KR102577210B1 - Method and apparatus for editing adaptive media - Google Patents

Abstract

Various embodiments of the present disclosure relate to a video editing method of a media processing server in an adaptive media streaming environment. A method for this includes: receiving a request to edit a media stream file; An operation of generating editing work instructions for media stream files for multiple resolutions and transmitting them to a work queue; An operation of determining a resolution at which editing will be performed by outputting an editing work instruction stored in the work queue; An operation of receiving a first original file corresponding to the determined resolution; and editing the first original file to create a second original file.

Description

Adaptive media editing method and apparatus {METHOD AND APPARATUS FOR EDITING ADAPTIVE MEDIA}

Various embodiments of the present disclosure relate to adaptive media editing methods and devices.

Media streaming service refers to a service that provides various multimedia data such as sound, music, and video in a streaming manner. Streaming refers to a technology that can continuously and endlessly process and play transmitted data. Normally, multimedia is played after downloading the entire file, so in the case of large files, it may take a long time to download the entire file. However, using streaming technology, multimedia playback is possible before the entire file is transferred.

Adaptive media streaming means preparing video of various resolutions stored in the form of several small pieces in advance and providing streaming as the original (piece) optimized for the network environment when the user watches the video. Examples of adaptive media streaming services include HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH).

Videos in the adaptive video streaming service on a media server are stored on the server by dividing one video file into several small pieces of time, and when editing a video, the edited parts of various versions of the video must be individually identified and modified. do.

Even when a user edits a video that is being streamed adaptive video, it must be possible to provide adaptive video streaming by quickly reflecting the video edits.

Various embodiments of the present disclosure relate to adaptive media editing methods. A method for this includes: receiving a request to edit a media stream file; An operation of generating editing work instructions for media stream files for multiple resolutions and transmitting them to a work queue; An operation of determining a resolution at which editing will be performed by outputting an editing work instruction stored in the work queue; An operation of receiving a first original file corresponding to the determined resolution; and editing the first original file to create a second original file.

In one embodiment, the method may further include storing information associated with the second original file in an external database.

In one embodiment, the operation of transmitting the second original file may be further included.

In one embodiment, the operation of determining the resolution at which editing will be performed by outputting the editing work instruction stored in the work queue may be performed in parallel by a plurality of editing engines in the media processing server.

Various embodiments of the present disclosure relate to a media processing server in an adaptive media streaming environment. The media processing server includes a video editing controller configured to receive editing requests for media stream files, generate and deliver editing work instructions for media stream files for a plurality of resolutions; a task queue configured to receive and store editing task instructions from the video editing controller; and output an editing work instruction stored in the work queue to determine a resolution at which editing will be performed, receive a first original file corresponding to the determined resolution, and edit the first original file to generate a second original file. It may include a configured video editing engine.

According to various embodiments of the present disclosure, even when editing a video in an adaptive video streaming service, adaptive video streaming can be provided by accurately and quickly reflecting the video editing content.

According to various embodiments of the present disclosure, adaptive video streaming can be provided by processing editing of a plurality of original files stored for adaptive streaming in parallel and quickly reflecting the edited content.

The effects that can be obtained from the exemplary embodiments of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned are common knowledge in the technical field to which the exemplary embodiments of the present disclosure belong from the following description. It can be clearly derived and understood by those who have it. That is, unintended effects resulting from implementing the exemplary embodiments of the present disclosure may also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

FIG. 1 is a diagram schematically illustrating an adaptive media streaming system environment including a content providing terminal, a media processing server, a media server, and a network connecting them, according to various embodiments of the present disclosure.
FIG. 2 is a block diagram illustrating the functional configuration of the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.
FIG. 3 is a schematic flowchart of a content upload method for an adaptive media streaming service operating in the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.
FIG. 4 is a schematic flowchart of a content editing method for an adaptive media streaming service operating in the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.
FIG. 5 is a diagram illustrating a function for deleting content for an adaptive media streaming service operating in the media processing server shown in FIG. 1 according to an embodiment of the present disclosure.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the function of the present disclosure, but this may vary depending on the intention of the technician working in the art, precedents, the emergence of new technologies, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant embodiment. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. In addition, terms such as "...unit" and "...module" used in the specification refer to a unit that processes at least one function or operation, which is implemented as hardware or software, or as a combination of hardware and software. It can be.

Hereinafter, with reference to the drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In the drawings, the same reference numerals refer to the same components, and the size of each component in the drawings may be exaggerated for clarity of explanation.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

FIG. 1 is a diagram schematically illustrating an adaptive media streaming system environment including a content providing terminal, a media processing server, a media server, a user terminal, and a network connecting them, according to various embodiments of the present disclosure.

According to various embodiments, the adaptive media streaming system 100 may include a content providing terminal 110, a media processing server 120, a media server 130, and a user terminal 140.

According to an embodiment of the present disclosure, each of the content providing terminals 110 may be any user electronic device equipped with a wired or wireless communication function. It should be noted that each of the content providing terminals 110 may be a variety of wired or wireless communication terminals, including, for example, a smart phone, tablet PC, desktop, laptop, PDA, digital TV, etc., and is not limited to a specific form. In this drawing, only one content providing terminal is shown, but the present disclosure is not limited thereto. According to another embodiment of the present disclosure, it should be noted that the system environment 100 may include a greater number of various types of content providing terminals.

According to an embodiment of the present disclosure, each of the content providing terminals 110 may communicate with the media processing server 120 or the media server 130 through a network (not shown), that is, transmit and receive necessary information. According to an embodiment of the present disclosure, the content providing terminal 110 may transmit stored media content to the media processing server 120 through a network. According to an embodiment of the present disclosure, the content providing terminal 110 may transmit an edit request for video streaming being serviced by the media server 130 to the media processing server 120 through a network. According to an embodiment of the present disclosure, the content providing terminal 110 may receive various signals transmitted from the outside (eg, the media processing server 120 or the media server 130) through a network.

According to an embodiment of the present disclosure, the media processing server 120 may perform procedures for providing an adaptive media streaming service. According to an embodiment of the present disclosure, the media processing server 120 may receive a media stream file from the content providing terminal 110. According to an embodiment of the present disclosure, the media processing server 120 may process media stream files to generate a plurality of media stream files of various resolutions to support an adaptive streaming service. According to an embodiment of the present disclosure, the media processing server 120 may transmit a plurality of media stream files of various resolutions to the media server 130.

According to an embodiment of the present disclosure, the media processing server 120 may perform a procedure for editing a media stream providing an adaptive media streaming service. According to an embodiment of the present disclosure, the media processing server 120 may receive a request to edit a media stream file from the content providing terminal 110. For example, the content providing terminal 110 may request deletion of a specific section by requesting editing of a specific media stream file. According to one embodiment, the media processing server 120 may receive a plurality of media stream files from the media server 130 and perform a requested editing task on the corresponding files. For example, you can create an edited file that deletes a specific section (e.g., 0:35-0:53) from a specific media stream file. According to an embodiment of the present disclosure, the media processing server 120 can perform editing in parallel on a plurality of media stream files of various resolutions. For example, multiple media stream files may have resolutions of 360p, 480p, 720p, and 1080p. According to an embodiment of the present disclosure, the media processing server 120 may transmit a plurality of media stream files of various resolutions to the media server 130.

According to an embodiment of the present disclosure, the media server 130 may provide an adaptive media streaming service to the user terminal 140. In one embodiment, the media server 130 receives a plurality of media stream files of various resolutions from the media processing server 120, and selects a specific resolution according to a request from the user terminal 110 in service or the state of the user terminal 110. of media stream files can be provided. For example, the media server 130 may be Amazon S3.

According to an embodiment of the present disclosure, the user terminal 140 is a device that receives an adaptive media streaming service from the media server 130. In one embodiment, the user terminal 140 may communicate with the media server 130 through a network, that is, transmit and receive necessary information.

According to an embodiment of the present disclosure, the user terminal 140 may be any user electronic device each equipped with a wired or wireless communication function. It should be noted that the user terminal 140 may be a variety of wired or wireless communication terminals, including, for example, a smart phone, tablet PC, desktop, laptop, PDA, digital TV, etc., and is not limited to a specific form. In this drawing, only one user terminal is shown, but the present disclosure is not limited thereto. It should be noted that according to another embodiment of the present disclosure, the system environment 100 may include a larger number of various types of user terminals.

According to one embodiment of the present disclosure, the network may include any wired or wireless communication network, such as a TCP/IP communication network. According to an embodiment of the present disclosure, the network may include, for example, a Wi-Fi network, a LAN network, a WAN network, an Internet network, etc., and the present disclosure is not limited thereto. According to an embodiment of the present disclosure, the network may be any variety of wired or wireless communication such as Ethernet, GSM, EDGE (Enhanced Data GSM Environment), CDMA, TDMA, OFDM, OFDMA, Bluetooth, VoIP, Wi-MAX, Wibro, etc. It can be implemented using a protocol.

FIG. 2 is a block diagram illustrating the functional configuration of the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.

As shown, the media processing server 120 may include a video edit controller 210, a job queue 220, and a plurality of video edit engines 230.

In one embodiment of the present disclosure, the media processing server 120 may perform procedures for providing an adaptive media streaming service. According to one embodiment, media processing server 120 may receive a media stream file. According to an embodiment of the present disclosure, the media processing server 120 may process media stream files to generate a plurality of media stream files of various resolutions to support an adaptive streaming service. According to an embodiment of the present disclosure, the media processing server 120 may transmit a plurality of media stream files of various resolutions to the media server 130.

In one embodiment of the present disclosure, the video editing controller 210 may perform a procedure for editing a media stream providing an adaptive media streaming service. According to an embodiment of the present disclosure, the video editing controller 210 may receive a request for editing a media stream file from the content providing terminal 110. For example, the content providing terminal 110 may request deletion of a specific section by requesting editing of a specific media stream file. According to one embodiment, the video editing controller 210 may control to perform a requested editing task on video originals of multiple resolutions associated with a specific media stream file.

In one embodiment of the present disclosure, the video editing controller 210 may create an editing task for each resolution that must support adaptive media streaming and transmit it to the task queue 220. For example, if a plurality of media stream files provided by the media server 130 have resolutions of 360p, 480p, 720p, and 1080p, the video editing controller 210 performs 360p editing, 480p editing, 720p editing, and 1080p editing. , a total of four editing tasks can be created and delivered to the task queue 220.

According to one embodiment of the present disclosure, the task queue 220 is a queue that stores editing tasks. In one embodiment, the task queue 220 may receive a task from the video editing controller 210 and output the task to the video editing engine 230, which will be described later.

According to an embodiment of the present disclosure, the video editing engine 230 may perform an editing operation on an original video file of a specific resolution. In one embodiment, video editing engine 230 may retrieve editing jobs from task queue 220. In one embodiment, video editing engine 230 may determine the resolution at which to perform editing based on the imported editing job. In one embodiment, the video editing engine 230 may receive an original video file corresponding to the determined resolution from the media server 130. In one embodiment, the video editing engine 230 may edit an original video file to create a new original video file. For example, the video editing engine 230 may create a new original file by deleting some sections (eg, 1:30-2:45) from the original video file stored in the media server 130 and being serviced.

According to an embodiment of the present disclosure, the video editing engine 230 may perform editing through an editing function (eg, doVideoEdit) that edits a portion of a media stream of a specific resolution. A more detailed description of the editing function according to an embodiment of the present disclosure will be described later with reference to FIG. 5 and Table 1.

According to an embodiment of the present disclosure, the video editing engine 230 may transmit an edited media stream, that is, a new original created by performing editing, to the media server 130. In one embodiment, there may be a plurality of video editing engines 230, and each of the plurality of video editing engines 230 may transmit each of the edited media streams of a plurality of resolutions to the media server 130.

FIG. 3 is a schematic flowchart of a content upload method for an adaptive media streaming service operating in the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.

First, the media processing server 120 may receive a media stream from the content providing terminal 110. In one embodiment, the media stream may be an image such as a video (S301). Next, the media processing server 120 may process the media stream file to generate a plurality of media stream files of various resolutions to support an adaptive streaming service (S302). Thereafter, the media processing server 120 may transmit a plurality of media stream files of various resolutions to the media server 130 (S303). Although not shown, in one embodiment, the media processing server 120 may store information related to the original media stream in an external video information database. Information related to the media stream source may include, for example, URL information of the file.

FIG. 4 is a schematic flowchart of a content editing method for an adaptive media streaming service operating in the media processing server shown in FIG. 1, according to an embodiment of the present disclosure.

First, the video editing controller 210 of the media processing server 120 may perform a procedure for editing a media stream providing an adaptive media streaming service. The video editing controller 210 may receive a request to edit a media stream file from the content providing terminal 110 (S401). For example, the video editing controller 210 may receive a request from the content providing terminal 110 to delete a specific section of the media stream file. In one embodiment, media stream files are being served in various resolutions, such as 360p, 480p, 720p, and 1080p resolution.

Next, in order to perform the requested editing operation on the original media file stored in the media server 130, the video editing controller 210 is connected to the media stream file for each resolution that must support adaptive media streaming. Editing work instructions can be created (S402). For example, the video editing controller 210 may provide an editing operation instruction for deleting a specific section (e.g., 1:30-2:45) from a specific video file to a plurality of media streams of various resolutions to support an adaptive streaming service. Each file can be created separately.

Next, the video editing controller 210 may transmit an editing task instruction to the task queue 220 (S403). For example, if a plurality of media stream files provided by the media server 130 have resolutions of 360p, 480p, 720p, and 1080p, the video editing controller 210 performs 360p editing, 480p editing, 720p editing, and 1080p editing. , a total of four editing tasks can be created and delivered to the task queue 220.

Next, the video editing engine 230 may retrieve the editing work instructions input to the work queue 220 and determine the resolution at which editing will be performed based on the editing work instructions (S404).

Next, the video editing engine 230 may receive the video original file corresponding to the determined resolution from the media server 130 (S405).

Next, the video editing engine 230 may edit the received original video file, that is, delete a specific section, and create a new original video file (S406). For example, the video editing engine 230 may create a new original file by deleting some sections (eg, 1:30-2:45) from the original video file stored in the media server 130 and being serviced. The function for deleting the original video file received from the video editing engine 230 will be described with reference to FIG. 5 below.

In one embodiment, the video editing engine 230 is composed of a plurality of video editing engines 230 and can perform editing operations in parallel on each of the media files of a plurality of resolutions.

Finally, the video editing engine 230 may transmit the new edited original file to the media server 130 (S407). In one embodiment, each of the plurality of video editing engines 230 may transmit each of the edited media streams of the plurality of resolutions to the media server 130.

FIG. 5 is a diagram illustrating a function for deleting content for an adaptive media streaming service operating in the media processing server shown in FIG. 1 according to an embodiment of the present disclosure.

Referring to the drawing, each video file having multiple resolutions consists of 10 chunks. Here, chunks represent pieces of a media file, and the data structure may be an array. Here, chunk_duration represents the period for each chunk and the data structure may be an array. Here, total_chunk_count represents the total number of media file fragments, and may be, for example, 10. Here, chunk_time_unit represents the time of the media file fragment and may be, for example, 10 seconds. Here, startTime represents the start time to edit the media, and endTime represents the end time to edit the media.

The duration for each chunk is initially set to the chunk_time_unit value. The duration for each chunk starts with the same value, but may decrease after editing, so the chunk_duration[i] variable is used.

Since the time (chunk_time_unit) of the current media file fragment is 10 seconds, if the user enters deletion of a section between 35 and 53 seconds, part of chunk 3, chunk 4, and part of chunk 5 must be deleted.

Table 1 shows the number code of an editing function that edits a portion of a media stream of a specific resolution, according to an embodiment of the present disclosure.

The function doVideoEdit according to an embodiment of the present disclosure is a function that edits (eg, deletes) a media stream of a specific resolution. By receiving chunks of a media file as arguments, you can create a new media file by deleting media files for the requested deletion time. To explain, for all chunks ( for i = 0 ~ total_chunk_count ), set to the time (initial value) of the media file fragment (chunk_duration[ i ] = chunk_time_unit).

function getCumulativeDurtion( index ) is a function that determines the total media time by summing the duration of each chunk up to the argument (index).

function getChunkIndex( total_count , value ) is a function that determines which chunk corresponds to the time (start time or end time) requested for deletion.

function saveChunk( chunk, from, to ) is a function that returns only a part (e.g. from-to) within one chunk. For example, if chunk 2 is 10 seconds long, calling saveChunk(chunk 2, 0, 4) deletes 5 to 10 seconds of chunk 2 and returns the remaining chunk from 0 to 4 seconds. .

In one embodiment, when the media processing server receives a request to edit (e.g., delete) a media stream, the media processing server divides the edits into cases where editing is required within one chunk and cases where editing is required across multiple chunks. Perform.

In the case of deletion within one chunk ( if (startIndex == endIndex)), the media processing server performs partial deletion of the corresponding chunk using the function saveChunk. To explain, when the media processing server edits the middle part of one chunk, the function saveChunk is used twice for one chunk, with the former saveChunk function returning the first half and the latter saveChunk function returning the second half. For example, when deleting 4-5 seconds out of 0-10 seconds of one chunk, the first saveChunk returns 0-3 seconds, and the second saveChunk returns 6-10 seconds. The media processing server merges the first half chunk and the second half chunk to create a new chunk and updates the duration (chunk_duration[endIndex]) of the new chunk.

If editing is required across multiple chunks, the media processing server and use function saveChunk for the last chunk. The media processing server performs partial deletion using the function saveChunk for the starting chunk requested for deletion, deletes all of the middle chunks, and performs partial deletion using the function saveChunk for the last chunk requested for deletion.

As described above, the media processing server 120 performs editing operations in parallel on each of the media files of a plurality of resolutions in the plurality of video editing engines 230 and uploads them to the media server 130 to edit the video content. Adaptive media streaming can be provided through rapid reflection.

Programs executed by the content providing terminal 110, media processing server 120, and media server 130 described through this disclosure include hardware components, software components, and/or hardware components and software components. It can be implemented in combination. A program can be executed by any system that can execute computer-readable instructions.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software may be implemented as a computer program including instructions stored on computer-readable storage media. Computer-readable recording media include, for example, magnetic storage media (e.g., ROM (Read-Only Memory), RAM (Random-Access Memory), floppy disk, hard disk, etc.) and optical read media (e.g., CD-ROM). (CD-ROM), DVD (Digital Versatile Disc), etc. The computer-readable recording medium is distributed among computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner. The media may be readable by a computer, stored in memory, and executed by a processor.

Computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' only means that the storage medium does not contain signals and is tangible, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

Additionally, programs according to embodiments of the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product is a product in the form of a software program (e.g., a downloadable application) that is distributed electronically by the device manufacturer or through an electronic marketplace (e.g., Google Play Store, App Store). may include. For electronic distribution, at least a portion of the software program may be stored on a storage medium or created temporarily. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic market server, or a relay server that temporarily stores a software program.

A computer program product, in a system comprised of a server and a device, may include a storage medium of a server or a storage medium of a device. Alternatively, if there is a third device (e.g., a smartphone) connected to the server or device for communication, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from a server to the device or a third device, or from a third device to the device. In this case, one of the server, the device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of a server, a device, and a third device may execute the computer program product and perform the methods according to the disclosed embodiments in a distributed manner. For example, the server may execute a computer program product stored on the server and control a device connected to the server to perform the method according to the disclosed embodiments. As another example, a third device may execute a computer program product to control a device communication-connected to the third device to perform the method according to the disclosed embodiment. When the third device executes the computer program product, the third device may download the computer program product from the server and execute the downloaded computer program product. Alternatively, a third device may perform the methods according to the disclosed embodiments by executing a computer program product provided in a pre-loaded state.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and/or components, such as a described computer system or module, may be combined or combined in a form different from the described method, or other components or equivalents may be used. Appropriate results can be achieved even if replaced or replaced by .

110: Content provision terminal
120: Media processing server
130: media server
140: User terminal

Claims (9)

In a video editing method of a media processing server in an adaptive media streaming environment,
Receiving a request to edit a media stream file;
An operation of generating editing work instructions for media stream files for multiple resolutions and transmitting them to a work queue;
An operation of determining a plurality of resolutions at which editing will be performed by outputting an editing work instruction stored in the work queue;
An operation of receiving a plurality of first original files corresponding to the determined plurality of resolutions, each first original file of the plurality of first original files having an array structure including one or more chunks; and
Editing the plurality of first original files to create a plurality of second original files;
Including,
The operation of editing the plurality of first original files to create a plurality of second original files,
determining a chunk to be edited among the one or more chunks of each first original file of the plurality of first original files based on the editing operation instruction; and
The method comprising: performing editing in parallel in a plurality of editing engines in the media processing server on a determined chunk of each first original file of the plurality of first original files. According to paragraph 1,
The method further includes storing information associated with the plurality of second original files in an external database. According to paragraph 1,
The method further includes transmitting the plurality of second original files. According to paragraph 1,
The operation of determining a plurality of resolutions at which to perform editing by outputting an editing work instruction stored in the work queue is performed in parallel by a plurality of editing engines in the media processing server. A computer-readable recording medium containing one or more computer-readable instructions executable by a computer, wherein the one or more computer-readable instructions, when executed by the computer, cause the computer to perform the instructions of claims 1 to 4. A computer-readable recording medium that allows performing the method according to any one of the above. As a media processing server in an adaptive media streaming environment,
a video editing controller configured to receive editing requests for media stream files, and generate and deliver editing work instructions for media stream files for a plurality of resolutions;
a task queue configured to receive and store editing task instructions from the video editing controller; and
Output an editing work instruction stored in the work queue to determine a plurality of resolutions to perform editing, receive a plurality of first original files corresponding to the determined plurality of resolutions, and edit the plurality of first original files. , a plurality of video editing engines configured to generate a plurality of second original files;
Including,
Each first original file of the plurality of first original files has an array structure including one or more chunks,
Editing the plurality of first original files to create a plurality of second original files,
Based on the editing operation instruction, determine a chunk to be edited among the one or more chunks of each first original file of the plurality of first original files, and determine a chunk to be edited, A media processing server comprising performing editing in parallel in the plurality of video editing engines for the determined chunks. According to clause 6,
wherein the plurality of video editing engines are further configured to transmit information associated with the plurality of second original files to an external database. In clause 7,
and the plurality of video editing engines are further configured to transmit the plurality of second original files. delete