KR102439599B1 - Cloud video editing service system and method thereof - Google Patents

Abstract

The present invention relates to a cloud image editing system and method. The cloud image editing system includes: an image editing service server which receives an image to be edited from a user terminal and provides an editing result; a highlight extraction module for extracting a part of an image corresponding to highlight information set by a user from the image of the image editing service server and designating the part of the image as a highlight; an editing module for homogenizing the color and volume of the image; a rendering module for rendering the image into a high-definition image; and a background music recommendation module for recommending suitable background music from a background music database by analyzing the contents of the image through artificial intelligence.

Description

Cloud video editing service system and method thereof

The present invention relates to a cloud image editing service system and method, and more particularly, to a system and method for providing an image editing service for single creators and image editors.

With the recent development of video platforms, the scale of consumption of various video contents is rapidly increasing, and the number of single creators and broadcasters is also increasing.

As of 2020, the number of domestic YouTube channels is predicted to be 43.19 million, and 58% of the 15,122 domestic YouTube channel samples were confirmed to have uploaded more than one video.

In order to be competitive in the growing video content market, it is necessary to satisfy both the quantity and quality of the content, and proper editing of the video is required to satisfy the quality aspect of the content.

In particular, depending on how the highlight displayed in the video list is selected, the number of content views can vary greatly, and the selection of background music also plays a very important role.

In addition, in order to satisfy the quantitative aspect of the content, it is required to quickly edit the video that can shorten the production cycle of the video content. However, the performance of individual editing equipment is relatively low, and there is a problem in that the time of a task with a lot of calculations such as rendering is delayed.

Conventionally, time inefficiency may occur because it is necessary to check the entire image and extract the main part in order to grasp important scenes, flows, etc. of the image for highlight extraction.

In order to solve this problem, registered patent No. 10-2171657 (registered on October 23, 2020, a method and system for video editing based on context identification using artificial intelligence) sets a context and extracts a clip that matches the context, A technique for presenting to a user and confirming a clip selected by the user as a highlight is described.

However, this method has a problem in that a video clip irrelevant to the highlight may be selected due to an error in setting the context, and a relatively time loss occurs because the user has to check and select a plurality of video clips again.

In addition, the actual video editing spends a lot of time on simple repetitive editing tasks such as cut editing, audio equalization, and color correction, but the above registered patent has a disadvantage in that it cannot perform overall editing of the video.

In addition, there was a problem in that it was not possible to provide a service related to the selection of background music suitable for the scene. In particular, there is a problem that there is always a concern of copyright infringement when uploaded to a video platform by using background music without considering the copyright of the background music, a mistake that single creators often make.

The technical problem to be solved by the present invention in consideration of the above problems is to provide a cloud image editing service system and method capable of simultaneously performing overall image editing, highlight extraction, and background music selection and application.

In particular, it is an object of the present invention to provide a cloud image editing service system and method in which a user selects a condition of a highlight image and extracts a highlight image that meets the selection condition.

Another object of the present invention is to provide a cloud image editing service system and method that can select and apply background music suitable for an image, but select background music without fear of copyright infringement.

In addition, it is to provide a cloud video editing service system and method that can share knowledge so that single creators or video editors can easily edit video by mutually sharing video templates of video producers.

A cloud video editing service system according to an aspect of the present invention for solving the above problems includes a video editing service server that receives an image to be edited from a user terminal and provides an editing result, and an image of the video editing service server A highlight extraction module for extracting a part of an image matching the highlight information set by the user and designating it as a highlight, an editing module for homogenizing the color and volume of the image, a rendering module for rendering the image as a high-definition image; It may include a background music recommendation module that analyzes the content of the image through artificial intelligence and recommends suitable background music from a background music database.

In an embodiment of the present invention, an editing style transaction module for extracting the editing style of the video, and providing the extracted editing style to the highlight extraction module, the editing module, and the background music recommendation module to be applied to subsequent video editing may further include.

In an embodiment of the present invention, the editing style transaction module publishes the extracted editing style on a transaction page, and when an editing style posted in another user terminal is selected, the editing style selected for editing the image of the other user terminal can be applied.

In an embodiment of the present invention, the highlight extraction module may extract a highlight section from an image by using an acoustic feature, an image feature, or a combination of an acoustic feature and an image feature included in the highlight information.

In an embodiment of the present invention, the background music may be copyright-free or free of charge.

In addition, the cloud video editing service method according to another aspect of the present invention comprises the steps of: a) transmitting video and highlight extraction conditions from a user terminal to a video editing service server; b) using a highlight extraction module in the video editing service server extracting highlights that meet the highlight extraction conditions; c) converting the image to high quality using a rendering module in the image editing service server; d) adjusting the color and volume by using the editing module in the image editing service server It may include equalizing, and e) recommending background music matching the screen switching speed and sound information of the image by using the background music recommendation module in the image editing service server.

In an embodiment of the present invention, after performing step e), the editing style of the edited video is extracted using the editing style transaction module, and the editing extracted by the highlight extraction module, the editing module, and the background music recommendation module The method may further include providing a style and applying the style to subsequent video editing.

In an embodiment of the present invention, using the editing style transaction module, the extracted editing style is posted on a transaction page, and when an editing style posted in another user terminal is selected, the editing style selected for editing the image of the other user terminal is selected. The method may further include applying an editing style.

In an embodiment of the present invention, in step b), a highlight section may be extracted from an image using an acoustic feature, an image feature, or a combination of an acoustic feature and an image feature included in the highlight extraction condition.

In an embodiment of the present invention, the background music may be copyright-free or free of charge.

The present invention can extract an accurate highlight section from an image according to the user's intention, and recommend background music that does not infringe copyright, thereby improving the convenience of image editing and shortening the time required for image editing. there is

In addition, it is possible to provide a high-definition image rendering service by building a system that renders high-definition images at a high speed, so that creators or editors can obtain high-definition images in a short time without using expensive editing equipment.

In addition, the present invention has the effect of improving the convenience of editing by automatically performing audio equalization and color correction between edit cuts.

1 is a block diagram of a cloud video editing system of the present invention.
2 is a flowchart of a cloud video editing method of the present invention.

Hereinafter, the cloud image editing system and method of the present invention will be described in detail with reference to the accompanying drawings.

The embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art, and the embodiments described below may be modified in various other forms, The scope is not limited to the following examples. Rather, these embodiments are provided so as to more fully and complete the present invention, and to fully convey the spirit of the present invention to those skilled in the art.

The terminology used herein is used to describe specific embodiments, and is not intended to limit the present invention. As used herein, the singular form may include the plural form unless the context clearly dictates otherwise. Also, as used herein, “comprise” and/or “comprising” refers to the presence of the recited shapes, numbers, steps, actions, members, elements, and/or groups thereof. and does not exclude the presence or addition of one or more other shapes, numbers, movements, members, elements and/or groups. As used herein, the term “and/or” includes any one and any combination of one or more of those listed items.

Although the terms first, second, etc. are used herein to describe various elements, regions and/or regions, it is to be understood that these elements, parts, regions, layers, and/or regions are not limited by these terms. . These terms do not imply a specific order, upper and lower, or superiority, and are used only to distinguish one member, region or region from another member, region or region. Accordingly, a first member, region, or region described below may refer to a second member, region, or region without departing from the teachings of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings schematically illustrating embodiments of the present invention. In the drawings, variations of the illustrated shape may be expected, for example depending on manufacturing technology and/or tolerances. Therefore, the embodiment of the present invention should not be construed as limited to the specific shape of the region shown in the present specification, but should include, for example, a change in shape caused by manufacturing.

1 is a block diagram of a cloud video editing system according to a preferred embodiment of the present invention.

Referring to FIG. 1 , the cloud image editing system of the present invention receives an image to be edited from the user terminal 80 , and provides the edited image to the user terminal 80 or the user terminal 80 to the video platform server 90 . A video editing service server 10 that provides a video editing service for registering as an account of the video editing service server 10, and extracting a part of the video corresponding to the highlight information set by the user from the video of the video editing service server 10 and designating it as a highlight a highlight extraction module 20, an editing module 40 for homogenizing the color and volume of the image, a rendering module 30 for rendering the edited image into a high-definition image, and analyzing the content of the image through artificial intelligence The background music recommendation module 50 for recommending suitable background music from the background music database 60, and the editing style for extracting the editing style of each video and posting the editing style transaction between the user terminals 80 It may include a transaction module 70 .

Hereinafter, the configuration and operation of the cloud video editing system according to a preferred embodiment of the present invention configured as described above will be described in more detail.

First, the user terminal 80 is a device that provides the first edited video to the video editing service server 10 , and is a device that can be connected to the video editing service server 10 through at least a network. For example, it may be a device such as a laptop computer, a PC, or a smartphone.

The first edited image may be an image captured by a camera, or may be an image in which cuts of images captured in units of cuts are connected. The first edited video includes at least sound.

At this time, the sound may be a voice, background music, sound effect, or other sound such as a sound of chewing food or a sound of boiling liquid.

The first edited image may be of low quality, background music may be omitted, may be an image in which the sound is not uniform as a whole, and may be an image requiring color correction due to lighting effects.

The first edited image of the user terminal 80 is received by classifying the user from the image service server 10 , and processing is performed according to the editing information provided by the user terminal 80 .

The editing information provided by the user terminal 80 may be at least one of highlight extraction, rendering, editing such as color correction, and background music recommendation.

The edit information of highlight extraction may further include sub-editing information on the highlight section.

The sub-editing information of the highlight extraction editing information includes audio-related and video-related information. For example, in audio-related terms, the video part with the loudest laughter, the sound of eating food or the sound of boiling, etc. may be The most used parts may be included.

In addition, the video-related information may be a portion with the most people in one frame, a portion in which a person is photographed the largest on a screen, a portion with the most movement, and the like.

Such sub-editing information may be selected by the user terminal 80 on the connection screen of the image editing service server 10 .

The highlight extraction module 20 extracts a highlight portion from the image according to the sub-editing information selected by the user terminal 80 .

For example, when the sub-editing information is a part having the largest laughter, the highlight extraction module 20 detects a section corresponding to the laughter from the audio information of the video, and selects a part having the largest volume among them.

It is assumed that the highlight extraction module 20 includes at least an artificial intelligence module to learn about the sub-editing information, and to detect a highlight section according to the learning result.

In the above example, the highlight extraction module 20 may learn audio information from a plurality of images, store a learning result on a human laughter, and detect a highlight section by comparing it with the learning result.

In the same way, a screen in which a person is most close-up may be detected from an image, or a screen having the largest number of people may be detected by recognizing a person in the image, and a certain area may be extracted as a highlight image based on the screen.

And the editing module 40 homogenizes the color and volume of the set image. For example, if there is a difference in the color and volume of an image for each cut due to a difference in lighting of each cut or a difference in settings of recording equipment in the same image, the color and volume are homogenized using the average value of each difference.

The rendering module 30 renders the low-quality image as a high-quality image such as 4k.

In this case, the rendering module 30 includes a high-speed, high-performance rendering equipment, so that it is possible to shorten the time compared to rendering in the user terminal 80 .

Therefore, it is possible to shorten the working time compared to the individual directly performing the rendering.

The background music recommendation module 50 may recommend suitable background music using various information of the image.

The background music recommendation module 50 uses the audio information of the image or recommends suitable background music according to the switching speed of the screen.

For example, from the audio information included in the video, music with a fast or slow tempo can be recommended according to the pitch of the sound. background music can be recommended.

In addition, the tempo of the background music is determined in consideration of the video transition speed, and the background music that is dark or has a horror atmosphere can be recommended according to the overall brightness of the image.

The background music recommendation module 50 may recommend a plurality of background music and provide it to the user terminal 80 , or may automatically insert background music into an image.

In this case, the background music is stored in the background music database 60, but it is assumed that there is no copyright.

Then, the editing style transaction module 70 detects a unique editing style for each user (editor or creator) of the user terminals 80 from the video.

That is, the editor learns his/her own editing style structure in the editing style transaction module 70 , and provides this information to the editing module 40 , the background music recommendation module 50 and the highlight extraction module 20 . Therefore, by applying it to the editing of the video of the corresponding editor, it is possible to further shorten the time required for editing, and finally, it is possible to provide a personalized editing service.

In addition, the editing style transaction module 70 uploads the editing style of a specific editor to the transaction page, checks the transaction page in another user terminal 80, selects and purchases a favorite editing style, and uses it to edit one's own video. can be applied.

A fee may be paid for such interpersonal editing style transactions, and editors with a beautiful editing style can generate profits.

In addition, from the standpoint of the user terminal 80 who has purchased the editing style, it is possible to lower the entry barrier for video editing, and the effect of obtaining a more stylish video editing effect can be expected.

2 is a flowchart of a cloud image editing method according to a preferred embodiment of the present invention.

Referring to FIG. 2 , the present invention includes a step of transmitting an image and highlight extraction conditions from the user terminal 80 to the image editing service server 10 ( S21 ), and a highlight extraction module 20 from the image editing service server 10 . ) extracting highlights that meet the highlight extraction conditions (S22), equalizing the color and volume of the image using the editing module 40 (S23), and the background music recommendation module 50 Recommending background music by using the screen switching speed and sound information of the image (S24), converting the image to high quality using the rendering module 30 (S25), and the editing style transaction module 70 ) to extract the editing style for each user of the user terminal 80 who uploaded the video, and use the extracted editing style to the highlight extraction module 20, the editing module 40 and the background music recommendation module 50 The steps of uploading the editing style to the transaction page (S26) and providing the edited video to the user terminal 80 or the video platform server 90 (S27) are provided so that it can be used for subsequent editing. consists of including

Hereinafter, the configuration and operation of the cloud image editing method of the present invention configured as described above will be described in more detail.

First, in step S21 , the user terminal 80 transmits video and highlight extraction conditions to the video editing service server 10 .

In this case, the highlight extraction condition may be a condition suggested by the image editing service server 10 selected by the user terminal 80 .

The highlight extraction condition may be a characteristic of a sound or a characteristic of an image frame, and may be a combination of a characteristic of a sound and a characteristic of an image frame.

Next, in step S22 , the video editing service server 10 extracts a highlight that satisfies the highlight extraction condition using the highlight extraction module 20 .

As in the specific example described above, the highlight can be extracted using the characteristics of the sound, such as the part with the loudest laughter. Allows you to extract parts.

Next, in step S23 , the color and volume of the image are equalized using the editing module 40 .

In general, an image is made by combining cuts taken in one take, and in this case, each image cut may have a difference in color or volume depending on shooting conditions, etc., and may be equalized through the editing module 40 .

Next, as in step S24 , the image editing service server 10 recommends background music using the background music recommendation module 50 and the screen switching speed and sound information of the image.

At this time, it is assumed that the recommended background music is stored in the background music database 60 , and the background music stored in the background music database 60 has no copyright or is free of charge.

Although it has been described that the steps S22, S23, and S24 are sequentially performed, they may be performed regardless of the order.

Next, in step S25 , the image editing service server 10 converts the image to high quality using the rendering module 30 .

At this time, the rendering module 30 is capable of high-speed rendering, and converts it into a high-definition image such as 4K.

Next, as in step S26, an editing style for each user of the user terminal 80 who has uploaded the image is extracted by using the editing style transaction module 70 .

In this case, the editing style may be information about condition information for highlight extraction, color and volume, and selection of background music, and the extracted editing style may be applied again to subsequent editing.

That is, the extracted editing style may be provided to the highlight extraction module 20 , the editing module 40 , and the background music recommendation module 50 to be used for subsequent editing.

Then, as in step S27, the editing style is uploaded to the transaction page, and when another user terminal 80 selects the corresponding editing style, the information is extracted from the highlight extraction module 20, the editing module 40, and the background music recommendation module. (50) so that the editing style of other editors can be applied as it is.

Posting on the transaction page as described in step S27 is not necessarily a step, but may be performed as needed.

Thereafter, the edited image may be provided to the user terminal 80 or the video platform server 90 .

Through this process, the present invention can automatically perform overall editing of the image of the user terminal 80 in the cloud environment.

It is obvious to those of ordinary skill in the art that the present invention is not limited to the above embodiment and can be variously modified and modified within the scope without departing from the technical gist of the present invention. will be.

10: Video editing service server 20: Highlight extraction module
30: rendering module 40: editing module
50: background music recommendation module 60: background music database
70: editing style transaction module 80: user terminal
90: video platform server

Claims (10)

an image editing service server that receives an image to be edited from a user terminal and provides an editing result;
a highlight extraction module for automatically extracting a part of an image corresponding to audio information and video information, which is highlight information set by a user, from the image of the image editing service server and designating it as a highlight;
an editing module for homogenizing the color and volume of the image with an average value of the color and volume of the cuts constituting the image;
a rendering module for rendering the image as a high-definition image; and
and a background music recommendation module that analyzes the content of the image through artificial intelligence, selects suitable background music from a background music database, and recommends it to the user terminal. According to claim 1,
Cloud video editing further comprising an editing style transaction module for extracting the editing style of the video, and providing the extracted editing style to the highlight extraction module, the editing module, and the background music recommendation module to be applied to subsequent video editing service system. 3. The method of claim 2,
The editing style transaction module,
Post the extracted editorial style to the transaction page;
Cloud video editing service system, characterized in that when selecting an editing style published by another user terminal, the selected editing style is applied to the editing of the video of the other user terminal. delete According to claim 1,
The background music is
Cloud video editing service system, characterized in that there is no copyright or free of charge. a) transmitting the image and highlight extraction conditions from the user terminal to the image editing service server;
b) extracting a highlight matching audio information and video information, which are highlight extraction conditions, by using a highlight extraction module in the image editing service server;
c) converting the image to high quality using a rendering module in the image editing service server;
d) obtaining an average value of color and volume of cuts constituting an image using the editing module in the image editing service server, and equalizing the color and volume with the average value; and
e) using a background music recommendation module in the image editing service server to select background music that matches the screen switching speed and sound information of the image, and recommending it to the user terminal. 7. The method of claim 6,
After performing step e),
Extracting the editing style of the edited video by using the editing style transaction module, and providing the extracted editing style to the highlight extraction module, the editing module, and the background music recommendation module to apply to the subsequent video editing Cloud video editing service method. 8. The method of claim 7,
Using the editing style transaction module,
Post the extracted editorial style to the transaction page;
The cloud image editing service method further comprising the step of applying the selected editing style to the editing of the image of the other user terminal when selecting an editing style published in another user terminal. delete 7. The method of claim 6,
The background music is
Cloud video editing service method, characterized in that there is no copyright or free.

Images