KR20170130729A - Method, Apparatus, and Computer-Readable Medium for Automatic editing moving picture data and audio data - Google Patents

Abstract

According to the present invention, it is possible to provide a user with second data obtained by matching and combining automatically pre-stored audio data with a whole or a part of a reproduction section of first data by using image analysis data derived from attribute information input by a user with respect to the first data that is video data, and variation and color information of a plurality of image data included in the first data. Accordingly, when a user edits a video, a process of selecting each audio data and a process of selecting a reproduction section of a video to be combined with the audio data can be minimized. Also, since a user does not need to know an editing technique of video data and audio data, it is possible to minimize technical barriers in editing video data and to provide a user with utmost convenience through omission of an editing process.

Description

[0001] The present invention relates to a method and a computer-readable medium for automatically editing video data and audio data,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a moving picture editing technique such as combining moving picture data and voice data, and more specifically, to a technique for analyzing moving picture data to automatically combine voice data such as an effect sound and a background sound .

Recently, as the usage rate of various video sharing sites increases and the usage rate of social network services increases, users make videos or share them with individuals or groups, thereby gaining interest or gaining popularity. Such a moving image is often produced by shooting a moving image using a photographing apparatus and applying various effects to the photographed moving image, and is shared with a combination of moving image and audio data.

Audio data included in a moving picture is often recorded directly in a photographing apparatus, that is, voice data recorded together when moving pictures are captured, and voice data combined when moving pictures are edited.

Of these, specialized programs are used for editing and editing voice data recorded together or for combining new voice data when editing moving images. Although the difficulty has been lowered relatively recently due to the spread of various types of popular video editing programs and applications, the user directly searches for voice data to be edited and stores the voice data in his / her terminal, and at the same time, There is a time-consuming and inconvenient time for editing a moving picture because it is inevitable that a troublesome work such as directly determining whether to be combined in a section and editing it is inevitable.

Korean Unexamined Patent Publication No. 2007-0098362 discloses a technology for generating moving picture data by setting a playback interval for the background music to be played back and combining the background music with the background music. However, in the above-mentioned technique, the user has to directly select the background music. Consequently, there has been inconvenience that the user must directly select the playback interval of the background music on the timeline control tool.

Accordingly, the present invention provides a technology for combining a large amount of audio data with moving picture data, and by matching the audio data automatically matching the moving picture data to the user, the inconvenience of the user selecting a large number of audio data is minimized, And a technique for automatically analyzing and selecting sections is provided so that background sound or effect sound is automatically combined with a moving picture to provide a technique for maximizing the convenience of editing a moving picture of a user.

According to an aspect of the present invention, there is provided a method for automatic editing of moving picture data and audio data, the method comprising the steps of: A method of automatic editing of data and audio data, the method comprising the steps of: receiving, as moving image data to be edited from a user terminal, a plurality of image data arranged in order of reproduction time, Receiving first data; Extracting image analysis data including color information of a plurality of image data and information on a change pattern according to moving picture reproduction of a plurality of image data embedded in the received first data; Comparing the attribute information of the first data and the image analysis data stored in one area of the first data with pre-stored audio data, and storing at least one audio data in all or a part of the reproduction time period of the first data Matching step; And combining the first data and the audio data matched with the first data to generate second data in which the moving picture data and the audio data are combined and providing the second data to the user terminal.

According to the present invention, by using image analysis data derived from variation and color information of a plurality of image data included in the first data and attribute information inputted by a user with respect to first data that is moving image data, The data is matched to all or a part of the playback period of the first data and the combined second data is provided to the user.

Accordingly, it is possible to minimize the process of selecting the audio data at the time of editing the moving picture and the process of selecting the reproduction section of the moving picture to be combined with the audio data, It is possible to minimize the technical barrier in the editing of the moving picture data and to provide the user with the convenience of omitting the editing process to the maximum.

In addition, in the aspect of a voice maker who directly produces voice data, there is an additional effect that voice data can be provided in providing the technology according to the present invention, and additional profit can be obtained therefrom.

1 to 5 are flowcharts of a method of automatically editing moving picture data and voice data according to each embodiment of the present invention.
6 to 9 are examples of screens output to the user terminal according to the implementation of each embodiment of the present invention.

Various embodiments and / or aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it will also be appreciated by those of ordinary skill in the art that such aspect (s) may be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more aspects. It is to be understood, however, that such aspects are illustrative and that some of the various ways of practicing various aspects of the principles of various aspects may be utilized, and that the description set forth is intended to include all such aspects and their equivalents.

In addition, various aspects and features will be presented by a system that may include multiple devices, components and / or modules, and so forth. It should be understood that the various systems may include additional devices, components and / or modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in connection with the drawings Must be understood and understood.

As used herein, the terms "an embodiment," "an embodiment," " an embodiment, "" an embodiment ", etc. are intended to indicate that any aspect or design described is better or worse than other aspects or designs. . As used herein, the terms 'component,' 'module,' 'system,' 'interface,' and the like generally refer to a computer-related entity and include, for example, hardware, It can mean software.

In addition, the term "or" is intended to mean " exclusive or " That is, it is intended to mean one of the natural inclusive substitutions "X uses A or B ", unless otherwise specified or unclear in context. That is, X uses A; X uses B; Or when X uses both A and B, "X uses A or B" can be applied to either of these cases. It should also be understood that the term "and / or" as used herein refers to and includes all possible combinations of one or more of the listed related items.

It is also to be understood that the term " comprises "and / or" comprising " means that the feature and / or component is present, but does not exclude the presence or addition of one or more other features, components and / It should be understood that it does not.

It is also to be understood that the singular forms "a" and "an" above, which do not expressly state otherwise in this specification, include plural representations. Thus, in one example, a " component surface " includes one or more component surfaces.

Also, terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, are intended to be inclusive in a manner that is generally understood by those of ordinary skill in the art to which this invention belongs. Have the same meaning. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and, unless explicitly defined in the embodiments of the present invention, are intended to mean ideal or overly formal .

1 to 5 are flowcharts of a method of automatically editing moving picture data and audio data according to each embodiment of the present invention.

Referring first to FIG. 1, a method for automatic editing of moving picture data and voice data in the present invention is performed in a computing device including a main memory storing instructions executable by one or more processors and processors as mentioned above It will be understood. Hereinafter, a method for automatically editing moving picture data and voice data according to the present invention will be described as being performed by an 'apparatus', which means all the apparatuses capable of performing the functions of the present invention including the above-described computing apparatus It will be understood.

First, referring to FIG. 1, the apparatus includes moving image data to be edited from a user terminal, the moving image data including a plurality of image data arranged in order of reproduction time, (S10) of receiving data.

In the present invention, moving picture means original data to be edited in the form of combining video or audio data. That is, it means data immediately before the function of the present invention, in which voice data is automatically matched and combined, is stored. Accordingly, it will be understood that the first data may mean data in which only image data exists, data in which image and audio data are combined, and data in which the performance of the embodiment of the present invention is already applied.

As described above, the first data may include a plurality of pieces of image data arranged in order of reproduction time, and data on a combining order and a combining relation of each image data. This generally means a plurality of still image data included in moving image data. It will be understood that the moving picture data is expressed as moving a plurality of still pictures, such as a film, in accordance with reproduction, so a detailed description of the contents will be omitted.

The plurality of image data may be composed of pixel data and pixel combination data such that a plurality of pixels constituting each image data are positioned in the corresponding pixel region as described later. Each pixel data is configured to have an RGB value so that one pixel of the image data is constituted as each pixel data is expressed in the area where each pixel data should be located based on the pixel-combination data.

A plurality of image data is determined for each reproduction time point, and image data mapped at a predetermined point in time according to the reproduction is loaded and reproduced as a moving image.

Meanwhile, as described later, the first data may include information input from the user terminal when generating the first data. The information may be included in the first data as attribute information of the first data as described later. The attribute information of the first data may include preference information on the music inputted by the user. The preference information for the music may include information about the music genre, tempo, atmosphere, lyrics, composer, and the like. That is, it can be understood that the attribute information of the first data includes at least keyword information on the music inputted by the user.

In step S10, the apparatus receives all the information included in the above-mentioned first data from the user terminal, thereby performing a pre-processing operation for performing the function of the present invention to be described later.

The apparatus then performs step S20 of extracting image analysis data including color information of a plurality of image data and information on a change pattern according to moving picture reproduction of a plurality of image data included in the received first data do. A concrete example of step S20 is shown in Fig.

Referring to FIG. 2, in step S20, the apparatus first calculates object data included in a plurality of image data based on a combination of a plurality of pixels included in a plurality of image data included in the first data, A step S21 of extracting RGB values of pixels is performed.

In step S21, for example, the apparatus extracts pixels representing a boundary of an object among the plurality of pixels based on a change in an RGB value between the plurality of pixels, and extracts object data based on a combination form of the pixels. For example, the object data may include header information for identifying a specific object such as a group of pixels meaning a specific object, a group of pixels representing a person, and data about pixels included in the object.

When step S21 is performed, the apparatus performs step S22 of calculating variation information of the object data included in the plurality of image data, using the reproduction order of the plurality of image data.

For example, the group of pixels included in the object data may be configured differently for each image data as the image data is reproduced as a moving image. That is, as the group of pixels included in the object data changes, the object data can be perceived as fluctuating.

For example, a group of pixels included in object data representing a person may be composed of different pixel groups in adjacent image data according to a reproduction order of image data in which the object data is recognized. This is an event in which the size of the object data of a person is changed, the person is expressed as moving, or the color of object data representing a person is changed.

That is, in step S22, when it is detected that the information of the group of pixels included in the object data is changed, the variation information of the object data is calculated to detect whether the recognized object is changed.

When the operation of step S22 is completed, the apparatus processes the variation information of the object data into information on the change pattern, and uses the RGB values of the pixels to process the color information of the plurality of image data, And the step of extracting image analysis data (S23) is performed.

The information generated as a result of performing the steps S20 and S23 may be information on patterning the variation information of the object data described above. For example, image analysis data for a particular playback interval may be interpreted as meaning a moving image of several people wearing bright clothing. In this case, the change of the object in which the grass grass is shaken can be detected in the object data representing several people and the grassland, and the change in the object data can be detected . Also, grass and meadow grass and sunlight can be detected as color information.

In addition to the method described in FIG. 2, it will be appreciated that various methods of sensing motion pictures may be used in step S20.

Referring back to FIG. 1, when the step S20 is completed, the apparatus stores attribute information of the first data stored in one area of the first data and image analysis data extracted and generated in step S20, (S30) of comparing at least one piece of audio data with the audio data previously stored in the library and matching the audio data within the reproduction time interval of the first data with a part of the reproduction time interval.

In the present invention, voice data refers to voice data input from an external music producer through a specific process. That is, this means audio data produced by a corresponding producer input from a terminal of a producer registered as a producer of audio data in a device, excluding other audio data whose copyright may be a problem. Of course, other external audio data may be used to the extent that the problem of copyright is solved.

Step S30 is a series of processes in which the device automatically selects a specific playback interval and automatically selects audio data to be combined with the corresponding playback interval. As described above, in the corresponding process, the apparatus selects the reproduction interval and the audio data (audio data) using the attribute information of the first data, which is the information about the music preferred by the user, or the image analysis data analyzed in step S20, Lt; / RTI >

That is, in step S30, in accordance with the pattern / tempo information including the variation information of the image derived as the device analyzes the image, the color information, and the like, based on the preference information inputted by the user about the first data, And selects the audio data matching the corresponding information and automatically matches the audio data to the reproduction section.

In the present invention, in order to match with all or a part of the first data, the information on which the matching is based may be combined or stored. For example, pre-stored audio data includes first identification information indicating whether the data is at least background sound data or effect sound data, second identification information indicating a genre of the audio data, and a tempo of the audio data The third identification information indicating the third identification information. For example, the voice data may be background sound data, the genre may be set to dance, and the tempo may be set to a medium tempo. As described above, the first to third identification information of the voice data may be received together with the voice data from the voice data producer terminal, but may be automatically generated by the voice recognition program and stored in the voice data.

A concrete example of step S30 will be described with reference to flowcharts shown in FIGS. 3 to 5. FIG.

Referring to FIG. 3, in the matching of audio data and first data, the apparatus determines whether a variation rate of an average value or a mode value of extracted RGB values among a plurality of image data included in a reproduction time period of the first data is preset A step S31 of calculating an average value or a mode value of the extracted RGB values for a plurality of image data of a successive section that is less than the critical variation rate is performed.

It is difficult to match the same voice data when the color of the corresponding image data is rapidly changed. Therefore, the above-mentioned one period including the continuous plurality of image data existing within the critical variation ratio is matched with the voice data To be reproduced. For example, the threshold variation can be set to 25% of the RGB value, for example, as the rate at which the color completely fluctuates.

When the reproduction interval in which the audio data is to be matched is set in step S31, the apparatus determines whether the audio data having the above-described second identification information or the third identification information matching the color identification period to which the average value or the mode value of the RGB value belongs (S32) of matching the audio data and the first data to be reproduced during a reproduction time of a plurality of image data of one section.

As the first to third identification information of the audio data, the matching information may be set in advance according to the above-described color identification period. That is, when the hue is in the red section, all of the voice data of the fast tempo of the dance genre can be matched. It will be appreciated that other implementations may be variously modified and applied.

In the present invention, the first identification information of the audio data is a reproduction interval of a plurality of image data automatically grasped as described above, that is, a variation rate of an average value or a mode value of extracted RGB values among a plurality of image data in the example of FIG. 3 Is used to determine whether to match the background sound data or the sound effect data with the first data according to the length of the playback period of the image data in a successive section that is less than a predetermined threshold variation rate.

For example, when the length of one section recognized by the above procedure exceeds a predetermined length or a predetermined length (for example, a length corresponding to 70% of the entire playback period), the first data The matching voice data can be matched using the second and third identification information. When the length of one section is less than a predetermined length or ratio (for example, a length exceeding 5% of the entire reproduction section), the first data of the sound effect data is matched using the second and third identification information Can be matched. Of course, the predetermined lengths or the total to relative ratios may be modified according to various embodiments of the present invention.

In addition, an embodiment may be used in which the playback section is directly selected from the user terminal, and the first identification information is applied according to the playback section according to a similar criterion to the first example, so that the background data or the sound effect data is applied to the first data. Of course, at this time, instead of automatically selecting the playback interval and automatically matching the audio data, the plurality of image data included in the playback interval selected from the user terminal is automatically analyzed according to the criteria described above or described below, A process of automatically matching voice data on the basis will be performed.

Referring to FIG. 4, the apparatus includes information on the size of the object data included in the variation information of the object data among the plurality of image data included in the reproduction time period of the first data, (Step S33).

At this time, the playback section may be automatically selected as shown in FIG. For example, the apparatus can automatically select a playback interval from a time point at which the variation starts, such as a size of the object data increases or decreases, to a point at which the variation of the size of the object data stops, based on the variation information of the size of the object data have. Further, the apparatus may process the motion information, analyze the pattern of movement of the object data, and select the time for which the pattern is held as the playback section. Of course, in addition to the above example, various embodiments can be applied in which the playback period can be selected according to the size and movement of the object data.

When the step S33 is performed, the apparatus calculates a value calculated by using the variation information of the size of the object data or the movement information as at least one of the size variation rate of the object data, the size variation ratio of the object data, The audio data having the second identification information or the third identification information is reproduced during the reproduction time of the plurality of image data in the time interval in which the movement information of the object data is calculated, (Step S34).

For example, when the object data is gradually decreased, all the voice data having the slow tempo of the ballad genre as the second and third identification information can be matched to the first data of the corresponding reproduction section. Alternatively, when the moving pattern is recognized as the object data is slowly walking, the voice data having the slow tempo of the ballad genre as the second and third identification information is selected as the background sound or the effect sound according to the length of the playback section, 1 < / RTI > data. Of course, various examples similar to those described above can be applied.

Referring back to FIG. 1, the attribute information of the first data described above may be reflected in all the embodiments described in the description of FIGS. 3 and 4. FIG.

As described above, the attribute information of the first data may include preference information on the music input when generating the first data from the user terminal. Also, as described above, there may be at least one, e.g., a plurality of, audio data each matching the first data.

At this time, in step S30, the apparatus can control the audio data corresponding to the preference information of the music included in the attribute information of the first data among the audio data matched to the first data to finally match the first data.

For example, in a state in which both the first data and the A-voice data of the medium tempo of the ballad genre and the B-voice data of the medium tempo of the dance genre are all matched in the same first reproduction period and the preference information In the case where the dance genre is included, the B voice data is finally matched to the reproduction section of the first data.

On the other hand, when a plurality of pieces of audio data are matched in the first data to be the same matching object as described above, a process similar to the embodiment of Fig. 5 can be performed.

Referring to FIG. 5, in operation S30, the apparatus selects at least one audio data to be matched, and transmits at least one selected audio data to the user terminal (S35).

In step S35, for example, a process of providing voice data to a user terminal so that a user can confirm a plurality of voice data matched to the corresponding playback interval on a screen of an interface output to use the process described above it means.

When step S35 is performed, the apparatus performs step S36 of receiving a selection input for one of the voice data transmitted in step S34 from the user terminal, and transmits voice data corresponding to the selection input of the user terminal 1 data (step S37).

Finally, the embodiment of FIG. 5 refers to a process of recommending voice data to the user in accordance with automatic matching and finally matching the voice data with the first data based on the user's selection of the recommended voice data. For the purpose of such a process, the embodiment of Fig. 5 may be performed in combination with the embodiment of Figs. 1-4.

That is, when a plurality of audio data is selected, predetermined audio data is selected by the device so that the user's music preference information is reflected, and the audio data selected from the user terminal is finally matched to the first data after transmitting the selected audio data to the user terminal It is.

1, when the above-described process is completed, the apparatus combines the first data and the first data and the audio data matched with each other, thereby generating second data in a form in which the moving image data and the audio data are finally combined To the user terminal (S40), and the automatic editing process of the moving picture data and the audio data is completed.

According to the above-described process of the present invention, the user can use an editing process of selecting any one of the voice data recommended by the device and combining the selected voice data to the moving picture without checking the voice data.

In addition, the device can automatically select a combining period of the voice data, thereby providing convenience to the user.

Accordingly, it is possible to minimize the process of selecting the audio data at the time of editing the moving picture and the process of selecting the reproduction section of the moving picture to be combined with the audio data, It is possible to minimize the technical barrier in the editing of the moving picture data and to provide the user with the convenience of omitting the editing process to the maximum.

In addition, in the aspect of a voice maker who directly produces voice data, there is an additional effect that voice data can be provided in providing the technology according to the present invention, and additional profit can be obtained therefrom.

6 to 9 are examples of screens output to the user terminal according to the implementation of each embodiment of the present invention. In the following description, portions overlapping with those in the description of Figs. 1 to 5 will be omitted.

Through the screen 100 of FIG. 6, users can input moving image data to be edited, that is, first data. At this time, the editing interface as shown in FIGS. 7 to 9 may be output to the user terminal.

Referring to the screen 110 of FIG. 7, it is possible to confirm the reproduction of the moving picture through the screen 111 on which the moving picture is reproduced, and to confirm a plurality of image data constituting the moving picture through the frame screen 112. Through the analysis screen 113, the image analysis data can be confirmed as a graph and a text through the above-described image analysis data for a plurality of image data constituting the moving image.

Referring to the screen 120 of FIG. 8, in addition to the screen 121 on which the moving image is reproduced, the playback interval information 122 can be confirmed. At this time, a playback interval T1 in which audio data is automatically matched may be set, or a user may select the playback interval T1 as described above.

In the screen 120 of FIG. 8, the users can check the list 123 of the sound data, that is, the sound data automatically matched to the corresponding reproduction interval T1, and can preview each voice data through the reproduction menu 124 have. Thereafter, any one of the audio data can be selected through the selection menu 125, and the audio data can be finally matched with the audio data for the reproduction section (T1).

On the other hand, in the screen 130 of FIG. 9, the users check the playback interval information 132 in addition to the screen 121 on which the moving picture is played back, and for example, the background sound data automatically matched to be reproduced with respect to the entire interval, (133), and each audio data can be previewed through the playback menu (134). Then, any one of the audio data can be selected through the selection menu 135, and the audio data can be finally matched as a background sound for the entire section.

In one or more exemplary implementations, the functions presented herein may be implemented in hardware, software, firmware, or a combination thereof. When implemented in software, the functions may be stored on or transmitted via one or more instructions or code on a computer readable medium. Computer-readable media includes computer storage media and communication media including any medium for facilitating transfer of a computer program from one place to another. The storage medium may be any general purpose computer or any available medium that can be accessed by a special purpose computer. By way of example, and not limitation, such computer-readable media can comprise any form of computer readable medium, such as RAM, ROM, EEPROM, CD-ROM or other optical disk storage media, magnetic disk storage media or other magnetic storage devices, But not limited to, a general purpose computer, a special purpose computer, a general purpose processor, or any other medium that can be accessed by a particular processor.

In addition, any connection means may be considered as a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source over wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared radio, and microwave, Wireless technologies such as cable, fiber optic cable, twisted pair, DSL, or infrared radio, and microwave may be included within the definition of such medium. The disks and discs used herein may be compact discs (CDs), laser discs, optical discs, DVDs, floppy disks,

This disc contains discs where discs reproduce data magnetically, while discs reproduce data optically through a laser. The combinations may also be included within the scope of computer readable media.

Those of ordinary skill in the art will appreciate that the various illustrative elements, components, logical blocks, modules, and algorithm steps described above may be implemented as electronic hardware, computer software, or combinations thereof. In order to clarify the interchangeability of hardware and software, various illustrative components, blocks, modules and steps have been described in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement these functions in varying ways for each particular application, but such implementation decisions are not necessarily outside the scope of the invention.

The various illustrative logical blocks and modules described herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, Discrete hardware components, or any combination designed to implement the functions described herein. A general purpose processor may be a microprocessor; In an alternative embodiment, such a processor may be an existing processor, controller, microcontroller, or state machine. A processor may be implemented as a combination of computing devices, such as, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or a combination of such configurations.

For a hardware implementation, the various illustrative logic, logic blocks, and modules of processing units described in connection with the aspects disclosed herein may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs) Such as digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), discrete gate or transistor logic, discrete hardware components, general purpose processors, Controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. The general purpose processor may be a microprocessor, but, in the alternative, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other suitable configuration). Additionally, at least one processor may include one or more modules capable of implementing one or more of the steps and / or operations described herein.

In addition, various aspects or features described herein may be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. Moreover, steps and / or operations of a method or algorithm described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. Additionally, in some aspects, steps or acts of a method or algorithm may be present as a machine-readable medium, or as a combination of at least one or any combination of codes or instructions on a computer-readable medium, It can be integrated into computer program stuff. The term article of manufacture as used herein is intended to encompass a computer program accessible from any suitable computer-readable device or medium.

The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features presented herein.

Claims (9)

A method for automatic editing of moving picture data and audio data, which is performed in a computing device including at least one processor and a main memory for storing instructions executable by the processor,
Receiving first data as a moving picture data to be edited from a user terminal, the first data including a plurality of image data arranged in order of reproduction time, the data being reproduced by the plurality of image data during reproduction;
Extracting image analysis data including color information of a plurality of image data and information on a change pattern according to moving picture reproduction of a plurality of image data embedded in the received first data;
Comparing the attribute information of the first data and the image analysis data stored in one area of the first data with pre-stored audio data, and storing at least one audio data in all or a part of the reproduction time period of the first data Matching step; And
And combining the first data and the audio data matched with the first data to generate second data in which the moving picture data and the audio data are combined and providing the second data to the user terminal. Automatic editing of data. The method according to claim 1,
Wherein the extracting comprises:
Extracting object data included in a plurality of image data and RGB values of each pixel based on a combination form among pixels included in the plurality of image data included in the first data;
Calculating variation information of object data included in a plurality of image data using a reproduction order of the plurality of image data; And
Processing the variation data of the object data into information on the variation pattern and extracting the image analysis data by processing the color information of the plurality of image data using the RGB values of the pixels; Wherein the video data and the audio data are automatically edited. 3. The method of claim 2,
Wherein the pre-stored voice data includes at least first identification information indicating whether the data is at least background sound data or effect sound data, second identification information indicating a genre of voice data, and a second identification information indicating a tempo of voice data 3 < / RTI > identification information. The method of claim 3,
The matching step comprises:
The image processing apparatus according to claim 1, wherein, in a case where a plurality of pieces of image data included in a reproduction time period of the first data include a plurality of pieces of RGB data extracted for a plurality of image data of a continuous interval having a variation rate of an average value or a mode value of extracted RGB values less than a predetermined threshold variation rate Calculating an average value or a mode value; And
The audio data having the second identification information or the third identification information matched with the color identification section to which the average value or the mode value of the RGB value belongs is reproduced during the reproduction time of the plurality of image data of the continuous section, And automatically matching the audio data with the audio data. The method of claim 3,
The matching step comprises:
Calculating movement information of the object data included in the variation information of the object data or movement information of the object data among the plurality of image data included in the reproduction time period of the first data; And
A second identification that matches at least one of a size variation rate of the object data, a size variation ratio of the object data, and a movement speed of the object data, the value being calculated using the variation information of the size of the object data or the movement information, Information or third identification information is reproduced during the reproduction time of the plurality of image data in the time interval in which the movement information of the object data is calculated or the variation information of the size of the object data is matched with the first data And automatically editing the moving picture data and the audio data. The method of claim 3,
The matching step comprises:
And the background sound data or sound effect data is matched to the first data by using the first identification information of the audio data in accordance with the length of the reproduction section of the plurality of image data included in the extracted image analysis data. And a method for automatically editing voice data. 7. The method according to any one of claims 3 to 6,
Wherein the attribute information of the first data is preference information on music input from the user terminal when the first data is generated and the audio data matching the first data is at least one,
The matching step comprises:
And finally matching the audio data corresponding to the preference information on the music included in the attribute information of the first data with the first data among the audio data matching the first data. Automatic editing method. The method according to claim 1,
The matching step comprises:
Selecting at least one of the matching voice data and transmitting the selected voice data to the user terminal;
Receiving a selection input for one of the voice data transmitted from the user terminal; And
And finally matching the audio data corresponding to the selection input of the user terminal to the first data. 22. A computer-readable medium,
The computer-readable medium storing instructions that cause a computer to perform the steps of:
Receiving first data as a moving picture data to be edited from a user terminal, the first data including a plurality of image data arranged in order of reproduction time, the data being reproduced by the plurality of image data during reproduction;
Extracting image analysis data including color information of a plurality of image data and information on a change pattern according to moving picture reproduction of a plurality of image data embedded in the received first data;
Comparing the attribute information of the first data and the image analysis data stored in one area of the first data with pre-stored audio data, and storing at least one audio data in all or a part of the reproduction time period of the first data Matching step; And
And combining the first data and the audio data matched with the first data to generate second data combined with the moving picture data and the audio data and providing the generated second data to the user terminal. media.

Images