KR102435244B1 - An apparatus for providing a producing service of transformed multimedia contents using matching of video resources - Google Patents

Abstract

The present invention relates to a device for providing multimedia conversion content production service using video resource matching of audio information. The service providing device according to an embodiment of the present invention comprises: a voice data processing module collecting voice data of a first language; an input unit converting the voice data of the first language received from the voice data processing module into a text sentence of the first language according to text recognition, obtaining the converted text sentence as conversion target data, and processing the identification of the format of the conversion target data; a target language setting unit configured to set a target language corresponding to the conversion target data; an element information extraction unit extracting element information corresponding to the target language from the conversion target data; a content composition/conversion unit configured to obtain multimedia converted content by synthesizing and converting multimedia content based on the target language-based image resource matching corresponding to the element information; and an output unit outputting the multimedia converted contents. The present invention is to easily and quickly create multimedia conversion contents based on target data without the need for a separate specialized tool or expert participation.

Description

Device for providing multimedia conversion contents production service using video resource matching of audio information

The present invention relates to a service providing apparatus. More specifically, the present invention relates to an apparatus for providing a multimedia conversion content production service using image resource matching of text converted from voice information.

With the recent development of video and content production technology, various personalized multimedia contents are being produced and distributed and consumed through social networking services. In the multimedia content sharing service represented by YouTube, the number of creators and users is increasing exponentially, and various specialized tools for video content production are also being developed in terms of user convenience.

However, it is still not convenient to use these specialized tools. In order to produce high-quality multimedia contents, the time and technical manpower of experts who use specialized tools well must be invested, and the difficulty of manufacturing data for production and production costs are incurred accordingly.

In particular, this problem is more pronounced in the work of converting existing text format documents into multimedia contents to enhance information delivery power. In order to produce a text document such as a normal report as video content, collection of related video data, use of specialized tools, technical personnel and time costs of experts are excessively required.

In addition, time and money may be required to pre-produce a text format document, and there is a problem in that it is necessary to have a specialized text worker to create video content.

The present invention has been devised to solve the above problems, and resource matching, conversion, and processing processing according to preset and learned analysis process for target data including text information automatically converted from voice information rather than multimedia content format Accordingly, an object of the present invention is to provide a method and an apparatus for providing a converted content creation service that can easily, easily, and quickly produce multimedia converted content based on target data without a separate professional tool or expert participation.

According to an embodiment of the present invention, there is provided a method for operating a service providing apparatus, the method comprising: collecting voice data of a first language; obtaining conversion target data from the voice data of the first language; when a target language corresponding to the conversion target data is set, extracting element information corresponding to the target language from the target data; performing multimedia content synthesis and conversion processing based on the target language-based image resource matching of the element information to obtain multimedia converted content; and outputting the multimedia conversion content.

In addition, an apparatus according to an embodiment of the present invention for solving the above problems, a service providing apparatus, comprising: a voice data processing module for collecting voice data of a first language; a target language setting unit configured to set a target language corresponding to the conversion target data; an element information extraction unit for extracting element information corresponding to the target language from the target data; a content synthesizing and converting unit for obtaining multimedia converted content by performing multimedia content synthesizing and converting processing based on the target language-based image resource matching corresponding to the element information; and an output unit for outputting the multimedia conversion content.

On the other hand, the method according to an embodiment of the present invention for solving the above problems may be implemented as a program for executing the method in a computer and a recording medium in which the program is recorded.

According to an embodiment of the present invention, when voice data of a first language is collected and obtained as the conversion target data, and a target language is set, element information is extracted from the target data, and a target language-based video resource corresponding to the element information It is possible to provide a production interface based on matching, and by performing multimedia content synthesis and conversion service according to a user input to the production interface, it is convenient to produce multimedia image conversion content converted from target data.

Accordingly, the service providing apparatus according to an embodiment of the present invention performs resource matching, conversion, and resource matching based on a target language according to a preset and learned analysis process for target data such as a normal document or voice rather than a multimedia content format. It can be processed, and it enables the creation of global multimedia conversion contents based on the target data easily, easily and quickly without the participation of a separate professional tool, translator or expert.

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.
2 is a block diagram illustrating a service providing apparatus according to an embodiment of the present invention in more detail.
3 is a flowchart illustrating an operation of a service providing apparatus according to an embodiment of the present invention.
4 is an exemplary diagram of synthesized-converted video multimedia content according to an embodiment of the present invention.
5 is a diagram for explaining a process of converting input data into multimedia content data according to an embodiment of the present invention.
6 to 7 are diagrams for explaining a resource database according to an embodiment of the present invention.
9 is a view for explaining in more detail a manufacturing interface according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices that, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. Further, it is to be understood that all conditional terms and examples listed herein are, in principle, expressly intended solely for the purpose of enabling the concept of the present invention to be understood, and not limited to such specifically enumerated embodiments and states. should be

Moreover, it is to be understood that all detailed description reciting the principles, aspects, and embodiments of the invention, as well as specific embodiments, are intended to cover structural and functional equivalents of such matters. It is also to be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on a computer-readable medium and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

In addition, the clear use of terms presented as processor, control or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile memory. Other common hardware may also be included.

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in the description of the present invention, when it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically illustrating an entire system according to an embodiment of the present invention.

Referring to FIG. 1 , a system according to an embodiment of the present invention may include a service providing apparatus 100 , a user terminal 200 , and a multimedia content server 300 .

The service providing apparatus 100 according to an embodiment of the present invention may process the conversion target data from the user terminal 200 as input data, and perform resource matching-based multimedia content conversion of element information corresponding thereto, The converted multimedia content may be output to the multimedia content server 300 and distributed to one or more service user terminals.

Furthermore, the service providing apparatus 100 according to an embodiment of the present invention may further include a voice data processing module 210 , and the voice data processing module 210 may be input to the user terminal 200 or a separate external device. A function of collecting voice data of the first language input from the device and transmitting it to the service providing device 100 may be performed.

Accordingly, the conversion target data may be obtained from the voice data of the first language. In this case, the service providing apparatus 100 sets the target language corresponding to the conversion target data from the target data. Extracting element information corresponding to , and performing multimedia content synthesis and conversion processing based on the target language-based image resource matching of the element information to obtain multimedia converted content, wherein the multimedia converted content is a multimedia content server (300) It may be output as , and distributed to one or more service user terminals or user terminals 200 .

To this end, more specifically, the service providing apparatus 100, when the conversion target data is input from the user terminal 200, extracts element information from the target data, and based on image resource matching corresponding to the element information Provides a production interface to the user terminal 200, performs multimedia content synthesis and conversion processing according to a user input to the production interface to obtain multimedia conversion content, and transfers the multimedia conversion content to the multimedia content server 300 output as

In addition, the service providing apparatus 100, when the voice data of the first language input through the voice data processing module 210 is obtained from the user terminal 200, acquires conversion target data based on the voice data, When a target language corresponding to the transformation target data is set, element information corresponding to the target language is extracted from the target data, and based on target language-based image resource matching corresponding to the element information, to the user terminal 200 It provides a production interface, performs multimedia content synthesis and conversion processing according to a user input to the production interface, obtains multimedia converted content, and outputs the multimedia converted content to the multimedia content server 300 .

Accordingly, the multimedia converted content converted from the input target data may be distributed to one or more other user terminals through the multimedia content server 300, and the multimedia content server 300 provides various information providing services based on the multimedia converted content. can be processed

The user terminal 200, the service providing apparatus 100, and the multimedia content server 300 may be connected by wire or wirelessly through a network, and each user terminal 200 and the service providing apparatus 100 may be connected to each other for communication between networks. ) and the multimedia content server 300 transmit and receive data through an Internet network, LAN, WAN, PSTN (Public Switched Telephone Network), PSDN (Public Switched Data Network), cable TV network, WIFI, mobile communication network and other wireless communication networks, etc. can do. The user terminal 200 , the service providing apparatus 100 , and the multimedia content server 300 may include respective communication modules for communicating with a protocol corresponding to each communication network.

In addition, the user terminal 200 described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and the like. may be included, but the present invention is not limited thereto, and may be various devices capable of user input and information display other than that.

In such a system, the user terminal 200 may receive a multimedia content conversion service based on resource matching of input data from the service providing device 100 , and may receive an additional information service based on the converted multimedia content data. .

In particular, in extracting element information, the service providing apparatus 100 according to an embodiment of the present invention extracts key element information according to a pattern and statistical similarity of input data to be converted based on text according to a preset natural language processing algorithm. resource matching processing that optimizes matching video, image, text, subtitle, animation, font (color, size, font) and audio by frame merge layer using the extracted text-based element information, It is possible to provide a production interface using matching-processed element information and to generate optimized multimedia conversion content based on the frame merging layer according to a user input.

Accordingly, the service providing apparatus 100 according to an embodiment of the present invention analyzes elements even when general documents or image data such as market reports, statistical reports, company introductions, commercial flyers, resumes, and self-introductions in various formats are input. By processing element information and video content resource matching extracted through Even non-experts in image editing enable the production of general document-based multimedia image content according to the matching proposal of the service providing apparatus 100 .

Furthermore, when conversion target data is obtained based on voice data recognition of the first language, image content resource matching in various target languages can be processed, so that for conversion target data input in one language, various countries Multimedia conversion content suitable for culture can be created. This has the effect of making it possible to easily and easily create global multimedia conversion contents suitable for each country's language or culture, not only for non-experts in video editing, but also for users without knowledge of the language and culture of each country.

2 is a block diagram illustrating in more detail an apparatus for providing a service according to an embodiment of the present invention.

Referring to FIG. 2 , the service providing apparatus 100 according to an embodiment of the present invention includes a target data input unit 110 , a target language setting unit 117 , an element information extraction unit 120 , and an image resource matching unit 130 . ), a production interface providing unit 140 , a content synthesis conversion unit 150 , a learning database 160 , a resource database 180 , and an output unit 170 .

First, the input unit 110 receives target data for multimedia content conversion from the user terminal 200 and transmits it to the element information extraction unit 120 .

The input unit 110 may include one or more input interfaces for receiving target data from the user terminal 200 . For example, the target data may be document data input from the user terminal 200 , and may include data in various formats, such as a report, a company introduction, a self-introduction letter, and a commercial advertisement document. Furthermore, the target data may be a news article document extracted from a specific site, or may include a social media (SNS) document.

In addition, the input unit 110 converts the voice data of the first language received from the voice data processing module 210 into a text sentence of the first language according to text recognition, and converts the converted text sentence into the conversion target data. may be obtained.

For example, the target data may be voice data of the user's first language input through the voice data processing module 210, company introduction voice data, self-introduction voice data, commercial advertisement voice data, report voice data, etc. It may include voice data in various formats. Furthermore, the voice data of the first language may be voice data of a video extracted from a specific site, or may include voice data acquired from social media (SNS).

In addition, the input unit 110 may process the format identification of the target data input from the user terminal 200 , and the format identification information may be transmitted to the element information extraction unit 120 .

The format identification information may indicate, for example, a document type, and various document types such as novels, essays, news articles, drafts, plans, plans, sales reports, settlement reports, and conference reports may be exemplified.

In addition, the format identification information may indicate, for example, a voice data type, and various voice data types such as company introduction announcement, self-introduction, commercial advertisement, face-to-face report, song, music video, and press conference may be exemplified.

Also, the input unit 110 may further receive main element data corresponding to the target data. The main element data may include, for example, a key keyword input from the user terminal 200, a type of voice data, a report type, company characteristic information, a main company name, a main company name, a main person name, etc., and the element information extraction unit When the element information of (120) is extracted, a weight corresponding to the main element data may be assigned.

The element information extraction unit 120 may extract element information for classifying the input target data into one or more element data matching the image resource.

Here, the element information extraction extracts element data in a text format from the target data using a preset natural language processing algorithm, and transmits the extracted element information to the image resource matching unit 130 .

In particular, the element information extractor 120 may determine a natural language processing process of the target data for matching with an image resource, based on main element data and format identification information of the target data. Here, the natural language processing process may be exemplified by a text summary process previously learned by a deep learning process.

Accordingly, the element information extraction unit 120 may perform a text summary process, extract important sentences or words from the target data, synthesize one or more summary sentences, and output it as element information.

Also, in determining the text summary process, the element information extractor 120 may apply one or more different language models according to the format identification information of the target data. An extraction model or a synthesis model may be exemplified as the language model, and different models may be determined according to the characteristics of the company and the type of document.

For example, the element information extraction unit 120, when large or medium-sized company information is included in the main element information input in response to the target data, in response to the format identification information of a mass document such as a report or terms and conditions, the target By applying the extraction model to the data, important sentence information in the original text can be extracted as element information.

In addition, the element information extraction unit 120, when the main element information input in response to the target data includes small business, startup, or creator information, the format of a small document such as a news column, lecture material, lifestyle material, etc. Corresponding to the identification information, by applying a synthesis model to the target data, important keyword information in the original text is selected, and sentence information synthesized as one summary sentence may be extracted as element information.

Accordingly, element information according to an embodiment of the present invention may include one or more important sentence information extracted from target data or obtained based on a synthetic language model. Such sentence information may correspond to a layer unit of one image resource matching frame, and appropriate resource matching may be processed for each sentence information to constitute one image frame layer unit.

And, the image resource matching unit 130, based on the learning database 160 and the resource database 180, performs an optimized resource matching process in response to the element information, and converts the resource matching information into a content synthesis and conversion unit ( 150) and the manufacturing interface providing unit 140 .

More specifically, the image resource matching unit 130 performs resource matching processing for content synthesizing transformation corresponding to element information, and the resource for content synthesizing transformation is a background image processed within a preset frame layer unit; It may include various contents such as a background image, background music, layout, motion, animation, and the like, and may be pre-stored in the resource database 180 .

In addition, the resource database 180 may store and manage resource content data received from various content servers connected through an external network. Here, the resource content data may include at least one of content attribute information, content identification information, content link information, and content data information, and the matched resource information is sent to the production interface providing unit 140 or the content synthesis conversion unit 150 . can be transmitted.

Furthermore, the resource database 180 may include a plurality of resource databases 180 classified and configured for each country and language. In this case, resource content data suitable for a country and a language corresponding to a target language set during content synthesis and conversion may be extracted and matched by the image resource matching unit 130 .

In addition, the image resource matching unit 130 may build and utilize the learning database 160 in order to match more appropriate content corresponding to the element information from the resource database 180 . The learning database 160 may build a relationship learning model for learning relationship information between resource content and element information, and in particular, a weight variable that allows more suitable resource content to be matched in response to the type of target data and main element information. can be set. Accordingly, the image resource matching unit 130 may calculate matching information in which the optimal resource content corresponding to the element information is matched by using the learning database 160 , and the calculated matching information is provided by the production interface providing unit ( 140) and the content synthesis conversion unit 150 .

For example, the image resource matching unit 130, in response to the sentence information of the element information, the background, sound, character type, target language, etc. for each image frame layer unit divided by a predetermined time unit to the learning database 160 Based on the pre-built resource database 180 can be matched.

The learning database 160 may define a large classification category and a detailed classification category of each sentence information, and analyze the correlation between the large classification and the deep learning learning result of the detailed classification, and the matched background, sound, or character type of the target document It is possible to arithmetically analyze how probabilistically it is related to the business purpose corresponding to the format.

Accordingly, the image resource matching unit 130 may acquire resource contents such as background, sound, character type, and target language for which the most optimized correlation is calculated as matching information for matching the image frame layer unit.

In addition, for example, the video resource matching unit 130 according to an embodiment of the present invention may directly create an image or audio resource content depicting a sentence of element information, or search in the resource database 180, The generated or searched resource content may be transmitted to the production interface providing unit 140 and the content synthesis converting unit 150 .

In addition, the production interface providing unit 140 configures a production interface capable of synthesizing and converting the content matched by the image resource matching unit 130 based on the matching information, and provides it to the user terminal 200 .

The production interface providing unit 140 transmits the resource content data and resource matching information to an interface application executed in the user terminal 200, or transmits the resource content data and resource matching information to the user terminal 200 through a separate API. ), or may be provided to the user terminal 200 by configuring a real-time web production interface based on the resource content data and resource matching information.

Accordingly, in the user terminal 200, in response to the target data input by the user, element information is extracted and it is possible to check the resource content matched with the image resource, process appropriate editing and processing, and input a synthesis conversion command. can Also, depending on the settings of the user terminal 200 , a conversion request to the content synthesis and conversion unit 150 may be directly input without separate editing or processing in the user terminal 200 .

The content synthesis and conversion unit 150 synthesizes and converts the target data into multimedia converted content based on the resource content data, resource matching information, and input information of the user terminal 200 .

Accordingly, the multimedia conversion content may include multimedia data in which at least one of an image, a sound, an image, an animation, a subtitle, and a font is synthesized and converted according to the target data. The synthesized and converted multimedia content may be provided to the production interface providing unit 140 , and may be transmitted to the output unit 170 according to the confirmation or upload input of the production interface providing unit 140 .

The output unit 170 may output the finally determined multimedia conversion content as converted content of the target data, which is provided to the multimedia content server 300, and may be used for various information providing services based on the target data, It may be shared to one or more other user terminals through a social network service.

For example, the information providing service may include a multimedia content conversion service using various document data such as news articles, reports, novels, essays, and blogs, and a multimedia content streaming service based thereon may be exemplified.

In addition, the service providing apparatus 100 according to an embodiment of the present invention processes image resource matching according to element information extraction not only for report data composed of long sentences, but also various newsletters composed of relatively short sentences, online comments, SNS data, etc. It can also be synthesized and converted into multimedia content through

Furthermore, the service providing apparatus 100 according to an embodiment of the present invention performs synthesis conversion processing into global multimedia content through resource matching processing corresponding to the target language for each country in response to voice data configured in a first language. be able to perform For example, even if Korean voice data is converted and input as target data, the image, music, and video content resource database for each country corresponding to the target language, Vietnamese or English, is utilized to process image resource matching corresponding to each country or language. can be performed.

Furthermore, the content synthesis and conversion unit 150 translates target data obtained by converting voice data of the first language into text into the target language, and converts the translated data back into voice or subtitle data according to user settings. Thus, the synthesis-converted multimedia content may be synthesized, and accordingly, the multimedia content may be output in a state in which voice description information or subtitle description information composed of a target language other than the first language is synthesized.

3 is a flowchart illustrating an operation of a service providing apparatus according to an embodiment of the present invention.

Referring to FIG. 3 , first, the service providing apparatus 100 receives conversion target data from the user terminal 200 or converts the conversion target data according to a recognition conversion process corresponding to the voice data reception of the voice data processing module 210 . Acquire (S101).

Then, the service providing apparatus 100 extracts element information from the target data (S103).

Thereafter, the service providing apparatus 100 processes image resource matching corresponding to the element information ( S105 ).

Then, the service providing apparatus 100 provides a production interface based on the matched image resource content to the user terminal 200 (S107).

Thereafter, the service providing apparatus 100 performs media content synthesis and conversion processing according to a user input to the production interface (S109).

Then, the service providing apparatus 100 outputs and distributes the converted multimedia content (S111).

4 is a diagram illustrating synthesis-converted video multimedia content according to an embodiment of the present invention, and FIG. 5 is a diagram for explaining a process of converting input data into multimedia content data according to an embodiment of the present invention.

First, referring to FIG. 4 , as described above, the element information extracting unit 120, as the main sentence from the target data input in the first language, says, "Go to a wonderful beach and collect seals and wonderful ships on the beach sand rocks. Sentences such as "I saw" can be extracted as element information.

Then, the image resource matching unit 130, when the target language is set to English, corresponding to the target language, the most suitable resource content corresponding to the word keyword of each element information based on the learning database 160 each country and language It can be obtained from the resource database 180 classified by each. For example, the US beach image resource corresponding to the beach keyword, the rock image resource corresponding to the beach sand and rock keyword, the seal image resource corresponding to the seal keyword, and the US ship image resource corresponding to the ship keyword may be matched. have.

Also, the image resource matching unit 130 may match subtitle, font, and font resources corresponding to the sentence information of the element information, and may perform matching processing with audio converted from the sentence information as an audio resource. In addition, the image resource matching unit 130 may match the animation information corresponding to the sentence information.

Accordingly, the content synthesis and transformation unit 150 corresponds to the image frame layer unit of the preset time section, the image resource, subtitle, font and font resource, the sound resource is matched and converted according to the layout and animation information It will be possible to create multimedia content.

For example, multimedia content related to one sentence output as a caption may be played in an image of a frame layer unit section, and the content synthesis and conversion unit 150 converts subtitles, images, and images into an image of a frame layer unit section. Arranged together, they can be synthesized and converted so that the sound is output at a preset timing. The image resource matching unit 130 may match an appropriate content data combination, animation effect, and arrangement of the content synthesis and transformation unit 150 through machine learning technology, deep learning technology, or the like.

In addition, the content synthesis and conversion unit 150 may match partial image contents for each text element, perform boundary processing between the matched partial images, and converge them into one image, and generate an image frame set using the fused image. can be built

Furthermore, the content synthesis and conversion unit 150 may refine text elements, set output timing for each text element, and match the image frame set to output timing. Then, the content synthesis conversion unit 150 inserts a synthesized subtitle synchronized with the output timing for each text element, merges the TTS voice of the synthesized subtitle, or assigns a voice characteristic to the TTS voice according to a user setting. Further processing may be performed. Accordingly, even for multimedia content corresponding to voice data in the first language, appropriate multimedia content for each target language may be fused, transformed, and synthesized.

This matching process can be understood in more detail with reference to FIG. 5 . As shown in FIG. 5A , target data composed of voice data or text of a first language is input to the input unit 110 and the target language-based element information extraction processing by the element information extraction unit 120 is performed. can

According to the element information extraction process, one or more core sentence data may be extracted as element information as shown in FIG. Based on the matching process as shown in FIG. 5(C), resource content matching with one or more images, sounds, or images stored or linked in the resource database 180 may be processed.

Here, the resource database 180 may be an internal or external database of the service providing apparatus 100 , and may use resource content service providing servers of well-known service providers as shown in FIG. 5C .

And, as shown in FIG. 5(D), the multimedia content synthesized and converted in the content synthesis and conversion unit 150 based on the matching information of the image resource matching unit 130 is transmitted through the output unit 170 to the multimedia content server. 300, and may be distributed and shared with other users.

6 to 7 are diagrams for explaining a resource database according to an embodiment of the present invention.

Referring to FIG. 6 , the resource database 180 according to an embodiment of the present invention includes an interface unit 185 , a logical model management unit 181 , a physical environment management unit 183 , a metastore database 183 , and a data storage unit. (184).

According to an embodiment of the present invention, the resource database 180 may classify and label media content data based on meta information, and load it in a form that can be analyzed in the learning database 160, and the resource content Facilitate data sharing.

To this end, the resource database 180 may perform redundant data removal, missing data correction, and abnormal data detection through pre-processing of resource content data, perform a scaling process of pre-processed data, and a well-known Long Short-Term Memory It is possible to perform data classification processing to build the learning database 160 using an algorithm such as models (LSTM).

More specifically, the interface unit 185 performs distributed input/output interface processing of resource content data classified and stored in each of the management units 181 and 182 .

The logical model manager 181 may classify, store, and manage resource contents through the metastore database 183 . Here, the metastore database 183 may store and manage metadata for indexing the big data-based content data of the data storage unit 184 physically stored in the physical environment management unit 182 . The metadata may include, for example, at least one of classification information for each user, classification information for each function, and storage classification information, and each classification information may correspond to a storage structure of the physically distributed and stored data storage unit 184 . .

For example, the data storage unit 184 may store animation, background image, sound, font (font), layout information, etc. as resource content.

7 is an example of a resource content format stored according to an embodiment of the present invention, illustrating that data type information such as video, sound, image, etc., identifier information, tag information, URL information, virtual hosting URL information, and the like are included. have.

The metastore database 183 may store and manage metadata as shown in Table 1 below as classification information.

Data classification Meta information 1 meta information 2 meta information 3 type animation /store /data /animation background image /image acoustic /sound Font (font) /log /realtime Layout information /batch

As shown in Table 1, meta information may be classified by each classification information according to data classification, and thus, necessary resources may be indexed using meta information. Accordingly, according to an embodiment of the present invention, The resource database 180 may manage the data storage unit 184 of a big data structure that is physically distributed and stored, and may index necessary resource contents using meta information of the metastore database 183 .

Therefore, the resource database 180 according to an embodiment of the present invention is not only for the purpose of storage, but it can be built in consideration of the aspect of loading stored data in a form that can be analyzed and sharing necessary data in various analysis environments. Furthermore, by enabling SQL-based data information inquiry, it is possible to increase the convenience and speed of data access.

9 is a view for explaining in more detail a manufacturing interface according to an embodiment of the present invention.

Referring to FIG. 9 , the production interface according to an embodiment of the present invention may include a graphic user interface output through the user terminal 200 , and includes a target data input interface 201 , an image editing interface 204 , and , a subtitle editing interface 202 , and a sound source and language editing interface 203 .

In addition, the service providing apparatus 100 according to an embodiment of the present invention may receive text data of a specific document through the target data input interface 201, and the input text data is an element according to a summary button input, etc. It may be used for element information extraction processing in the information extraction unit 120 .

Then, the recommended resource content according to the matching processing of the image resource matching unit 130 based on the extracted element information is transmitted to each image editing interface 204, subtitle editing interface 102, and sound source and language editing interface 203. It may be proposed as a recommended item. Accordingly, the user terminal 200 may generate the multimedia conversion content by selecting the recommended resource content.

A user of the user terminal 200 may select resource content in each of the editing interfaces, and input image conversion and SNS upload through the output interface 205 , and accordingly, conversion processing in the content synthesis conversion unit 150 . is performed, and the result may be output to the user terminal 200 or uploaded to the multimedia content server 300 and shared through a preset SNS account.

Meanwhile, the above-described method according to various embodiments of the present invention may be implemented as a program and provided to each server or device in a state stored in various non-transitory computer readable media. Accordingly, the user terminal 100 may access the server or device and download the program.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (3)

In the service providing device,
a voice data processing module for collecting voice data of the first language; Converts the voice data of the first language received from the voice data processing module into a text sentence of the first language according to text recognition, obtains the converted text sentence as the transformation target data, and identifies the format of the transformation target data an input unit for processing; a target language setting unit configured to set a target language corresponding to the conversion target data; an element information extraction unit for extracting element information corresponding to the target language from the conversion target data; an interface providing unit that provides a production interface based on image resource matching corresponding to the element information; and a content synthesizing/converting unit configured to obtain multimedia converted content by performing multimedia content synthesizing and converting processing based on the target language-based image resource matching corresponding to the element information according to a user input to the production interface; and an output unit for outputting the multimedia conversion content,
The element information extraction unit,
Extracting one or more sentence information for matching an image resource from the transformation target data, and performing a text summary process of the transformation target data to extract sentence information, wherein the text summary process identifies the format of the transformation target data It is a process using different language models determined according to information, wherein the language model includes an extraction model or a synthesis model,
The image resource matching includes a process of matching resource content for each image frame layer unit divided by a predetermined time unit with a pre-built resource database in response to the element information, wherein the resource content can match the element information At least one of video, background, image, sound, text type, or animation
Service providing device. According to claim 1,
the first language and the target language are on each other;
The target language can be set according to the image resource classification information for the image resource matching.
Service providing device. According to claim 1,
The target language includes one or more languages selected by a user among a plurality of languages,
The video resource, characterized in that extracted from the resource database for each country corresponding to the selected one or more languages
Service providing device.

Images