KR101848467B1 - Speech recognition database construction system using digital content - Google Patents

Abstract

Various embodiments of the present invention relate to a voice recognition database construction system using digital content. To this end, the voice recognition database construction system using digital content comprises: a data acquiring part acquiring digital content including voice information; a data extracting part extracting voice data from the digital content acquired from the data acquiring part; a data matching part matching subtitle data corresponding to the voice data extracted from the data extracting part; a data storing part storing the voice data and the subtitle data matched by the data matching part; and a data distributing part distributing the corresponding voice data and subtitle data from the data storing part, if a confirmation request of the voice data exists.

Description

[0001] Speech recognition database construction system using digital contents [

Various embodiments of the present invention are directed to a system for building a speech recognition database utilizing digital content.

The conventional speech recognition database system has provided a basic voice search database based on data recorded by voice actors or people limited to a specific field. Therefore, conventionally, there has been a problem that the recognition rate of a coined word or the like is remarkably low because the speech recognition database system is not continuously and / or automatically updated.

In addition, the conventional speech recognition database system has not been able to establish a speech database distinguished by age group (for example, children or the elderly), and thus has a problem that speech recognition rate is remarkably low at a specific age range.

In addition, since the conventional speech recognition database system is constructed based on contents recorded in a studio, there is a problem that speech recognition rate is remarkably low in various environments such as outdoor.

Moreover, the conventional speech recognition database system does not include diverse contents in addition to the standard language, such as dialect, and also has a problem that the voice recognition rate for foreign languages other than Korean is remarkably low.

The above-described information disclosed in the background of the present invention is only for improving the understanding of the background of the present invention, and thus may include information not constituting the prior art.

A problem to be solved by the present invention is to provide a system for constructing a voice recognition database utilizing various digital contents such as a recorded content such as a real time content such as a television or a radio, a recorded broadcast, a streaming broadcast, an audio book, . That is, a problem to be solved by the present invention is to extract and utilize voice files and other data from video files used in broadcasting and various digital contents, (Or update) a variety of voice data compared to the conventional voice search method, and to provide a new service such as a real-time broadcast language or the like Or a composite system that can be used as a solution.

A system for constructing a voice recognition database using digital contents according to various embodiments of the present invention includes a data acquisition unit for acquiring digital contents including voice information; A data extracting unit for extracting audio data from the digital contents acquired by the data acquiring unit; A data matching unit for matching the caption data corresponding to the audio data extracted from the data extracting unit; A data storage unit for storing audio data and caption data matched by the data matching unit; And a data distribution unit for distributing the audio data and the caption data from the data storage unit when there is a request for verification of the audio data.

And a data classifier for classifying the voice data stored in the data storage unit.

The data classification unit may classify the stored voice data by morpheme, word, sentence, date, age, place, time or genre.

The digital content may be a recorded content including a real-time broadcast content including a television broadcast or a radio broadcast, a recorded broadcast, a streaming broadcast, or an audio book, or a broadcast program information.

If the digital content includes caption information, the data extraction unit may extract caption data from the digital content.

The data matching unit compares the audio data and the caption data used for a specific period of time and operates in an intersection scheme in which the most frequently overlapped audio data and caption data are matched.

The data matching unit converts the separated caption data into voice data through a TTS (Text-To-Speech) function or a STT (Speech-To-Text) function, and separates the separated voice data, And can operate in a TTS or STT scheme matching the separated caption data.

The data matching unit may operate as a time error analysis method of calculating the error between the separated voice data and the caption data statistically and matching the voice data and the caption data.

The data matching unit may operate in a voice engine interlocking manner in which the separated subtitle data is interlocked with a speech engine and the output result is matched with the separated speech data.

The data matching unit may operate in association with a past multimedia content associating interface that matches audio data with caption data in cooperation with a self-database built on a past multimedia content.

The voice recognition database construction system utilizing digital contents according to various embodiments of the present invention utilizes / extracts voice files and other data from video files used in broadcasting and various digital contents to separate / combine / match voice data A large data or an integrated system that provides a voice data base used for voice recognition through a variety of fields and can continuously generate various voice data compared to a conventional voice search method, To provide a new solution or a hybrid solution.

That is, in the conventional voice database, a basic voice search database is used based on data recorded by a voice actor or a person limited to a specific field. However, the speech recognition database construction system according to various embodiments of the present invention is used in various broadcasts To be used for the voice search base.

In addition, the speech recognition database construction system according to various embodiments of the present invention divides the constructed speech database by sentence, word, morpheme, and letter, and classifies the speech database by time, Make it available for different situations.

In addition, the speech recognition database construction system according to various embodiments of the present invention constructs various interworking systems so that the constructed speech database can be utilized for various services.

In addition, the speech recognition database construction system according to various embodiments of the present invention may also construct a real-time popular broadcast language ranking system.

1 is a block diagram showing a configuration of a speech recognition database construction system utilizing digital contents according to various embodiments of the present invention.
2 is a flowchart illustrating an operation flow of a system for building a voice recognition database using digital contents according to various embodiments of the present invention.
FIG. 3 is a diagram illustrating an example of voice / subtitle multiplex intersection analysis during operation of a voice recognition database construction system utilizing digital contents according to various embodiments of the present invention.
FIG. 4 is a diagram illustrating an example of meta-analysis during operation of a speech recognition database building system utilizing digital contents according to various embodiments of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The embodiments of the present invention are described in order to more fully explain the present invention to those skilled in the art, and the following embodiments may be modified into various other forms, It is not limited to the embodiment. Rather, these embodiments are provided so that this disclosure will be more faithful and complete, and will fully convey the scope of the invention to those skilled in the art.

In the following drawings, thickness and size of each layer are exaggerated for convenience and clarity of description, and the same reference numerals denote the same elements in the drawings. As used herein, the term "and / or" includes any and all combinations of one or more of the listed items. In the present specification, the term " connected "means not only the case where the A member and the B member are directly connected but also the case where the C member is interposed between the A member and the B member and the A member and the B member are indirectly connected do.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" include singular forms unless the context clearly dictates otherwise. Also, " comprise, " and / or "comprising, " when used in this specification, are intended to be interchangeable with the said forms, numbers, steps, operations, elements, elements and / And does not preclude the presence or addition of one or more other features, integers, operations, elements, elements, and / or groups.

Although the terms first, second, etc. are used herein to describe various elements, components, regions, layers and / or portions, these members, components, regions, layers and / It is obvious that no. These terms are only used to distinguish one member, component, region, layer or section from another region, layer or section. Thus, a first member, component, region, layer or section described below may refer to a second member, component, region, layer or section without departing from the teachings of the present invention.

It is to be understood that the terms related to space such as "beneath," "below," "lower," "above, But may be utilized for an easy understanding of other elements or features. Terms related to such a space are for easy understanding of the present invention depending on various process states or use conditions of the present invention, and are not intended to limit the present invention. For example, if an element or feature of the drawing is inverted, the element or feature described as "lower" or "below" will be "upper" or "above." Thus, "below" is a concept covering "upper" or "lower ".

In addition, the system (controller), server and / or other associated device or component in accordance with the present invention may be implemented using any suitable hardware, firmware (e.g., on-demand semiconductor), software, or a suitable combination of software, Can be implemented. For example, various components of a system (controller), a server, and / or other associated equipment or components according to the present invention may be formed on one integrated circuit chip or on a separate integrated circuit chip. In addition, the various components of the system (controller), server may be implemented on a flexible printed circuit film and formed on the same substrate as a tape carrier package, printed circuit board, or system (controller), server . In addition, the various components of the system (controller), the server, may be a process or thread running on one or more processors in one or more computing devices, which may be implemented as computer program instructions And interact with other components. The computer program instructions are stored in a memory that can be executed on a computing device using standard memory devices, such as, for example, random access memory. The computer program instructions may also be stored in other non-transitory computer readable media, such as, for example, CD-ROMs, flash drives, and the like. Further, those skilled in the art will appreciate that the functions of the various computing devices may be combined with one another, integrated into one computing device, or the functionality of a particular computing device may be implemented within one or more other computing devices Lt; / RTI > can be dispersed in the < / RTI >

As an example, a system (controller) according to the present invention, a server may be a central processing unit, a mass storage device such as a hard disk or solid state disk, an input device such as a volatile memory device, a keyboard or a mouse, Lt; RTI ID = 0.0 > commercial < / RTI >

1 is a block diagram showing the configuration of a speech recognition database construction system 100 using digital contents according to various embodiments of the present invention.

1, a speech recognition database construction system 100 according to various embodiments of the present invention includes a data input unit 110, a data extraction unit 120, a data matching unit 130, (150) and a data distribution unit (170).

In addition, the speech recognition database construction system 100 according to various embodiments of the present invention may further include a data verification unit 140 and / or a data classification unit 160.

The system 100 for constructing a speech recognition database according to various embodiments of the present invention includes a primary data storage unit 121 at the rear end of the data extraction unit 120 and a secondary data storage unit 121 at the rear end of the data matching unit 130. [ The data storage unit 131 may be connected to the tertiary data storage unit 141 at the rear end of the data verification unit 140.

In addition, the settlement management server 180 may be further connected to the data distribution unit 170.

The data receiving unit 110 plays a role of obtaining various kinds of digital contents including audio information and / or caption information. Here, the digital content includes, for example, but not limited to, television broadcasting (satellite broadcasting, IP TV, wireless broadcasting, cable broadcasting, DMB broadcasting) or radio broadcasting (wireless broadcasting, Internet broadcasting, DMB broadcasting) And / or broadcast programming information (EPG) content, including real-time broadcast content, recorded broadcasts, streaming broadcasts and / or audiobooks, and the like. Further, such digital contents generally include voice information, but may or may not include corresponding caption information. The digital contents including the caption information and the digital contents not including the caption information are slightly different from each other, and this will be described below again.

Here, the data acquisition unit 110 may include, for example, but not limited to, a connection server, a main system, and / or a collection server. The connection server connects, for example, but not limited to, a real-time broadcast content including a television broadcast or a radio broadcast, and transmits the live broadcast content to the main system. Also, the collection server collects, for example, but not limited to, recorded contents including broadcast programs, streaming broadcasts and / or audio books, and / or broadcast programming information (EPG) contents and transmits them to the main system .

The data extracting unit 120 extracts audio data from various types of digital contents obtained by the data obtaining unit 110. [ The data extracting unit 120 extracts caption data from the digital content when the digital content includes the caption information. Furthermore, the data extracting unit 120 extracts additional data (e.g., viewable age, background place, creation time, genre, creator, creative company, etc.) from the digital contents. Here, when the digital content does not have the caption information, the caption information may not be extracted.

The primary data storage unit 121 stores the audio data, the additional information, and / or the caption data extracted from the data extracting unit 120 as described above or in one piece.

The data matching unit 130 matches the subtitle data corresponding to the extracted audio data from the data extraction unit 120 and / or the primary data storage unit 121 as described above. The data matching can be performed by various methods such as a method using a voice engine, an intersection matching method, and a guess matching method.

For example, the method using the speech engine is a method in which the data matching unit 130 matches the separated resultant audio data with the output result of the separated subtitle data interlocked with the speech engine . Here, the speech engine may be, for example, but not limited to, software or a program for converting subtitle data into audio data.

As another example, the intersection matching method may include a method of comparing the audio data and the caption data used by the data matching unit 130 for a specific period or interval and matching the caption data with the audio data that is most frequently overlapped to be.

As another example, the data matching unit 130 may convert the separated caption data into speech data through a text-to-speech (TTS) function or a speech-to-text (STT) And operates in a TTS or STT scheme for matching the above-described separated voice data most similar to the voice data to the separated caption data.

Here, the STT method is useful for matching voice data and caption data by converting voice data into caption data using the STT function when the digital contents do not have caption information.

As another example, the data matching unit 130 may operate as a time error analysis method of calculating the error between the separated voice data and the caption data statistically and matching the voice data and the caption data.

As another example, although not limited, the data matching unit 130 may operate in a past multimedia content association interworking system that matches voice data and caption data in cooperation with a self-built database already built on the basis of past multimedia contents have.

The secondary data storage unit 131 stores data matching the audio data and the caption data from the data matching unit 130 through a speech engine, intersection matching, speculative matching, or the like as described above.

The data verification unit 140 serves to verify whether the audio data and the caption data matched with each other through audio verification and / or type verification, for example, are not limited. For example, the data verifying unit 140 verifies whether the matched voice data and the caption data are appropriate through pattern comparison with previously established data.

The tertiary data storage unit 141 stores the matched voice data and the caption data that have been verified through the data verification unit 140 or passed the verification as described above.

The data storage unit 150 receives and stores audio data extracted from the above-described primary data storage unit 121, extracted additional data, and / or extracted subtitle data, for example, but not limited thereto, Further, the speech data and the subtitling data are received and stored from the secondary data storage unit 131, and the voice data and / or the type verification is transmitted from the tertiary data storage unit 141 And transmits and stores the completed audio data and subtitle data.

The data classification unit 160 classifies various types of audio data and caption data stored in the data storage unit 150 by morpheme, word, sentence, and / or date, for example, . The data classification unit 160 may classify the audio data and the caption data stored in the data storage unit 150 by age, place, time, and / or genre, for example, but not limited to, have.

The data distribution unit 170 distributes the audio data and the caption data from the data storage unit 150, for example, when there is a request for confirmation of audio data from the inside or outside, for example, but not limited thereto.

The voice data and the caption data constructed by the data distribution unit 170 can be utilized for various services. That is, the established database is distributed for use in various devices and / or services using speech recognition.

The settlement management server 180 allows, for example, but not limited to, charge data settlement for each one-by-one provider when voice data and caption data are generated using pay digital contents.

In this manner, the system 100 for building a voice recognition database using digital contents according to various embodiments of the present invention can be automatically constructed and upgraded. In other words, the database is constantly generated by system construction, and new terms can automatically be included in the database.

In addition, the system 100 for building a voice recognition database using digital contents according to various embodiments of the present invention can construct databases for various ages. That is, by using various digital contents, it is possible to secure databases for various ages such as children who can not be held in the conventional voice database.

In addition, the speech recognition database construction system 100 using the digital contents according to various embodiments of the present invention can construct a database for various situations. That is, the recognition rate is improved by using the broadcast contents recorded in various environments such as the studio and the outside.

In addition, the system 100 for building a voice recognition database using digital contents according to various embodiments of the present invention can construct various databases such as a standard language and a dialect, and can also construct a multilingual language other than Korean.

In addition, various embodiments of the present invention can improve the quality by utilizing the digital contents that are generated many times to improve the voice recognition rate by utilizing the voice database system in various fields. For example, a database for each age can be constructed (currently, there is no separate voice database for infants and elderly people) and a database for each situation (voice database including dates, seasons, and places) A speech database including a special word such as a coined word or a foreign word can be constructed, and a filtering speech database for adult-related speech data can be constructed (a filtering database can be constructed by utilizing the voice of an adult channel).

In addition, various embodiments of the present invention may provide, for example, but not limited to, real-time broadcast information and sequences to broadcasters and the like through the data distribution unit 170. [ That is, the embodiment of the present invention can provide the broadcasters with the most frequently used words in the broadcast (similar to the real-time search term of the portal site).

Further, the various embodiments of the present invention can be utilized for a broadcasting monitoring system (used for regulation of broadcasting deliberation and the like), and can be used for providing TV automatic subtitle broadcasting function and can be used as a basic function of automatic translation.

FIG. 2 is a flowchart illustrating an operation flow of a system 100 for building a voice recognition database using digital contents according to various embodiments of the present invention. Here, the flowchart of FIG. 2 is substantially similar to the configuration of FIG. 1, but is rearranged according to time for easy understanding of the present invention.

2, the speech recognition database construction system 100 roughly includes a digital content acquisition step S1, a digital content source separation step S2, a voice database matching step S3, And a database storage step (S4).

In the digital contents acquisition step S1, for example, a TV live broadcast (for example, a satellite broadcast) 10, a recorded video (e.g., VOD, DVD) file 20, Information 30 and / or existing voice information 40 may be obtained.

In relation to TV live broadcast (10), video / audio / subtitles are separated by the source of satellite broadcast and IP TV broadcast data and used for database matching and generation (in this case, .

Regarding the recorded video (for example, VOD, DVD) file 20, the audio file and the subtitle file are separated from each other in the video file and used for database matching and generation (in this case, the subtitle and audio sync may coincide) can do.

For example, in the case of satellite broadcasting, when the satellite broadcasting is obtained through the broadcasting receiving system 11, the reproduction 12 and the image storage 13 are performed simultaneously, and the subtitle / (14). Then, the audio data separation 15 and the caption data separation 16 are performed respectively.

With respect to the broadcast meta information 30, image related metadata (e.g., performer, place and / or time) can be extracted / separated 31 from the schedule table information and utilized for database matching generation.

In addition, with respect to the existing voice information 40, the voice database matching step S3 can be performed immediately without any separate extraction or separation operation.

In the digital content source separation step S2, for example, in the case of broadcasting including subtitles during live broadcasting, although not limited, in the case of broadcasting including subtitles during live broadcasting, in the case of subtitling broadcasting among recorded video files, The case of non-captioned broadcasting, and the like, and the source separation step can be performed.

First, in the case of live broadcast / closed caption broadcasting, it is possible to separate the audio file and the caption file from the satellite broadcasting source by time (in this case, the corresponding file can be separated in the case of multi language support).

Next, in the case of live broadcasting / subtitle non-broadcasting, audio files can be separated by morphemes (in this case, corresponding files can be separated in the case of multi-language support).

In the case of the recording video file / subtitle broadcasting, it is possible to separately separate the audio file and the subtitle file from the broadcasting source according to the exposure time (in this case, the corresponding file can also be separated in the case of multi language support).

In the case of a recorded video file / subtitle-free broadcasting, it is possible to separately separate the audio file from the broadcasting source (in this case, the corresponding file can be separated in the case of multi-language support).

In the voice database matching step S3, for example, a voice / caption multiplex intersection analysis method 51 (see FIG. 3), a voice caption matching method 52 (see FIG. 4) A database matching analysis method 53 and / or a TTS / SST matching analysis method 54 may be used.

In the case of the audio / subtitle multiplex intersection analysis method (see FIG. 3) 51, the audio database is matched / generated as follows.

 First, N pieces of subtitles and sets delayed by N seconds are stored at the input voice time point. For example, N seconds can be set to about 1 to 3 seconds by big data analysis, and N is a morpheme included in about 1 second. .

Subsequently, a source matching a maximum frequency among the databases constituting the set of audio and subtitles is distinguished.

Finally, the sets containing the same words in the subtitles are checked by intersection analysis (for example, all the sources of subtitles at 9 o'clock are analyzed to distinguish similar patterns of speech). Here, unmatched sources are also databaseized and used in the next set to increase speed / accuracy. In addition, if the type of a voice wave (wav) recognized as the same word is analyzed and deviates from the error range, the corresponding database is excluded and managed separately.

In the case of the audio subtitle matching (see FIG. 4) scheme (speculative analysis) 52, the audio database is matched / generated as follows.

First, the broadcast related meta information is obtained. Then, the subtitle file is compared with the voice file classified by the morpheme. Finally, the voice database is matched / created by separately storing the voice file showing the same pattern.

In the case of the existing speech database matching analysis method 53, a speech database is generated as follows.

First, a voice file is generated which is divided into morphemes. Then, the audio database is matched / generated by confirming the caption of the file by analyzing the voice file wavelength of the existing voice database.

Next, TTS / SST matching  In the case of the analysis method (54), the separated subtitle data is converted into the TTS function or STT  Function and converts the above-described separated voice data, which is the most similar to the converted voice data, into the separated caption data By matching , Voice database matching / Create.

here, STT  If the digital content does not have subtitle information, STT  Function is used to convert audio data into caption data, which is useful for matching / generating audio data and caption data.

Finally, the voice database storage step ( S4 ), The audio data and the caption data are reproduced using any one of the four methods as described above Matched  And then determines whether the matched voice data and caption data are appropriate according to the internal verification algorithm of the database (62). Then, the data classification (63) and the final voice data And stored (64) as a base.

FIG. 3 is a diagram illustrating an example of voice / caption multiplex intersection analysis during operation of the speech recognition database construction system 100 using digital contents according to various embodiments of the present invention.

As shown in FIG. 3, the voice is blanked without voice recognition, the voice data is divided first, the divided data is repeatedly compared, words of similar patterns are stored, and data of different patterns are deleted. This is a way to increase accuracy.

FIG. 4 is a diagram illustrating an example of meta-analysis during operation of the speech recognition database building system 100 using digital contents according to various embodiments of the present invention.

As shown in FIG. 4, the extracted voice and broadcast programming information (EPG) are utilized to provide more data for predicting the preceding and succeeding words, thereby improving the voice search accuracy. For example, in August, when the title of the movie "Haeundae" is used most frequently in broadcasting, "Haeundae" can be exposed as a priority in voice search.

The present invention is not limited to the above-described embodiments, and various modifications and changes may be made without departing from the scope of the present invention. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

100; Speech Recognition Database Construction System
110; Data acquisition unit 120; The data extracting unit
121; A primary data storage unit 130; The data-
131; A secondary data storage unit 140; The data verification unit
141; A tertiary data storage unit 150; The data storage unit
160; A data classifier 170; The data distribution unit
180; Settlement management server

Claims (10)

A data acquisition unit for acquiring digital contents including audio information;
A data extracting unit for extracting audio data from the digital contents acquired by the data acquiring unit;
A data matching unit for matching the caption data corresponding to the audio data extracted from the data extracting unit;
A data storage unit for storing audio data and caption data matched by the data matching unit; And
And a data distribution unit for distributing the audio data and the caption data from the data storage unit when there is a request for confirming the audio data,
If the digital content includes caption information, the data extracting unit also extracts caption data from the digital content,
Wherein the data matching unit compares the audio data and the caption data used for a specific period of time and operates in an intersection scheme for matching the most overlapping audio data with caption data. system. The method according to claim 1,
Further comprising a data classifier for classifying the voice data stored in the data storage unit. 3. The method of claim 2,
Wherein the data classification unit classifies the stored voice data by morpheme, word, sentence, date, age, place, time, or genre. The method according to claim 1,
Wherein the digital content is a recorded content including a real-time broadcast content including a television broadcast or a radio broadcast, a recorded broadcast, a streaming broadcast, or an audio book, or broadcast programming information content. system. delete delete delete delete delete delete

Images