KR101590078B1 - Apparatus and method for voice archiving - Google Patents

Abstract

In a voice archiving process, a voice is recorded. The recorded voice data is converted into text. At least one keyword is extracted from the converted text. A voice file which includes the voce data from a record start point to a record end point, is matched to a keyword. It is stored. A voice file player for reproducing the stored voice file is provided. Provided is a graphic user interface which outputs a keyword as a marker in at least one position on the voice file when the voice file player is executed.

Description

[0001] APPARATUS AND METHOD FOR VOICE ARCHIVING [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice archiving apparatus and method for recording and recording voice of a user.

In general, digital conversation through online / mobile messenger and text is easy to store data, but it is virtually impossible for a speaker to memorize conversation contents in actual conversation in offline. Accordingly, methods of directly writing or recording important contents to be stored through a medium such as a portable terminal are used. However, there is a problem that it is difficult to concentrate on the conversation due to the action for storing the conversation contents, except for the recording of the voice conversation, and accordingly, important conversation contents are often missed. In addition, the content that has been handled trivially at the time of conversation may become important later, and a method for efficiently storing and managing conversation contents is needed.

Accordingly, conventionally, a method of recording a voice conversation through a voice recorder or the like is used. However, in the case of the conventional voice recording method, there is a limit in that direct access to a specific position for reproducing desired information immediately is not possible. In addition, only basic information such as title, date, and number is displayed in relation to the recorded voice file, making it impossible to efficiently search for desired information. In addition, the user interface (UI) is not intuitive as compared with the functions provided by a voice recording device such as a voice recorder. That is, by using a limited number of buttons on the device for controlling the user interface, many functions having no relevance to one button are often matched, so that it is difficult to manipulate the functions desired by the user.

In this regard, Korean Patent Laid-Open Publication No. 10-2013-0134156 (an electronic apparatus and method for automatically recording a voice in a call standby state) and a call standby status with the first electronic device, The process of confirming that it has begun; Confirming the call waiting state and starting voice recording; And transmitting the recorded voice to the second electronic device of either the first electronic device or the at least one second electronic device according to a call state, Electronic devices and methods.

In order to solve the problems of the related art described above, an embodiment of the present invention records a voice and records it together with related information, but also outputs related information when a voice file is provided so that a user can intuitively And to provide a voice archiving device and method therefor.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided a voice archiving apparatus comprising: a recording processor for recording a voice; A voice-to-text conversion unit for converting the voice data recorded through the recording processing unit into text; A keyword extracting unit for extracting at least one keyword from the speech data converted into the text; A storage unit for storing the recorded voice data as an audio file including a recording start time to a recording end time; And a voice file providing unit for providing a voice file player for reproducing a voice file selected from the voice files stored in the storage unit, wherein the voice file providing unit includes at least one position And outputting the keyword as a marker.

According to another aspect of the present invention, there is provided a voice archiving method using a voice archiving apparatus, comprising: recording voice; Converting the recorded voice data into text; Extracting at least one keyword from the text; Storing the speech data as a speech file including the speech data from the start of recording to the end of recording; And providing a voice file player for playing back the stored voice file, wherein, when the voice file player is executed, a graphical user interface is provided for outputting the keyword as a marker at at least one location on the voice file .

According to still another aspect of the present invention, there is provided a recording medium including: recording a voice; Converting the recorded voice data into text; Extracting at least one keyword from the text; Storing the speech data as a speech file including the speech data from the start of recording to the end of recording; And a voice file player for reproducing the stored voice file, wherein the voice file player comprises a graphical user interface for outputting the keyword as a marker at at least one position on the voice file when the voice file player is executed, A program for allowing the computer to provide the information is recorded.

According to any one of the above-mentioned objects of the present invention, it is possible to easily record conversations between users in daily life, and to provide refined information by intelligently classifying recorded voice conversations.

In addition, according to any one of the tasks of the present invention, a user can automatically start voice recording using a simple operation or a reserved word through a voice archiving device such as a smart device, automatically recognize the end of a conversation, It is convenient to record.

Further, according to any one of the tasks of the present invention, there is an effect that the user can intuitively recognize the conversation position including specific information in the voice file by marking the keyword for each section of the voice file.

In addition, according to any one of the tasks of the present invention, the user can separately check the important related information when recording a voice file, and can effectively utilize it when searching for a voice file.

Further, according to any one of the tasks of the present invention, it is possible to systematically manage the conversation information by matching the recorded voice conversation with the information related to the voice file and archiving it in a database.

1 is a block diagram showing a configuration of a voice archiving apparatus according to an embodiment of the present invention.
2 is a diagram for explaining a structure of a voice file database according to an embodiment of the present invention.
3 is a diagram for explaining a recording end processing method according to an embodiment of the present invention.
4 is a diagram for explaining a conversation space processing method according to an embodiment of the present invention.
5 is a diagram illustrating an example of a graphical user interface of a voice file player according to an embodiment of the present invention.
6 is a flowchart for explaining a voice archiving method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 is a block diagram showing a configuration of a voice archiving apparatus according to an embodiment of the present invention.

1, the voice archiving apparatus 100 according to an exemplary embodiment of the present invention includes a recording processing unit 110, a voice-to-text conversion unit 120, a keyword extraction unit 130, a storage unit 140, And an audio file providing unit 150.

1 may be implemented in hardware such as software or an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform predetermined roles can do. However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors. Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

The recording processing unit 110 records and processes the user's voice.

The recording processing unit 110 may record voice (e.g., " at least one user-to-user conversation ") in units of intervals of a predetermined length (i.e., time), but may record a continuous section. At this time, the recording processing unit 110 continuously transmits the audio data of the section unit to the voice-to-text conversion unit 120 and the storage unit 140. For reference, audio data for each section from the recording start time to the recording end time are combined and stored as one audio file through the storage unit 140. [

The recording processing unit 110 can automatically process the start and end of voice recording.

Specifically, the recording processing unit 110 recognizes the occurrence of a preset recording command event and can automatically start voice recording. For example, the recording processing unit 110 may acquire voice or operation information sensed through at least one type of sensor (not shown) included in the voice archiving apparatus 110, If it is recognized, it is determined that a recording command event has occurred and the voice recording can be automatically started.

The record processing unit 110 detects the presence or absence of the text data according to the speech recognition in the section based on the text conversion result through the speech-to-text conversion unit 120 to be described below, Can be automatically terminated.

For example, FIG. 3 illustrates a recording end processing method according to an embodiment of the present invention.

The recording processing unit 110 receives feedback on the speech recognition result (i.e., text conversion result) of the speech data per section through the speech-data conversion unit 120. As shown in FIG. 3, the recording processing unit 110 can automatically end the voice recording when there is no voice recognition result in at least two consecutive intervals (that is, when there is no text data) have. That is, as shown in FIG. 3, if the result of speech recognition from the first section to the (n + 1) th section is not included in the n + 1th section after the text conversion result value is not included in the nth section , It is determined that the voice conversation has ended and the automatic recording end process can be performed.

In addition, the recording processing unit 110 may pre-process an audio file in which a conversation gap exists and store the audio file as a single audio file through the storage unit 140. [

For example, FIG. 4 is a diagram for explaining a conversation space processing method according to an embodiment of the present invention.

As shown in Fig. 4, when the recording start processing for the new voice file is executed within a predetermined time after the end of the voice recording, the first voice file whose recording has been completed and the second voice file May be determined as one voice file, combined and stored through the storage unit 140. [ At this time, the recording processing unit 110 may perform at least one blank section including no text conversion result at the ending point of each of the first and second audio files by blanking the blank space. Accordingly, even when the voice recording is temporarily terminated due to the silence of the user or the like, it is possible to effectively manage the data by creating one voice file by combining the related voice files. In addition, quality of voice archiving can be improved by eliminating unnecessary data (i.e., blank time, etc.) when combining voice files.

Meanwhile, the recording processor 110 may provide a user interface (UI) that enables a user to input at least one kind of related information about a voice file when storing the voice file. At this time, the related information may include at least one of an audio file name, speaker information, tag information, recording place information, recording time information, and conversation contents summary information. The related information input through the user is stored in a matching manner with the corresponding voice file through the storage unit 140. Through the voice file player provided by the voice file providing unit 150 to be described later, And can be matched and displayed with the voice file. Accordingly, the user can specify the title of the recorded voice file, and related information such as the speaker participating in the conversation, the tag for the voice file, the conversation place, the date and time, and the conversation contents can be managed and displayed for each voice file, There is an effect that it becomes very easy to select a voice file to be heard by the user in the future.

1, the voice-to-text conversion unit 120 converts the recorded voice data through the recording processing unit 110 into text.

Here, the speech-to-text conversion unit 120 receives speech data in units of a predetermined length from the speech processing unit 110, and converts the received speech data into a predetermined speech-to-text (STT) API, Application Programming Interface).

For example, the voice-to-text conversion unit 120 may transmit the voice data to a public / non-public voice recognition server that is linked with the voice recognition server through the STT API, And receives data. The voice-to-text conversion unit 120 transmits the received text conversion result data to the keyword extraction unit 130 and the recording processing unit 110. For reference, the voice-to-text conversion unit 120 may transmit only the text conversion result of the voice data section to the recording processor 110.

The keyword extracting unit 130 extracts at least one keyword from the text change result data received through the voice-to-text conversion unit 120 (i.e., the voice data converted into text).

Specifically, the keyword extracting unit 130 morphologizes the converted text from the speech data to obtain at least one word, and determines keywords among the acquired words according to predetermined conditions. At this time, the predetermined condition may be set as the appearance frequency of the corresponding word in each section of the voice file, and the keyword having the highest appearance frequency or a plurality of words from the highest frequency to the predetermined ranking may be determined as the keyword.

The storage unit 140 stores the voice data recorded through the voice recording unit 110 as a voice file including the voice data from the start of recording to the end of voice recording.

Specifically, the storage unit 140 stores information such as a voice file ID, a voice file name, a keyword ID, a keyword, a keyword appearance period information, and a keyword appearance frequency information in each section with respect to each voice file, .

For example, FIG. 2 illustrates a structure of a voice file database according to an embodiment of the present invention.

As shown in Fig. 2, the database configured with a table for expressing the relationship between the voice file and the keyword includes a unique ID (archive_id) for identifying the voice file, a keyword ID for identifying the keyword included in the voice file (keyword_id), a keyword appearance interval (archive_fragment) for identifying the number of pieces (i.e., sections) of the keyword in the corresponding voice file, and a keyword occurrence frequency in the section for identifying the number of times the keyword appeared in the corresponding section (keyword_weight) item is included.

As shown in FIG. 2, the storage unit 140 may store a matching ID by giving a unique ID to each word beforehand. For reference, a plurality of words set in advance may mean words included in a dictionary. In addition, the storage unit 140 may store an automatic input for each voice file ID or an archive file name (archive_name) input from the user. Accordingly, the voice file providing unit 150, which will be described later, can detect and output the related information for each voice file by referring to the database stored through the storage unit 140. [

The storage unit 140 may be a nonvolatile memory such as a cache, a read only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) A memory device, a volatile memory device such as a RAM (Random Access Memory), or a storage medium such as a hard disk drive (HDD), a CD-ROM, or the like.

Referring back to FIG. 1, the voice file providing unit 150 provides a voice file player for playing a voice file selected by the user among voice files stored in the storage unit 140.

At this time, when the voice file player 150 executes the voice file, the voice file providing unit 150 transmits the at least one keyword extracted through the keyword extracting unit 130 to at least one position on the voice file, as a marker, Interface (GUI, Graphical User Interface) is provided.

Specifically, when executing the voice file player, the voice file providing unit 150 extracts voice file names and keywords for each voice file from the storage unit 140, and outputs the voice file list including the extracted voice file names and keywords to the GUI Lt; / RTI > At this time, the voice file providing unit 150 can additionally display at least one piece of related information stored in the voice file matching the voice file through the GUI.

When the user selects one of the voice files through the GUI, the voice file providing unit 150 extracts keyword and keyword appearance period information corresponding to the voice file according to the input selection information from the storage unit 140 , And displays the keyword on a section of a predetermined length of the voice file through the GUI based on the extracted result.

For example, FIG. 5 illustrates an example of a graphical user interface of a voice file player according to an exemplary embodiment of the present invention.

5, on the GUI provided through the audio file providing unit 150, starting point information P10 and at least one keyword (for example, < RTI ID = 0.0 & (P20) may be displayed as a marker. Further, on the GUI, a boundary line P30 indicating a start point of each section can be further displayed so that the user can intuitively identify each piece (i.e., section) of the audio file, and a playback bar (bar) It is also possible to display different colors for each section (P40).

For reference, the voice file providing unit 150 refers to the keyword-weight item in the section stored in the storage unit 140 to sort and display the keywords in descending order of frequency from the most frequently occurring keyword It is possible.

The voice archiving apparatus 100 according to an embodiment of the present invention described above with reference to FIG. 1 through FIG. 5 has a function of adding a computer support function such as an Internet communication and an information search to a mobile terminal such as a smart phone As an intelligent terminal, it can be implemented as a smart device capable of installing an application desired by the user. That is, the smart device to which the present invention is applied is not limited in functions such as a smart phone, a tablet PC, and a smart pad, and is a device capable of changing and expanding functions by installing a plurality of applications (applications). Such a smart device may include a display unit configured to display a screen and configured with a touch screen or the like.

The smart device 100 executes the operations of the recording processing unit 110, the voice-to-text conversion unit 120, the keyword extraction unit 130, the storage unit 140, and the voice file providing unit 150 And stores the data stored in the storage unit 140 in the memory. And, the smart device 100 includes a processor for executing a program stored in the memory. At this time, as the processor executes the program, the keyword start and end operations of the voice recording, the text conversion operation of the voice data, the keyword extraction operation from the converted text, and the keyword All sequential or parallel processing for voice archiving such as display and voice file reproduction operation is performed. For reference, a program executed in the smart device 100 may be mounted on the smart device 100 in the form of at least one application.

Hereinafter, a voice archiving method according to an embodiment of the present invention will be described in detail with reference to FIG.

First, voice recording for constituting one voice file is started (S610).

At this time, in the step of starting the voice recording, after the occurrence of the preset recording command event is recognized, the voice recording can be automatically started according to the recognition.

Next, the recorded voice data is converted into text (S620).

Specifically, the speech data of the interval unit having the predetermined length recorded through the step S610 is received, and the received speech data is converted into text through a predetermined STT API.

Then, at least one keyword is extracted from the speech data converted into the text according to predetermined conditions (S630).

Specifically, morphological analysis processing is performed on the text converted from the speech data to acquire at least one word, and a keyword among the acquired words is determined according to a predetermined condition. At this time, the predetermined condition may be set as the occurrence frequency of the corresponding word for each section of the voice file.

Then, an audio file including audio data from the recording start point to the end point is matched with the corresponding keyword and stored (S640).

At this time, the voice file ID, the voice file name, the keyword ID, the keyword, the keyword appearance period information, and the keyword occurrence frequency information in the section can be stored in the voice file. In step S640, a user interface may be provided to allow the user to input at least one kind of related information for the voice file. For reference, the related information may include at least one of an audio file name, speaker information, tag information, recording place information, recording time information, and conversation contents summary information. Accordingly, the related information can be matched and stored in the voice file, and at the step of providing the voice file player to be described later, at least one related information item can be displayed by matching with the voice file through the graphical user interface (GUI) .

In the step S640, the presence or absence of the text data according to the speech recognition in the segment is detected based on the result of converting the speech data into text, and if there is no speech recognition result in at least two segments If the voice recording can be processed automatically. When the start processing of the new voice file is executed within a predetermined time after the voice recording is automatically terminated, the voice file whose voice recording has been completed can be combined with the new voice file and stored as one voice file. For reference, it is also possible to remove whitespace between a voice file and a new voice file in which voice recording has been completed during the combining process.

Next, an audio file player for reproducing a voice file selected by a user from among the stored voice files is provided. In operation S650, the keyword is output as a marker to a position of each voice file on a section of the voice file.

Specifically, when executing the voice file player, a voice file name and a keyword stored for each voice file are extracted, a voice file list including the extracted voice file name and the keyword is displayed through a graphical user interface (GUI) Extracts keyword and keyword appearance period information corresponding to the voice file according to the selection information, and displays a keyword for each interval of a predetermined length of the voice file according to the selection information based on the extracted result do. At this time, through the graphical user interface, the start point and the keyword can be displayed as markers in the voice file for each section.

The above-described voice archiving method according to an embodiment of the present invention can also be implemented in the form of a recording medium including instructions executable by a computer such as a program module executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: Voice archiving device
110:
120: voice-to-text conversion unit
130:
140:
150: voice file provisioning

Claims (18)

In the voice archiving device,
A recording processor for recording a voice;
A voice-to-text conversion unit for converting the voice data recorded through the recording processing unit into text;
A keyword extracting unit for extracting at least one keyword from the speech data converted into the text;
A storage unit for storing the recorded voice data as a voice file including a time from a start of recording to an end of voice recording, the voice file matching the voice file and storing the voice file; And
And a voice file player for providing a voice file player for playing back the selected voice file among the voice files stored in the storage unit,
The audio file providing unit,
And a graphical user interface for outputting the keyword as a marker at at least one position on the voice file when the voice file player is executed,
The recording processing unit,
When the preset word and operation set in advance are recognized based on the voice or operation information sensed by the one or more types of sensors included in the voice archiving device, it is determined that a predetermined recording command event has occurred and the voice recording is automatically started,
The speech data is transmitted to the speech-to-text conversion unit in units of intervals of a predetermined length, the presence or absence of the text data according to the speech recognition in the interval is detected based on the result of text conversion through the speech- Wherein the speech recognition apparatus automatically terminates the voice recording when there is no voice recognition result in at least two of the sections. The method according to claim 1,
Wherein the voice-to-
Receiving audio data of a section unit having a predetermined length from the recording processing unit,
And converting the received voice data into text through a predetermined STT (Application to Speech to Text) application programming interface. 3. The method of claim 2,
The audio file providing unit,
And displays the starting point and the keyword by a marker in each of the sections in the voice file through the graphical user interface. The method according to claim 1,
The keyword extracting unit extracts,
A morphological analysis processing unit for morphing the converted text from the speech data to obtain at least one word and determining a keyword among the acquired words according to a predetermined condition,
Preferably,
And the frequency of appearance of words is set for each interval of a predetermined length unit of the voice file. The method according to claim 1,
Wherein,
Wherein the database stores a voice file ID, a voice file name, a keyword ID, a keyword, a keyword appearance period information, and a keyword occurrence frequency information in each section of the voice file. 6. The method of claim 5,
The audio file providing unit,
Extracting a voice file name and a keyword for each voice file from the storage unit when the voice file player is executed, displaying a voice file list including the extracted voice file name and the keyword through the graphical user interface,
Extracting a keyword and keyword appearance period information corresponding to a voice file corresponding to the selection information from the storage unit, and outputting, based on the extracted result, a voice file corresponding to the selection information And said keyword is displayed for each section of a predetermined length of said keyword. The method according to claim 1,
The recording processing unit,
A user interface for allowing a user to input at least one kind of related information about the voice file when storing the voice file,
The above-
An audio file name, speaker information, tag information, recording place information, recording time information, and conversation contents summary information,
The voice file is matched and stored through the storage unit,
Wherein at least one item is displayed and matched with the voice file through the graphical user interface. delete delete The method according to claim 1,
The recording processing unit,
If a recording start process for a new voice file is performed within a predetermined time after the end of the voice recording, the voice file whose voice recording has been finished is combined with the new voice file to be stored as one voice file in the storage unit The voice archiving device storing the voice archiving device. 11. The method of claim 10,
The recording processing unit,
Wherein the voice archiving apparatus removes spaces between the voice file whose voice recording is terminated and the new voice file at the time of combining. A voice archiving method using a voice archiving device,
Recording a voice;
Converting the recorded voice data into text;
Extracting at least one keyword from the text;
Storing the speech data as a speech file including the speech data from the start of recording to the end of recording; And
And providing a voice file player for playing back the stored voice file,
And a graphical user interface for outputting the keyword as a marker at at least one position on the voice file when the voice file player is executed,
The step of recording the voice includes:
Determining that a predetermined recording command event is generated when a preset word and an operation set in advance are recognized based on voice or operation information sensed through one or more types of sensors included in the voice archiving device; And
And automatically starting the voice recording according to the determination,
After the storing step,
Detecting presence or absence of text data according to speech recognition in the section based on a result of converting the speech data continuously recorded in units of intervals of a predetermined length into text; And
And automatically terminating the voice recording when at least two consecutive voice recognition results in the section do not exist. 13. The method of claim 12,
Wherein the step of converting the speech data into text comprises:
Receiving voice data of a predetermined interval in a unit of interval through voice recording; And
And converting the received speech data into text through a predetermined Speech to Text (STT) application programming interface. 13. The method of claim 12,
Wherein the storing step comprises:
Wherein the voice file ID, voice file name, keyword ID, keyword, keyword appearance period information, and keyword occurrence frequency information are matched and stored in the voice file. 15. The method of claim 14,
Wherein providing the voice file player comprises:
Extracting the stored voice file name and keyword for each voice file when the voice file player is executed;
Displaying a list of voice files including the extracted voice file names and keywords through the graphical user interface;
Receiving selection information for any one of the audio files;
Extracting the stored keyword and keyword appearance period information corresponding to the voice file according to the selection information; And
And displaying the keyword on a section of a predetermined length of an audio file according to the selection information based on the extracted result. delete delete A recording medium on which a program is recorded enabling a computer to perform the method of any one of claims 12 to 15.

Images