KR102208954B1 - Coputer device for providing dialogues services - Google Patents

Abstract

In an embodiment of the present disclosure for solving the above-described problem, as a computer program stored in a computer-readable storage medium executable by one or more processors, the computer program causes the one or more processors to perform the following operations. The operations include: receiving one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user, each of the one or more voice data using a voice recognition model. Through the conversion to one or more voice recognition results, the operation of generating a conversation record and generating a user interface including the conversation record and providing the user interface to the one or more user terminals may be included.

Description

A computing device that provides conversation log service {COPUTER DEVICE FOR PROVIDING DIALOGUES SERVICES}

The present disclosure relates to a computing device that provides a chat log service, and more particularly, to a computing device that provides a chat log service based on a voice file including chat contents between a plurality of users.

Today, office workers frequently hold meetings with employees inside and outside the company. In the course of meetings, making minutes for meeting content is very helpful in terms of work efficiency of the organization and individuals. However, since a separate stenographer is required to write minutes while the meeting is in progress, additional costs may be incurred, and the quality of the minutes may vary depending on the capabilities of the stenographer.

On the other hand, in recent years, the market demand for artificial intelligence is increasing according to issues such as AlphaGo, and artificial intelligence is evolving at a rapid pace as the amount of information that can be processed increases with the development of computer technology. In addition, as artificial intelligence evolves at a rapid pace, the accuracy of speech recognition using artificial neural networks is increasing.

Accordingly, in Korean Patent Laid-Open No. 10-2008-0037402 (2008.04.30), an algorithm for separating a plurality of speakers based on a frequency band for a voice file input through a single channel is performed by separating speakers through text, A system for automatically taking minutes of meetings is being launched.

However, in the prior art, in the process of separating a speaker through a frequency band from a voice file input through a single channel, accurate speaker separation may not be achieved due to overlapping speech sounds or noise generated by an external environment. That is, in the prior art, a text file corresponding to a plurality of speakers may be inaccurate, and there is a concern that incorrect information may be provided to a plurality of users using the system.

The present disclosure is conceived in response to the above-described background technology, and is to provide a computing device that provides a conversation record service based on a voice file including conversation contents between a plurality of users.

In one embodiment of the present disclosure for solving the above-described problem, a computer program stored in a computer-readable medium executable by one or more processors is disclosed. The computer program causes the one or more processors to perform the following operations, the operations being: one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user- Receiving an operation, an operation of converting each of the one or more speech data into one or more speech recognition results through a speech recognition model to generate a conversation record, and an operation of generating a user interface including the conversation record and providing it to the one or more user terminals It may include.

Alternatively, the operation of normalizing the size of a signal included in each of the one or more voice data based on a comparison of the size of a signal included in each of the one or more voice data with a size of a predetermined reference signal, and the one or more voice The operation of removing at least one specific frequency by analyzing frequencies included in each of the data may be further included.

Alternatively, an operation of identifying one or more speech sections based on one or more silent sections included in the voice data, and an action of separating a speaker based on the size of the spoken speech in one or more speech sections included in the voice data It may further include.

Alternatively, the operation of separating a speaker based on a speech size in one or more speech sections included in each of the one or more speech data includes the size of a speech speech having a predetermined size or more among one or more speeches included in the speech data. An operation of determining an utterance as a specific speaker corresponding to the voice data, an operation of removing voices other than the determined specific speaker's voice from the voice data, and identification information of the user terminal that transmitted the voice data to the specific speaker Matching with and storing may be included.

Alternatively, the operation of generating a conversation record by converting each of the one or more speech data into one or more speech recognition results through a speech recognition model is an operation of dividing the speech data into one or more speech data sub-data based on a speech section. , An operation of identifying a start point and an end point of each of the one or more voice data sub-data, and one or more corresponding to each of the one or more voice data sub-data by using each of the one or more voice data sub-data as an input of the voice recognition model An operation of outputting a speech recognition sub-result and a start time and an end time identified in each of the voice data sub-data that are the basis of the one or more voice recognition sub-results on the conversation log of each of the one or more voice recognition sub-results. It may include an operation of determining the order.

Alternatively, the one or more voice data includes first voice data and second voice data received from each user terminal, and converts each of the one or more voice data into one or more voice recognition results through a voice recognition model. The operation of generating a conversation record may include dividing the first voice data into one or more first voice data sub-data based on a speech section, and dividing the second voice data into one or more second voice data sub-data based on the speech section Dividing the first voice data sub-data and the second voice data sub-data into a start time and an end time of each of the first voice data sub-data and the second voice data sub-data, the voice recognition of the first voice data sub-data and the second voice data sub-data Outputting one or more first voice recognition sub-results and one or more second voice-recognition sub-results corresponding to each sub-data as an input of a model, the one or more first voice recognition sub-results, and the one or more second The at least one first speech recognition sub result and the at least one second speech recognition sub result on the conversation log based on the start time and end time identified in each of the voice data sub data that are the basis of the voice recognition sub result. It may include an operation of determining an order and an operation of generating a conversation record based on the determined order and identification information of each of the one or more user terminals.

Alternatively, one or more first voice recognition sub-results and second voice-recognition sub-results corresponding to each sub-data by using the first voice data sub-data and the second voice data sub-data as inputs of the voice recognition model The outputting of the first voice data sub-data and the second voice data sub-data is an operation of identifying an overlapping section based on a start time and an end time of each of the first voice data sub-data, and when the overlapping section is identified, the overlapping section Determining whether to divide at least one of the corresponding first voice recognition sub-result and the second voice recognition sub-result, and if the division is determined, dividing the voice recognition sub-result corresponding to the overlapping section can do.

Alternatively, when the overlapping section is identified, the operation of determining whether to divide at least one of the first voice recognition sub-result and the second voice recognition sub-result corresponding to the overlapping section corresponds to the overlapping section. It may include an operation of determining whether to divide the speech recognition result through a statistical language model with respect to the speech recognition result.

Alternatively, determining one or more keywords for the conversation log based on at least one of a result of semantic analysis of the one or more speech recognition results, the number of repetitions for each word included in the one or more speech recognition results, and a predetermined conversation topic. And identifying a location of each of the one or more keywords in the one or more speech recognition results.

Alternatively, the user interface may include a user's profile, a list of one or more other users registered by the user, a registration request button for the user to register an additional user, and a user indicating whether to access each of the one or more other users Including a list display screen, and information on the names of each of the one or more conversations the user participated in, a conversation search button for searching for a specific conversation among the one or more conversations, and a chat room opening button for opening a chat room with one or more other users It may include a conversation list display screen that displays.

Alternatively, the processor, when receiving a selection input for the chat room opening button from the user terminal, sends a control signal to the user terminal to provide a user selection input screen for including the one or more other users in the chat room. It may further include an operation of determining to transmit.

Alternatively, when receiving a selection input for a specific chat room from a list of one or more chat rooms in which the user participates from a user terminal, the processor provides a chat log display screen including a chat log and one or more keywords corresponding to the specific chat room. It may further include an operation of determining to transmit a control signal to be performed to the user terminal.

Alternatively, the processor, when receiving a selection input for a specific keyword among the one or more keywords from the user terminal, searches for a speech recognition result corresponding to the keyword, and based on the generation of the searched speech recognition result. The method may further include determining to transmit a control signal to the user terminal to provide a keyword playback screen for reproducing the speech section of the voice data.

Alternatively, when a new chat room is opened, the user interface includes a recording start button for receiving voice data from each of one or more users included in the chat room, and one or more buttons received from each of one or more users included in the chat room. It may include a conversation log display screen that displays one or more voice recognition results generated based on voice data.

Alternatively, the user interface includes a text editing screen for receiving a request for modification of one or more speech recognition results included in the conversation log from a user terminal, and the processor displays the text editing screen from the user terminal. When receiving a request to modify one or more speech recognition results included in the conversation log through, the operation of correcting the speech recognition model so that the speech recognition model outputs a speech recognition result corresponding to the modification request. have.

In another embodiment of the present disclosure, a method of providing a conversation log service is disclosed. The method includes receiving one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user, each of the one or more voice data through a speech recognition model. It may include converting into a speech recognition result and generating a conversation log, and generating a user interface including the conversation log and providing the user interface to the one or more user terminals.

In another embodiment of the present disclosure, as a server providing a conversation log service, a server processor including one or more cores, a server memory storing program codes executable in the processor, and a server network unit for transmitting and receiving data with one or more user terminals. And the processor receives one or more voice data from each of the one or more user terminals, the voice data including one or more utterances for each user, and converts each of the one or more voice data into a voice recognition model Through the conversion into one or more voice recognition results, a conversation record may be generated, and a user interface including the conversation record may be generated and provided to the one or more user terminals.

The present disclosure may provide a computing device that provides a conversation record service based on a voice file including conversation contents between a plurality of users.

Various aspects are now described with reference to the drawings, where like reference numbers are used collectively to refer to like elements. In the examples that follow, for illustrative purposes, a number of specific details are presented to provide a thorough understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details.
1 is a conceptual diagram illustrating an overall system for providing a conversation log service according to an embodiment of the present disclosure.
FIG. 2 is a block diagram of a computing device providing a conversation log service according to an embodiment of the present disclosure.
3 is a flow chart for providing a chat log service related to an embodiment of the present disclosure.
4 is a diagram illustrating an exemplary user interface including a user list display screen and a conversation list display screen according to an embodiment of the present disclosure.
5 is a diagram illustrating a process of opening a chat room and a user interface for a chat room including a plurality of users according to an embodiment of the present disclosure.
6 is a diagram illustrating an exemplary user interface for providing a conversation log display screen in response to a selection input of a user terminal for a specific conversation among one or more conversation lists performed by a user according to an embodiment of the present disclosure.
7 is a diagram illustrating an exemplary user interface in which a screen for reproducing a specific time point of voice data is displayed in response to a user's selection input for a voice recognition result and a keyword according to an embodiment of the present disclosure.
8 shows a means for providing a chat log service related to an embodiment of the present disclosure.
9 illustrates a module for providing a chat log service related to an embodiment of the present disclosure.
10 is a diagram illustrating logic for providing a chat log service related to an embodiment of the present disclosure.
11 illustrates a circuit for providing a conversation log service related to an embodiment of the present disclosure.
12 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is clear that these embodiments may be implemented without this specific description.

The terms "component", "module", "system" and the like as used herein refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component may be, but is not limited to, a process executed on a processor, a processor, an object, an execution thread, a program, and/or a computer. For example, both an application running on a computing device and a computing device may be components. One or more components may reside within a processor and/or thread of execution. A component can be localized on a single computer. A component can be distributed between two or more computers. In addition, these components can execute from a variety of computer readable media having various data structures stored therein. Components can be, for example, via a signal with one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, and/or a signal through another system and a network such as the Internet. Depending on the data being transmitted), it may communicate via local and/or remote processes.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise or is not clear from the context, "X employs A or B" is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or, when X uses both A and B, “X uses A or B” can be applied to either of these cases. In addition, the term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

In addition, the terms "comprising" and/or "comprising" are to be understood as meaning that the corresponding features and/or components are present. However, it is to be understood that the terms "comprising" and/or "comprising" do not exclude the presence or addition of one or more other features, elements, and/or groups thereof. Further, unless otherwise specified or when the context is not clear as indicating a singular form, the singular in the specification and claims should be interpreted as meaning "one or more" in general.

Those of skill in the art would further describe the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein, including electronic hardware, computer software, or a combination of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the specific application and design restrictions imposed on the overall system. Skilled technicians can implement the described functionality in various ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

1 is a conceptual diagram illustrating an overall system for providing a chat log service related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 and one or more user terminals 10 may transmit and receive data through wireless and/or wired interconnection. The computing device of the present disclosure may include all kinds of computing devices capable of calculating data in electronic form, and for example, general computing devices such as personal computers and server computers, and mobile terminals (smartphones, It may include a computing device having limited computing power, such as a tablet.

According to an embodiment of the present disclosure, the computing device 100 may provide a conversation log service based on voice data including conversation contents between a plurality of users. Specifically, the computing device 100 receives one or more voice data from each of the one or more user terminals 10, and converts each of the received one or more voice data into text through a voice recognition model, You can create a conversation log for conversation content. At this time, the one or more user terminals 10 may include a personal computer (PC), a notebook (note book), a mobile terminal, a smart phone, a tablet PC, and the like. /Can include all types of terminals that can access the wireless network.

In addition, the computing device 100 may provide a texted conversation log based on each of the one or more voice data received from each of the one or more user terminals 10. In this case, the texted conversation log is a textualization of voice data of a plurality of users who participated in the conversation, and may be arranged by dividing by time and speaker by converting voice data on conversations in which the plurality of users participated. Accordingly, when a conversation (or meeting) between multiple users is conducted, a plurality of users do not need to write a conversation log through a separate stenographer or write a conversation log by themselves, and simply voice data including the conversation contents of a plurality of users. Through this, a textual conversation log including conversation contents between a plurality of users may be provided.

In addition, the computing device 100 may provide a user interface including a conversation log to one or more user terminals. In this case, the user interface provides information on each of the one or more users who participated in the conversation, and the conversation contents of one or more users are classified by time and speaker and recorded in the form of a chat window. Convenience may be provided to a plurality of users who use the computing device 100 of the present disclosure, including various user interfaces (UIs)/user experiences (UXs) to improve.

In the present disclosure, a detailed description of a process in which the computing device 100 provides a chat log service to one or more user terminals 10 and a user interface including the chat log will be described later with reference to the following drawings.

FIG. 2 is a block diagram of a computing device providing a conversation log service according to an embodiment of the present disclosure.

Components of the computing device 100 providing the conversation log service shown in FIG. 2 are exemplary. Only some of the components shown in FIG. 2 may constitute the computing device 100 providing the chat log service, and additional component(s) other than the component shown in FIG. 2 may be added to the computing device 100 providing the chat log service. May be included.

As shown in FIG. 2, the computing device 100 providing a conversation log service may include a processor 110, a memory 120, and a network unit 130.

According to an embodiment of the present disclosure, the processor 110 may receive voice data from each of one or more user terminals 10. In this case, the voice data received by the processor 110 is received from each of the one or more user terminals 10, and one or more may exist. For example, when three users conduct a conversation, three voice data may be received from each of three user terminals held by each of the three users. That is, the processor 110 may receive one or more voice data from one or more user terminals 10 each possessed by a plurality of users, rather than receiving one voice data through one user terminal. The detailed description of the number of users conducting the above-described conversation and the number of user terminals corresponding thereto is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may perform preprocessing on each of one or more voice data received from each of the one or more user terminals 10. The processor 110 may perform pre-processing to increase a recognition rate for voice data. In this case, the preprocessing performed by the processor 110 on each of the one or more voice data may include preprocessing for removing noise and preprocessing for separating speakers based on the speech size in the speech section.

According to an embodiment of the present disclosure, the processor 110 may remove noise from each of one or more voice data received from each of the one or more user terminals 10. Removing noise from each of the one or more voice data may be for removing noise included in the one or more voice data. More specifically, the processor 110 may standardize the size of a signal included in each of the one or more voice data based on a comparison between the size of a signal included in each of the one or more voice data and a size of a predetermined reference signal. When the size of a signal included in each of the voice data received from one or more user terminals 10 is less than a predetermined reference signal, the processor 110 increases the size of the corresponding signal, and When the size of a signal included in each of the received voice data is greater than or equal to a predetermined reference signal, audio signal compression normalization may be performed to adjust the size of the corresponding signal to be small (ie, not to be clipping). The noise removal operation described above is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 may remove sounds other than speech by analyzing a waveform of a voice included in each of one or more voice data. More specifically, the processor 110 may remove at least one specific frequency by analyzing the frequency of the voice data. Each of the one or more voice data received by the processor 110 may include white noise having a uniform frequency, random frequency noise having an irregular frequency, or various noises generated according to a recording device or a signal processing process. That is, the processor 110 may perform preprocessing on noise having various frequencies included in each of one or more voice data. Specifically, the processor 110 classifies using a machine learning algorithm such as SVM (Supporting Vector Machine) in order to determine the type of noise included in each of one or more voice data, and each noise including different frequencies Noise can be removed through a corresponding noise reduction algorithm. The aforementioned noise reduction algorithm is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 may identify one or more speech sections for each of the one or more voice data. The processor 110 may identify one or more utterances included in the voice data. Specifically, the processor 110 may identify the user's speech section based on one or more silent sections present in the voice data. For example, the processor 110 may identify between the first silent section and the second silent section included in the voice data as the user's first speech section, and between the second silent section and the third silent section. It can be identified as a second speech section. That is, in the voice data including the conversation contents of one or more users, a point in time at which the voice of the one or more users (i.e., speech) is not recognized (for example, a point in time when only sound of a certain size or less is detected) is identified, and , It is possible to identify one or more user's speech sections based on the identified time point (ie, silent section). The detailed description of the above-described user's speech section is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may identify each voice of one or more users in each of one or more speech sections included in the voice data. In addition, the processor 110 may determine, as a specific speaker corresponding to the voice data, a speech having a voice size equal to or greater than a predetermined size among the identified voices of each of the one or more users. More specifically, the processor 110 may determine a specific speaker corresponding to the voice data when the size of a specific user's voice among one or more user voices included in the voice data is equal to or greater than a predetermined size. For example, when receiving voice data including voices of two users (user A and user B) from the first user terminal, the processor 110 determines that the voice of user A is equal to or greater than a predetermined size. , The user A may be determined as a specific speaker corresponding to the voice data received from the first user terminal. That is, in the present disclosure, since the user terminal may be a terminal disposed to each user in order to collect the voice of each user, the processor 110 is the user for which the corresponding voice data is collected when an utterance of a specific size or more It can be determined that it is the utterance of the user of the terminal. Also, the processor 110 may determine a specific speaker corresponding to the voice data based on a comparison of the sizes of voices of one or more users included in the voice data. For example, when voice data including the voices of two users (user A and user B) is received from the first user terminal, the processor 110 is the voice of user B among the voices of two users included in the voice data. It is determined that the volume of the voice is greater than that of the user A, and the user B may be determined as a specific speaker corresponding to the voice data received from the first user terminal. The detailed description of the loudness of the voices of one or more users and one or more users described above is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 may remove voices other than the voice of a specific speaker from voice data received from the user terminal 10. More specifically, when determining a specific speaker corresponding to the voice data, the processor 110 may remove the voice of a user other than the voice of the specific speaker from the voice data. For example, if the voice data includes the voices of user A and user B, and the user A is determined as a specific speaker corresponding to the voice data, the processor 110 may remove the voice of user B from the voice data. I can. That is, the processor 110 may more clearly separate the voice of a specific speaker (eg, user A) by removing the voice of another user (eg, user B) from the voice data. A detailed description of a user corresponding to the above-described voice data is only an example, and the present disclosure is not limited thereto. Accordingly, when a conversation record is written based on the voice data later, the separation of each speaker can be accurately performed, and a conversation record with higher accuracy can be generated.

In addition, the processor 110 determines a specific speaker corresponding to each of the one or more voice data, and when removing other voices other than the voice of a specific speaker determined from each of the voice data, the specific speaker is identified by one or more user terminals. Each piece of information can be matched and stored. For example, user A determined as a specific speaker corresponding to the first voice data may be matched with identification information of user A terminal, and user B determined as a specific speaker corresponding to the second voice data is matched with identification information of user B You can save it. Accordingly, a speaker corresponding to each of the one or more voice data received from each of the one or more user terminals is determined, and each speaker is matched to each user terminal, thereby more accurately separating each of the plurality of speakers having a conversation. . The detailed description of the above-described voice data and a user terminal corresponding thereto is only an example, and the present disclosure is not limited thereto.

Also, the processor 110 may divide the voice data into one or more voice data sub-data based on the speech section. For example, when receiving the first voice data through the user terminal A, the processor 110 converts the first voice data into one or more voice data sub-data based on one or more speech sections identified in the first voice data. Can be divided. For a more specific example, when there are 5 speech intervals identified in the first speech data, the processor 110 divides the first speech data into 5 speech data sub-data corresponding to each of the identified 5 speech intervals. can do. The detailed description of the number of one or more voice data sub-data of the above-described voice data is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may assign a unified viewpoint to each of one or more voice data received from each of the one or more user terminals 10.

Specifically, the processor 110 may tag global time information on each of one or more voice data received from each of the one or more user terminals 10. In this case, the global time information may be time information applied to each of a plurality of voice data received from each of a plurality of user terminals. More specifically, the global time information may be time information that serves as a reference given to clarify the relationship between the plurality of voice data. For example, the global time information may be time information determined based on time information of the computing device. The method of determining global time information is only an example, and the present disclosure is not limited thereto. The local time information may be time information applied to each voice data. That is, the local time information may be information about an elapsed time from the start of the voice data (eg, in the case of voice data for 8 minutes, a 32 second point, a 40 second point, etc.). The foregoing description of local time information is only an example, and the present disclosure is not limited thereto. That is, when receiving a plurality of voice data from each of a plurality of user terminals 10, the processor 110 matches and stores global time information with each of the plurality of voice data, thereby It is possible to clearly determine the relationship between the sub-data of one or more divided voice data. For example, when two voice data (first voice data and second voice data) are received through two user terminals (user terminal A and user terminal B), each speech included in the two voice data It may be difficult to grasp the precedence relationship of each voice data only with local time information. That is, when the recording start time points of the respective voice data are different from each other, the processor 110 may be difficult to clearly grasp the precedence and subsequent relationship of each utterance included in the plurality of voice data using only local time information. For example, the first voice data received from user A may be voice data for 11:10 to 11:20 (that is, voice data for 10 minutes), and received through user terminal B. The second voice data may be voice data from 11:12 to 11:20 (i.e., voice data for 8 minutes), so that the start points of each voice data are different from each other, so that the same local time information is provided. The utterances at the point of excitation may actually be utterances at different points in time. Accordingly, since the precedence relationship between the voice data sub-data included in each voice data is not clearly identified, it may cause a problem in generating a conversation log through each voice data, so that the processor 110 2 By matching global time information to each voice data (i.e., displaying the current time corresponding to each speech section) and unifying the time point of each data, it is divided according to the speech section in each of the first voice data and the second voice data. It is possible to clearly distinguish between the preceding and following relations of one or more voice data sub-data. The above-described detailed description of the number of user terminals, the number of voice data, and the time point of each voice data are only examples, and the present disclosure is not limited thereto.

Also, the processor 110 may determine a pre-and-after relationship of speech included in the speech data based on the elapsed time of each of the plurality of speech data. The processor 110 may determine a reference point in time applied to all of the plurality of voice data, and determine a precedence relationship between speech included in the plurality of voice data based on an elapsed time from the reference point in time. For example, the processor 110 determines the start time of the voice data from which recording is started first among the plurality of voice data as a reference time point, and determines the elapsed time based on the reference time point in other voice data, It is possible to determine the predecessor relationship of a plurality of utterances included in the data. Specifically, the processor 110 records a time when each of the one or more voice data received from each of the one or more user terminals 10 is initially generated, and each voice data based on the first time recorded in each of the one or more voice data. By identifying the progress time of the one or more voice data, a unified time point may be given to determine a precedence relationship between one or more voice data sub data included in each of the one or more voice data.

More specifically, the processor 110 determines the silence section to the voice data from which recording has been started, as much as the difference between the recording start point of the voice data where recording is first started and the recording start point of the voice data from which the recording has started. Can be added before starting. That is, the processor 110 matches the reference time point applied to the plurality of voice data (in this example, the recording start time of the voice data at which recording was first started, etc.), and determines the difference between each voice data from the reference time point. By reflecting it, you can determine the relationship between the utterances before and after. For example, if the recording of the second voice data is started 5 seconds after the recording of the first voice data is started, the processor 110 is at a reference time (for example, the time of the computing device) for the generation of voice data. This can be determined based on this, and by giving a 5-second silent period before the start of recording of the second voice data, the progress time of the first voice data and the second voice data may coincide with the actual speech time. The foregoing description is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 adds to the progression time of the voice data from which recording is started later, as much as the difference between the recording start point of the voice data where the recording is started first and the recording start point of the voice data from which the recording is started later, in addition to the pre- and post relationship of the speech Can judge. For example, if recording of the second voice data is started 5 seconds after the start of recording of the first voice data, a speech located 10 seconds from the start of recording of the second voice data is 15 seconds from the start of recording of the first voice data. It may be an utterance located in seconds. Accordingly, the processor 110 may determine a precedence relationship between each speech included in each speech data by adding a time equal to the difference between the first speech data and the start time to the second speech data to the progress time. The foregoing description is only an example, and the present disclosure is not limited thereto.

As described above, the processor 110 may match and store global time information with voice data, or match and store information on elapsed time from a reference point in time, thereby determining the predecessor relationship of a plurality of speeches included in the voice data have.

The above-described detailed description of the number of user terminals, the number of voice data, and the time point of each voice data are only examples, and the present disclosure is not limited thereto.

In the description of the voice data described below, the start point and the end point are based on at least one of global time information and reference point information of the voice data, and a precedence relationship between each of the one or more voice data sub-data included in each of the one or more voice data. It is based on a unified point of view assigned to each of one or more voice data in order to distinguish between them.

According to an embodiment of the present disclosure, the processor 110 may identify a start time and an end time of each of one or more voice data sub-data. Specifically, the processor 110 may identify a start time and an end time of each of the one or more voice data sub-data divided by the voice data based on the speech section included in the voice data. For example, when voice data is divided into two voice data sub-data (speech data first sub-data and voice data second sub-data) corresponding to two speech sections, the processor 110 The start time and end time of the data may be identified as 3 seconds and 32 seconds, respectively, and the start time and end time of the second sub data of the voice data may be identified as 1 minute 10 seconds and 1 minute 52 seconds, respectively. The detailed description of the start time and end time of the above-described voice data sub-data is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may output one or more voice recognition sub-results corresponding to each of the one or more voice data sub-data by using each of the one or more voice data sub-data as an input of the voice recognition model. . In this case, the speech recognition model is a model for converting a speech language spoken by a user into text data, and may include one or more network functions pre-trained through the training data. In addition, the processor 110 may determine the order of each of the one or more voice recognition sub-results in the conversation log based on a start time and an end time identified in each voice data sub-data that are the basis of one or more voice recognition sub-results . For a specific example, the processor 110 may input each of the first voice data sub-data and the second voice data sub-data into the voice recognition model to output the first voice recognition sub result and the second voice recognition sub result. The processor 110 identifies the start time and end time of the voice data first sub data based on the output of the voice recognition first sub result as 3 seconds and 32 seconds, respectively, and the voice data based on the output of the voice recognition second sub result The start and end times of the second sub data may be identified as 1 minute 10 seconds and 1 minute 52 seconds, respectively. The processor 110 may determine an order of the first voice recognition sub result and the second voice recognition sub result in the conversation log based on the determined time information. That is, the processor 110 identifies the start time and end time of the voice data sub-data corresponding to each of the one or more voice recognition sub-results, and at least one of the identified start time and end time An order in which each of the one or more voice recognition sub-results is displayed in the conversation log may be determined based on one. That is, one or more voice recognition sub-results in the conversation log may be arranged in order according to a start time point and an end time point. For example, if the start time of the speech recognition first sub result is earlier than the start time of the voice recognition second sub result, the processor 110 displays the first sub result of speech recognition before the second sub result of speech recognition in the conversation log. You can decide the order if possible. For another example, when the end point of the second voice recognition sub result is earlier than the end point of the first voice recognition result, the processor 110 displays the second voice recognition result before the first voice recognition sub result in the conversation log. You can also decide the order if possible. The detailed description of the start time and end time corresponding to the above-described speech recognition first sub-result and speech recognition second sub-result are examples only, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may generate a conversation log by converting each of one or more voice data into one or more voice recognition results through a voice recognition model. In this case, the one or more voice data may include first voice data and second voice data received from each user terminal. Further, the processor 110 divides the first voice data into one or more first voice data sub data based on the speech section, and divides the second voice data sub data into one or more second voice data sub data based on the speech section. Can be divided into For example, when there are three speech intervals identified in the first speech data, the processor 110 uses the first speech data as the first speech data, first sub data, and the first speech data, based on the three speech intervals. It may be divided into sub data and first voice data and third sub data, respectively. For another example, when there are two speech intervals identified in the second speech data, the processor 110 applies the second speech data to the second speech data, first sub data, and the second speech data, based on the two speech intervals. Each can be divided into 2 sub-data. The description of specific values of the speech section included in the above-described first voice data and second voice data is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 may identify a start point and an end point of each of the one or more first voice data sub-data and the one or more second voice data sub-data. For example, the first voice data is divided into first voice data, first sub data, first voice data, second sub data, and first voice data, and third sub data, based on three speech sections, and the second voice data When is divided into second voice data first sub-data and second voice data second sub-data based on two speech intervals, the processor 110 sets the starting points of the five voice data sub-data for 10 seconds ( First audio data, first sub data), 21 seconds (first audio data, second sub data), 32 seconds (first audio data, third sub data), 17 seconds (second audio data, first sub data) and 25 It can be identified by second (second voice data, second sub data). The detailed description of the starting point of the voice data sub-data is only an example, and the present disclosure is not limited thereto.

In addition, the processor 110 uses one or more first voice data sub-data and one or more second voice data sub-data as inputs of the voice recognition model, and one or more first voice recognition sub-results corresponding to each of the sub-data and one The above second voice recognition sub result may be output. For example, when the first sub-data of the first voice data is input to the voice recognition model, the first sub-result of the first voice recognition may be output, and if the first sub-data of the second voice data is input, the second The second sub result of speech recognition may be output.

In addition, the processor 110 may provide one or more presentations on the conversation log based on the start time and end time identified in each of the voice data sub-data that are the basis of the one or more first voice recognition sub-results and the one or more second voice recognition sub-results. An order in which each of the first voice recognition sub-result and one or more second voice recognition sub-results are displayed may be determined. For a specific example, in the above example, the first voice recognition first sub result, the first voice recognition second sub result, the first voice recognition third sub result, the second voice recognition first sub result, and the second voice recognition second Each of the two sub-results is based on the first voice data, the first sub data, the first voice data, the second sub data, the first voice data, the third sub data, the second voice data, the first sub data, and the second voice data sub data. Can be created. The processor 110 sets the starting points of the five voice data sub-data based on the generation of the five voice recognition sub-results, respectively, for 10 seconds (first voice data first sub data) and 21 seconds (first voice data first sub data). 2 sub data), 32 seconds (first voice data, third sub data), 17 seconds (second voice data, first sub data), and 25 seconds (second voice data, second sub data). Also, the processor 110 may determine an order in which the five voice recognition sub-results are displayed in the conversation log based on the identified start time point. That is, in the conversation log, the first voice recognition first sub result (10 seconds), the second voice recognition first sub result (17 seconds), the first voice recognition second sub result (21 seconds), and the second voice recognition second sub result. Results (25 seconds) may be arranged in the order of the first voice recognition third sub-result (32 seconds).

Also, the processor 110 may generate a conversation log based on the determined order of each of the one or more first voice recognition sub-results and the one or more second voice recognition sub-results, and identification information of each of the one or more user terminals 10. For example, the processor 110 may identify a speaker based on the identification information of the first user terminal matched with each of the one or more first voice recognition sub-results. In addition, the processor 110 may identify the speaker based on identification information of the second user terminal matched with each of the one or more second voice recognition sub-results. Accordingly, each voice recognition sub-result may be recorded in the chat log in the form of a chat window based on the identification information of each user terminal. That is, the processor 110 may generate a separate conversation log for each talker (eg, a first user using a first user terminal and a second user using a second user terminal).

According to an embodiment of the present disclosure, the processor 110 determines an overlapping section of the utterances of a plurality of users based on a start time and an end time of each of one or more first voice data sub-data and one or more second voice data sub-data. Can be identified. At this time, the first voice data sub-data is included in the first voice data received through the first user terminal, and the second voice data sub-data is included in the second voice data received through the second user terminal, Each may be generated based on voice data received from different user terminals. Specifically, the processor 110 divides the first voice data and the second voice data received from different terminals on the basis of each speech section, and at least one first voice data sub-data and one or more second voice data sub-data. Whether each of the first voice data sub-data and the second voice data sub-data are temporally overlapped may be identified through the start time and end time of each data. That is, the processor 110 may identify a section in which the utterances overlap among the plurality of utterances of a plurality of users in each voice data. For example, start and end of each of the first voice data, the first sub data, the first voice data, the second sub data, and the first voice data, the third sub data included in the first voice data received through the first user terminal. The time points are 10 to 15 seconds, 21 to 28 seconds, and 32 to 35 seconds, respectively, and the second voice data included in the second voice data received through the second user terminal First sub data, second voice data and second sub data When each start and end time is 17 to 20 seconds and 25 to 31 seconds, respectively, the processor 110 identifies that the second sub data of the first voice data and the second sub data of the second voice data overlap, so that from 25 seconds to 28 seconds. Up to seconds can be identified as overlapping intervals. The detailed description of the start time and end time of each of the above-described first voice recognition sub result and the second voice recognition sub result are only examples, and the present disclosure is not limited thereto.

Also, when the overlapping section is identified, the processor 110 may determine an order of the speech recognition sub-results corresponding to the overlapping section. In the following example, the first voice recognition sub result is a voice recognition sub result of one of the sub data of the first voice data included in the first voice data received from the first user terminal, and the second voice recognition sub result is the second voice recognition sub result. 2 It may be a voice recognition sub result of one of the sub data of the second voice data included in the second voice data received from the user terminal. The processor 110 may identify that the first voice recognition sub-result and the second voice recognition sub-result are utterances of times overlapping each other based on time information of the first voice data sub-data and the second voice data sub-data. In this case, the processor 110 may determine the order of the first voice recognition sub-result and the second voice recognition sub-result in the conversation log according to a predetermined criterion. For example, if the start time of the first voice data sub data, which is the basis of the first voice recognition sub result, is earlier than the start time of the second voice data sub data, the processor 110 May be placed before the second voice recognition sub result.

Also, the processor 110 may divide at least one of the speech recognition sub-results corresponding to the overlapping section. Specifically, the processor 110 is based on the start time and end time of each of the first subset of speech data and the second subset of speech data that are based on the output of each of the first speech recognition sub-result and the second speech recognition sub-result. Thus, when overlapping of the speech is identified, it may be determined to divide at least one of the first speech recognition sub-result and the second speech recognition sub-result. More specifically, when the second voice data sub-data starts between the start and end times of the first voice data sub-data, which is the basis of the first voice recognition sub-result, the processor 110 determines the first voice recognition sub-result. The second voice data may be divided based on the start time of the sub data. For example, if the first voice data sub-data is data including a 10-second speech, it may be assumed that the second voice data sub-data starts to utter five seconds after the first voice data sub-data starts to utter. have. In this case, the processor 110 may divide the first voice recognition sub-result, which is the voice recognition result for the speech of the first voice data sub-data for 10 seconds, based on 5 seconds and arrange them on the conversation log. That is, in this example, the processor 110 is the first text in a section between 0 seconds and 5 seconds of the first voice recognition sub result in the conversation log, the text of the second voice recognition sub result, and 5 seconds of the first voice recognition sub result. The second text in the interval between 10 seconds can be placed in order. The description of the time and division of the voice data sub-data described above is only an example, and the present disclosure is not limited thereto.

In addition, when the overlapping section is identified, the processor 110 may divide each voice recognition sub-result based on meaning. The processor 110 may determine a division point of the speech recognition sub-result based on natural language processing on the speech recognition sub-result. In addition, the processor 110 analyzes the combination probability of words included in the respective speech recognition sub-results based on the statistical language model for each of the first speech recognition sub-result and the second speech recognition sub-result corresponding to the overlapping section. It is possible to determine the end point in the sentence for the speech recognition sub-result. At this time, the statistical language model may be a model capable of calculating the probability of a sentence, or a probability of a next word appearing when previous words are input, and how appropriate is a combination of words, or how much a corresponding sentence is. It may be a model that can determine whether it is appropriate. Also, the processor 110 may determine whether to divide the speech recognition sub-results based on the determined end point of each speech recognition sub-result. For example, when the start and end times of the first voice data sub-data are identified as 5 seconds and 20 seconds, respectively, and the start and end times of the second voice data sub-data are identified as 12 seconds and 18 seconds, respectively (i.e. , When an overlapping section of speech for 6 seconds between 12 seconds and 18 seconds is identified), the processor 110 uses the first speech recognition sub result as an input of the statistical language model, and each included in the first speech recognition sub result. It is possible to determine the end point in the sentence for the first speech recognition sub-result by outputting the word of s and the suitability for a plurality of combinations of each word. Also, when the end point is determined from the result of the first voice recognition server, the processor 110 may determine to divide the first voice recognition sub result. For a more specific example, the speech of the voice data including the above-described overlapping section, "I want to see you (copyright related?) because I want to receive a copyright-related consultation. Yes, that is correct." Is assumed. In the above-described assumption, the utterance in parentheses may be the second speaker (ie, the utterance recognized from the second voice data), and the utterance outside the parenthesis may be the first speaker (ie, the utterance recognized from the first voice data). Statistical linguistic model in this case, each of the words included in the first speech recognition sub-result was'copyright','related','counseling','receiving','wanting','seeking','seeing', and'seeking'. ','yes','yes' can be recognized, and the combination of the words'was' and'yes' can be output to a lower fit, and accordingly, the processor 110 indicates'was' at the end of the sentence. Can be judged by points. Also, the processor 110 may determine to divide the first voice recognition sub-result based on an end point in the sentence determined in the first voice recognition sub-result. The detailed description of the start time and end time of each of the first voice data sub-data and the second voice data sub-data described above is only an example, and the present disclosure is not limited thereto.

Also, when it is determined that at least one of the first voice recognition sub-result and the second voice recognition sub-result is determined, the processor 110 may divide the voice recognition sub-result corresponding to the overlapping section. For a specific example, the start and end times of the first voice data sub-data are identified as 5 seconds and 20 seconds, respectively, and the start and end times of the second voice data sub-data are identified as 12 seconds and 18 seconds, respectively, Existence was identified, and each of the words included in the first voice recognition sub-result was'copyright','related','counseling','receiving','wanting','finding','seeing','see' When','yes','correct', and each of the words included in the second voice recognition sub-result is'copyright' and'relevant?', the processor 110 is a combination of words output from the statistical language model. Based on the suitability for,'it' of the first voice recognition sub-result may be determined as an end point in the sentence for the first voice recognition sub-result. In addition, the processor 110 may divide the first voice recognition sub result into'I want to receive copyright-related counseling' and'Yes, I am right' based on the end point determined from the first voice recognition sub result. . In other words, the processor 110 identifies that the second voice recognition sub-result, which is output based on the utterance of another user, exists between the start time and the end time corresponding to the first voice recognition sub-result (that is, the overlapping section). Identification) and grasping the meaning of the words included in the speech recognition sub-result in the overlapping section, the speech recognition sub-result can be divided so that the precedence relationship of each speech recognition sub-result is clearly displayed. That is, the processor 110 converts the first voice recognition sub-result to “copyright-related” so that the words included in the second voice recognition sub-result (that is, “copyright related?”) are placed between the first voice recognition sub results. It can be divided into “I’m here to meet you for consultation” and “Yes, that’s right”. The detailed description of the start time and end time of each of the above-described first voice data sub data and the second voice data sub data, and the specific description of the first voice recognition sub result and the second voice recognition sub result are only examples. The disclosure is not limited thereto.

As described above, the present disclosure determines the order of one or more voice recognition results output from the two voice data by receiving two voice data from each user terminal, and the two user terminals receiving the two voice data. Although the conversation record may be generated based on the identification information, it may be obvious to a person skilled in the art that the conversation record may be generated by receiving two or more pieces of voice data through the processes of the present disclosure.

According to an embodiment of the present disclosure, the processor 110 may determine a keyword corresponding to the conversation log. Specifically, the processor 110 may determine one or more keywords for the conversation log based on at least one of a semantic analysis result of one or more speech recognition results, a number of repetitions for each word included in the one or more speech recognition results, and a predetermined conversation topic. have.

The processor 110 may determine one or more keywords for the conversation log based on a result of semantic analysis of the one or more speech recognition results. More specifically, the processor 110 may determine one or more keywords based on the importance of the meaning of each of the one or more words included in each of the one or more speech recognition results. For example, if each of the words included in the speech recognition result is'copyright','related','consultation','receive','want','find','see you','been', the processor (110) may determine that the meaning of'copyright' or'counseling' is of relatively high importance, and thus'copyright' or'counseling' as a keyword. One or more words included in the above-described speech recognition result and a detailed description of the importance of each of the one or more words are only examples, and the present disclosure is not limited thereto.

Also, the processor 110 may determine one or more keywords for the conversation log based on the number of repetitions for each word included in each of the one or more speech recognition results. Specifically, the processor 110 may identify the number of repetitions (ie, the number of duplicates) of each of one or more words included in the speech recognition result, and determine a word having a high repetition number as a keyword for the conversation log.

Also, the processor 110 may determine a keyword based on a predetermined conversation topic. More specifically, the processor 110 may determine a topic of conversation from a semantic analysis of the result of speech recognition, or receive a topic of conversation from the user terminal 10, and determine a keyword in response to the received topic. have. For example, when receiving a'copyright' as a conversation topic from the user terminal 10, the processor 110 uses'infringement','complaint', and'confession' related to the topic (i.e. You can decide. In this case, one or more keywords may be pre-stored in the memory 120 corresponding to each of the plurality of conversation topics. The detailed description of the above-described conversation topic and keywords corresponding to the conversation topic is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may identify locations of one or more keywords in one or more speech recognition results. Specifically, the processor 110 may identify at least one of a position on the chat log and a position on the voice data of each of one or more keywords among one or more speech recognition results included in the chat log. In addition, the processor 110 may identify the position of each of the one or more keywords in the result of one or more speech recognition included in the conversation log, and reproduce the voice data sub-data corresponding to each of the one or more keywords. In detail, the processor 110 identifies a keyword in the conversation log generated through one or more speech recognition results output by using each of the one or more speech data as an input of the speech recognition model, and the identified keyword in response to the identified keyword. Audio data sub-data including keywords can be reproduced. That is, the processor 110 may identify only a portion of the voice data received from the user terminal 10 corresponding to a specific keyword in the conversation log and provide it as a voice recognition result, and the voice data subsidiary corresponding to the identified voice recognition result Since the data section can be reproduced, convenience can be provided to the user. In other words, the user can only be provided with an important specific part of the chat log, and a playback screen that allows the user to play the section corresponding to that specific part of the recorded data (i.e., voice data) can be provided. It is possible to search only specific important parts (ie, keywords) as needed without having to listen to all the data, so that convenience can be provided to grasp the point.

According to an embodiment of the present disclosure, the processor 110 may provide a user interface including a conversation log generated based on voice data received from each of the one or more user terminals 10 to one or more user terminals.

A detailed description of the user interface provided by the processor 110 to one or more user terminals 10 will be described later with reference to FIGS. 4 to 7.

4 is a diagram illustrating an exemplary user interface including a user list display screen and a conversation list display screen according to an embodiment of the present disclosure.

The user interface may be displayed on the display means of the user terminal 10.

The user interface generated by the processor 110 and provided to the one or more user terminals 10 includes profile information 310 of a user using the conversation log service, as shown in FIG. 4A, and other users registered by the user A user list display screen 340 may be provided to display a list of users, access display information 320 regarding whether other registered users are connected, and a registration request button 330 for additional registration of other users. However, since the information displayed on the corresponding user list display screen is not essential, additional information may be further included in the user list display information.

The user list display screen 340 included in the user interface may display the user's profile information 310. For example, the user's profile information displayed on the user interface may include information related to the user's name, company name, position, and contact information. In addition, the user list display screen 340 included in the user interface may display a list of one or more other users registered as friends by the user, and the connection status of one or more other users corresponding to each list of one or more other users. The indicated connection indication information 320 may be displayed. For example, as shown in FIG. 4, the user interface may display another user named'Choi Wonhyuk' registered by the user on the user list display screen, and access display information indicating the current connection status of'Choi Wonhyuk' Can be displayed together. For a more specific example, when a user terminal corresponding to'Choi Won-hyuk' among one or more other users does not access the computing device 100, the user interface may indicate access indication information such as reference number 321 for the user. When user terminals corresponding to each of'Taebin Jang' and'Seonhyung Kim' access the computing device 100, the user interface may indicate access indication information such as 322 and 323. The detailed description of the user list display screen included in the user interface is only an example, and the present disclosure is not limited thereto.

In addition, the user list display screen 340 included in the user interface may additionally display a friend add button 330 for registering another user, and the processor 110 may provide a corresponding friend add button from the user terminal 10 ( When a selection input for 330) is received, it may be determined to transmit a control signal to the user terminal to provide an ID input window through which a unique ID of another user who intends to conduct a conversation or conference can be input. In addition, when receiving information about another user's ID from the user terminal 10 through the corresponding ID input window, the processor 110 allows the other user's information (e.g., ID ) May be added to the user list display screen 340 to transmit a control signal for display to the user terminal 10.

In addition, the user interface may include a conversation list display screen 350 on which one or more conversations performed by a user with other users are displayed. Specifically, the user interface may provide a screen displaying the names of each of one or more conversations performed with other users based on a user's selection input on the conversation list display screen 350. In this case, the conversation list display screen 350 included in the user interface includes one or more conversation items 370 performed by the user, a conversation content search button 360, and a chat room opening button 390 for opening a chat room. I can.

The conversation list display screen 350 included in the user interface may arrange and display the names of each of one or more conversations performed by a user with each of one or more other users. In this case, the list of one or more conversations, as shown in Figure 4, IR meeting 371, weekly meeting 372, OO company meeting 373, OOO's interview 374, OOO's consultation 375, and It can be displayed aligned with the entire conference 376. In addition, the name of each conversation may be generated based on a selection input received from the user terminal when opening a chat room, and may be generated as a result of semantic analysis for each conversation or based on a keyword determined in each conversation. It may have been created.

Also, the conversation list display screen 350 included in the user interface may display one or more viewpoint information corresponding to each one or more conversations performed by the user. For example, as illustrated in FIG. 4, when the name of the conversation performed by the user is the IR meeting 371, the '2018.10.03. 14:00' can be displayed. Also, the names of each of the one or more conversations displayed on the conversation list display screen may be arranged in descending or ascending order based on viewpoint information.

In addition, the conversation list display screen 350 included in the user interface may display a conversation search button 360 for searching for a specific conversation among one or more conversations performed by the user. When a selection input for a conversation search button is received from the user terminal 10 through the conversation list display screen 350, the processor 110 transmits a control signal to provide a search window for inputting a number or text. 10) can decide to send. In addition, when receiving information on numbers or texts from the user terminal 10 in the search window, the processor 110 searches for and provides a conversation corresponding to the information on numbers or texts received from the user terminal 10 It may be determined to transmit a control signal to be performed to the user terminal 10. In this case, the selection input that the processor 110 receives from the user terminal 10 may be information on at least one of a name of a conversation, information about a time point of the conversation, and information on one or more users included in the conversation. That is, the processor 110 may specify one or more conversations based on the name of the conversation, information about the time point of the conversation, and information on one or more users included in the conversation (eg, ID information for each of the one or more users who have performed the conversation) You can search for conversations.

In addition, when receiving a selection input from a user terminal for a specific conversation name among one or more conversation names displayed on the conversation list display screen 350 included in the user interface, the processor 110 is based on the selection input of the user terminal. It may be determined to transmit a control signal for providing a conversation log display screen corresponding to the corresponding conversation name to the user terminal 10. A detailed description of the conversation log display screen provided to the user terminal based on a selection input by the user terminal for a specific conversation name among one or more conversation names performed by the user will be described later with reference to FIG. 6.

In addition, the conversation list display screen 350 included in the user interface may include a chat room opening button for receiving a selection input from the user terminal 10 in order for a user to open a chat room with one or more other users. In addition, when receiving a selection input for the chat room opening button from the user terminal 10, the processor 110 provides a control signal for providing a user selection input screen for including one or more other users in the chat room. ) To send. A detailed description of a chat room opened based on a user selection input screen included in the user interface and a selection input of a user terminal for the user selection input screen will be described later with reference to FIG. 5.

5 is a diagram illustrating a process of opening a chat room and a user interface for a chat room including a plurality of users according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the processor 110 selects a user for including one or more other users in the chat room in response to a selection input of the user terminal 10 for a chat room opening button (reference numeral 390 in FIG. 4). It may be determined to transmit a control signal for providing an input screen to the user terminal 10. In this case, the user selection input screen may display a list of one or more other users registered by the user, and a selection button for selecting the one or more other users corresponding to each of the one or more other users, as shown in FIG. 5. It may include. For a more specific example, referring to FIG. 5, when a user wants to open a chat room including other users with IDs'Taebin Jang','Seonhyung Kim' and'Kihoon Byun' among one or more other users registered by the user ,'Taebin Jang','Seonhyung Kim' and'Kihun Byun', respectively, each of the corresponding selection buttons (reference numbers 422, 423, 424) can be selected and input, and a button for selection completion can be selected and input. In other words, the processor 110 responds to the selection input for each of one or more other users (reference numbers 422, 423, 444) and the confirmation button 430 for selection completion through the user selection input screen from the user terminal. In response, a control signal for opening a chat room may be determined to be transmitted to the user terminal 10. The foregoing description is only an example, and the present disclosure is not limited thereto.

In addition, when a new chat room between the user and one or more other users is opened, the processor 110 may determine to transmit a control signal for providing a chat room display screen to the user terminal. For example, when the processor 110 receives a selection input of a user terminal for one or more other users ('Taebin Jang','Seonhyung Kim' and'Kihoon Byun') as shown in FIG. The chat room display screen as shown in (b) can be provided. In this case, the chat room display screen may display information about the time when the chat room was opened, as shown in (b) of FIG. 5, and information of users included in the chat room (eg, user ID information (' ','Kim Sun-hyung' and'Byeon Ki-hoon')) can be displayed. The foregoing description is only an example, and the present disclosure is not limited thereto.

In addition, the chat room display screen included in the user interface may include a recording start button 440 for receiving one or more voice data from each of one or more users included in the chat room. In addition, when receiving a selection input for the recording start button displayed on the chat room display screen from the user terminal, the processor 110 may receive one or more voice data from each of the one or more user terminals based on the time when the selection input is received. I can.

In addition, the chat room display screen included in the user interface may display one or more voice recognition results generated based on one or more voice data received from each of one or more users included in the chat room, arranged by time and by speaker. The configuration in which one or more voice recognition results generated based on one or more voice data received from each user are sorted and displayed by time and by speaker has been described in detail with reference to FIG. 2, and redundant descriptions will be omitted. The user interface displays the voice recognition result output based on the voice data received from the user's own terminal on one side of the chat room display screen, and the voice output based on each voice data received from one or more other user terminals. The recognition result may be displayed on the other side of the chat room display screen. For example, as shown in (b) of FIG. 5, the voice recognition result output through the voice data of a specific user 450 is placed on the left side of the chat room display screen in response to the specific user 450 (451 ), and the voice recognition result output through voice data of one or more other users 460 except for a specific user may be disposed 461 on the right side of the chat room display screen. That is, the user interface can provide convenience to the user by securing the readability of the text by displaying the conversation history of a plurality of users in the form of a chat window so that the user can easily grasp the conversation content.

In addition, when receiving a selection input for a specific conversation name among one or more conversation names displayed on the conversation log list display screen 350 from the user terminal, the processor 110 transmits a control signal for providing the conversation log display screen to the user terminal. Can decide. A detailed description of the conversation log display screen will be described later with reference to FIG. 6.

6 is a diagram illustrating an exemplary user interface for providing a conversation log display screen in response to a selection input of a user terminal for a specific conversation among one or more conversation lists performed by a user according to an embodiment of the present disclosure.

When receiving a selection input from a user terminal for a specific conversation name among one or more conversation names displayed on the conversation list display screen 350 included in the user interface, the processor 110 determines the corresponding conversation name based on the selection input of the user terminal. It may be determined to transmit a control signal for providing a conversation log display screen corresponding to the user terminal. For a specific example, as shown in (a) of FIG. 6, a selection input 510 for a conversation name corresponding to a'service planning meeting' among one or more conversation names displayed on the conversation list display screen is input to the user terminal 10. ), the processor 110 may transmit a control signal to the user terminal 10 to provide a conversation log display screen corresponding to the'service planning meeting'. In this case, the conversation log display screen may include items as shown in (b) of FIG. 6. More specifically, the conversation log display screen includes information 530 on each of one or more users who have performed a conversation, information on one or more keywords related to the chat room 540, and one or more speech recognition results listed by time and speaker. A voice reproduction screen 560 for reproducing information 550 and a voice section corresponding to a voice recognition result may be included. The foregoing description is only an example, and the present disclosure is not limited thereto.

In addition, when receiving a selection input for at least one of one or more keywords and one or more speech recognition results displayed on the conversation log display screen 350 from the user terminal 10, the processor 110 Can play.

A detailed description of how the processor 110 reproduces voice data based on a selection input for one or more keywords and one or more voice recognition results from the user terminal 10 will be described later with reference to FIG. 7.

7 is a diagram illustrating an exemplary user interface in which a screen for reproducing a specific time point of voice data is displayed in response to a user's selection input for a voice recognition result and a keyword according to an embodiment of the present disclosure.

The user interface may display one or more voice recognition results on the conversation log display screen, and the processor 110 responds to a selection input from the user terminal for at least one voice recognition result among the one or more voice recognition results. It may be determined to transmit a control signal to the user terminal 10 to provide a voice reproducing screen 630 for reproducing voice data at the generation point of For a specific example, as shown in FIG. 7, the conversation log display screen may display one or more voice recognition results, such as 611, 612, and 613. In this case, when receiving a selection input for reference number 612 from the user terminal 10, the processor 110 reproduces the voice data sub-data starting from the speech section of the voice data corresponding to the voice recognition result. It may be determined to transmit a control signal for providing the playback screen 630 to the user terminal 10. That is, the processor 110 may provide a conversation log including one or more voice recognition results, and in the conversation log, play the corresponding voice data sub-data in response to a selection input for a specific voice recognition result through the user terminal 10. I can. Accordingly, a user can receive a conversation performed with one or more other users in both a text form and a voice form. In addition, since it is difficult to grasp the contents of a specific section in the voice file, the user can quickly identify a specific section in the conversation content through one or more speech recognition results (i.e., text) displayed in the conversation log and serve the voice data corresponding to the specific section. Since data can be reproduced, it is possible to easily grasp the contents of the conversation performed.

In addition, the processor 110 reproduces the voice data sub-data at the time of generating one or more voice recognition results matched with the corresponding keyword in response to a selection input for a specific keyword among one or more keywords from the user terminal 10 It may be determined to transmit a control signal for providing the screen 630 to the user terminal 10. In this case, the positions of one or more keywords may be identified in each of one or more speech recognition results, and a specific keyword may be identified in one or more positions among one or more speech recognition results included in the conversation log. Accordingly, the processor 110 may display one or more locations of the identified keywords on the voice reproduction screen 630. For a more specific example, as shown in FIG. 7, one or more keywords displayed on the conversation log display screen include'law','counseling' and'copyright', and selection for a specific'copyright' from the user terminal 10 When receiving an input, the processor 110 may provide a control signal to the user terminal 10 to provide a voice reproduction screen 630 for reproducing the voice data sub-data of the section corresponding to the keyword, One or more sections corresponding to “copyright” (ie, a keyword selected by the user) in the voice data may be displayed on the voice playback screen 630. That is, the processor 110 may display a section (ie, voice data sub-data) in which the keyword exists on the voice reproduction screen 630 so that the user can easily search for a section corresponding to a specific keyword. Accordingly, the user can easily reproduce a section corresponding to a specific keyword in the voice data including the content of the conversation performed with another user, and thus can easily perform a search for specific content.

According to an embodiment of the present disclosure, the user interface may include a text editing screen for receiving a request for correction of one or more speech recognition results included in a conversation log from a user terminal. In addition, when the processor 110 receives a request to modify a specific speech recognition result among one or more speech recognition results included in the conversation log from the user terminal 10, the speech recognition model is The speech recognition model can be corrected to output. In this case, the speech recognition model may output a speech recognition result corresponding to each of one or more speech data sub-data by using a speech to text (STT) algorithm. That is, the processor 110 may improve the performance of the STT algorithm by reflecting the feedback corresponding to the request to modify the speech recognition result received from the user terminal 10 to the STT algorithm.

According to an embodiment of the present disclosure, the memory 120 is configured to input a program code executable by the processor 110, identification information for each of one or more user terminals, and voice data to a speech recognition model that outputs a speech recognition result. It is possible to store information related to information about the natural language processing algorithm and the like. For example, information on a modification request for modifying one or more speech recognition results received from a user terminal through a user interface may be stored.

According to an embodiment of the present disclosure, the memory 120 is a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, For example, SD or XD memory), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read) -Only Memory), a magnetic memory, a magnetic disk, and an optical disk. The computing device 100 may operate in connection with a web storage that performs a storage function of the memory on the Internet. The description of the above-described memory is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the network unit 130 may transmit and receive information with one or more user terminals 10. More specifically, the network unit 130 may include a wired/wireless Internet module for network access. As a wireless Internet technology, WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), and the like may be used. As a wired Internet technology, XDSL (Digital Subscriber Line), FTTH (Fibers to the Home), PLC (Power Line Communication), etc. may be used.

In addition, the network unit 130 may include a short range communication module and may be located relatively close to the user terminal and transmit and receive data to and from an electronic device including a short range communication module. As short range communication technology, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

3 is a flowchart for providing a chat log service according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the computing device 100 may receive one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user (210). . In this case, the voice data received by the computing device 100 is received from each of the one or more user terminals 10, and one or more may exist.

According to an embodiment of the present disclosure, the computing device 100 may generate a conversation log by converting each of one or more voice data into one or more voice recognition results through a voice recognition model (220).

The computing device 100 may identify one or more speech sections for each of the one or more voice data. The computing device 100 may identify one or more speeches included in the voice data. Specifically, the computing device 100 may identify the user's speech section based on one or more silent sections present in the voice data. In addition, the computing device 100 may divide the voice data into one or more voice data sub-data based on the speech section.

In addition, the computing device 100 may identify each voice of one or more users in each of one or more speech sections included in the voice data. In addition, the computing device 100 may determine, as a specific speaker corresponding to the speech data, a speech having a volume equal to or greater than a predetermined size among the identified voices of each of the one or more users. In addition, the computing device 100 may match and store a specific speaker corresponding to the voice data with identification information of the user terminal.

In addition, the computing device 100 may assign a unified viewpoint to each of the one or more voice data received from each of the one or more user terminals 10.

In addition, the computing device 100 may identify a start point and an end point of each of one or more voice data sub-data. Specifically, the computing device 100 may identify a start time point and an end time point of each of the one or more voice data sub-data divided by the voice data based on the speech section included in the voice data.

In addition, the computing device 100 may output one or more voice recognition sub-results corresponding to each of the one or more voice data sub-data by using each of the one or more voice data sub-data as an input of the voice recognition model. In this case, the speech recognition model is a model for converting a speech language spoken by a user into text data, and may include one or more network functions pre-trained through the training data. In addition, the computing device 100 may determine the order of each of the one or more voice recognition sub-results in the conversation log based on the start time and the end time identified in each voice data sub-data that are the basis of one or more voice recognition sub-results. have.

Also, the computing device 100 may generate a conversation log based on the order of each of one or more voice recognition sub-results.

According to an embodiment of the present disclosure, the computing device 100 may generate a user interface including a conversation log and provide it to one or more user terminals (230).

8 shows a means for providing a chat log service related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, in order to provide a chat log service, the computing device 100 includes one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user. Means for receiving (710), means (720) for converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record, and a user interface including the conversation record It may include a means 730 for providing to the above user terminal.

Alternatively, means for normalizing the size of a signal included in each of the one or more voice data based on a comparison of the size of a signal included in each of the one or more voice data with a size of a predetermined reference signal, and the one or more It may further include a means for removing at least one specific frequency by analyzing the frequencies included in each of the voice data.

Alternatively, means for identifying at least one speech section based on one or more silent sections included in the voice data, and separating a speaker based on the size of the spoken voice in at least one speech section included in the voice data It may further include means for.

Alternatively, the means for separating a speaker based on a speech size in one or more speech sections included in each of the one or more speech data includes a size of a speech speech having a predetermined size or more among the one or more speeches included in the speech data Means for determining a speech having a specific speaker corresponding to the voice data, means for removing voices other than the determined specific speaker's voice from the voice data, and a user terminal for transmitting the voice data to the specific speaker It may include means for matching and storing the identification information of.

Alternatively, the means for generating a conversation record by converting each of the one or more speech data into one or more speech recognition results through a speech recognition model comprises: dividing the speech data into one or more speech data sub-data based on the speech section. Means for identifying a start time and an end time of each of the one or more voice data sub-data, and corresponding to each of the one or more voice data sub-data by using each of the one or more voice data sub-data as inputs of the voice recognition model Means for outputting one or more speech recognition sub-results, and each of the one or more speech recognition sub-results based on a start time and an end time identified in each of the voice data sub-data on which the one or more voice recognition sub-results are based. It may comprise means for determining the order in the conversation log.

Alternatively, the one or more voice data includes first voice data and second voice data received from each user terminal, and converts each of the one or more voice data into one or more voice recognition results through a voice recognition model. The means for generating the conversation record comprises: dividing the first voice data into one or more first voice data sub-data based on the speech section, and dividing the second voice data into one or more second voice data sub-data based on the speech section Means for dividing into data, means for identifying a start time and an end time of each of the first voice data sub data and the second voice data sub data, the first voice data sub data and the second voice data sub data Means for outputting at least one first speech recognition sub result and at least one second speech recognition sub result corresponding to each of the sub data as an input of the speech recognition model, the at least one first speech recognition sub result, and the Each of the one or more first voice recognition sub-results and the one or more second voice recognition sub-results based on the start and end times identified in each of the voice data sub-data on which the one or more second voice recognition sub-results are based. And means for determining an order on the chat log, and means for generating a chat log based on the determined order and identification information of each of the one or more user terminals.

Alternatively, one or more first voice recognition sub-results and second voice-recognition sub-results corresponding to each sub-data by using the first voice data sub-data and the second voice data sub-data as inputs of the voice recognition model The means for outputting is means for identifying an overlapping section based on a start time and an end time of each of the first voice data sub-data and the second voice data sub-data, when the overlapping section is identified, the overlapping A means for determining whether to divide at least one of the first voice recognition sub-result and the second voice recognition sub-result corresponding to a section, and if the division is determined, dividing the voice recognition sub-result corresponding to the overlapping section It may include means for.

Alternatively, when the overlapping section is identified, means for determining whether to divide at least one of the first voice recognition sub-result and the second voice recognition sub-result corresponding to the overlapping section is in the overlapping section. It may include means for determining whether to divide the speech recognition result with respect to the corresponding speech recognition result through a statistical language model.

Alternatively, for determining one or more keywords for the conversation log based on at least one of a result of semantic analysis of the one or more speech recognition results, the number of repetitions for each word included in the one or more speech recognition results, and a predetermined conversation topic. It may further comprise means and means for identifying the location of each of the one or more keywords in the one or more speech recognition results.

Alternatively, the user interface may include a user's profile, a list of one or more other users registered by the user, a registration request button for the user to register an additional user, and a user indicating whether to access each of the one or more other users Including a list display screen, and information on the names of each of the one or more conversations the user participated in, a conversation search button for searching for a specific conversation among the one or more conversations, and a chat room opening button for opening a chat room with one or more other users It may include a conversation list display screen that displays.

Alternatively, the processor, when receiving a selection input for the chat room opening button from the user terminal, sends a control signal to the user terminal to provide a user selection input screen for including the one or more other users in the chat room. It may further comprise means for determining what to transmit.

Alternatively, when receiving a selection input for a specific chat room from a list of one or more chat rooms in which the user participates from a user terminal, the processor provides a chat log display screen including a chat log and one or more keywords corresponding to the specific chat room. It may further comprise means for deciding to transmit to the user terminal a control signal for causing.

Alternatively, the processor, when receiving a selection input for a specific keyword among the one or more keywords from the user terminal, searches for a speech recognition result corresponding to the keyword, and based on the generation of the searched speech recognition result. It may further include means for determining to transmit a control signal to the user terminal to provide a keyword reproduction screen for reproducing the speech section of the voice data.

Alternatively, when a new chat room is opened, the user interface includes a recording start button for receiving voice data from each of one or more users included in the chat room, and one or more buttons received from each of one or more users included in the chat room. It may include a conversation log display screen that displays one or more voice recognition results generated based on voice data.

Alternatively, the user interface includes a text editing screen for receiving a request for modification of one or more speech recognition results included in the conversation log from a user terminal, and the processor displays the text editing screen from the user terminal. When receiving a request for modification of one or more voice recognition results included in the conversation log through, means for correcting the voice recognition model so that the voice recognition model outputs a voice recognition result corresponding to the modification request. I can.

9 illustrates a module for providing a chat log service related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, in order to provide a chat log service, the computing device 100 includes one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user. A module 810 for receiving, a module 820 for generating a conversation record by converting each of the one or more voice data into one or more voice recognition results through a voice recognition model, and a user interface including the conversation record It may include a module 830 for providing to the above user terminal.

10 is a diagram illustrating logic for providing a chat log service related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, in order to provide a chat log service, the computing device 100 includes one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user. A logic 910 for receiving, a logic 920 for generating a conversation record by converting each of the one or more voice data into one or more voice recognition results through a voice recognition model, and a user interface including the conversation record. It may include a logic 930 for providing to the above user terminal.

11 illustrates a circuit for providing a conversation log service related to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, in order to provide a chat log service, the computing device 100 includes one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user. A circuit 1010 for receiving, a circuit 1020 for generating a conversation record by converting each of the one or more voice data into one or more voice recognition results through a voice recognition model, and a user interface including the conversation record. A circuit 1030 for providing to the above user terminal may be included.

Those of skill in the art would further describe the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein, including electronic hardware, computer software, or a combination of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the specific application and design restrictions imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

12 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

While the present disclosure has generally been described above with respect to computer-executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Further, to those skilled in the art, the method of the present disclosure is not limited to single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable household appliances, etc. It will be appreciated that it may be implemented with other computer system configurations, including one or more associated devices).

The described embodiments of the present disclosure may also be practiced in a distributed computing environment where certain tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Computer-readable media can be any computer-readable media, including volatile and non-volatile media, transitory and non-transitory media, removable and non-transitory media. Includes removable media. By way of example, and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Includes the medium. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device, Or any other medium that can be accessed by a computer and used to store desired information.

Computer-readable transmission media typically implement computer-readable instructions, data structures, program modules or other data on a modulated data signal such as a carrier wave or other transport mechanism. Includes all information transmission media. The term modulated data signal refers to a signal in which one or more of the characteristics of the signal is set or changed to encode information in the signal. By way of example, and not limitation, computer-readable transmission media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above-described media are also intended to be included within the scope of computer-readable transmission media.

An exemplary environment 1100 is shown that implements various aspects of the present disclosure, including a computer 1102, which includes a processing device 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to, system memory 1106 to processing device 1104. The processing unit 1104 may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be used as processing unit 1104.

The system bus 1108 may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read-only memory (ROM) 2110 and random access memory (RAM) 2112. The basic input/output system (BIOS) is stored in nonvolatile memory 2110 such as ROM, EPROM, EEPROM, etc. This BIOS is a basic input/output system that helps transfer information between components in the computer 1102, such as during startup. Includes routines. RAM 2112 may also include high speed RAM, such as static RAM, for caching data.

The computer 1102 also has an internal hard disk drive (HDD) 2114 (e.g., EIDE, SATA), and the internal hard disk drive 2114 can also be configured for external use within a suitable chassis (not shown). ), a magnetic floppy disk drive (FDD) 2116 (for example, to read from or write to a removable diskette 2118), and an optical disk drive 1120 (e.g., a CD-ROM disk (For reading 1122) or for reading from or writing to other high-capacity optical media such as DVDs). The hard disk drive 2114, the magnetic disk drive 2116 and the optical disk drive 1120 are each connected to the system bus 1108 by the hard disk drive interface 1124, the magnetic disk drive interface 1126 and the optical drive interface 1128. ) Can be connected. The interface 1124 for implementing an external drive includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of the computer-readable medium above refers to a removable optical medium such as a HDD, a removable magnetic disk, and a CD or DVD, those skilled in the art may use a zip drive, a magnetic cassette, a flash memory card, a cartridge, etc. It will be appreciated that other types of computer-readable media, such as the like, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

A number of program modules, including an operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136, may be stored in the drive and RAM 2112. All or part of the operating system, applications, modules, and/or data may also be cached in RAM 2112. It will be appreciated that the present disclosure may be implemented in a number of commercially available operating systems or combinations of operating systems.

A user may input commands and information into the computer 1102 through one or more wired/wireless input devices, for example, a pointing device such as a keyboard 2138 and a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, a parallel port, IEEE 1394 serial port, game port, USB port, IR interface, Can be connected by other interfaces such as etc.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, the computer generally includes other peripheral output devices (not shown) such as speakers, printers, etc.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communication. The remote computer(s) 1148 may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and is generally connected to the computer 1102. Although it includes many or all of the components described for simplicity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or to a larger network, eg, a wide area network (WAN) 1154. Such LAN and WAN networking environments are common in offices and companies, and facilitate an enterprise-wide computer network such as an intranet, all of which can be connected to a worldwide computer network, for example the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 via a wired and/or wireless communication network interface or adapter 1156. Adapter 1156 may facilitate wired or wireless communication to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158, connected to a communications computing device on the WAN 1154, or through the Internet, to establish communications over the WAN 1154. Have other means. The modem 1158, which may be an internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for the computer 1102 or portions thereof may be stored in the remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers may be used.

Computer 1102 is associated with any wireless device or entity deployed and operated in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant (PDA), communication satellite, wireless detectable tag. It operates to communicate with any equipment or place and phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Thus, the communication may be a predefined structure as in a conventional network or may simply be ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) allows you to connect to the Internet, etc. without wires. Wi-Fi is a wireless technology such as a cell phone that allows such devices, for example computers, to transmit and receive data indoors and outdoors, ie anywhere within the coverage area of a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a,b,g, etc.) to provide a secure, reliable and high-speed wireless connection. Wi-Fi can be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz radio bands, for example at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). have.

Those of ordinary skill in the art of the present disclosure will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields. Or particles, or any combination thereof.

A person of ordinary skill in the art of the present disclosure includes various exemplary logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein, electronic hardware, (convenience). For the sake of clarity, it will be appreciated that it may be implemented by various forms of program or design code or a combination of both (referred to herein as "software"). To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. A person of ordinary skill in the art of the present disclosure may implement the described functions in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CD, DVD, etc.), smart cards, and flash memory. Devices (eg, EEPROM, card, stick, key drive, etc.), but is not limited to these. In addition, the various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on the design priorities, it is to be understood that within the scope of the present disclosure a specific order or hierarchy of steps in processes may be rearranged. The appended method claims provide elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art, and general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but is to be interpreted in the widest scope consistent with the principles and novel features presented herein.

Claims (17)

A computer program stored in a computer-readable storage medium executable by one or more processors, the computer program causing the one or more processors to perform the following operations, the operations:
Receiving one or more voice data from each of one or more user terminals, the voice data including one or more utterances for each user;
Converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record; And
Generating a user interface including the conversation log and providing it to the one or more user terminals;
Contains, and
The operation of converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record,
Dividing the speech recognition result for each speaker by identifying an overlapping section of speech based on time information of the one or more speech data and analyzing the speech recognition result of the overlapping section by using a statistical language model;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
Standardizing the size of a signal included in each of the one or more voice data based on a comparison of the size of a signal included in each of the one or more voice data with a predetermined size of a reference signal; And
Removing at least one specific frequency by analyzing a frequency included in each of the one or more voice data;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
Identifying one or more speech sections based on one or more silent sections included in the voice data; And
Separating a speaker based on the size of the spoken speech in one or more speech sections included in the speech data;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 3,
The operation of separating a speaker based on a speech size in one or more speech sections included in each of the one or more speech data,
Determining, as a specific speaker corresponding to the speech data, a speech having a size of an uttered speech equal to or greater than a predetermined size among at least one speech included in the speech data;
Removing voices other than the determined specific speaker's voice from the voice data; And
Matching and storing the specific speaker with identification information of a user terminal that has transmitted the voice data;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The operation of converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record,
Dividing the voice data into one or more voice data sub-data based on a speech section;
Identifying a start time and an end time of each of the one or more voice data sub-data;
Outputting one or more voice recognition sub-results corresponding to each of the one or more voice data sub-data by using each of the one or more voice data sub-data as an input of the voice recognition model; And
Determining an order of each of the one or more voice recognition sub-results in the conversation log based on a start time and an end time point identified in each of the voice data sub-data on which the one or more voice recognition sub-results are based;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The one or more voice data includes first voice data and second voice data received from each user terminal,
The operation of converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record,
Dividing the first voice data into one or more first voice data sub-data based on a speech section, and dividing the second voice data into one or more second voice data sub-data based on a speech section;
Identifying a start time point and an end time point of each of the first voice data sub-data and the second voice data sub-data;
Using the first voice data sub-data and the second voice data sub-data as inputs of the voice recognition model, one or more first voice recognition sub-results and one or more second voice recognition sub-results corresponding to each of the sub-data are obtained. Outputting;
The one or more first voice recognition sub-results and the one or more first voice recognition sub-results based on a start time and an end time identified in each voice data sub-data based on the one or more first voice recognition sub-results and Determining an order of each of the one or more second voice recognition sub-results on the conversation log; And
Generating a conversation log based on the determined order and identification information of each of the one or more user terminals;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 6,
Outputting one or more first voice recognition sub-results and second voice-recognition sub-results corresponding to each sub-data by using the first voice data sub-data and the second voice data sub-data as inputs of the voice recognition model silver,
Identifying an overlapping section based on a start time point and an end time point of each of the first voice data sub-data and the second voice data sub-data;
Determining whether to divide at least one of the first voice recognition sub-result and the second voice recognition sub-result corresponding to the overlapping section when the overlapping section is identified; And
If the division is determined, dividing the speech recognition sub-result corresponding to the overlapping section;
Containing,
A computer program stored on a computer-readable storage medium.
delete The method of claim 1,
Determining one or more keywords for the conversation log based on at least one of a result of semantic analysis of the one or more speech recognition results, a number of repetitions for each word included in the one or more speech recognition results, and a predetermined conversation topic; And
Identifying a location of each of the one or more keywords in the one or more speech recognition results;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The user interface,
A user profile, a list of one or more other users registered by the user, a registration request button for registering an additional user by the user, and a user list display screen indicating whether access to each of the one or more other users, and
A conversation list display screen displaying information on the names of each of the one or more conversations the user has participated in, a conversation search button for searching for a specific conversation among the one or more conversations, and a chat room opening button for opening a chat room with one or more other users. Included,
A computer program stored on a computer-readable storage medium.
The method of claim 10,
The processor,
When receiving a selection input for the chat room opening button from the user terminal, determining to transmit a control signal to the user terminal to provide a user selection input screen for including the one or more other users in the chat room;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 10,
The processor,
When receiving a selection input for a specific chat room from a list of one or more chat rooms in which the user participates from a user terminal, a control signal for providing a chat log display screen including a chat log and one or more keywords corresponding to the specific chat room is transmitted to the user terminal Determining to send to;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 12,
The processor,
When receiving a selection input for a specific keyword among the one or more keywords from the user terminal, searching for a speech recognition result corresponding to the keyword, and reproducing a speech section of speech data based on the generation of the searched speech recognition result Determining to transmit a control signal for providing a keyword playback screen for the user terminal;
Further comprising,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The user interface,
When a new chat room is opened, a recording start button for receiving voice data from each of the one or more users included in the chat room and one or more generated based on one or more voice data received from each of the one or more users included in the chat room Including a conversation log display screen for displaying a speech recognition result,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The user interface,
A text editing screen for receiving a request for modification of one or more speech recognition results included in the conversation log from a user terminal, and
The processor,
When receiving a request to modify one or more voice recognition results included in the conversation log from the user terminal through the text editing screen, the voice recognition model so that the voice recognition model outputs a voice recognition result corresponding to the modification request The operation of correcting;
Further comprising,
A computer program stored on a computer-readable storage medium.
As a method for providing a conversation log service,
Receiving one or more voice data from each of one or more user terminals, wherein the voice data includes one or more utterances for each user;
Converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation record; And
Generating a user interface including the conversation log and providing it to the one or more user terminals;
Contains, and
Converting each of the one or more voice data into one or more voice recognition results through a voice recognition model to generate a conversation log,
Dividing the speech recognition result for each speaker by identifying an overlapping section of the speech based on time information of the one or more speech data, and analyzing the speech recognition result of the overlapping section by using a statistical language model;
Containing,
Method for providing conversation log service.
As a server to provide conversation log service,
A server processor including one or more cores;
A server memory storing program codes executable in the server processor; And
A server network unit for transmitting and receiving data to and from one or more user terminals;
Contains, and
The server processor,
Receiving one or more voice data from each of one or more user terminals, wherein the voice data includes one or more utterances for each user,
Each of the one or more speech data is converted into one or more speech recognition results through a speech recognition model, an overlapping section of speech is identified based on time information of the one or more speech data, and the speech recognition result of the overlapping section is a statistical language model By analyzing using, the speech recognition result is divided for each speaker to generate a conversation record, and
Generating a user interface including the conversation log and providing it to the one or more user terminals,
Server that provides chat log service.

Images