KR102136393B1 - Apparatus and Method for managing text changed from voice in call - Google Patents

Abstract

Disclosed is an apparatus and method for converting and managing voice during a call into text. According to an aspect, an apparatus for text-converting a voice of a service subscriber and managing the converted text includes: a receiver configured to receive voice data of a call in which a service subscriber becomes a caller or a receiver; A classification unit for classifying the received voice data into outgoing voice data and received voice data, respectively; A conversion unit that converts the classified voice data into data of outgoing text and received text, respectively; A call text unit for distinguishing the converted caller's calling text and recipient's receiving text and arranging them in chronological order to generate call text; And a provision unit for retrieving the generated call text at the request of the service subscriber and providing it to the terminal of the service subscriber.

Description

Apparatus and Method for managing text changed from voice in call}

The present invention relates to an apparatus and method for converting a conversational voice of a caller and a recipient into text during a call, and managing the converted text during a call.

Previously, the method of storing the contents of a call was the only service for recording the contents of the call. The user installs a recording application on the mobile terminal, and presses the recording button of the application during a mobile call to save the call as a voice file.

Here, the user must listen to the recorded voice file from beginning to end in order to confirm the recorded contents. If there are multiple recorded voice files, the user must listen to all of the plurality of voice files until the desired content is found.

Therefore, in the case of a plurality of voice files, it is difficult to check which voice file the user wants and which location of the voice file exists.

Korean Registered Patent 10-0935524 (Dec. 28, 2009)

The present invention is to solve the above problems, and an object of the present invention is to provide an apparatus and method for converting a call voice between a caller and a receiver into text and managing the converted text.

According to an aspect, an apparatus for text-converting a voice of a service subscriber and managing the converted text includes: a receiver configured to receive voice data of a call in which the service subscriber becomes a caller or a receiver; A classification unit classifying the received voice data into outgoing voice data and received voice data, respectively; A conversion unit for converting the classified voice data into data of outgoing text and received text, respectively; A call text unit for distinguishing the converted sender's calling text and recipient's received text and arranging them in chronological order to generate call text; And a provision unit for querying the generated call text and providing it to the terminal of the service subscriber at the request of the service subscriber.

The apparatus further includes a DTMF unit that receives a dual tone multiple frequency (DTMF) signal requesting text conversion from the calling terminal of the service subscriber in a call, and the classifying unit classifies the received DTMF signal.

The classification unit classifies voice data of outgoing and receiving by referring to the source ID and port of the SIP (Session Initiation Protocol) message, and the source ID and port of the receiving side and the synchronization source ID of the Realtime Transfer Protocol (RTP) packet.

The apparatus further includes a silence removing unit that removes a Silence Indicator (SID) packet corresponding to silence to leave only a voice packet among the RTP packets of the classified outgoing voice data and received voice data, and the conversion unit leaves the remaining voice packet The text is converted to text using the STT (Speech To Text) engine.

The apparatus further includes a timestamp unit for calculating a voice generation time as timestamp information of the converted text by using the time amount of the RTP packet referenced in the ptime value of the Session Description Protocol (SDP) of the SIP message.

The timestamp unit checks codec information of the RTP packet, checks a timestamp increasing per second from the sample rate of the identified codec, and calculates the time amount of the SID packet from the timestamp value of the SID packet increased from the previous packet. Calculate and calculate the voice generation time by using the time amount of the RTP packet and the time amount of the calculated SID packet.

The call text unit stores the call text including at least one set of outgoing phone number, incoming call number, total call time, outgoing text data and voice generation time, and at least one set of received text data and voice generation time do.

The apparatus, at a starting position of the voice data that has arrived first among the outgoing voice data and the received voice data, is configured to display the voice data of the late arrival or reception voice data by the time of the RTP packet of the first arrival voice data. Further comprising a mixing unit for setting a starting position, and mixing the voice stream of each of the outgoing voice data and the received voice data into a single stream of integrated voice data using the set starting position, the providing unit is the integrated Provide voice data.

The providing unit provides the call text to a terminal of a service subscriber by using at least one of a text message, an email, a social network service (SNS), and a web page.

According to another aspect, a method for a device to text-convert a service subscriber's call voice and manage the converted text includes: the service subscriber receiving voice data of a call to be a caller or a recipient; Classifying the received voice data into outgoing voice data and received voice data, respectively; Converting the classified voice data into data of outgoing text and received text, respectively; Generating call texts by separating the converted sender's calling text and the recipient's receiving text and listing them in chronological order; And at the request of the service subscriber, querying the generated call text and providing it to the terminal of the service subscriber.

According to an aspect of the present invention, by managing the call content as text, the user can search and search the text of the call content as needed.

In text conversion of the call content, the processing load of the STT engine can be reduced by removing the silent packet and converting only the packet of the call voice.

When storing caller's and receiver's call contents as one integrated voice data, cross talk by setting the start position of the voice packet of the sender or receiver arriving late by the amount of time of the packet of the sender or receiver arriving first Can store one unified voice data that has been removed.

The following drawings attached to this specification are intended to illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with the detailed description of the invention described below, and thus the present invention is described in such drawings. It should not be interpreted as being limited to.
1 is a schematic configuration diagram of a call management system according to an embodiment of the present invention.
FIG. 2 is a schematic configuration diagram of the text management server of FIG. 1.
3 is an exemplary diagram in which the text management server of FIG. 2 classifies voice data of Tx and Rx.
4A and 4B are exemplary diagrams in which the text management server of FIG. 2 classifies call voice packets into outgoing voice packets and incoming voice packets.
5 is an exemplary diagram of an SID packet among call voice packets received by the text management server of FIG. 2.
FIG. 6 is an exemplary diagram in which the text management server of FIG. 2 calculates a timestamp time of the SID packet of FIG. 5.
FIG. 7 is an exemplary diagram in which the text management server of FIG. 1 mixes outgoing voice data and received voice data into one integrated voice stream according to another embodiment of the present invention.
8A and 8B are exemplary diagrams of delaying the start position of voice data of a calling or receiving side that arrives late in the mixing of FIG. 7.
9 is a schematic flowchart of a text management method of a voice call according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to ordinary or lexical meanings, and the inventor appropriately explains the concept of terms to explain his or her invention in the best way. Based on the principle that it can be defined, it should be interpreted as meanings and concepts consistent with the technical spirit of the present invention. Accordingly, the embodiments shown in the embodiments and the drawings described in this specification are only the most preferred embodiments of the present invention and do not represent all of the technical spirit of the present invention, and thus can replace them at the time of application. It should be understood that there may be equivalents and variations.

1 is a schematic configuration diagram of a call management system 100 according to an embodiment of the present invention.

The call management system 100 according to an embodiment of the present invention includes a caller terminal 110, a text management server 130, a telephony application server (TAS) 140, a call session control function (CSCF) 150, and It comprises a receiver call terminal 170.

When a call call is sent from the caller terminal 110 to the caller terminal 170, call data (call data) is relayed through the carrier server, and when the caller terminal 170 receives the incoming call, the call is connected. The voice call of the caller and receiver is started. The call data is divided into signal data (eg, SIP messages) and voice data (eg, RTP packets), and each classified data has a separate processing and transmission path.

In the VoLTE environment, the carrier server includes the TAS 140 and the CSCF 150 for processing the signal data. The CSCF 150 relays the SIP message of the call data from the call processing server. In addition, the TAS 140 is a server that provides call-based multimedia supplementary services (eg, caller ID service, call ring tone service, call waiting service, etc.) to subscribers of the corresponding supplementary service. When a subscriber of a call service subscribes to the call management service of the present invention, service subscription information is managed by the TAS 140. The voice data of the call data is relayed to the caller terminal 110 and the caller terminal 170 through the communication company's call processing server.

Here, the present invention implements the call processing server as a text management server (130). The text management server 130 provides a text conversion service of a call content to a subscriber of the call management service in the process of relaying voice data of call data between the caller terminal 110 and the caller terminal 170. That is, the call management service may provide a query and search service to the subscriber by converting the voice call contents into text and managing the converted text at the request of the service subscriber.

2 is a schematic configuration diagram of the text management server 130 of FIG. 1.

The text management server 130 according to an embodiment of the present invention includes a receiving unit 231, a DTMF unit 232, a classification unit 233, a silence removing unit 234, a conversion unit 235, and a time stamp unit 236. ), a call text unit 237 and a provision unit 239.

Assuming that the text management server 130 is a computer terminal composed of a memory and a processor, each component 231 to 239 may be loaded into a memory in the form of a program and executed through the processor. For example, each of the components 231 to 239 is produced by a text management program, and then executed by a processor of the text management server 130 to provide the text management service using text converted from voice of a call. can do.

The receiving unit 231 receives each voice data of the caller and the receiver generated during the call in RTP packets. Of course, the outgoing voice data is relayed to the destination call terminal 170 via the text management server 130 as a destination, and the received voice data is relayed to the destination call terminal 110 via the text management server 130 as a destination. do.

The DTMF unit 232 receives a DTMF signal requesting text conversion of a call voice from a subscriber's call terminal during a call of a subscriber subscribed to the call management service of the present invention. That is, when the subscriber of the call management service requests text conversion of a call voice during a call, a specific key (eg, * key) is pressed to generate a DTMF signal.

When the DTMF signal of the text management service is confirmed by the DTMF unit 232, the classification unit 233 classifies the voice data received from the reception unit 231 into outgoing voice data and received voice data, respectively. For reference, the classification of the voice data will be described later with reference to FIGS. 3 to 4B.

The silence removing unit 234 removes the silent RTP packet because the silence does not need to be converted to text when each RTP packet of the outgoing voice data and the received voice data classified in the classification unit 233 corresponds to silence. By the removal, only RTP packets containing a call voice can be left. For reference, the silent RTP packet will be described later with reference to FIGS. 5 and 6.

The conversion unit 235 converts the voice of the outgoing RTP packet into the outgoing text, and the voice of the received RTP packet into the received text. The conversion unit 235 is composed of an STT engine, and may be constructed as a separate STT server.

Here, the conversion unit 235 has the advantage of reducing the data processing load because only the RTP packets of the call voice in which the silent RTP packet is removed by the silence removing unit 234 can be input and converted to STT-based text. .

The time stamp unit 236 calculates the time stamp value of the RTP packet as time, and sets the calculated time as time stamp information corresponding to the talk time of the converted text. For example, the converted text of the call voice is timestamped with the occurrence time like a message.

Here, the reason why the time stamp unit 236 calculates the time is that the time of the text management server 130, the caller terminal 110, and the caller terminal 170 is different, and the path of the RTP packet and the path are passed through. Because time is different. In addition, since the arrival order of RTP packets arriving at the text management server 130 does not match the voice generation order, the arrival time should not be used as the voice generation time. In consideration of this, the timestamp unit 236 sets the call start time based on the call start time for accurate call voice generation time, and then calculates the generation time based on the timestamp value included in each RTP packet. Therefore, by the above calculation, the speech generation time of text converted from speech has the advantage of accuracy. The call start time corresponding to the reference time may be the arrival time of the first packet. For reference, the calculation will be described later with reference to FIG. 6.

The call text unit 237 generates call text by distinguishing the caller's calling text and the callee's received text converted by the conversion unit 235, and arranged in chronological order calculated by the time stamp unit 356, and the generated call Save the text.

Here, the stored call text information is to be provided to service subscribers, at least one set of an outgoing phone number, an incoming phone number, total call time, outgoing text data and voice generation time, at least one of received text data and voice generation time Includes the above set. That is, the set consists of each text sentence and its voice generation time. The screen UI on which the call text information is displayed can be configured similarly to the messenger screen, and does not place any special restrictions.

The providing unit 239 queries the call text data managed by the call text unit 237 at the request of the service subscriber, and provides it to the terminal of the service subscriber.

The call text information may be provided to a terminal of a service subscriber using at least one of a text message, an email, a social network service (SNS), and a web page. For example, the web page in which the caller's and receiver's call contents are listed as text information may be provided to the terminal of the service subscriber, such as the screen UI of KakaoTalk.

3 is an exemplary diagram in which the text management server 130 of FIG. 2 classifies voice data of Tx and Rx.

When the service subscriber A becomes the caller and calls the recipient, the Rx stream of the outgoing data is the voice content of A, and the Tx stream of the received data is the voice content of the receiver that the caller A hears.

The classification unit 233 of the text management server 130 includes a source IP and a port (src IP/Port), a destination IP and a port (dst IP/Port) included in a SIP (Session Initiation Protocol) message of call data. The RTP packet of the call data is classified into the Tx stream of the outgoing voice data and the Rx stream of the received voice data by referring to the Synchronized Source ID (SSRC) of the RTP packet. SSRC corresponds to an identifier of a call source sound source connected to a stream by synchronization.

For example, when the service subscriber A is a caller and makes a call to the recipient, the source IP/Port is the call terminal of A and the destination IP/Port is the text management server 130. It is classified as a Tx stream. In addition, the RTP packets whose source IP/Port is the text management server 130 and the destination IP/Port is the call terminal of A are classified into Rx streams of received voice data that the other party speaks and A hears.

4A and 4B are exemplary diagrams in which the text management server 130 of FIG. 2 classifies a call voice packet into an outgoing voice packet and a received voice packet.

4A, a voice RTP packet of a sender and a receiver arriving at the text management server 130 is shown. Assuming that RTP packet 1 is the incoming voice and RTP packet 2 is the outgoing voice, after the first three packets of RTP packet 1 of the receiving side arrive at the text management server 130, then the two packets of RTP packet 2 of the calling side It arrives. As the conversation progresses during a call, each RTP packet arrives at the text management server 130.

Here, when the time portion of the RTP packet has 20 ms of voice data, it can be interpreted that the receiver of the RTP packet 1 speaks for 60 ms first, and then the sender of the RTP packet 2 speaks.

Referring to FIG. 4B, the classification unit 233 classifies the voice packets of the RTP packet 1 of the receiving side and the RTP packet 2 of the calling side, respectively, with reference to the IP/Port value and SSRC value as in FIG.

FIG. 5 is an exemplary diagram of the SID packet 500 among the call voice packets received by the text management server of FIG. 2.

The RTP packet of the voice data is divided into a voice packet including a real human voice and a silent SID packet 500. The SID packet 500 is a silent RTP packet generated when a user stops speaking and listens.

Looking at the actual conversation in the call, the other party hears while the caller or recipient speaks. When the voice data is separated into Tx and Rx data on the receiving side, about 1/2 of each separated data is silent. That is, half of RTP packets of call data may be SID packets 500 corresponding to the silence.

When the value defined for each codec is checked in the "FT" field defined in the payload of the RTP packet, the silent SID packet 500 can be known. For reference, in the case of the AMR-NB codec of FIG. 5, when the value “8” of “FT” is confirmed, the corresponding RTP packet is a SID packet 500 in which the user's voice is silent. Also, in the case of the AMR-WB codec, if the value "9" of "FT" is confirmed, it is the SID packet 500. For reference, a method of checking codec information will be described later with reference to FIG. 6. The silence removing unit 234 filters the SID packet 500 by checking the value of the “FT” field in the RTP packet, and leaves only the voice packet. If only the remaining voice packets are transmitted to the STT server conversion unit 235 for text conversion, the load of the STT server is reduced.

FIG. 6 is an exemplary diagram in which the text management server 130 of FIG. 2 calculates the timestamp time of the SID packet 500 of FIG. 5.

It is assumed that according to the voice order, the n-1th RTP packet, the nth RTP packet 600, the n+1st SID packet 500, and the n+2th RTP packet are listed. Each RTP packet includes a timestamp value. In the RTP packet, the packet containing the voice has the same amount of time, but the time amount of the SID packet 500 is different for each codec, so the time amount must be calculated from the information of each codec.

In the case of an RTP packet including voice, the time stamp unit 236 of the text management server 130 defines the time amount of the RTP packet containing voice as a ptime value (eg, 20 ms) of the SDP of the SIP message.

For the RTP packet corresponding to the SID packet 500, the time stamp unit 236 calculates the amount of time by referring to the codec information. The timestamp unit 236 may check the codec header in the RTP payload or obtain codec information from the length of the payload. For reference, the VoLTE terminal uses EVS, AMR-WB, and AMR-NB codecs. For example, in the case of the EVS codec, the SID packet 500 is a silent packet, which has a shorter payload length than the normal voice packet, and the normal voice packets are transmitted every 20 ms, and the SID packet 500 is transmitted every 20 ms. It can be transmitted or delivered over a longer period (multiple of 20 ms). That is, the SID packet 500 is required to calculate a time amount corresponding to the packet size, and the time amount can be calculated using the time stamp value of the RTP packet.

For example, assuming that the timestamp of the nth RTP packet 600 is 1,000 and the timestamp of the n+1st SID packet 500 is 2,600, the timestamp unit 236 sets the difference value of the timestamp to 1,600. Is calculated as In the RTP packets 500 and 600, the timestamp field is a relative value related to the sampling rate of the call codec. Therefore, in order to calculate the absolute amount of time through the time stamp of RTP, it is necessary to know the sampling rate of the call codec. The sampling rate can be known by checking the SDP of the SIP message when a call is established. Assuming that a call is established with the AMR-WB codec, the time stamp unit 236 refers to the sampling rate of the AMR-WB codec from SDP 16 kHz. Since 16 kHz means sampling 16000 times per second, 16 times are sampled in 1 ms, and the timestamp value of RTP increases by 16 per 1 ms. In the case of the voice RTP packet 600, since the RTP packet is transmitted every 20 ms, the timestamp value increases by 320 (16*20).

Here, since the difference value of the timestamp of the RTP packet 600 and the SID packet 500 was calculated to be 1,600, 1,600/320=5, and 5*20 ms = 100 ms is calculated. Then, the time stamp unit 236 calculates the time amount of the SID packet 500 as 100 ms. The combined time amount of the RTP packet 600 and the SID packet 500 is calculated as 20+100 = 120 ms. Accordingly, in the order of the n-1th RTP packet, the nth RTP packet 600, the n+1st SID packet 500, and the n+2th RTP packet, the timestamp unit 236 of the RTP packet 600 When the timestamp information of the time t1 is set in the text, the occurrence time t1+120 can be calculated in the text of the n+2th RTP packet and set as the timestamp information.

7 is an exemplary diagram in which the text management server 130 of FIG. 1 mixes outgoing voice data and received voice data into one integrated voice stream according to another embodiment of the present invention.

In another embodiment of the present invention, when the service subscriber receives and confirms the call text from the text management server 130 and requests the call voice content, the mixing unit of FIG. 2 provides a voice stream of the call content to the service subscriber 238 mixes the outgoing call voice and the incoming call voice into one unified voice.

Here, it is assumed that the RTP packet 1 stream 710 of the received voice and the RTP packet 2 stream 730 of the outgoing voice are mixed into one integrated voice stream.

As shown in FIG. 4B, when mixing the same starting positions of the RTP packets 400 and 410 for reception and transmission, a cross talk occurs in which the calling voice and the calling voice overlap. In order to prevent cross talk and to mix a call voice stream that is the same as the actual call content as much as possible, the mixing unit 238 RTP packet 2 of outgoing voice arriving late 3 packets of RTP packet 1 of incoming voice arriving first to stream 730 Delays by 3*20 ms = 60 ms.

8A and 8B are exemplary diagrams of delaying the start position of voice data of a calling or receiving side that arrives late in the mixing of FIG. 7.

Referring to FIG. 8A, the actual call contents of A and B are shown based on the time axis. The call starts with the voice of A, and the conversation between A and B continues.

Referring to FIG. 8B, when the starting positions of the RTP packets of Tx and Rx are the same or based on the arrival time of the packets without applying the technology of the present invention, A is generated in the areas 810 and 830 where the cross talk occurs. The voices of and B may overlap.

In order to exclude the crosstalk, the present invention mixes by arranging the stream of the RTP packet of B at the starting position where the mixing unit 238 is delayed back by the amount of time of the RTP packet of A arriving first. Thus, A and B call voices are mixed into one unified call stream without crosstalk.

9 is a schematic flowchart of a text management method of a voice call according to an embodiment of the present invention.

When a call is initiated between the caller terminal 110 and the caller terminal 170, the text management server 130 receives the call and receive call data (S910). Here, the text management server 130 receives the SIP message of the signal data through the TAS 140. In addition, the text management server receives RTP packets of voice data.

During a call, when a service subscriber presses a key button of a call terminal to request a text conversion service of the present invention, the text management server 130 receives a DTMF signal of a corresponding service key from the call terminal of the subscriber (S920).

By receiving the DTMF signal, the text management server 130 classifies the RTP packet of the outgoing voice data and the packet of the received voice data (S930). The classification of the packet is processed by referring to the source IP and port, the destination IP and port, and the SSRC value as shown in FIG. 3. Also, as shown in FIG. 4B, each is classified into a receiving side RTP packet 1 400 and a calling side RTP packet 2 410.

When the RTP packets of the sender and the receiver are classified, the text management server 130 removes the silent SID packet 500 from each of the classified RTP packets, and leaves RTP packets including voice (S940).

After the silent SID packet 500 is removed, the text conversion server 130 performs text conversion on the RTP packets including the voice using the STT engine (S950).

Here, the text conversion server 130 calculates the start time of each packet by setting the reference time as the initial call generation time relative to each RTP packet (400, 410) arranged in the order of voice generation as shown in FIG. 4B (S951) ). As described above with reference to FIG. 6, the time amount of the SID packet 500 is calculated with reference to codec information.

After the text conversion and time calculation are completed, the text conversion server 130 matches the converted test and the calculated voice generation time to one set, and displays the set screen UI to display a call including each matched set. The text is generated and stored (S960).

Thereafter, when the service subscriber requests the provision of the call text, the text conversion server 130 inquires and provides the call text corresponding to the request of the subscriber (S970). That is, the text conversion server 130 may provide a search and search service for call text changed from the call content at the request of the service subscriber.

Although the present invention has been described by means of limited embodiments and drawings, the present invention is not limited by this, and the technical idea of the present invention and claims to be described below by those skilled in the art to which the present invention pertains Of course, various modifications and variations are possible within the scope of the scope.

100: system 110: caller terminal
120: text management server 140: TAS
150: CSCF 170: call recipient terminal

Claims (18)

In the apparatus for text conversion of the voice of the service subscriber, and managing the converted text,
A receiver configured to receive voice data of a call in which the service subscriber becomes a caller or a receiver;
A classification unit classifying the received voice data into outgoing voice data and received voice data, respectively;
A conversion unit for converting the classified voice data into data of outgoing text and received text, respectively;
A call text unit for distinguishing the converted sender's calling text and recipient's received text and arranging them in chronological order to generate call text;
At the request of the service subscriber, a provision unit for querying the generated call text and providing it to the terminal of the service subscriber; And
And a silence removing unit for removing a Silence Indicator (SID) packet corresponding to silence in order to leave only a voice packet among RTP packets of the classified outgoing voice data and received voice data,
The conversion unit converts the remaining voice packet into text using a speech to text (STT) engine. According to claim 1,
Further comprising a DTMF unit for receiving a dual tone multiple frequency (DTMF) signal requesting text conversion from the call terminal of the service subscriber in a call,
The classification unit is characterized in that the classification processing by the received DTMF signal. According to claim 1,
The classification unit,
A device characterized by classifying voice data of transmission and reception by referring to a synchronization source ID of an IP and a port of a SIP (Session Initiation Protocol) message, an IP and a port of a receiver, and a Realtime Transfer Protocol (RTP) packet. delete In the apparatus for text conversion of the voice of the service subscriber, and managing the converted text,
A receiver configured to receive voice data of a call in which the service subscriber becomes a caller or a receiver;
A classification unit classifying the received voice data into outgoing voice data and received voice data, respectively;
A conversion unit for converting the classified voice data into data of outgoing text and received text, respectively;
A call text unit for distinguishing the converted sender's calling text and recipient's received text and arranging them in chronological order to generate call text;
At the request of the service subscriber, a provision unit for querying the generated call text and providing it to the terminal of the service subscriber; And
And a timestamp unit for calculating a voice generation time as timestamp information of the text converted using the time amount of the RTP packet referenced in the ptime value of the Session Description Protocol (SDP) of the SIP message. The method of claim 5,
The time stamp portion,
Check the codec information of the RTP packet, check the timestamp increasing per second from the confirmed sampling rate of the codec, calculate the time amount of the SID packet from the timestamp value of the SID packet increased from the previous packet, and calculate the RTP And calculating the voice generation time using the time amount of the packet and the time amount of the calculated SID packet. The method of claim 1 or 5,
The currency text section,
Characterized by storing the call text comprising at least one set of outgoing phone number, incoming phone number, total call time, outgoing text data and voice generation time, at least one set of incoming text data and voice generation time Device. In the apparatus for text conversion of the voice of the service subscriber, and managing the converted text,
A receiver configured to receive voice data of a call in which the service subscriber becomes a caller or a receiver;
A classification unit classifying the received voice data into outgoing voice data and received voice data, respectively;
A conversion unit for converting the classified voice data into data of outgoing text and received text, respectively;
A call text unit for distinguishing the converted sender's calling text and recipient's received text and arranging them in chronological order to generate call text;
At the request of the service subscriber, a provision unit for querying the generated call text and providing it to the terminal of the service subscriber; And
Set the starting position of the voice data of the outgoing or received voice data from the starting position of the voice data that has arrived first among the outgoing voice data and the received voice data by the time of the RTP packet of the first arrived voice data. And a mixing unit for mixing each of the outgoing voice data and the received voice data into a single integrated voice data stream using the set starting position.
And the providing unit provides the integrated voice data. The method of any one of claims 1, 5, or 8,
The providing unit,
Device for providing the call text to the terminal of the service subscriber by using at least one of a text message, an email, a social network service (SNS), and a web page. A method for a device to text-convert a service subscriber's call voice and manage the converted text,
The service subscriber receiving voice data of a call to be a caller or a recipient;
Classifying the received voice data into outgoing voice data and received voice data, respectively;
Converting the classified voice data into data of outgoing text and received text, respectively;
Generating call texts by distinguishing the converted sender's calling text and recipient's receiving text and arranging them in chronological order; And
And in response to the request of the service subscriber, querying the generated call text and providing it to the terminal of the service subscriber,
Since the step of sorting,
Further comprising the step of removing the SID (Silence Indicator) packet corresponding to the silence to leave only the voice packet from the RTP packets of the classified outgoing voice data and received voice data,
The step of converting is a step of converting the remaining voice packet into text using a speech to text (STT) engine. The method of claim 10,
Before the sorting step,
The method further includes receiving a dual tone multiple frequency (DTMF) signal requesting text conversion from the calling terminal of the service subscriber in a call,
The step of classifying is a step of classifying by the received DTMF signal. The method of claim 10,
The classification step,
It is a step of classifying voice data of sending and receiving by referring to a synchronization source ID of an IP and a port of a SIP (Session Initiation Protocol) message, an IP and a port of a receiver, and a Realtime Transfer Protocol (RTP) packet, respectively. Way. delete A method for a device to text-convert a service subscriber's call voice and manage the converted text,
The service subscriber receiving voice data of a call to be a caller or a recipient;
Classifying the received voice data into outgoing voice data and received voice data, respectively;
Converting the classified voice data into data of outgoing text and received text, respectively;
Generating call texts by distinguishing the converted sender's calling text and recipient's receiving text and arranging them in chronological order; And
And in response to the request of the service subscriber, querying the generated call text and providing it to the terminal of the service subscriber,
Since the step of converting,
And calculating a voice generation time as timestamp information of the converted text by using the time portion of the RTP packet referenced in the ptime value of the Session Description Protocol (SDP) of the SIP message. The method of claim 14,
The calculating step,
Check the codec information of the RTP packet, check the timestamp increasing per second from the confirmed sampling rate of the codec, calculate the time amount of the SID packet from the timestamp value of the SID packet increased from the previous packet, and calculate the RTP And calculating the voice generation time using the time amount of the packet and the time amount of the calculated SID packet. The method of claim 10 or 14,
The generating step,
And storing the call text including at least one set of outgoing phone number, incoming phone number, total call time, outgoing text data and timestamp, and at least one set of incoming text data and timestamp. Way. A method for a device to text-convert a service subscriber's call voice and manage the converted text,
The service subscriber receiving voice data of a call to be a caller or a recipient;
Classifying the received voice data into outgoing voice data and received voice data, respectively;
Converting the classified voice data into data of outgoing text and received text, respectively;
Generating call texts by distinguishing the converted sender's calling text and recipient's receiving text and arranging them in chronological order; And
And in response to the request of the service subscriber, querying the generated call text and providing it to the terminal of the service subscriber,
Prior to the providing step above,
Set the starting position of the voice data of the outgoing or received voice data from the starting position of the voice data that has arrived first among the outgoing voice data and the received voice data by the time of the RTP packet of the first arrived voice data. And, further comprising the step of mixing the voice stream of each of the outgoing voice data and the received voice data into a single stream of integrated voice data using the set starting position,
The providing step is a method of providing the integrated voice data. The method of any one of claims 10, 14 or 17,
The providing step,
And providing the call text to a terminal of a service subscriber using at least one of a text message, an email, a social network service (SNS), and a web page.

Images