JP2007228114A - Telephone system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a telephone system wherein the voice is transmitted from the sending side and received on the receiving side, which displays the voice text on the receiving side in sync with the voice even if the voice is interrupted on the receiving side as well as highlighting the interrupted portion of the voice, and allows the receiving side to inform the sending side of the interruption of the voice and display that information on the receiving side during the talk between the sending side and the receiving side over the telephone system. <P>SOLUTION: The telephone system includes a means which turns the voice into a packet and turns the text information made by voice recognition of the voice into a packet on the sending side, adds identification information indicating synchronization between the voice and the text to these packets, and then sends these packets; and a means which receives the packets on the receiving side, and outputs the voice and the text information in sync based on the identification information when outputting the voice and text information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a telephone system that transmits voice on the transmission side and receives it on the reception side.

  Conventionally, a telephone system using IP technology has been devised so that voice (packet) is preferentially processed by a packet prioritization technique in a network so as not to interrupt the voice as much as possible.

  In addition, there is a technique in which a communication server is arranged on a network, voice (packets) from a telephone system is recognized in the server, converted into text, and delivered to a designated destination by mail (Patent Document 1).

Also, if the incoming call cannot be answered, the voice message from the caller is accumulated, recognized and converted into a text message, and the converted text message is sent when there is a message notification timing or user designation from the caller. There is a technique for displaying the image (Patent Document 2).
JP 2005-12391 A JP 2004-328525 A

  However, even if the voice (packet) is preferentially processed by the conventional packet prioritization technology, it is difficult to operate the priority control completely when the telephone system is fully IP, and the voice is interrupted. There is a problem that it may occur.

  For this reason, for example, on the transmitting side, it is not known that a situation has occurred in which the audio is not normally reproduced on the receiving side, so there is a need to continue the conversation as it is and to supplement the contents of the conversation later or to repeat the conversation from the middle. There was a problem that it would occur.

  Further, in the conventional technique of the former patent document, a server is provided on the network, the voice is recognized by the server and distributed by mail, and the text data recognized by the voice cannot be displayed in synchronization with the voice. There is a problem in that it is impossible to recognize that a situation has occurred in which the sound is not normally reproduced on the transmission side on the transmission side.

  Further, in the latter technique of the latter patent document, when the incoming call cannot be answered, the voice message from the caller is displayed as text data and cannot be displayed in synchronization with the voice. At the same time, there has been a problem that it is impossible to recognize that a situation has occurred in which the audio is not normally reproduced on the transmission side on the transmission side.

  In order to solve these problems, the present invention displays and synchronizes voice text even when a break occurs in the middle as speech on the receiving side when a call is made between the sending side and the receiving side in the telephone system. The highlighted part is highlighted, or the fact that there is a break from the reception side to the transmission side is transmitted and displayed.

  In the present invention, when a call is made between the transmission side and the reception side in the telephone system, even if a break occurs in the middle as a voice on the reception side, the voice text is synchronously displayed and the broken portion is highlighted, In addition, by sending and displaying the interruption from the receiving side to the transmitting side, even if there is a voice interruption in the IP phone, the voice text is displayed reliably and the missing part is highlighted and transmitted. It is possible to transmit and display the fact that the sound is interrupted to the side.

  In the present invention, when a call is made between the transmission side and the reception side in the telephone system, even if a break occurs in the middle as a voice on the reception side, the voice text is synchronously displayed and the broken portion is highlighted, It has been possible to send and display the fact that there is a break from the receiving side to the sending side.

FIG. 1 shows a system configuration diagram of the present invention.
FIG. 1A shows an overall system configuration diagram.

  In FIG. 1A, a terminal 1 is a terminal that makes a call such as a fixed-line telephone / mobile phone, and here, the terminal on the transmission side is the terminal A and the terminal on the reception side is the terminal B. Yes, for displaying text information that has been called and synchronized with each other.

  The accommodated base station 2 is connected to the network 4 and communicates with the terminal 1 via the subscriber line (wireless or wired) in a packet, and here, between the terminal A on the transmission side, The accommodating base station A is the one that makes a call, and the accommodating base station B is the one that makes a call with the terminal B on the receiving side. Between the terminal 1 and the accommodating base station 2, voice information and text information obtained by character recognition of the voice information and identification information are mutually transmitted and received during a call (see FIGS. 2 to 11).

  The server 3 is connected to the network 4 and performs call management (call connection, billing management, etc.) between the terminal A on the transmission side and the terminal B on the reception side.

  The network 4 is a communication path for communicating packets with each other. Here, the accommodation base station A, the accommodation base station B, a server, and the like are connected, and the terminals A and B communicate with each other (call and text information). Display).

  FIG. 1B shows an example packet. This shows an example of a packet transmitted from the terminal A to the terminal B via the network 4.

  In FIG. 1B, the transmission block 100 is configured such that the terminal A on the transmission side in FIG. 1A samples an analog voice signal uttered by the user and converts it into a digital voice signal. Including the voice information packet 300 set in the packet and the text signal by recognizing the voice signal as text information, and including the text information packet 400 set in the packet. To send to.

  The reception block 200 indicates a reception block received by the terminal B on the receiving side from the network 4. The voice information and the text information are respectively extracted from the reception block 200.

  FIG. 1C shows an example of voice collation information. This is because, with respect to the text information of FIG. 1B, since voice collation information 402 is set in the data portion in addition to the text information main body 401, it is taken out and combined with voice utterance as described later. The text information body 401 is displayed synchronously, or the missing portion in the text information is highlighted when a voice packet is missing (see FIGS. 2 to 11).

FIG. 2 shows an example of the terminal (sending side) of the present invention.
In FIG. 2, the call setting / management means 11 sets and manages a call with the network 4 (further, the terminal B on the receiving side). When this happens, the start information is sent to the input sampling means 12, and the call start information is embedded in the voice information (see (1) A in FIG. 5).

  The input sampling means 12 records the input sampling time of the voice signal from the user A, and embeds the call start information from the call setting / management means 11 in the voice signal (FIG. 5). (1) See A).

  The encoding processing means 13 generates a digital audio signal by sampling an audio signal (including call start information) in sampling time units (see (2) B in FIG. 5).

  The RTP information generating unit 14 generates packet data (RTP information) by bundling a plurality of digitally encoded audio information generated by the encoding processing unit 13 ((5) E in FIG. 6). reference).

  The voice information packetizing means 15 converts packet data (RTP information) into a packet. Then, the generated packet is sent to the network.

  The text processing means 16 recognizes speech as text information and adds voice collation information to the text information. The text processing means 16 includes a voice reading means 161, a text information converting means 162, and the like (see FIG. 5 (4) D).

The voice reading means 161 reads voice.
The text information converting means 162 records information indicating which input sampling time the audio signal is converted into text (see (4) D in FIG. 5).

  The time information collating unit 17 matches the input sampling time including the start information with the RTP timestamp position, and transmits time correspondence information to the text information packetizing unit 18 (see (4) D in FIG. 5). .

  The text information packetizing means 18 generates, for example, an IETF standard packet in order to maintain network compatibility and interconnection (see (6) F in FIG. 6).

  Next, the operation of the configuration of FIGS. 1 and 2 will be described in detail according to the order of the flowcharts of FIGS. 3 and 4.

  FIG. 3 is a flowchart for explaining the operation of the present invention (from call setting to call start). Here, the terminal A is the utterance side terminal A in FIGS. 1 and 2, the server 3 is the server 3 in FIGS. 1 and 2, and the terminal B is the reception side terminal B in FIGS. User A (transmission) represents number operation, utterance, and the like of user A on the transmission side. The audio information and text information of the terminal A represent processing such as encoding of the audio information and text information on the terminal A side. The voice information and text information of the terminal B represent processing such as display of voice information and text information on the terminal B side.

  In FIG. 3, user A (utterance) inputs a number in S1. In this case, the user A on the speaking side dials the telephone number of the terminal B on the receiving side.

  In S2, call processing is performed. In response to the dialing by user A in S1, this number is called.

  In S3, the server 3 recognizes the receiving terminal. This is because the server 3 recognizes the telephone number of the terminal B on the receiving side that is called in S2.

  In S4, call processing is performed. This calls to the telephone number of the terminal B on the receiving side recognized by the server 3 in S3.

In S5, the terminal B performs incoming processing.
S6 calls. In S5 and S6, the terminal B on the receiving side called from the server 3 in S4 performs an incoming call process and rings a bell for calling the user B.

  In S7, an incoming call is notified. Since the terminal B recognizes that it has received an incoming call in S5, it returns the incoming response to the server 3.

In step S8, call establishment monitoring is started, and the calling side is returned to the transmitting side.
In S9, a call-in-progress display (or a calling voice is made) is displayed on the terminal A on the transmission side.

  S11 goes off-hook. In response to the call of S6, the user B picks up the receiver of the terminal B on the receiver side.

In S12, a response is notified.
In S13, the server 3 starts charging in response to the response notification in S12.

In S14, the terminal B starts a call session.
In S15, the server 3 transmits a call establishment notification to the terminal A.

In S16, the terminal A starts a call session.
In S17, a call is started. As a result, the user A of the terminal A on the speaking side and the user B of the terminal B on the receiving side can mutually communicate and display text information in synchronization with the call.

Next, the synchronous display of a call and text information will be described in detail.
In FIG. 3, user A makes a call in S21.

In S22, speech encoding is performed.
In S23, voice packetization is performed. When the user A transmits to the handset of the terminal A on the transmitting side by S22 and S23, it is sampled and encoded, and further transmitted as a packet toward the terminal B on the receiving side via the network.

In S24, the transmitted voice is read.
In S25, it is converted into a voice text.

  In S26, a text packet is formed. When the user A transmits to the receiver of the terminal A on the transmitting side through S24 to S26, the transmitted voice is read and recognized, added to text and voice verification information, and further, these are packetized. Then, the data are sequentially transmitted to the terminal B on the receiving side via the network.

  When the user A transmits to the receiver of the terminal A on the transmitting side by the above S21 to S26, the speech encoding is performed to packetize the speech, and the speech is recognized and converted into text and speech verification information is added to be packetized. The packet can be transmitted toward the terminal B on the receiving side. At this time, the text information packet is transmitted when the voice is smaller than a predetermined threshold or when there is no voice.

S27 is initialized.
In S28, buffer processing is performed.

In S29, voice decoding is performed.
S30 receives a call. These S28, S29, and S30 are delayed for a predetermined time after the terminal B receives the packet obtained by packetizing the transmission of the user A on the transmitting side in S23 and temporarily stores the received packet in the buffer. Then, the corresponding packet is taken out from the buffer in order of time, decoded to be the original voice signal, returned to the original voice by the receiver, and output to the user B.

In S31, a synchronization process is performed.
In S32, display processing is performed.

  S33 displays text. These S31, S32, and S33 are based on the voice collation information in the packet when the terminal B receives a packet obtained by packetizing the text information obtained by character recognition of the transmission of the user A on the transmission side in S26. And the text information is displayed on the display screen of the terminal B in synchronization with the voice (see FIGS. 7 to 11).

  Similarly, as described as “repeat”, by repeating S21 to S33, the voice and the text information recognizing the voice and the voice collation information are packetized from the transmitting terminal A to the receiving terminal B. It is possible to transmit and receive and display the utterance and text information in synchronization (see FIGS. 7 to 11).

FIG. 4 is a flowchart for explaining the operation of the present invention (from a call to a call release).
In FIG. 4, S41 ends the call.

  In S42, user B (received call) goes off-hook. In this case, the user B puts the handset of the terminal B and ends the call.

In S43, the terminal B sends a call session end notification to the server 3.
In S44, the server 3 finishes charging.

In S45, the server 3 notifies the terminal A on the transmission side of the charging notification.
In S46, the terminal A ends the call session.

In S47, the terminal A ends the encoding of the voice.
In S48, the terminal A ends text conversion.

In S49, the server 3 releases the call.
In S50, the terminal B ends the text information display process.

In S51, terminal B finishes decoding the voice packet.
As described above, the call termination process can be performed.

  5 and 6 are explanatory diagrams of the present invention. Here, the information forms at the points A to F in FIG. 2 are shown below. The audio is sampled in units of 125 μs, and the RTP interval is set to 20 ms.

  (1) A in FIG. 5 shows an example of a signal input from the input sampling means 12 in FIG. 2 to the encoding processing means 13. Here, the start information from the call setting / management means 11 is inserted into the illustrated position in the voice signal transmitted by the user A. The sampling time is 125 μs. This makes it possible to synchronize the voice (digital voice sampled at intervals of 125 μs) with the text information obtained by character recognition of the voice with reference to the position of the start information.

  (2) B in FIG. 5 shows an example of a signal input to the RTP information generating unit 14 from the encoding processing unit 13 in FIG. Here, as shown in the figure, for each sampling time (125 μs interval), information is generated by combining the sampling time and the sampled audio signal (digital value).

  (3) C in FIG. 5 shows an example of a signal output from the time information matching unit 17 in FIG. 2 to the text information packetizing unit 18. Here, the input sampling time is associated with the RTP time stamp (ms) (including start information). By checking the position of the start information, the input sampling time and the RTP time stamp can be associated. The RTP time stamp takes a value at every packet generation interval (20 ms in the example in the figure).

  (4) D in FIG. 5 shows an example of a signal input from the text processing unit 16 in FIG. 2 to the text information packetizing unit 18. Here, the input sampling time, the text information sequence number, and the text information contents are associated with each other. As for text information, the position of the start information and the input sampling time at the beginning of the text are recorded, and it is determined which RTP the text information corresponds to.

  (5) E in FIG. 6 shows an example of a signal input to the voice information packetizing means 15 from the RTP information generating means 14 in FIG. Here, the RTP information is generated with a standard specification (the collation information with the text information is not described here). This is to maintain communication compatibility with terminals that cannot display text.

  (6) F in FIG. 6 shows an example of a signal output from the text information packetizing means 7 in FIG. Here, in the text information header (header portion), a sequence number indicating the order of text conversion and corresponding RTP time stamp information are described. As described above with reference to FIG. 1C, voice collation information (here, sequence number, RTP time stamp) and text information main body are described in the data portion.

Next, the configuration and operation of the terminal B on the receiving side will be described in detail with reference to FIGS.
FIG. 7 shows an example of the terminal (receiving side) of the present invention.

  In FIG. 7, the call setting / management means 21 sets and manages a call, and here, instructs to start reception (at the start of a call session).

  The voice information reception processing unit 22 receives a voice information packet. Here, the voice information reception processing unit 22 reads time stamp information from the voice information of the voice information packet and passes it to the synchronization processing unit 26.

  The buffer 23 is for temporarily storing the received voice packet and taking out a packet that has been adjusted and synchronized for a predetermined time based on the notification of the adjustment time from the synchronization processing means 26.

The decoding processing means 24 decodes the voice packet and returns digital voice.
The text information reception processing unit 25 receives a text information packet. Here, the text information reception processing unit 25 extracts the time stamp range of the text information voice information packet as the voice collation information in the text information, and synchronizes the processing unit 26. Etc.

  The synchronization processing means 26 is a time stamp range when the time stamp information of the voice packet passed from the voice information reception processing means 22 and the voice of the voice packet passed from the text information reception processing means 25 are packetized. On the basis of this information, the depth of the buffer 23 is adjusted so as to synchronize with the text information, and the voice packet is synchronized (see FIGS. 8 to 11).

The display processing means 27 displays text information (see FIGS. 8 to 11).
FIG. 8 shows a configuration example of the reception block 200 of the present invention. As described above with reference to FIGS. 1B and 1C, the reception block 200 is a mixture of a voice information packet and a text information packet. The information packet has the arrival time difference t1 shown in the figure.

FIG. 9 is an explanatory diagram of the present invention (arrival time difference, etc.).
In FIG. 9, when the buffer depth of the buffer 23 is tB (time), the specified voice information interval is tc (time), and the deviation of the voice packet from the specified voice information interval is Δt (time), the voice information packet T2 from when the signal is input to the buffer 23 until it is output,
t2 = tB−Δt
It becomes. Then, the voice packet having the irregular t2 time is temporarily stored in the buffer 23, and the voice packet having a constant tc is output from the output to the decoding processing means 24 of FIG. Is possible. Also, synchronization is performed by adjusting the depth of the buffer so that the arrival time difference t1 already described in FIG. 8 is almost equal to tB, and the voice information packet and the text information packet are decoded. It is possible to synchronize with the converted text information.

FIG. 10 shows a display example of the present invention (text display example when voice information is missing).
FIG. 10A shows an example of the original sound. Here, an example of “Hyakumanen” (million yen) is shown.

  FIG. 10B shows an example of the transmission side output voice packet. Here, the voice packet is output without being lost.

  FIG. 10C shows an example of a voice information packet that has arrived at the receiving side. Here, it is assumed that a black mark is missing. As shown in the figure, the missing packet is displayed with a black mark to inform the user B.

  FIG. 10 (d) shows an example of the reproduced sound. As for the reproduced sound, since the packets in the voice group “MA” and “N” are missing from the voice information packet in FIG. 10C, the sounds “MA” and “N” cannot be played back. Therefore, there is a possibility that the situation will be reproduced as “Hyaku--en” and mistakenly recognized as “100 yen”. At this time, if a voice information packet is missing, the missing voice part (sequence number, time stamp) is notified to the transmitting side, and the missing information is also displayed on the transmitting side as shown in FIG. The packet portion may be highlighted.

  FIG. 10E shows an example of text display. This shows an example in which text information “Hyakumanen” obtained by character recognition of the original voice information of the transmission side output voice information packet of FIG. 10B is received and displayed on the receiving side.

FIG. 11 shows a display example of the present invention.
FIG. 11A shows an example of a packet. This shows an example of a packet received at the receiving side. Highlight missing packets.

  FIG. 11B shows an example of a voice information packet. This shows an example in which only the voice information packet is extracted from the received packet of FIG. 11A and the missing packet is highlighted.

  FIG. 11C shows an example of a text information packet. This shows an example in which only the text information packet is extracted from the received packet in FIG. 11A and the missing packet is highlighted.

  In the present invention, when a call is made between the transmission side and the reception side in the telephone system, even if a break occurs in the middle as a voice on the reception side, the voice text is synchronously displayed and the broken portion is highlighted, Furthermore, a message indicating that the interruption has occurred is transmitted and displayed from the receiving side to the transmitting side, and even if there is a voice interruption in the IP phone, the voice text is displayed in a synchronized manner, the missing part is highlighted, and the transmitting side is highlighted. The present invention relates to a telephone system that transmits, displays, and notifies that a voice is interrupted.

It is a system configuration diagram of the present invention. It is a terminal (sending side) of the present invention. It is an operation | movement explanatory flowchart (from call setting to call start) of this invention. It is an operation | movement explanatory flowchart (from a telephone call to call release) of this invention. It is explanatory drawing (the 1) of this invention. It is explanatory drawing (the 2) of this invention. It is a terminal (receiving side) of the present invention. It is a structural example of the receiving block 200 of this invention. It is explanatory drawing (arrival time difference etc.) of this invention. It is a display example (text display example at the time of audio | voice information missing) of this invention. It is a display example of the present invention.

Explanation of symbols

1: Terminal 2: Accommodating base station 3: Server 4: Network 11, 21: Call setting / management means 12: Input sampling means 13: Encoding processing means 14: RTP information generating means 15: Voice information packetizing means 16: Text Processing means 161: speech reading means 162: text information converting means 17: time information collating means 18: text information packetizing means 22: speech information receiving processing means 23: buffer 24: decoding processing means 25: text information receiving processing means 26 : Synchronization processing means 27: Display processing means

Claims (7)

In a telephone system that sends audio on the sending side and receives it on the receiving side,
A means for transmitting voice by packetizing text information obtained by voice recognition of the voice on the transmission side, and adding identification information indicating synchronization between the two,
A telephone system comprising: means for synchronously outputting the voice and the text information based on the identification information when the receiving side receives the packet and outputs the voice and text information.   2. The telephone system according to claim 1, wherein when the voice level is lower than a predetermined threshold or when there is no voice on the transmission side, the identification information is added to the packet and transmitted.   A buffer that, as the identification information, adds a sequence number of a voice packet to be synchronized to the text information on the transmission side, and temporarily stores a voice packet based on the sequence number in the text information received on the reception side 3. The telephone system according to claim 1 or 2, wherein the voice is output in synchronization with the depth of the sound being adjusted.   As the identification information, a buffer that adds a time stamp of a voice packet to be synchronized to the text information on the transmission side and temporarily stores the voice packet based on the time stamp in the text information received on the reception side 3. The telephone system according to claim 1 or 2, wherein the voice is output in synchronization with the depth of the sound being adjusted.   5. The voice data received on the receiving side is highlighted when a missing or unacceptable packet is received within a predetermined time. Phone system as described in.   6. The text information packet received on the receiving side is highlighted when the packet is missing or cannot be received within a predetermined time. The phone system described in Crab.   When a voice packet received on the receiving side is lost or cannot be received within a predetermined time, the fact that the voice packet was not received can be output to the sending side by sending a message to that effect. The telephone system according to any one of claims 1 to 6, wherein the telephone system is characterized.

Images