KR101089444B1 - System and method for emotion expression of soft phone user - Google Patents

Abstract

The present invention relates to a softphone user emotional expression system and method.

In the softphone user emotion expression system and method according to the present invention, when the emotion expression apparatus requests a voice file of the current softphone user as a recorder, the voice recorder collects an RTP packet transmitted from the softphone to the voice file of the current softphone user. The emotion expression apparatus extracts text from the voice file transmitted from the recorder, and retrieves the emotion state corresponding to the text from the emotion expression database to generate emotion state information, and the softphone The emotion state of the current softphone user is displayed according to the emotion state information transmitted from the presentation device.

Description

Softphone user emotion expression system and method {System and method for emotion expression of soft phone user}

The present invention relates to a system and method for expressing a soft phone user emotion, and more particularly, to extract text from a voice file generated by collecting a Session Initiation Protocol (RTP) packet generated during a soft phone call. The present invention relates to a system and method for expressing an emotion of a softphone user by displaying an emotional state corresponding to text on a softphone.

Internet phone service is a phone call service that delivers voice, video, fax, etc. over the Internet using Voice over Internet Protocol (VoIP). The Internet telephony service is provided by a user, for example, by accessing the Internet from a personal computer (PC), using an independent device with an Internet protocol, or by dialing a gateway from an existing public switched telephone network (PSTN) terminal. Can be done. Signaling means for VoIP include H.323 of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and Session Initiation Protocol (SIP) of the Internet Engineering Task Force (IETF). Although many VoIP services have been developed, recently, SIP, which is easy to parse and compile, has excellent scalability, and is text-based, has been widely used.

On the other hand, in connection with the Internet telephony service, a soft phone is a software-based internet phone installed in a PC, a personal digital assistant (PDA) or the like. In addition to the ability to provide voice and / or video call service by accessing the Internet through VoIP, V2oIP, etc., it can provide various application functions through application software.

One of them is a function of displaying an emotional state of a softphone user. In the related art, a user (phone caller) enters a text indicating his or her emotional state or selects an emoticon during a call and selects an emoticon. In this case, a technology for displaying the emotion state of a softphone user by displaying the corresponding text or emoticon on the screen by the counterpart phone receiving the same has been introduced.

However, the emotion expression technology of the softphone user according to the related art is not a method of naturally catching and expressing the emotion state of the current user, but a method in which the user artificially selects text and / or emoticons and transmits them to the counterpart phone.

Therefore, since the user arbitrarily sets the emotional state and transmits it, the emotional state of the user cannot be displayed according to the objective call contents, and there is an inconvenience of inputting text or setting an emoticon during the call.

The present invention was made to solve the above-mentioned problems and an object of the present invention is to provide a softphone user emotion expression system and method that can display the current emotional state of the softphone user based on the natural content of the call during the Internet phone call To provide.

Another object of the present invention is to generate a voice file of a current softphone user by collecting RTP packets during an internet phone call, extracting text from the voice file, and expressing a corresponding emotional state on the softphone in real time. It is to provide a system and method.

Another object of the present invention is to extend the function of displaying only the physical state information such as login, logout, away, meeting, etc. in the existing softphone software that can display the current emotional state of the softphone user in real time It is to provide a phone user emotion expression system and method.

To this end, the softphone user emotion expression system of one embodiment of the present invention includes: a recorder for collecting a RTP packet transmitted from a softphone to generate a voice file of a current softphone user; An emotion expression apparatus for extracting text from the voice file transmitted by the recorder, and generating emotion state information by searching for an emotion state corresponding to the extracted text in an emotion expression database; And a soft phone displaying an emotion state of the current soft phone user in correspondence with the emotion state information transmitted from the emotion expression apparatus.

In addition, an emotion expression apparatus for expressing an emotion of a softphone user according to one embodiment of the present invention includes: an emotion expression storage unit for matching text representing an emotion with an emotion state and storing the text in an emotion expression DB (Database) form; An emotional state analyzer configured to extract text from the voice file transmitted from the recorder, and to generate emotional state information by searching for an emotional state corresponding to the extracted text in an emotional expression database; And a SIP processing unit for transmitting the MESSAGE message including the emotional state information to the corresponding soft phone.

On the other hand, a softphone user emotion expression method of one embodiment of the present invention includes the steps of: a) requesting a voice file of a current softphone user by recording the emotion expression device; b) receiving, by the emotion expression apparatus, a voice file of a current softphone user generated by collecting a Real-time Transport Protocol (RTP) packet transmitted from the recorder by the softphone; c) the emotion expression apparatus extracting text from the voice file transmitted from the recorder, and searching the emotion state corresponding to the extracted text in an emotion expression database to generate emotion state information ; And d) transmitting the emotion state information to the softphone by the emotion expression apparatus.

According to the present invention, it is possible to display an emotion state of a current softphone user based on a natural call content during an internet phone call, out of the level of expressing emotions by artificially setting texts or emoticons.

In addition, according to the present invention, by extending the function of the existing softphone to display only the physical state information such as login, logout, away, meeting, etc. to display the current emotional state of the user in real time state information Has the effect of diversifying the display.

In addition, according to the present invention, by displaying the current emotional state of the softphone user in real time during the Internet phone call has the effect of stimulating the user to activate the communication.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings and preferred embodiments. For reference, in the following description, detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention are omitted.

First, the terms used in the present invention will be described before explaining the configuration of the present invention.

Session Initiation Protocol (SIP) is an application layer signaling protocol that specifies the procedures for intelligent terminals that want to communicate on the Internet to identify each other, locate their locations, and create, delete, or change multimedia communication sessions between them. to be. This is a simpler protocol that corresponds to H.323 in ITU-T. SIP-based messages are text-based and consist of a request message and a response message model between SIP elements. For reference, Table 1 below illustrates types of SIP-based request messages according to the Session Description Protocol (SDP), and Table 2 below illustrates types of SIP-based response messages according to the SDP.

[Table 1] SIP based request message

Table 2 SIP-Based Response Messages

Real-time Transport Protocol (RTP) is a transport layer protocol for transmitting and receiving voice and video in real time, and is defined together with RTCP (RTP control protocol) for a video conferencing system in a LAN / Internet environment. Unlike RSVP (Resource Reservation Protocol), this is a feature that is executed between terminals without relying on a network device such as a router, and is usually used as a higher communication protocol of UDP (User Datagram Protocol). With RTP, the sender can abandon packets with large delays based on time stamps, and the receiver checks transmission delays, bandwidth, etc., and uses RTCP to send higher layer applications to the sender. The quality of service (QoS) can be controlled by adjusting a coding rate.

Hereinafter, a softphone user emotion expression system and method according to a preferred embodiment of the present invention will be described with reference to FIGS. 1 to 7.

First, Figure 1 is a schematic diagram of a softphone user emotion expression system according to the present invention.

Referring to FIG. 1, a plurality of soft phones 100 are interconnected via a network to provide an Internet telephony service. The network also includes a recorder 200 and an emotion expressing device 300 according to the present invention. Emotional expression for the softphone user is performed in real time.

2 is a block diagram of the softphone 100, the recorder 200, and the emotion expression apparatus 300 according to an exemplary embodiment of the present invention, and shows a mutual relationship.

2, first, the soft phone 100 includes an RTP processing unit 110, a SIP processing unit 120, an emotional state display unit 130, and the like.

The RTP processing unit 110 converts the voice of the softphone user into an RTP packet using a Real-time Transport Protocol (RTP), and transmits the voice of the softphone user to the counterpart phone (not shown) through the network. Similarly, the RTP packet received from the other party's phone is received and converted into voice.

The SIP processing unit 120 supports a SIP-based VoIP service in conjunction with a SIP server (not shown). For example, the SIP processing unit 120 sets up the session by transmitting an INVITE message to the counterpart phone and requesting session establishment. When the SIP processor 120 receives a 200 OK message from the counterpart phone, the SIP processor 120 transmits an ACK message to the counterpart phone. When the SIP processing unit 120 receives a BYE message for requesting call termination from the other party's phone, the SIP processing unit 120 transmits a 200 OK message for confirming the call and terminates the call. In addition, in connection with the present invention, the SIP processing unit 120 processes the MESSAGE message including the emotional state information transmitted from the emotion expression apparatus 300 and transmits the emotional state information to the emotional state display unit 130.

In addition, the emotional state display unit 130 displays the emotional state of the current user according to the emotional state information transmitted from the SIP processing unit 120. For example, the emotional state display unit 130 displays text and / or emoticons corresponding to the emotional state information on the screen of the softphone. In this case, the emotional state display unit 130 may be implemented to display the emotional state of the sender and the receiver together on the screen according to the user's setting, or to display the emotional state of the corresponding sender or the receiver in real time when there is a change in the emotional state. have.

Referring to FIG. 2, the recorder 200 according to the present invention includes an RTP processor 210, an emotion expression device interworking unit 220, and the like.

The RTP processing unit 210 collects RTP packets sent from the softphone 100 according to a request of the emotion expression apparatus 300, and generates a voice file using the collected RTP packets using RTP. For example, the RTP processing unit 210 collects the RTP packets sent in real time from the soft phone 100, and arranges the collected RTP packets in order according to a sequence number to generate a voice file.

The emotion expression device linkage unit 220 performs a function of interworking with the emotion expression device 300 (specifically, the recorder linkage unit 320). For example, when the emotion expression device linkage unit 220 receives a voice file request of the current softphone user from the emotion expression device 300, the emotion expression device interworking unit 220 notifies the RTP processing unit 210 of the current voice file request and reports the user voice file generated by the RTP processing unit 210. Send to the emotion expression device 300.

Similarly, referring to FIG. 2, the emotion expression apparatus 300 according to the present invention includes a recorder linkage unit 310, an emotion expression storage unit 320, an emotion state analyzer 330, a SIP processor 340, and the like. Include.

The recorder linkage unit 310 performs a function of interlocking with the recorder 200 (specifically, the emotion expression device linkage unit 220). For example, the recorder interlocking unit 310 requests the voice file of the current softphone user from the recorder 200 to express the emotion of the current softphone user, and the voice file of the current softphone user from the recorder 200. Receives and transmits to the emotional state analysis unit 330.

The emotion expression storage unit 320 matches text related to the emotion expression with an emotion state and stores the text in the form of an emotion expression DB. 3 illustrates a structure of an emotional expression database (Database) according to an embodiment of the present invention. As shown, the emotional expression DB sets various emotional states and selects texts associated with each emotional state. Save it. In this case, the text may be stored in at least one of sentences and words, and the emotion expression storage unit 320 may inform a corresponding emotion state when the text is provided from the emotion state analyzer 330.

The emotion state analyzer 330 analyzes the voice of the current softphone user, extracts text, and retrieves emotion state information corresponding to the text from the emotion expression storage 320. For example, the emotion state analyzer 330 extracts text from a voice file of the current softphone user transmitted through the recorder interlocking unit 310 using technology such as Speech To Text (STT), and the emotion expression storage unit. In operation 320, an emotional state corresponding to the extracted text is searched to generate emotional state information.

As described above, the SIP processing unit 340 includes the emotional state information generated by the emotional state analyzing unit 330 in the SIP-based MESSAGE message and transmits it to the soft phone 100, and responds to the soft phone 100. Receive 200 OK message to confirm the normal reception of the emotional state information.

For reference, the above-described emotion expression apparatus 300 according to the present invention may be implemented with an EAS (Emotion Pattern Analysis System).

Hereinafter, a method of expressing softphone user emotions according to a preferred embodiment of the present invention will be described with reference to FIGS. 4 to 7. For reference, the softphone user emotion expression method according to the present embodiment is based on the above-described softphone user emotion expression system, and the lacking in the above description may refer to the following description.

4 is a flowchart illustrating a method for expressing softphone user emotions according to an embodiment of the present invention, which schematically illustrates a message flow between a softphone, a recorder, and an emotion expressing device.

Referring to FIG. 4, in operation S410, the emotion expression apparatus 300 requests the voice file of the current softphone user to the recorder 200 to express the emotional state of the current softphone user. Then, in step S420, the recorder 200 collects the RTP packets sent from the softphone 100 on the network, and arranges them in order according to the sequence number to generate a voice file. In operation S430, the recorder 200 transmits the voice file of the current softphone user generated to the emotion expression apparatus 300.

On the other hand, in step S440, the emotion expression apparatus 300 extracts the text from the voice file of the current softphone user transmitted from the recorder 200 by using technology such as Speech To Text (STT), and extracted text The emotional state corresponding to the current state of the softphone user is determined by searching the emotional expression DB. In operation S450, the emotion expression apparatus 300 includes the emotion state information according to the emotion state of the current softphone user in the SIP-based MESSAGE message and transmits the information to the soft phone 100. In this case, the emotion expression apparatus 300 preferably transmits identification information (eg, a phone number) together with the softphone 100 to identify the current softphone user.

Then, in step S460, the softphone 100 transmits a 200 OK message in response to the MESSAGE message to the emotion expression apparatus 300 to notify the normal reception, and in step S470, the emotion state of the user according to the emotion state information (for example, , Emoticons, text, etc.) are displayed on the soft phone screen. For reference, step S470 may be performed before step S470 or may be performed simultaneously. In this case, the softphone 100 may be a caller softphone and / or a receiver softphone, in which case the softphone 100 may check the emotional state of the caller and the receiver (by checking the identification information of the current softphone user). When the screen is displayed together or when the emotional state changes, the emotional state of the corresponding sender or receiver may be displayed in real time.

FIG. 5 is a flowchart illustrating an operation of the softphone 100 according to an exemplary embodiment of the present invention with reference to FIG. 4.

Referring to FIG. 5, in step S510, when a user enters a phone number into the soft phone 100 to make a call to the other party, in step S520, the soft phone 100 connects a call with the counterpart phone in association with a SIP server. (That is, connect an Internet phone). Thereafter, the soft phone 100 performs a voice call with the other party phone using the RTP.

On the other hand, while the phone call is in progress, the emotion expression apparatus 300 interlocks with the recorder 200 to generate emotional state information for the current softphone user, and the softphone 100 includes MESSAGE including emotional state information. Send a message.

Then, in step S530, the soft phone 100 checks whether the MESSAGE message including the emotion state information is received from the emotion expression apparatus 300, and if the emotion state information is normally received, in step S540, Accordingly, the corresponding emotional state is displayed on the soft phone screen.

6 is a flowchart illustrating an operation of a recorder 200 according to an exemplary embodiment of the present invention with reference to FIG. 4.

Referring to FIG. 6, in step S610, the emotion expression apparatus 300 requests the voice file of the current user to the recorder 200 to express the emotion state of the current softphone user, and the recorder 200 does this. Receive.

Then, in step S620, the recorder 200 collects the RTP packets sent from the softphone 100 in response to the request of the emotion expression apparatus 300, and sorts the collected RTP packets in order according to the sequence number. Create a file. Thereafter, in step S630, the recorder 200 transmits the voice file of the current softphone user to the emotion expression apparatus 300.

FIG. 7 is a flowchart illustrating an operation of the apparatus for expressing emotion 300 according to an embodiment of the present invention with reference to FIG. 4.

Referring to FIG. 7, in operation S710, the emotion expression apparatus 300 requests the voice file of the current user to the recorder 200 in order to express the emotion state of the current softphone user. In operation S720, the emotion expression apparatus 300 receives a voice file of the current softphone user from the recorder 200. In operation S730, the emotion expression apparatus 300 analyzes the voice file of the current softphone user. To determine the current emotional state. For example, the emotion expression apparatus 300 extracts text from a voice file transmitted from the recorder 200 using a technique such as Speech To Text (STT), and extracts an emotional state corresponding to the extracted text. Emotion state information is generated by searching in the emotion expression database (Database).

In step S740, it is determined whether there is a change in the emotional state of the current softphone user. If there is a change in the emotional state, the flow proceeds to step S750, and the MESSAGE message including the emotional state information is sent to the softphone 100. send. In this case, the emotion expression apparatus 300 also transmits identification information (eg, a phone number) for identifying the current softphone user. Then, in step S760, the softphone 100 transmits a 200 OK message in response to the MESSAGE message to the emotion expression apparatus 300, the emotion expression apparatus 300 confirms the reception of the 200 OK message to the corresponding softphone Confirm that the emotional state information has been received normally. On the other hand, if there is no change in the emotional state in step S740, of course, even in this case may proceed to step S750 and transmit the emotional state information, but this is the same as the previous emotional state, so proceed to S710 as shown in step S710 to Repeat the process S740.

Although the present invention has been described in detail with reference to preferred embodiments, it will be apparent to those skilled in the art that the present invention may be embodied in other specific various forms without changing the technical spirit or essential features of the present invention. One embodiment is to be understood in all respects as illustrative and not restrictive.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. .

1 is a schematic diagram of a softphone user emotion expression system according to an embodiment of the present invention.

2 is a block diagram of a soft phone, a recorder, and an emotion expression apparatus according to an embodiment of the present invention.

3 illustrates a structure of an emotional expression DB according to an embodiment of the present invention.

4 is a flowchart illustrating a method of expressing softphone user emotions according to an embodiment of the present invention, and illustrates a message flow between a softphone, a recorder, and an emotion expressing device.

5 is a flowchart illustrating an operation of a soft phone according to an embodiment of the present invention.

6 is an operation flowchart of a recorder according to an embodiment of the present invention.

7 is a flowchart illustrating an operation of an emotion expression apparatus according to an embodiment of the present invention.

Claims (20)

Softphone user emotion expression system, A recorder which collects RTP packets transmitted from the softphone and generates a voice file of the current softphone user; An emotion expression apparatus for extracting text from the voice file transmitted by the recorder, and generating emotion state information by searching for an emotion state corresponding to the extracted text in an emotion expression database; And And a soft phone displaying an emotion state of a current soft phone user in correspondence with the emotion state information transmitted from the emotion expression apparatus. The recorder collects the RTP packets transmitted from the softphone and arranges them in sequence according to sequence numbers to generate a voice file, and an emotion expression device for transmitting the voice file generated by the RTP processor to the emotion expression apparatus. Including the linkage, The emotion expression apparatus may include a recorder linkage unit that receives a voice file of a current softphone user from the recorder, an emotion expression storage unit that matches an emotion state text with an emotion state, and stores it in the form of an emotion expression DB (Database); Extracts text from the voice file transmitted by the recorder using Speech To Text (STT) technology, and generates emotional state information by searching for an emotional state corresponding to the extracted text in an emotional expression database (DB). An emotional state analyzing unit, and a SIP processing unit transmitting a MESSAGE message including the emotional state information to the softphone; The emotion expression DB is a soft phone user emotion expression system, characterized in that for storing the text to express the emotion in at least one form of each sentence and word. delete delete The method of claim 1, And the emotion expression apparatus confirms the normal reception of the emotion state information by receiving a 200 OK message in response to the MESSAGE message from the soft phone. delete The method according to claim 1 or 4, wherein the softphone An RTP processing unit for converting a voice of a softphone user into an RTP packet; A SIP processor which receives a MESSAGE message including emotion state information from the emotion expression apparatus; And And an emotion state display unit that displays an emotion state of a current softphone user in correspondence with the emotion state information. delete An emotion expression apparatus for expressing an emotion of a softphone user, A recorder interlocking unit for receiving a voice file of a current softphone user from a recorder which collects RTP packets transmitted from an Internet phone and arranges them in sequence according to a sequence number to generate a voice file; An emotion expression storage unit matching text representing emotions with an emotion state and storing the text in an emotion expression database form; Extracts text from the voice file transmitted by the recorder using Speech To Text (STT) technology, and retrieves the emotional state corresponding to the extracted text from the emotion expression database to retrieve the emotional state information. Generating an emotional state analyzer; And Including a SIP processing unit for transmitting the MESSAGE message containing the emotional state information to the softphone, The emotion expression DB is an emotion expression apparatus, characterized in that for storing the text to express the emotion in at least one form of each sentence and word. delete The method of claim 8, And the SIP processing unit confirms the normal reception of the emotional state information by receiving a 200 OK message in response to the MESSAGE message from the soft phone. delete delete As a softphone user emotion expression method, Requesting the voice file of the current softphone user by recording the emotion expression apparatus; Receiving, by the emotion expression apparatus, a voice file of a current softphone user generated by collecting Real-time Transport Protocol (RTP) packets transmitted from the recorder by the recorder and sorting the packets according to sequence numbers; The emotion expression apparatus extracts text from the voice file transmitted from the recorder using speech to text (STT) technology, and retrieves the emotion state corresponding to the extracted text from the emotion expression database (Database). Generating emotional state information; The emotion expression apparatus including the emotion state information in a MESSAGE message and transmitting the same to the softphone; And Displaying, by the softphone, an emotional state of a current softphone user corresponding to the emotional state information; The emotion expression DB is a soft phone user emotion expression method characterized in that for storing the text to express the emotion in at least one form of each sentence and word. delete delete delete The method of claim 13, And receiving, by the emotion expression apparatus, a 200 OK message in response to the MESSAGE message to confirm normal reception of the emotion state information. delete delete The method according to claim 13 or 17, In the transmitting, softphone user emotion expression method characterized in that the identification information of the current softphone user is transmitted to the softphone with the emotion state information.

Images