WO2020045712A1

WO2020045712A1 - Device, method and computer-readable recording medium for providing asynchronous instant messaging service

Info

Publication number: WO2020045712A1
Application number: PCT/KR2018/010172
Authority: WO
Inventors: 장준수; 윤용기; 장재웅; 김세미; 신희욱; 김영상; 임중신; 정정화
Original assignee: 주식회사 닫닫닫
Priority date: 2018-08-27
Filing date: 2018-08-31
Publication date: 2020-03-05
Also published as: KR20200023814A

Abstract

A method for providing an instant messaging service is provided. One exemplary method can be performed under the control of a computing device of a first user. The method can comprise the steps of: receiving one or more voice messages, which comprise a voice message transmitted by means of a computing device of a second user, and one or more voice-recognized text messages respectively corresponding to the one or more voice messages; receiving a voice message play request from the first user; and, in response to the voice message play request, playing one or more voice messages which have been received sequentially.

Description

Apparatus, method and computer readable storage medium for providing asynchronous instant message service

The present disclosure relates to an apparatus, a method and a computer readable storage medium for providing an asynchronous instant message service.

Unless stated otherwise herein, the contents described in this section are not prior art to the claims in this application and should not be admitted to be prior art for the purposes described in this section.

A user using an instant message service can transfer messages between two or more users relatively quickly and simply. Recently, as mobile devices such as smart phones are widely used, the use of instant messaging services has exploded. In recent years, in addition to the conventional text message, the instant message service enables the transmission of a relatively short voice message. The voice message is easier to input than the text message, and may transmit various features that the user who inputs the voice message wants to deliver. However, the data size of the voice message is generally larger than that of the text message, and the user must perform an operation of playing each voice message (eg, clicking, touching, etc.) the voice message, and listening to the voice message being played. In that sense, voice messages may have spatial constraints, such as temporal constraints or memory space or physical space, compared to text messages that can be quickly identified by eye.

Republic of Korea Patent Publication No. 10-1863776 (hereinafter referred to as Prior Art Document 1), when a user inputs a voice message, performs a voice recognition from the voice message to generate a text message, and extracts the user's emotion from the voice message A text representation method of changing and outputting a font of a text message generated from a voice message is disclosed.

As described above, the prior art document 1 extracts information on various emotions from a voice message and generates a text message using the information, but only a part of information that a user wants to deliver through the voice message can be obtained. The rest of the information may be lost.

SUMMARY The present disclosure is directed to solving the above problems, and provides an apparatus, method, and computer readable storage medium that are convenient for playing a voice message in an instant message service and are efficient in data management. In addition, the present disclosure provides an apparatus, method, and computer readable storage medium capable of providing an improved instant message service utilizing characters in an instant message service.

In some embodiments of the present disclosure, a method of providing an instant message service performed under the control of a computing device of a first user is described. One example method includes receiving one or more voice messages including a voice message sent by a computing device of a second user and one or more voice recognized text messages corresponding to each of the one or more voice messages; Receiving a request for playing a voice message from a first user; And responsive to the voice message reproducing request, reproducing one or more voice messages sequentially received. In some examples, the method may include storing each of the one or more voice recognized text messages corresponding to the one or more voice messages. In some examples, the method may further include displaying, in response to the playing of the one or more voice messages, the one or more voice recognized text messages.

In some additional examples, the exemplary method may further include receiving character information for the character of the second user. In this example, sequentially playing the received one or more voice messages may include playing back the voice message along with displaying the character of the second user based on the character information.

In some embodiments, an instant message service providing apparatus is described. An exemplary instant message service providing apparatus may include a communication module, a user interface module, a voice playback module, a display module, and a memory module. The communication module may be configured to receive one or more voice messages including voice messages sent by the sender's external computing device and one or more voice recognized text messages corresponding to the one or more voice messages, respectively. The user interface module may be configured to receive input for an instant message service. The user interface module may receive a voice message playing request from a user. The voice playback module may be configured to sequentially reproduce one or more voice messages received by the communication module in response to the voice message playback request received by the user interface module. The display module may be configured to sequentially display one or more voice recognized text messages in response to the playback of the one or more voice messages by the voice playback module. The memory module may be configured to store one or more voice messages and one or more voice recognized text messages.

In some embodiments, the user interface module can be configured to receive an input voice message from the user. In this embodiment, the instant message service providing apparatus may further include a voice recognition module and a text recognition module. The voice recognition module may be configured to perform voice recognition on the input voice message to obtain a voice recognized input text message. The character recognition module may be configured to detect an operation character that enables selecting an operation of a character of a user displayed by the display module, from the voice recognized input text message obtained by the speech recognition module. In a further example, the instant message service providing apparatus may further include a camera module and an expression determining module. The camera module may be configured to obtain face information of the user. The facial expression determination module may be configured to determine the facial expression of the character of the user based on the face information.

In some embodiments, a computer readable storage medium having stored thereon a computer program for providing an instant message service is described. The computer program stored in the computer readable storage medium, when executed, causes the first user's computing device to correspond to the one or more voice messages and the one or more voice messages, respectively, including the voice message sent by the second user's computing device. Receiving one or more voice recognized text messages; Storing at least one voice message and at least one voice recognized text message; Receiving a request for playing a voice message from a first user; Responsive to the voice message reproducing request, reproducing one or more voice messages sequentially received; And displaying the one or more voice recognized text messages in response to the reproduction of the one or more voice messages.

The above brief summary and description of the effects are merely exemplary and are not intended to limit the technical matters intended in the present disclosure. By referring to the following detailed description and the accompanying drawings, in addition to the above-described exemplary embodiments and technical features, additional embodiments and technical features will be understood.

Features and other additional features of the present disclosure described above are described in detail below with reference to the accompanying drawings. These drawings illustrate only a few embodiments in accordance with the present disclosure and should not be regarded as limiting the scope of the spirit of the present disclosure. The technical spirit of the present disclosure will be described in more detail and in detail using the accompanying drawings.

1 is an exemplary environmental diagram illustrating an environment in which an instant message service is provided in accordance with at least some embodiments of the present disclosure;

2 illustrates an example of using an instant message service in accordance with the present disclosure;

3 illustrates an example of displaying and playing a message on a user's mobile device when using an instant message service according to FIG. 2;

4 shows an example showing a log of an instant message, in the example according to FIGS. 2 and 3;

5 is a block diagram schematically illustrating an apparatus for providing an instant message service according to at least some embodiments of the present disclosure;

6 is a flow diagram illustrating an example process for a method of providing instant message services, in accordance with at least some embodiments of the present disclosure;

7 is a flowchart illustrating another example process for a method of providing instant message services, in accordance with at least some embodiments of the present disclosure;

8 illustrates an example computer program product that may be used to provide an instant message service, in accordance with at least some embodiments of the present disclosure.

9 is a block diagram schematically illustrating a server for providing an instant message service according to at least some embodiments of the present disclosure.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments and embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present disclosure. However, the present disclosure may be embodied in many different forms and should not be construed as limited to the embodiments and examples set forth herein.

The present disclosure generally relates to an apparatus, a method and a computer readable storage medium for providing an instant message service.

Hereinafter, "instant message service" may refer to a service in which a message received by a recipient is displayed and / or played back if the sender sends a message, such as a text message, voice message, image, or the like, to one or more recipients. The term " character " means an object that is represented by a computer graphic and has a face, and can be expressed in various forms such as, for example, a person, an animal, a virtual animal, a robot, and the like, and according to the present disclosure, a character is instant It is an object displayed on the message service and can be operated by the control of the user or the user's device.

In the following, the term “module” may refer to an apparatus, a server, a program unit, or a suitable combination thereof. For example, the term " memory module ", which will be described below, refers not only to hardware for storing data in memory, but also to devices, servers, program units or their suitable controls for deleting data stored in such hardware according to predetermined conditions. May refer to a combination.

1 is an exemplary environmental diagram illustrating an environment 100 in which an instant message service is provided, in accordance with at least some embodiments of the present disclosure. Exemplary environment 100 includes network environment 110, one or more user devices 120-1, 120-2, 120-3, 120-4,. (130-1, 130-2, 130-3, ...; hereinafter referred to as 130). The network environment 110 represents various environments for connecting the user device 120 and the user device 130 by wired or wireless communication. The network environment 110 may include a server 115 for providing an instant message service.

In some embodiments, the user device 120 may send an instant message to or receive an instant message from the user device 130 through the server 115 providing an instant message service. In some other examples, user device 120 may send a notification for an instant message to user device 130 via server 115, where user device 130 receives the notification from the server and sends the user an instant message. Can be received directly from the device 120. In such an embodiment, server 115 may serve as a relay server, and after user device 130 receives the notification, user device 130 may, for example, be peer-to-peer. An instant message can be received by connecting directly to the user device 120 using a peer technique.

In various embodiments, network environment 110 may further include a communication environment, such as a wired environment, a wireless environment, a base station, or the like, between

user devices

120, 130. In some examples, the server 115 stores the instant message sent by the user device 120 and then, when the user device 130 is connected to the server 115, receives the instant message received from the user device 120. Can be configured to transmit. In the example of directly sending an instant message in a peer-to-peer technique, server 115 may assist network environment 110 to establish a peer-to-peer connection between

user devices

120, 130.

In FIG. 1,

user devices

120 and 130 include communicable devices, such as smartphones, tablet computers, desktop computers, laptop computers, mobile phones, personal digital assistants (PDAs), special purpose devices, or any of the above functions. Small form factor portable (mobile) electronic devices, such as a fusion device. As shown in FIG. 1,

user devices

120 and 130 may perform one-to-one or many-to-many instant message communications as well as one-to-one instant message communications, and server 115 provides such instant messaging services. can do.

In some examples, a user (first user) of user device 120 may enter an instant message to send to user (second user) of user device 130 using user device 120. In some examples, the instant message may be a voice message entered by the first user. For example, the first user may input a voice message to the user device 120 for a predetermined time. Thereafter, the user device 120 may perform voice recognition on the voice message to generate a voice recognized text message, and various voice recognition techniques well known in the art according to the present disclosure may be used. The user device 120 may transmit a voice message and a voice recognized text message corresponding to the voice message to the user device 130 by an input of the first user. Voice and text messages may be sent from the user device 120 to the user device 130 through the server 115 or directly from the user device 120 to the user device 130.

In some examples, the user device 120 may combine two or more voice messages input by the first user and transmit them in one voice message. In some other examples, the user device 120 may combine one or more voice messages input by the first user and one or more text messages input by the first user and send them into one instant message.

In some further embodiments, while the instant messaging service is being provided, a character for the first user and a character for the second user may be displayed. The user device 120 may transmit character information about the character for the first user while transmitting the instant message to the user device 130. The character information may include information on at least one of the type of the character of the first user, the expression of the character, or the operation of the character.

In some examples, the user device 120 may obtain face information of the first user using a device such as an image / video camera, a depth camera, and the like, and based on the acquired face information, a character for the first user. Can determine the facial expression. In one example, the facial expression of the character for the first user may be determined while the first user is entering an instant message. In some other examples, the facial expression of the character may be selected by the first user. In some examples, the first user may select an action of the character from a list of predetermined actions. In some other examples, the user device 120 may recognize the characters associated with the character's movement from the voice recognized text message or typed text message. The first user may select an action of the character by selecting the recognized character. The first user may check a voice recognized text message, a facial expression and motion of the character, and determine the transmission of the voice message.

In some examples where an instant message entered by a first user is sent from the user device 120 to the user device 130 through the server 115, the server 115 may receive the instant message received from the user device 120. For example, voice messages, typed text messages, etc.). The server 115 may store the voice message corresponding to the voice recognized text message. In some further examples, the server 115 may receive character information from the user device 120 along with the instant message and store it in correspondence with the received instant message.

In some examples where an instant message entered by a first user is sent directly from user device 120 to user device 130, server 115 may send a notification to user device 130, and the user device ( 130 may receive a notification message for an instant message received from the user device 120 from the server 115.

The user device 130 may access the server 115. In some embodiments, the user device 130 may receive from the user device 120 one or more voice messages including voice messages input by the first user and one or more voice recognized text messages respectively corresponding to the one or more voice messages. Can be received. In the example where the user device 130 receives a notification from the server 115, the notification may include an indication of one or more voice messages to the user device 130 including the voice message received from the user device 120. have.

In some embodiments, user device 130 may store therein one or more voice messages and corresponding one or more voice recognized text messages. In some examples, user device 130 may store the received one or more voice messages and the voice recognized text message in correspondence.

In some embodiments, user device 130 may receive a voice message playback request from a user (second user) to play the received one or more voice messages. The user device 130 may sequentially play one or more received voice messages in response to the voice message play request. As such, the voice message can be played back asynchronously with the reception of the voice message. In addition, playback of the voice message may be controlled to pause, stop, or play the previous voice message or the next voice message at the request of the user. In some examples, the user device 130 may display one or more voice recognized text messages in response to playback of the one or more voice messages.

In some further embodiments, before the user device 130 plays one or more voice messages, the user device 130 may filter the one or more voice recognized text messages based on a predetermined censoring condition. For example, predetermined censorship conditions may include, but are not limited to, abusive language, slang, and the like. The user device 130 may play one or more voice messages based on the result of the filtering. For example, if a slang is included in the voice recognized text message, the user device 130 may mute and reproduce at least a part of the corresponding voice message. This filtering process is not limited to what the user device 130 performs, and in some embodiments, the server 115 may perform the filtering process before sending the one or more voice messages and the corresponding one or more voice recognized text messages. have.

In some embodiments, user device 130 may delete one or more voice messages based on a predetermined condition. In some examples, the user device 130 may delete the corresponding voice message when playback ends. In some other examples, user device 130 may delete the old voice message based on a predetermined storage capacity condition.

In some additional embodiments, the user device 130 may receive character information for the character of the first user from the server 115 in addition to receiving one or more voice messages. In some examples, when the user device 130 sequentially plays one or more voice messages, the user device 130 may display the character of the first user based on the received character information. In some examples, the user device 130 may display a character whose expression, motion, or the like of the character of the first user is controlled based on the character information.

2 illustrates an example of using an instant message service according to the present disclosure, and FIG. 3 illustrates an example of displaying and playing a message on a user's mobile device when using the instant message service according to FIG. 2. As shown in FIG. 2, the first user 210, the second user 220, and the third user 230 are respectively connected to the user device 212, the user device 222, and the user device 232. I use an instant messaging service. In the example of FIG. 2, the

users

210, 220, and 230 may share an instant message transmitted and received at the request of at least one of the

users

210, 220, and 230. For example, when the first user 210 transmits an instant message, the second user 220 and the third user 230 may receive the instant message. As illustrated in FIG. 2, the first user 210 and the second user 220 may transmit a voice message.

In some examples, first user 210 may select character 216 and second user 220 may select character 226. The first user 210 may input a voice message 214 that is "not cold?" When the first user 210 inputs the voice message 214, the user device 212 may detect the facial expression of the first user 210 and may determine the facial expression of the character 216. The user device 212 can obtain the speech recognized text message 214-2 by performing speech recognition on the voice message 214. In addition, the first user 210 selects one of a list of predetermined actions or removes a character (eg, “cold” is recognized) that the user device 212 recognizes from the voice recognized text message 214-2. 1 The user 210 can determine the operation of the character 216 by selecting. The voice message 214 may then be sent towards the user devices 222, 232 of the second and

third users

220, 230.

After the voice message 214 is transmitted, the second user 220 may input the voice message 224 "I'm hot!" When the second user 220 inputs the voice message 224, the user device 222 may detect an expression of the second user 220 and may determine an expression of the character 226. The user device 222 may obtain the voice recognized text message 224-2 from the voice message 224. In addition, the second user 220 selects one of a list of predetermined actions, or selects a character recognized by the user device 222 from the voice recognized text message 224-2 (eg, “hot” is recognized). The operation of the character 226 may be determined by the selection of the second user 220. The voice message 224 may then be sent towards the

user devices

212, 232 of the first and

third users

210, 230.

Thereafter, the third user 230 may access a server (not shown) that provides an instant message service using the user device 232, and the user device 232 may have a voice message 214 and a voice message 224. ) Can be received. In addition, the user device 232 may receive the voice recognized text messages 214-2, 224-2. As shown in FIGS. 3A and 3B, the voice recognized text messages 214-2 and 224-2 may be displayed in response to playback of the

voice messages

214 and 224. The third user 230 may input a voice message playback request using the user interface 240 displayed on the user device 232. When a voice message reproduction request is input, as shown in Fig. 3A, the voice message 214 is reproduced with the display of the character 216. Also, while voice message 214 is played back, voice recognized text message 214-2 may be displayed in response to playback of voice message 214. The character 216 may show facial expressions and actions determined by the user device 212 of the first user 210. Thereafter, as shown in FIG. 3B, the voice message 224 is played along with the display of the character 226. In addition, while the voice message 224 is reproduced, the voice recognized text message 224-2 may be displayed in response to the reproduction of the voice message 224. The character 226 may show facial expressions and actions determined by the user device 222 of the second user 220.

4 shows an example showing a log of an instant message in the example according to FIGS. 2 and 3. In the user device 232 of the third user 230, the third user 230 may be configured to receive instant messages 214-2 and 224-2 received from the first user 210 and the second user 220. An example displayed sequentially is shown. Although not shown in the example of FIG. 4, instant messages transmitted by the user device 232 of the third user 330 may also be sequentially displayed based on the transmission time and / or the reception time.

5 is a block diagram schematically illustrating an instant message service providing apparatus 500 according to at least some embodiments of the present disclosure. As illustrated in FIG. 5, the instant message service providing apparatus 500 may include a communication module 510, a user interface module 520, a voice playback module 530, a display module 540, a memory module 550, and a voice. Recognition module 570 may be included. In addition, the instant message service providing apparatus 500 may further include a message filter 560, a text recognition module 580, a camera module 590-1, and an facial expression determination module 590-2. Components included in the instant message service providing apparatus 500 may be individually implemented or two or more of the components may be combined to form one component. As described in more detail below, the instant message service providing apparatus 500 transmits a voice message and a voice recognized text message, while the text message received from an external computing device such as the instant message service providing apparatus 500. Can be configured to display and play the voice message at the request of the user. The instant message service providing device 500 may be a variety of computing devices, such as, for example, smartphones, tablet computers, desktop computers, laptop computers, mobile phones, personal digital assistants (PDAs), special purpose devices, or any of the above functions. Small form factor portable (mobile) electronic devices, such as including fusion devices.

The communication module 510 may be configured to connect the instant message service providing apparatus 500 to a server. The communication module 510 may be configured to receive one or more voice messages and one or more voice recognized text messages corresponding to one or more voice messages, respectively, from an external computing device. In some examples, communication module 510 may receive one or more voice messages and corresponding one or more voice recognized text messages from an external computing device via a server. In some other examples, communication module 510 may receive a notification from a server. The notification may indicate that one or more voice messages have been sent. In this example, the communication module 510 receives one or more voice messages and corresponding one or more voice recognized text messages directly from an external computing device, such as in a manner such as a peer to peer connection, based on this notification. can do.

The user interface module 520 may be configured to receive a voice message playing request for the received one or more voice messages from the user. The voice playing module 530 may be configured to sequentially play one or more voice messages received by the communication module 510 in response to a voice message playing request received by the user interface module 520. The user interface module 520 may receive various requests from the user for pausing, stopping, playing the previous voice message, or playing the next voice message, as required. ) May play the voice message according to the request received by the user interface module 520.

In addition, the display module 540 may be configured to display one or more voice recognized text messages corresponding to one or more voice messages, respectively, in response to playback of the one or more voice messages. The memory module 550 may be configured to store one or more voice messages received by the communication module 510 and corresponding one or more voice recognized text messages. In some examples, the memory module 550 may store one or more received voice messages and voice recognized text messages in correspondence. In some embodiments, the memory module 550 may delete one or more voice messages based on a predetermined condition. In some examples, the memory module 550 may delete the corresponding voice message stored in the memory module 550 when the playback of the voice message ends. In some other examples, the memory module 550 may delete the old voice message based on the predetermined storage capacity condition. In one example, the memory module 550 may delete the oldest voice message if the total capacity of the stored voice message exceeds a predetermined value. In another example, the memory module 550 may delete the voice message when the voice message, for which playback ends, exceeds a predetermined value.

In a further embodiment, the message filter 560 may filter the one or more voice recognized text messages based on a predetermined censoring condition before the voice playback module 530 plays the received one or more voice messages. The censoring method by the message filter 560 may use a well-known censoring method for a text message. For example, the predetermined censorship condition may include, but is not limited to, abusive language, slang, and the like. The voice reproduction module 530 may reproduce one or more voice messages based on the filtering result of the message filter 560. For example, when a slang is included in a voice recognized text message, the voice reproducing module 530 may mute the corresponding voice message.

In addition, the communication module 510 may receive character information about the character of the sender from the server while receiving one or more voice messages. In some examples, when the voice playback module 530 sequentially plays one or more voice messages, the display module 540 may display the sender's character based on the received character information. The display module 540 may display a character whose expression, motion, or the like is controlled based on the character information.

In some embodiments, the user interface module 520 may be configured to receive a voice message (hereinafter, “input voice message”) input by a user of the instant message service providing apparatus 500. In some examples, user interface module 520 may receive an input voice message for a predetermined time. In some examples, the user interface module 520 may generate one input voice message by combining two or more input voice messages received for a predetermined time at a user's request. The voice recognition module 570 may generate a voice recognized text message by performing voice recognition on the input voice message received by the user interface module 520, and various voices well known in the art according to the present disclosure. Recognition techniques can be used. It is also possible for a user to type a text message through the user interface module 520. The communication module 510 may transmit an input voice message and a corresponding voice recognized text message to an external computing device. The voice recognized text message corresponding to the input voice message may be transmitted to the external computing device or directly to the external computing device through the server.

In some further embodiments, while the instant message service is being provided, the display module 540 may display a character of the user of the instant message service providing apparatus 500. The communication module 510 may transmit character information about the character of the user. The character information may include information about at least one of a type of a character of a user, an expression of a character, or an operation of the character.

In some examples, text recognition module 580 may recognize text associated with the movement of the character from a speech recognized text message or typed text message generated by speech recognition module 570. The user can select an action of the character by selecting the recognized character. In some other examples, the user may select an action of the character from the list of predetermined actions of the character via the user interface module 520.

In some examples, the camera module 590-1 may acquire face information of the user using a device such as an image / video camera, a depth camera, or the like. The facial expression determination module 590-2 may determine the facial expression of the character of the user based on the face information obtained by the camera module 590-1. In one example, while the user interface module 520 receives an instant message, such as a voice message or text message, the camera module 590-1 obtains face information of the user, and the facial expression determination module 590-2 The facial expression of the character may be determined based on the acquired face information.

Additionally or alternatively, the user may determine the transmission of an instant message, such as a voice message, after confirming a text message, a facial expression, an action, or the like appearing on the display module 540.

6 and 7 are flowcharts illustrating example processes 600 and 700 for a method of providing instant message services, in accordance with at least some embodiments of the present disclosure. 6 illustrates a process 600 for receiving an instant message, and FIG. 7 illustrates a process 700 for sending an instant message. For example, the

processes

600 and 700 may be performed under the control of a computing device such as the

user device

120, 130 of FIG. 1, or the instant message service providing device 500 of FIG. 5. The process 600 shown in FIG. 6 may include one or more operations, functions or actions as illustrated by

blocks

610, 620, 630, 640 and / or 650. In addition, the process 700 shown in FIG. 7 may include one or more operations, functions, or actions as illustrated by

blocks

710, 720, 730, and / or 740. The various blocks are not intended to be limited to the described embodiments. For example, those skilled in the art will appreciate that, for the present processes disclosed herein, the functions performed in the processes and methods may be implemented in a different order. The schematic operations illustrated in FIGS. 6 and 7 are provided by way of example only, and some of the operations may be optional, may be combined in fewer operations, or extended to additional operations without departing from the spirit of the disclosed embodiment. Can be.

The process 600 shown in FIG. 6 begins at block 610 connecting to a server. At block 610, the computing device may connect to the server. Process 600 may continue to block 620 to receive one or more voice messages and one or more voice recognized text messages at block 610.

At block 620, the computing device may receive one or more voice messages and one or more voice recognized text messages corresponding to each of the one or more voice messages from the sender's external computing device. In some examples, the server may send a notification for one or more voice messages received from the sender, and the computing device may receive such a notification from the server. The computing device may then receive one or more voice messages and corresponding one or more voice recognized text messages that appear in the notification directly from an external computing device or through a server. Process 600 may continue at block 620 with block 630 storing the received one or more voice messages and one or more voice recognized text messages.

At block 630, the computing device may store therein one or more voice messages and corresponding one or more voice recognized text messages therein. In some examples, the computing device may store the received one or more voice messages and the one or more voice recognized text messages in correspondence. Process 600 may continue to block 640 to receive a voice message playback request from a user at block 630.

At block 640, the computing device may receive a voice message playback request from the user to play the received one or more voice messages. Process 600 may continue to block 650 to sequentially play one or more voice messages at block 640.

At block 650, the computing device may sequentially play one or more voice messages received in response to the voice message playback request. In addition, playback of the voice message may be controlled to pause, stop, or play the previous voice message or the next voice message at the request of the user. In some examples, the computing device may delete the voice message for which playback has ended based on a predetermined condition. In some examples of receiving the sender's character information, the computing device may play one or more voice messages sequentially while displaying the sender's character based on the character information.

In addition, the computing device may filter the one or more voice recognized text messages based on the predetermined censoring condition before performing block 650. Thereafter, at block 650, the computing device may play one or more voice messages based on the result of the filtering. For example, when a slang is included in the voice recognized text message, the computing device may mute and reproduce at least a portion of the corresponding voice message.

Process 700 shown in FIG. 7 may begin at block 710 for receiving an input voice message from a user. At block 710, a user can enter a voice message to be sent using the computing device. In some examples, a user may enter a voice message into the computing device, for example, for a predetermined time, and the computing device may receive this input voice message. Process 700 may continue to block 720 to obtain a voice recognized input text message at block 710.

In block 720, the computing device may perform voice recognition on the input voice message to generate a voice recognized input text message. Speech recognition for input voice messages can use a variety of known techniques. In some examples, the computing device may display a voice recognized input text message and the user may confirm the displayed message. Process 700 may continue to block 730 to obtain character information at block 720.

In block 730, the computing device may obtain character information for the character of the user to be displayed on the instant message service. In some examples, the computing device may obtain facial information of the user using an accessory device connected to the computing device, such as an image / video camera, a depth camera, and the like, and based on the acquired facial information, the facial expression of the character to the user Can be determined. In some other examples, the facial expression of the character may be selected by the user. In some examples, the computing device may obtain gesture information of the character selected from the list of gestures predefined by the user. In some other examples, the computing device may recognize a character associated with the movement of the character from the voice recognized input text message, and when the user selects one of the recognized characters, the computing device may retrieve the movement information of the character corresponding to the character. Can be obtained. As described above, the character information may include not only the type of character but also the expression of the character and the operation of the character. Process 700 may continue from block 730 to block 740 where the computing device may transmit a voice message, a voice recognized input text message and character information to an external computing device.

Thus, by providing an instant message service, when receiving a voice message, it is easier to understand the contents of the voice message by sequentially playing the voice messages without inputting a separate request for playing each of the received voice messages. Lose. It also stores and displays voice-recognized text messages along with voice messages, making it easy to quickly understand and search for conversations made during the provision of instant messaging services, even if the voice messages are not played or the voice messages are erased due to capacity issues. Become.

8 illustrates an example computer program product 800 that may be used to perform defect inspection in accordance with at least some embodiments of the present disclosure. An example embodiment of an example computer program product is provided using a signal bearing medium 802. In some embodiments, signal bearing media 802 of one or more computer program products 700 may include computer readable media 806, recordable media 808, and / or communication media 810.

The instructions 804 included in the signal bearing medium 802 may be executed by a computing device such as the

user device

120, 130 shown in FIG. 1 and / or the instant message service providing device shown in FIG. 5. The instruction 804 may, when executed, provide an instant message service for the first user in accordance with the present disclosure. The instructions 804 may include one or more instructions for receiving one or more voice messages including a voice message sent by the computing device of the second user and one or more voice recognized text messages respectively corresponding to the one or more voice messages; One or more instructions for storing one or more voice messages and one or more voice recognized text messages; One or more instructions for receiving a voice message playing request from a first user; One or more instructions for reproducing the one or more voice messages sequentially received in response to the voice message playback request; And one or more instructions for displaying the one or more voice recognized text messages in response to playing of the one or more voice messages.

9 is a block diagram schematically illustrating an instant message service providing server 900 according to at least some embodiments of the present disclosure. As illustrated in FIG. 9, the instant message service providing server 900 may include a communication module 910, a character module 920, a voice memory 930, and a text memory 940. The communication module 910 may receive instant message and character information, such as a voice message and / or text message, from the sender. In addition, the communication module 910 may transmit a notification and / or an instant message for the instant message to the recipient of the instant message. In some examples, the communication module 910 may transmit a voice message, a voice recognized text message, and character information of the sender. The character module 920 may store character information received from the sender, for example, information on a type, facial expression, motion, and the like of the sender's character. The voice memory 930 may store a voice message received from the sender. In some examples, the voice message stored in the voice memory 930 may be deleted according to a predetermined condition. The text memory 940 can store voice recognized text messages and typed text messages. In some examples, the text memory 940 may store the voice recognized text message corresponding to the voice message stored in the voice memory 930, and the character module 920 may store the character information in the voice message stored in the voice memory 930. And / or corresponding to the voice recognized text message or typed text message stored in the text memory 940.

The claimed subject matter is not limited in scope to the specific embodiments described herein. For example, some implementations may be in hardware, such as may be used to operate on a device or combination of devices, while other implementations may be in software and / or firmware, for example. Likewise, the claimed subject matter is not limited in scope in this respect, but some embodiments may include one or more articles, such as signal bearing media, storage media. Such storage media, such as CD-ROMs, computer disks, flash memory, etc., may be executed by computing devices such as, for example, computing systems, computing platforms, or other systems, for example, to claimed subject matter, such as one of the embodiments described above. As a result, instructions may be stored that may cause the processor to execute. As one possibility, the computing device may comprise one or more processing units or processors, one or more input / output devices such as displays, keyboards and / or mice, and static random access memory, dynamic random access memory, flash memory and / or hard drives. It may include one or more of the same memory.

There is little distinction between hardware and software implementations of aspects of the system; The use of hardware or software is generally a design choice that represents a tradeoff in cost-efficiency (but not always in the sense that the choice between hardware and software can be important in some contexts). . There are various vehicles (eg, hardware, software and / or firmware) in which the processes and / or systems and / or other techniques described in this disclosure can be affected, and preferred means are processes and / or systems and And / or other techniques will change depending on the context in which it is used. For example, if an implementer decides that speed and accuracy are the most important, then the implementer may primarily choose hardware and / or firmware means; if flexibility is paramount, the implementer may choose a software implementation primarily; Or, as another alternative, the implementer may choose any combination of hardware, software and / or firmware.

The foregoing detailed description has described various embodiments of the apparatus and / or process via block diagrams, flow diagrams, and / or examples. As long as such block diagrams, flowcharts, and / or examples include one or more functions and / or operations, one of ordinary skill in the art will appreciate that each function and / or operation within such block diagrams, flowcharts, or examples is hardware, software, firmware, or their It will be understood that they may be implemented individually and / or collectively by a wide range of substantially any combination. In one embodiment, some portions of the subject matter described in this disclosure may be implemented via an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), Digital Signal Processor (DSP), or other integrated form. However, those skilled in the art will appreciate that some aspects of the embodiments of the present disclosure may include one or more computer programs running on one or more computers (eg, one or more programs running on one or more computer systems), one running on one or more processors. Software, and / or firmware that may be implemented in integrated circuits, in whole or in part, as one or more programs (eg, one or more programs running on one or more microprocessors), firmware, or substantially any combination thereof It will be appreciated that the writing of code for and / or the design of circuitry is within the skill of one of ordinary skill in the art in light of this disclosure. Moreover, those skilled in the art will understand that the mechanisms of the subject matter of the present disclosure may be distributed in various forms of program products, and examples of the subject matter of the present disclosure are specific types of signal bearing media used to actually perform the distribution. It will be understood that it is applied regardless of.

While certain example techniques have been described and illustrated herein using various methods and systems, it should be understood by those skilled in the art that various other modifications may be made and equivalents may be substituted without departing from the claimed subject matter. In addition, many modifications may be made to adapt a particular situation to the teachings of the claimed subject matter without departing from the central concept described herein. Thus, while the claimed subject matter is not limited to the specific examples disclosed, it is intended that such claimed subject matter may also include all embodiments falling within the scope of the appended claims and their equivalents.

Claims

An instant message service providing method performed under the control of a computing device of a first user,

Receiving at least one voice message comprising a voice message sent by the computing device of a second user, at least one voice recognized text message corresponding to each of the at least one voice message, and information about a character of the second user ;

Storing the one or more voice messages and the one or more voice recognized text messages;

Receiving a request for playing a voice message from the first user; And

In response to the voice message reproduction request, displaying the character of the second user based on the character information and sequentially playing the received one or more voice messages;

Instant message service providing method comprising a.
The method of claim 1,

Receiving information about the one or more voice messages, the one or more voice recognized text messages and the character of the second user may include:

Receiving a notification from the server for the one or more voice messages; And

Receiving the one or more voice messages and the one or more voice recognized text messages directly from the computing device of the second user.

That includes, instant messaging service providing method.
The method of claim 1,

Receiving information about the one or more voice messages, the one or more voice recognized text messages and the character of the second user may include:

Receiving the one or more voice messages and the one or more voice recognized text messages sent by the computing device of the second user from a server storing the one or more voice messages and the one or more voice recognized text messages.

That includes, instant messaging service providing method.
The method of claim 1,

Displaying the one or more voice recognized text messages in response to playing the one or more voice messages.

It further comprises, instant message service providing method.
The method of claim 1,

Prior to sequentially playing the received one or more voice messages,

Filtering the one or more voice recognized text messages based on a predetermined censoring condition

More,

Playing the received one or more voice messages comprises playing the one or more voice messages based on a result of the filtering.
The method of claim 1,

After sequentially playing the received one or more voice messages,

Deleting at least a portion of the one or more voice messages based on a predetermined condition

Instant message service providing method further comprising.
The method of claim 1,

The character information includes information on at least one of the type of the character of the second user, the expression of the character or the operation of the character.
An instant message service providing apparatus,

A communication module configured to receive one or more voice messages including voice messages sent by the sender's external computing device, one or more voice recognized text messages corresponding to each of the one or more voice messages, and information about the character of the sender;

A user interface module configured to receive input for an instant message service from a user;

Voice playback module;

Display module; And

A memory module operatively coupled to the communication module

Including,

The user interface module is configured to receive a voice message playback request from the user,

The voice playing module is configured to sequentially play the one or more voice messages received by the communication module in response to the voice message playing request received by the user interface module,

The display module is configured to display the one or more voice recognized text messages in response to the playback of the one or more voice messages by the voice playback module,

The display module is further configured to display a character of the sender based on the character information, with the reproduction of the one or more voice messages by the voice reproducing module,

The memory module is configured to store the one or more voice messages and the one or more voice recognized text messages,

Device for providing instant message service.
The method of claim 8,

The communication module,

Receive a notification from the server for the one or more voice messages; And

And receive the one or more voice messages and the one or more voice recognized text messages directly from the external computing device.
The method of claim 8

The communication module is configured to receive the one or more voice messages and the one or more voice recognized text messages sent by the external computing device from a server that stores the one or more voice messages and the one or more voice recognized text messages. And an instant message service providing device.
The method of claim 8

The memory module

Store the one or more voice messages and the one or more voice recognized text messages,

And delete at least some of the one or more voice messages based on a predetermined condition.
The method of claim 11,

And the memory module is configured to delete the oldest voice message when the total capacity of the one or more voice messages stored in the memory module exceeds a predetermined value.
The method of claim 8,

A message filter configured to filter the one or more voice recognized text messages based on a predetermined censoring condition

More,

And the voice playback module is configured to play the one or more voice messages based on a result of the filter by the message filter.
The method of claim 8,

And the character information includes information on at least one of a type of the character of the sender, an expression of the character, or an operation of the character.
According to claim 8,

The user interface module is configured to receive an input voice message from the user,

The instant message service providing apparatus

A voice recognition module configured to perform voice recognition on the input voice message to obtain a voice recognized input text message

Instant message service providing device further comprising.
The method of claim 15,

A character recognition module configured to detect a motion character from the speech recognized input text message obtained by the speech recognition module to enable selection of an action of the character of the user displayed by the display module

Instant message service providing device further comprising.
The method of claim 15,

A camera module configured to obtain face information of the user; And

An expression determining module configured to determine an expression of a character of the user based on the face information obtained by the camera module.

Instant message service providing device further comprising.
A computer readable storage medium having stored thereon a computer program for providing an instant message service, wherein the computer program, when executed, causes the first user's computing device to:

Receiving at least one voice message comprising a voice message sent by a computing device of a second user, at least one voice recognized text message corresponding to each of the at least one voice message, and information about the character of the second user ;

Storing the one or more voice messages and the one or more voice recognized text messages;

Receiving a voice message playing request from the first user;

In response to the voice message reproduction request, displaying the character of the second user based on the character information and sequentially playing the received one or more voice messages; And

Displaying the one or more voice recognized text messages in response to playing the one or more voice messages.

And one or more computer executable instructions for making the operations executable.