KR101968669B1 - Method for providing call service and computer program for executing the method - Google Patents

Abstract

According to an embodiment of the present invention, there is provided a method of providing a call service in a first terminal, comprising: receiving a sound source from a second terminal; Converting the received sound source into text using a speech recognition module; Displaying the converted text on a display unit of the first terminal; Receiving text from a user; Converting the received text into a sound source using a speech synthesis module; And transmitting the converted sound source to the second terminal.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a call service providing method and a computer program,

A method of providing a call service, and a computer program.

Telephone refers to a technique of converting a voice into an electric signal, transmitting it to another device, receiving an electric signal, and reproducing voice again to enable communication between two devices at a remote location. Since the development of commercial mobile phones, it has become possible to make calls using mobile phones anytime and anywhere, and thereafter, even communication between computer devices via the Internet has become possible. Here, the computer device refers to all devices capable of being connected to the Internet, and includes a smart phone.

On the other hand, since the telephone basically transmits voice, a user who is unable to reproduce the voice, or a user who has difficulty in recognizing the voice due to the hearing impairment has a limitation in using the telephone.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

And a computer program for providing a call service including a sound source-text conversion function. The technical problem to be solved by this embodiment is not limited to the above-mentioned technical problems, and other technical problems can be deduced from the following embodiments.

According to an embodiment of the present invention, there is provided a method of providing a call service in a first terminal, comprising: receiving a sound source from a second terminal; Converting the received sound source into text using a speech recognition module; Displaying the converted text on a display unit of the first terminal; Receiving text from a user; Converting the received text into a sound source using a speech synthesis module; And transmitting the converted sound source to the second terminal.

According to an embodiment, the displaying step may display the text in a format corresponding to the amplitude of the received sound source, and the relation between the amplitude and the format may be set so that the larger the amplitude is, have.

According to an embodiment, the step of receiving and the step of converting into text are performed a plurality of times, and the displaying step may accumulatively display the converted texts.

According to an embodiment of the present invention, the method may further include scrolling the accumulated text corresponding to a first input input by the user to the first terminal.

According to an embodiment, the screen displayed on the display unit of the terminal includes a first area for displaying the converted text and a second area for receiving text from the user, and the scrolling is performed in the first area And the second area can be fixedly displayed even if the scrolling is performed.

According to an embodiment, the screen displayed on the display unit of the terminal includes a first area for displaying the converted text and a second area for receiving text from the user, and when the scrolling is performed, The first area may be extended to a displayed position, and only the first area may be displayed on the display unit.

According to one embodiment, at least one of the words included in the text is selected in consideration of at least one of the part of speech, the frequency of each word included in the accumulated text, and whether or not each word matches a predetermined format The method comprising the steps of:

According to an embodiment, the selecting and displaying step may include selecting at least one of a font, a size, a color, and a display of each word in consideration of at least one of a part of speech, frequency of the word, One or more of the positions may be determined.

According to an embodiment, the first terminal may alternatively provide a text mode for outputting data received in a call process to a display unit in a text format, and a sound source mode for outputting through a speaker in a sound source format, Selecting the text mode by the first terminal, wherein the receiving step or the transmitting step is performed after the selecting step, and when the sound source mode is selected by the user, A sound source may be output to the speaker and a sound source received from the microphone of the first terminal may be transmitted to the second terminal.

According to an embodiment, in the receiving step, the text stored in the first terminal or the server is selected from a user and the converting the selected text into the sound source may convert the selected text into a sound source using a sound synthesis module .

According to another embodiment of the present invention, the receiving step may include converting a character input through the at least one drag operation to the touch screen of the first terminal into text using a character recognition module, The converted text can be converted into a sound source using a speech synthesis module by using a module.

According to one embodiment, the displaying step may translate the converted text into a second language specified by the user in the first language using the translation module, and display the translated text in the second language on the display unit have.

According to an exemplary embodiment, the transmitting may include transmitting the converted sound source and the sound source filter information selected by the user to the server so that the server can apply the sound source filter to the converted sound source and transmit the sound source filter to the second terminal have.

According to an embodiment of the present invention, there is provided a method of providing a call service in a first terminal, the method comprising: receiving text from a second terminal; Displaying the text on a display of the first terminal; Receiving text from a user; Converting the received text into a sound source using a speech synthesis module; And transmitting the converted sound source to the second terminal.

Receiving a call initiation request with a second terminal according to an embodiment; And initiating a call by transmitting a call initiation request to the second terminal and receiving a call initiation acknowledgment from the second terminal, wherein the call initiation request includes an output format information And the data output format may be specified as either a text or a sound source.

According to an embodiment of the present invention, there is provided a method for receiving a sound source, the method comprising: receiving a sound source from a second terminal; Converting the sound source into text using a speech recognition module, and transmitting the converted text to a first terminal; Receiving text from a first terminal; And converting the received text into a sound source using a speech synthesis module, and transmitting the converted sound source to the second terminal.

According to an embodiment of the present invention, a computer program stored on a medium for executing a method according to any one of the above embodiments using a computer is disclosed.

A call service providing method and a computer program according to embodiments of the present invention provide a text-based call service through a sound source-text conversion function, thereby enabling a user who can not reproduce a voice, Users with difficulties can also use the service smoothly.

1 is a schematic illustration of a call system in accordance with an embodiment of the present invention.
2 is a block diagram schematically illustrating an apparatus for providing a call service according to an embodiment of the present invention.
3 is a flowchart schematically showing a call service providing method according to the first embodiment.
4 is a flowchart schematically showing a modification of the call service providing method according to the first embodiment.
5 is a flowchart schematically showing a call service providing method according to the second embodiment.
6 is a flowchart schematically illustrating a method for providing a call service according to the first embodiment in which a first terminal and a second terminal divide and process a sound source-text conversion function.
7 to 13 are various examples of screens displayed on the terminal.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and particular embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by terms. Terms are used only for the purpose of distinguishing one component from another.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, Should not be construed to preclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, or combinations thereof.

Hereinafter, the present invention will be described in more detail with reference to the preferred embodiments of the present invention shown in the accompanying drawings.

1 is a schematic illustration of a call system in accordance with an embodiment of the present invention.

Referring to FIG. 1, the communication system includes a terminal 10, which is a call subject, and a server 20, which communicates with the terminal 10 to mediate a call service between the terminals 10. The server 20 and the terminal 10 can communicate through the communication network 30. [ The communication network 30 may be an Internet network and / or a wired telephone network. However, the communication network 30 is not limited to the LAN network such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs , Wireless LANs, CDMA, Bluetooth, satellite communication, 3G, 4G, 5G, LTE, and the like.

The terminal 10 includes a terminal device capable of making a call using a wired telephone network and all communication terminal devices capable of installing and executing an Internet site access or service dedicated application (hereinafter referred to as a "service app") associated with the server 20 It can mean. For example, the terminal 10 may be a device such as a telephone, a smart phone, a tablet, a PC, or a notebook computer. The terminal 10 can perform service overall operations such as service screen configuration, data input, data transmission / reception, and data storage under the control of the server 20 and / or the service application. In FIG. 1, the first terminal 11 and the second terminal 12 are illustrated as an example to explain that a plurality of terminals 10 exist.

The server 20 is a communication server that communicates with the first terminal 11 and the second terminal 12 via the Internet network or communicates with the first terminal 11 and the second terminal 12 via the telephone network A base station, a communication server, and a base station. The specific configuration of the server 20 may vary according to each embodiment. Hereinafter, the configuration of the server 20 will be described while explaining each embodiment.

Data transmission from the first terminal 11 to the second terminal 12 and data transmission from the second terminal 12 to the first terminal 11 are described in the first When the first terminal 11 and the second terminal 12 transmit and receive data through the server 20 as well as when the terminal 11 and the second terminal 12 directly transmit and receive data, May be included.

The call service provided by the call system according to an embodiment of the present invention can expand the call service that was previously limited to the voice based call to the text based service by providing the function of converting between the text and the sound source.

In the following, embodiments of the present invention will be described. When a user wants to use a text-based call service for reasons such as being unable to reproduce a voice in a place such as a movie theater, 1 terminal 11 and that the communication partner terminal of the first terminal 11 is the second terminal 12. [ The distinction between "first" and "second" is intended to distinguish each of a plurality of devices for convenience of explanation. The first terminal 11 may include a display unit capable of displaying text, which is a terminal to use a text-based call service.

2 is a block diagram schematically illustrating an apparatus for providing a call service according to an embodiment of the present invention.

The components shown in FIG. 2 may be provided in the terminal 10 or may be provided in the server 20 according to the design of the embodiments of the present invention. Or a part of the components shown in FIG. 2 may be provided in the terminal 10, and the remaining part may be provided in the server 20. FIG.

2 may be provided in the terminal 10 and the server 20 may exchange data transmitted / received between the first terminal 11 and the second terminal 12 Only the role can be performed. That is, the sound source-text conversion function used in the call service can be processed in the terminal 10. In this example, the sound source-text conversion function may be processed only in the terminal of either the first terminal 11 or the second terminal 12, which is a communication party, or may be processed separately in both terminals.

According to the second embodiment, the display control unit 122 of the components of FIG. 2 is provided in the terminal 10 and the remaining components are provided in the server 20, The data transmitted / received between the first terminal 11 and the second terminal 12 can be processed and transmitted. In the second embodiment, the call control unit 121 may be provided in the server 10 and the display control unit 122 may be provided in the terminal 10. [ In this case, the role of the call control unit 121 may be performed by the communication server, by the base station, or by cooperation of the communication server and the base station. The following description of each component in FIG. 2 can basically be applied to both the first and second embodiments described above, and if there are special circumstances, a description thereof will be added.

Referring to FIG. 2, the call service providing apparatus 100 may include a memory 110 and a processor 120. The call service providing apparatus 100 shown in FIG. 2 shows the components related to the embodiments of the present invention to prevent the features of the embodiments of the present invention from being blurred. Therefore, components other than the components shown in FIG. 2 may be further included, such as general-purpose components such as an input / output unit, a display unit, and a communication unit.

The memory 110 may store various modules required for providing a call service. In one embodiment, the memory 110 includes a speech recognition module 111 and a speech synthesis module 112. The memory 110 may further include a character recognition module 113 and / or a translation module 114 to further provide additional functionality.

The processor 120 may perform overall data processing to provide a call service. The processor 120 may include a call control unit 121 for controlling overall data processing related to a call and a display control unit 122 for controlling a screen displayed on a display unit of the terminal 10 and providing a user interface.

The speech recognition module 111 according to one embodiment provides a speech recognition function. Speech recognition may include a process of interpreting a speech language and converting its contents into text, a process of extracting text from a sound source, a process of converting a sound source into text, and the like. As a concrete method for speech recognition, various techniques related to STT (Speech-to-Text) can be borrowed without limitation. An example of a representative algorithm is HMM (hidden markov model).

The speech synthesis module 112 according to one embodiment provides a speech synthesis function. Speech synthesis may refer to a process of synthesizing a voice that reads the text from text, a process of converting the text into a sound source, and the like. As a concrete method for speech synthesis, various techniques related to TTS (Text-to-Speech) can be borrowed without limitation.

The character recognition module 113 according to one embodiment refers to a process of recognizing a written character and converting it into a text. For example, the character recognition module 113 can read a handwritten character directly input from a user through a touch screen or the like, and convert the handwritten character into text. A handwritten character may consist of one or more drag inputs, or consist of one or more consecutive inputs input with an electronic pen, or may be composed of a combination of these inputs. The character recognition module 113 can determine which character is the last by determining the closest model pattern among the pre-stored model patterns as the recognition result in consideration of the coordinates of each input, the writing order, and the like.

The translation module 114 according to one embodiment may provide a translation function between a plurality of languages. For example, it is possible to provide a function of translating a first language into a second language different from the first language.

The call control unit 121 according to an exemplary embodiment may perform all the processes related to the call. The call control unit 121 performs a process of converting a sound source into text using the speech recognition module 111 of the memory 111, a process of converting text into a sound source using the sound synthesis module 112, a sound source and / From the external device or to the external device. In addition, the call control unit 121 may process the sound source and / or text data transmitted / received during the call process to extract meaningful data for providing to the user. The call control unit 121 can determine the format of the text to be displayed on the display unit of the terminal 10. [

The display control unit 122 according to an embodiment can control a screen displayed on the display unit of the terminal 10. [ The display control unit 122 may provide a user interface related to the call service.

3 is a flowchart schematically showing a call service providing method according to the first embodiment.

In the flowchart shown in FIG. 3, the first terminal 11 and the second terminal 12 are shown to directly exchange data, but the present invention is not limited thereto. For example, the communication between the first terminal 11 and the second terminal 12 may be via the server 20 as described above. The server 20 in this example may include only a communication server, only a base station, or a communication server and a base station. That is, the call between the first terminal 11 and the second terminal 12 according to the first embodiment can be Internet-based through the Internet-based server, and can also be made via the base station.

The flowchart shown in Fig. 3 shows an example of processing all the sound-text conversion functions in the first terminal 11 in the first embodiment.

Referring to FIG. 3, in step 31, the second terminal 12 receives a sound source (user's voice, etc.) from a user through a microphone or the like.

In step 32, the first terminal 11 receives the sound source from the second terminal 12.

In step 33, the call control unit 121 of the first terminal 11 converts the sound source received in step 32 into text using the speech recognition module 111.

In step 34, the display control unit 122 of the first terminal 11 displays the converted text in step 32 on the display unit of the first terminal 11.

In step 35, the display control unit 122 of the first terminal 11 displays on the display unit a user interface through which the user can input text, and receives text from the user through the user interface.

In step 36, the call control unit 121 of the first terminal 11 converts the text input in step 35 into a sound source using the speech synthesis module 112. [

In step 37, the call control unit 121 of the first terminal 11 transmits the sound source converted in step 36 to the second terminal 12.

In step 38, the second terminal 12 outputs the sound source received in step 37.

According to the first embodiment, the sound source-text conversion function is performed in the first terminal 11, and the first terminal 11 installs a service application providing a call service according to an embodiment of the present invention, The service application installed in the first terminal 11 can provide functions necessary for the call service.

On the other hand, in the first embodiment, the second terminal 12 does not perform the sound source-text conversion function and thus can not install the above-mentioned service application, and can communicate with the first terminal 11 through a normal telephone call method .

According to the first embodiment, the first terminal 11 outputs the sound source received from the second terminal 12 as text, receives the text from the user, converts the text, and provides the converted text to the second terminal 12, The user of the first terminal 11 can read the text displayed on the display unit and enter a text to make a telephone conversation with the second terminal 12. [

The call in the first embodiment can be applied to various types of communication such as an Internet telephone, a landline telephone, and a video call.

The steps shown in FIG. 3 may be repeatedly performed a plurality of times in the course of a call.

4 is a flowchart schematically showing a modification of the call service providing method according to the first embodiment. In particular, FIG. 4 shows an example of a case where a call between the first terminal 11 and the second terminal 12 is a video call in the embodiment of FIG.

In step 41, the second terminal 12 receives video from the user. The video may include a sound source component and an image component.

In step 42, the first terminal 11 receives video including a sound source component and an image component from the second terminal 12.

In step 43, the call control unit 121 of the first terminal 11 converts the sound source component included in the video into text using the voice recognition module 111. [

In step 44, the display control unit 122 of the first terminal 11 displays the image components included in the video on the display unit, and displays the converted text on the display unit in step 43. [

According to one embodiment, steps 43 and 44 may be performed separately in time, or may be performed in parallel. For example, the first terminal 11 can display the image components included in the video on the display unit, simultaneously convert the sound source components included in the video into text using the voice recognition module 111, Can be displayed on the display unit.

According to one embodiment, in step 44, the first terminal 11 can output the sound source component included in the video to the speaker, but the user of the first terminal 11 can preset whether to output the sound source.

In step 45, the first terminal 11 receives the video through the camera, and receives text input by the user through the user interface displayed on the display unit of the first terminal 11.

In step 46, the first terminal 11 converts the text input in step 45 into a sound source using the speech synthesis module 112.

In step 47, the first terminal 11 may transmit video to the second terminal 12. At this time, the video transmitted to the second terminal 12 may include the sound source converted in step 46 as a sound source component. In addition, the video may include an image component of the video captured at step 45 as an image component.

According to one embodiment, the sound source component of the video transmitted in step 47 may include both the sound source component of the video shot in step 45 and the sound source converted in step 46. [

In step 48, the second terminal 12 outputs the video received in step 47. The video component of the video can be output via the display, and the sound component can be output via the speaker.

5 is a flowchart schematically showing a call service providing method according to the second embodiment.

The communication between the first terminal 11 and the second terminal 12 in the flow chart shown in Fig. 5 may be via the server 20 as described above. The server 20 in this example includes a communication server, and may further include a base station. That is, the communication between the first terminal 11 and the second terminal 12 according to the second embodiment can be an internet telephone through an Internet-based server, or a combination of an Internet telephone and a wire telephone. In the embodiment shown in FIG. 5, the memory 110 and the call control unit 121 of FIG. 2 may be provided in the server 20, and the display control unit 122 may be provided in the first terminal 11. On the other hand, the second terminal 12 performs a normal telephone conversation in which a sound source is received and transmitted, and a sound source is received and output, so that a separate service application may not be installed.

Referring to FIG. 5, in step 51, the second terminal 12 receives a sound source (user's voice, etc.) from a user through a device such as a microphone.

In step 52, the server 20 receives the sound source from the second terminal 12.

In step 53, the call control unit 121 of the server 20 uses the voice recognition module 111 to convert the sound source received in step 52 into text.

In step 54, the server 20 transmits the converted text to the first terminal 11.

In step 551, the display control unit 122 of the first terminal 11 displays the text received in step 54 on the display unit of the first terminal 11.

In step 552, the display control unit 122 of the first terminal 11 displays on the display unit a user interface through which the user can input text, and receives text from the user through the user interface.

In step 56, the first terminal 11 transmits the text input in step 552 to the server 20.

In step 57, the call control unit 121 of the server 20 converts the text input in step 56 into a sound source using the speech synthesis module 112. [

In step 58, the call control unit 121 of the server 20 transmits the sound source converted in step 57 to the second terminal 12.

In step 59, the second terminal 12 outputs the sound source received in step 58 using a device such as a speaker.

 Meanwhile, it is also possible to modify the flowchart shown in FIG. 3 and apply it to the video call by modifying the flowchart shown in FIG. 5 in the same way as applied to the video call as shown in FIG.

6 is a flowchart schematically illustrating a method of providing a call service according to a case where the first terminal 11 and the second terminal 12 divide and process a sound source-text conversion function in the first embodiment.

6, the first terminal 11 and the second terminal 12 can communicate with the configurations of the communication service providing apparatus 100, for example, one or more modules included in the memory 110 and the call control unit 121 And a display control unit 122, and can process sound source-text conversion functions, respectively. To this end, service apps provided by the server 20 may be installed in the first terminal 11 and the second terminal 12, respectively, according to an embodiment of the present invention.

Referring to FIG. 6, in step 61, the second terminal 12 receives a sound source from a user via a microphone.

In step 62, the second terminal 12 converts the sound source into text using the speech recognition module 111. [

In step 63, the second terminal 12 transmits the converted text to the first terminal 11. The first terminal 11 receives text from the second terminal 12.

In step 64, the first terminal 11 displays the text received in step 63 on the display of the first terminal 11.

In step 65, the first terminal 11 receives text from the user.

In step 66, the first terminal 11 converts the text input in step 65 into a sound source using the speech synthesis module 112.

In step 67, the first terminal 11 transmits the converted sound source to the second terminal 12 in step 66.

In step 68, the second terminal 12 outputs the sound source received in step 67.

In the embodiment shown in FIG. 6, the data input / output format handled by the user at the second terminal 12 is a sound source, and the data input / output format handled by the user at the first terminal 11 is set to text. That is, the user of the second terminal 12 can talk to the microphone and listen to the talk of the other party through the speaker in a conventional manner, and the user of the second terminal 11 can confirm the story of the other party by text, You can enter the words you want to tell in text. Such settings may be specified by the user at the beginning of a call or through a preset in the service app.

For example, before step 61, when the first terminal 11 receives a call initiation request from the user with the second terminal 12, the first terminal 11 sends a call initiation request to the second terminal 12 And initiating a call by receiving the call initiation acknowledgment from the second terminal 12 by the first terminal 11, and the like. At this time, the call initiation request may include information on the data input / output format of the first terminal 11, and the information on the data input / output format may be designated as either text or sound source. The call initiation acknowledgment may include information about the data input / output format at the second terminal 12. In the example of FIG. 6, the first terminal 11 may transmit a call initiation request to the second terminal 12 specifying a data input / output format as text. The second terminal 12 can transmit the call initiation approval to the first terminal 11 in which the data input / output format is designated as the sound source. Hereinafter, a call in a state in which a data input / output format is specified as text is referred to as a text mode, and a call in a state in which a data input / output format is specified as a sound source is referred to as a sound source mode.

6 shows an example in which the format of data is converted and transmitted on the data transmitting side. However, the present invention can also provide an embodiment for converting the format of data on the data receiving side.

For example, step 62 may be performed at the first terminal 11, and step 66 may be performed at the second terminal 12. [ The first terminal 11 can receive the sound source from the second terminal 12 and convert it into text and display it on the display unit and the second terminal 12 can receive the sound source from the first terminal 11, And converts it into a sound source and outputs it to a speaker.

Figs. 7 to 13 show various examples of screens displayed on the terminal 10. Fig. Hereinafter, the call system of FIG. 1 and the function of the call service providing apparatus of FIG. 2 will be described in more detail with reference to FIGS.

FIG. 7 shows an example of a screen displayed on the first terminal 11.

7, the first terminal 11 is in video communication with the second terminal 12, the video component 71 of the video captured by the second terminal 12 is displayed, and the second terminal 12 12, the converted text 72 was displayed from the sound source component of the video.

Referring to FIG. 7, the text 72 is displayed, indicating that the first terminal 11 is using the text mode call service. 7, the call control unit 121 may provide a toggle function to select or cancel the text mode, and the display control unit 122 may display the toggle icon 73 on the screen. Through the icon 73, the first terminal 11 can alternatively provide a text mode for outputting data received in the course of a call in text format and a sound source mode for outputting through a speaker in a sound source format .

In FIG. 7, the icon 73 is clicked by the user, the text mode is selected, and the call is proceeding in the text mode. If the user deselects the icon 73 to release the text mode, the first terminal 11 is set to the sound source mode and does not display the text 72, Or the sound source component of the video photographed by the second terminal 12) to the speaker of the first terminal 11. In the sound source mode, the first terminal 11 can receive a sound source through the microphone of the first terminal 11 and transmit the sound source to the second terminal 12.

The image 74 photographed through the camera of the first terminal 11 can be displayed in one area of the screen.

The first terminal 11 may further display icons providing various functions. For example, an icon 75 for providing a handwriting mode for inputting text by directly writing a character using a touch screen or an electronic pen, and a keyboard (both a physical keyboard or a keyboard displayed on the touch screen) An icon 76 that provides a typing mode for typing text, and a call termination icon 77 may be provided. In addition, an icon 78 for providing a voice filter function, an icon 79 for loading stored text, an icon 710 for providing a translation function, and a flashing icon 711 can be provided.

In Fig. 7, the icon 710 is indicated as " A ", which indicates that the sound source received from the second terminal 12 is recognized in Korean and the current language is set to Korean and displayed on the screen. When the user selects the icon 710, the used language may be changed. For example, the user can select the icon 710 and change the language used to Japanese. In this case, the call control unit 121 converts the text converted from the sound source received from the second terminal 12 using the speech recognition module 111 into the first language of the converted text using the translation module 114, In a second language designated by the user in the language used by the user. The display control unit 122 can display the translated text in the second language on the display unit.

When the icon 711 is selected, the first terminal 11 can transmit a flashing-on command to the second terminal 12, and the second terminal 12 can turn on the flashing in a predetermined pattern according to the flashing- have. The user of the first terminal 11 can call the user of the second terminal 12 using the icon 711. [

8 is another example of a screen displayed on the first terminal 11. In FIG. 8 is an example of a screen displayed on the first terminal 11 when a general landline telephone is made rather than a video call. In this example, the second terminal 12, which is the other party of the call, can proceed with a normal wired telephone call.

Referring to FIG. 8, it can be seen that the typing mode is activated. When the typing mode is activated, the screen 80 displayed on the display unit of the first terminal 11 is divided into a first area 81 for displaying the text T2 converted from the sound source input from the second terminal 12, And a second area 82 for receiving text from a user of the terminal 11.

Meanwhile, the first area 81 may further display the text T1 recently input by the user of the first terminal 11. The text T1 may be converted to a sound source and transmitted to the second terminal 12.

The first area 81 can accumulate and display the text converted from the sound source input from the second terminal 12 and the text input from the first terminal 11 in chronological order.

The text converted from the sound source input from the second terminal 12 in the first area 81 and the text input from the first terminal 11 can be displayed in different formats.

9A is another example of a screen displayed on the first terminal 11. As shown in FIG.

Referring to FIG. 9A, the call control unit 121 accumulates and arranges, in chronological order, the text (T2) converted from the sound source input from the second terminal 12 and the text (T1) input from the first terminal 11 Can be displayed.

In FIG. 9A, the text T1 and the text T2 are displayed in different formats in order to distinguish the speakers. In the example of FIG. 9A, the text (T1) is distinguished from the text (T2) by applying an "italic" format to the text (T1) and bolding the text (T2) by applying a "bold" format. However, the present invention is not limited thereto. For example, the format of text color, shade, font, display position, alignment method (right alignment, or left alignment) can be specified differently.

According to an embodiment of the present invention, the call control unit 121 can determine a format for displaying the text (T2) converted from the sound source in accordance with the amplitude of the sound source. For example, the call control unit 121 can determine a format for displaying the first text T21 converted from the first sound source and the second text T22 converted from the second sound source, respectively, and the amplitude of the first sound source The format of the first text T21 corresponding to the amplitude of the second sound source and the format of the second text T22 corresponding to the amplitude of the second sound source can be determined. The display control unit 122 can display the text in a format determined in accordance with the amplitude of the sound source.

Determining the format corresponding to the amplitude of the sound source may be to determine the format so that the larger the amplitude of the sound source is, the larger the text is displayed. For example, if the amplitude of a sound source exceeds a predetermined threshold, the format may be determined to be greater than the default value. Alternatively, the at least one sound source amplitude section and the corresponding format may be set in advance, and the call control section 121 may determine the format of the text converted from the sound source in the format corresponding to the section corresponding to the amplitude of the sound source .

9A shows an example in which the amplitude of the second sound source exceeds a predetermined threshold value and the second text T22 is displayed larger than the remaining text.

Referring to FIG. 9A, the user of the first terminal 11 can infer the information included in the sound source, for example, the voice size, emotion, etc. of the speaker through the format of the text without directly listening to the sound source.

9B and 9C show another example of the screen displayed on the first terminal 11. 9B and 9C are examples of screens in which the accumulated dialog text is scrolled in response to the first input inputted to the first terminal 11 by the user in the state where the screen shown in FIG. 9A is displayed.

The first input may be a scroll input using a scroll bar (not shown), or a drag or flicking input that moves the touch point up / down while touching a point on the screen of FIG. 9A.

According to one example, when the user touches one point of the screen shown in FIG. 9A for a predetermined time or more, the first terminal 11 determines that the scroll operation has been started. As shown in FIG. 9B, 91) can be further displayed.

According to an example, when the user inputs dragging 92 or flicking to move the touch point up / down while maintaining the state of touching one point of the screen shown in FIG. 9A, the first terminal 11 It is possible to scroll the conversation history corresponding to the direction of the drag 92 (or flicking). In FIG. 9C, the drag 92 is input in the downward direction, and the conversation history is also scrolled down, so that the past conversation history is additionally displayed at the top.

8, the display unit of the first terminal 11 displays a first area 81 for displaying the text converted from the sound source received from the second terminal 12, The first terminal 11 is displayed only in the first area 81 when dragging is input in the first area 81. In this case, You can perform scrolling. Accordingly, even if scrolling is performed in the first area 81, the second area 82 can be fixedly displayed on the screen.

According to another example, when a drag is input in the first area 81, the display control part 122 of the first terminal 11 expands and displays the area 81 to the position where the second area 82 is displayed , It is possible to perform scrolling in the extended first area 81. [ According to this, only the first area 81 can be displayed on the screen after the dragging is inputted. The user can display the second area 82 again using an icon (handwriting input icon, typing input icon, etc.) on the screen.

10 is another example of a screen displayed on the first terminal 11. In FIG. 10 is an example of a screen that can be displayed when the icon 76 and the icon 78 are selected on the screen of Fig.

The first terminal 11 may display the keyboard 1001 on the touch screen and the second area 1002 in which the typed text is displayed corresponding to the selection of the typing function icon 76. [ Also, corresponding to the selection of the icon 78, the voice filter list 1003 can be displayed.

When a specific filter is selected in the list 1003, the first terminal 11 can transmit the sound source filter information selected by the user to the server 20. The server 20 applies the sound source filter corresponding to the sound source filter information to the sound source synthesized from the text input from the first terminal 11 (corresponding to " o " in Fig. 10) , To the second terminal 12. Here, the synthesis of the sound source can be processed in the first terminal 11 or the server 20. [ In this example, the first terminal 11 can store only the sound source filter list, and the actual filter data necessary for applying each sound source filter to the sound source can be stored by the server 20.

11 is another example of a screen displayed on the first terminal 11. In FIG. 11 is an example of a screen that can be displayed when the icon 79 is selected on the screen of Fig. The user can directly type the text to be input, as well as select the icon 79 to retrieve the pre-stored text.

The call control unit 121 of the first terminal 11 can provide the text that was previously stored in the mailbox when the icon 79 is selected. The text of the mailbox may be stored in the first memory 11 of the first terminal 11, or in the server 20. In FIG. 11, one text 1102 stored in the mailbox is displayed on the screen, and the user can inquire other text stored in the mailbox by using the scroll bar 1101 or dragging the text 1102 left / right. The user can select the text 1102 by touching and clicking, or by dragging the text 1102 onto the screen where the image is displayed at the top.

The first terminal 11 can receive the text stored in the first terminal 11 or the server 20 from the user. When the text is selected, the call control unit 121 converts the selected text into a sound source using the sound synthesis module 112 and provides the sound source to the second terminal 12.

12 is another example of a screen displayed on the first terminal 11. [ 12 is an example of a screen that can be displayed when the icon 75 is selected on the screen of Fig. In addition to typing the text to be input, the user can write directly using an electronic pen or a touch screen. Referring to Figure 12, the user has directly typed the word " yes " on the touch screen with one or more drag operations, and the handwritten characters 1201 are shown. The first terminal 11 can convert the handwritten character 1201 into text using the character recognition module 113 and display the converted text 1202 at the bottom of the screen. When the user selects transmission 1203, the call control unit 121 can convert the converted text 1202 into a sound source using the speech synthesis module 112 and transmit the converted sound to the second terminal 12. The user can exit the handwriting mode by selecting the close icon in the upper right corner.

13 is another example of a screen displayed on the first terminal 11. [

The call control unit 121 can store the call history after the call is terminated, and can provide call summary contents. 13 is an example of a screen in which call summary contents are displayed.

The call summary content may include one or more keywords selected from words included in the text accumulated during the call. The call control unit 121 may select one or more keywords among the words included in the accumulated text in consideration of at least one of the part of speech, frequency, and whether or not each word matches the predetermined format. The call control unit 121 may assign a score or a priority to each keyword according to the above-mentioned conditions, and may select a predetermined number of keywords in descending order of score or priority. For example, if a part of a word is a noun, it may be assigned a higher score or priority. The higher the frequency of the word, the higher the score or priority can be given. The call control unit 121 may determine at least one of the font, size, color, and display position of each keyword in consideration of at least one of the above listed conditions. For example, a noun may be determined in a first format, and a keyword having the highest frequency may be determined in a second format. Alternatively, the size of the keyword can be determined differentially according to the frequency. For example, the first style may be to apply "bold", to display in a specific color, or to apply a specific display effect (eg shading).

The call summary content may provide a summary image summarizing the call image. The summary image may include some of the entire call images. The summary image may be generated so as to include only a section in which the word selected as the keyword appears in the entire call image. The summary image may be generated so as to include only a section in which a predetermined number of keywords assigned with the highest score or the highest priority among the words selected as keywords among the entire call images. The method of generating the summary image is not limited thereto.

According to one example, the call control unit 121 can automatically provide the summary contents after the call is terminated. According to an example, the call control unit 121 can provide a list of stored call history, and the user can select a desired call history from the list and play back the call history again. On the other hand, when a specific call history is selected from the list, the call control unit 121 provides the call summary contents for the call history, so that the user can check the call summary contents first and then select whether to play the entire call history .

According to an embodiment of the present invention, since the contents of communication with the other party are inputted or outputted one or more times in the form of text, various services using such text can be additionally provided. The call summary content is an example of an additional service.

The medium may be one that continues to store computer executable programs, or temporarily store them for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a combination of a single hardware or a plurality of hardware, but is not limited to a medium directly connected to a computer system, but may be dispersed on a network. Examples of the medium include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floptical disk, And program instructions including ROM, RAM, flash memory, and the like. As another example of the medium, a recording medium or a storage medium managed by a site or a server that supplies or distributes an application store or various other software to distribute the application may be mentioned.

The present invention has been described with reference to the preferred embodiments. While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. It will be appreciated that other equivalent embodiments are possible. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

10: Terminal
11:
12:
20: Server
100: a call service providing device
110: Memory
120: Processor
111: Speech recognition module
112: speech synthesis module
113: Character recognition module
114: translation module
121:
122:

Claims (17)

A method for providing a call service in a first terminal,
Receiving a sound source from a second terminal;
Converting the received sound source into text using a speech recognition module;
Receiving the sound source, and converting the text into a plurality of times;
Accumulating and displaying the converted texts on a display unit of the first terminal;
Receiving text from a user;
Converting the received text into a sound source using a speech synthesis module; And
And transmitting the converted sound source to the second terminal,
And scrolling the accumulated text corresponding to a first input to the first terminal by the user.
The method according to claim 1,
Wherein the displaying comprises:
Displaying the text in a format corresponding to the amplitude of the received sound source,
Wherein the relationship between the amplitude and the format is such that the larger the amplitude is,
A method of providing a call service.
delete delete The method according to claim 1,
Wherein the screen displayed on the display unit of the terminal includes a first area for displaying the converted text and a second area for receiving text from the user,
Wherein the scrolling is performed in the first area and the second area is fixedly displayed even if the scrolling is performed,
A method of providing a call service.
The method according to claim 1,
Wherein the screen displayed on the display unit of the terminal includes a first area for displaying the converted text and a second area for receiving text from the user,
When the scrolling is performed, the first area extends to a position where the second area is displayed, and only the first area is displayed on the display unit
A method of providing a call service.
The method according to claim 1,
Selecting at least one of the words included in the text in consideration of at least one of a part of the words included in the accumulated text, a frequency of the words included in the accumulated text, and whether or not each of the words matches the predetermined format; More included
A method of providing a call service.
8. The method of claim 7,
Wherein said selecting and displaying comprises:
Determining at least one of a font, a size, a color, and a display position of each word in consideration of at least one of a part of the word, a frequency of the word, and whether or not each word matches the predetermined format
A method of providing a call service.
The method according to claim 1,
The first terminal,
A text mode for outputting data received in a call process in a text format on a display unit, and a sound source mode for outputting through a speaker in a sound source format,
The method comprises:
Further comprising: selecting the text mode by a user,
The receiving step or the transmitting step is performed after the selecting step,
When the sound source mode is selected by the user, the sound source received from the second terminal is output to the speaker, and the sound source received from the microphone of the first terminal is transmitted to the second terminal
A method of providing a call service.
The method according to claim 1,
The method of claim 1,
Selecting text stored in the first terminal or server from a user,
Wherein the step of converting the selected text into the sound source includes converting the selected text into a sound source using a sound synthesis module
A method of providing a call service.
A method for providing a call service in a first terminal,
Receiving a sound source from a second terminal;
Converting the received sound source into text using a speech recognition module;
Displaying the converted text on a display unit of the first terminal;
Converting a character input to the touch screen of the first terminal into at least one drag operation by using a character recognition module;
Converting the converted text into a sound source using a speech synthesis module using the character recognition module; And
And transmitting the converted sound source to the second terminal.
The method according to claim 1,
Wherein the displaying comprises:
Translating the converted text into a second language specified by the user in the first language using the translation module and displaying the translated text in the second language on the display unit
A method of providing a call service.
The method according to claim 1,
Wherein the transmitting comprises:
And transmitting the converted sound source and the sound source filter information selected by the user to the server so that the server can apply the sound source filter to the converted sound source and transmit the sound source filter to the second terminal.
A method of providing a call service.
A method for providing a call service in a first terminal,
Receiving a plurality of texts from a second terminal;
Accumulating and displaying the text on a display unit of the first terminal;
Receiving text from a user;
Converting the received text into a sound source using a speech synthesis module; And
And transmitting the converted sound source to the second terminal,
And scrolling the accumulated text corresponding to a first input to the first terminal by the user.
15. The method of claim 14,
Receiving a call initiation request with the second terminal; And
Further comprising: transmitting a call initiation request to the second terminal, receiving a call initiation acknowledgment from the second terminal, and initiating a call;
Wherein the call initiation request includes the output format information of the first terminal and the output format information is specified as either a text or a sound source,
A method of providing a call service.
delete A computer program stored on a medium for performing the method of any one of claims 1, 2, and 5 to 15 using a computer.

Images