WO2018216857A1

WO2018216857A1 - Conversation search method, conversation-searchable portable terminal, and conversation management server

Info

Publication number: WO2018216857A1
Application number: PCT/KR2017/010225
Authority: WO
Inventors: 이성노
Original assignee: 텍스토리 주식회사
Priority date: 2017-05-24
Filing date: 2017-09-19
Publication date: 2018-11-29
Also published as: KR20180128653A

Abstract

According to one aspect of the present invention, a conversation search method comprises the steps of: converting, into text data, voice data corresponding to conversation content of an image icon displayed on a portable terminal; storing the converted text data; and searching for an inputted search text from conversation content including the converted text data.

Description

Conversation search method, portable terminal and conversation management server capable of conversation search

The present invention relates to a conversation retrieval method, a portable terminal capable of a conversation retrieval, and a conversation management server. Specifically, an integrated retrieval can be performed for various voice conversations made between dialogues and various non-voice conversations made between dialogues. , A conversation search method, a portable terminal capable of a conversation search, and a conversation management server.

A user using a smartphone can talk to a specific other user through various channels. For example, a user may conduct a conversation with a specific user by using a voice call, a text, a mail, a messenger, and the like.

Smartphone users can receive and send various conversations with specific users through various conversation channels. For example, smartphone users can have voice conversations with other parties through voice calls, You can send and receive various conversations.

The contents, audio, video, etc. of messages transmitted and received may be stored through a program corresponding to each conversation channel. For example, a voice call program of a smart phone stores call history and even a voice recording in the form of a file, and various text, mail, and messenger programs display time (year, month, day, and time of various conversations transmitted and received through a corresponding communication channel). Save by). As such, the smart phone user may transmit and receive various conversations with a specific user through various conversation channels and store the corresponding conversation through a program of each conversation channel.

Smartphone users need to search for a variety of interspersed conversations. For example, when a user wants to confirm a conversation with another user, it is not easy for the user to find it in the recorded voice file. In addition, it is sometimes difficult for a user to even find out which of the various chat channels the conversation to be searched through. For example, sometimes you don't even remember whether a particular matter was through a text message or a voice conversation with a conversation partner.

As described above, there is a need for a conversation retrieval method, a portable terminal capable of conversation retrieval, and a conversation management server that can solve various problems caused by retrieving various conversations transmitted between specific users.

The present invention has been made to solve the above-described problems, and the conversation search method, conversation search to convert the voice file recording the conversation between the user to the text to enable the content search for the converted text file It is an object of the present invention to provide a portable terminal and a conversation management server.

In addition, the present invention maps the voice file and the converted text file on a block-by-block basis, a conversation retrieval method for enabling quick retrieval of a voice file and confirmation of a corresponding voice, a portable terminal capable of a conversation retrieval, and a conversation management server. The purpose is to provide.

Another object of the present invention is to provide a conversation retrieval method, a portable terminal capable of conversation retrieval, and a conversation management server, which enable integrated retrieval for conversations transmitted and received through various conversation channels with a conversation partner.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

The conversation retrieval method according to an aspect of the present invention comprises the steps of: a mobile terminal or a conversation management server converting voice data corresponding to the conversation content of an image icon displayed on the mobile terminal into text data; Storing the converted text data by the portable terminal; And searching, by the portable terminal or the chat management server, in the chat content including the converted text data.

In the above-mentioned conversation retrieval method, before converting the voice data into text data, the portable terminal transmits image icons corresponding to conversation contents transmitted and received to a single conversation partner designated through a plurality of programs according to the transmission and reception time. And displaying each of the image icons in a sequence of transmission / reception times of the respective conversation contents in different images according to the type and transmission / reception type of the program that generated the corresponding conversation content.

In the above conversation retrieval method, the plurality of programs includes two or more of a voice telephone program, a mail program, a text message program, and a text messenger program.

In the above-mentioned conversation retrieval method, before displaying the image icons, the portable terminal determines conversation contents transmitted and received from a single conversation partner designated from the plurality of programs through an integrated story management program stored in the portable terminal. And constructing an order list by mapping conversation contents and corresponding image icons to transmission and reception times.

In the above-mentioned conversation retrieval method, the retrieval step includes the conversation contents composed of the converted text data and the text and configured by a plurality of programs to perform a text search on the conversation contents transmitted and received with a designated single conversation partner. The search text input to the target is searched, and the conversation search method further includes the step of displaying, by the portable terminal, the conversation content or the image icon where the search text is found.

In the above-mentioned conversation retrieval method, in the configuration step, the integrated story management program transmits and receives a conversation content corresponding to the mobile telephone number of the designated single conversation partner from the plurality of programs and the transmission / reception type and transmission time at the time of loading the program. And the image icon is determined according to the type and the transmission / reception type of the program that has delivered the conversation content, and the determined image icon is mapped to the conversation content so that the conversation contents and the corresponding image icons received by the plurality of programs can be transmitted and received in order of transmission and reception time. Construct an ordered list to form a unified chat story for a single designated contact.

In the above-mentioned conversation retrieval method, after the retrieval step, the mobile terminal displays the converted text data; And reproducing the voice from the unit block corresponding to the user input by using the corresponding voice data according to the user input for the displayed converted text data by the portable terminal.

According to an aspect of the present invention, a portable terminal capable of searching for a conversation may include a storage unit configured to store an image icon and conversation contents; A display unit which displays the image icon and the conversation content; An input unit to receive a user input; And a controller configured to convert voice data of the conversation content corresponding to the image icon displayed according to the user input into text data, and to map the converted text data to the conversation content and to store the conversation content in the storage unit. The search text input through the input unit is searched for in the conversation content including the converted text data.

In the above portable terminal, the control unit before the conversion of the voice data into the text data, according to the transmission time and the image icons corresponding to the conversation contents transmitted and received for a single conversation partner designated through a plurality of programs The image icons are output to the display unit, and each of the image icons is displayed in the order of the transmission and reception times of the respective conversation contents in different images according to the type and transmission / reception type of the program generating the corresponding conversation contents.

In the above portable terminal, the storage unit further stores a plurality of programs capable of generating an integrated story management program and conversation contents, and the controller controls the mobile phone numbers from the plurality of programs through the integrated story management program. The conversation contents transmitted and received from the single conversation partner identified by the user are determined, and the order contents are constructed by mapping the conversation contents and the corresponding image icons according to the transmission and reception times.

In the portable terminal, the control unit, after the search, outputs the converted text data to the display unit and uses the voice data corresponding to the converted text data according to the user input to the converted text data through the input unit. The audio is played back from the unit block corresponding to the.

A conversation management server according to an aspect of the present invention includes a communication unit for transmitting and receiving data; A storage unit which stores the integrated conversation story for each designated conversation partner configured using a plurality of programs driven in the portable terminal; And a control unit which receives a processing request for the integrated conversation story of a designated conversation partner from the portable terminal through the communication unit and transmits a processing result to the portable terminal through the communication unit. According to the request for conversion of the voice data corresponding to the conversation content from the terminal, the voice data of the conversation content is converted into text data, and the converted text data is mapped to the conversation content in the integrated conversation story to be stored and processed in the storage unit. As a result, the converted text data is transmitted to the portable terminal, and the controller searches for the search text in the conversation contents including the converted text in the integrated conversation story according to a search request including the search text from the portable terminal. And search results Transmit to the portable terminal.

In the above conversation management server, the control unit is configured according to an integrated conversation story update request including a conversation content received from a portable terminal through a communication unit, a type of a program that generated the conversation content, a transmission / reception type, and a transmission / reception time. Create and add a new conversation item to the unified conversation story of the storage unit.

The conversation search method according to the present invention, a portable terminal capable of a conversation search, and a conversation management server convert a voice file recording a conversation between users into text so that a content search can be performed on the converted text file. It works.

In addition, the conversation search method according to the present invention, a portable terminal capable of a conversation search and a conversation management server map the voice file and the converted text file on a block-by-block basis to quickly select a voice file corresponding to the voice file. It has the effect of making it possible to confirm.

In addition, the conversation search method according to the present invention as described above, the portable terminal and the conversation management server capable of a conversation search to be integrated search for the conversations transmitted and received through various conversation channels with the conversation partner, and thus the conversation channel Even if you do not remember the effect can be effectively searched.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a diagram illustrating an example of a connection configuration between devices included in a conversation retrieval system according to the present invention.

2 illustrates an exemplary block diagram of a conversation management server.

3 is a diagram illustrating an exemplary block diagram of a portable terminal.

4 is a diagram illustrating an example of a control flow in which a conversation search is performed through content conversion.

5 illustrates an example of a screen configuration in which a story is displayed on a portable terminal.

The above objects, features, and advantages will become more apparent from the detailed description given hereinafter with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains may share the technical idea of the present invention. It will be easy to implement. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As can be seen in the example of FIG. 1, the conversation retrieval system includes one or more portable terminals 100, a conversation management server 200, and one or more conversation sharing terminals 300. The portable terminal 100, the conversation management server 200, and the conversation sharing terminal 300 may be connected through the internet network 400 to transmit and receive various conversation contents with other users.

Referring to the conversation search system through the example of FIG. 1, the portable terminal 100 is a terminal capable of receiving a conversation search service according to the present invention. The portable terminal 100 may be, for example, a mobile phone, a smartphone, a tablet PC, or the like.

The portable terminal 100 may perform a voice call through a mobile communication network (which may be separate from the Internet network 400 or may be included in the Internet network 400) or the Internet network 400, and various text messages, mails, It is configured to send and receive messenger messages. The portable terminal 100 includes a voice telephone program for performing a voice telephone call, a text message program for sending and receiving a text message, a mail program for sending and receiving mail, a messenger program for sending and receiving a messenger message, and a respective program. It is configured to deliver and receive voice calls, texts, mails, messenger messages, etc. to designated users.

In particular, the portable terminal 100 is provided with an integrated story management program (hereinafter referred to simply as a management program) to integrate with the same specific conversation partner by interworking with a voice call program, a mail program, a text message program, and a text messenger program. You can compose a conversation story.

Through the integrated story management program, the portable terminal 100 can arrange various conversation contents (hereinafter, also referred to as 'conversations') for a specific designated user in a line according to the transmission and reception times and output them to the user in a dialogue story form. And search for various conversations within the conversation story. The conversation story may be configured as a list of orders in which conversation contents (conversation items) transmitted and received by various programs for a single conversation partner identified by the user identifier are configured according to transmission and reception times of the conversation contents.

In particular, the portable terminal 100 is configured to enable text search for the conversation contents converted to text by converting text data into voice data according to voice call recording or reception of an audio file. Text conversion and search may be performed directly by the portable terminal 100 or by the conversation management server 200.

The portable terminal 100 will be described in more detail with reference to FIGS. 3 and 4.

The conversation management server 200 is a server capable of adding, deleting and managing conversation content between designated conversation partners in the portable terminal 100. The conversation management server 200 stores a conversation story about a conversation partner designated for each user of the portable terminal 100. The conversation story may be updated in conjunction with the portable terminal 100.

Furthermore, when the conversation content is composed of voice data, the conversation management server 200 converts the voice data into text data according to a conversion request from the portable terminal 100 and stores it in the conversation story and / or the portable terminal 100. ) Can be sent. In addition, the conversation management server 200 searches whether the search text of the search request exists in the conversation contents in the conversation story according to the search request from the portable terminal 100, and displays the search result in the portable terminal 100. Can be sent to. The search result may include an identifier of the conversation content (conversation item) identified within the conversation story and a location area (start position and / or end position) of the search text within the conversation content.

The conversation management server 200 will be described in more detail with reference to FIGS. 2 and 4.

The conversation sharing terminal 300 is a terminal capable of sharing the conversation content generated by the portable terminal 100. The conversation sharing terminal 300 may be, for example, a personal computer or a laptop. The conversation sharing terminal 300 may access the conversation management server 200 using a user ID and a password, and search for and download an integrated conversation story for one or more designated conversation partners who are stored for the user. It is possible to synchronize the conversation content with the portable terminal 100 or the conversation management server 200.

The internet network 400 is a network capable of transmitting and receiving an IP (Internet Protocol) communication packet. The internet network 400 may transmit and receive communication packets based on the Internet protocol, including a wired network, a wireless network, and / or a mobile communication network.

2 is a diagram illustrating an exemplary block diagram of a conversation management server 200.

According to FIG. 2, the conversation management server 200 includes a communication unit 201, a storage unit 203, a connection unit 205 and a control unit 207. 2 preferably shows a functional block diagram and each functional block has a corresponding hardware block. The conversation management server 200 of FIG. 2 may be configured using various hardware. For example, the conversation management server 200 may be configured using one or more PCs, workstations and / or mainframes. Other blocks not shown in FIG. 2 may be further included in this block diagram according to design variations.

Referring to the conversation management server 200 through FIG. 2, the communication unit 201 is provided with a chipset for wired LAN and / or Wi-Fi, and is provided with one or more portable terminals 100 and / or a conversation sharing terminal through the internet network 400. Send and receive data with the 300.

For example, the communication unit 201 receives various processing requests from the portable terminal 100 or the conversation sharing terminal 300 and transmits them to the control unit 207. In addition, the communication unit 201 may receive the processing result by the control unit 207 and transmit it to the portable terminal 100 or the conversation sharing terminal 300.

The storage unit 203 stores various data and programs, including a mass storage medium such as a volatile memory, a nonvolatile memory, and preferably a hard disk.

The storage unit 203 stores a plurality of user-specific user entries of the portable terminal 100 so as to collectively manage conversation stories for each user. The plurality of user entries may consist of a database.

Each user entry includes a user identifier (ID) and authentication data. The user identifier includes a user ID composed of a combination of numbers, letters, and / or special characters, a mobile phone number of the portable terminal 100 used by the user, and / or an e-mail address used by the user. Certain programs may use certain types of user identifiers and other programs may utilize other types of user identifiers. The mobile terminal 100 performing a voice call through the mobile communication network may access a specific user entry through the mobile phone number, and the mobile terminal 100 processing a mail may access a specific user entry through a mail address or the like. have. The authentication data may include password and / or biometric data such as iris data or fingerprint data consisting of a combination of numbers, letters and / or special characters.

In addition, the user entry includes a specified number of integrated conversation entries that store the integrated conversation story for each of the specified plurality of conversation partners. Each unified conversation entry includes a user identifier of a conversation partner. The user identifier may include a mobile phone number and further include a user ID or an e-mail address. The e-mail address can be determined directly by retrieval of the user entry and the mobile phone number can be entered by the user upon new setup of the unified conversation story.

The integrated conversation entry may be newly configured by the user, and after the new creation, the conversation content may be added to display a conversation history with a specific conversation partner according to a transmission / reception time of the conversation.

The integrated conversation entry is composed of a series of conversation items that divide various conversation contents made with the corresponding conversation partner according to transmission and reception times. One conversation item is data of the conversation content itself, a program type that can represent a specific program among various programs that can generate the conversation content, a type of transmission and reception indicating whether the conversation content has been output to or received from the conversation partner, transmission and reception of the conversation content. Types of conversation contents representing time, voice, text, image, video, etc. are stored as fields. The conversation item may further include converted conversation content converted from the specific conversation content.

The conversation content is composed of voice data such as a voice recording file or an audio file recorded at the time of telephone call or text data such as a text message, mail, messenger, and the like. The converted conversation content may be composed of text data converted from text data into text, for example, the conversation content converted from the original conversation content and its type.

Each conversation item may be ordered according to a transmission / reception time and may be displayed on the portable terminal 100 or the like.

In addition, the integrated conversation entry may be updated or changed under control by the control unit 207.

As such, the storage unit 203 stores the integrated conversation story for each designated conversation partner, and the integrated conversation story includes a plurality of programs (eg, a voice call program, a mail program, a text message program, and a text) driven by the portable terminal 100. Messenger program).

The connection unit 205 transmits and receives data between blocks in the conversation management server 200. The connection unit 205 is configured using a local area network, a parallel bus, a serial bus, or the like.

The control unit 207 includes one or more execution units to load a program or the like stored in the storage unit 203 into the execution unit and control the conversation management server 200 through execution of command codes of the programs.

Referring to the control process performed in the control unit 207, in response to a new user registration request from the portable terminal 100, the control unit 207 generates a new user entry in the storage unit 203 and portables the user identifier and authentication data. Set in conjunction with the terminal 100.

In addition, the control unit 207 creates an integrated conversation entry in the corresponding user entry and sets the user identifier of the conversation partner in accordance with the request for establishing the conversation partner from the portable terminal 100 of the specific user entry. The user identifier may be received as included in the setting request and may include one or more of a mobile phone number, a user ID (a combination of numbers, letters, and special characters) and a mail address.

After setting up the integrated conversation entry and the user entry, the control unit 207 receives a processing request for the conversation content of the designated conversation partner via the communication unit 201 and uses the user entry and the integrated conversation entry in the user entry. It is configured to process and transmit the processing result to the portable terminal (100).

For example, the control unit 207 receives a request for conversion of voice data corresponding to the conversation content from the portable terminal 100 via the communication unit 201. The conversion request may include or recognize the user identifier of the portable terminal 100 and the user identifier of the conversation partner, and includes a conversation item identifier for identifying the conversation content to be converted or the corresponding conversation content in the integrated conversation entry. The conversation item identifier may be a serial number, for example.

The control unit 207 converts the speech data of the conversation content into the text data using a known speech recognition algorithm, a speech-to-text conversion algorithm, or the like, and sends the communication unit 201 to the portable terminal 100 that has requested the conversion of the converted text data. Send it through. In addition, when the conversation item identifier is included in the conversion request, the control unit 207 identifies the conversation item in the integrated conversation entry of the user entry and stores the converted text data as the conversion conversation in the conversation item. In this way, the control unit 207 maps the converted text data to the conversation content requested to be converted and stores it in the storage unit 203.

The text data and voice data converted by the control unit 207 are configured to be mapped for each designated unit block so that their correspondence can be recognized. For example, the converted text data is divided into word units or sentence units. Each unit block is mapped to a specific location of the voice data so that the mapping relationship is transmitted to the portable terminal 100 or stored in the conversation item.

In addition, the control unit 207 receives a search request containing the search text from the portable terminal 100 via the communication unit 201. The search request is configured to include or recognize the user identifier of the portable terminal 100 and the user identifier of the conversation partner. The control unit 207 identifies the user entry and the integrated conversation entry using the user identifier and the conversation partner identifier.

The control unit 207 retrieves the search text in the conversation contents in the identified integrated conversation entry and transmits the search result to the portable terminal 100. The control unit 207 is configured to perform a search using the converted conversation in case it is not the same type of conversation content as the search text in the search process. The control unit 207 may search the search text for the converted text data corresponding to the voice data.

The search result may include the presence or absence of the search text, an identifier of one or more conversation items in which the search text exists, a start position and an end position in the conversation item, and the like.

In addition, the control unit 207 may receive the integrated conversation story update request to the portable terminal 100 through the communication unit 201. The integrated conversation story update request is configured to include or recognize the user identifier of the portable terminal 100 and the user identifier of the conversation partner.

The control unit 207 identifies the integrated conversation story (entry) using the user identifier of the integrated conversation story update request and the user identifier of the conversation partner, creates and adds a new conversation item to the integrated conversation story, and sets up a new conversation item. do. The control unit 207 may set the conversation content included in the integrated conversation story update request, the type of the program that generated the conversation content, the transmission / reception type, and the transmission / reception time as a new conversation item in the integrated conversation story and store it in the storage unit 203. have. The control unit 207 may order the conversation items of the integrated conversation story according to the transmission / reception time and store them in the order list form.

If the unified conversation story update request further includes a conversation content identifier (conversation item identifier), the control unit 207 receives the existing conversation item identified by the conversation content identifier, the conversation content received, the type of program that generated the conversation content, Updates can be made to the transmission / reception type, transmission / reception time and converted text data.

In addition, the control unit 207 receives the integrated conversation content download request from the portable terminal 100 via the communication unit 201. The integrated conversation content download request may be configured to include or recognize the user identifier of the portable terminal 100 and the user identifier of the conversation partner, and may further include a start transmission / reception time.

The control unit 207 identifies the integrated conversation story (entry) by using the user identifier of the integrated conversation content download request and the user identifier of the conversation partner, identifies the conversation items throughout the integrated conversation story or after the start transmission and reception time, and identifies the identified conversation. The items are sent to the portable terminal 100 in response.

In addition, the control unit 207 receives a synchronization request from the conversation sharing terminal 300 via the communication unit 201 ('synchronization' here includes partial or total sharing performed only at the request of a user). do). The synchronization request is configured to include or recognize the user identifier and the authentication data of the portable terminal 100.

According to the synchronization request, the control unit 207 identifies the user entry of the storage unit 203 by the user identifier and authenticates the user by the authentication data. If authenticated, the control unit 207 may transmit all the integrated conversation entries of the user entry to the requested conversation sharing terminal 300 to proceed with the synchronization of the conversation contents between the conversation sharing terminal 300 and the conversation management server 200. In this process, the control unit 207 may transmit only the chat content which does not exist in the chat sharing terminal 300 to the chat sharing terminal 300.

Through such a control process, the conversation management server 200 can integrate and manage conversation contents for each conversation partner, convert them into text data when necessary, integrate conversation contents scattered among various programs into one, and target the integrated conversation contents. Can provide services such as search.

3 is a diagram illustrating an exemplary block diagram of the portable terminal 100.

According to FIG. 3, the portable terminal 100 includes a communication unit 101, an input unit 103, an output unit 105, a storage unit 107, a display unit 109, a connection unit 111, and a control unit 113. do. 3 preferably shows a functional block diagram and each functional block has a corresponding hardware block.

Looking at each block of the portable terminal 100, the communication unit 101 transmits and receives various data through the Internet network 400 and / or mobile communication network. The communication unit 101 may include a communication chipset for wirelessly connecting to the Internet network 400 or a mobile communication network to transmit and receive wireless communication packets through the communication chipset. The mobile terminal 100 may transmit / receive voice calls, texts, data, image data, audio data, video data, and the like with the mobile terminal 100 of the conversation partner through the communication unit 101.

The input unit 103 receives a user input. The input unit 103 may include a button, a touch pad, a touch panel, and the like to receive a specific button input or a user input of a specific location and transmit the received user input to the controller 113.

The output unit 105 outputs data generated by the control unit 113. The output unit 105 includes an audio codec, a speaker, an earphone terminal, and the like, and is configured to output audio data generated by the control unit 113 as an audio signal (vibration signal). The output unit 105 may output a voice of a conversation partner of a voice call or a voice (audio) signal of a voice (audio) file (data).

The storage unit 107 stores various data and programs, including a nonvolatile memory and further a volatile memory. The storage unit 107 stores image icons and dialog contents for indicating the type of conversation contents and further stores various programs used in the present invention.

The image icons are configured to know which program among the plurality of programs and the image is set to distinguish whether the conversation content in the program is sent or received. The conversation contents may be stored separately for each conversation partner and may be ordered according to transmission and reception times of the conversation contents.

The programs stored in the storage unit 107 include an integrated story management program for managing an integrated conversation story for each conversation partner and a plurality of programs for generating conversation contents. The plurality of programs include, for example, a voice telephone program, a mail program, a text message program and a text messenger program, or consist of two or more combinations.

The display unit 109 includes an LCD display module, an LED display module, and the like, and outputs data output by the control unit 113 to the display module. For example, the display unit 109 may display a series of image icon and conversation content pairs output by the controller 113.

The connection unit 111 transmits and receives data between blocks in the portable terminal 100. The connection 111 may be, for example, a parallel bus or a serial bus. The connection unit 111 may be controlled by the control unit 113 to transmit and receive data to and from another block.

The controller 113 controls the blocks of the portable terminal 100. The control unit 113 may include one or more execution units capable of executing the command code of the program and control the blocks by loading the program of the storage unit 107. In the following description, the execution subject is mixedly displayed by the control unit 113 or the program.

Referring to a representative control process performed by the controller 113, each of a plurality of programs capable of generating chat content may be loaded into the controller 113 to transmit / receive chat content, and store the chat content that is transmitted / received in the storage unit 107. .

For example, the voice call program performs a voice call to the mobile party's mobile phone number through a mobile communication network, and the call start time, call end time, mobile phone number, recorded voice data (when voice recording through the input unit 103). ) Is stored in the manageable storage unit 107. Other mail programs, text messaging programs, and text messenger programs also include the contact's user identifier (email address, mobile phone number, user ID, etc.), the transmission time of the mail (message), and the conversation content (e.g., mail content, message content, etc.). ) Is stored in the storage unit 107.

In addition, the controller 113 may load the integrated story management program and execute the program to manage the integrated conversation contents for each conversation partner. In this way, the control unit 113 can manage the integrated conversation contents by executing the program, and will be described below as the subject of the program if necessary.

For example, the management program may be loaded into the controller 113 to set a user of the portable terminal 100 and set up a conversation partner. The setting of the conversation partner may be automatically performed according to registration of a telephone number or the like. In association with the conversation management server 200, the management program may newly register a user of the portable terminal 100 in the conversation management server 200. The management program may generate the user identifier and the authentication data through the input unit 103 and the like and include the same in a new user registration request to the conversation management server 200 through the communication unit 101 and receive a response of the registration request. . The user identifier may include a mobile phone number, a mail address and / or a user ID used in a plurality of programs.

The management program also registers the chat partner as the user is registered. According to the input of the user identifier of the contact person through the input unit 103, the management program constructs a contact setup request including the user identifier of the contact person and transmits the request to the chat management server 200 through the communication unit 101. do. The user identifier may include a mobile phone number, a mail address and / or a user ID used in a plurality of programs.

The management program can manage the integrated chat contents for a specific chat partner by receiving a response (processing result) to the chat partner setting registration. The management program stores the user identifier of the portable terminal 100 and the user identifiers of the conversation partner in the storage unit 107.

When the management program is loaded into the controller 113 or when a specific conversation partner is selected, the management program constructs an integrated conversation story for each conversation partner. For example, the manager may identify a conversation story of a selected (all) conversation partner in the storage unit 107 or configure an integrated conversation content download request and transmit it to the conversation management server 200, and configure the integrated conversation story in response. Receive conversation items.

In addition, the management program receives new conversation contents of each registered conversation partner from a plurality of programs each time it is loaded into the control unit 113 or whenever a new conversation partner is set. For example, the management program transmits the user identifier of the conversation partner and the update start time to the plurality of programs, and each of the plurality of programs transmits and receives the conversation contents transmitted or received to the corresponding user identifier after the update start time. To the management program.

The integrated story management program stores the conversation content, transmission / reception type, and transmission / reception time received from each of the plurality of programs for each conversation partner as the conversation item in the storage unit 107. The conversation item to be stored further includes an image icon mapped to the conversation content and determined according to the transmission / reception type and the program type, and a program type and a conversation content type indicating a program generating the conversation content. The conversation content type is received from each of the plurality of programs or analyzes the conversation content or is determined according to the type of program. The management program sets the conversation content as an image icon having a different image shape according to the program type and the transmission / reception type, and stores it in the conversation item.

The management program (control unit 113) adds the newly created conversation item to the order list of the existing conversation item. For example, the management program may configure the integrated conversation story update request and transmit the integrated conversation story update request to the conversation management server 200 or insert the integrated conversation story update request in the order list of the conversation items stored in the storage unit 107 at positions corresponding to the transmission and reception times. Through this process, the control unit 113 (management program) determines conversation contents transmitted and received from a single conversation partner identified by a user identifier such as a mobile phone number through a plurality of programs and displays image icons corresponding to the conversation contents. The order list may be configured by mapping according to transmission / reception times. The order list may be stored in advance in the storage unit 107 or received from the conversation management server 200.

The control unit 113 (management program) outputs a list of the conversation managers who are being managed together to the display unit 109 and according to the conversation partner selection through the input unit 103, the conversation contents that have been transmitted and received for a specific single conversation partner. The corresponding image icons are output to the display unit 109 according to the transmission and reception times.

For example, the controller 113 outputs, to the display unit 109, image data and an image icon indicating the conversation content of the conversation item of the order list determined for the selected conversation partner and ordered according to the transmission / reception time of the conversation items. do. Pairs of the conversation content and the corresponding image icon are displayed in order of transmission and reception time. The image data of the chat content may be the text representing the content of the chat (for example, the content of a text message (messenger)) or may be text representing the chat content (for example, 'call', 'receive mail', etc.).

The user of the portable terminal 100 selects the conversation content, and the management program may output the selected conversation content that has been transmitted and received at a specific time through the output unit 105 as a voice or through the new pop-up window to the display unit 109. have.

In addition, the management program (control unit 113) may convert the voice into text and provide an integrated search function for the integrated conversation story including the converted text. For example, the management program (control unit 113) receives a text conversion input for the displayed conversation content via the input unit 103. The text conversion input may be configured in various forms according to the configuration of the management program.

According to the text conversion input, the management program (control unit 113) determines the corresponding conversation item in the order list, converts the voice data of the conversation content of the conversation item into text data, and maps the converted text data to the conversation content to communicate. Update and save the item.

For example, the management program (control unit 113) converts the voice data directly into the text data and maps the text data converted as the conversation dialogue into the conversation item corresponding to the speech data to the conversation contents of the speech data to store the contents 107. ). The converted text data (including a conversation item) may be included in the integrated conversation story update request and transferred to the conversation management server 200 to update the corresponding conversation item.

Alternatively, the management program (control unit 113) may include a conversation item identifier for identifying a conversation item corresponding to the voice data to be converted and an identifier of a conversation partner, and further recognize a user identifier of the portable terminal 100. The conversion request may be configured and transmitted to the conversation management server 200, and the converted text data may be received as a processing result.

The voice data corresponding to the specific conversation content may be converted into text, stored in the conversation item, and further output to the display unit 109.

The management program (control unit 113) outputs the text data converted according to the user input through the input unit 103 to the display unit 109 through a new pop-up window. The converted text data is mapped with the voice data in units of designated blocks as described above with reference to FIG. 2.

The user may select the text data in units of blocks through the input unit 103 (for example, when a cursor is positioned on a specific character, the block including the character is highlighted), and the text block corresponding to the selected block of the text data may be selected. The voice data may be reproduced from the unit block corresponding to the user input using the voice data and output to the output unit 105. The speech reproduction process may be performed after the conversion of the text data, or may be performed after the conversion and retrieval of the text data.

In addition, the management program (control unit 113) may provide a search box for the search on the display unit 109 and receive the search text according to a search input (when text input is completed in the search box) through the input unit 103. have.

As the search input is received in the process of outputting the specific integrated conversation story to the display unit 109, the management program (control unit 113) searches the conversation contents in the specific integrated conversation story. Preferably, the management program (control unit 113) searches for the presence of the search text in the conversation contents including the converted text data, and highlights the search position on the display unit 109 according to the presence of the search text. Can be.

Search of the text may be performed directly by the portable terminal 100 or by the conversation management server 200. When made by the conversation management server 200, the management program (control unit 113) constructs a search request including the search text and transmits it to the conversation management server 200 through the communication unit 101 and receives a search result. can do.

The conversation contents in which the search text exists among the converted text data may be highlighted (highlighted) with an accent color, and the converted text data corresponding to the conversation contents may be separated by a user's selection through the input unit 103. Is displayed through.

The user can select text data in units of designated blocks, and the management program (control unit 113) retrieves the voice data corresponding to the selected text data from the conversation item (conversation) and decodes the retrieved voice data to the output unit 105. Output Accordingly, the user can check the converted text data and can check the voice data in which the specific search text exists as a voice signal in units of words or sentences.

The control flow of FIG. 4 is performed in collaboration of the portable terminal 100 and the conversation management server 200 and is performed by the control unit 113 of the portable terminal 100 and the control unit 207 of the conversation management server 200. . The control unit 113 and the conversation management server 200 perform the control flow through the provided program, and the control unit 113 performs the control flow through the integrated story management program. Hereinafter, the contents described in detail with reference to FIGS. 1 to 3 will be described below.

First, a user of the portable terminal 100 registers a new user through the conversation management server 200 and sets up a conversation partner after registering a new user (S10). The controller 113 may set various user identifiers in association with the conversation management server 200 through a new user registration request and a conversation partner setting request.

The management program (control unit 113) of the portable terminal 100 determines a conversation content for a registered conversation partner through a plurality of other programs, and constructs an ordered list in which the conversation content is ordered according to transmission / reception time (S20). )do.

For example, the management program (control unit 113) configures an integrated conversation story through a request for downloading the integrated conversation content for each conversation partner or retrieves the integrated conversation story for each conversation partner stored in the storage unit 107. You can compose the story. The unified conversation story is composed of a plurality of conversation items and each conversation item includes transmission and reception times so that they can be ordered or ordered according to transmission and reception times.

The configuration process S20 is performed when the control unit 113 of the management program is loaded or when a specific conversation partner is selected. The management program (control unit 113) can add a new conversation content to form an integrated conversation story.

For example, the management program (control unit 113) receives new conversation contents for each conversation partner identified by the mobile telephone number together with the transmission / reception time, transmission / reception type, and the like from each of the plurality of programs. The management program (control unit 113) determines an image icon according to the type and transmission / reception type of the program that delivered the conversation content, and maps the determined image icon and the received conversation icon to form one conversation item. The management program (control unit 113) may add a new conversation item to the existing integrated conversation story, and configure the integrated conversation story for a single conversation partner by composing the integrated conversation story in the order of transmission / reception time order.

The configuration of the integrated conversation story can be made directly by the management program (control unit 113) or in conjunction with the conversation management server 200. For example, the management program (control unit 113) may configure the integrated conversation story update request and transmit it to the conversation management server 200.

The portable terminal 100 displays the conversation contents and the corresponding image icons transmitted and received for a single conversation partner designated by a plurality of programs according to a selection input of a specific conversation partner according to the transmission and reception time (S30) (for example, FIG. 5). Is the same).

Specifically, the management program (control unit 113) maps the conversation items of the integrated conversation story to the transmission / reception time and includes the conversation contents (including an icon that can access the conversation contents) and the corresponding image icon in the ordered order list. These are sequentially displayed according to the transmission and reception time together with the transmission and reception time and output to the display unit 109. The previous conversation content is displayed first (previous position of the display unit 109) and then the conversation content is displayed later (down position in the display unit 109). The image data of the displayed conversation content may be text representing the content of the conversation or representative text of the conversation content.

Each of the image icons displayed in pairs together with the conversation contents (image data of) is configured with different image types according to the type of the program generating the conversation contents and the transmission / reception type, and are displayed together with the conversation contents in the order of transmission / reception time.

The user of the portable terminal 100 may select a specific conversation content through the input unit 103, check the type of the selected conversation content, and request conversion to text data when the conversation content is composed of voice data.

The portable terminal 100 or the conversation management server 200 converts the voice data corresponding to the image icon selected from the displayed image icons into text data according to the conversion request to the text data (S40). Or you can use the one already saved by the previous transformation.

For example, the portable terminal 100 may directly convert the voice data into text data, or the portable terminal 100 may configure a conversion request, and the conversation management server 200 may convert the voice data of the conversion request into text data. .

Thereafter, the portable terminal 100 stores the converted text data (S50). For example, the portable terminal 100 stores the converted text data in the local storage 107 and in the conversation management server 200.

After the conversion of the text data, the portable terminal 100 or the conversation management server 200 searches (S60) the search text input by the user on the series of conversation contents including the converted text data.

For example, the portable terminal 100 provides a search box to the user to perform a text search on the conversation contents that have been transmitted and received with a single conversation partner designated by the user.

The portable terminal 100 compares the input search text with the conversation contents of the integrated conversation story with the designated conversation partner configured in the storage unit 107 and determines whether the search text exists. Alternatively, the portable terminal 100 may construct a search request including the search text, transmit the search request to the conversation management server 200, and receive a search result.

The conversation contents that are searched are conversations generated by a plurality of programs for a single conversation partner. In addition, specific conversation contents are converted into text data and searched for the converted text data, thereby enabling integrated search for voice.

The search results received or determined by the portable terminal 100 may include an identifier of the conversation content (conversation item) identified within the unified conversation story and a location area (start position and / or end position) within the conversation content. have.

The mobile terminal 100 highlights the conversation content and / or the corresponding image icon where the search text is found using the search result and displays the result via the display unit 109 (S70).

The specific conversation content may be a conversation containing the converted text and the conversation content may be highlighted through text search. The user may select the conversation content including the text data converted through the user input, and the portable terminal 100 displays the converted text data through a pop-up window (S80).

The unit block including the search text or the search text among the converted text data may be highlighted.

The user of the portable terminal 100 may select the converted text data for each unit block, and the portable terminal 100 reproduces a voice corresponding to the user selection input using the corresponding voice data from the selected unit block (S90).

Through such a control flow, it is possible to integrate conversation content management for each conversation partner, and further unified conversation search is possible.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

Claims

Converting, by the portable terminal or the conversation management server, voice data corresponding to the conversation content of the image icon displayed on the portable terminal into text data;

Storing the converted text data by the portable terminal; And

And searching, by the portable terminal or the conversation management server, in the conversation contents including the converted text data.

How to search conversations.
The method of claim 1,

Before converting the voice data into text data, displaying, by the portable terminal, image icons corresponding to the contents of the conversation transmitted and received for a single conversation partner designated through a plurality of programs according to the transmission and reception time; ,

Each of the image icons is displayed in the order of the transmission and reception time of each conversation content in different images according to the type and transmission / reception type of the program that generated the corresponding conversation content.

How to search conversations.
The method of claim 2,

The plurality of programs includes two or more of a voice call program, a mail program, a text message program, and a text messenger program.

How to search conversations.
The method of claim 2,

Prior to displaying the image icons, the portable terminal determines the conversation contents transmitted and received from a single conversation partner designated from the plurality of programs through an integrated story management program stored in the portable terminal, and the conversation contents and the corresponding image icon. Comprising the step of mapping the transmission and reception times to construct a list; further comprising,

How to search conversations.
The method of claim 2,

In the searching step, the search text inputted to the conversation contents composed of the converted text data and the text and configured by a plurality of programs is searched for the text contents to be transmitted and received with the designated single conversation partner. ,

The conversation search method may further include highlighting, by the portable terminal, the conversation content or the image icon where the search text is found;

How to search conversations.
The method of claim 4, wherein

In the configuring step, the integrated story management program receives the conversation content corresponding to the mobile telephone number of the designated single conversation partner with the transmission / reception type and transmission time from each of the plurality of programs at the time of loading the program, and receives the conversation content. The image icon is determined according to the type of program and the type of transmission and reception, and the determined image icon is mapped to the conversation content. The conversation contents and corresponding image icons delivered by the plurality of programs are arranged in the order of transmission and reception times. To organize a unified conversation story for your partner,

How to search conversations.
The method of claim 1,

After the searching step, the portable terminal displaying the converted text data; And

Reproducing the voice from the unit block corresponding to the user input by using the corresponding voice data according to the user input for the displayed converted text data;

How to search conversations.
A portable terminal capable of a conversation search,

A storage unit which stores image icons and conversation contents;

A display unit which displays the image icon and the conversation content;

An input unit to receive a user input; And

And a controller configured to convert voice data of the conversation content corresponding to the image icon displayed according to the user input into text data, and map the converted text data to the conversation content and store the converted text data in the storage unit.

The control unit searches for the search text input through the input unit in the conversation content including the converted text data.

Portable terminal.
The method of claim 8,

Before converting the voice data into text data, the controller outputs image icons corresponding to the conversation contents transmitted and received to a single conversation partner designated through a plurality of programs to the display unit according to the transmission and reception time,

Each of the image icons is displayed in the order of the transmission and reception time of each conversation content in different images according to the type and transmission / reception type of the program that generated the corresponding conversation content.

Portable terminal.
The method of claim 9,

The storage unit further stores a plurality of programs capable of generating an integrated story management program and conversation content,

The control unit determines conversation contents transmitted and received from a single conversation partner identified by the mobile phone numbers designated by the plurality of programs through the integrated story management program, and maps the conversation contents and the corresponding image icons according to the transmission and reception times, and then order list. Consisting of,

Portable terminal.
The method of claim 8,

After the search, the control unit outputs the converted text data to the display unit and starts from the unit block corresponding to the user input using the voice data corresponding to the converted text data according to the user input for the converted text data through the input unit. To play,

Portable terminal.
A communication unit for transmitting and receiving data;

A storage unit which stores the integrated conversation story for each designated conversation partner configured using a plurality of programs driven in the portable terminal; And

And a control unit which receives a processing request for the integrated conversation story of a designated conversation partner from the portable terminal through the communication unit and transmits a processing result to the portable terminal through the communication unit.

The control unit converts the speech data of the conversation content into text data according to a request for converting the speech data corresponding to the conversation content from the portable terminal, and maps the converted text data to the conversation content in the integrated conversation story. Storing in the unit and transmitting the converted text data as the processing result to the portable terminal,

The control unit searches for the search text in conversation contents including the converted text in the integrated conversation story according to a search request including the search text from the portable terminal, and transmits a search result to the portable terminal.

Dialog management server.
The method of claim 12,

The control unit is new to the integrated conversation story of the storage unit in response to the integrated conversation story update request including the conversation content received from the portable terminal through the communication unit, the type of the program that generated the conversation content, the transmission / reception type, and the transmission / reception time. To create and add conversation items,

Dialog management server.