JP2008104080A - Method and system for linking data information and audio information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To link audio information accumulated in an audio accumulation server and data information accumulated in a data utilization server. <P>SOLUTION: An audio accumulation server records audio information received from a terminal device and stores a telephone number of the terminal device and an audio arrival time as audio associated information and a data utilization server receives from the terminal device terminal device transmission data containing a first time that is a time prior to the time at which the audio information is sent from the terminal device, a second time that is a time after the end of audio recording, and the telephone number, accesses the audio accumulation server to extract audio associated information which contains the same telephone number as the telephone number contained in the terminal device transmission data and contains an audio arrival time after the first time and prior to the second time, acquires audio information corresponding to the relevant audio associated information and correspondingly stores the relevant audio information and the data information in a utilization data storage means. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for integrating data information and audio information that are to be united from a mobile phone and stored in separate devices. is there.

The technological progress of mobile phones has been remarkable, and various information can be input or acquired by one mobile phone. Such information includes still images or moving images acquired from the camera provided in the mobile phone, GPS information (location information of the mobile phone) if the model has a GPS function, and further acquired from a microphone. Audio information.
By transmitting these pieces of information to a server connected to the network, it becomes possible to use these pieces of information for various purposes in the server. In particular, by transmitting audio information and image information to the server as an integrated unit, it is possible to construct an application that explains the image content using audio.

  For example, in Patent Document 1, environmental survey integrated data including photo data and audio data acquired using a portable PC is stored in a recording medium, and the integrated data is externally used using a mobile phone as necessary. A technique for transmitting data to a PC and performing data analysis or the like on the external PC is described.

In Patent Document 2, an image file and a voice file acquired by a mobile phone in an emergency are transmitted to a communication service provider server, and the communication service provider server creates an emergency contact mail with the image file and the voice file attached thereto. It is described that it is sent to the contact.
These technologies are based on the premise that a storage medium or the like having a certain storage capacity is mounted on a portable PC or a mobile phone, and that the information is stored in these storage media. However, a large-capacity storage medium is required to store data recorded for a long time, and if these data are sent to a specific server wirelessly, the amount of communication increases. The communication time will also increase. In order to avoid such a situation, if an upper limit is set for the recording time, the user gets an evaluation that it is difficult to use.
On the other hand, with respect to recording voice, it is necessary to record voice on a remote voice storage server instead of storing voice data in a mobile phone or the like as in Patent Document 1 or Patent Document 2. Services that can be heard according to the situation are widely spread.
For example, Patent Document 3 describes a recording service device that records voice information transmitted from a mobile phone and reproduces the recorded voice information in response to a request from a user of the mobile phone. Patent Document 4 describes a technique in which voice is input to a mobile phone, the voice is transmitted to a server, and the server records the voice.

Generally, when recording to a voice storage server using a mobile phone, a call is made to the voice storage server in the same manner as a voice call, and the call connection is made. Recorded in the voice storage server. When playing back the recorded voice, the voice storage server is called and the voice is played back from the voice storage server.
If a voice storage server is used, a large-capacity storage medium is not required for the mobile phone, so that voice recording / playback is possible even in a relatively low-priced mobile phone that does not have a large-capacity storage medium.

  However, since image information, text information, and the like are transmitted to a server (a general Web server) via a data communication path different from the voice communication path, the voice information and the image information are integrated. If an audio storage server such as the one described above is used to store audio in an application that needs to be handled as a group of audio information stored in the audio storage server, image information stored in the Web server, etc. It is necessary to perform association for recognizing the group.

  As a simple countermeasure against this problem, it is conceivable to give the same identification information (corresponding to ID) to both pieces of information. Usually, when recording the voice into the voice storage server, the caller together with the voice Only the telephone number and the recording time on the side are recorded, and in order to assign IDs other than these, it is necessary to modify the voice storage server.

As another countermeasure, the voice storage server records "caller number + voice arrival time", and the server records "caller number + data information outgoing time". A method for determining whether or not there is a relationship is possible. However, since the voice is transmitted by utterance and the data information is transmitted by the operation of the mobile phone, the incoming time of the voice and the outgoing time of the data information cannot match. In addition, even if a determination logic is adopted such that a time difference of a certain time (for example, a difference of 5 seconds) is regarded as the same time, the time difference between the voice transmission and the data transmission is not necessarily within the certain time difference. There is no guarantee. Therefore, when such a determination logic is adopted, it may occur that the voice information and the data information to be linked cannot be linked, or the irrelevant voice information and the data information are linked. It is not an effective method for judging the unity between information and data information.
JP 2002-352040 A Japanese Patent Laying-Open No. 2005-039700 JP 2001-177653 A JP 2004-336319 A

  The present invention has been made in view of the above points, and voice information stored in a voice storage server via a voice communication path from a mobile phone and a data utilization server from the mobile phone via a data communication path It is an object of the present invention to realize a technology for linking data information stored in a mobile phone without adding a large function to an existing voice storage server and mobile phone.

  The above problem is that a terminal device having a function of acquiring data information and transmitting it via a data communication path and a function of transmitting voice information via a voice communication path, and voice information received from the terminal apparatus are obtained. In a system comprising a voice storage server having a function of storing data and a data use server having a function of receiving and storing data information from the terminal device, the method combines the data information and the voice information, The terminal device acquiring a first time and storing the first time in a storage unit; the terminal device acquiring the data information based on a user operation; and storing the data information in the storage unit; A device performs a call connection with the voice storage server and transmits voice information from a user's utterance to the voice storage server; Storing the voice information received from the terminal device in the voice information storage means, and storing the telephone number and voice reception time of the terminal device in the voice related information storage means as voice related information in association with the voice information; The terminal device after the call connection with the voice storage server is terminated, the data information, the first time, and a second time that is later than the time when the call connection is terminated; Transmitting the telephone number of the terminal device to the data utilization server as terminal device transmission data, and receiving the terminal device transmission data and storing the terminal device transmission data in the terminal device transmission data storage means; The data utilization server accesses the voice storage server to obtain the same telephone number as the telephone number included in the terminal device transmission data. And the voice related information including the voice incoming time that is after the first time and before the second time is extracted, voice information corresponding to the voice related information is acquired, and the voice information is acquired. And the step of storing the data information in the utilization data storage means in association with each other. For example, a Web server can be used as the data use server.

  The present invention also has a function of accumulating voice information received from a terminal device having a function of acquiring data information and transmitting the data information via a data communication path and a function of transmitting voice information via the voice communication path. A voice storage server, and a data use server having a function of receiving and storing data information from the terminal device, the system having a function of linking the data information and the voice information, the voice storage server Makes a call connection with the terminal device, stores the voice information received from the terminal device in the voice information storage means, and associates the telephone number of the terminal device and the voice incoming time with the voice information to Means for storing the information in a voice related information storage means, and the data utilization server sends the data information and the voice information from the terminal device. Terminal device transmission data including a first time that is a time before the time at which the call is made, a second time that is a time after the call connection is terminated, and a telephone number of the terminal device. Means for receiving from the device and storing in the terminal device transmission data storage means; and accessing the voice storage server, thereby including the same telephone number as the telephone number included in the terminal device transmission data, and the first The voice related information including the voice incoming time that is later than the second time and before the second time is extracted, voice information corresponding to the voice related information is acquired, and the voice information and the data information are associated with each other. It can also be configured as a system characterized by having a means for storing in the utilization data storage means.

  According to the present invention, the voice information stored in the voice storage server via the voice communication path from the mobile phone is linked to the data information stored in the data utilization server via the data communication path from the mobile phone. For this reason, it is possible to realize the technology for this without adding a large function to the existing voice storage server and mobile phone.

  Embodiments of the present invention will be described below with reference to the drawings. In the following embodiment, image information (photograph), text information, and GPS information (these are referred to as data information) are transmitted from a mobile phone to a Web server, and voice is transmitted to a voice storage server. By linking information and data information, a screen in which location information based on GPS information is mapped on a map is displayed on a terminal accessing a Web server, and the user of the terminal clicks on the map It assumes a service that allows users to refer to photos and text information while listening to audio. In addition, it is assumed that the time of the mobile phone and the time of the voice storage server match to some extent accurately.

(System configuration)
FIG. 1 shows a system configuration in the embodiment of the present invention. As shown in FIG. 1, the system in the present embodiment includes a mobile phone 10, a voice storage server 20, and a Web server 30.

  The cellular phone 10 makes a call connection with the voice storage server 20 via the voice communication path 40 of the mobile network, and transmits voice to the voice storage server 20 via the voice communication path 40. In addition, the cellular phone 10 transmits data information to the Web server 30 via the data communication path 50 of the cellular network.

  The mobile phone 10 is a commercially available general device, and includes a GPS receiver 11, a camera 12, a character input unit 13, a voice input unit 14, a data storage unit 15, a data transmission / reception unit 16, a voice transmission / reception unit 17, and an antenna. 18 The GPS receiving unit 11 receives GPS information from a GPS satellite, the camera 12 takes a picture and acquires image information, and the character input unit 13 acquires text information based on a user operation and records it in the data storage unit 15. To do. The data information stored in the data storage unit 15 is transmitted to the Web server 30 by the data transmission / reception unit 16 via the data communication path 50.

  In addition, the voice input unit 14 acquires voice generated by the user's utterance, and the voice transmission / reception unit 17 transmits the acquired voice to the voice storage server 20 via the voice communication path 40. The voice transmission / reception unit 17 has a telephone function such as call connection.

  The mobile phone 10 is further provided with a function unit (CPU, memory, etc.) for executing an application program (hereinafter simply referred to as an application) for inputting / transmitting data information. A data information input transmission unit for inputting and transmitting data information is generated. Furthermore, the mobile phone 10 has a display unit (not shown) for performing screen display.

  The voice storage server 20 includes a voice information storage unit 21 that stores voice received from the mobile phone 10 as voice information. The voice storage server 20 includes a voice related information storage unit 22 that stores voice related information (caller telephone number and voice incoming time) in association with the voice information.

  The Web server 30 has a usage data storage unit 31 that acquires voice information to be associated with data information received from the mobile phone 10 from the voice storage server 20 and stores the data information and voice information. The Web server 30 has a map information storage unit 32 that stores map information. The map information may be acquired from an external map information server as necessary. Further, the Web server 30 has a function of providing a service to the user using the data stored in the usage data storage unit 31 and the map information stored in the map information storage unit 32.

(System operation)
Next, the operation of the system in the present embodiment will be described with reference to the flowchart of FIG.

  First, the cellular phone 10 activates an application based on an operation by the user, and the data information input transmission unit generated by the activated application saves the time when the application is activated (referred to as application activation time) in the data storage unit 15 (step S1). M1).

  The data information input transmission unit displays a screen for supporting the user to input the data information on the display unit of the mobile phone 10, and the user takes a picture, inputs a text according to the information shown on the screen, An operation for acquiring GPS information is performed. The image information, text information, and GPS information obtained by these operations are stored in the data storage unit 15 by the data information input / transmission unit (step M2).

  A button for making a call to the voice storage server 20 is displayed on the screen. When the user depresses (clicks) the button, the data information input transmission unit performs a call process to the voice storage server 20. Perform (step M3). Thereafter, the execution of the application is suspended. That is, it becomes impossible to input data using an application. Further, the application may be configured such that data information cannot be transmitted before the call is made to the voice storage server 20.

  When the call connection to the voice storage server 20 is completed, the user of the mobile phone 10 utters the content related to the data information input in step 2 according to the guidance and the like flowing from the voice storage server 20, and the spoken voice is voice communication It is transmitted to the voice storage server 20 via the path 40 (step M4).

  On the other hand, the voice storage server 20 connected to the mobile phone 10 in step V1 stores the caller telephone number (phone number of the mobile phone 10) and the voice incoming time in the voice related information storage unit 22 as voice related information. (Step V2). The caller telephone number may be notified using a caller telephone number notification function in the mobile network, or may be notified by the mobile phone 10. The voice incoming time is the time that the voice storage server 20 acquires when voice recording is started.

  The voice storage server 20 stores (records) the voice transmitted from the mobile phone 10 in the voice information storage unit 21 (step V3). Note that the voice related information including the caller telephone number and the voice incoming time stored in the voice related information storage unit 22 and the voice information stored in the voice information storage unit 21 are associated with each other. For example, this association can be performed by including identification information (such as a file name) of the audio information in the audio related information.

  On the mobile phone 10 side, when the voice input is finished, the call is disconnected (call connection is finished), the call function in the mobile phone 10 is finished, and then the application is restarted (re-executed). The input transmission unit operates (step M5).

  The data information input transmission unit displays a screen including a data transmission button on the display unit. When the user depresses (clicks) the data transmission button displayed on the screen, the data information input / transmission unit stores the depressing time (referred to as data transmission time) in the data storage unit 15 (step M6). Further, the data information input / transmission unit transmits the data information input / saved in step M 2, the application start time, the data transmission time, and the telephone number of the mobile phone 10 to the Web server 30 via the data communication path 50. Transmit (step M7).

  The Web server 30 receives the data information transmitted from the mobile phone 10, the application start time, the data transmission time, the telephone number of the mobile phone 10 (collectively referred to as mobile phone transmission data), and the Web server 30 The data is stored in a primary data storage unit (RAM or the like) of the server 30 (step W1).

  Then, the Web server 30 refers to the voice related information storage unit 22 in the voice storage server 20, has the same caller telephone number as the caller telephone number included in the mobile phone transmission data received in Step W1, and receives a voice call. The voice related information whose time is between the application start time and the data transmission time is extracted (step W2). Then, the Web server 30 acquires audio information corresponding to the audio related information, and stores it in the usage data storage unit 31 together with the data information received in Step W1 (Step W3). The reason why the voice information corresponding to the voice related information extracted in the process of step W2 and the data information received in step W1 may be regarded as one body will be described below.

  FIG. 3 shows the processing in the case of transmitting and storing data and voice in time series according to the flowchart shown in FIG.

  That is, the application is first activated (application activation time: T1). After data input / editing, a call is made to the voice storage server, and voice recording is started (voice reception time: T2). Then, after the voice recording is completed, the application is re-executed along with the termination of the call function, transmission of data information is started by pressing the transmission button (data transmission time: T3), and data transmission is terminated.

  If the application is not activated, no call is made to the voice storage server 20, so that T1 <T2. In addition, since data transmission is not performed unless the voice recording is completed, T2 <T3 is always satisfied. Therefore, T1 <T2 <T3 always holds. Also, since data input / editing and calling to the voice storage server 20 are continuously performed using the same application, voice recording related to other data information of the same user is recorded between T1 and T3. Never done.

  Therefore, it has the same caller telephone number as the caller telephone number included in the mobile phone transmission data, and the voice incoming time T2 corresponds to the voice related information existing between the application start time T1 and the data transmission time T3. The voice information can be handled as one piece with the data information included in the mobile phone transmission data.

  Note that T1 may be any time before the voice incoming time T2, and is not necessarily the application activation time. Further, T3 may be a time after the voice incoming time T2, and is not necessarily the data transmission time (time when the transmission button is pressed). Further, T2 may be a time between T1 and T3, and does not necessarily need to be a time when voice recording is started. For example, it may be a time during voice recording or a time when voice recording ends.

  An example of processing after step W3 in FIG. 2 will be described next. The Web server 30 acquires map information including the position indicated by the GPS information based on the GPS information included in the data information stored in the usage data storage unit 31, includes an image indicating the position, and clicks on the position Sometimes, a web page that displays a photograph and text information and outputs a voice is created and stored in the usage data storage unit 31 in association with user identification information (user ID, telephone number, etc.). Then, the web server 30 transmits the web page to the user terminal in response to access from the user terminal. On the user terminal, a map image including an image indicating a position associated with the data information is displayed (FIG. 4A), and by clicking on the position indicated by the image indicating the position, the photograph and text information are displayed. Is displayed (FIG. 4B), and the user can listen to the voice.

(Detailed configuration of each device)
Next, the functional configuration of each device that performs the above operation will be described in more detail.

  FIG. 5 shows a functional configuration of the data information input transmission unit of the mobile phone 10 when the application is executed. As shown in FIG. 5, the data information input / transmission unit includes a time acquisition unit 1, a data information input / editing unit 3, a calling unit 5, and a data information transmission processing unit 7.

  The time acquisition unit 1 acquires the time from the clock function unit included in the mobile phone 10 when the application is activated and when the transmission button is pressed, and records the time in the data storage unit 15. The data information input / editing unit 3 has a function of acquiring an image obtained by photographing and recording it in the data storage unit 15 and a function of displaying a text input screen and recording the input text in the data storage unit 15 The GPS receiving unit 11 has a function of recording GPS information received in the data storage unit 15. The time information, the image information, the text information, and the GPS information are recorded in the data storage unit 15 in association with each other.

  The calling unit 7 makes a call for making a call connection to the voice storage server 20 based on an operation from the user. In addition, the data information transmission processing unit 5 has a function for transmitting time information and information obtained by adding a caller telephone number to the data information to the Web server 30 via the data communication path 50 in response to pressing of the transmission button. Have.

  FIG. 6 is a functional configuration diagram of the voice storage server 20. The voice storage server 20 includes a telephone function unit 23, a voice storage processing unit 24, a Web server cooperation processing unit 25, a voice related information storage unit 22, and a voice information storage unit 21.

  The telephone function unit 23 is a function unit for performing call connection with another device through the voice communication path 40 and transmitting and receiving voice. The voice accumulation processing unit 24 is a processing unit for storing the voice incoming time, the caller telephone number, and the voice identification information in the voice related information storage unit 22 in association with each other. The voice accumulation processing unit 24 performs processing for storing the voice received from the telephone function unit 23 in the voice information storage unit 21 as voice information.

  The Web server cooperation processing unit 25 has a function of receiving access from the Web server 30 and causing the Web server 30 to search the audio related information storage unit 22 and transmitting audio related information and audio information to the Web server. ing.

  The voice storage processing unit 24 and the Web server cooperation processing unit 25 can be realized by executing a program for voice storage and a program for Web server cooperation processing on the voice storage server 20, respectively.

  FIG. 7 is a functional configuration diagram of the Web server 30. The web server 30 includes a data communication function unit 33, a web data processing unit 34, a voice storage server cooperation processing unit 35, a primary data storage unit 36, a usage data storage unit 31, and a map information storage unit 32.

  The data communication function unit 33 is a function unit for performing data communication with other devices via the data communication path 50. The Web data processing unit 34 receives the mobile phone transmission data from the mobile phone 10 and stores it in the primary data storage unit 36, the data information received from the mobile phone 10, and the voice storage server cooperation processing unit 35 4 (a) and 4 (b) using the function of storing the voice information to be integrated with the data information in the usage data storage unit 31 and the data stored in the usage data storage unit 31. It has a function to provide such services.

  The voice storage server cooperation processing unit 35 receives the application start time, data transmission time, and caller telephone number from the Web data processing unit 34, and accesses the voice storage server 20 to access the application start time, data transmission time, and caller. The voice related information storage unit 22 is searched using the telephone number, the voice related information having the same caller telephone number as the caller telephone number, and the voice incoming time is between the application start time and the data transmission time. It has a function to extract information, acquire the voice information from the voice information storage unit 21 using the voice identification information included in the voice related information, and pass it to the Web data processing unit 34.

  The voice storage server cooperation processing unit 35 acquires all the voice related information having the same caller telephone number, the Web data processing unit 34 stores the voice related information in the primary data storage unit 36, and the Web data processing The unit 34 may extract the voice related information corresponding to the target data information from the voice related information stored in the primary data storage unit 36 using the application activation time and the data transmission time. In this case, the Web data processing unit 34 acquires the identification information of the voice information from the extracted voice related information, and uses the identification information to send the voice information from the voice storage server 30 via the voice storage server cooperation processing unit 35. To get.

  The Web data processing unit 34 and the voice storage server cooperation processing unit 35 can be realized by executing a program for Web data processing and a program for voice storage server cooperation processing on the Web server 30, respectively.

  FIG. 8 shows an example of data stored in the usage data storage unit 31. In the example shown in FIG. 8, an image file, a sound file, text information, latitude / longitude information (GPS information), caller information (ID, etc.), and the like are stored for each chunk.

  By using the technology described in the present embodiment, for example, a service can be easily provided in which a photograph is sent to a Web server and the contents of the photograph are supplemented by voice. In addition, by including GPS information in the data information and performing mapping to the map as described above, an answering machine function that does not need to explain where the call is made can be realized. In addition, since the technology in this embodiment can be realized with a commercially available mobile phone without a special device, it is also possible to expand the application range to create a hazard map using live information by collecting information from the general public. is there.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

It is a system configuration figure in an embodiment of the invention. It is a flowchart which shows operation | movement of the system in embodiment of this invention. It is the figure which showed the flow of the process in time series. It is a figure for demonstrating an example of the service provided. 3 is a functional configuration diagram of a data information input transmission unit of the mobile phone 10; FIG. 3 is a functional configuration diagram of a voice storage server 20. FIG. 2 is a functional configuration diagram of a Web server 30. FIG. 4 is a diagram illustrating an example of data stored in a usage data storage unit 31. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile telephone 11 GPS receiving part 12 Camera 13 Character input part 14 Voice input part 15 Data storage part 16 Data transmission / reception part 17 Voice transmission / reception part 18 Antenna 20 Voice storage server 21 Voice information storage part 22 Voice related information storage part 23 Telephone function part 24 voice storage processing unit 25 Web server cooperation processing unit 30 Web server 31 usage data storage unit 32 map information storage unit 33 data communication function unit 34 Web data processing unit 35 voice storage server cooperation processing unit 36 primary data storage unit 40 voice communication path 50 Data communication path

Claims (2)

A terminal device having a function of acquiring data information and transmitting it via a data communication path and a function of transmitting voice information via a voice communication path; and a function of accumulating voice information received from the terminal device In a system comprising a voice storage server and a data use server having a function of receiving and storing data information from the terminal device, a method for linking the data information and the voice information,
The terminal device acquiring a first time and storing it in a storage means;
The terminal device acquires the data information based on a user's operation and stores it in the storage means;
The terminal device performs a call connection with the voice storage server, and transmits voice information from a user's utterance to the voice storage server;
The voice storage server stores voice information received from the terminal device in voice information storage means, and associates the telephone number and voice reception time of the terminal device with the voice information as voice related information. Storing in the storage means;
The terminal device, after the call connection with the voice storage server is terminated, the data information, the first time, a second time that is later than the time when the call connection is terminated, Transmitting the telephone number of the terminal device to the data utilization server as terminal device transmission data;
The data utilization server receiving the terminal device transmission data and storing it in the terminal device transmission data storage means;
The data utilization server includes the same telephone number as the telephone number included in the terminal device transmission data by accessing the voice storage server, and is later than the first time and the second time Extracting voice related information including a voice incoming time that is earlier, obtaining voice information corresponding to the voice related information, and storing the voice information and the data information in association with each other in the usage data storage unit; A method characterized by comprising: A voice storage server having a function of storing voice information received from a terminal device having a function of acquiring data information and transmitting the data information via a data communication path and a function of transmitting voice information via the voice communication path; A data use server having a function of receiving and storing data information from the terminal device, and a system having a function of linking the data information and the voice information,
The voice storage server performs call connection with the terminal device, stores voice information received from the terminal device in voice information storage means, and associates the telephone number and voice reception time of the terminal device with the voice information. In addition, it has means for storing in the voice related information storage means as voice related information,
The data utilization server includes a first time that is a time before the time when the data information and the voice information are transmitted from the terminal device, and a second time that is after the time when the call connection is terminated. Means for receiving terminal device transmission data including the time of the terminal device and the telephone number of the terminal device from the terminal device, and storing it in the terminal device transmission data storage means;
By receiving the voice storage server, an incoming voice call that includes the same telephone number as the telephone number included in the terminal device transmission data and that is later than the first time and before the second time Extracting voice related information including time, acquiring voice information corresponding to the voice related information, and storing the voice information and the data information in association with each other in the usage data storage means, System.

Images