JP2006080850A - Communication terminal and its communication method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily extract intended information regardless of a large amount of information to be retrieved. <P>SOLUTION: An information storage processing part 102 performs processing about information storage. An information storing part 103 stores information to be stored by the information storage processing part 102. An opposite communication party information acquiring part 104 acquires information about the opposite communication party during communication. A stored information retrieving part 105 retrieves the information stored in the information storing part 103 with the information about the opposite communication party acquired by the opposite communication information acquiring part 104 as key information and retrieves and extracts stored information deeply related to the opposite communication party. An extracted information reproducing part 106 reproduces the retrieved information deeply related to the opposite communication party extracted by the stored information retrieving part 105 through an outputting part 107. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a communication terminal that performs communication such as a videophone and a telephone and a communication method thereof, and in particular, stores received information such as image information of a communication partner during communication such as a videophone and retrieves the stored information. The present invention relates to a communication terminal and a communication method thereof.

  As a support device for making conversation more enjoyable when communicating using a videophone, for example, a co-view screen phone system has been proposed (for example, Patent Document 1).

The above conventional methods generally have different information held by each terminal. Therefore, before starting a call, the information stored in the database held by each terminal is unified into the same state. Thus, it is a support device that smoothly advances a conversation by searching information in a database and displaying data obtained by the search on each terminal.
Japanese Patent Laid-Open No. 3-49385

  However, although the above-described conventional method describes a search method, there is a problem that an information storage method for quickly and efficiently performing a search is not considered.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a communication terminal capable of easily extracting intended information and a communication method thereof even when the amount of information to be searched is large. .

  The communication terminal of the present invention includes an information storage unit that stores reception information including at least image information received from a communication partner, and communication partner information acquisition that acquires communication partner information related to the reception information including the image information from the communication partner. And an information storage processing unit that stores the received information including the image information received from the communication partner and the communication partner information acquired by the communication partner information acquisition unit in association with each other.

  According to this configuration, since the reception information including the image information received from the communication partner is stored in association with the communication partner information, the reception information can be searched with reference to the communication partner information. Even if the amount is large, intended information can be easily extracted.

  The communication method of the present invention includes an information storage step for storing in a memory reception information including at least image information received from a communication partner, and a communication partner for acquiring communication partner information related to the reception information including the image information from the communication partner. An information acquisition step, and an information storage processing step for storing in the memory the received information including the image information received from the communication partner and the communication partner information acquired in the communication partner information acquisition step in association with each other. I tried to do it.

  According to this method, since the reception information including the image information received from the communication partner and the communication partner information are stored in association with each other, the reception information can be searched with reference to the communication partner information. Even if the amount is large, intended information can be easily extracted.

  According to the present invention, even if the amount of information to be searched is large, intended information can be easily extracted.

  The main point of the present invention is that when storing received information including at least image information received from a communicating partner, information relating to the communicating partner is stored in association with the received information including the image information, and the stored information is searched. At this time, the information stored in association with each other is used as search information, thereby enabling quick information search.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
1 to 7 are diagrams for explaining the first embodiment.

  First, the configuration of the first embodiment will be described.

  In the communication system according to the present invention, as shown in FIG. 2, a plurality of communication terminals (201 to 203) are connected via a communication network 220.

  The communication terminals (201 to 203) of the present invention are, for example, mobile phones, and the communication terminals (201 to 203) are connected to the communication network 220 via base stations (211 to 213).

  As illustrated in FIG. 1, the communication terminal 100 includes a communication unit 101, an information storage processing unit 102, an information storage unit 103, a communication partner information acquisition unit 104, a stored information search unit 105, and an extracted information reproduction unit 106. And an output unit 107.

  The communication unit 101 has a function of performing communication such as a videophone, and performs outgoing / incoming call connection and transmission / reception of data such as video information, sound information, and text-based information.

  The text base information is arbitrary data information and includes displayed data and control information.

  The information storage processing unit 102 performs processing related to storage of information, such as processing of storing a playback image on a videophone, processing of storing transmitted / received mail, and processing of storing user information in an address book.

  The information storage unit 103 stores information to be stored by the information storage processing unit 102.

  The information storage unit 103 includes a memory built in each of the communication terminals 201 to 203, specifically, an electrically erasable nonvolatile memory (EEPROM), a readable / writable volatile memory (SRAM or DRAM), a magnetic Either memory (Hard Disk) can be used. The information storage unit 103 may be an external storage device that is detachable from the communication terminals 201 to 203, such as a card-type memory or a stick-type memory. Furthermore, a storage device of a network server constructed by a wireless network or a wired network can be used as the information storage unit 103.

  The communication partner information acquisition unit 104 acquires information regarding a communication partner during communication.

  Information related to the communication partner is identification information for uniquely identifying the communication partner. Information related to the communication partner includes personal information including preference information, information related to the location of the communication partner terminal, and information related to time. You may make it include the information concerned.

  The stored information search unit 105 searches the information stored in the information storage unit 103 using the information related to the communication partner acquired by the communication partner information acquisition unit 104 as key information (search key information), and relates to the communication partner. Search and extract deeply stored information (searched information).

  The extracted information reproduction unit 106 reproduces the searched information closely related to the communication partner extracted by the stored information search unit 105 via the output unit 107.

  The output unit 107 is configured by a display or a speaker that displays an image.

  Therefore, the extracted information reproduction unit 106 causes the output unit 107 to display the extracted image information on the display, and reproduces the extracted audio information on the speaker.

  Accordingly, the communication terminals 201 to 203 respectively include the communication unit 101, the information storage processing unit 102, the information storage unit 103, the communication partner information acquisition unit 104, the storage information search unit 105, the extracted information reproduction unit 106, and the like illustrated in FIG. An output unit 107 is provided.

  Next, a communication method in the communication system shown in FIG. 2 will be described with reference to FIGS.

  In the following description, when the communication terminal 201 communicates with the communication terminal 202 and the communication terminal 201 stores information including image information from the communication terminal 202, the communication terminal 201 acquires information (person information) about the communication partner during communication. An example will be described in which when the information including the received image information is stored, the acquired personal information of the communication partner is stored as attribute information in association with the received image information.

  Therefore, when searching and extracting the storage information stored in the information storage unit 103 of the communication terminal 201, the search can be quickly performed by using the person information (attribute information) as the key information.

  (1) First, a sequence for storing together video information of a communication partner currently communicating with a videophone and personal information related to the video information will be described with reference to FIG.

  The communication terminal 201 makes a connection request by videophone to the communication terminal 202 (step S301).

  The communication terminal 202 makes a permission response to the connection request via the videophone (step S302).

  After the connection process described above, data transmission settings for videophone communication are performed.

  In the communication system of the present invention, ITU-T recommendation H.264, which is a standard method for multiplexing multimedia information, is used for videophone communication. The 3G-324M protocol of the 3rd generation mobile phone standard using data multiplexing of H.223 is used.

  In accordance with the 3G-324M protocol, the communication terminal opens a data transmission channel capable of transmitting arbitrary data in addition to a video channel for transmitting video information and an audio channel for transmitting sound information.

  The communication terminal becomes capable of videophone communication by opening the 3G-324M protocol (step S303). At this point, the mutual video display and voice conversation are possible between the mutual communication terminals.

  During communication between the communication terminal 201 and the communication terminal 202 (step S303), when the communication terminal 201 stores image information received from the communication terminal 202, the communication terminal 201 acquires personal information from the communication terminal 202. A request is made (step S304).

  The communication terminal 202 extracts target person information in response to a request for acquiring person information from the communication terminal 201, and transmits the extracted person information to the communication terminal 201 (step S305).

  (2) Next, a processing flow in which the communication terminal 201 associates and stores image information of the communication partner and person information (attribute information) related to the communication partner will be described with reference to FIG.

  The communication terminal 201 displays an image of the communication partner during a videophone call (step S501).

  When the communication terminal 201 detects an instruction to save the image information of the communication partner by a key input operation or the like from the user of the communication terminal 201 (step S502), a request for acquiring personal information is sent to the communication terminal 202 which is the communication partner. (Step S304 in FIG. 3), and the personal information of the communication partner is received from the communication terminal 202 (Step S503).

  The information storage processing unit 102 of the communication terminal 201 associates the received image information of the communication partner with the personal information including the telephone number that is information for uniquely specifying the communication partner acquired in step S503, and communicates with the communication terminal 201. Is stored in the information storage unit 103 (step S504).

  (3) Next, a processing flow in the communication terminal 202 that receives a person information request from the communication terminal 201 will be described with reference to FIG.

  The communication terminal 202 displays an image of the communication partner during videophone communication with the communication terminal 201 (step S601).

  The communication terminal 202 receives a personal information acquisition request from the communication terminal 201 via the data transmission channel (step S602).

  Upon receiving the personal information acquisition request, the communication terminal 202 refers to the stored information of the communication terminal 202 and extracts the personal information of the user who uses the communication terminal 202 (step S603). The person information is transmitted to the communication terminal 201 via the data transmission channel (step S604).

  (4) Next, the stored information retrieval processing flow in the communication terminal 201 will be described with reference to FIG.

  The communication terminal 201 establishes videophone communication with the communication terminal 202 and is in communication (step S701).

  H. The image of the communication partner received through the video channel in the H.223 multiplexing is displayed. In addition, a conversation is performed using an audio channel (audio channel).

  The communication terminal 201 makes a request for personal information, which is communication partner information, to the communication terminal 202, which is a communication partner, via a data transmission channel (step S702), and the user (user B) of the communication terminal 202 is requested. Person information is acquired (step S703).

  Therefore, the communication terminal 202 acquires identification information that uniquely identifies the communication partner as the personal information of the communication partner. The identification information that uniquely identifies the communication partner is a telephone number.

  The stored information search unit 105 is stored in the information storage unit 103 using the telephone number, which is personal information of the communication partner, which is information for uniquely identifying the communication partner obtained by the communication partner information acquisition unit 104, as search key information. The stored information is searched (step S704).

  The information storage unit 103 stores the personal information related to the image information together with the image information as attribute information. Accordingly, the stored information search unit 105 can extract information related to the user B from the stored information in the information storage unit 103 using the telephone number that uniquely identifies the user B as the communication partner as search key information.

  Then, the extracted information reproduction unit 106 causes the output unit 107 to display information related to the extracted communication partner (searched information) (step S705).

  FIG. 4 shows an example in which image information displayed on the communication terminal 201 and data extracted by the search are displayed.

  In FIG. 4, an image 401 of a partner in communication via a video channel is displayed on the upper side, and image information (searched information) 402 related to the communication partner extracted by the search is displayed on the lower side. .

  As described above, according to the first embodiment, the reception information including the image information received from the communication partner is stored in association with the communication partner information such as the person information. Information can be searched, and even if the amount of information to be searched is large, intended information can be easily extracted.

  In the first embodiment, the information related to the communication partner acquired from the communication partner is personal information as identification information for uniquely identifying the communication partner. However, the information related to the communication partner is not limited to this. Etc. may be included in the person information.

  In addition, by storing information such as date of birth, gender, marriage calendar, occupation, current address, hobbies, and preferences as personal information of the communication partner, use that personal information when searching for stored information This makes it possible to narrow down the information to be searched.

  Note that the communication terminal does not use information that uniquely identifies the communication partner as person information in the search for stored information, but when using preference information or the like, without being limited to stored information related to the communication partner, What is necessary is just to comprise so that it may search. For example, information related to the artist may be searched using favorite artist information as a search condition.

  In the above description, an example is shown in which a telephone number is used as personal information to uniquely identify a communication partner. However, identification information that uniquely identifies a communication partner does not have to be a telephone number. It may be configured to use the user identifier assigned based on the above. Further, a user name or mail address information registered in address information or the like may be used.

  In the above description, the example in which the person information is used as the information related to the communication partner has been described. However, not only the person information but also environment information such as position information and time information may be used. For example, the communication terminal 201 may be configured to request information related to the environment instead of the person information request step S304 of FIG.

  The information related to the environment is information related to the environmental situation in which the communication terminal 202 that is the communication partner is placed, for example, information related to a location such as latitude / longitude information or information related to time such as the current time. is there.

  For example, when the communication terminal 201 requests position information as information relating to a location, the communication terminal 202 acquires position information using a position measurement function of a GPS function (not shown). The communication terminal 202 transmits the acquired position information to the communication terminal 201 via the data transmission channel.

  By this method, the communication terminal 201 can record the position information of the communication terminal 202 as the communication partner terminal together as attribute information when storing the image information of the communication partner. The communication terminal 201 can extract related information based on the position information in a later search.

  The position measurement function may not be the GPS function, and may be another method as long as the position information can be acquired. For example, the position information may be acquired from a cellular network.

  Further, the information related to the location may not be the latitude / longitude information, but may be information related to the location. For example, it may be a zip code, address information, area information, landmark information, or the like.

  Further, based on the obtained latitude / longitude information, it may be converted into information relating to a different type of place. For example, it may be converted into area information and landmark information based on latitude / longitude information.

  The conversion to the information related to the location of another format may be performed on the side requesting the information related to the location, or may be performed on the requested side.

  Further, it may be configured such that the requesting side designates which type of information related to the location is good and requests the information related to the location. In this case, the communication terminal that has requested the information related to the location in the designated format converts the information into information related to the appropriate location and responds.

  The information about the environment acquired from the communication partner is not limited to the information about the place, but may be information about other environments. That is, time information of a communication partner may be acquired as information related to the environment, and the acquired time information may be stored in association with the received image information.

  In this case, in a situation where there is a time difference between the communication partner and the own terminal, it is possible to save information reflecting the time in the partner environment.

  The technical meaning of “associating received information including received image information and communication partner information such as person information” has a one-to-one relationship between certain person information (attribute information) and the image information. In addition, when certain personal information (attribute information) is extracted, the personal information (attribute information) and the image information have a one-to-n relationship. This means that image information corresponding to 1 or image information corresponding to 1 to n is extracted from the extracted person information (attribute information).

(Embodiment 2)
In the second embodiment, the communication terminal shown in FIG. 1 recognizes image information or audio information received from the communication partner, obtains information related to the communication partner, and receives the recognition information. A communication method for storing information in association with it will be described with reference to FIGS.

  That is, in FIG. 8, when the communication terminal 201 communicates with the communication terminal 202 and the communication terminal 201 stores information including image information or audio information from the communication terminal 202, image information or audio received from the communication partner. A description will be given of an example in which characteristics related to a communication partner are acquired by performing information recognition processing, and the acquired information related to the communication partner is stored in association with the received image information or audio information.

  First, in step S801, the communication terminal 201 displays an image of a communication partner during a videophone call, as in the first embodiment.

  Next, in step S802, the communication terminal 201 detects a recording instruction for recording image information or audio information received from the communication partner.

  The detection of the recording instruction is detection of a key operation instructing recording from the user, for example, by pressing a software key or a hardware key.

  In step 802, when an instruction to store the received image information or audio information is detected, the communication terminal 201 performs processing for recognizing image information or audio information received from the communication partner in step S803, and obtains information related to the communication partner. .

  Next, in step S804, the communication terminal 201 obtains image information or audio information received from the communication partner and communication partner information (attribute information) related to the communication partner based on the feature information acquired by the recognition process in step S803. The information is stored in the information storage unit 103 in association with each other.

  In the information storage unit 103 of the communication terminal 201, image information and person characteristic information related to the image information are stored together. Further, it is assumed that the feature information about the person includes feature information related to sound.

  Next, with respect to the recognition processing in step S803, an example will be described in which received video is recognized and information related to the person of the communication partner is obtained.

  The communication terminal 201 stores identification information for identifying a plurality of users in the information storage unit 103 and personal characteristic information corresponding to the identification information in association with each other. For example, the feature information of each user's face image and the feature of the voice are stored in association with the telephone number of the address book that is user information.

  In the video recognition processing by the communication terminal 201, the feature information of the person obtained by analyzing the video information received via the video channel is compared with the feature information of each user included in the user information stored in the information storage unit 103. By doing so, a recognition process for uniquely identifying the person of the communication partner is performed.

  As described above, according to the second embodiment, the image information or the voice information received from the communication partner is recognized to obtain the characteristics related to the communication partner, and the acquired information related to the communication partner is received. Since it is stored in association with information or voice information, it is possible to search for received information with reference to information related to the communication partner, and even if the amount of information to be searched is large, the intended information can be easily extracted.

  In this embodiment, an example in which information related to a person is extracted by image recognition as image recognition has been described. However, the present invention is not limited to this, and information related to a landscape may be extracted.

  For example, when recognizing image information received from a communication terminal of a communication partner, information that is characteristic as a landscape, for example, information about a building is extracted, and the feature information is stored together with the received image information. In this case, when searching for stored information, it is possible to search for information to be stored using feature information to be stored. For example, presenting information that is judged to be more closely related by comparing feature information obtained by image recognition of landscape information that is video information from a communication partner and feature information related to the landscape stored in the information storage unit Is possible. Further, it is possible to search for stored information having feature information that is closely related to the feature information of the building as a search condition.

  Note that the communication terminal 100 may recognize the image with priority given to the image information of the indicated portion by designating the position of the partner video screen to be displayed by the pointing device.

  With this configuration, it is possible to recognize an image by designating a recognition target. For example, if a plurality of persons or both persons and buildings are displayed as images, it is possible to specify a priority recognition target by pointing one of them with a pointing device. It is possible to specify feature information to be stored together when saving.

  Note that the attribute information related to the video object corresponding to the position of the partner video specified by the pointing device may be acquired from the communication partner and stored together. With this configuration, attribute information related to a target pointed by the pointing device can be used as a search condition to search for stored information.

  Next, an example in which the communication partner information is acquired by recognizing the voice of the communication partner and the stored information is searched based on the acquired key information will be described with reference to FIG.

  Also in this case, the communication terminal 201 stores the identification information for identifying a plurality of users in the information storage unit 103 and the characteristic information about the person image corresponding to the identification information in association with each other. For example, voice feature information of each user is stored in association with a telephone number in an address book that is user information.

  Voice recognition processing by the communication terminal 201 is performed by separating voice information from sound information received via an audio channel and analyzing human voice feature information obtained by analyzing the separated voice information, and an information storage unit. A recognition process for uniquely identifying the person of the communication partner is performed by comparing the feature information of each user's voice included in the user information stored in 103.

  The specific flow is as shown below.

  First, when the communication terminal 201 makes a connection request via videophone to the communication terminal 202 and receives an acceptance response from the communication terminal 202, the H. Videophone communication is performed by the 3G-324M protocol using 223 multiplexing (step S901).

  The video is transmitted and received using the video channel, and the other party's video is displayed by the output unit 107 (step S902). It also has a conversation through the audio channel.

  The communication partner information acquisition unit 104 of the communication terminal 201 extracts the other party's voice information in the conversation using the audio channel (step S903), and the communication partner information acquisition unit 104 further extracts the other party's voice information. The communication partner is uniquely identified by performing analysis and referring to the voice database storing the voice characteristics stored in the information storage unit 103 (step S904).

  The stored information search unit 105 of the communication terminal 201 searches the stored information stored in the information storage unit 103 based on the specified communication partner information, and extracts information related to the user B who is the communication partner (step S905).

  The extracted information reproducing unit 106 of the communication terminal 201 displays the image information, which is information related to the extracted user B, via the output unit 107 (step S906).

  Of course, these steps are performed automatically in succession.

  Note that a user interface for controlling the timing for performing voice recognition may be provided. For example, in order to control the timing for performing voice recognition, software keys and / or hardware keys are provided, and voice recognition is performed using a predetermined key press as a trigger. The timing for performing voice recognition may be performed at any time determined in advance during communication.

  When voice recognition is performed as needed, even when the target user (user using the terminal) changes in the communication partner terminal, the user can quickly re-identify the user as the communication partner without any operation. .

  The communication terminal can improve speech recognition by storing a term database that assists speech recognition.

  The voice recognition may be performed not only for voice received from the communication partner but also for voice of the user of the terminal itself.

  Further, the sound information to be recognized is not limited to the sound information but may be an environmental sound. For example, by recognizing the environmental sound, information such as where the communication partner is and what situation is obtained. For example, it is distinguished that the vehicle is moving indoors, outdoors, or in a car, and the information is stored as feature information. Further, it can be recognized that it is a specific place, for example, a concert venue or a station where a sound related to a train can be heard. When configured in this way, it is possible to easily extract information in a similar situation at the time of a search performed later.

  INDUSTRIAL APPLICABILITY The present invention is used for a communication terminal such as a mobile phone, a personal digital assistant (PDA), a notebook personal computer, a personal computer on a network constructed by a wireless LAN or a wired LAN, and its communication method. It is suitable for.

The block diagram which shows the structure of the communication terminal which concerns on Embodiment 1 of this invention. Schematic diagram showing a communication system according to Embodiment 1 of the present invention. The sequence diagram which shows the information acquisition method regarding the communication other party which concerns on Embodiment 1 of this invention The figure which shows the example of a display of the reproduction information which concerns on Embodiment 1 of this invention. FIG. 5 is a flowchart showing a method for storing received image information according to the first embodiment of the present invention. The flowchart which shows the response method with respect to the communicating party information request which concerns on Embodiment 1 of this invention The flowchart which shows the search method of the storage information based on the communication other party information which concerns on Embodiment 1 of this invention. FIG. 7 is a flowchart showing a storage information storage method based on image recognition according to Embodiment 2 of the present invention; The flowchart which shows the retrieval method of the preservation | save information based on the speech recognition which concerns on Embodiment 2 of this invention

Explanation of symbols

100, 201-203 Communication terminal 102 Information storage processing unit 103 Information storage unit 104 Communication partner information acquisition unit 105 Stored information search unit 106 Extracted information reproduction unit 107 Output unit 211-213 Base station

Claims (8)

Information storage means for storing received information including at least image information received from a communication partner;
Communication partner information acquisition means for acquiring communication partner information related to the reception information including the image information from the communication partner;
Information storage processing means for storing the received information including the image information received from the communication partner in association with the communication partner information acquired by the communication partner information acquiring means;
A communication terminal comprising:   2. The communication terminal according to claim 1, wherein the communication partner information acquisition unit captures the communication partner information and stores the information in the information storage unit when detecting an instruction to store the reception information including the image information.   3. The communication terminal according to claim 1, further comprising search means for searching for information in the information storage means based on the received key information for specifying a communication partner.   The communication terminal according to any one of claims 1 to 3, wherein the communication partner information acquisition unit recognizes an image of an area designated by a pointing device. An information storage step for storing in a memory reception information including at least image information received from a communication partner;
A communication partner information acquisition step of acquiring communication partner information related to the reception information including the image information from the communication partner;
An information storage processing step for storing in the memory the received information including the image information received from the communication partner and the communication partner information acquired in the communication partner information acquisition step in association with each other;
A communication method comprising:   6. The communication method according to claim 5, wherein said communication partner information acquisition step captures said communication partner information and stores it in said memory when an instruction to save received information including said image information is detected.   7. The communication method according to claim 5, further comprising a search step of searching for information in the memory based on the received key information for specifying a communication partner.   The communication method according to claim 5, wherein the communication partner information acquisition step recognizes an image of an area designated by a pointing device.

Images