JP4636903B2 - Video phone equipment - Google Patents

Description

The present invention also relates to a television telephone equipment.

  In recent years, communication services performed via an ISDN line, the Internet, a mobile phone network, and the like have become widespread, and audiovisual services such as a video phone system and a video conference system using such a digital line have attracted attention. Among them, some applied technologies have been proposed for videophones.

  For example, Patent Literature 1 discloses a multimedia communication device that allows an operator to select whether or not to transmit a self-image after confirming who the other party is at the start of communication after call setting.

  According to this apparatus, it is possible to make a call by transmitting only the voice of the operator without transmitting the self-portrait to a partner who does not want to transmit the self-portrait.

In addition, there is a telephone that receives a caller's telephone number and displays a name corresponding to the telephone number as a telephone that can identify a caller.
Japanese Patent Laid-Open No. 05-091214

  However, when another person who is not the owner of the telephone makes a call to the receiver, even if the telephone number displayed by the caller number notification function or the like is confirmed, the receiver cannot confirm that the other person has made the call.

  The present invention has been made to solve such problems, and an object of the present invention is to be able to confirm the actual sender on the receiver side.

In order to achieve the above object, the present invention has the following configuration. That is, a communication unit that establishes a session for a call with the calling party, a transmitting / receiving unit that transmits and receives audio and images, and a representative image that includes the face of the sender on the calling side created from the image received by the transmitting / receiving unit Representative image creation means, speech recognition means for converting speech received by the transmission / reception means into text, summary message creation means for summarizing the converted text and creating a summary message, and representative image creation means A representative image created by the user and storage means for storing identification information used for callback to the sender on the calling side, a representative image created by the representative image creation means, and a summary message creating means Output means for outputting the summary message to the display means, and that the representative image or summary message output to the display means is selected. A detection means for issuing, and a control means for controlling the transmission / reception means so as not to be in a call ready state until a call start instruction is received from the called user after the session is established. When a set time has elapsed, an answering machine function for storing image and audio information received by the transmission / reception means in the storage means is enabled, and the voice received by the transmission / reception means and the summary message creation means are The created summary message is stored in the storage means. When the summary message is selected, the voice stored in the storage means is played back. When the representative image is selected, a callback to the sender is performed. It is a videophone device to perform .

  According to the present invention, the actual caller can be confirmed on the receiver side.

  A videophone device according to an embodiment of the present invention will be described below with reference to the drawings. The configuration of the embodiment is an exemplification, and the present invention is not limited to the configuration of the embodiment. The present invention can be implemented by hardware and software. When the program is executed by software consisting of a program, various functions can be realized by installing the program constituting the software in hardware such as a computer. The program is installed in a computer or the like through a communication line or using a computer-readable storage medium.

  Here, the computer-readable storage medium refers to a storage medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from the computer. Examples of such storage media that can be removed from the computer include a flexible disk, a magneto-optical disk, a CD-ROM, a CD-R / W, a DVD, a DAT, a magnetic tape, and a memory card. Further, there are a hard disk, a ROM (read only memory) and the like as a storage medium fixed to the computer.

  FIG. 1 is a block diagram showing a configuration of a videophone system according to an embodiment of the present invention. In FIG. 1, the videophone system 100 on the transmission side (calling side) and the videophone system 200 on the reception side (calling side) have the same configuration. Therefore, for the same components, “a” is added to the code on the transmitting side, and “b” is added to the code on the receiving side.

  The videophone system 100 (200) includes a videophone device 1A (1B), an image / audio server 8a (8b), and a display device 9a (9b). Each of the image / sound server 8a (8b) and the display device 9a (9b) can be provided separately or integrally with the videophone device 1A (1B).

  When the display device is provided integrally with the videophone device, the display device operates as a display unit of the videophone device (corresponding to “display means” of the present invention). Further, even when the videophone device has a display unit, it may be connectable to an external display device.

  Next, an example in which the videophone system according to the embodiment of the present invention is applied to a television receiver as a separate display device will be described as an example of the present invention.

  In FIG. 1, reference numerals 1a and 1b denote interfaces for performing communication including still images or moving images that are actually transmitted and received during connection processing for establishing a call and during a call. The videophone apparatus processes images and sound as signals or data, respectively.

  Reference numerals 2a and 2b denote connection control units including a ROM, a RAM for storing a program necessary for controlling the videophone system of the present invention, and a CPU for executing various processes by executing the program.

Reference numerals 3a and 3b denote image / sound transmission / reception units in the videophone system of the present invention.
Reference numerals 4a and 4b denote transmission image input cameras (corresponding to "imaging means" of the present invention) in the videophone system of the present invention.

Reference numerals 5a and 5b denote receivers for inputting and outputting sound in the videophone system of the present invention. The receivers 5a and 5b are interfaces with a person when making a call, and include a microphone for inputting voice, a speaker for outputting voice, a button for inputting a telephone number, and the like. The receivers 5a and 5b are connected to the videophone device by wire or wirelessly.

  Reference numerals 6a and 6b denote videophone I / Fs that connect the display devices 9a and 9b and the videophone devices 1A and 1B in the videophone system of the present invention.

  7a and 7b are icon creation units (corresponding to "representative image creation means" of the present invention) for creating a reduced image including the face of the sender from the received image on the sender side in the videophone system of the present invention. is there.

  8a and 8b are (i) a created icon, (ii) an image stored during an answering machine (answering machine image), (iii) an image stored during a call (calling image), and (iv) in the videophone system of the present invention. 4) An image / sound server (corresponding to the “storage device” of the present invention) that stores image information or the like by associating a reply function for replying to the sender with at least one of the summary messages shown in FIG.

  For example, when the reply function is associated with the icon shown in FIG. 3, the reply function is a function that can make a call to the sender by performing a selection operation such as pressing down the icon. The image / sound servers 8a and 8b store addresses such as a telephone number of the other party in association with an icon or the like.

  Reference numerals 9a and 9b denote television receivers that can display images and information related to a videophone, output sound, and project a television broadcast in the videophone system of the present invention.

  FIG. 2 is a block diagram showing a configuration of connection control units 2a and 2b used in the videophone apparatus according to the embodiment of the present invention. The connection control units 2a and 2b are a central processing unit CPU 11, a ROM 12 storing a processing procedure according to the present invention as a program, a RAM 13 temporarily storing various information, a voice recognition unit 14, a summary message creating unit 15, a human An image analysis unit 16, a human face image analysis unit 17, an image cutout unit 18 and the like are included.

  In FIG. 2, the CPU 11 processes various information handled by the connection control units 2a and 2b in the present invention. Next, main operations of each component of the connection control unit on the transmission side and the reception side will be described.

  The CPU 11 on the transmission side operates the image / sound creation unit to transmit an image obtained from the camera after the session with the reception side is established. Further, the CPU 11 on the receiving side stores the image received from the transmitting side in the image / sound server.

  The ROM 12 on the transmission side stores a processing procedure for determining the establishment of a session and transmitting the image to the reception side (FIG. 5, steps 53 and 54). Further, the receiving ROM 12 stores the processing procedures of steps 52, 53, 55 to 58 of FIG. 5 and the processing procedures of steps 70, 71, 73 to 77 of FIG.

  In FIG. 2, the RAM 13, the voice recognition unit 14, the summary message creation unit 15, the human image analysis unit 16, the human face image analysis unit 17, and the image cutout unit 18 operate on the reception side in order to process received images and voices. To do.

The RAM 13 temporarily stores the received image, sound, text converted from the sound, or other information being processed. The voice recognition unit 14 converts voice received by the answering machine function into text (document). The summary message creation unit 15 starts the display device 9b from the text.
Create a summary message for display on The human image analysis unit 16 analyzes the image to determine whether a person is included in the received image.

  The human face image analysis unit 17 analyzes the image in order to determine whether the received image includes a human face portion. The image cutout unit 18 cuts out a person image or a person face image from the image according to the analysis result of the human image analysis unit 16 or the human face image analysis unit 17.

  Hereinafter, an example of an embodiment for confirming an actual caller on the receiving side will be described with reference to FIG. FIG. 5 shows that when a receiving side user is watching a TV program of a TV broadcast, the receiving side videophone device 1B detects a call request (connection request) from the transmitting side videophone device 1A. It is an example of an operation | movement of the video telephone system until it will be in the state which can call. In the following embodiments, description will be made on the assumption that the display devices (television receivers) on the transmission side and the reception side are turned on. In the following description, it is assumed that the display device 9a on the transmission side is the television receiver 9a and the display device 9b on the reception side is the television receiver 9b.

  First, it is assumed that both the television receiver 9a on the transmission side and the television receiver 9b on the reception side project a television program.

  First, in step 51, the sender operates the receiver 5a to call the videophone device 1B on the receiving side. At this time, the videophone device 1A on the transmission side identifies information such as an identification ID such as an address of the transmission destination and a videophone number input by the connection control unit 2a (corresponding to “identification information” of the present invention). Is used to transmit a call request for starting a call to the videophone device 1B via the interface 1a. The videophone device 1B stores information (corresponding to “identification information” in the present invention) for identifying a partner such as an identification ID such as an address on the transmission side and a videophone number in the image / sound server 8b. The identification information on the transmission side is used in the above reply function.

  In step 52, in the videophone 1B, the connection control unit 2b performs a connection process in accordance with a communication procedure normally used in a videophone in response to a call request on the transmission side (corresponding to "communication means" of the present invention). In step 53, a communication session is established between both systems.

  At this time, although the communication line is established, the connection control units 2a and 2b on both the transmission side and the reception side do not make the actual call possible state until there is an instruction to start the call from the user on the reception side. The image / sound transmitting / receiving units 3a and 3b are controlled (corresponding to "control means" of the present invention). Then, the connection control units 2a and 2b continue to play the ringing sound through the videophone I / F 6a to the transmitting side television receiver 9a until the substantial call is possible in step 59. The call ready state is a state in which voice and / or images can be transmitted to each other.

  In step 54, the connection control unit 2a of the transmission side system transmits the image of the sender imaged through the camera 4a to the reception side through the image / sound transmission / reception unit 3a and the interface 1a.

  The connection control unit 2a of the transmission side system cuts out an image from which the sender's face can be recognized from the image of the sender imaged through the camera 4a by the icon creation unit 7a (corresponding to “representative image creation means” of the present invention) -You may transmit to the receiving side through the audio | voice transmission / reception part 3a and the interface 1a. It is also conceivable that the connection control unit 2a transmits a still image or a moving image, and an icon creation unit 7b (corresponding to the “representative image creation unit” of the present invention) cuts out an image from which the receiving system can recognize the face. .

  Next, in step 55, the connection control unit 2b of the videophone 1B stores the still image received through the interface 1b and the image / sound transmission / reception unit 3b in the image / sound server 8b, and simultaneously sends it to the icon creation unit 7b.

  In step 56, the icon creation unit 7b performs face recognition from the acquired still image using the human face image analysis unit 17 of the connection control unit 2b. The icon creation unit 7b cuts out and enlarges or reduces a partial image including a face to create an icon. The icon creation unit 7b may create a display icon without cutting out the acquired image.

  Before executing step 55, in step 54, the videophone 1 </ b> B may be provided with a human image analysis unit 16 that analyzes whether an image of a person is included in the image transmitted by the videophone 1 </ b> A. The device 1B may have a function of displaying an icon on the television receiver 9b when the human image analysis unit 16 of the connection control unit 2b analyzes that a person is included in the image.

  In addition, when the human image analysis unit 16 or the human face image analysis unit 17 analyzes that the person image or the face image is not captured, the video phone device 1B does not display an icon or the like on the video receiver 9b, and the video phone system A notification function may be provided to notify 100 that a person's face or the like is not shown. Here, the connection control units 2a and 2b may have a function of notifying the transmission source and a function of receiving a notification from the transmission destination.

  In step 54, when the human image analyzing unit 16 analyzes that an image other than a person is included in the transmitted image, (i) a function of displaying an icon on the television receiver 9b; and (ii) a television receiver 9b. It is also possible to switch between two functions, i.e., a function that does not display an icon.

  Further, the connection control units 2a and 2b may have a function of comparing a received acquaintance image with a received image and determining the similarity between the images. In addition, when the connection control units 2a and 2b determine that the similarity is equal to or less than a predetermined threshold, a function of enabling or operating the answering machine function may be provided. Thereby, if it is not an acquaintance, the answering machine process mentioned later can be performed automatically.

  The icon created in step 56 has a connection link function (reply function) to the transmission source. The image / sound server 8b accumulates the created icons.

  In step 57, the connection control unit 2b on the receiving side displays the created icon on the screen of the television receiver 9b through the videophone I / F 6b. For example, the connection control unit 2b superimposes an icon on the TV program content being displayed. Further, the connection control unit 2b on the receiving side informs the receiving side user that there is a call request for a call from the other party of the image displayed as the icon by blinking an icon on the television receiver 9b. For example, as shown in FIG. 3, an icon is displayed on a television receiver. Notify you that you received a videophone call.

In step 58, the receiving user selects the displayed icon with a selection unit such as a remote controller, and transmits a control signal instructing the start of the call to the television receiver 9b. When the television receiver 9b receives the control signal, the connection control unit 2b on the receiving side receives the videophone I / F.
A control signal is received through 6b. The connection control unit 2b negotiates with the connection control unit 2a on the transmission side via both the interfaces 1a and 1b in accordance with the control signal, and performs a substantial call connection with the sender. Thereafter, in step 59, image and sound can be transmitted and received bidirectionally, and a call using the videophone system is possible.

  When a television receiver is used as the display device, a wireless receiver with a built-in camera may be used as a remote controller. At this time, an image received from the transmission side is displayed on the television receiver, and a telephone call can be made with the receiver.

  When a call on the videophone system is started, the connection control unit 2a on the transmission side stops the ringing tone. Further, both the connection control units 2a and 2b completely switch the image such as the TV program displayed on the transmission side and the reception side to the image received by the videophone system or display a reduced screen.

  Next, a second embodiment according to the present invention will be described with reference to FIG. FIG. 6 shows a flowchart when the receiving user watching the TV program is not a videophone. That is, FIG. 6 shows processing for executing the answering machine function.

  In the present embodiment, when the TV receiver 9b on the receiving side is in the OFF state, when the sender leaves an answering machine message with an image and voice (answering message recording), the receiver plays back the answering message recording. explain. If the CPU 11 of the connection control unit 2b determines that the power of the television receiver 9b is OFF, the answering machine function may be enabled.

  Steps 51 to 57 are the same processing as in the first embodiment. However, step 57 is a process that is not executed when the receiving user has not turned on the power of the television receiver 9b. The features of the second embodiment are shown in steps 70-77. Hereinafter, the processing after step 70 will be described.

  In the second embodiment, the TV receiver 9b on the receiving side is in the OFF state, and the answering machine function is set. After the image and voice are received by the answering machine function, the processing until the receiving user turns on the television receiver 9b and confirms the contents stored by the answering machine function will be described. Even if the power supply of the TV receiver 9b on the receiving side is ON, the answering machine function may be set by operating a button or the like of the videophone device 1B.

  In step 70, the user of the videophone system 200 on the receiving side operates the operation unit of the videophone 1B or the button of the receiver 5b to preset the time (number of calls) until the answering machine function is executed. .

  When the transmitting-side videophone system 100 transmits a call request to the receiving side, the calling is performed for a set time. In step 71, after the set time has elapsed, the connection control unit 2b on the reception side has previously stored in the connection control unit 2a on the transmission side in the image / sound server 8b (corresponding to the “storage means” of the present invention). An answering machine message is transmitted via the image / sound transmission / reception unit 3b and the interface 1b. After the set time elapses, the connection control unit 2b activates the answering machine function of the videophone system 200 (recording mode).

  In step 72, after the sender listens to the answering machine message received from the videophone system 200, the videophone system 100 transmits the image / sound (answering machine message).

  In step 73, the reception side connection control unit 2b stores the contents of the answering machine acquired through the interface 1b and the image / sound transmission / reception unit 3b in the reception side image / sound server 8b through the videophone I / F 6b.

  Next, in step 74, the connection control unit 2b on the receiving side acquires the voice accumulated in the image / sound server 8b through the videophone I / F 6b. The summary message creating unit 15 converts the speech recognized by the speech recognition unit 14 of the connection control unit 2b into text by the summary message creating unit 15 and creates a summary message.

  The image / sound server 8b stores the summary message and the icon created in step 55 in association with each other. When the power of the television receiver 9b is turned on, the television receiver 9b displays the icon and the summary message on the screen of the television receiver 9b (FIG. 4).

  The summary message creating method of step 74 may have a function of creating a summary message by extracting a text portion that matches a preset keyword. Thus, by setting keywords in advance, only the contents that the user wants to know can be picked up and displayed as a summary message.

  Further, the connection control units 2a and 2b may have a marking function such as changing the font color of the “keyword” of the summary message displayed on the television receiver 9b. As a result, a display method in which the main points are easily understood by the receiver can be realized.

  Furthermore, the keyword is given in advance an attribute of order of urgency (corresponding to “priority” of the present invention), and when a plurality of answering machine messages are received, the answering machine message is displayed according to the urgency of the keyword. It is also possible to determine the priority (reproduction order). The connection control unit 2b displays an answering machine message according to the urgency of the keyword.

  In step 75, when the receiving-side user turns on the power of the television receiver 9b, the connection control unit 2b detects that the television receiver 9b is turned on. In step 76, the connection control unit 2b on the receiving side notifies the user that there is an image / sound stored in the answering machine function in the video receiver 9b on the receiving side through the videophone I / F 6b. The icon of the sending user and a summary message are displayed (FIG. 4).

  Next, in step 77, when the receiving side user selects the summary message (icon) displayed by the television receiver 9b, the connection control unit 2b stores the answering machine image stored in the image / sound server 8b through the videophone I / F 6b. And audio playback starts.

  At this time, if the receiving side user selects the icon (summary message) displayed on the television receiver 9b through the videophone I / F 6b, the connection control unit 2b may perform a call back to the sender. Good.

  According to the present invention, after confirming the image, it is possible to make a call state, so that it is not known that the other party has received the call, so that the other party is not given a bad impression. In addition, when a call comes in while watching TV, it is possible to prevent the screen from being switched to the TV phone screen immediately. Further, the receiving side user can visually and easily understand when and from what kind of business he / she has called before playing the answering machine message.

  In addition, in a videophone system having a TV broadcast receiving function, an icon or the like is provided in one section (part) of the TV image to notify who has come from without immediately switching from the TV image to the videophone image. Can be displayed. As described above, according to the present invention, it is possible to give the receiver information for determining whether or not to answer the videophone.

Furthermore, according to the present invention, when answering a videophone, the same interface as that used for receiving a normal videophone is used, and when and from whom and what kind of business calls are received. It is possible to provide a function of notifying the recipient visually and simply.

  According to the present invention, after a communication session is established with a videophone or videoconferencing communication protocol, a communication procedure, etc., an image of the first videophone caller using the higher-level protocol is displayed on the receiving side, and the receiving side calls If you do not start, you will not be in a call state.

  The videophone system using the present invention is intended for videophone devices attached to or incorporated in display devices such as analog televisions, digital televisions, personal computers, etc., but is also applicable to videophones using cellular phones. Is possible. Further, the present invention can be applied regardless of the type of communication line such as the Internet, ISDN, telephone line, and cellular phone network.

  As described above, in the videophone system of the present invention, the videophone receiver can see the image sent from the transmission side before both systems are substantially in a call-ready state, Can be determined.

It is a block diagram which shows the structure of the videophone system by this invention. It is a block diagram which shows the structure of the connection control part used with the videophone system by this invention. It is a figure which shows one Embodiment of the answering machine process of the videophone system by this invention. It is a figure which shows one Embodiment of the answering machine process of the videophone system by this invention. It is a detailed flowchart of the process in the transmission side system of the video telephone system by this invention, and a receiving side system. It is a detailed flowchart of the process in the transmission side system of the video telephone system by this invention, and a receiving side system.

Explanation of symbols

1a, 1b Interface 2a, 2b Connection control unit 3a, 3b Image / audio transmission / reception unit 4a, 4b Camera 5a, 5b Receiver 6a, 6b Videophone I / F
7a, 7b Icon creation units 8a, 8b Image / audio servers 9a, 9b Display device 11 CPU
12 ROM
13 RAM
14 Voice recognition unit 15 Summary message creation unit 16 Human image analysis unit 17 Human face image analysis unit 18 Image segmentation unit

Claims (4)

A communication means for establishing a session for a call with the calling party;
Transmitting and receiving means for transmitting and receiving sound and images;
Representative image creating means for creating a representative image including the face of the calling sender created from the image received by the transmitting / receiving means;
Voice recognition means for converting voice received by the transmission / reception means into text;
Summary message creation means for summarizing the converted text by the voice recognition means and creating a summary message;
Storage means for storing a representative image created by the representative image creation means, and identification information used for callback to a sender on the calling side;
An output means for outputting a representative image created by the representative image creating means and a summary message created by the summary message creating means to a display means;
Detection means for detecting that the representative image or summary message output to the display means is selected;
After session establishment, viewed including control means for controlling said transmitting and receiving means so as not to call established status until receiving an indication of a call initiated from the called user,
The control means enables an answering machine function to store the image and voice information received by the transmission / reception means in the storage means when a preset time has elapsed after the session is established, and the voice received by the transmission / reception means. And storing the summary message created by the summary message creation means in the storage means,
A videophone device that plays back the voice stored in the storage means when the summary message is selected, and calls back to the sender when the representative image is selected . A human face image analyzing means for analyzing whether the received image includes a human face image;
2. The videophone apparatus according to claim 1, wherein when the human face image analyzing means analyzes that a human face image is included, the output means outputs an image. Human face image analyzing means for analyzing whether or not the received image includes a human face image;
In addition,
When the person facial image analysis means the analysis and does not include a face image of a person, in the output unit does not output the image, according to claim 1, wherein notifying the sender that the image is not photographed about the person Video phone equipment. Before SL control unit analyzes the received image, and comparing the received image and registered image in advance, to determine the similarity between the images, when the degree of similarity is equal to or less than a predetermined threshold value, prior to KiTome Mamoru No. videophone apparatus according to claim 1, wherein you enable telephone function.

Images