JPH09331385A - Voice image communication system and image information retrieval system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the voice image communication system with excellent operating convenience by which retrieval and confirmation are facilitated by making a voice keyword corresponding to image information and telephone number information and storing the information to a storage section. SOLUTION: A database 201 corresponding to voice keywords 200 is stored in linking to a storage section 107. Furthermore, an image information database 300 and a telephone number database 300 in cross reference with the image information are stored in the database 201 of the storage section 107. Then a plurality of sets of image information are in cross reference with one voice keyword and one telephone number is linked with a plurality of the sets of the image information. Thus, a plurality of objects are retrieved by one voice keyword and visual information such as image information corresponding to the keyword is provided. Moreover, even when one voice keyword for retrieval is forgotten, it is relieved by making a plurality of keywords in cross reference with one telephone number.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice image communication system and an image information retrieval system, and more particularly to a system having a multimedia function in which a computer, a telephone, an image transmitting / receiving function, a television broadcasting receiving function and the like are integrated. Of technology.

[0002]

2. Description of the Related Art Conventionally, a telephone having a voice dial function for making a call using voice recognition, command control, and the like have been proposed. For example, in the case of voice dialing, a word corresponding to a telephone number is uttered in advance, and a feature amount obtained by voice analysis processing is registered as a standard pattern in a memory in association with a telephone number at a word level. When making a call
Instead of keying in the phone number, speak the word corresponding to the phone number, and perform word recognition by matching the feature amount obtained by voice analysis processing with the standard pattern, and the number associated with the recognition result. A method has been proposed in which information is transmitted to a telephone line (Japanese Patent Laid-Open No. 2-1436).
51). Then, only the telephone having no image display at present uses voice output or character output as a means for confirming the result of voice recognition.

By the way, recently, ISDN (Integrated
Due to the spread of Services Digital Network, desktop-type videophones and videoconference systems in which a personal computer (PC) or workstation (WS) and a camera input are integrated are becoming widespread. Then, in order to connect the line to the other party, for example, in a star type multipoint video conference, you can key in individual telephone numbers to select multiple parties, or you can enter names or telephone numbers registered in advance. The method of selecting numbers etc. is used.

In addition to making a telephone call, a method of digitally inputting and recording image information such as a digital still camera or a digital video camera has recently been proposed. Such digital information is taken into a personal computer or a workstation, and the information is used for editing such as processing, correction, change, etc., and its application is widely considered. When searching, the date and N are currently added to the image information.
o. If there is additional information in chronological order such as, you can use it to search using a keyboard or mouse.

[0005]

The methods known so far require a registration operation in advance as a preparation before making a call using the voice dial function.
One phone number is associated with one voice keyword. The reason for doing this is that the conventional method has a limitation in the means for confirming the result and it is difficult to assign a plurality of telephone numbers with one voice keyword. Therefore, when calling to a plurality of parties, it is necessary to remember different voice keywords and input them by voice, which is troublesome for the user and inconvenient.

When a plurality of telephone numbers are associated with one voice keyword, there arises a problem of how to confirm a plurality of telephone numbers obtained by voice recognition.

On the other hand, digital information such as an image includes date, time, No. When it is not possible to specify the date and time when the user took a picture after it was once downloaded to a PC or workstation using only time-series additional information such as
Searching is not easy. Therefore, as the amount of information such as images to be stored in the memory increases, the time and effort for the confirmation work for the search increase, which is troublesome for the user, and there is a possibility that the originally intended editing work may not be performed sufficiently. There is.

It is an object of the present invention that even if a plurality of telephone numbers are associated with one voice keyword and registered, the desired telephone number can be specified reliably and easily by using image information. To provide a good audio-visual communication system.

It is another object of the present invention to add a voice information index to the image information when registering the image information into a personal computer or a work station so that the information can be easily searched and confirmed. An object of the present invention is to provide an image information retrieval system that combines various information and voice information.

[0010]

In order to achieve the above object, the voice image communication system of the present invention comprises a voice input section for inputting voice and a key input section for inputting a telephone number by a key or a mouse. , A telephone number storage unit that stores the telephone number entered from the key input unit for redialing, an image storage unit that captures the still image of the other party, a telephone number, a still image of the other party, and a voice input section that are stored for redialing A storage unit that stores the input voice in association with each other is provided.

Further, the voice image communication system of the present invention is provided with a storage unit for storing a voice keyword and a plurality of image information corresponding thereto at the time of keyword registration.

Further, in the voice image communication system of the present invention, when the recognition result is displayed on the display, when voice keywords corresponding to a plurality of images are input, the corresponding image information is read from the storage unit and divided into a plurality of pieces. An image display unit that can display the image is provided. With this, the user can easily visually confirm.

Further, in the image information retrieval system of the present invention, a storage unit for registering a voice keyword corresponding to the image information as an index, and an image display unit capable of reading the corresponding image information from the storage unit and displaying the image in a plurality of divided areas. And.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a terminal device (terminal device with a telephone function) of a voice image communication system according to an embodiment of the present invention.

In FIG. 1, 100 is a key input unit for inputting a telephone number, 101 is a display unit for displaying a telephone number and other messages, 102 is an image input unit for inputting still images and moving images, and 103 is An image display unit that outputs the image of the other party or the other party, 104 is a voice input unit for inputting voice, 105 is a voice output unit that outputs the voice of the other party or the other party, or some voice such as a confirmation message of registration, 106 is a control unit that controls each unit in the terminal device, 107 is a storage unit that stores a database of voice keywords, image information, telephone number information, and a database of their correspondence, 108 is an analysis process of the input image An image processing unit 109, an image CODEC 109 for transmitting an image, a voice processing unit 110 for analyzing input voice, 1 Voice C for 1 to transmit speech
The ODEC 112 is a line connection unit for connecting to a line network such as a public line network, and the terminal device with a telephone function of the present embodiment is mainly configured by the above units 100 to 112. Reference numeral 113 is a public telephone line network or a line network such as ISDN which is widely used.

In this embodiment, first in the off-hook state,
A telephone number is input from the key input unit 100 such as a numeric keypad. The telephone number input at this time is temporarily stored in the memory for redialing. Next, the input telephone number is the line connection unit 1 controlled by the control unit 106.
The call is sent from 12 to the network 113 and sent to a general telephone exchange to be connected to the other party. Then, the information of a certain time during which the other party goes off-hook and the image information is transmitted is captured as a still image, and is used as visual confirmation data when making a call again. In addition, the voice signal input from the voice input unit 104 is analyzed by the voice processing unit 110 and converted into a characteristic parameter. Then, the registration is completed by storing the characteristic parameter, the telephone number input from the key input unit 100, and the captured still image in association with each other.

The off-hook state and the key input section 100
Voice input unit 10 with no telephone number input
When a voice is input from 4, the voice processing unit 110 obtains a characteristic parameter, calculates a standard pattern of the voice keyword stored in the storage unit 107 and the similarity, and obtains the word having the highest similarity. After that, the image information corresponding to the obtained keyword word is referred from the storage unit 107, a plurality of data is sent from the control unit 106 to the image processing unit 108, and the image display unit 10
At 3, the screen is divided and multi-displayed. Selection or confirmation is performed on the displayed screen, the telephone number corresponding to the selected image is read from the storage unit 107 to the control unit 106, the telephone number is sent to the line connection unit 112, and the line network 1
Call 13.

As shown in FIG. 2, the storage unit 107 stores a database 201 linked to the voice keywords 200.

As shown in FIG. 3, the database 201 of the storage unit 107 includes a database 300 of image information.
Is stored, and a plurality of image information is stored in association with one voice keyword. Further, a plurality of voice keywords are stored corresponding to one image information.

Further, as shown in FIG. 4, the database 201 of the storage unit 107 includes a database 30 of image information.
0 and a telephone number database 400 corresponding thereto are stored. A plurality of image information correspond to one voice keyword, and one telephone number can be linked to a plurality of image information to be associated with each other. In this way, if you associate one telephone number with multiple keywords, you can search
Even if you forget one voice keyword, you can deal with it,
A wide range of search systems can be constructed without limiting the search vocabulary by voice.

In FIG. 5, the storage unit 107 is replaced by the server 50.
By having it on the 0 side, a plurality of clients 504 are connected to the LAN 503 and the public line 50 in the database server 501.
It shows that it can be shared and accessed through 2.

FIG. 6 is a flow chart of a sequence for registering a voice keyword and image information in the telephone number information in the videophone and the videoconference system to which the present embodiment is applied.

First, when the telephone number is input from the key input unit 100 in step 601 from the off-hook state in step 600, the process proceeds to step 602, and the telephone number is stored in the storage unit 1.
No. 07 telephone number database 400 is stored. The input telephone number is transmitted from the line connection unit 112 controlled by the control unit 106 to the line network 113, and is sent to a general telephone exchange to be connected to the other party. Then, in step 603, it is determined whether or not the other party's telephone is a telephone capable of transmitting and receiving a video signal, and if it is a telephone capable of transmitting and receiving a video signal, the process proceeds to step 604, and the video signal transmitted after the other party goes off hook is transmitted. To receive.
Then, in step 605, the received video signal is processed as a still image from the image processing unit 108 through the image CODEC 109. In this process, the control unit 106 takes in information of a certain time of an image transmitted from the other party as a still image. For example, after the other party goes off-hook, the voice processing unit 110 sets the voice section in which the voice is being uttered.
The control unit 106 automatically captures the received image at that time from the image processing unit 108. Alternatively, the user can register any received image in the storage unit 107 by using the selection key or the like from the key input unit 100. Image registration is also possible by having the partner send the selected image. The captured image information is stored in the image database 3 in step 608.
00, and the process proceeds to step 609.

On the other hand, if it is determined in step 603 that the called party is not a telephone capable of transmitting and receiving a video signal, the process proceeds to step 606 to allow the user to select whether to register image information. When registering an image, the process proceeds to step 607, and a still image such as a photograph or a picture is input by the image input unit 102 by itself. Alternatively, instead of capturing the image, it is possible to input the information of the character instead of the image from the key input unit 100 using a mouse or the like and store the information so that the information can be confirmed later.

After the call is completed, the process proceeds to step 609 to perform on-hook, and in step 610, the voice keyword input from the voice input unit 104 is stored in the memory. Regarding the registration of voice keywords, conventionally, when using voice recognition, it was necessary to register the words to be recognized by voice to create a standard pattern for a specific speaker, but recently, it is possible to register freely from text data. There is also a method that can create standard patterns for unspecified speakers. Therefore, the index is registered by inputting text from the key input unit 100 using a keyboard or a mouse, a method of inputting voice and converting it into text data, or storing voice information itself in the storage unit 107. It is possible. After that, the process proceeds to step 611, where the image information and the telephone number information are
The voice keyword is associated and stored in the storage unit 107.

FIG. 7 shows an example of a method for capturing a still image from a moving image. It is important that the still image can be captured easily without the user having to be aware of the operation.
Further, it is also important to avoid a failure such as having to try again in the case where the other party cannot be identified from the captured image or the image information is not intended by the user, and it is possible to reliably capture the image.

Therefore, considering that the user usually talks while looking at the face of the other party, even when making a voice call in a videophone or a video conference, it is displayed on the camera or on the screen while facing the front. It is highly possible that you are watching the video. Therefore, the voice processing unit 110 detects the voice section 701 of the voice signal transmitted while the other party is talking through the voice CODEC 111. Then, with respect to the moving image transmitted from the other party, an image is captured in the voice section at a constant cycle T, for example. The memory 70 temporarily stores the captured image data.
3 is prepared and stored sequentially. Memory 703
Since there is a limit to the capacity of the memory 703, when the memory 703 becomes full, the new information is temporally overwritten and the new information is always stored to save the memory.

Then, the information in the memory 703 is sequentially taken out by the control unit 106, the data is sent to the image processing unit 108, and a plurality of images are displayed in time series on the multi-screen on the image display unit 103. Then, the still image confirmed by the user's confirmation and selection is stored in the storage unit 10 from the memory 703.
7 is stored. In this way, since the memory 703 is temporarily stored, the still image can be registered easily and surely not only after the start of the call but also after the end of the call.

FIG. 8 is a flow chart of a sequence in which a voice keyword is uttered, recognition processing is performed, and a call is transmitted in a videophone or a videoconference system to which the present embodiment is applied.

First, from the off-hook state in step 800, when a voice keyword is uttered in step 801, the voice keyword is captured by the voice input unit 104, and the process proceeds to step 802. In step 802, the voice processing unit 110 performs processing and sends the recognition result to the control unit 106. In step 803, the control unit 106 refers to the information corresponding to the recognition result from the storage unit 107. Then, in step 804, it is confirmed whether or not the image information referred to in the image database 300 exists. If there is, go to step 805, and if not, go to step 810.

In step 805, it is further determined whether or not one voice keyword corresponds to a plurality of image information and telephone numbers. If a plurality of pieces of image information correspond to each other, go to step 806, and if not, go to step 807.
Proceed to.

In step 806, the control unit 106 sends a plurality of data to the image processing unit 108, and the image display unit 10
In step 3, the screen is divided and multi-displayed, and step 808
Proceed to. This makes it easier for the user to visually confirm.

On the other hand, in step 807, data is similarly sent from the control unit 106 to the image processing unit 108, and the image display unit 103 displays the data on one screen. At this time, if not only the image information corresponding to the keyword with the highest similarity obtained as a result of the recognition process but also the information of the next candidate is displayed on the multi-screen at the same time, the size of each screen display is set to It is also possible to make it easier to visually confirm by dividing in proportion to the height.

In step 808, selection or confirmation is performed based on the displayed image, in step 809, the telephone number corresponding to the selected image is read from the telephone number database 400 in the storage unit 107, and in step 810.
The corresponding telephone number is input from the control unit 106 to the line connection unit 112.
To the line network 113.

As described above, by displaying a plurality of image information corresponding to one voice keyword on the screen, it is possible to simultaneously call a plurality of parties in a multipoint TV conference system such as a personal computer or a workstation. In this case, since it is possible to make a call by simply speaking one keyword, it is possible to save the trouble of selecting each keyword. Further, when one telephone number corresponds to one keyword, the size of the screen display is made larger than the next candidate according to the degree of similarity obtained as a result of the recognition process, so that it can be visually confirmed. Can be made easier. Alternatively, in the case of a telephone that can send and receive a video signal, if the image information of the other party is stored, the voice keyword is uttered in the case of remembering the other party's face but not the telephone number when making a call. Since the other party can be confirmed by displaying the image, it is possible to avoid making an erroneous call. for that reason,
This method has many advantages such as when the same voice keyword is attached to a plurality of parties, for example, in the case where the names are the same, it is possible to confirm the image even if the telephone number cannot confirm. As described above, by using the image information, the visual confirmation can be easily performed, a plurality of telephone numbers can be associated with the same voice keyword, and the burden on the user to select each can be reduced.

As described above, the voice dialing can be realized in combination with the reference of the image information, and the convenience is greatly improved.

Next, an image information retrieval system can be considered as an application other than dialing the system of the present invention. For example, when the image information is loaded into a personal computer or a workstation using an image scanner or the like for editing work, a search function is required as the amount of information stored in the memory increases. In addition, there is a strong desire to shorten the time required for searching for information that is frequently referred to and to perform effective editing work. In such a case, if a voice index corresponding to the image information is added, the search can be facilitated.

FIG. 9 shows a flow chart of a registration sequence in the image information search system as another embodiment of the present invention. The configuration of the image information retrieval system is the same as that of FIG. 1, and when it is not necessary to communicate with the outside through the line network 113, the line connection unit 11 is used.
2 can be omitted.

First, in step 900, image information is input to the personal computer or workstation from the image input unit 102 such as an image scanner. Then, step 901
Then, the digital data is stored in the image database 300 of the storage unit 107, and the process proceeds to step 902. Here, when capturing a still image from a moving image, if there is a sound recorded in synchronization with the moving image data,
As described with reference to FIG. 7, it is possible to detect the voice section and divide it into a plurality of still images in synchronization with it. In step 902, it is confirmed whether a voice index is added to the captured image data. When adding an index, the process proceeds to step 903 and the index is added. As a method of assigning a voice index, it is possible to input text using a keyboard or a mouse, a method of inputting voice and converting it into text data, or storing voice information itself in the storage unit 107. Then, in step 904, the storage unit 10 associates the input image information with the index information.
7 is stored and the registration is completed. Also, if the data that has been captured once is stored as original information, when the original information is processed or edited, for example, the same information as the original is automatically added without adding the index information each time. You can also do it.

FIG. 10 shows a flow chart of a recognition sequence in the image information retrieval system as another embodiment of the present invention.

The processing flow in FIG. 10 is the off-hook of step 800, the confirmation of whether there is image information of step 804, and the step 80 of FIG.
Reading phone numbers from the database in step 9, step 8
This is a common process except that the call is sent to the ten network 113.

In the image information retrieval system as described above,
When referring to different image information while looking at the screen which is the editing work, the search can be performed by the voice keyword. Also, even if the same category or directory is referenced, no key or mouse operation is required, and even if the layers become deeper as the amount of information increases, it is possible to search by voice. It does not take time to operate or check. Further, for example, if a voice index is added to the original image, even when searching after editing, by displaying a link like a tree to the original image information, which original image was edited Can be checked, and other information related to the original image can be searched at the same time. As a result, the degree of freedom in searching is further expanded. In addition to the above explanation, of course, when storing image information in a digital camera or digital video camera terminal, voice keywords are added at the same time as a data format,
It can also be used by directly importing it to a personal computer or the like.

Further, by using the image information retrieval system described above, the following uses can be considered. For example, when displaying explanatory materials for an audio graphics conference on the screen, if voice keywords and image information are stored in advance in the storage unit 107 in association with each other, the storage unit 107 can be referred to when necessary. You can search by voice without spending time. Furthermore, by using voice keywords for unspecified speakers, other users can use the keywords on their own terminals by simultaneously transmitting the image information and the voice keyword that is the index to the other party's terminal. become. As a result, there is no need to newly register by oneself, and the search time can be greatly shortened, which is a great advantage. Alternatively, even if the terminal side does not hold the storage unit 107 in which the voice keyword and the image information are individually associated with each other, by holding the database for the unspecified speaker on the server side, many users can It is also possible to make it accessible and used in common.

Furthermore, services in the medical field can be considered. For example, even for medical image databases,
Searching can be facilitated by associating voice keywords. Therefore, it can be used for pathological diagnosis in which a remote hospital is connected to a specialty hospital in an urban area with an ISDN line to transfer and exchange patient diagnostic data.

The present invention has been described above with reference to some examples, but it goes without saying that various other modifications can be made without departing from the spirit of the present invention.

[0046]

As described above in detail, according to the present invention, a plurality of objects can be searched by one voice keyword, and visual information such as image information corresponding to them can be provided, so that the user can search and confirm. The work load can be reduced.

[Brief description of drawings]

FIG. 1 is a block diagram of a terminal device (terminal device with a telephone function) of a voice image communication system according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a storage unit in FIG.

FIG. 3 is an explanatory diagram showing an image database in a storage unit of FIG.

FIG. 4 is an explanatory diagram showing an image information database and a telephone number information database in the storage unit of FIG.

FIG. 5 is an explanatory diagram showing an example of a network configuration in the audiovisual communication system according to the embodiment of the present invention.

FIG. 6 is a flowchart showing an example of a registration sequence according to the embodiment of the present invention.

FIG. 7 is an explanatory diagram showing an example of capturing a still image at the time of registration according to the embodiment of the present invention.

FIG. 8 is a flowchart of a sequence in which a voice keyword is uttered, recognition processing is performed, and a call is transmitted in a videophone, a videoconference system, and the like as an application example of one embodiment of the present invention.

FIG. 9 is a flowchart of a registration sequence in the image information search system according to another embodiment of the present invention.

FIG. 10 is a flowchart of a recognition sequence in the image information search system according to another embodiment of the present invention.

[Explanation of symbols]

 100 key input unit 101 display unit 102 image input unit 103 image display unit 104 voice input unit 105 voice output unit 106 control unit 107 storage unit 108 image processing unit 109 image CODEC 110 voice processing unit 111 voice CODEC 112 line connection unit 113 network

Claims (4)

[Claims] 1. A voice image communication system for transmitting a telephone number using voice, comprising: a voice input section for inputting a voice, a key input section for inputting a telephone number, and an image input section for inputting an image. A storage unit that stores a telephone number input from the key input unit for redialing, a storage unit that stores an image from the image input unit, a voice keyword input from the voice input unit, and the image An audiovisual communication system comprising: a storage unit that stores an image input from an input unit and a telephone number input from the key input unit in association with each other. 2. In a voice image communication system for transmitting a telephone number using voice, a voice input section for inputting voice, a key input section for inputting a telephone number, and an image input section for inputting an image. A storage unit that stores a telephone number input from the key input unit for redialing, a storage unit that stores an image from the image input unit, and one voice keyword input from the voice input unit. And a plurality of image information input from the image input unit and a storage unit that stores the corresponding telephone number input from the key input unit in association with each other. system. 3. A voice image communication system for transmitting a telephone number using voice, comprising: a voice input section for inputting a voice, a key input section for inputting a telephone number, and an image input section for inputting an image. A storage unit that stores a telephone number input from the key input unit for redialing, a storage unit that stores an image from the image input unit, a voice keyword input from the voice input unit, and the image An image input unit, a storage unit that stores the telephone number input from the key input unit in association with each other, and an image display unit capable of multi-displaying a plurality of image information simultaneously. Audio-visual communication system. 4. An image information retrieval system for retrieving image information using voice, wherein an image input section for inputting an image, a voice input section for inputting voice, and the input voice are stored as voice keywords. A storage unit; a storage unit that stores the image from the image input unit; a storage unit that stores a plurality of image information in association with one voice keyword input from the voice input unit; An image information retrieval system, comprising: an image display unit capable of simultaneously displaying multiple pieces of image information.