JPH11355747A - Video/sound communication equipment and video conference equipment using the same equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To unnecessitate transmission of a picture signal to be subordinately used during a conference by preliminarily transmitting a picture to the receiver side by adding an identifier to it and registering it at the transmitter side, selecting a picture corresponding to the identifier from a picture database and displaying it at the receiver side when the identifier is transmitted during the conference. SOLUTION: A video, a sound and a picture signal obtained by picture and sound encoders 41, 42 and a compressor 43 from a camera 1, a microphone 2 and a picture database 3 are processed, multiplexed and transmitted by the transmitter side. The received multiplexed signal is separated by a separator 51 and an original signal is outputted to a monitor 6, a speaker 7 and a picture database 58 via a switch 52 by picture and sound decoders 53, 54 and an extension device 55 at the receiver side. The picture signal is preliminarily transmitted to the reception side, and registered there by adding the identifier to it at display control signal input 91 by the transmission side and when the identifier is transmitted during the conference, the corresponding picture is displayed on the monitor 6. Thus, a subordinate picture is displayed without using paper.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio communication device for communicating video and audio, and more particularly, to a communication capable of transmitting an image prepared in advance in addition to video / audio and displaying the image on a monitor of the other party. The device is applied to a video conference system or the like.

[0002]

2. Description of the Related Art In general, when a video conference is used, the video of the other party is often viewed during the conference, but in the case of contents that are difficult to understand only by audio, it is necessary to supplementally use image information to operate the conference. May be effective. For example, there is a case in which a document such as a chart is desired to be displayed and explained to the other party during the meeting.

[0003]

However, in such a case, in a conventional video conference, it is difficult to freely select such supplementary information from a remote side. Thus, a method of selecting necessary supplementary information according to an instruction from a remote side is adopted. In this method, paper information is absolutely necessary. In addition, the other party, that is, the person receiving the explanation based on the supplementary information, needs to receive the instruction of the presenter through the video conference system and then select and view the corresponding supplementary information, which is time-consuming and takes time from the video screen. There are also problems such as having to keep an eye on them.

[0004] A presentation conferencing system as disclosed in Japanese Patent Application Laid-Open No. 6-169456, for example, is known as long as it enables only real-time transfer of presentation information. In this conference system, the presentation information created by the editing unit is transferred to the recording unit of the terminal of the conference member and recorded before the conference starts, and at the time of the conference, the presentation is sent to the execution unit simply by sending necessary instructions. Let's do it.

However, the method disclosed in Japanese Patent Application Laid-Open No. Hei 6-169456 merely transmits the presentation information including the image and the audio information to the presentation terminal in advance and operates the timing of performing the presentation from a remote place. However, there is a problem in that it is not possible to appropriately display necessary image information or the like on a monitor in place of a video during a video conference or the like.

It is an object of the present invention to provide a video / audio communication apparatus capable of displaying auxiliary image information in a timely manner without using paper, instead of a normal video or simultaneously.

[0007]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, a video / audio communication apparatus according to the present invention comprises: a transmitting apparatus for transmitting video and audio; a receiving unit for receiving video and audio; A video / audio communication device comprising: a receiving device having a reproduction unit that reproduces audio; and the transmitting device further specifies an image transmitting unit that transmits an image, and a specific image from the images. A display control signal transmitting unit for transmitting a display control signal for instructing display of a lever image. On the other hand, the receiving device further includes an image storage unit that receives and stores the image, and an image display unit that receives the display control signal from the transmitting device, selects the specified image from the image storage unit, and displays the selected image. ing. Then, when the video / audio receiving device receives the display control signal, an image specified to be displayed is displayed on the video display unit instead of the video. This display may be displayed on the entire video display unit instead of the video, or may be displayed on a part of the video.

In the video / audio communication apparatus according to the present invention, instead of inputting the above-described display control signal for designating an image on the transmitting apparatus side, display may be controlled based on audio. That is, in addition to the above basic configuration, the transmitting device includes an image transmitting unit that transmits an image corresponding to the voice keyword, while the receiving device includes an image storing unit that receives and stores the image, and an image storing unit that receives the image. A voice recognition unit for recognizing a keyword. When a voice keyword is extracted from a voice, an image corresponding to the voice keyword is selected from the image storage unit and displayed. Here, the audio keyword may indicate not only the image to be displayed but also the start or end of the display.

As a more specific configuration of the video / audio communication apparatus according to the present invention, a video signal transmitting section for converting an input video to a video signal and transmitting the same, and converting an input audio to an audio signal for transmission. Video / audio transmitting device having an audio signal transmitting unit for transmitting a video signal, a video signal receiving unit for receiving a video signal, an audio signal receiving unit for receiving an audio signal, and a video reproducing unit for reproducing the received video signal into video And a video / audio transmission device having a video / audio receiving device having an audio reproduction unit for reproducing a received audio signal into audio. The video / audio transmission device further includes an image transmission unit that transmits the image to the video / audio reception device, and a display control signal that designates a specific image from the images and instructs display of the image. And a display control signal transmitting unit for transmitting. On the other hand, the video / audio receiving unit further includes an image database that receives and stores images and audio keywords, and an image display unit that receives a display control signal and selects and displays a specified image from the image storage unit. It has.

As described above, an image to be displayed on the receiving device side is provided with a voice recognition unit for recognizing a voice keyword from a voice signal instead of inputting a display control signal on the transmitting side, When a voice keyword is extracted, an image corresponding to the voice keyword may be selected from the image database and displayed on the image display unit.

In such a configuration, the video signal transmitting section has a video encoder for encoding a video and outputting a video encoded signal, and the audio signal transmitting section encodes audio and outputs an audio encoded signal. Audio encoder. Further, the image is transmitted by a compressed image signal compressed by the compressor. Further, the video, audio, and image may be multiplexed with the coded video signal, the coded audio signal, and the compressed video signal, and may be transmitted by the multiplexed signal. The multiplexed signal is separated into a video coded signal, a voice coded signal, and a compressed image signal by a demultiplexer on the receiving device side, and further separated by a video decoder, a voice decoder and a decompressor. The signal is reproduced.

The image signal selected by the voice keyword and extracted from the image database and the decoded video signal may be synthesized by a synthesizer and displayed simultaneously on a video display unit.

Using the video / audio communication apparatus as described above, a camera for recording video in the video signal transmitting section, a microphone for recording audio in the audio signal transmitting section, and a monitor for displaying video in the video reproducing section are provided. Alternatively, the audio reproducing unit may be provided with a speaker for outputting audio, so that the video conference apparatus can be configured.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, a video / audio communication apparatus of the present invention and a video conference apparatus using the same will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing the configuration of an embodiment of a video / audio communication apparatus according to the present invention.

As shown in FIG. 1, in this embodiment, in order to facilitate understanding, the transmitting device 4 is shown as comprising a receiving device 5 and a transmission line 45 connecting the two. . When applied to an actual video conference device or a communication device, it is normal to provide a transmission device and a reception device on both sides. Therefore, if the communication device of the present embodiment shown in FIG. Good.

In FIG. 1, a transmitting device 4 includes a camera 1 for capturing video, a microphone 2 for capturing audio, and an image encoder 4 for encoding captured video and audio, respectively.
1. An audio encoder 42 is provided. Further, it is provided with an image database 3 for transmitting an image desired to be shown to the other party in addition to a normal video used for a video conference or the like, and a compressor 43 for compressing information taken from the image database 3. The image database 3 may be configured so that necessary images on the spot are sequentially input to the compressor 43 by a scanner or the like, or all image information to be transmitted is recorded in advance. Is also good. The image referred to here may be not only a still image but also a moving image, or may include text information such as a text, a table, and a graph.

The video, audio, and image signals obtained by the image encoder 41, the audio encoder 42, and the compressor 43 are multiplexed by the multiplexer through the switch 44, and are multiplexed by the receiver via the transmission line 45. 5 is transmitted. In the present embodiment, the image information to be supplementarily used is transmitted to the receiving device 5 in advance.

The receiver 5 separates the received multiplexed signal into a video signal, an audio signal, and an image signal, a switch 52, an image decoder 53, an audio decoder 54, and a decompressor. And a vessel 55. The original video signal, audio signal, and image signal are reproduced by the image decoder 53, the audio decoder 54, and the expander 55, respectively.

The previously transmitted image signal is expanded by an expander 55 and then stored in an image database 58. Note that each image signal is provided with an identifier so as to be able to respond to a display instruction transmitted from the transmitting apparatus later. That is, the transmitting device registers in advance the image used in the conference or the like in the image database 3 together with the identifier. The registered image signal is compressed by the compressor 43 and transmitted to the other party, decompressed by the decompressor 55, and registered in the image database 58 of the other party.

Next, the operation of displaying an image during a conference or the like will be described. During a conference, a video signal captured by the camera 1 of the transmission device 4 is normally transmitted and displayed on the monitor 6 of the reception device 5. Also, the transmitting device 4
The sound on the side is also transmitted to the receiving device 5 side and reproduced by the speaker 7.

If the transmitting device 4 wants to use, for example, an image registered in advance in the receiving device 5 for the sake of explanation, the transmitting device 4 uses the identifier for identifying the image to be displayed. The display control signal including the display control signal is input from the display control signal input 91 and transmitted. The identifier for specifying this image may be input from a keyboard or the like.

The receiving device 5 that has received the display control signal is separated into a display control signal output 92, and a corresponding image is selected from the image database 58 by the extracted display control signal and displayed on the monitor 6. This display may be completely replaced with a normal image (the monitor is not shown when it is displayed on a monitor different from the image), or the display screen is divided and partially displayed. May be. When a part of the image is displayed, as shown in FIG. 1, the video signal from the image decoder 59 and the image signal extracted from the image database 58 are synthesized by the synthesizer 59.

Next, a description will be given of a display using identification of an image by a keyword using sound. In the above embodiment, the image to be displayed is specified by directly inputting the identifier, but in the video / audio communication device of the present invention,
A feature is that a keyword for specifying an image is extracted from a sound, and a screen to be displayed is specified by the keyword.

FIG. 2 is a block diagram showing a configuration of a video / audio communication apparatus according to a second embodiment of the present invention.

The structures of the transmitting device 4 and the receiving device 5 are almost the same as those of the first embodiment of the present invention shown in FIG.
However, the present embodiment is different in that a speech recognizer 56 and a data discriminator 57 are provided after the speech decoder 54. On the other hand, the transmitting device 4 does not need a function for inputting an identifier for specifying an image to be displayed.

In the present embodiment, the auxiliary image is first registered in the image database 3 of the transmitting device 4 together with the voice as the keyword. The registered data is compressed by the compressor 43 and transmitted to the destination, decompressed by the decompressor 55, and registered in the image database 58 of the destination. The keyword is also registered in the speech recognizer 56 and associated with the image database.

An image from the camera 1 is transmitted to an image encoder 41.
And a multiplexer 45 via a switch 44
Supplied to The sound from the microphone 2 is sent to a sound encoder 4
2 and the multiplexor 4 through the switch 44
5. The switch 44 switches between video, audio, and data. In the multiplexer 45, the coded video and the coded audio are multiplexed by the multiplexor 45 and transmitted to the receiving device 5 via the transmission line 45.

In the receiving device 5, the multiplexed signal transmitted via the transmission line 45 by the demultiplexer 51 is separated into coded video and coded audio. Sent to. The encoded video is supplied to the image decoder 53, decoded, and supplied to the synthesizer 59. On the other hand, the coded speech is supplied to the speech decoder 54, decoded, and supplied to the speech recognizer 56 and the speaker 7.

Here, the speech recognizer 56 is configured to recognize the inputted speech and, if the same speech as the keyword registered in advance is found, send the result to the data decision unit 57. ing. The data determiner 57 selects data corresponding to the recognized result from the image database 58 and supplies the data to the synthesizer 59. In the synthesizer 59, the decoded image and the data are synthesized and sent to the monitor 6. On the monitor 6, a composite screen of the transmitted video and the data selected by the audio is displayed.

The selection and display of an image by a keyword using the communication device having the above-described configuration will be described more specifically.

On the transmitting device 4 side, the voice uttered for a conference or the like is encoded by the encoder 42 and
Sent to the side. Now, when the speaker wants to supplementally display an image on the monitor of the receiving device 5 for explanation or the like, the speaker utters predetermined words for specifying and displaying the image. For example, if a plurality of drawings and photographs are registered in the image database 58 of the receiving device 5 in advance, and if it is desired to display FIG. 5 on the monitor 6 from among them, a word including the keyword “display FIG. ,
For example, a word "Please see FIG. 5 for display."

Then, the audio signal is decoded by the audio decoder 54 of the receiving device 5 and contains the keyword for specifying the image “FIG. 5” by the audio recognizer 56. FIG. 5 is selected from images registered in the image database 58 in advance. Then, since the keyword “display” is also extracted, the image is displayed on the monitor 6. This display may be displayed instead of the image displayed on the monitor 6, or may be partially displayed, as in the first embodiment.

When the keyword is extracted on the receiving device 5 side and the display of the corresponding image is started, for example, “end”
Such a keyword for ending the display of the image is registered, and when this sound is extracted, the display of the auxiliary image is ended, and the image is automatically returned to the original image.

As described above, in addition to the keyword for specifying the image, a keyword for identifying and controlling the start and end of the display may be provided. After the time display, the display may automatically return to the original video screen, or may have a keyword indicating the end only in the case of the end.

By adopting the configuration of the second embodiment of the present invention, auxiliary video data paired with a keyword is stored in the image database, and the auxiliary video data corresponding to the keyword of the received audio is stored. Is selected and combined with the received video and displayed, so that data can be selected in a timely manner without operation on the receiving side.

Although the video / audio communication apparatus of the present invention described above has been described by taking as an example a communication apparatus in which a transmitting apparatus and a receiving apparatus are paired, each terminal normally has a pair of a transmitting apparatus and a receiving apparatus. A video conference is configured so that video and audio can be communicated in both directions. In the above embodiment,
In order to make the understanding of the invention easier, a communication mode between only two terminals has been described as an example. However, a video conference device is configured between three or more terminals, and a voice transmitted from one transmission device is used. It goes without saying that keywords for image display can be extracted by two or more receiving devices, and an image can be displayed on each receiving device.

[0038]

As described above, the video / audio communication apparatus according to the present invention transmits an image to the receiving apparatus with an identifier for identifying the image in advance, registers the image, and registers the identifier during the conference. Since the image is selected and displayed from the image database on the receiving device side by transmitting, the transmission of the image signal to be used as an auxiliary during the conference is unnecessary. For this reason, the image desired to be displayed instantaneously on the receiving side can be displayed.

Further, the video / audio communication apparatus of the present invention
By registering the auxiliary video paired with the keyword in advance in the image database, the received voice is recognized by voice recognition, data corresponding to the voice is selected from the image database, and instead of the received video, or It is also possible to adopt a configuration in which images are combined and displayed. According to this configuration, in a video conference or the like, data corresponding to the sound on the transmitting side can be displayed on the receiving side simultaneously with the sound without operation on the receiving side.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of a video / audio communication device according to the present invention.

FIG. 2 is a block diagram showing a configuration of a video / audio communication device according to a second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Camera 2 Microphone 3 Image database 4 Codec transmission unit 41 Image encoder 42 Audio encoder 43 Compressor 44 Switcher 45 Multiplexer 5 Codec receiver 51 Separator 52 Switcher 53 Image decoder 54 Audio Decoder 55 Decompressor 56 Speech Recognizer 57 Data Judge 58 Image Database 59 Synthesizer 6 Monitor 7 Speaker 8 Transmission Line 91 Display Control Signal Input 92 Display Control Signal Output

Claims (17)

[Claims] 1. A video comprising: a transmitting device for transmitting video and audio; a receiving unit for receiving the video and audio; and a video / audio receiving device having a reproducing unit for reproducing the received video and audio. An audio communication device, wherein the video / audio transmission device further includes: an image transmission unit that transmits an image; and a display control signal that designates a specific image from among the images and instructs display of the image. A display control signal transmitting means for transmitting, the video / audio receiving apparatus further comprising: an image storing means for receiving and storing the image; and storing the specified image by receiving the display control signal. And an image display means for selecting from the means and displaying the selected image instead of the image. 2. The video / audio communication device according to claim 1, wherein the image is displayed on a part of the video. 3. A video, comprising: a transmitting device for transmitting video and audio; a receiving unit for receiving the video and audio; and a video / audio receiving device including a reproducing unit for reproducing the received video and audio. An audio communication device, wherein the video / audio transmission device further includes an image transmission unit that transmits an image corresponding to an audio keyword; the video / audio reception device further receives and stores the image; And a voice recognition unit that recognizes the keyword from the received voice. When the voice keyword is extracted, the image corresponding to the voice keyword is selected from the image storage unit and displayed. A video / audio communication device characterized in that: 4. The video / audio communication device according to claim 3, wherein the image is displayed in place of the video. 5. The video / audio communication device according to claim 3, wherein the image is displayed as a part of the video. 6. The image processing apparatus according to claim 3, wherein the voice keyword includes a display start keyword for instructing a start of display of the image and a display end keyword for instructing an end of display of the image. The video / audio communication device according to claim 5. 7. A video / audio transmission device having video signal transmission means for converting an input video into a video signal and transmitting the video signal, and audio signal transmission means for converting an input audio to an audio signal and transmitting the audio signal. A video signal receiving unit that receives the video signal; an audio signal receiving unit that receives the audio signal; a video reproducing unit that reproduces the received video signal into a video and displays the video on a video display unit; A video / audio transmission device comprising: a video / audio reception device having a video / audio reception device for reproducing the audio signal into voice. Image transmitting means for transmitting to a receiving device; display control signal transmitting means for transmitting a display control signal designating a specific image by designating a specific image from the images; The receiving apparatus further includes: an image database that receives and stores the image and the audio keyword; and receives the display control signal, selects a designated image from the image storage unit, and displays the selected image on the video display unit. 7. The video / audio communication device according to claim 6, wherein the image is displayed in place of the video. 8. The image display device according to claim 7, wherein the image display unit displays the image on a part of the video display unit.
The video / audio communication device as described. 9. A video / audio transmission device having video signal transmission means for converting an input video into a video signal and transmitting the video signal, and audio signal transmission means for converting an input audio into an audio signal and transmitting the audio signal. A video signal receiving means for receiving the video signal; an audio signal receiving means for receiving the audio signal; a video reproducing means for reproducing the received video signal into a video; and converting the received audio signal to audio. A video / audio transmission device comprising: a video / audio reception device having an audio reproduction unit for reproducing, wherein the video / audio transmission device further transmits the image and the audio keyword to the video / audio reception device. The video / audio receiving apparatus further comprises: an image database configured to receive and store the image and the audio keyword; and Voice recognition means for recognizing a voice keyword; and image display means for selecting and displaying the image corresponding to the voice keyword from the image database when the voice keyword is extracted. Video and audio transmission equipment. 10. The video playback unit includes a video display unit that displays the video, and the image display unit displays the image instead of the video displayed on the video display unit. The video / audio communication device according to claim 9. 11. The image display device according to claim 9, wherein the video playback unit includes a video display unit that displays the video, and the image display unit displays the image on a part of the video display unit. Video and audio communication device. 12. The voice recognition unit further includes a start instruction keyword for instructing start of display of the image, and an end instruction keyword for instructing termination of display of the image. Image display start means for starting display of the image when a start instruction keyword is extracted; and image display end means for ending display of the image when the end instruction keyword is extracted. The video / audio communication device according to any one of claims 8 to 11, characterized in that: 13. The video signal transmitting unit includes a video encoder that encodes the video and outputs a video encoded signal, and the audio signal transmitting unit encodes the audio to generate an audio encoded signal. An audio encoder for outputting, the video / audio transmission device further includes: a transmission-side image database for storing the image; a compressor for compressing the image to output a compressed image signal; and the video signal. Instead of the audio signal and the image signal,
A multiplexer that multiplexes the video encoded signal, the audio encoded signal, and the compressed image signal and transmits a multiplexed signal, the video / audio receiving device further includes: A separator that separates the encoded signal, the audio encoded signal, and the compressed image signal from each other; a video decoder that decodes the encoded video signal and outputs a decoded video signal; and decodes the encoded audio signal. 10. An audio decoder for converting an encoded video signal into an encoded audio signal and outputting an encoded image signal to output an expanded image signal. Video and audio communication equipment. 14. The video reproducing means according to claim 8, further comprising a synthesizer for synthesizing the image signal selected by the audio keyword and extracted from the image database and the decoded video signal. The video / audio communication device according to claim 11. 15. The video signal transmitting unit includes a video encoder that encodes the video and outputs a video encoded signal, and the audio signal transmitting unit encodes the audio to generate an audio encoded signal. An audio encoder for outputting, the video / audio transmission device further includes: a transmission-side image database for storing the image; a compressor for compressing the image to output a compressed image signal; and the video signal. Instead of the audio signal and the image signal,
A multiplexer that multiplexes the video encoded signal, the audio encoded signal, and the compressed image signal and transmits a multiplexed signal, the video / audio receiving device further includes: A separator that separates the encoded signal, the audio encoded signal, and the compressed image signal from each other; a video decoder that decodes the encoded video signal and outputs a decoded video signal; and decodes the encoded audio signal. 15. An audio decoder for converting an encoded video signal into an audio decoded signal and outputting an expanded video signal by expanding the encoded video signal. The video / audio communication device according to claim 1. 16. The video / audio communication apparatus according to claim 1, wherein the image includes a still image, a moving image, or character data. 17. A video conference apparatus comprising the video / audio communication device according to claim 7, wherein the video signal transmitting means includes a camera for recording video. The audio signal transmitting unit includes a microphone that records audio, the video reproduction unit includes a monitor that displays the video, and the audio reproduction unit includes a speaker that outputs the audio. Characteristic video conference equipment.