JP2004228737A - Conference supporting device, system and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a conference supporting device for letting attendees at a conference know contents of proceedings when the attendees go back to their sheets even if the attendees leave halfway through the conference, and to provide a conference supporting system and a conference supporting program. <P>SOLUTION: Picture of entire attendees at the conference are imaged by an imaging section 105. A section 113 for recognizing the attendees who leave halfway through the conference detects a change in input image data containing the picture by the pattern matching of image data, recognizes the attendees who leave halfway through the conference and then go back to their sheets from the change in the detected input image data, and measures a time and a time interval when the attendees leave halfway through the conference. Based on the measured time and time interval when the attendees leave halfway through the conference, a speech signal stored in a speech signal storage section 111 is converted to a voice by a voice output section 112, thus notifying the contents of proceedings of the attendees who leave halfway through the conference and go back to their sheets. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

で表される。ただし、ｃ^２＝ａ^２＋ｂ^２であり、ａおよびｂは双曲面の形状を定義する定数であり、ｃは焦点の位置を定義する定数である。
【００８１】
この双曲面ミラー１０４ａは、２つの焦点▲１▼と▲２▼とを有し、外部から一方の焦点▲１▼に向かう光は双曲面ミラー１０４ａで反射され、全て他方の焦点に向かうという特徴を有する。したがって、双曲面ミラー１０４ａの回転軸と撮像レンズの光軸とを一致させると共に、他方の焦点位置▲２▼に撮像レンズの第１主点を配置することにより、撮像部１０５で撮影された画像が、一方の焦点▲１▼を視点中心として視野方向によって視点位置が変わらない画像となる。
【００８２】
また、上記撮像部１０５は、ビデオカメラ等であり、図３における双曲面ミラー１０４ａを介して得られる光学像を、ＣＣＤやＣＭＯＳ等の固体撮像素子を利用して画像データに変換する。なお、撮像部１０５のレンズは、一般的な球面レンズであっても非球面レンズであってもよく、第一主点が焦点位置▲２▼にあればよい。
【００８３】
上記撮像部１０５は、ＣＣＤやＣＭＯＳなどの固体撮像素子を利用した撮像カメラで構成され、この撮像カメラにより双曲面ミラー１０４ａに投影された光学像を全方位画像データとして撮像する。
【００８４】
図４はこの実施形態による会議の全出席者および発言者の映像を含む画像を示す図である。
【００８５】
また、図４（ａ）に示す画像は、会議室の中央部に設置された撮像カメラ（光学系１０４、撮像部１０５）により撮像された会議の全出席者Ａ、Ｂ、Ｃ、Ｄの映像を含む全方位画像であり、全方位画像データとしてフレーム単位で会議支援装置１のバッファメモリ１０３に順次一時記憶される。
【００８６】
また、図４（ｂ）に示す画像は、画像データ生成部１０７により、バッファメモリ１０３に一時記憶された全方位画像データから変換情報に基づいて生成された会議の全出席者Ａ、Ｂ、Ｃ、Ｄの映像を含むパノラマ画像と発言者認識部１０９により認識された発言者Ａの映像を含む透視画像である。この図４（ｂ）に示す画像が、外部から会議に参加した端末装置２の利用者に送信される。
【００８７】
図５，図６，図７，図８は中座者認識部１１３の動作を説明するための図で、図５は例えば会議の全出席者が揃ったときの映像であり、図６は１人が欠席または中座したときの映像である。また、図７は出席者および座席に対応したマイクのＩＤコードの１例を示す図である。さらに、図８は会議のタイムテーブルの１例を示す図である。
【００８８】
中座者認識部１１３の中座者認識プログラムは、タイマーなどにより逐次所定時間間隔で撮像部１０５で取り込んだ画像データの変化をパターンマッチングにより比較検出しており、例えば図５から図６のように画像データが変化したとき、中座者を図７のＩＤコードから割り出すとともに図８に示すようなタイムテーブルに座席位置と出席者コードを対応させて記録しておき、画像データが図５のように戻ったとき、その間の時間間隔や中座者を容易に特定できる。
【００８９】
図９〜図１１はこの実施形態による会議支援システムの基本動作の手順を示すフローチャートである。なお、図９〜図１１において、会議支援装置１は、所定の端末装置２と、通信回線３を介して、画像データ、音声、議事録、ｅメールなどを通信可能に接続されているものとし、会議支援装置制御部１０１の制御により、会議室内に設置された会議支援装置１が起動し、会議が開始されたとする。
【００９０】
まず、基本動作を開始すると、ステップＳ１で、会議支援装置１において、音声入力部１０８または通信部１１８に入力情報があるか否かを判定する。そうであれば、ステップＳ２に進み、そうでないならば、入力待ち状態になる。
【００９１】
次に、ステップＳ２に進み、音声入力部１０８または通信部１１８に入力情報があれば、会議室内に設置された会議支援装置１の音声入力部１０８（マイクロフォン）からの音声の入力か、端末装置２からの入力かを判定する。音声入力部１０８からの音声の入力ならば、ステップＳ３に進み、端末装置２からの入力ならば、ステップＳ４に進む。
【００９２】
そして、ステップＳ３で音声入力部１０８からの音声の入力であれば、発言者の音声を音声信号に変換する。
【００９３】
また、ステップＳ４で端末装置２からの入力情報が音声信号か、ｅメールの文字データかを判定する。音声信号ならば、図１０に示すステップＳ５に進む一方、ｅメールの文字データならば、図１０に示すステップＳ１３に進む。
【００９４】
次に、図１０に示すステップＳ５で発言者認識部１０９を用いて、使用されたマイクロフォンに付加されたＩＤコードまたは端末装置２のＩＤコードから発言者を認識し、対応する発言者コードを特定する。
【００９５】
次に、ステップＳ６に進み、入力された音声信号を発言者コードと対応して音声信号記憶部１１１に記憶する。
【００９６】
次に、ステップＳ７に進み、光学系１０４において、会議室周囲３６０°の視野領域を有する光学像を投影する。
【００９７】
次に、ステップＳ８に進み、撮像部１０５において、光学系１０４に投影された光学像を全方位画像データとして撮像する。
【００９８】
次に、ステップＳ９に進み、撮像された全方位画像データを会議支援装置１のバッファメモリ１０３にフレーム単位で順次一時記憶する。
【００９９】
次に、ステップＳ１０に進み、画像データ生成部１０７を用いて、バッファメモリ１０３に一時記憶された全方位画像データから会議の全出席者の映像を含むパノラマ画像データおよび発言者認識部１０９により認識された発言者の映像を含む透視画像データを生成する。このとき、位置座標特定部１１０により、発言者の入力画像の位置座標が特定される。生成した画像データを表示部１１７に表示してもよい。
【０１００】
次に、ステップＳ１１に進み、通信部１１８を用いて、通信回線３を介して、発言者の音声信号と一緒に、画像データ生成部１０７により生成画像データ（パノラマ画像データおよび透視画像データ）を所定の端末装置２に送信する。なお、発言者が端末装置２の利用者ならば、音声信号を送信しない。
【０１０１】
次に、ステップＳ１２に進み、音声信号変換部１１４を用いて、音声信号記憶部１１１に記憶された音声信号を文字データに変換する。
【０１０２】
次に、ステップＳ１３に進み、変換された文字データを議事録として議事録記録部の一例としての記憶媒体１１９に記憶する。
【０１０３】
次に、ステップＳ１４に進み、作成した議事録を表示部１１７に表示する。
【０１０４】
次に、図１１に示すステップＳ１５に進み、端末装置２から議事録の送信要求があったか否かを判定する。そうならば、ステップＳ１６に進む一方、そうでないならば、ステップＳ１７に進む。
【０１０５】
そして、ステップＳ１６で端末装置２から議事録の送信要求があれば、画面に表示されている現在の議事録を端末装置２に順次送信する。
【０１０６】
次に、ステップＳ１７で中座者認識部１１３を用いて、会議を中座した出席者がいるか否かを判定する。そうならば、ステップＳ１８に進む一方、そうでないならば、図９のステップＳ１に戻る。
【０１０７】
ここでは、中座者認識部１１３は、バッファメモリ１０３にフレーム単位で相前後して記憶された全方位画像データの中座者の動きにより発生する差を、パターンマッチングにより検知し、検知した画像データの差から中座者であるか否かを認識するように構成されているものとする。
【０１０８】
次に、ステップＳ１８で会議を中座した出席者がいるならば、出席者が中座した時刻および時間間隔を測定する。
【０１０９】
次に、ステップＳ１９に進み、中座した出席者が元の場所に戻ってきたか否かを判定する。そうならば、ステップＳ２０に進み、そうでないならば、ステップＳ１８に戻る。
【０１１０】
そして、ステップＳ２０で元の場所に戻ってきた出席者が、中座中の発言内容の音声出力を要求したか否かを判定する。そうならば、ステップＳ２１に進む一方、そうでないならば、図９のステップＳ１に戻る。
【０１１１】
次に、ステップＳ２１で音声信号記憶部１１１から中座中の発言内容の音声信号を検索して音声出力部１４に出力する。
【０１１２】
次に、ステップＳ２２に進み、会議の終了指示が操作されたか否かを判定する。そうならば、ステップＳ２３に進む一方、そうでないならば、図９のステップＳ１に戻り、入力待ち状態になる。
【０１１３】
次に、ステップＳ２３で会議の終了指示が操作されたならば、端末装置２に会議終了を通知し、会議を終了する。
【０１１４】
以上、会議支援装置１と端末装置２とからなる会議支援システムについて説明したが、例えば、外部の複数の会議室に同じ会議支援装置１を互いに通信可能にそれぞれ設置して、会議支援装置１の各表示部１１７の画面に各会議室の出席者、発言者などを自動表示して、お互いに顔を見ながら会議を進行することができる。
【０１１５】
また、上記実施の形態では、最大周囲３６０°の視野領域を有する光学像を投影する光学系１０４を用いた会議支援装置について説明したが、光学系はこれに限らず、周囲３６０°の視野領域の少なくとも一部を有する光学像を投影する光学系を備えた会議支援装置にこの発明を適用してもよい。
【０１１６】
なお、上記端末装置２において、利用者が中座して戻ってきたときに、受信情報記憶部２０５に記憶された中座中の音声信号を音声出力部２０８により音声に変換して出力し、受信情報記憶部２０５と音声出力部２０８で、戻ってきた利用者に中座中の議事内容を通知する議事内容通知部を構成してもよい。
【０１１７】
ところで、上記実施の形態の会議支援装置の機能は、プログラム記録媒体に記録されたプログラムによって実現される。そこで、このようなプログラムを、コンピュータ読取可能な記録媒体に記録させて、プログラム製品として提供することもできる。
【０１１８】
上記コンピュータ読取可能な記録媒体は、ＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）等のプログラムメディアである。あるいは、外部補助記憶装置に装着され読出されるプログラムメディアであってもよい。なお、いずれの場合においても、上記プログラムメディアから情報管理プログラムを読み出すプログラム読出手段は、上記プログラムメディアに直接アクセスして読み出す構成を有してもよいし、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）に設けられたプログラム記憶エリアにロードし、上記プログラム記憶エリアにアクセスして読み出す構成を有してもよい。なお、上記プログラムメディアから上記ＲＡＭのプログラム記憶エリアにロードするためのロードプログラムは、予め装置本体に格納されているものとする。
【０１１９】
ここで、上記プログラムメディアとは、本体側と分離可能に構成され、磁気テープやカセットテープ等のテープ系、フレキシブルディスク、ハードディスク等の磁気ディスクやＣＤ−ＲＯＭ（Ｃｏｍｐａｃｔ Ｄｉｓｃ−Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ），ＭＯ（ＭａｇｎｅｔＯｐｔｉｃａｌ ｄｉｓｋ：光磁気）ディスク、ＭＤ（ミニディスク）、ＤＶＤ（Ｄｉｇｉｔａｌ Ｖｅｒｓａｔｉｌｅ Ｄｉｓｃ：ディジタル多用途ディスク）等の光ディスクのディスク系、ＩＣ（集積回路）カードや光カード等のカード系、マスクＲＯＭ、ＥＰＲＯＭ（Ｅｒａｓａｂｌｅ Ｐｒｏｇｒａｍｍａｂｌｅ Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ：紫外線消去型ＲＯＭ）、ＥＥＰＲＯＭ（Ｅｌｅｃｔｒｉｏｎｉｃａｌｌｙ Ｅｒａｓａｂｌｅ Ｐｒｏｇｒａｍｍａｂｌｅ Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ：電気的消去型ＲＯＭ）、フラッシュＲＯＭ等の半導体メモリ系を含めた固定的にプログラムを坦持する媒体である。
【０１２０】
また、上記実施形態における会議支援システムは、モデム、ＬＡＮカード等を備えてインターネットを含む通信ネットワークと接続可能な構成を有している場合は、上記プログラムメディアは、通信ネットワークからのダウンロード等によって流動的にプログラムを坦持する媒体であっても差し支えない。なお、その場合における上記通信ネットワークからダウンロードするためのダウンロードプログラムは、予め本体装置に格納されているものとする。あるいは、別の記録媒体からインストールされるものとする。
【０１２１】
なお、上記記録媒体に記録されるものはプログラムのみに限定されるものではなく、データも記録することができる。
【０１２２】
【発明の効果】
以上より明らかなように、この発明の会議支援装置および会議支援システムおよび会議支援プログラムによれば、会議の全出席者の映像を含む入力画像データの変化を画像データのパターンマッチングにより検知し、検知した入力画像データの変化から会議の途中で中座した出席者または元の場所に戻ってきた出席者を認識し、その出席者が中座した時刻および時間間隔を測定する中座者認識部と、その測定された上記中座した時刻および時間間隔に基づいて、戻ってきた出席者に中座中の議事内容を通知する議事内容通知部とを備えたことにより、出席者のうちの誰かが途中で中座し、戻ってきたとき、その間の議事については他の出席者に確認する必要がなくなり、議事進行の妨げとならず、効率的に支障なく円滑に進めることができる。
【０１２３】
さらに、この発明によれば、通信回線を介して互いにデータ通信が可能な会議支援装置と端末装置とからなる電子会議システムにおいて、例えば１台の３６０°の視野領域を撮像するカメラを会議室の所定の箇所に設置することにより、外部から会議に出席した端末装置の利用者に対して、発言者の音声だけでなく、会議の出席者や発言者の画像データを会議支援装置から端末装置に送信することができる。よって、端末装置の利用者が、遠隔地から会議に参加しても、会議の雰囲気を十分に理解しながら討議できる会議支援システムを提供することができる。
【図面の簡単な説明】
【図１】図１はこの発明の実施の一形態の会議支援システムに用いられる会議支援装置の基本構成を示すブロック図である。
【図２】図２は上記会議支援システムに用いられる端末装置の基本構成を示すブロック図である。
【図３】図３は上記会議支援システムの光学系の構成を示す斜視図である。
【図４】図４は上記会議支援システムによる会議の全出席者および発言者の映像を含む画像を示す図である。
【図５】図５は上記会議支援システムにおいて会議の全出席者が揃ったときの画像を示す図である。
【図６】図６は上記会議支援システムにおいて１人が欠席または中座したときの画像を示す図である。
【図７】図７は上記会議支援システムにおける出席者および座席に対応したマイクのＩＤコードの例を示す図である。
【図８】図８は上記会議支援システムにおける会議のタイムテーブルの例を示す図である。
【図９】図９は上記会議支援システムの基本動作の手順を示すフローチャートである。
【図１０】図１０は図９に続くフローチャートである。
【図１１】図１１は図１０に続くフローチャートである。
【符号の説明】
１…会議支援装置、
１０１…会議支援装置制御部、
１０２…プログラムメモリ、
１０３…バッファメモリ、
１０４…光学系、
１０５…撮像部、
１０６…変換情報記憶部、
１０７…画像データ生成部、
１０８…音声入力部、
１０９…発言者認識部、
１１０…位置座標特定部、
１１１…音声信号記憶部、
１１２…音声出力部、
１１３…中座者認識部、
１１４…音声信号変換部、
１１５…表示部、
１１６…通信部、
１１７…受信情報記憶部、
１１８…操作部、
１１９…記憶媒体、
１２０…記憶媒体読取部、
１２１…バス、
２…端末装置、
２０１…端末装置制御部、
２０２…プログラムメモリ、
２０３…バッファメモリ、
２０４…通信部、
２０５…受信情報記憶部、
２０６…表示部、
２０７…音声入力部、
２０８…音声出力部、
２０９…カメラ部、
２１０…入力部、
２１１…バス。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a conference support device, a conference support system, and a conference support program, and more particularly, to a conference support device, a conference support system, and a conference that generate image information that conveys the atmosphere of a conference and transmit the image information to an external device via a communication line. Regarding support programs.
[0002]
[Prior art]
In recent years, with the spread of communication lines and multimedia devices capable of handling a plurality of multimedia information such as characters, graphics, images, and voices, conference rooms at points separated from each other are connected by communication lines, and image information and 2. Description of the Related Art A video conference system is known as a conference support system that allows a plurality of participants to share the same information by transmitting voices to each other and to have a conference while watching each other on a monitor.
[0003]
As a conference support system that does not require a special conference room, a voice input / output device or a user terminal (mobile phone), a conference server device for supporting a conference, a voice management device, and a network device for telephone and information communication are provided. To transmit voice and electronic documents (e-mails) during the conference to the terminal (mobile phone) of the user who attended the conference via the network device, and connect the terminal of the user who attended the conference to the network device. There has been proposed an electronic conference system that can automatically manage attendance / exit / absence of a prospective electronic conference attendee based on the result (for example, see Patent Document 1).
[0004]
[Patent Document 1]
JP-A-8-79391
[0005]
[Problems to be solved by the invention]
By the way, the electronic conference system for automatically managing attendance / leaving / absence of a prospective electronic conference attendee does not disclose a case where any of the attendees exits on the way. This electronic conferencing system has a problem that when one of the attendees stops halfway and returns, it is necessary to confirm the contents of the proceedings to other attendees, which hinders the proceedings of the proceedings. . Also. A conventional video conference system in which a conference is performed while watching each other on a monitor has a similar problem when any of the attendees gets out on the way.
[0006]
Further, the electronic conference system is a conference support system in which all attendees including the conference chairman hold a conference using the terminal device, and each attendee including the conference chairman uses a mobile phone as the terminal device. Therefore, even if the user can listen to the audio while the meeting is in progress, or partially grasp the contents of the proceedings as necessary, he / she cannot see the face of each participant. However, since the amount of information on the contents of the proceedings displayed on the display unit is limited, there is a problem that the atmosphere of the meeting is not easily transmitted.
[0007]
Therefore, an object of the present invention is to provide a conference support device, a conference support system, and a conference support program that can know the contents of the proceedings when the attendee returns, even if the attendee of the conference is in the middle. Is to do.
[0008]
Another object of the present invention is to provide a conference support device that allows a user of another device connected via a communication line to participate in a conference from a remote place and to discuss while fully understanding the atmosphere of the conference. And a conference support system and a conference support program.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, a conference support device according to the present invention detects an image capturing unit capturing images of all attendees of a conference, and detects a change in input image data including video from the image capturing unit by pattern matching of the image data. Then, based on the detected change in the input image data, the system recognizes the participant who has been in the middle of the meeting or the participant who has returned to the original location, and measures the time and time interval when the participant was in the middle. A participant recognition unit, and an agenda content notification unit for notifying the returning attendees of the content of the agenda being held in the middle, based on the time and the time interval of the inset, which are measured by the intruder recognition unit. It is characterized by:
[0010]
According to this configuration, the attendant who has been in the middle of the conference or the attendant who has returned to the original place is recognized by the intruder recognition unit from the change of the input image data including the video from the imaging unit, It is possible to measure the time and the time interval of the inset, and when the intruder returns, the time interval between them and the other inset can be easily specified. Then, based on the measured time and time interval of the in-between, the returning person can know the contents of the in-between items by the proceedings content notification unit.
[0011]
In addition, the conference support device according to one embodiment includes a communication unit that mutually communicates with the other device to hold a conference with a user of another device connected via a communication line.
[0012]
According to the conference support device of the above-described embodiment, the communication unit communicates with another device (for example, another conference support device or a terminal device) connected via a communication line, thereby providing a remote location. Can participate in conferences.
[0013]
The conference support device according to one embodiment is an optical device that projects an optical image of at least a part of a wide field of view of a 360 ° surrounding field, and which is captured as wide field image data by the imaging unit. System, a voice input unit that converts the voice of the speaker into a voice signal and inputs the voice, and a speaker recognition unit that recognizes the speaker in the voice input by the voice input unit and specifies the voice code. At least one of panoramic image data including images of all the attendees of the conference from the wide-field image data captured by the imaging unit or fluoroscopic image data including the image of the speaker recognized by the speaker recognition unit. An image data generating unit for generating, the communication unit, together with the voice signal of the speaker in the speech input by the voice input unit, the image data generated by the image data generating unit Panorama image data or via at least one communication line of the fluoroscopic image data and transmits to the other device.
[0014]
According to the conference support device of the above-described embodiment, for example, a camera that captures at least a part of a 360 ° field of view in a predetermined area of a conference room is provided to allow a conference to be performed from another external device. For the attending user, not only the voice of the speaker but also the image data of the attendees and the speaker of the conference can be transmitted from the conference support device to the terminal device. Therefore, even if a user of another device participates in a conference from a remote place, it is possible to discuss while fully understanding the atmosphere of the conference.
[0015]
In one embodiment of the conference support device, the proceeding content notifying unit stores an audio signal input by the audio input unit in association with a speaker code specified by the speaker recognition unit. And an audio output unit that converts the audio signal stored in the audio signal storage unit into audio and outputs the audio signal, and in response to a request from the attendee who has returned to the original location, the audio signal storage unit The stored voice signal is searched for a voice signal in the middle, converted into voice by the voice output unit, and output.
[0016]
According to the meeting support device of the above embodiment, in response to the request of the attendee who has returned to the original place, the above-mentioned meeting contents notifying section converts the voice signal stored in the voice signal storage section into the voice signal of the middle seat. With the simple configuration of searching for and converting to voice by the voice output unit, the contents of the meeting in the middle can be notified.
[0017]
In one embodiment, the conference support device further includes a position coordinate specifying unit that specifies position coordinates of at least one of the panoramic image data and the perspective image data of the speaker recognized by the speaker recognition unit.
[0018]
According to the conference support device of the above embodiment, the image data generation unit includes the speaker image by specifying the position coordinates of the speaker recognized by the speaker recognition unit by the position coordinate specifying unit. Perspective image data can be generated.
[0019]
In one embodiment, the speaker recognition unit recognizes the speaker based on a voice signal input to the voice input unit and an ID code previously added to the voice input unit. And recognizing the speaker on the other device side based on the voice signal transmitted from the other device and at least one of a telephone number, a registration number, and a URL previously added to the other device. I do.
[0020]
Here, the URL (Uniform Resource Locator) indicates an address of a Web page on the Internet.
[0021]
According to the conference support device of the above embodiment, not only a speaker who has input a voice to the voice input unit but also a speaker who has input a voice signal to a voice input unit of another device can be identified.
[0022]
For example, by simply associating the ID code with the speaker code indicating the name and affiliation, the minutes of the utterance with the name and affiliation of the utterer added can be created, so understanding who spoke what and what Can be.
[0023]
In one embodiment, the optical system includes a convex rotating mirror.
[0024]
According to the conference assisting apparatus of the above embodiment, an optical image having a 360 ° field of view and extending in all directions in the horizontal and vertical directions can be projected on the convex rotating mirror.
[0025]
In one embodiment, the conference support apparatus includes conversion information for converting at least one of the panoramic image data or the perspective image data from the wide-field image data captured by the imaging unit by the image data generation unit. Is provided.
[0026]
According to the conference support device of the above embodiment, based on the conversion information stored in the conversion information storage unit, panoramic image data including images of all attendees of the conference from omnidirectional image data captured by the imaging unit Alternatively, at least one of the fluoroscopic image data including the image of the speaker can be easily generated by the image data generating unit.
[0027]
Further, the conference support system of the present invention is characterized in that a plurality of conference support devices provided with the above-mentioned communication unit are connected via a communication line.
[0028]
According to the conference support system having the above-described configuration, a plurality of conference support devices communicate with each other via a communication line to measure a time and a time interval in which a plurality of people participate from a remote place. And when you return, you can easily identify the time intervals and other intellectuals in between. Thus, the proceedings content notification unit of each meeting support device enables the returning person to know the proceedings during the suspension based on the measured time and time interval of the suspension.
[0029]
Further, a conference support system according to one embodiment includes the conference support device and a terminal device connected to the conference support device via a communication line.
[0030]
According to the conference support system of the above-described embodiment, the conference support device and the terminal device communicate with each other via the communication line, so that the time and the time at which the user participates in the conference in which a plurality of persons participate from a remote place The interval can be measured and when returning, the time interval between them and other intellectuals can be easily identified. Thus, the meeting content notification unit of the meeting support device can notify the returning person of the content of the held meeting based on the measured time and time interval of the held position. Note that the proceedings content notification section of the conference support device informs the contents of the proceedings being conducted to the user via a communication line when the user of the terminal device stands back and returns. Is also good.
[0031]
According to another aspect of the present invention, a function of capturing images of all attendees of a conference and a change in input image data including the captured images of all attendees are detected by pattern matching of image data. A function of recognizing a participant who has been in the middle of a meeting or a participant who has returned to the original place from the detected change in the input image data, and measuring the time and time interval when the participant was in the middle There is provided a meeting support program for causing a computer to execute a function of notifying returning attendees of the contents of an attended meeting based on the time and the time interval at which the attendant has been attended.
[0032]
By causing the computer to execute the conference support program, the attendee who has been in the middle of the conference or the attendee who has returned to the original place can be identified from the change in the input image data including the video from the imaging unit. Recognition by the recognition unit, it is possible to measure the time and time interval of the inset, and when the intruder returns, can easily identify the time interval and other insets during that time, and Based on the sitting time and the time interval, it is possible for the returning middleman to know the contents of the proceedings while the sitting.
[0033]
Further, in the conference support program according to one embodiment, a function of capturing an optical image obtained by an optical system of at least a part of a wide field of view around a 360 ° field of view as wide field image data by the image pickup unit; A function of converting a speaker's voice into a voice signal by a voice input unit and inputting the same, a function of specifying a speaker code of the input speaker, and a function of a conference from the captured wide-field image data. A function of generating at least one of panoramic image data including the image of the attendee or fluoroscopic image data including the image of the recognized speaker, together with the voice signal of the speaker in the input speech, Transmitting at least one of the panoramic image data or the perspective image data to another device via a communication line. Conference support program is provided.
[0034]
By causing the computer to execute the conference support program of the above embodiment, not only the voice of the speaker but also the image data of the attendees and the speaker of the conference are transmitted to the user who has attended the conference from another external device. It can be transmitted from the conference support device to another device. Therefore, even if the user of the terminal device participates in the conference from a remote place, it is possible to discuss while fully understanding the atmosphere of the conference.
[0035]
Further, the terminal device of the present invention is a terminal device that performs mutual communication with the conference support device via a communication line, and a communication unit that receives information on the contents of a meeting from the conference support device; A reception information storage unit for storing information on the contents of the proceedings received by the communication unit; and a reception information storage unit for storing the contents of the proceedings stored in the reception information storage unit when the user returns with a pause. It is characterized in that it is provided with a proceedings content notifying section for notifying the returning user of the contents of the proceedings being held based on the information.
[0036]
According to the terminal device having the above-described configuration, even if a user participates in a conference from a remote place, it is possible to discuss while fully understanding the atmosphere of the conference, and the user can stop when the user returns to the center. The contents of the proceedings can be known by the proceedings contents notification unit based on the information on the contents of the proceedings stored in the received information storage unit.
[0037]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a conference support apparatus, a conference support system, and a conference support program of the present invention will be described in detail with reference to the illustrated embodiments. Note that the conference support device, the conference support system, and the conference support program of the present invention are not limited to the embodiments.
[0038]
FIG. 1 is a block diagram showing a basic configuration of a conference support device of a conference support system according to an embodiment of the present invention. As shown in FIG. 1, the conference support system of the present invention includes a conference support device 1 and a terminal device 2 capable of performing data communication with each other.
[0039]
The conference support device 1 includes a conference support device control unit 101, a program memory 102, a buffer memory 103, an optical system 104, an imaging unit 105, a conversion information storage unit 106, an image data generation unit 107, The input unit 108, the speaker recognition unit 109, the position coordinate specifying unit 110, the voice signal storage unit 111, the voice output unit 112, the truss person recognition unit 113, the voice signal conversion unit 114, and the display unit 115 , A communication unit 116, a reception information storage unit 117, an operation unit 118, a storage medium 119, a storage medium reading unit 120, and a bus 121.
[0040]
The conference support device 1 can connect the external terminal device 2 to the communication unit 116 via the communication line 3 according to the connection instruction of the operation unit 118.
[0041]
Hereinafter, the basic configuration of the conference support device 1 will be described in detail.
[0042]
The conference support device control unit 101 includes, for example, a CPU (central processing unit) and an MPU (microprocessing device) of a computer, and executes a control program (conference support program) stored in the program memory 102 or the storage medium 119. Accordingly, the entire conference support device 1 is controlled via the bus 121.
[0043]
The program memory 102 is configured by a storage medium such as a ROM (Read Only Memory), an EEPROM (Electrically Erasable and Writable Read Only Memory), and a flash memory. A control program for controlling the device 1 is stored.
[0044]
Further, the buffer memory 103 is configured by a storage medium such as a RAM (random access memory) and a flash memory, and the input / output data of each unit when the conference support device control unit 101 controls the entire conference support device 1. Is temporarily stored, and the generated image data is stored.
[0045]
The optical system 104 includes, for example, a parabolic or hyperboloidal reflecting mirror as an example of a convex rotating body mirror, a cylindrical translucent body that holds a CCD (charge coupled device) camera, and the like. And an optical image having a 360 ° field of view (an optical image obtained by condensing reflected light from a parabolic or hyperboloidal reflecting mirror with a lens) is projected.
[0046]
The imaging unit 105 includes, for example, a CCD camera including an imaging lens, a CCD, an A / D (analog / digital) conversion circuit, and an image processing circuit, and converts an optical image projected on the optical system 104 into omnidirectional image data. As an image. Note that the optical image projected on the optical system 104 may not be captured as omnidirectional image data, but may be captured as wide-field image data of at least a part of a 360-degree surrounding field of view.
[0047]
Note that the optical system 104 and the imaging unit 105 may be integrally configured as an imaging camera. This imaging camera is fixed at a predetermined height with a tripod, for example, and one camera is installed in a conference room.
[0048]
The conversion information storage unit 106 is configured by a storage medium such as a ROM, an EEPROM, or a flash memory, and has a relational expression and a constant between an optical image projected by the optical system 104 and omnidirectional image data captured by the imaging unit 105. Is stored.
[0049]
Further, the image data generation unit 107 is configured by, for example, an image data generation program, and is configured to perform panoramic image data including images of all attendees of the conference from captured omnidirectional image data and fluoroscopy including images of recognized speakers. Image data is generated based on the conversion information stored in the conversion information storage unit 106.
[0050]
The voice input unit 108 includes, for example, a plurality of microphones to which an ID code (identification number) is added in advance, an A / D conversion circuit, and the like, and converts voice of each speaker who has attended the conference into a voice signal. Enter For example, a speaker code indicating the name and affiliation of a participant of a conference sitting in front of a microphone is associated with the ID code (identification number, registration number).
[0051]
The speaker recognition unit 109 includes, for example, a speaker recognition program, a speaker code storage table, and the like, and is added in advance to the audio signal input to the audio input unit 108 of the conference support device 1 and the audio input unit 108. Based on the ID code, the speaker on the conference support device side is recognized. At this time, the speaker recognizing unit 109 determines whether or not the signal is a voice signal input to the microphone of another voice input unit 108 based on the strength of the voice signal input to one microphone of the voice input unit 108. . Further, the speaker recognition unit 109 recognizes the speaker (user) on the terminal device side based on the transmitted telephone number, registration number, URL, and the like.
[0052]
The speaker recognition program is a program for recognizing a speaker based on an ID code stored in a speaker code storage table, a telephone number, a registration number, and a URL added to the terminal device 2. The speaker code storage table stores an ID code added to the microphone of the voice input unit 108, and a telephone number, a registration number, and a URL added to the terminal device 2.
[0053]
The position coordinate specifying unit 110 is configured by, for example, a position coordinate specifying program, and specifies the position coordinates of the input image of the speaker recognized by the speaker recognition unit 109. That is, as described above, the voice input unit 108 of the conference support device 1 includes a plurality of microphones to which ID codes are assigned in advance, and the microphones include, for example, an on / off switch and a display lamp that lights up when turned on (see FIG. (Not shown). When the speaker turns on the microphone, the indicator light turns on. The position coordinate specifying unit 110 can specify the position of the microphone by identifying the indicator light. Further, the position of the speaker is specified from the position of the microphone. Accordingly, the image data generation unit 107 can generate fluoroscopic image data including the image of the speaker based on the specified position coordinates.
[0054]
Further, the position coordinate specifying unit 110 may add an identification mark to the microphone, specify the position of the microphone to which the voice is input by using the identification mark, and further specify the position of the speaker from the position of the microphone. good. Further, when the microphone is fixed to a conference table or the like, its position coordinates may be stored in advance. In this case, the position of the microphone to which the voice is input can be specified by associating the position with the position coordinates stored in advance. Still further, the position of the speaker may be specified by directly detecting, for example, the eyes of a person on the input image.
[0055]
The audio signal storage unit 111 is configured by a storage medium such as a RAM, a flash memory, a hard disk, and a magneto-optical disk, and stores the speaker's audio signal corresponding to the speaker's code.
[0056]
The audio output unit 112 includes, for example, a plurality of earphones, headphones, a D / A (digital / analog) conversion circuit, and the like, and converts an audio signal stored in the audio signal storage unit 111 into audio and outputs the audio. . Further, the audio output unit 112 may be configured as a pair with the audio input unit 108.
[0057]
Further, the messenger recognition unit 113 is constituted by, for example, a messengers recognition program, a timer, etc., and detects a change in image data including video of all attendees of the conference by pattern matching of the image data, and detects the detected image. The system is configured to recognize a participant who has been in the middle of the meeting or an attendant who has returned to the original place from a change in the data, and measures the time and time interval when the participant has been in the middle of the meeting. In addition, when the attendant who has been in the middle for a predetermined period of time returns to the original place, the middle-seat recognition unit 113 searches the sound signal storage unit 111 for the sound signal of the middle-seat, converts the sound signal into a sound, and outputs the sound. The output from the unit 112 may be adopted. The audio signal storage unit 111 and the audio output unit 112 constitute a meeting content notification unit. Note that the proceedings content notification unit may be configured by displaying the contents of the proceedings in the middle on the display unit 115.
[0058]
The audio signal conversion unit 114 includes, for example, an audio signal conversion program, an audio conversion dictionary, and the like, and converts the audio signal stored in the audio signal storage unit 111 into character data.
[0059]
In addition, the display unit 115 includes, for example, a large-sized LCD (liquid crystal display), PD (plasma display), or ELD (electro luminescence display) connected wirelessly or by wire, and displays the minutes of the meeting on a screen. To be displayed.
[0060]
Further, the communication unit 116 is constituted by, for example, a modem, a signal modulation / demodulation circuit, etc., is connected to a predetermined terminal device 2 via the communication line 3, and together with the voice signal of the speaker, all the attendees of the conference. A function of transmitting panoramic image data including video, fluoroscopic image data including the video of the recognized speaker, and minutes to the terminal device 2; a voice signal, image data, and e-mail ( Document data) and the like.
[0061]
In addition, the reception information storage unit 117 is configured by a storage medium such as a RAM and a flash memory, and stores the speaker's voice signal, image data, and document data received from the terminal device 2.
[0062]
The operation unit 118 includes, for example, a keyboard, a touch panel, and the like connected to the bus 121 wirelessly or by wire. Under the control of the conference support device control unit 101, a connection instruction between the terminal device 2 and the communication unit 116, , Start / end instructions, audio signals, image data, minutes transmission instructions, etc. are input.
[0063]
Further, the storage medium 119 is constituted by a storage medium such as a hard disk and a magneto-optical disk, and is used as an auxiliary storage medium for storing a control program.
[0064]
The storage medium reading unit 120 includes a driving unit that drives the FD, HD, and CD-R / W, and an R / W unit that writes / reads data.
[0065]
Further, a function of projecting an optical image having a viewing area of 360 ° at the maximum circumference using the optical system 104, capturing the optical image as omnidirectional image data using the imaging unit 105, and remarking using the voice input unit 108 A function of converting the speaker's voice into a voice signal and inputting the same, a function of recognizing a speaker who is speaking using the speaker recognition unit 109 and specifying the speaker code, and a function of using the voice signal storage unit 111. Using the function of storing the input audio signal in association with the speaker code and the image data generation unit 107, the panoramic image data including the video of all the attendees of the conference or the image data is recognized from the captured omnidirectional image data. The panoramic image data or the fluoroscopic image data together with the speaker's voice signal through a communication line using the function of generating fluoroscopic image data including the speaker's video. The conference support program for executing the function of transmitting to the predetermined terminal device to the conference support apparatus 1 of the computer may be stored in the storage medium 119.
[0066]
Therefore, by reading the conference support program stored in the storage medium 119 by the storage medium reading unit 120 and sending it to the conference support device control unit 101, the conference support device of the present invention can be realized.
[0067]
FIG. 2 is a block diagram showing a basic configuration of a terminal device connected to the conference support device according to one embodiment of the present invention. As shown in FIG. 2, the terminal device 2 includes a terminal device control unit 201, a program memory 202, a buffer memory 203, a communication unit 204, a reception information storage unit 205, a display unit 206, and a voice input unit. 207, an audio output unit 208, a camera unit 209, an input unit 210, and a bus 211. The terminal device 2 can connect to the conference support device 1 via the communication line 3 and participate in a conference from outside.
[0068]
Hereinafter, the basic configuration of the terminal device 2 will be described in detail.
[0069]
The terminal device control unit 201 is configured by, for example, a CPU or MPU of a computer, and controls the entire terminal device via the bus 211 by executing a control program stored in the program memory 202.
[0070]
The program memory 202 is configured by a storage medium such as a ROM and an EEPROM, for example, and stores a control program for the terminal device control unit 201 to control the terminal device 2.
[0071]
The buffer memory 203 is configured by a storage medium such as a RAM and an EEPROM, and temporarily stores input / output data of each unit when the terminal device control unit 201 controls the entire terminal device.
[0072]
The communication unit 204 includes, for example, a modem, a signal modulation / demodulation circuit, and the like. The communication unit 204 is connected to the conference support device 1 via the communication line 3, and transmits the entire conference together with the audio signal transmitted from the conference support device 1. A function to receive panoramic image data including an attendee's image and perspective image data including a recognized speaker's image, minutes, etc., and to support audio signals, image data, e-mail, and the like input by the terminal device 2 for meeting support It has a function of transmitting to the device 1.
[0073]
Further, the reception information storage unit 205 is configured by a storage medium such as a RAM and an EEPROM, and as an example of information on the contents of a meeting, a voice signal of a speaker, image data, and minutes of the meeting received from the meeting support device 1. Is stored.
[0074]
The display unit 206 includes, for example, a small-sized LCD (liquid crystal display), a PD (plasma display), or an ELD (electro luminescence display), and displays received image data and minutes.
[0075]
The voice input unit 207 includes, for example, a small microphone, an A / D conversion circuit, and the like, and converts the voice of the terminal device user into a voice signal and inputs the voice signal.
[0076]
The audio output unit 208 includes, for example, a small speaker, a D / A conversion circuit, and the like, and converts a received audio signal into audio and outputs the audio.
[0077]
The camera unit 209 includes, for example, a CCD camera including an optical system, a CCD, an A / D conversion circuit, and an image processing circuit, and captures an image around the terminal device. The camera unit 209 may be configured by, for example, a parabolic or hyperboloidal reflecting mirror or an optical system including an imaging lens and an imaging unit including a CCD. Thereby, it is possible to capture images in all directions in the horizontal and vertical directions having a 360 ° viewing area.
[0078]
The input unit 210 includes, for example, a numeric keypad, a key switch, and a touch panel. Under the control of the terminal device control unit 201, input of characters, connection instructions with the conference support device 1, audio signals, image data, minutes Input a transmission request instruction.
[0079]
FIG. 3 is a perspective view showing the configuration of the optical system of the conference support device 1 of this embodiment. The optical system 104 of the conference support device 1 shown in FIG. 3 is constituted by a hyperboloid mirror 104a.
[0080]
In FIG. 3, a hyperboloid mirror 104 a is a convex surface of a curved surface (a region of Z> 0) of a curved surface (a bilobal hyperboloid) obtained by rotating a hyperbola about the Z axis. It was formed. This two-lobe hyperboloid is
(Equation 1)

Is represented by Where c ² = A ² + B ² Where a and b are constants defining the shape of the hyperboloid, and c is a constant defining the position of the focal point.
[0081]
The hyperboloid mirror 104a has two focal points (1) and (2), and light traveling from the outside to one focal point (1) is reflected by the hyperboloid mirror 104a, and all light goes to the other focal point. Having. Therefore, by aligning the rotation axis of the hyperboloid mirror 104a with the optical axis of the imaging lens and arranging the first principal point of the imaging lens at the other focal position {circle around (2)}, the image captured by the imaging unit 105 is obtained. However, an image in which the viewpoint position does not change depending on the viewing direction with one focus (1) as the viewpoint center is obtained.
[0082]
The imaging unit 105 is a video camera or the like, and converts an optical image obtained via the hyperboloid mirror 104a in FIG. 3 into image data using a solid-state imaging device such as a CCD or a CMOS. Note that the lens of the imaging unit 105 may be a general spherical lens or an aspherical lens, and the first principal point only needs to be at the focal position (2).
[0083]
The imaging unit 105 is configured by an imaging camera using a solid-state imaging device such as a CCD or a CMOS, and captures an optical image projected on the hyperboloid mirror 104a by the imaging camera as omnidirectional image data.
[0084]
FIG. 4 is a diagram showing images including videos of all attendees and speakers of the conference according to this embodiment.
[0085]
The image shown in FIG. 4A is a video of all the attendees A, B, C, and D of the conference captured by the imaging cameras (optical system 104 and imaging unit 105) installed in the center of the conference room. , And is temporarily stored in the buffer memory 103 of the conference support apparatus 1 sequentially in frame units as omnidirectional image data.
[0086]
4B are all attendees A, B, and C of the conference generated by the image data generation unit 107 from the omnidirectional image data temporarily stored in the buffer memory 103 based on the conversion information. , D and a perspective image including the image of the speaker A recognized by the speaker recognition unit 109. The image shown in FIG. 4B is transmitted from the outside to the user of the terminal device 2 participating in the conference.
[0087]
5, 6, 7, and 8 are diagrams for explaining the operation of the messenger recognition unit 113. FIG. 5 is an image when all attendees of the conference are present, for example, and FIG. This is an image when a person is absent or seated. FIG. 7 is a diagram showing an example of a microphone ID code corresponding to an attendee and a seat. FIG. 8 is a diagram showing an example of a meeting time table.
[0088]
The middle person recognition program of the middle person recognition unit 113 compares and detects, by pattern matching, changes in the image data captured by the imaging unit 105 at predetermined time intervals by a timer or the like. For example, as shown in FIGS. When the image data changes, the indenter is identified from the ID code in FIG. 7 and the seat position and the attendee code are recorded in a time table as shown in FIG. When you return, you can easily identify the time interval and the messengers during that time.
[0089]
9 to 11 are flowcharts showing the procedure of the basic operation of the conference support system according to this embodiment. 9 to 11, it is assumed that the conference support device 1 is communicably connected to a predetermined terminal device 2 via a communication line 3 for image data, voice, minutes, e-mail, and the like. Assume that the conference support device 1 installed in the conference room is activated by the control of the conference support device control unit 101, and the conference is started.
[0090]
First, when the basic operation is started, in a step S1, it is determined whether or not there is input information in the voice input unit 108 or the communication unit 118 in the conference support device 1. If so, proceed to step S2; otherwise, enter an input waiting state.
[0091]
Next, proceeding to step S2, if there is input information in the voice input unit 108 or the communication unit 118, the input of voice from the voice input unit 108 (microphone) of the conference support device 1 installed in the conference room or the terminal device It is determined whether the input is from the second. If the voice is input from the voice input unit 108, the process proceeds to step S3. If the voice is input from the terminal device 2, the process proceeds to step S4.
[0092]
Then, if the voice is input from the voice input unit 108 in step S3, the voice of the speaker is converted into a voice signal.
[0093]
In step S4, it is determined whether the input information from the terminal device 2 is a voice signal or character data of an e-mail. If it is a voice signal, the process proceeds to step S5 shown in FIG. 10, while if it is e-mail character data, the process proceeds to step S13 shown in FIG.
[0094]
Next, in step S5 shown in FIG. 10, the speaker is recognized from the ID code added to the microphone used or the ID code of the terminal device 2 by using the speaker recognition unit 109, and the corresponding speaker code is specified. I do.
[0095]
Next, proceeding to step S6, the input voice signal is stored in the voice signal storage unit 111 in correspondence with the speaker code.
[0096]
Next, proceeding to step S7, the optical system 104 projects an optical image having a 360 ° field of view around the conference room.
[0097]
Next, proceeding to step S8, the imaging unit 105 captures the optical image projected on the optical system 104 as omnidirectional image data.
[0098]
Next, the process proceeds to step S9, in which the captured omnidirectional image data is sequentially and temporarily stored in the buffer memory 103 of the conference support device 1 in frame units.
[0099]
Next, the process proceeds to step S10, in which the image data generation unit 107 is used to recognize from the omnidirectional image data temporarily stored in the buffer memory 103 by the panoramic image data including the video of all the attendees of the conference and the speaker recognition unit 109. Then, fluoroscopic image data including the image of the speaker is generated. At this time, the position coordinates specifying unit 110 specifies the position coordinates of the input image of the speaker. The generated image data may be displayed on the display unit 117.
[0100]
Next, the process proceeds to step S11, in which the image data generation unit 107 generates generated image data (panoramic image data and perspective image data) together with the speaker's voice signal using the communication unit 118 via the communication line 3. It transmits to a predetermined terminal device 2. If the speaker is a user of the terminal device 2, the voice signal is not transmitted.
[0101]
Next, the process proceeds to step S12, in which the audio signal stored in the audio signal storage unit 111 is converted into character data using the audio signal conversion unit 114.
[0102]
Next, the process proceeds to step S13, where the converted character data is stored as minutes in a storage medium 119 as an example of a minutes recording unit.
[0103]
Next, the process proceeds to step S14, where the created minutes are displayed on the display unit 117.
[0104]
Next, the process proceeds to step S15 shown in FIG. 11, and it is determined whether or not a request for transmission of minutes is issued from the terminal device 2. If so, go to step S16, otherwise go to step S17.
[0105]
Then, if there is a request to transmit the minutes from the terminal device 2 in step S16, the current minutes displayed on the screen are sequentially transmitted to the terminal device 2.
[0106]
Next, in step S17, it is determined whether or not there is an attendant who has intervened in the conference by using the intruder recognition unit 113. If so, the process proceeds to step S18; otherwise, the process returns to step S1 in FIG.
[0107]
Here, the cadaver recognition unit 113 detects, by pattern matching, a difference caused by the movement of the cadaver of the omnidirectional image data stored in the buffer memory 103 one after another in frame units, and detects the detected image. It is assumed that the system is configured so as to recognize whether or not the person is a mediocre from the data difference.
[0108]
Next, in step S18, if there is an attendee who has been in the middle of the conference, the time and the time interval when the attendee has been in the middle are measured.
[0109]
Next, the process proceeds to step S19, and it is determined whether or not the in-between attendant has returned to the original place. If so, the process proceeds to step S20; otherwise, the process returns to step S18.
[0110]
Then, it is determined whether or not the attendee who has returned to the original place in step S20 has requested a voice output of the contents of the statement in the middle. If so, the process proceeds to step S21. If not, the process returns to step S1 in FIG.
[0111]
Next, in step S21, the voice signal of the content of the utterance in the middle is retrieved from the voice signal storage unit 111 and output to the voice output unit 14.
[0112]
Next, the process proceeds to step S22, and it is determined whether or not an instruction to end the conference has been operated. If so, the process proceeds to step S23. If not, the process returns to step S1 in FIG. 9 to be in an input waiting state.
[0113]
Next, if the end instruction of the conference is operated in step S23, the terminal apparatus 2 is notified of the end of the conference, and the conference is ended.
[0114]
As described above, the conference support system including the conference support device 1 and the terminal device 2 has been described. For example, the same conference support device 1 is installed in a plurality of external conference rooms so as to be able to communicate with each other, and The attendees, speakers, and the like in each conference room are automatically displayed on the screen of each display unit 117, and the conference can proceed while looking at each other's faces.
[0115]
Further, in the above-described embodiment, the conference support apparatus using the optical system 104 that projects an optical image having a maximum 360-degree viewing field has been described, but the optical system is not limited to this, and the 360-degree viewing field may be used. The present invention may be applied to a conference support apparatus provided with an optical system that projects an optical image having at least a part of the above.
[0116]
Note that, in the terminal device 2, when the user returns to the center position and returns, the sound signal in the center position stored in the reception information storage unit 205 is converted into sound by the sound output unit 208 and output. The reception information storage unit 205 and the audio output unit 208 may constitute a proceedings content notification unit that notifies the returning user of the contents of the proceeding midterm.
[0117]
By the way, the function of the conference support device of the above embodiment is realized by a program recorded on a program recording medium. Therefore, such a program can be recorded on a computer-readable recording medium and provided as a program product.
[0118]
The computer-readable recording medium is a program medium such as a ROM (Read Only Memory). Alternatively, it may be a program medium that is mounted on and read from an external auxiliary storage device. In any case, the program reading means for reading the information management program from the program medium may have a configuration in which the program medium is directly accessed and read, or a program provided in a RAM (Random Access Memory). A configuration may be adopted in which the program is loaded into a storage area, and the program storage area is accessed and read. It is assumed that a load program for loading from the program medium to the program storage area of the RAM is stored in the apparatus main body in advance.
[0119]
Here, the above-mentioned program medium is configured to be separable from the main body side, such as a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a flexible disk and a hard disk, a CD-ROM (Compact Disc-Read Only Memory), and an MO. (Magnet Optical Disk: Magneto-Optical) Disk, MD (Mini Disk), DVD (Digital Versatile Disc: Digital Versatile Disk) and other optical disk disks, IC (integrated circuit) cards, optical cards and other card systems, mask ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable ROM) e Read Only Memory: electrically erasable ROM), a medium for carrying the fixedly programmed, including semiconductor memory, such as a flash ROM.
[0120]
In the case where the conference support system in the above embodiment includes a modem, a LAN card, and the like, and has a configuration that can be connected to a communication network including the Internet, the program medium flows by downloading from the communication network or the like. It may be a medium that carries the program as a whole. In this case, it is assumed that the download program for downloading from the communication network is stored in the main device in advance. Alternatively, it shall be installed from another recording medium.
[0121]
Note that what is recorded on the recording medium is not limited to only a program, and data can also be recorded.
[0122]
【The invention's effect】
As is apparent from the above, according to the conference support apparatus, the conference support system, and the conference support program of the present invention, a change in input image data including video of all attendees of a conference is detected by pattern matching of the image data. A participant recognition part for recognizing the participant who has been in the middle of the meeting or the participant who has returned to the original place from the change in the input image data, and measures the time and time interval when the participant was in the middle. And a meeting content notification unit for notifying the returning attendee of the contents of the meeting being held based on the measured time and time interval of the above-mentioned in-between, so that any of the attendees When you return and return, you do not have to check with other attendees about the proceedings during that time, so that you can proceed efficiently and smoothly without hindering the proceedings.
[0123]
Further, according to the present invention, in an electronic conference system including a conference support device and a terminal device capable of performing data communication with each other via a communication line, for example, one camera for imaging a 360 ° field of view is provided in the conference room. By installing the terminal at a predetermined location, not only the voice of the speaker but also the image data of the attendees and the speaker of the conference can be transmitted from the conference support device to the terminal device for the user of the terminal device attending the conference from the outside. Can be sent. Therefore, even if the user of the terminal device participates in the conference from a remote place, it is possible to provide a conference support system which allows the user to discuss while fully understanding the atmosphere of the conference.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a basic configuration of a conference support device used in a conference support system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a basic configuration of a terminal device used in the conference support system.
FIG. 3 is a perspective view showing a configuration of an optical system of the conference support system.
FIG. 4 is a diagram showing an image including images of all attendees and speakers of the conference by the conference support system.
FIG. 5 is a diagram showing an image when all attendees of a conference are present in the conference support system.
FIG. 6 is a diagram showing an image when one person is absent or seated in the conference support system.
FIG. 7 is a diagram showing an example of a microphone ID code corresponding to an attendee and a seat in the conference support system.
FIG. 8 is a diagram showing an example of a conference time table in the conference support system.
FIG. 9 is a flowchart showing a procedure of a basic operation of the conference support system.
FIG. 10 is a flowchart following FIG. 9;
FIG. 11 is a flowchart following FIG. 10;
[Explanation of symbols]
1. Meeting support device
101: conference support device control unit,
102: Program memory,
103 ... buffer memory,
104 ... optical system,
105 ... Imaging unit,
106 ... Conversion information storage unit,
107 ... Image data generation unit
108 ... voice input unit,
109 ... Speaker recognition unit,
110 ... Position coordinate specifying unit,
111: audio signal storage unit,
112 ... audio output unit,
113: Intermediate person recognition unit,
114 ... audio signal conversion unit,
115 ... display unit,
116 ... communication unit,
117: reception information storage unit,
118 ... operation unit,
119 ... storage medium,
120: storage medium reading unit,
121 ... bus,
2. Terminal device,
201: terminal device control unit,
202: Program memory,
203: buffer memory,
204: communication unit,
205: reception information storage unit,
206 ... display unit,
207 ... voice input unit,
208: audio output unit,
209 ... Camera part,
210 ... input unit,
211 ... Bus.

Claims (13)

An imaging unit for capturing images of all attendees of the conference,
A change in the input image data including the video from the image pickup unit is detected by pattern matching of the image data, and the attendee who has returned to the original place after sitting in the middle of the conference is recognized from the detected change in the input image data. And a center attendant recognition unit that measures the time and time interval when the attendee sits down,
A meeting content notifying unit for notifying the returning attendees of the contents of the meeting during the sitting based on the time and time interval of the sitting at the center, which is measured by the seated person recognizing unit. Meeting support device. The conference support device according to claim 1,
A conference support device, comprising: a communication unit that performs mutual communication with the other device to hold a conference with a user of another device connected via a communication line. The conference support device according to claim 2,
An optical system that projects an optical image of at least a part of a wide-field region of a 360-degree surrounding field region and is captured as wide-field image data by the imaging unit;
A voice input unit for converting the voice of the speaker into a voice signal and inputting the voice signal;
A speaker recognition unit that recognizes a speaker in the speech input by the voice input unit and specifies the speaker code;
Generating at least one of panoramic image data including images of all attendees of a conference or fluoroscopic image data including images of speakers recognized by the speaker recognition unit from the wide-field image data captured by the imaging unit And an image data generation unit that performs
The communication unit communicates at least one of the panoramic image data or the fluoroscopic image data generated by the image data generation unit together with the voice signal of the speaking speaker input by the voice input unit. A conference support device for transmitting the information to the other device via the Internet. The conference support device according to claim 3,
The meeting content notifying unit is a voice signal storage unit that stores a voice signal input by the voice input unit in association with the speaker code specified by the speaker recognition unit, and is stored in the voice signal storage unit. A voice output unit that converts the voice signal into a voice and outputs the voice signal, and in response to a request from the attendee who has returned to the original location, the voice signal stored in the voice signal storage unit and the voice in the middle position A conference support apparatus for retrieving a signal, converting the signal into voice by the voice output unit, and outputting the voice. The conference support device according to claim 3,
A meeting support device comprising: a position coordinate specifying unit that specifies position coordinates of at least one of the panoramic image data and the perspective image data of the speaker recognized by the speaker recognition unit. The conference support device according to claim 3,
The speaker recognition unit recognizes the speaker based on a voice signal input to the voice input unit and an ID code previously added to the voice input unit, and transmits the voice from the other device. A conference support device for recognizing a speaker on the other device based on at least one of a voice signal, a telephone number, a registration number, and a URL previously added to the other device. The conference support device according to claim 3,
The conference supporting apparatus, wherein the optical system has a convex rotating body mirror. The conference support device according to claim 3,
A conversion information storage unit storing conversion information for converting at least one of the panoramic image data or the perspective image data from the wide-field image data captured by the imaging unit by the image data generation unit; A meeting support device characterized by the above-mentioned. A conference support system, wherein a plurality of the conference support devices according to claim 3 are connected via a communication line. A meeting support device according to claim 1,
A conference support system, comprising: a terminal device connected to the conference support device via a communication line. A function of capturing images of all attendees of the conference by an imaging unit,
A change in the input image data including the images of all the attended participants is detected by pattern matching of the image data, and the attendance returns to the original place after being interrupted in the middle of the meeting from the detected change in the input image data. The ability to recognize the attendee and measure the time and interval at which the attendee sits;
A conference support program for causing a computer to execute a function of notifying a return attendee of the contents of an in-between meeting based on the time and time interval of the in-between. The conference support program according to claim 11,
A function of capturing, as wide-field image data, the optical image obtained by the optical system of at least a part of the wide-field field of the 360-degree field of view by the imaging unit;
A function of converting the voice of the speaker into a voice signal by a voice input unit and inputting the signal;
A function for identifying the speaker code of the speaker in the input speech,
A function of generating at least one of panoramic image data including images of all attendees of the conference or fluoroscopic image data including images of recognized speakers from the captured wide-view image data,
Causing the computer to execute a function of transmitting at least one of the panoramic image data or the perspective image data to another device via a communication line together with the input speech signal of the speaker who is speaking. A featured meeting support program. A terminal device for mutually communicating with the conference support device according to claim 2 via a communication line,
A communication unit that receives information about the contents of the meeting from the meeting support device;
A reception information storage unit that stores information on the contents of the proceedings received by the communication unit,
When the user returns with a pause, the contents of the proceedings are notified to the returning user based on the information on the details of the proceedings stored in the received information storage unit. A terminal device comprising: a notifying unit.

Images