JP4502896B2

JP4502896B2 - Projection display

Info

Publication number: JP4502896B2
Application number: JP2005207768A
Authority: JP
Inventors: 光浩嶺
Original assignee: NEC Display Solutions Ltd
Current assignee: Sharp NEC Display Solutions Ltd
Priority date: 2005-07-15
Filing date: 2005-07-15
Publication date: 2010-07-14
Anticipated expiration: 2025-07-15
Also published as: JP2007025295A

Description

本発明は投写型表示装置に関し、特に電子ペンと組み合わせて使用され発信器位置検出部と画面操作信号識別部とを有する投写型表示装置に関する。 The present invention relates to a projection display device, and more particularly to a projection display device that is used in combination with an electronic pen and has a transmitter position detection unit and a screen operation signal identification unit.

近年投写型表示装置がプレゼンテーション用として広く用いられるようになり、指し棒を発信器を有する電子ペンとして、発表者が電子ペンで指した投写映像中のポインティング位置を自動的に検出し、検出した位置を投写型表示装置や投写型表示装置に接続されたパソコンのマウスの変わりとして使用できるシステムが知られている（特許文献１、２参照）。特許文献１には、信号処理器をホワイトボードなどの互いに離れた場所に取り付け、電子ペンの超音波発生部から発信された超音波がそれぞれの超音波受信部で受信されるまでの所要時間に基づいて電子ペンからそれぞれの超音波受信部までの距離を算定することで電子ペンの位置を取得し、パソコンで座標として取り込むポインティング位置検出装置と画像表示装置が開示されている。その仕組みとしては、例えばホワイトボードに投写型表示装置から映像を投影して説明者が電子ペンを用いて説明する場合に、信号処理器をホワイトボードの２箇所以上の隅部に取り付け、パソコンの映像を投写型表示装置でホワイトボードに投射し、投射映像の特定の位置（例えば４隅）を電子ペンで指定することで投射映像の基準位置を前記信号処理器に覚えこませ、その後は電子ペンの位置と先に覚えこませた特定の位置を比較することで座標を算出し、算出された座標をパソコンに渡し映像内のマウスカーソルなどを動かす処理を行う。特許文献２で提案されている座標検出部を搭載したミラー投射型投写型表示装置は、特許文献１のような投写型表示装置やスクリーンとは別体の位置検出部やデジタイザーなどを必要としないためシステム構成が簡単となり移動も容易となる。
特開２００１−１２５７４０号公報特開２００５−１２２５３４号公報 In recent years, projection display devices have been widely used for presentations, and the pointing position in the projected image pointed by the presenter is automatically detected by using the pointing stick as an electronic pen with a transmitter. There is known a system that can be used as a projection display device or a mouse of a personal computer connected to the projection display device (see Patent Documents 1 and 2). In Patent Document 1, the signal processor is attached to a place separated from each other such as a whiteboard, and the time required until the ultrasonic wave transmitted from the ultrasonic wave generation unit of the electronic pen is received by each ultrasonic wave reception unit. A pointing position detection device and an image display device that acquire the position of the electronic pen by calculating the distance from the electronic pen to each ultrasonic wave reception unit based on the calculated distance and that are captured as coordinates by a personal computer are disclosed. As the mechanism, for example, when an image is projected from a projection display device on a whiteboard and an instructor explains using an electronic pen, signal processors are attached to two or more corners of the whiteboard, and The image is projected onto the whiteboard by the projection display device, and the specific position (for example, four corners) of the projected image is designated with the electronic pen so that the reference position of the projected image is memorized in the signal processor. The coordinates are calculated by comparing the position of the pen with the specific position previously learned, the calculated coordinates are transferred to the personal computer, and the mouse cursor in the video is moved. The mirror projection type projection display device equipped with the coordinate detection unit proposed in Patent Document 2 does not require a position detection unit or digitizer separate from the projection display device and the screen as in Patent Document 1. This simplifies the system configuration and facilitates movement.
JP 2001-125740 A JP 2005-122534 A

しかし、このような特許文献２で提案されている電子黒板機能をもったミラー投射型プロジェクタを利用したときにも、電子ペンでスクリーンを指そうとすると手とペンとが投射光を遮ってしまうことがあり、スクリーンにその影ができてしまい、電子ペンで何かを描画しようとしているときに、描いた軌跡とこれから描こうとしている部分とが見えないと非常に描画が難しいという課題があった。 However, even when the mirror projection type projector having the electronic blackboard function proposed in Patent Document 2 is used, if the user tries to point at the screen with the electronic pen, the hand and the pen block the projection light. In some cases, the shadows appear on the screen, and when you try to draw something with the electronic pen, it is very difficult to draw if you cannot see the trace you drew and the part you are about to draw. It was.

本発明の目的は、電子ペンで描画するときに電子ペンや手の影が支障とならない投写型表示装置を提供することにある。 An object of the present invention is to provide a projection display device in which the shadow of an electronic pen or hand does not hinder when drawing with the electronic pen.

本発明の投写型表示装置は、
電磁波発光部および波動発信部を備える発信器と組み合わせて使用され、電磁波受光部および互いに離れて配置された第１の波動受信部と第２の波動受信部とを有する投写型表示装置であって、発信器の発信する波動および電磁波の受信時刻に基づいて、発信器と第１の波動受信部および発信器と第２の波動受信部のそれぞれの距離を算出して発信器の位置座標を算出し、算出した位置座標を記憶して逐次更新する発信器位置算出部と、入力した画面操作信号から操作内容を識別して対応する画像操作を画像制御部に指示する画面操作信号識別部とを有する。 The projection display device of the present invention is
A projection display device that is used in combination with a transmitter including an electromagnetic wave light emitting unit and a wave transmitting unit, and includes an electromagnetic wave light receiving unit and a first wave receiving unit and a second wave receiving unit that are arranged apart from each other. Based on the wave and electromagnetic wave reception time transmitted by the transmitter, the distance between the transmitter and the first wave receiver and the distance between the transmitter and the second wave receiver is calculated to calculate the position coordinates of the transmitter. A transmitter position calculation unit that stores and sequentially updates the calculated position coordinates, and a screen operation signal identification unit that identifies an operation content from the input screen operation signal and instructs the image control unit to perform a corresponding image operation. Have.

第１の態様では、画像制御部に指示された画面操作がエリア複写指示信号であるときは、発信器の位置座標に対応して予め設定されている複写範囲、発信器の位置座標に対応して予め設定されている嵌めこみ位置の条件に従って、予め設定されている複写範囲の映像情報を複写して、映像情報における予め設定されている嵌めこみ位置に嵌め込んで投射面に投写する複写映像生成貼り付け部を有する。 In the first aspect, when the screen operation instructed to the image control unit is an area copy instruction signal, it corresponds to the copy range and the position coordinate of the transmitter set in advance corresponding to the position coordinate of the transmitter. Copying the video information in the preset copy range in accordance with the preset fitting position conditions, and fitting the preset video image position into the preset fitting position in the video information and projecting it onto the projection surface It has a generation pasting part.

第２の態様では、画像制御部に指示された画面操作信号がエリア複写指示信号であるときは、発信器の位置座標と対応して予め設定されている複写範囲、発信器の位置座標と対応して予め設定されている嵌めこみ位置の条件に従って、発信器の位置座標の移動に伴って予め設定されている複写範囲の映像情報を逐次複写して、発信器の位置座標の移動に伴って予め設定されている嵌めこみ位置に嵌め込んで投射面に投写する複写映像生成貼り付け部を有する。 In the second aspect, when the screen operation signal instructed to the image control unit is an area copy instruction signal, it corresponds to the preset copy range and the position coordinate of the transmitter corresponding to the position coordinate of the transmitter. Then, in accordance with the preset fitting position condition, the image information of the preset copy range is sequentially copied with the movement of the transmitter position coordinate, and the movement of the transmitter position coordinate is performed. It has a copy image generation and pasting unit that fits in a preset fitting position and projects it onto the projection surface.

画面操作信号がカーソルを用いた描画信号であるときは、発信器位置をカーソル位置として投射面にカーソルを投射し、その発信器の位置の移動に対応して投射画像に描画が行われる描画部を有してもよい。 When the screen operation signal is a drawing signal using a cursor, a drawing unit that projects the cursor on the projection surface with the transmitter position as the cursor position, and draws the projected image corresponding to the movement of the transmitter position You may have.

発信器は画面操作信号を波動発信部から出力する波動に乗せて発信する操作部を有し、画面操作信号識別部は、波動受信部で受信した波動から画面操作信号を識別して画像制御部に出力してもよく、発信部は電子ペンに設けられていてもよい。また、画面操作信号識別部は、投写型表示装置の操作部から入力した画面操作信号を識別して画像制御部に出力してもよく、投写型表示装置に接続された装置の操作部から入力した画面操作信号を識別して画像制御部に出力してもよい。 The transmitter has an operation unit that transmits the screen operation signal on the wave output from the wave transmission unit, and the screen operation signal identification unit identifies the screen operation signal from the wave received by the wave reception unit and controls the image control unit. The transmitter may be provided in the electronic pen. The screen operation signal identification unit may identify the screen operation signal input from the operation unit of the projection display device and output it to the image control unit, or input from the operation unit of the device connected to the projection display device. The screen operation signal may be identified and output to the image control unit.

時間差解析発信器位置算出部は、電磁波受光部における電磁波受光時刻と第１の波動受信部および第２の波動受信部における波動受信時刻との差を基に、波動発信部から第１の波動受信部および第２の波動受信部までの波動到達時間を算出する波動到達時間算出手段と、算出された波動到達時間から、波動発信部から第１の波動受信部および第２の波動受信部までの距離を算出する距離算出手段と、算出された波動発信部から第１の波動受信部および第２の波動受信部までの距離と、予め記憶されている第１の波動受信部と第２の波動受信部との間隔および位置算出の基準となる波動受信部と投射面との間の垂直距離とから、発信器の波動発生部の投射面上の位置座標を算出して、投写映像内における対応する位置座標を算定する発信器座標算定手段と、算定した位置座標を逐次格納して投写型表示装置の画像制御部に出力する発信器座標格納部とを備えていてもよく、投写型表示装置が、開口構造を有する筐体に画像投射のための投射ミラーを備える投写型表示装置であってもよく、電磁波は赤外線であり、波動は超音波であってもよい。 The time difference analysis transmitter position calculation unit receives the first wave reception from the wave transmission unit based on the difference between the electromagnetic wave reception time in the electromagnetic wave reception unit and the wave reception times in the first wave reception unit and the second wave reception unit. Wave arrival time calculating means for calculating the wave arrival time to the first wave receiving unit and the second wave receiving unit, and from the calculated wave arrival time to the first wave receiving unit and the second wave receiving unit. Distance calculating means for calculating the distance, the calculated distance from the wave transmitting unit to the first wave receiving unit and the second wave receiving unit, the first wave receiving unit and the second wave stored in advance The position coordinates on the projection surface of the wave generator of the transmitter are calculated from the distance between the receiver and the vertical distance between the wave receiver, which is the reference for position calculation, and the projection surface. Transmitter coordinates to calculate the position coordinates And a transmitter coordinate storage unit that sequentially stores the calculated position coordinates and outputs them to the image control unit of the projection display device, and the projection display device is provided in a casing having an opening structure. A projection display device including a projection mirror for image projection may be used, the electromagnetic waves may be infrared rays, and the waves may be ultrasonic waves.

発表者がスクリーン上の投写映像内を電子ペンで指したときに、出力される赤外線パルスと超音波パルスを投写型表示装置の赤外線受光部と２つ以上の超音波受信部で受信して、投写映像内のポインティング位置に対応する液晶パネル内の座標を算出し、算出された座標の周辺の映像データを入力映像データから抜き出し、あらかじめ設定されたエリアに複写することができ、電子ペンを移動させると複写された画像データは電子ペンの移動位置に対応して移動すると共に新しい位置の映像データに更新されるので、手やペンの影を気にすることなく描画を行うことができる。 When the presenter points at the projected image on the screen with the electronic pen, the output infrared pulse and ultrasonic pulse are received by the infrared receiving unit and two or more ultrasonic receiving units of the projection display device, Calculate the coordinates in the liquid crystal panel corresponding to the pointing position in the projected image, extract the video data around the calculated coordinates from the input video data, copy it to the preset area, and move the electronic pen Then, the copied image data moves in accordance with the movement position of the electronic pen and is updated to video data at a new position, so that drawing can be performed without worrying about the shadow of the hand or the pen.

本発明の第1の効果は、発表者が投写型表示装置やパソコンの黒板機能でスクリーンの投射映像内に文字や図形を描画しているときに、自分の手やペンの影で見えない部分が陰にならないエリアに複写表示されるため、発表者は既に書いた画像とこれからペンをドラッグさせて書こうとする部分の画像を確認しながら作業をすることができるようになることである。 The first effect of the present invention is that when the presenter draws characters or figures in the projected image on the screen with the blackboard function of the projection display device or the personal computer, the portion that cannot be seen by the shadow of his hand or pen Since the image is copied and displayed in an area that is not shaded, the presenter can work while confirming the image already written and the image of the part to be written by dragging the pen.

第２の効果は、発表者が描画しているとき、発表を見ている人々には描いている発表者の陰になった部分は直接見えないが、複写された画像で発表者が描画している画像をリアルタイムに見ることができるようになることである。 The second effect is that when the presenter is drawing, people viewing the presentation cannot directly see the shaded part of the presenter who is drawing, but the presenter draws the copied image. It is to be able to see the images that are in real time.

本発明の投写型表示装置は、電子ペンを投写画像の中で使用するとき、手やペンの影で見えなくなった部分を他のエリアに表示する手段を備えることを特徴としている。 The projection display device according to the present invention is characterized by comprising means for displaying, in another area, a portion that has disappeared due to the shadow of a hand or a pen when the electronic pen is used in a projected image.

本発明の実施の形態について図面を参照して説明する。図１は本発明の実施の形態の投写型表示装置の模式的斜視図であり、図２は本発明の実施の形態の投写型表示装置の模式的ブロック構成図である。 Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic perspective view of a projection display apparatus according to an embodiment of the present invention, and FIG. 2 is a schematic block diagram of the projection display apparatus according to the embodiment of the present invention.

本発明の実施の形態の投写型表示装置１０と電子ペン７０と投射面８０との構成と動作とを図１、図２を参照して説明する。電子ペン７０は電子黒板用としてよく使用され、トリガパルスとして赤外線を発光する赤外線発光部７２および位置検出用として超音波を発信する超音波発生部７１を備える発信器が先端近傍に設けられており、電子ペン７０の手元には超音波発生部７１と赤外線発光部７２のオンオフや、複写信号や描画信号を含む所定の画面操作信号を超音波発生部７１の超音波に乗せて発信できる操作部７６を有し、画面操作信号発信の指示があった場合は超音波発生部７１は画面操作信号を所定の超音波信号として発信する。 The configuration and operation of the projection display device 10, the electronic pen 70, and the projection surface 80 according to the embodiment of the present invention will be described with reference to FIGS. The electronic pen 70 is often used for an electronic blackboard, and a transmitter including an infrared light emitting unit 72 that emits infrared light as a trigger pulse and an ultrasonic wave generating unit 71 that transmits ultrasonic waves for position detection is provided near the tip. On the hand of the electronic pen 70, an operation unit capable of transmitting on / off of the ultrasonic wave generation unit 71 and the infrared light emission unit 72 and transmitting a predetermined screen operation signal including a copy signal and a drawing signal on the ultrasonic wave of the ultrasonic wave generation unit 71 When the screen operation signal is instructed to be transmitted, the ultrasonic generator 71 transmits the screen operation signal as a predetermined ultrasonic signal.

ここでは１５ｋＨｚより高い振動数の音波を超音波といい、０．７４μｍより長い波長の光を赤外線という。本実施の形態では電子ペンとして説明するが、発信器付指し棒であっても問題はなく、赤外線発光部と超音波発生部とを有する発信器が備えられていれば各種の形態が考えられる。この発信器は投射面を指示する部分の近傍に設けられていることが望ましい。 Here, sound waves having a frequency higher than 15 kHz are called ultrasonic waves, and light having a wavelength longer than 0.74 μm is called infrared rays. Although this embodiment will be described as an electronic pen, there is no problem even if it is a pointing stick with a transmitter, and various forms are possible if a transmitter having an infrared light emitting unit and an ultrasonic wave generating unit is provided. . This transmitter is preferably provided in the vicinity of a portion indicating the projection surface.

発信器の超音波発生部７１と赤外線発光部７２は同期して一周期に一度だけ超音波信号、および赤外線パルスを発生する。ここでは、超音波発生部７１と赤外線発光部７２のオンオフは電子ペン７０の手元の操作部７６の操作で行われることとしているが、先端部を投射面に押し付けることによって行なわれてもよい。 The ultrasonic generator 71 and the infrared light emitter 72 of the transmitter generate an ultrasonic signal and an infrared pulse only once in a cycle in synchronization. Here, the ultrasonic generator 71 and the infrared light emitter 72 are turned on and off by the operation of the operation unit 76 at the hand of the electronic pen 70, but may be performed by pressing the tip portion against the projection surface.

投写型表示装置１０は、投影部２０と、発信器位置算出部３０と、画面操作信号識別部３７と、画像制御部４０と、各部の制御を行う中央制御装置６０とを有する。投影部２０は、光源２１と液晶表示部やＤＭＤ（ＤｉｇｉｔａｌＭｉｌｌｅｒＤｅｖｉｃｅ）などの光束変調部２２と光学系２３とを有し、投写部２４から投射面８０に画像を投写する。ここでは超短焦点のミラー型プロジェクタを例として説明するが、レンズシフト機構を有する短焦点のレンズ投射型プロジェクタに対しても有効に適用できる。 The projection display apparatus 10 includes a projection unit 20, a transmitter position calculation unit 30, a screen operation signal identification unit 37, an image control unit 40, and a central control device 60 that controls each unit. The projection unit 20 includes a light source 21, a liquid crystal display unit, a light beam modulation unit 22 such as a DMD (Digital Miller Device), and an optical system 23, and projects an image from the projection unit 24 onto the projection surface 80. Although an ultra-short focus mirror type projector will be described here as an example, it can be effectively applied to a short focus lens projection type projector having a lens shift mechanism.

発信器位置算出部３０は、第１の超音波受信部３１と、第２の超音波受信部３２と、赤外線受光部３３と、時間差解析発信器位置算出部３４と、発信器座標算出部３５と、発信器座標格納部３６とを有する。 The transmitter position calculator 30 includes a first ultrasonic receiver 31, a second ultrasonic receiver 32, an infrared receiver 33, a time difference analysis transmitter position calculator 34, and a transmitter coordinate calculator 35. And a transmitter coordinate storage unit 36.

第１の超音波受信部３１および第２の超音波受信部３２は、投射光軸８５を含む鉛直面と直交する方向にそれぞれが水平方向に離れて配設されていて、電子ペン７０の超音波発生部７１で発生した超音波信号７３、７４をそれぞれが受信する。このような配置が発信器位置の計算上望ましいが、違った配置であっても補正することができる。また、赤外線受光部３３は、第１の超音波受信部３１と第２の超音波受信部３２の近傍、通常は中間に配置されて電子ペン７０の赤外線発光部７２で発生した赤外線パルス７５を受光する。赤外線受光部３３は、離れた場所に設けられていても受光時刻に大きな誤差を生ずることはない。 The first ultrasonic receiving unit 31 and the second ultrasonic receiving unit 32 are disposed in the horizontal direction in a direction orthogonal to the vertical plane including the projection optical axis 85, respectively, and Each of the ultrasonic signals 73 and 74 generated by the sound wave generation unit 71 is received. Such an arrangement is desirable for the calculation of the transmitter position, but a different arrangement can be corrected. The infrared light receiving unit 33 receives an infrared pulse 75 generated in the infrared light emitting unit 72 of the electronic pen 70 in the vicinity of the first ultrasonic receiving unit 31 and the second ultrasonic receiving unit 32, usually in the middle. Receive light. The infrared light receiving unit 33 does not cause a large error in the light receiving time even if it is provided at a remote location.

電子ペン７０の超音波発生部７１と赤外線発光部７２とは同期して一周期に一度だけ超音波信号、および赤外線パルスを発生し、赤外線の発光時刻と受光時刻とは殆ど同一と見なされるので、赤外線受光部３３の受光時刻と第１の超音波受信部３１の受信時刻の受信時刻の差、および赤外線受光部３３の受光時刻と第２の超音波受信部３２の受信時刻の受信時刻の差から、電子ペン７０の超音波発生部７１から第１の超音波受信部３１および超音波発生部７１から第２の超音波受信部３２までのそれぞれの超音波到達時間が算出されて、時間差解析発信器位置算出部３４で音速を用いて換算することによって、超音波発生部７１から第１の超音波受信部３１および超音波発生部７１から第２の超音波受信部３２のそれぞれの距離が算出できる。 Since the ultrasonic wave generation unit 71 and the infrared light emission unit 72 of the electronic pen 70 generate an ultrasonic signal and an infrared pulse only once in a single cycle, the infrared light emission time and the light reception time are considered to be almost the same. The difference between the reception time of the infrared light reception unit 33 and the reception time of the first ultrasonic reception unit 31, and the reception time of the reception time of the infrared reception unit 33 and the reception time of the second ultrasonic reception unit 32. From the difference, the ultrasonic arrival times of the electronic pen 70 from the ultrasonic generator 71 to the first ultrasonic receiver 31 and from the ultrasonic generator 71 to the second ultrasonic receiver 32 are calculated, and the time difference is calculated. The distances from the ultrasonic generator 71 to the first ultrasonic receiver 31 and from the ultrasonic generator 71 to the second ultrasonic receiver 32 are converted by using the speed of sound in the analysis transmitter position calculator 34. Can be calculated

発信器座標算出部３５では算出された距離情報に予め入力されている両超音波受信部の間隔、超音波受信部と投射面との距離を参照して投射面上の発信器の座標が算定され、それに基づいて投写映像上の発信器の座標が算出され、発信器座標格納部３６に逐次更新して格納される。座標の算出についての詳細は後述する。 The transmitter coordinate calculation unit 35 calculates the coordinates of the transmitter on the projection surface with reference to the distance between the two ultrasonic reception units and the distance between the ultrasonic reception unit and the projection surface, which are input in advance in the calculated distance information. Based on this, the coordinates of the transmitter on the projected image are calculated, and are sequentially updated and stored in the transmitter coordinate storage unit 36. Details of the coordinate calculation will be described later.

画面操作信号識別部３７は、電子ペン７０の操作部７６から超音波発生部７１を経由して超音波に載せられ発信された画面操作信号を超音波受信部３２から受信し、画面操作信号の内容を識別して画像制御部４０の画像操作部５０に出力する。 The screen operation signal identification unit 37 receives the screen operation signal transmitted from the operation unit 76 of the electronic pen 70 via the ultrasonic wave generation unit 71 and transmitted from the ultrasonic wave reception unit 32, and receives the screen operation signal. The contents are identified and output to the image operation unit 50 of the image control unit 40.

画像制御部４０は、入力した映像信号をデジタル信号にＡＤ変換する入力信号処理部４１、デジタル信号に変換された映像信号を液晶パネルの解像度に変換する解像度変換部４２、解像度変換された映像信号を一時的に保存しておく入力映像記憶部４３、画面操作信号識別部３７から入力した画面操作信号と発信器座標格納部３６から入力した発信器座標とに基づいて入力映像記憶部４３に記憶されている入力映像を参照して所定の画像操作を行う画像操作部５０、画像操作部５０で制作された映像データを入力映像記憶部４３に記憶されている入力映像に合成して出力する画像データ合成部４４、合成された映像のＤＡ変換を行うＤＡ変換部４５、およびＤＡ変換された映像に基づいて投影部２０の光束変調部２２を駆動する表示駆動部４６を有する。 The image control unit 40 includes an input signal processing unit 41 that AD converts the input video signal into a digital signal, a resolution conversion unit 42 that converts the video signal converted into the digital signal into the resolution of the liquid crystal panel, and a resolution-converted video signal Is stored in the input video storage unit 43 based on the screen operation signal input from the screen operation signal identification unit 37 and the transmitter coordinates input from the transmitter coordinate storage unit 36. An image operation unit 50 that performs a predetermined image operation with reference to the input video that has been input, and an image that is output by combining the video data produced by the image operation unit 50 with the input video stored in the input video storage unit 43 A data synthesis unit 44, a DA conversion unit 45 that performs DA conversion of the synthesized video, and a display drive unit 4 that drives the light beam modulation unit 22 of the projection unit 20 based on the DA-converted video. Having.

画像操作部５０には、複写映像生成貼り付け部５１や、描画部５２や、発生したカーソルに対し制御部でクリック動作を行うことで所望の操作を行わせたりする操作手段などが複数設けられている。複写映像生成貼り付け部５１では、発信器の位置座標と対応する入力映像内の所定のエリア内の映像情報が複写されて入力映像の所定の位置に嵌め込まれ、複写した映像の嵌め込み位置が、発信器位置の移動に対応して移動するとともに、複写した映像が新しい発信器位置の入力映像に対応して更新される。描画部５２では、発信器座標位置にカーソルを生成させ、発信器の移動に対応してカーソルを移動させて描画を行わせる。なお、複写映像生成貼り付け部５１では、嵌めこみ位置の移動や更新は行なわれなくてもよい。 The image operation unit 50 is provided with a plurality of operation means for performing a desired operation by performing a click operation on the generated cursor by the control unit with respect to the copy video generation and pasting unit 51, the drawing unit 52, and the generated cursor. ing. In the copied video generation / pasting unit 51, video information in a predetermined area in the input video corresponding to the position coordinates of the transmitter is copied and inserted in a predetermined position of the input video, and the insertion position of the copied video is The image is moved in response to the movement of the transmitter position, and the copied image is updated in response to the input image at the new transmitter position. In the drawing unit 52, a cursor is generated at the transmitter coordinate position, and drawing is performed by moving the cursor in accordance with the movement of the transmitter. Note that the copy video generation / pasting unit 51 does not need to move or update the fitting position.

次に、本発明の実施の形態の動作について説明する。まず、発表者が投射面８０に投写されている画像内の一点を電子ペン７０で突いたり、操作部７６のボタンを操作したりすると、電子ペン７０の赤外線発光部７２から赤外線パルス７５が出力され、超音波発生部７１から超音波信号７３、７４が出力される。 Next, the operation of the embodiment of the present invention will be described. First, when a presenter strikes a point in the image projected on the projection surface 80 with the electronic pen 70 or operates a button on the operation unit 76, an infrared pulse 75 is output from the infrared light emitting unit 72 of the electronic pen 70. Then, ultrasonic signals 73 and 74 are output from the ultrasonic generator 71.

投写型表示装置１０の発信器位置算出部３０の赤外線受光部３３で赤外線パルス７５を受光してから、第１および第２の超音波受信部３１、３２で超音波信号７３、７４を受信するまでの時間が時間差解析発信器位置算出部３４のカウンタ部でカウントされる。カウントされた時間から電子ペン７０で指し示した投射面８０上の点に対応する光束変調部２２内の映像での位置が発信器座標算出部３５で算出される。 After the infrared pulse 75 is received by the infrared light receiving unit 33 of the transmitter position calculating unit 30 of the projection display device 10, the first and second ultrasonic receiving units 31 and 32 receive the ultrasonic signals 73 and 74 . Is counted by the counter unit of the time difference analysis transmitter position calculation unit 34. From the counted time, the transmitter coordinate calculation unit 35 calculates the position in the image in the light beam modulation unit 22 corresponding to the point on the projection surface 80 pointed to by the electronic pen 70.

距離情報の取得原理および電子ペン７０で指し示した投射面８０上の点に対応する入力映像での座標の算出法を説明する。図３は電子ペンの発信器と投写型表示装置との間の赤外線と超音波の発信と受信との関係を示す模式図であり、図４は本発明の実施の形態の投写型表示装置の時間差解析発信器位置算出部と発信器座標算出部と発信器座標格納部の模式的ブロック構成図である。 The principle of distance information acquisition and a method of calculating coordinates in an input video corresponding to a point on the projection surface 80 pointed to by the electronic pen 70 will be described. FIG. 3 is a schematic diagram showing a relationship between transmission and reception of infrared rays and ultrasonic waves between the transmitter of the electronic pen and the projection display device, and FIG. 4 shows the projection display device according to the embodiment of the present invention. It is a typical block block diagram of a time difference analysis transmitter position calculation part, a transmitter coordinate calculation part, and a transmitter coordinate storage part.

指し示す位置に発信器を押し当て、あるいはスイッチ７６をＯＮとすると、図３に示すように、電子ペン７０の赤外線発光部７２から赤外線トリガパルス７０１が発光され、同時に超音波発生部７１から超音波信号７０２が発信される。投写型表示装置１０の赤外線受光部３３が赤外線トリガパルス７０１を赤外線トリガパルス３３１として受光し、第１の超音波受信部３１と第２の超音波受信部３２とが超音波信号７０２をそれぞれ超音波信号３１１、３２１として受信する。 When the transmitter is pressed at the indicated position or the switch 76 is turned ON, as shown in FIG. 3, an infrared trigger pulse 701 is emitted from the infrared light emitting unit 72 of the electronic pen 70, and at the same time, an ultrasonic wave is emitted from the ultrasonic wave generating unit 71. A signal 702 is emitted. The infrared receiving unit 33 of the projection display apparatus 10 receives the infrared trigger pulse 701 as the infrared trigger pulse 331, and the first ultrasonic receiving unit 31 and the second ultrasonic receiving unit 32 transmit the ultrasonic signal 702 respectively. The sound wave signals 311 and 321 are received.

赤外線受光部３３において受光される赤外線トリガパルス３３１の受光時刻ｔ₀は電子ペン７０の赤外線発光部７２から発光された赤外線トリガパルス７０１の発光時刻との間に時間遅延は殆どない。しかし、第１の超音波受信部３１にて受信される超音波信号３１１の受信時刻ｔ₁、第２の超音波受信部３２にて受信される超音波信号３２１の受信時刻ｔ₂は、超音波発生部７１から超音波信号７０２が発信された時刻に対して距離を音速（約３４０ｍ／ｓ）で割った時間分の遅れが生じている。この時刻ｔ₁、ｔ₂とｔ₀との差であるＴ₁、Ｔ₂を求めることで、それぞれの所用時間の情報が得られる。実際の受信時刻の測定では第１波の測定では誤差を生ずるおそれがあるので第２波以降の受信時刻を測定してｔ₁、ｔ₂を計算で求めてもよい。 There is almost no time delay between the light receiving time t ₀ of the infrared trigger pulse 331 received by the infrared light receiving unit 33 and the light emitting time of the infrared trigger pulse 701 emitted from the infrared light emitting unit 72 of the electronic pen 70. However, the reception time t ₁ of the ultrasonic signal 311 is received by the first ultrasonic wave receiver 31, the reception time t ₂ of the ultrasonic signal 321 is received by the second ultrasonic wave receiving unit 32, ultrasonic There is a delay corresponding to the time obtained by dividing the distance by the speed of sound (about 340 m / s) with respect to the time when the ultrasonic wave signal 702 is transmitted from the sound wave generator 71. By obtaining T ₁ and T ₂ , which are the differences between the times t ₁ and t ₂ and t ₀ , information on the required times can be obtained. In the actual measurement of the reception time, there is a possibility that an error may occur in the measurement of the first wave. Therefore, the reception times after the second wave may be measured to obtain t ₁ and t ₂ by calculation.

次に、発信器位置算出部３０の時間差解析発信器位置算出部３４と発信器座標算出部３５と発信器座標格納部３６の構成と動作を説明する。上述のように、超音波発生部７１と赤外線発光部７２は一周期に一度だけ超音波信号、および赤外線パルスを発生するので、赤外線パルスの受光時刻が超音波信号の発射時刻とほぼ同一となり、超音波信号の受信時刻と赤外線パルスの受光時刻との差が超音波の到達時間となり、音速を用いて超音波発生部７１と超音波受信部３１、３２との距離が算出できる。 Next, the configuration and operation of the time difference analysis transmitter position calculation unit 34, the transmitter coordinate calculation unit 35, and the transmitter coordinate storage unit 36 of the transmitter position calculation unit 30 will be described. As described above, since the ultrasonic wave generation unit 71 and the infrared light emission unit 72 generate an ultrasonic signal and an infrared pulse only once in one period, the light reception time of the infrared pulse is almost the same as the emission time of the ultrasonic signal, The difference between the reception time of the ultrasonic signal and the reception time of the infrared pulse is the arrival time of the ultrasonic wave, and the distance between the ultrasonic wave generation unit 71 and the ultrasonic wave reception units 31 and 32 can be calculated using the speed of sound.

図４に示すように、時間差解析発信器位置算出部３４は、赤外線受光時刻（ｔ0 ）取得部３４１、第１の超音波受信時刻（ｔ1 ）取得部３４２、第２の超音波受信時刻（ｔ2 ）取得部３４３、第１の超音波到達時間（ｔ1 −ｔ0 ＝Ｔ1 ）算出部３４４、第２の超音波到達時間（ｔ2 −ｔ0 ＝Ｔ2 ）算出部３４５、Ｔ1 距離換算部（＝Ｌ1 ）３４６、Ｔ2 距離換算部（＝Ｌ2 ）３４７、および発信器の投射面上のＸ、Ｙ座標算出部３４８を有する。発信器座標算出部３５は発信器の投射面上のＸ、Ｙ座標算出部３４８で算出された投射面上のＸ、Ｙ座標を光学関係の式により入力映像の座標に換算する投射面Ｘ、Ｙ座標の映像内座標換算部３５１を有する。発信器座標格納部３６は第１の超音波受信部−第２の超音波受信部距離Ｌ0 格納部３６１、超音波受信部−投射面間距離Ｌ4 格納部３６２および映像内座標格納部３６３を有する。第１の超音波受信部−第２の超音波受信部距離Ｌ0 格納部３６１には、予め第１の超音波受信部３１と第２の超音波受信部３２との間の距離Ｌ0 が格納され、超音波受信部−投射面間距離格納部３６２には、予め超音波受信部３１、３２−投射面８０の距離が格納され、映像内座標格納部３６３には投射面Ｘ、Ｙ座標の映像内座標換算部３５１で算出された発信器の映像内の座標が格納されている。 As shown in FIG. 4, the time difference analysis transmitter position calculation unit 34 includes an infrared light reception time (t0) acquisition unit 341 , a first ultrasonic reception time (t1) acquisition unit 342, and a second ultrasonic reception time (t2). ) Acquisition unit 343, first ultrasonic arrival time (t1-t0 = T1) calculation unit 344, second ultrasonic arrival time (t2-t0 = T2) calculation unit 345, T1 distance conversion unit (= L1) 346 , T2 distance conversion unit (= L2) 347 and X, Y coordinate calculation unit 348 on the projection surface of the transmitter. The transmitter coordinate calculation unit 35 converts the X and Y coordinates on the projection surface calculated by the X and Y coordinate calculation unit 348 on the projection surface of the transmitter into the coordinates of the input image using optical expressions, An in-video coordinate conversion unit 351 for Y coordinates is provided. The transmitter coordinate storage unit 36 includes a first ultrasonic reception unit-second ultrasonic reception unit distance L0 storage unit 361, an ultrasonic reception unit-projection plane distance L4 storage unit 362, and an in-video coordinate storage unit 363. . The distance L0 between the first ultrasonic receiver 31 and the second ultrasonic receiver 32 is stored in advance in the first ultrasonic receiver-second ultrasonic receiver distance L0 storage 361. The distance between the ultrasound receiver 31 and the projection plane distance storage unit 362 stores the distance between the ultrasound receivers 31 and 32 and the projection plane 80 in advance, and the image coordinate storage unit 363 stores images of the projection planes X and Y coordinates. The coordinates in the image of the transmitter calculated by the inner coordinate conversion unit 351 are stored.

赤外線受光時刻（ｔ₀）取得部３４１は、電子ペン７０の赤外線発光部７２から超音波発生部７１と同時に発光された赤外線パルス７５の赤外線受光部３３での受光時刻ｔ₀を取得する。第１の超音波受信時刻（ｔ₁）取得部３４２と第２の超音波受信時刻（ｔ₂）取得部３４３は、それぞれ電子ペン７０の超音波発生部７１から赤外線発光部７２と同時に発信された超音波信号７３、７４の第１の超音波受信部３１、第２の超音波受信部３２における受信時刻ｔ₁、ｔ₂を取得する。第１の超音波到達時間（ｔ₁−ｔ₀＝Ｔ₁）算出部３４４は、第１の超音波受信部３１の受信時刻ｔ₁と赤外線受光部３３の受光時刻ｔ₀との時間差Ｔ₁を算出する。第２の超音波到達時間（ｔ₂−ｔ₀＝Ｔ₂）算出部３４５は、第２の超音波受信部３２の受信時刻ｔ₂と赤外線受光部３３の受光時刻ｔ₀との時間差Ｔ₂を算出する。 The infrared light reception time (t ₀ ) acquisition unit 341 acquires the light reception time t ₀ at the infrared light reception unit 33 of the infrared pulse 75 emitted simultaneously with the ultrasonic wave generation unit 71 from the infrared light emission unit 72 of the electronic pen 70. The first ultrasonic wave reception time (t ₁ ) acquisition unit 342 and the second ultrasonic wave reception time (t ₂ ) acquisition unit 343 are transmitted simultaneously from the ultrasonic wave generation unit 71 of the electronic pen 70 and the infrared light emitting unit 72, respectively. The reception times t ₁ and t ₂ of the first ultrasonic receiving unit 31 and the second ultrasonic receiving unit 32 of the ultrasonic signals 73 and 74 are acquired. The first ultrasonic arrival time (t ₁ −t ₀ = T ₁ ) calculation unit 344 calculates the time difference T ₁ between the reception time t _{1 of the first} ultrasonic reception unit 31 and the light reception time t ₀ of the infrared light reception unit 33. Is calculated. The second ultrasonic arrival time (t ₂ −t ₀ = T ₂ ) calculation unit 345 calculates a time difference T ₂ between the reception time t _{2 of the second} ultrasonic reception unit 32 and the light reception time t ₀ of the infrared light receiving unit 33. Is calculated.

Ｔ₁距離換算部（＝Ｌ₁）３４６は、音速のデータを用いて時間差Ｔ₁を電子ペン７０の超音波発生部７１と第１の超音波受信部３１までの距離Ｌ₁に換算する。Ｔ₂距離換算部（＝Ｌ₂）３４７は、音速のデータを用いて時間差Ｔ₂を電子ペン７０の超音波発生部７１と第２の超音波受信部３２までの距離Ｌ₂に換算する。発信器の投射面上のＸ，Ｙ座標算出部３４８は、取得した距離Ｌ₁、Ｌ₂と発信器座標格納部３６に予め格納されている第１の超音波部受信部−第２の超音波部受信部距離Ｌ₀と超音波受信部−投射面距離Ｌ₄とから発信器の投射面上のＸ，Ｙ座標を後述の方法で算出する。 The T ₁ distance conversion unit (= L ₁ ) 346 converts the time difference T ₁ into the distance L ₁ between the ultrasonic wave generation unit 71 of the electronic pen 70 and the first ultrasonic wave reception unit 31 using sound speed data. The T ₂ distance conversion unit (= L ₂ ) 347 converts the time difference T ₂ into the distance L ₂ between the ultrasonic wave generation unit 71 of the electronic pen 70 and the second ultrasonic wave reception unit 32 using sound speed data. The X, Y coordinate calculation unit 348 on the projection plane of the transmitter transmits the acquired distances L ₁ and L ₂ and the first ultrasonic unit reception unit-second super stored in the transmitter coordinate storage unit 36 in advance. The X and Y coordinates on the projection surface of the transmitter are calculated by the method described later from the sound wave receiving portion distance L ₀ and the ultrasonic wave receiving portion-projection surface distance L ₄ .

発信器座標算出部３５の投射面Ｘ、Ｙ座標の映像内座標換算部３５１で、投射面上の発信器のＸ、Ｙ座標を入力映像のＸ、Ｙ座標に換算し、発信器座標格納部３６の映像内座標格納部３６３に記憶させる。 The in-video coordinate conversion unit 351 of the projection plane X and Y coordinates of the transmitter coordinate calculation unit 35 converts the X and Y coordinates of the transmitter on the projection surface into the X and Y coordinates of the input video, and the transmitter coordinate storage unit. 36 stored in the in-video coordinate storage unit 363.

ここで、発信器の投射面上のＸ、Ｙ座標算出部３４８における発信器の座標取得の原理について説明する。図５は発信器の投射面上のＸ、Ｙ座標算出部における投射面上の座標算出動作を説明するための模式図である。 Here, the principle of the coordinate acquisition of the transmitter in the X and Y coordinate calculation unit 348 on the projection surface of the transmitter will be described. FIG. 5 is a schematic diagram for explaining the coordinate calculation operation on the projection surface in the X and Y coordinate calculation unit on the projection surface of the transmitter.

図５において、第２の超音波受信部３２の位置をＡとし、第１の超音波受信部３１の位置をＢとし、電子ペン７０の発信器の超音波発生部７１の位置をＣとし、Ｃを通って投射面８０と直交する垂直面とＡ−Ｂを通る線分との交点をＤとし、Ｄより投射面８０に対して垂直な線と投射面８０との交点をＥとし、水平方向の座標をＸ、垂直方向の座標をＹとする。 In FIG. 5, the position of the second ultrasonic receiver 32 is A, the position of the first ultrasonic receiver 31 is B, the position of the ultrasonic generator 71 of the transmitter of the electronic pen 70 is C, An intersection of a vertical plane passing through C and perpendicular to the projection plane 80 and a line segment passing through AB is D, and an intersection of a line perpendicular to the projection plane 80 from D and the projection plane 80 is E. Let the coordinate in the direction be X and the coordinate in the vertical direction be Y.

△ＣＡＢにおいて辺ＣＡと辺ＡＢのなす角度をαとすると、
ｃｏｓα＝（Ｌ₂ ²＋Ｌ₀ ²−Ｌ₁ ²）／（２Ｌ₂ ×Ｌ₀）なので、
α＝ｃｏｓ^-1｛（Ｌ₂ ²＋Ｌ₀ ²−Ｌ₁ ²）／（２Ｌ₂ ×Ｌ₀）｝となる。 Δ When the angle between side CA and side AB is α in CAB,
Since cos α = (L ₂ ² + L ₀ ² −L ₁ ² ) / (2L ₂ × L ₀ ),
α = cos ⁻¹ {(L ₂ ² + L ₀ ² −L ₁ ² ) / (2L ₂ × L ₀ )}

一方、△ＣＤＡは直角三角形なので、
ｃｏｓ（１８０°−α）＝Ｘ／Ｌ₂ 従って、Ａを基点とした水平方向の座標Ｘは、
Ｘ＝ｃｏｓ（１８０°−α）×Ｌ₂ となり、先に求めたαを代入するとＸは求められる。 On the other hand, △ CDA is a right triangle, so
cos (180 ° −α) = X / L ₂ Therefore, the horizontal coordinate X based on A is
X = cos (180 ° −α) × L ₂ , and X can be obtained by substituting α obtained previously.

また、△ＣＤＡにおける辺ＣＤの長さＹ’は、
Ｙ’＝ｓｉｎ（１８０°−α）×Ｌ₂ で求められる。
一方、△ＣＥＤも直角三角形なので、
Ｙ²＝Ｙ’²−Ｌ₄ ² なので、垂直方向の座標Ｙは、
Ｙ＝√（Ｙ’²−Ｌ₄ ²）として求められる。
これらは都度計算してもよいし、予めＬ₀、Ｌ₁、Ｌ₂、Ｌ₄を変数とするテーブルを作成して、テーブルから求めてもよい。 Also, the length Y ′ of the side CD in ΔCDA is
Y ′ = sin (180 ° −α) × L ₂
On the other hand, △ CED is also a right triangle,
Since Y ² = Y ′ ² −L ₄ ² , the vertical coordinate Y is
Y = √ (Y ′ ² −L ₄ ² )
These may be calculated each time, or a table having L ₀ , L ₁ , L ₂ , and L ₄ as variables in advance may be obtained and obtained from the table.

画像制御部４０に入力された映像信号は入力信号処理部４１でデジタル信号に変換された後、解像度変換部４２で光束変調部２２の解像度に変換され、例えば図６のような映像データ９１が入力映像記憶部４３に書き込まれる。図６は画像制御部に入力した映像データの１例であり、図７は画像制御部に入力した映像データを投射面に投写し電子ペンを持った腕がその上に影を発生させた状態を示す模式図である。電子ペン７０と電子ペン７０を持つ腕８２が投射面８０上に影８３を発生させている。 The video signal input to the image control unit 40 is converted to a digital signal by the input signal processing unit 41, and then converted to the resolution of the light beam modulation unit 22 by the resolution conversion unit 42. For example, video data 91 as shown in FIG. It is written in the input video storage unit 43. FIG. 6 shows an example of the video data input to the image control unit, and FIG. 7 shows a state in which the video data input to the image control unit is projected onto the projection surface and the arm holding the electronic pen generates a shadow on the projection surface. It is a schematic diagram which shows. An electronic pen 70 and an arm 82 having the electronic pen 70 generate a shadow 83 on the projection surface 80.

電子ペン７０の操作部７６から超音波発生部７１を経由して画面操作信号を発信すると、画面操作信号は超音波に載せられて超音波受信部３２で受信され、画面操作信号識別部３７が画面操作信号の内容を識別して画像制御部４０の画像操作部５０に出力する。 When a screen operation signal is transmitted from the operation unit 76 of the electronic pen 70 via the ultrasonic wave generation unit 71, the screen operation signal is placed on the ultrasonic wave and received by the ultrasonic wave reception unit 32, and the screen operation signal identification unit 37 is The content of the screen operation signal is identified and output to the image operation unit 50 of the image control unit 40.

画像操作部５０には、上述のように発信器の位置座標と対応する入力映像内の所定のエリア内の映像情報を複写して入力映像の所定の位置に嵌め込み、複写した映像の嵌め込み位置を、発信器位置の移動に対応して移動させるとともに、複写した映像が新しい発信器の位置に対応して更新される複写映像生成貼り付け部５１や、投射面８０の発信器座標位置にカーソルを生成させ、発信器の移動に対応してカーソルを移動させて描画を行わせる描画部５２や、発生したカーソルに対し制御部でクリック動作を行うことで所望の操作を行わせたりする操作手段などが複数設けられている。 In the image operation unit 50, as described above, the video information in the predetermined area in the input video corresponding to the position coordinate of the transmitter is copied and inserted into the predetermined position of the input video, and the insertion position of the copied video is set. The cursor is moved to the transmitter coordinate position on the projection plane 80 or the copied video generation / pasting unit 51 where the copied video is updated corresponding to the new transmitter position. A drawing unit 52 that generates and moves the cursor in accordance with the movement of the transmitter to perform drawing, an operation means that performs a desired operation by performing a click operation on the generated cursor by the control unit, etc. Are provided.

画面操作信号が複写映像生成貼り付けであれば、画面操作信号は複写映像生成貼り付け部５１に送られ、複写映像生成貼り付け部５１は複写映像生成貼り付けを実行する。図８は複写映像生成貼り付け部の模式的ブロック構成図であり、図９は複写映像生成を説明するための映像データの模式図であり、（ａ）はエリア設定を示し、（ｂ）はエリアを複写した複写映像データを示し、図１０は映像データに複写映像データを貼り付けた状態を示す模式図である。 If the screen operation signal is copy video generation paste, the screen operation signal is sent to the copy video generation paste unit 51, and the copy video generation paste unit 51 executes copy video generation paste. FIG. 8 is a schematic block diagram of a copy video generation and pasting unit, FIG. 9 is a schematic diagram of video data for explaining copy video generation, (a) shows area setting, (b) shows FIG. 10 is a schematic diagram showing a state in which the copied video data is pasted on the video data.

複写映像生成貼り付け部５１は、操作信号受理部５１１、入力映像取得部５１２、発信器座標取得部５１３、複写範囲確定部５１４、映像複写実行部５１５、複写映像加工部５１６、貼り付け位置決定部５１７、貼り付け実行部５１８、継続実行判定部５１９、複写範囲条件格納部５３１、貼り付け位置条件格納部５３２、複写範囲映像加工内容格納部５３３、継続実行可否格納部５３４、複写映像格納部５３５を有する。 The copy video generation and paste unit 51 includes an operation signal reception unit 511, an input video acquisition unit 512, a transmitter coordinate acquisition unit 513, a copy range determination unit 514, a video copy execution unit 515, a copy video processing unit 516, and a paste position determination. Unit 517, pasting execution unit 518, continuous execution determination unit 519, copy range condition storage unit 531, paste position condition storage unit 532, copy range video processing content storage unit 533, continuous execution availability storage unit 534, copy video storage unit 535.

複写範囲条件格納部５３１には、予め発信器座標を基点とする複写範囲がいずれかの操作部により設定されており必要の都度変更される。貼り付け位置条件格納部５３２には、予め発信器座標を基点として基点の移動条件によって貼り付け位置が操作部により設定されており必要の都度変更される。複写範囲映像加工内容格納部５３３には、複写範囲映像に、例えば枠をつけたり拡大縮小処理を行なったりするといった加工内容が操作部により設定されており必要の都度変更される。継続実行可否格納部５３４には、複写映像生成を継続して行うか否かが操作部により設定されており必要の都度変更される。継続して行うと発信器の位置移動に対応して複写映像生成貼り付けが繰り返され、貼り付け位置が発信器の移動に対応して移動するとともに複写映像が新しい位置に対応して更新される。複写映像格納部５３５に最も新しい複写映像が格納される。 In the copy range condition storage unit 531, a copy range based on transmitter coordinates is set in advance by any of the operation units and is changed whenever necessary. In the pasting position condition storage unit 532, the pasting position is set in advance by the operation unit according to the moving condition of the base point with the transmitter coordinates as the base point, and is changed whenever necessary. In the copy range video processing content storage unit 533, for example, processing content such as adding a frame or enlarging / reducing processing to the copy range video is set by the operation unit and is changed whenever necessary. In the continuous execution availability storage unit 534, whether or not to continue to generate the copy video is set by the operation unit, and is changed whenever necessary. If it continues, copy video generation and pasting will be repeated in response to the movement of the transmitter, the pasting position will move in response to the movement of the transmitter, and the copy video will be updated to correspond to the new position. . The newest copy video is stored in the copy video storage unit 535.

操作信号受理部５１１が画面操作信号識別部３７から複写映像生成貼り付けの信号を受け取ると、入力映像取得部５１２が入力映像記憶部４３から現在の入力映像を取得し、発信器座標取得部５１３が発信器座標格納部３６から発信器の座標を取得する。複写範囲確定部５１４は、複写範囲条件格納部５３１に予め格納されている発信器座標を基点とする複写範囲を読み出し、現在の入力映像における複写範囲を確定する。映像複写実行部５１５は、確定した入力映像の複写範囲の複写を実行して複写映像を複写映像格納部５３５に格納する。複写映像加工部５１６は、複写範囲映像加工内容格納部５３３から映像加工内容を読み出し、加工の必要があれば複写映像格納部５３５に格納された複写映像を読み出して必要な加工を行って再び複写映像格納部５３５に格納する。貼り付け位置決定部５１７は、貼り付け位置条件格納部５３２に格納されている発信器座標を基点として基点の移動条件として示された貼り付け位置に従って貼り付け位置を入力映像の座標で決定する。貼り付け実行部５１８では、複写映像格納部５３５に格納されている複写映像に入力映像の貼り付け位置の座標を付して画像データ合成部４４に出力し、画像データ合成部４４は入力映像記憶部４３から読み出した入力映像に複写映像を貼り付ける。継続実行判定部５１９では、継続実行可否格納部５３４から継続実行可否格情報を読み出し、可であれば入力映像取得部５１２に戻ってステップを繰り返し、否であれば終了する。
When the operation signal receiving unit 511 receives a copy video generation and pasting signal from the screen operation signal identification unit 37, the input video acquisition unit 512 acquires the current input video from the input video storage unit 43 and the transmitter coordinate acquisition unit 513. Acquires the coordinates of the transmitter from the transmitter coordinate storage 36. The copy range determination unit 514 reads a copy range based on the transmitter coordinates stored in advance in the copy range condition storage unit 531 and determines the copy range in the current input video. The video copy execution unit 515 executes copying of the determined copy range of the input video and stores the copy video in the copy video storage unit 535. The copy video processing unit 516 reads the video processing content from the copy range video processing content storage unit 533, and if necessary, reads the copy video stored in the copy video storage unit 535, performs the necessary processing, and copies again. Stored in the video storage unit 535. The pasting position determination unit 517 determines the pasting position based on the coordinates of the input video according to the pasting position indicated as the base point movement condition with the transmitter coordinates stored in the pasting position condition storage unit 532 as the base point. The pasting execution unit 518 attaches the coordinates of the pasting position of the input video to the copy video stored in the copy video storage unit 535 and outputs it to the image data synthesis unit 44. The image data synthesis unit 44 stores the input video. The copy video is pasted on the input video read from the unit 43. The continuous execution determination unit 519 reads the continuous execution availability information from the continuous execution availability storage unit 534, and if yes, returns to the input video acquisition unit 512 to repeat the steps, and if no, ends.

このように、複写映像生成貼り付け部５１は、入力映像記憶部４３から入力映像を呼び出し、発信器座標格納部３６から発信器座標７９を呼び出して、図９（ａ）に示すように入力映像の発信器座標７９の周りの所定の範囲の映像データを、図９（ｂ）に示すように複写して、図１０に示すように画像データ合成部４４で複写した複写映像データ９３を入力した映像データ９１に貼り付けて映像データ９４とし、ＤＡ変換部４５、表示駆動部４６を経由して光束変調部２２に出力し、投射面８０に画像として投写する。 In this way, the copy video generation / pasting unit 51 calls the input video from the input video storage unit 43 and calls the transmitter coordinates 79 from the transmitter coordinate storage unit 36, and the input video as shown in FIG. The video data in a predetermined range around the transmitter coordinate 79 of the above is copied as shown in FIG. 9B, and the copied video data 93 copied by the image data composition unit 44 is input as shown in FIG. The image data 94 is pasted on the image data 91 to be output to the light beam modulation unit 22 via the DA conversion unit 45 and the display drive unit 46, and is projected as an image on the projection surface 80.

発信器の位置に従って複写エリアと貼り付け位置を移動する継続実行可の条件であれば、発信器の移動位置に対応してこの操作が繰り返されることによって、複写エリアの映像が更新され貼り付け位置が発信器の移動に対応して移動する。このように発表者が投射面８０上で電子ペン７０をドラッグさせると、複写される画像とその貼り付け位置がスクロールする。発表者は自分の手やペンの影で隠れた部分を複写される画像で確認しながら描画を続けることができるため、描きにくさが軽減される。 If it is a condition that allows continuous execution to move the copy area and paste position according to the transmitter position, the image of the copy area is updated and the paste position is repeated by repeating this operation according to the move position of the transmitter. Moves in response to the movement of the transmitter. When the presenter drags the electronic pen 70 on the projection surface 80 in this way, the image to be copied and its pasting position are scrolled. Since the presenter can continue drawing while confirming the part hidden by the shadow of his hand or pen in the copied image, the difficulty of drawing is reduced.

発表者が投射面８０から電子ペン７０を離して一定時間経過すると、自動的に複写される画像は消えるようにしてもよい。複写された画像が消えるまでの時間も発表者が設定できる。あるいは、複写された画像が表示されている箇所を電子ペン７０で突くか、投写型表示装置本体かリモコンのキーを押したときに消すという方法も取ることができる。 When the presenter releases the electronic pen 70 from the projection surface 80 and a predetermined time elapses, the automatically copied image may disappear. The presenter can also set the time until the copied image disappears. Alternatively, it is possible to take a method in which a portion where a copied image is displayed is pierced with the electronic pen 70 or is erased when a key of a projection display main body or a remote control is pressed.

ここでは、これらの設定は電子ペン７０の操作部７６から入力されることとして説明したが、投写型表示装置１０本体の不図示の操作部、投写型表示装置１０に接続する不図示のパソコンの操作部、あるいは、画面の発信器位置に表示されたカーソルの操作によって画面操作信号識別部３７に指示してもよい。 Here, it has been described that these settings are input from the operation unit 76 of the electronic pen 70. However, the operation unit (not shown) of the projection display device 10 main body and a personal computer (not shown) connected to the projection display device 10 are used. You may instruct | indicate to the screen operation signal identification part 37 by operation of the operation part or the cursor displayed on the transmitter position of the screen.

上述の実施の形態では、トリガパルスとして赤外線を、位置検出用として超音波を、それぞれ使用した。人間の目は赤外線に対しては感度が低いため、トリガパルスとしては赤外線を用いることが望ましいが、赤外線以外の電磁波を用いることもできる。例えば、波長０．４０μｍ以上の電磁波であれば、人体への影響も少ない。また、人間の耳は超音波に対しては感度が低いため、位置検出用としては超音波を用いることが望ましいが、超音波以外の波動を用いることもできる。例えば、１５ｋＨｚ以下の振動数の音波であっても、ユーザがうるさいと感じない程度の音量であれば、支障なく用いることができる。 In the above-described embodiment, infrared rays are used as trigger pulses, and ultrasonic waves are used for position detection. Since the human eye has low sensitivity to infrared rays, it is desirable to use infrared rays as a trigger pulse, but electromagnetic waves other than infrared rays can also be used. For example, an electromagnetic wave having a wavelength of 0.40 μm or more has little influence on the human body. In addition, since the human ear has low sensitivity to ultrasonic waves, it is desirable to use ultrasonic waves for position detection, but waves other than ultrasonic waves can also be used. For example, even a sound wave having a frequency of 15 kHz or less can be used without any problem as long as the sound volume is not felt by the user.

本発明の実施の形態の投写型表示装置の模式的斜視図である。1 is a schematic perspective view of a projection display apparatus according to an embodiment of the present invention. 本発明の実施の形態の投写型表示装置の模式的ブロック構成図である。1 is a schematic block configuration diagram of a projection display apparatus according to an embodiment of the present invention. 電子ペンの発信器と投写型表示装置との間の赤外線と超音波の発信と受信との関係を示す模式図である。It is a schematic diagram which shows the relationship between the transmission and reception of the infrared rays and ultrasonic waves between the transmitter of an electronic pen and a projection type display apparatus. 本発明の実施の形態の投写型表示装置の時間差解析発信器位置算出部と発信器座標算出部と発信器座標格納部の模式的ブロック構成図である。It is a typical block block diagram of the time difference analysis transmitter position calculation part, transmitter coordinate calculation part, and transmitter coordinate storage part of the projection type display apparatus of embodiment of this invention. 発信器の投射面上のＸ、Ｙ座標算出部における投射面上の座標算出動作を説明するための模式図である。It is a schematic diagram for demonstrating the coordinate calculation operation | movement on the projection surface in the X, Y coordinate calculation part on the projection surface of a transmitter. 画像制御部に入力した映像データの１例である。It is an example of the video data input into the image control part. 画像制御部に入力した映像データを投射面に投写し電子ペンを持った腕がその上に影を発生させた状態を示す模式図である。It is a schematic diagram which shows the state which projected the video data input into the image control part on a projection surface, and the arm with an electronic pen produced the shadow on it. 複写映像生成貼り付け部の模式的ブロック構成図である。It is a typical block block diagram of a copy image | video production | generation sticking part. 複写映像生成を説明するための映像データの模式図であり、（ａ）はエリア設定を示し、（ｂ）はエリアを複写した複写映像データを示す。4A and 4B are schematic diagrams of video data for explaining generation of a copied video, where (a) shows area setting and (b) shows copied video data obtained by copying the area. 映像データに複写映像データを貼り付けた状態を示す模式図である。It is a schematic diagram which shows the state which affixed copy image data to image data.

Explanation of symbols

１０投写型表示装置
２０投影部
２１光源
２２光束変調部
２３光学系
２４投写部
３０発信器位置算出部
３１第１の超音波受信部
３２第２の超音波受信部
３３赤外線受光部
３４時間差解析発信器位置算出部
３５発信器座標算出部
３６発信器座標格納部
３７画面操作信号識別部
４０画像制御部
４１入力信号処理部
４２解像度変換部
４３入力映像記憶部
４４画像データ合成部
４５ＤＡ変換部
４６表示駆動部
５０画像操作部
５１複写映像生成貼り付け部
５２描画部
６０中央制御装置
７０電子ペン
７１超音波発生部
７２赤外線発光部
７３、７４超音波信号
７５赤外線パルス
７６操作部
８０投射面
８２腕
８３影
８５投射光軸
９１、９２、９４映像データ
９３複写映像データ
３１１、３２１、７０２超音波信号
３３１、７０１赤外線パルス
３４１赤外線受光時刻（ｔ₀）取得部
３４２第１の超音波受信時刻（ｔ₁）取得部
３４３第２の超音波受信時刻（ｔ₂）取得部
３４４第１の超音波到達時間（ｔ₁−ｔ₀＝Ｔ₁）算出部
３４５第２の超音波到達時間（ｔ₂−ｔ₀＝Ｔ₂）算出部
３４６Ｔ₁距離換算部（＝Ｌ₁）
３４７Ｔ₂距離換算部（＝Ｌ₂）
３４８発信器の投射面上のＸ、Ｙ座標算出部
３５１投射面Ｘ、Ｙ座標の映像内座標変換部
３６１第１の超音波受信部−第２の超音波受信部Ｌ₀格納部
３６２超音波受信部−投射面間距離Ｌ₄格納部
３６３映像内座標格納部
５１１複写信号受理部
５１２入力映像取得部
５１３発信器座標取得部
５１４複写範囲確定部
５１５映像複写実行部
５１６複写映像加工部
５１７貼り付け位置決定部
５１８貼り付け実行部
５１９継続実行判定部
５３１複写範囲条件格納部
５３２貼り付け位置条件格納部
５３３複写範囲映像加工内容格納部
５３４継続実行可否格納部
５３５複写映像格納部 DESCRIPTION OF SYMBOLS 10 Projection type display apparatus 20 Projection part 21 Light source 22 Light flux modulation part 23 Optical system 24 Projection part 30 Transmitter position calculation part 31 1st ultrasonic reception part 32 2nd ultrasonic reception part 33 Infrared light receiving part 34 Time difference analysis transmission Transmitter position calculation unit 35 Transmitter coordinate calculation unit 36 Transmitter coordinate storage unit 37 Screen operation signal identification unit 40 Image control unit 41 Input signal processing unit 42 Resolution conversion unit 43 Input video storage unit 44 Image data synthesis unit 45 DA conversion unit 46 Display drive unit 50 Image operation unit 51 Copy video generation and pasting unit 52 Drawing unit 60 Central control unit 70 Electronic pen 71 Ultrasonic wave generation unit 72 Infrared light emission unit 73, 74 Ultrasonic signal 75 Infrared pulse 76 Operation unit 80 Projection surface 82 Arm 83 Shadow 85 Projection optical axis 91, 92, 94 Video data 93 Copy video data 311, 321, 702 Super sound It signals 331,701 IR pulses 341 infrared receiving time (t ₀₎ acquiring unit 342 first ultrasonic reception time (t ₁₎ acquiring section 343 second ultrasonic reception time (t ₂₎ acquiring unit 344 first ultrasonic Arrival time (t ₁ -t ₀ = T ₁ ) calculation unit 345 Second ultrasonic arrival time (t ₂ -t ₀ = T ₂ ) calculation unit 346 T ₁ distance conversion unit (= L ₁ )
347 T ₂ distance conversion part (= L ₂ )
348 X, Y coordinate calculation unit 351 on projection plane of transmitter 351 In-image coordinate conversion unit 361 of projection plane X, Y coordinate 361 First ultrasonic reception unit-second ultrasonic reception unit L ₀ storage unit 362 Ultrasonic wave Reception unit-projection plane distance L ₄ storage unit 363 In-video coordinate storage unit 511 Copy signal reception unit 512 Input video acquisition unit 513 Transmitter coordinate acquisition unit 514 Copy range determination unit 515 Video copy execution unit 516 Copy video processing unit 517 Paste Pasting position determination unit 518 Pasting execution unit 519 Continuous execution determination unit 531 Copying range condition storage unit 532 Pasting position condition storage unit 533 Copying range video processing content storage unit 534 Continuous execution availability storage unit 535 Copy video storage unit

Claims

A projection display device that is used in combination with a transmitter including an electromagnetic wave light emitting unit and a wave transmitting unit, and includes an electromagnetic wave light receiving unit and a first wave receiving unit and a second wave receiving unit that are arranged apart from each other. ,
Based on the difference between the electromagnetic wave light receiving time in the electromagnetic wave light receiving unit and the wave receiving time in the first wave receiving unit and the second wave receiving unit, the first wave receiving unit and the first wave receiving unit from the wave transmitting unit A wave arrival time calculating means for calculating a wave arrival time to the second wave receiving unit, and from the calculated wave arrival time and sound velocity, the wave transmitting unit to the first wave receiving unit and the second wave receiving unit. Distance calculating means for calculating the distance to the first wave receiving unit, the calculated distance from the wave transmitting unit to the first wave receiving unit and the second wave receiving unit, and the first wave receiving stored in advance. On the projection surface of the wave generation unit of the transmitter, from the distance between the wave reception unit and the second wave reception unit, and the vertical distance between the wave reception unit and the projection surface serving as a reference for position calculation Calculate the position coordinates and A transmitter comprising: transmitter coordinate calculating means for calculating a corresponding position coordinate in an image; and a transmitter coordinate storage unit for sequentially storing the calculated position coordinate and outputting it to the image control unit of the projection display device. A position calculation unit;
A screen operation signal identifying unit that identifies an operation content from the input screen operation signal and instructs the image control unit to perform a corresponding image operation;
When the screen operation instructed by the image control unit is an area copy instruction signal, a copy range set in advance corresponding to the position coordinates of the transmitter and a position corresponding to the position coordinates of the transmitter in advance Copy video generation and pasting to copy the video information in the preset copy range according to the set insertion position conditions, and insert the image information into the preset insertion position in the video information and project it on the projection surface And a projection display device.

A projection display device that is used in combination with a transmitter including an electromagnetic wave light emitting unit and a wave transmitting unit, and includes an electromagnetic wave light receiving unit and a first wave receiving unit and a second wave receiving unit that are arranged apart from each other. ,
Based on the difference between the electromagnetic wave light receiving time in the electromagnetic wave light receiving unit and the wave receiving time in the first wave receiving unit and the second wave receiving unit, the first wave receiving unit and the first wave receiving unit from the wave transmitting unit A wave arrival time calculating means for calculating a wave arrival time to the second wave receiving unit, and from the calculated wave arrival time and sound velocity, the wave transmitting unit to the first wave receiving unit and the second wave receiving unit. Distance calculating means for calculating the distance to the first wave receiving unit, the calculated distance from the wave transmitting unit to the first wave receiving unit and the second wave receiving unit, and the first wave receiving stored in advance. On the projection surface of the wave generation unit of the transmitter, from the distance between the wave reception unit and the second wave reception unit, and the vertical distance between the wave reception unit and the projection surface serving as a reference for position calculation Calculate the position coordinates and A transmitter comprising: transmitter coordinate calculating means for calculating a corresponding position coordinate in an image; and a transmitter coordinate storage unit for sequentially storing the calculated position coordinate and outputting it to the image control unit of the projection display device. A position calculation unit;
A screen operation signal identifying unit that identifies an operation content from the input screen operation signal and instructs the image control unit to perform a corresponding image operation;
When the screen operation signal instructed to the image control unit is an area copy instruction signal, a copy range set in advance corresponding to the position coordinate of the transmitter and a position coordinate of the transmitter In accordance with a preset fitting position condition, video information in a preset copy range is sequentially copied with the movement of the transmitter position coordinate, and the position coordinate of the transmitter is moved. A projection display apparatus comprising: a copy image generation and pasting unit that fits in a preset fitting position and projects onto a projection surface.

When the screen operation signal is a drawing signal using a cursor, the cursor is projected onto the projection surface with the transmitter position as the cursor position, and drawing is performed on the projected image corresponding to the movement of the transmitter position. The projection display device according to claim 1, further comprising a drawing unit.

The transmitter has an operation unit for transmitting the screen operation signal on a wave output from the wave transmission unit,
4. The projection type according to claim 1, wherein the screen operation signal identification unit identifies the screen operation signal from the wave received by the wave reception unit and outputs the screen operation signal to the image control unit. 5. Display device.

The projection display device according to claim 4, wherein the transmitter is provided in an electronic pen.

The said screen operation signal identification part identifies the said screen operation signal input from the operation part of the said projection type display apparatus, and outputs it to the said image control part. Projection display device.

The screen operation signal recognition unit identifies a screen operation signal input from an operation unit of an apparatus connected to the projection display device, and outputs the screen operation signal to the image control unit. The projection display device according to item.

The projection display device according to any one of claims 1 to 7, wherein the projection display device is a projection display device including a projection mirror for image projection in a housing having an opening structure.

The projection display device according to any one of claims 1 to 8, wherein the electromagnetic wave is an infrared ray, and the wave is an ultrasonic wave.