JP2004265185A - Image projecting device with camera/with digitizer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a desk projecting device with camera/digitizer capable of handling paper data or image data of a reference product or the like similarly to electronic data in a remote meeting or an electronic conference. <P>SOLUTION: This device comprises a camera and a projector provided on a stand-like base, so that a projected image corrected in oblique projection or an image corrected in oblique see-through can be acquired in an optional editing area on a table. This device is in general a digitizer device for displaying an object in the editing area on an external monitor with the echo back of a digitizer, which can acquire the object as image data by one operation. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

＜本実施例における斜め投射補正、斜め透視補正、デジタイザ補正の説明＞
編集領域の画像がカメラで撮影される場合の斜め透視歪 ：Ａ
カメラと投影機の視差による歪 ：Ｂ
を求めるための，構成手段を図５−１，５−２のフローチャートに示す。
【００６３】
また、ここで投影機の斜め投影歪はＡ（ｉｎｖｅｒｔ）と近似し、このとき
投影機によって編集領域に投影される画像の歪：Ａ（ｉｎｖｅｒｔ）×Ｂと
書くことができるという前提で該構成手段は成り立っている。
【００６４】
図５−１，５−２の各ステップは図中に示すように、図３−１，３−２，４−２，４−３，４−４に夫々対応している。
【００６５】
【発明の効果】
以上、説明したように本発明によって、例えば本や書類のような紙のデータを編集領域に置くことにより画像データとしてリアルタイムに入力することができ、かつ該書類の上面においても前記光学式座標入力装置が成り立っているので、例えば、前記、紙のデータに対してそれを汚すことなくリアルタイムで電子的な追加書きが可能である。
【００６６】
また、本方式の場合、立体形状のオブジェクトも画像データとしてリアルタイムで取り込むことができ、例えば動きのある立体物をデータとして保存しつつ外部モニターで表示ないしネット上に発送するなども可能である。
【００６７】
また、本発明装置の設置位置、設置の方向を適宜えらぶことにより、使用者は机上の任意領域を任意の大きさで編集領域として選ぶことができる。
【図面の簡単な説明】
【図１−１】実施例外観図
【図１−２】実施例外観図
【図１−３】実施例外観図
【図２−１】本発明の構成図
【図３−１】斜め透視補正、斜め投影補正説明の図
【図３−２】斜め透視補正、斜め投影補正説明の図
【図４−１】斜め透視補正、斜め投影補正説明の図
【図４−２】斜め透視補正、斜め投影補正説明の図
【図４−３】斜め透視補正、斜め投影補正説明の図
【図４−４】座標入力機能較正説明の図
【図５−１】斜め透視補正、斜め投影補正、座標入力機能較正説明の流れ図
【図５−２】斜め透視補正、斜め投影補正、座標入力機能較正説明の流れ図
【符号の説明】
１ 動画静止画カメラ
２ 投影機
３ 動画静止画カメラ１，投影機２の組み合わせからなるユニット
８ 平面
６ ペン型指示具
１３ 書類
１２ 軌跡[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a camera with a digitizer, which is a tool for operation input, electronic data input, and paper data input in the case of editing a document, giving a presentation in an electronic conference, etc., and holding a meeting with a remote place. The present invention relates to an image projection device provided with an image.
[0002]
[Prior art]
Conventionally, in the case of editing a document, giving a presentation in an electronic conference, etc., holding a meeting with a remote place, etc., mainly use a keyboard, a mouse, a digitizer with a liquid crystal display function etc. as tools for operation input As a tool for inputting electronic data, first, in the case of electronic data, a recording medium such as a download from a network, a floppy (R) disk, or a CDROM is used.
[0003]
In the case of paper data, means such as taking in with a scanner and taking a still image with a document camera have been used independently.
[0004]
[Problems to be solved by the invention]
In the above-described situation, paper data is input independently of operation input and electronic data input in particular. Therefore, in editing work and presentation work involving paper data, the paper data is once electronically input. After converting the data, the work is performed, which not only takes time and labor, but also hinders the work that proceeds in real time.
[0005]
The present invention is mainly an input / output means used by connecting to a computer.
[0006]
The present invention provides a projection device that displays an image without distortion in an editing work area appropriately selected by a user on a table, a moving image still image camera that captures an image of the editing area, and obtains an image without distortion. The three members constitute an optical coordinate input device having an echo back.
[0007]
Further, with such a configuration, for example, by placing paper data such as a book or a document in the editing area, image data can be input in real time, and the optical coordinate input device is also provided on the upper surface of the document. Holds, for example, it is possible to perform electronic additional writing in real time on the paper data without polluting it.
[0008]
In addition, in the case of this method, a three-dimensional object can be captured in real time as image data. For example, it is possible to display a moving three-dimensional object on an external monitor or send it to a network while saving the data as data.
[0009]
Another advantage of the present invention is simplicity.
[0010]
The present invention is mainly shaped like a desk lamp, and the moving image still image camera and the image projector are installed on a stand base, and both are integrated, and the direction can be freely changed to some extent. It has become.
[0011]
That is, by appropriately selecting the installation position and installation direction of the apparatus of the present invention, the user can select an arbitrary area on the desk as an editing area of an arbitrary size.
[0012]
In fact, in the present invention, for example, when the user is right-handed, the apparatus of the present invention is effective by being placed in the left front of the editing area. For example, when the user is left-handed, the apparatus of the present invention is placed in the right front of the editing area. It becomes effective by being done.
[0013]
In this manner, the moving image and still image camera and the image projector view the editing area obliquely from each position.
[0014]
The present invention also proposes a procedure for correcting the "oblique viewing", that is, correcting oblique projection and oblique perspective.
[0015]
Here, in terms of an apparatus with correction of oblique projection and oblique perspective, there are prior inventions similar to the present invention, such as JP-A-6-89085 and JP-A-2001-169221.
[0016]
However, the former does not propose echo back by the projection means, and the latter does not propose a coordinate input device that functions in synchronization.
[0017]
The present invention has been made in view of the above points, and has an image projection with a digitizer and a camera so that image data such as paper data and reference products can be handled in the same manner as electronic data in a remote meeting or an electronic conference. It is intended to provide a device.
[0018]
[Means for Solving the Problems]
The present invention can solve the above problem by providing the following configuration.
[0019]
(1) A device comprising a stand-shaped unit that can be installed at an arbitrary position on a table at an arbitrary angle and a coordinate input indicator having an infrared light emitting function at the tip, wherein the stand-shaped unit is: Image projection means capable of projecting an image sent from the computer onto an arbitrary editing area on the table, and a second correction means for correcting oblique projection distortion in advance so that the image projected by the image projection means has no distortion. 1 image correcting means, an image projected on a table by the image projecting means, an image of an object present in an editing area on the table, and infrared light emitted from the coordinate input indicator. Moving image still image photographing means for photographing an infrared image, and a second method for obtaining an image in which oblique perspective distortion has been corrected from a visible light image photographed by the moving image still image photographing means. An image correcting unit, a unit for performing the same image correction as the second image correcting unit from the infrared image captured by the moving image and still image capturing unit to obtain an image for coordinate calculation, and a first unit for outputting an image to the outside , A second image output means, and an image projection device with a camera with a digitizer.
[0020]
(2) The moving image and still image photographing means and the image projecting means of the above (1) are fixed to each other so that they face in substantially the same direction, and the spot light emitted from the image projecting means is set to a predetermined value set on a table. An image projection device with a camera with a digitizer, wherein the direction and position of the stand-shaped unit of (1) are set by adjusting to the area of (1).
[0021]
(3) The moving image still image photographing means, which is obtained by photographing a camera configuration pattern temporarily placed on a table by the moving image still image photographing means, Wherein the image is corrected based on a perspective distortion correction parameter row A (invert) which is an inverse matrix of the perspective distortion parameter matrix A resulting from the position and direction of the image projection apparatus.
[0022]
(4) The first image correcting means of (1) projects, by the image projecting means of (1), an image on which a correction equal to the perspective distortion parameter matrix A of (3) has been performed in advance. The image and the camera configuration pattern of (3) are photographed by the moving image and still image photographing means of (1), and the moving image and still image photographing device and the image are obtained from the difference between the projected image and the camera configuration pattern. An image projection apparatus with a camera with a digitizer, wherein an image is corrected based on a matrix represented by A × B (inv), where B is a matrix representing parallax distortion between the projection apparatus and the projection apparatus.
[0023]
(5) The image projection device with a camera with a digitizer, wherein the pointing device for coordinate input of (1) superimposes a pen up-down signal and a switch signal on the pen as a time-series signal on the infrared light emission.
[0024]
(6) The camera with a digitizer, wherein the unit of (1) includes a photosensor near the moving image and still image camera, and the photosensor extracts a time-series signal accompanying the infrared emission of (5). With image projection device.
[0025]
(7) The first image output means of (1) subtracts the image sent from the computer of (1) after delay correction from the output image of the second image correction means of (1). An image projection device with a camera with a digitizer, which outputs a first output image.
[0026]
(8) The second image output means of (1) switches between the output image of the second image correction means of (1) and the image sent from the computer of (1). An image projection device with a camera with a digitizer, which can be output as a second image output.
[0027]
(9) The unit of (1) can select either an image sent from the computer of (1) as an image to be projected by the image projection means or an illumination image for illuminating the editing area. An image projection device with a camera with a digitizer, characterized by the following.
[0028]
(10) A device comprising a stand-shaped unit that can be installed at an arbitrary angle at an arbitrary position on a table and a coordinate input indicator having an infrared light emitting function at the tip, wherein the stand-shaped unit is Image projection means capable of projecting an image sent from a computer onto an arbitrary editing area on a table; and correcting oblique projection distortion in advance so that the image projected by the image projection means has no distortion. First image correction means, an image projected on a table by the image projection means, an image of an object present in an editing area on the table, and infrared light emitted from the coordinate input indicator. Moving image still image photographing means for photographing an infrared image, and a second method for obtaining an image in which oblique perspective distortion has been corrected from a visible light image photographed by the moving image still image photographing means. Image correction means, means for performing the same image correction as the second image correction means from an infrared image captured by the moving image still image capturing means to obtain a coordinate calculation image, and outputting the image to the outside 1. An image projection apparatus with a camera having a digitizer, comprising: a first image output unit; and a first mode (normal digitizer mode), wherein the image projection unit projects an image including an echo back sent from a computer. The second image output means also projects an image including an echo back sent from the computer. In the second mode (object shooting mode), the image projection means outputs a solid image for illuminating the editing area. And the second image output function outputs an image including an echo back sent from the computer, and outputs the image from the first image output means. Image data of the object in the editing area is transmitted to the computer, and in a third mode (direct marking mode), the image projection means projects a composite image of a plain signal sent from the computer and an echo back, and 2. An image projection device with a camera with a digitizer, wherein the image output function of 2 outputs an image acquired by a moving image / still image acquisition means.
[0029]
BEST MODE FOR CARRYING OUT THE INVENTION
(Example 1)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0030]
FIG. 1-2 is a diagram that best illustrates the present invention.
[0031]
In this embodiment, a moving image and still image camera 1, a projector 2 (a unit 3 composed of a combination of 1 and 2), a plane 8 on which a predetermined editing area can be selected on a desk, and a pen-type pointing device 6 having an infrared light emitting element at its tip Consists of
[0032]
FIGS. 1-1, 1-2, and 1-3 represent different use situations. FIG. 1-1 is an external view of the projection state after performing the oblique perspective correction of the moving image and still image camera 1, the oblique projection correction of the projector 2, and the parallax correction of the camera and the projector. In this state, an image with little distortion is projected on the editing area.
[0033]
FIG. 1-2 shows a state in which, after the correction is established, a trajectory input by infrared emission is performed, and the result is echoed back correctly.
[0034]
FIG. 1-3 shows a state in which a document 13 is further placed in the editing area, and the trajectory 12 is electronically drawn thereon by a pen-shaped pointing tool.
[0035]
Next, the flow of this embodiment will be described with reference to FIG.
[0036]
In the present invention, an image is taken by a camera from a position oblique to the editing area and projected by a projector. Therefore, oblique perspective distortion and oblique projection distortion occur in each case.
[0037]
Here, since the camera and the projector are integrated, their facing directions are substantially the same. Therefore, the oblique perspective distortion and the oblique projection distortion have an inverse matrix relationship.
[0038]
However, the camera and the projector are not at the same position but at a short distance, but are separated by a predetermined distance, so that a parallax between the two occurs. Therefore, each distortion is classified as follows (all of them are matrices).
[0039]
Perspective distortion when an image in the editing area is captured by a camera: A
Distortion due to parallax between camera and projector: B
Also, if you define like this,
Distortion of image projected on editing area by projector: A (invert) × B
Can be written.
[0040]
Therefore, for the visible light image and infrared image captured by the camera,
The oblique perspective distortion correction to be performed after imaging is A (invert),
The correction to be performed in advance on the image projected by the projector is:
A × B (invert).
[0041]
In the present invention, first, the above-mentioned A and B are obtained by means described later. Here, assuming that these have already been obtained, the flow of this embodiment will be described below with reference to FIG.
[0042]
The first embodiment is mainly an input / output device for a computer.
[0043]
The image means for projection is sent from the computer for this embodiment. The normal display image 38 or the illumination image 41 is selected with the switch 2, the projection distortion correction A and the parallax distortion correction B (invert) are performed by the projection image correction processing unit, and the projection image is projected on the editing area by the projector 2. At this time, a projection image with little distortion is reproduced in the editing area.
[0044]
The moving image still image camera 5 captures a scene in the editing area, a projection image projected from the projector, and an infrared image from a wireless pen.
[0045]
Here, the scenery of the editing area expresses the existence of paper data such as a document and a photograph placed in the editing area or a three-dimensional object.
[0046]
In the moving picture still camera 5, an infrared image and a visible light image are separated by an infrared half mirror, and the visible light image is captured by an image sensor a and the infrared image is captured by an image sensor b.
[0047]
The visible light image is subjected to oblique perspective distortion correction A (invert) by the projection image correction processing unit.
) Is performed to obtain a corrected photographed image with less distortion. Here, the corrected photographed image is a composite image of a video of an object such as a document or a photograph in an editing area and a video projected by a projector. Is,
In this embodiment, in this part, only the former of the composite images needs to be stored in the memory as input information, and the latter (projection image) is unnecessary and needs to be removed.
[0048]
Therefore, the projection image subtraction processing unit 35 subtracts the image before correction of the projection image. In addition, this subtraction requires that the time of the subtracted image and the time of the subtracted image coincide in time. For this reason, the subtraction image is given a time delay in the delay processing unit. The object image from which the projection image has been removed in this way is sent to the computer main body as the first output image 37 and stored in the memory.
[0049]
On the other hand, the infrared image emitted from the pen-type pointing device and captured by the image sensor b is subjected to the oblique perspective distortion correction A (in) by the coordinate data correction processing unit 33 similarly to the visible light image.
(vert), and the corrected infrared image is obtained. Further, the coordinate calculation unit 34 calculates an actual coordinate value based on the corrected infrared image.
[0050]
Here, the infrared light emitted from the wireless pen includes pen up / down data, mode data, and the like as time-series information. The photo sensor 30 receives pen-up / down data, mode data, and the like by receiving infrared light emitted from the pointing device as the time-series information. These data are combined with coordinate values by the coordinate calculation unit 34 and sent to a USB port or the like of the computer as coordinate data.
[0051]
In this embodiment, the output images include the first output image 37 and the second output image 40. The first output image is an image for taking in the object in the editing area as data as described above, and the projection image is removed.
[0052]
The second image is an image for output to an external monitor.
[0053]
<Description of Three Modes in the Embodiment>
The present invention has three operation modes depending on the application.
[0054]
The first mode is a “normal digitizer mode”, the second mode is an “object capturing mode”, and the third mode is a “direct display mode”.
[0055]
The “normal digitizer mode” is a normal digitizer mode in which the object is not captured and both the second image output and the projected image are equal to the computer output image.
[0056]
The “object capturing mode” is a mode for capturing an object (document, document, object, or the like) in the editing area as moving image data or still image data, and the projector functions as a lighting device.
[0057]
Also, some normal computer output image is displayed on the external monitor.
[0058]
The "direct display mode" is a mode in which an electronic echo back is projected on a document or object in the editing area and the state is displayed on an external monitor as it is, when giving a presentation etc. using an object or paper document I use it.
[0059]
In this case, the application on the computer sends, to the present embodiment, an image obtained by combining only the echo back of the pen with an image such as a raster, and the present embodiment corrects the distortion and projects the image on the object in the editing area.
[0060]
This state is indicated on an external monitor as it is. Therefore, at this time, the first image is not captured. The coordinate data is imported.
[0061]
The operation in each mode will be described below.
[0062]

<Description of oblique projection correction, oblique perspective correction, and digitizer correction in the present embodiment>
Oblique perspective distortion when an image in the editing area is captured by a camera: A
Distortion due to parallax between camera and projector: B
5-1 are shown in the flowcharts of FIGS. 5-1 and 5-2.
[0063]
Also, here, the oblique projection distortion of the projector is approximated to A (invert), and the distortion of the image projected on the editing area by the projector at this time can be written as A (invert) × B. The means are working.
[0064]
The steps in FIGS. 5-1 and 5-2 correspond to FIGS. 3-1 and 3-2, 4-2, 4-3 and 4-4, respectively, as shown in the figure.
[0065]
【The invention's effect】
As described above, according to the present invention, paper data such as a book or a document can be input in real time as image data by placing the data in an editing area, and the optical coordinate input device can also be used on the upper surface of the document. Holds, for example, it is possible to perform electronic additional writing in real time on the paper data without polluting it.
[0066]
In addition, in the case of this method, a three-dimensional object can be captured in real time as image data. For example, it is possible to display a moving three-dimensional object on an external monitor or send it to a network while saving the data as data.
[0067]
In addition, by appropriately selecting the installation position and installation direction of the apparatus of the present invention, the user can select an arbitrary area on the desk as an editing area having an arbitrary size.
[Brief description of the drawings]
FIG. 1-1 is an external view of the embodiment. FIG. 1-2 is an external view of the embodiment. FIG. 1-3 is an external view of the embodiment. FIG. 2-1 is a configuration diagram of the present invention. Illustration of oblique projection correction. [FIG. 3-2] Illustration of oblique perspective correction and oblique projection correction. [FIG. 4-1] Illustration of oblique perspective correction and oblique projection correction. [FIG. 4-2] Oblique perspective correction, oblique. Illustration of projection correction [Figure 4-3] Illustration of oblique perspective correction, oblique projection correction [Figure 4-4] Illustration of coordinate input function calibration [Figure 5-1] Oblique perspective correction, oblique projection correction, coordinate input Flowchart for explaining function calibration [Fig. 5-2] Flowchart for explaining function calibration for oblique perspective correction, oblique projection correction, and coordinate input
DESCRIPTION OF SYMBOLS 1 Video still image camera 2 Projector 3 Unit 8 composed of a combination of video still image camera 1 and projector 2 Plane 6 Pen-shaped pointing device 13 Document 12 Trace

Claims (10)

A stand-shaped unit that can be installed at any position on the table at any angle,
An apparatus comprising a coordinate input indicator having an infrared light emitting function at the tip,
The stand-shaped unit is an image projection unit capable of projecting an image sent from a computer onto an arbitrary editing area on a table,
First image correction means for correcting oblique projection distortion in advance so that the image projected by the image projection means has no distortion;
An image projected on the table by the image projection means,
An image of an object existing in the editing area on the table;
An infrared image formed by infrared light emitted from the coordinate input indicator,
A moving picture still image shooting means to shoot,
A second image correction unit for obtaining an image in which oblique perspective distortion has been corrected from a visible light image captured by the moving image still image capturing unit,
Means for performing the same image correction as the second image correction means from the infrared image captured by the moving image still image capturing means to obtain a coordinate calculation image;
First and second image output means for outputting an image to the outside,
An image projection device with a camera with a digitizer, comprising: The moving image and still image photographing means and the image projecting means of claim 1 are fixed to each other so as to face in substantially the same direction, and adjust the spot light emitted from the image projecting means to a predetermined area set on the table. An image projection device with a camera with a digitizer, wherein the direction and the position of the stand-like unit according to claim 1 are set. 2. The position and direction of the moving image still image photographing means, wherein the second image correcting means is obtained by photographing a camera configuration pattern temporarily placed on a table by the moving image still image photographing means. is the inverse matrix of the perspective distortion parameter matrix a where due to, perspective distortion correction parameter line a (inv
ert), wherein the image is corrected based on (ert). The first image correcting means according to claim 1 projects, by the image projecting means according to claim 1, an image which has been corrected in advance to be equal to the perspective distortion parameter matrix A according to claim 3, and projects the projected image and the camera according to claim 3. A configuration pattern is photographed by the moving image and still image photographing means of claim 1, and a parallax distortion between the moving image and still image photographing apparatus and the image projection apparatus, which is obtained from a difference between the projected image and the camera composition pattern, is calculated. An image projection apparatus with a camera with a digitizer, wherein an image is corrected based on a matrix represented by A × B (inv), where B represents a matrix. 2. The image projection apparatus with a digitizer according to claim 1, wherein the pointing device for coordinate input superimposes a pen up / down signal and a switch signal on the pen as a time-series signal on the infrared light emission. 6. An image projection device with a digitizer according to claim 5, wherein the unit according to claim 1 further comprises a photosensor near the moving image and still image camera, and the photosensor extracts a time-series signal accompanying the infrared light emission. The first image output means of the first aspect outputs the first output image obtained by delay-correcting and subtracting the image sent from the computer of the first aspect from the output image of the second image correction means of the first aspect. An image projection device with a camera with a digitizer, which outputs the image. The second image output means of the first aspect switches any one of the output image of the second image correction means of the first aspect and an image sent from the computer of the first aspect as a second image output. An image projection device with a camera equipped with a digitizer, capable of outputting. 2. The digitizer according to claim 1, wherein the unit is capable of selecting one of an image sent from the computer as an image projected by the image projection means and an illumination image for illuminating the editing area. Image projection device with camera. A stand-shaped unit that can be installed at any angle on the table at any angle,
An apparatus comprising a coordinate input indicator having an infrared light emitting function at the tip,
The stand-shaped unit is an image projection unit capable of projecting an image sent from a computer onto an arbitrary editing area on a table,
First image correction means for correcting oblique projection distortion in advance so that the image projected by the image projection means has no distortion,
An image projected on the table by the image projection means,
An image of an object existing in the editing area on the table;
An infrared image formed by infrared light emitted from the coordinate input indicator,
A moving picture still image shooting means to shoot,
A second image correcting unit for obtaining an image in which oblique perspective distortion has been corrected from a visible light image captured by the moving image still image capturing unit,
Means for obtaining an image for coordinate calculation by performing the same image correction as the second image correcting means from an infrared image captured by the moving image still image capturing means;
An image projection device with a camera with a digitizer, comprising: first and second image output means for outputting an image to the outside,
In a first mode (usually a digitizer mode), the image projection means projects an image including an echo back sent from a computer,
The second image output means also projects an image including an echo back sent from the computer,
In a second mode (object shooting mode), the image projection means projects a plain image for illuminating an editing area,
The second image output function outputs an image including an echo back sent from the computer,
Image data of the object in the editing area is transmitted from the first image output means to the computer;
In a third mode (direct marking mode), the image projection means projects a composite image of a plain signal sent from a computer and an echo back,
An image projection device with a camera with a digitizer, wherein the second image output function outputs an image acquired by a moving image / still image acquisition unit.

Images