JP2012027769A - Interactive white board device, image display device and calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interactive white board device, an image display device and a calibration method with which processing time for calibration can be shortened.SOLUTION: An interactive white board device comprises: an image creation part 211 for creating multiple calibration images each of which includes points whose display position are different; an estimated position calculation part 214 for calculating an estimated position estimated to be indicated according to a point included in the next image which is a calibration image to be displayed next, based on an already detected indication position of each point included in the multiple calibration images; a coordinate position determination part 215 for determining whether or not an indication position of a point included in the next image detected by imaging means is within a predetermined range against the estimated position; and an execution part 216 for executing displaying or not of a calibration image after the next image and calibration images thereafter based on the determination result.

Description

  The present invention relates to an interactive whiteboard device, an image display device, and a calibration method.

Conventionally, a front projection touch panel system including a large touch panel, a projector, and a personal computer has been known (see, for example, Patent Document 1).
In the front-projection touch panel system described in Patent Document 1, calibration that corrects a deviation between a touch position on a large touch panel caused by a positional relationship between a projector and a personal computer and a position on the logical coordinate of the personal computer with respect to the touch position. Is executed.

Specifically, the personal computer generates an image including four marker points and outputs the image to a projector, and the projector projects the image on a large touch panel. When the four marker points displayed on the large touch panel are touched, the large touch panel detects the position coordinates of the touched four marker points and transmits the data of the position coordinates to the personal computer. Further, when an arbitrary point is touched on the large touch panel, the large touch panel transmits the position coordinates of the arbitrary point to the personal computer.
Then, based on the four marker points transmitted from the large touch panel and the position coordinates of the arbitrary point, the personal computer calculates a projection point for the arbitrary point, and the arbitrary point and the projection point are calculated. A calibration process for matching points is performed.

Japanese Patent Laid-Open No. 10-21007

  However, in such a front projection touch panel system described in Patent Document 1, it is necessary to perform calibration processing with any one point as an essential element in addition to the four marker points. For this reason, when performing the calibration process, it is necessary to instruct (touch) all five points, and there is a problem that the calibration process takes time.

  An object of the present invention is to provide an interactive whiteboard device, an image display device, and a calibration method that can shorten the calibration processing time.

  An interactive whiteboard apparatus according to the present invention includes an interactive image processing unit that outputs an image, an image display unit that displays the image, and a position detection unit that detects an indicated position instructed in the display area of the image. In the whiteboard device, any one of the image output unit and the image display unit is already detected by the image generation unit that generates a plurality of calibration images each including points having different display positions, and the position detection unit. Based on the indicated position of each point included in the plurality of calibration images, the estimated position estimated to be specified according to the point included in the next image that is the calibration image to be displayed next An estimated position calculation unit for calculating the estimated position and the estimated position by the position detection means. When the coordinate position determination unit that determines whether or not the indicated position of the point included in the subsequent image is within a predetermined range, and the coordinate position determination unit determines that the point is not within the predetermined range, When the image display means displays the calibration image further next to the next image and the coordinate position determination unit determines that the calibration image is located within the predetermined range, the calibration after the next calibration image is performed. And a control unit for stopping the display of the image.

  According to the present invention, when the displayed plurality of points are respectively indicated by, for example, an instruction unit that emits infrared light or ultrasonic waves, the position detection unit is set at the emission position of the infrared light or ultrasonic waves. Based on this, the designated position corresponding to each point is detected. The estimated position calculation unit calculates an estimated position estimated to be instructed according to a point included in the next image that is a calibration image to be displayed next based on the indicated position detected by the position detecting unit. calculate. Then, the coordinate position determination unit determines whether or not the estimated position and the indicated position of the point included in the next image detected by the position detection unit are within a predetermined range. If it is determined that the indicated position is not within the predetermined range with respect to the estimated position, the next calibration image of the next image is displayed. On the other hand, if it is determined to be within the predetermined range, the next calibration of the next image is performed. Stop the display after the image.

According to this, when the designated position and the estimated position are within a predetermined range, it is possible to perform appropriate calibration by displaying up to the calibration image including the point corresponding to the designated position. . Therefore, the number of points designated by the user, that is, the number of displayed calibration images can be reduced, and the calibration processing time can be shortened.
For example, when a calibration image including the third point is displayed, the designated position corresponding to the third point, the designated positions of the first point and the second point, and the calibration image If the estimated position of the third point calculated based on the position of the third point is within a predetermined range, the calibration can be appropriately performed based on the designated position of the first to third points. it can. Therefore, the image display can be completed until the display of the calibration image (next image) including the third point, and the calibration image (further next image) including the fourth point is displayed. Since there is no need, the number of point instruction operations by the user can be reduced. Accordingly, the calibration processing time can be shortened as compared with the case where processing is performed using the above-described five conventional points.

  In the present invention, when the next image is displayed, an end button display unit that displays an end button is provided, and when the position detecting unit detects that the end button has been instructed, It is preferable that display of the calibration image after the next calibration image is stopped.

  According to the present invention, for example, when the third point is displayed, the end button is displayed in the display area of the next image. According to this, if necessary (for example, when the estimated position is displayed, the position of the third point and the estimated position corresponding to the third point are within a predetermined range. By pressing the end button, the process can be terminated only by displaying the next image including the third point, and the number of point instruction operations by the user can be further reduced. Therefore, the calibration processing time can be further shortened.

In this invention, it is preferable to provide the range setting part which sets the said predetermined range.
Here, depending on the usage environment, the amount of deviation between the indicated position for position detection and the calculated estimated position may not affect the usage status. That is, there may be no problem even if the range set as the predetermined range is increased.
For this reason, in this invention, the said range can be suitably set according to a use environment or a use by the range setting part. Therefore, for example, even when the amount of deviation between the designated position and the estimated position is large, the calibration process can be executed with only the minimum necessary image display. Can be reduced.

  An image display device according to the present invention is an image display device comprising image display means for displaying an image and position detection means for detecting an indicated position designated in the display area of the image, wherein the display positions are different. Generating a plurality of calibration images each of which is included in the image display means and outputting them to the image display means, and at the indicated positions of the points included in the plurality of calibration images already detected by the position detection means Based on an estimated position that is estimated to be instructed according to a point included in the next image that is the calibration image that is displayed next, and for the estimated position, A coordinate position determination unit that determines whether or not the indicated position of the point included in the next image detected by the position detection means is within a predetermined range. When the coordinate position determination unit determines that the position is not within the predetermined range, the image display means displays the next calibration image after the next image, and the coordinate position determination unit causes the predetermined range to be displayed. And a control unit that stops the display of the calibration image after the next calibration image when it is determined that the position is located inside.

  Also in the image display device of the present invention, the same effects as those of the invention related to the interactive whiteboard device described above can be obtained.

In the present invention, when the next image is displayed, an end button display unit that displays an end button is provided, and when the position detecting unit detects that the end button has been instructed, It is preferable that display of the calibration image after the next calibration image is stopped.
Also in the present invention, the same effects as those of the invention related to the interactive whiteboard apparatus described above can be obtained.

In this invention, it is preferable to provide the range setting part which sets the said predetermined range.
Also in the present invention, the same effects as those of the invention related to the interactive whiteboard apparatus described above can be obtained.

The calibration method of the present invention is a calibration method executed by the above-described image display device, and generates a plurality of calibration images each including points with different display positions, and outputs the images to the image display means. A generation step, an indication position detection step for detecting an indication position indicated in the display area of the calibration image, and each of the points included in the plurality of calibration images already detected by the indication position detection step An estimated position calculating step for calculating an estimated position estimated to be instructed according to a point included in the next image that is the calibration image to be displayed next based on the indicated position; The point indicated position included in the next image detected by the indicated position detecting step A coordinate position determining step for determining whether or not the image is located within a predetermined range, and if it is determined by the coordinate position determining step that the image is not within the predetermined range, A control step of displaying the calibration image and stopping the display of the calibration image after the next calibration image when it is determined by the coordinate position determination step that the position is within the predetermined range. It is characterized by.
According to the present invention, the same effects as those of the image display device described above can be obtained.

1 is a schematic configuration diagram schematically showing an interactive whiteboard device according to a first embodiment of the present invention. The figure which shows an example of the input image in the said 1st Embodiment, a projection image, and a captured image. The block diagram of the interactive whiteboard apparatus in the said 1st Embodiment. The block diagram of PC in the said 1st Embodiment. The figure which shows the input image produced | generated by PC in the said 1st Embodiment. The figure which shows a mode that the calibration in the said 1st Embodiment is performed. The figure which shows a mode that the calibration in the said 1st Embodiment is performed. 5 is a flowchart showing calibration execution processing in the first embodiment. The block diagram of the interactive whiteboard apparatus which concerns on 2nd Embodiment of this invention. The figure which shows the end button displayed with the interactive whiteboard apparatus in the said 2nd Embodiment. The flowchart which shows the calibration execution process in the said 2nd Embodiment.

[First Embodiment]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment according to the invention will be described with reference to the drawings.
[Configuration of interactive whiteboard device]
FIG. 1 is a schematic configuration diagram schematically illustrating an interactive whiteboard device 1 according to the present embodiment.
As shown in FIG. 1, an interactive whiteboard device (hereinafter sometimes abbreviated as “IW device”) 1 according to the present embodiment includes a PC (Personal Computer) 2 as image output means and an image input from the PC 2. Projector 3 as an image display means or an image display device for projecting an input image based on information onto the projection surface 4, an electronic pen 5 (see FIG. 3), and position detection for detecting an indicated position designated by the electronic pen 5 The image pickup means 32 (refer FIG. 3) as a means is provided, and it is comprised as a system provided with said each apparatus.
The IW device 1 displays a projection image projected from the projector 3 on a projection surface 4 such as a screen or a white board, and the electronic pen 5 (see FIG. 3) that emits infrared light on the projection surface 4. The instructed position is detected, and a line or the like corresponding to the instructed position of the electronic pen 5 is drawn in the projection image.

  In such an IW device 1, as will be described in detail later, the projector 3 includes an imaging unit 32 that captures an imaging area including an area (projection area) on which an image is projected on the projection surface 4. Acquires a picked-up image obtained by picking up an image of the projected surface 4 including a locus or the like that is pseudo-written with the electronic pen 5 in the image pickup region. Then, the projector 3 acquires the captured image, analyzes the emission position of the infrared light in the captured image, and transmits the position information of the emission position to the PC 2 as the position information of the instruction position by the electronic pen 5. . On the other hand, the PC 2 generates an input image in which a line indicating the locus of the designated position is superimposed on an image of a presentation material or the like, and transmits image information of the input image to the projector 3. Thereby, a projection image in which a line indicating the locus of the electronic pen 5 is reflected on the projection surface 4 is projected by the projector 3.

FIG. 2 is a diagram illustrating an example of the input image Pi, the projection image Po, and the captured image Px. Specifically, FIG. 2A is a diagram showing an input image Pi generated by the PC 2 and input to the projector 3, and FIG. 2B is projected on the projection surface 4, and is shown in FIG. FIG. 2C is a captured image Px of the projected image Po that is displayed on the projection surface 4 and captured at the B viewpoint in FIG. 1.
Further, in such an IW device 1, it is necessary to perform a process called calibration before performing the above-described process. This calibration is performed by imaging the coordinate position in the input image Pi (FIG. 2A) input from the PC 2 to the projector 3 and the projection image Po projected on the projection surface 4 (FIG. 2B). In the image Px (FIG. 2C), the coordinate position (exit position) of the locus written by the electronic pen 5 with respect to the projection image Po is previously matched. That is, the IW device 1 performs calibration processing for aligning coordinates in the input image Pi generated by the PC 2 and input to the projector 3 with coordinates in the projected image Po projected from the projector 3 onto the projection surface 4. Execute.

This is because when the emission position of the electronic pen 5 and the position in the input image Pi do not match, the projector 3 projects an image reflecting a line indicating the locus of the electronic pen 5 on the projection surface 4. This is because the position of the trajectory is shifted from the position indicated by the electronic pen 5 in the projection image Po. In particular, when an image whose trapezoidal distortion has been corrected by the projector 3 is projected onto the projection surface 4, such a shift becomes significant, and thus calibration processing is necessary.
Details of the specific calibration process will be described later.

[Configuration of electronic pen]
Although not shown in detail, the electronic pen 5 is a pointing device having a size that can be grasped by a user's hand. The electronic pen 5 includes a control unit 51, an LED (Light Emitting Diode) 52 that emits infrared light, and a first switch that switches a light emission state of the infrared light emitted from the LED 52 to the tip of the electronic pen 5. 53 and a second switch 54 for switching the light emission pattern of the infrared light emitted from the LED 52 is provided substantially at the center. The electronic pen 5 is configured such that the first switch 53 is turned on and the LED 52 is lit while the tip is pressed against the projection surface 4. The control unit 51 controls the light emission state of the LED 52 based on the state of the first switch 53. That is, when the first switch 53 is in the on state, the LED 52 is caused to emit light in a predetermined light emission state indicating that the first switch 53 is in the on state.
When the second switch 54 is input by the user, the second switch 54 is turned on. When the second switch 54 is in an on state, the control unit 51 causes the LED 52 to emit light with a predetermined light emission pattern indicating that the second switch 54 is in an on state.

For this reason, when drawing a line etc. on a projection image at the time of execution of an interactive function realized by IW device 1, a user inputs the 1st switch 53 and makes infrared light emit from LED52. Further, when performing an operation such as a click operation and a drag operation as a pointing device, the user inputs the second switch 54 and switches the emission pattern of infrared light emitted from the LED 52.
Note that when the first switch 53 and the second switch 54 are in the off state, the control unit 51 maintains the state in which the LED 52 is turned off. Various devices such as a pressure sensor and a tact switch can be employed as the first switch 53 and the second switch 54 described above.

[Projector configuration]
FIG. 3 is a block diagram showing the configuration of the PC 2 and the projector 3 that constitute the IW device 1.
As shown in FIG. 3, the projector 3 includes an image projection unit 31 as an image display unit, an imaging unit 32 as a position detection unit, a storage unit 33, and an image reception unit in an exterior housing 30 (see FIG. 1). Means 34 and control means 35 are provided.
The image projection unit 31 forms image light corresponding to the image information of the input image input from the PC 2, enlarges and projects the image light, and displays the projection image on the projection surface 4. At this time, when the input image Pi shown in FIG. 2A is input from the PC 2, the projection image Po including the point M1 is displayed.
The image projection unit 31 includes a light source device 311, a light modulation device 312, and a projection lens 313.

Although not shown, the light source device 311 includes a light source lamp such as a high-pressure mercury lamp and a reflector that is a reflecting mirror, or a solid light source such as an LED, and irradiates the light modulation device 312 with a light beam.
Although not shown, the light modulation device 312 includes a liquid crystal panel that modulates a light beam emitted from the light source device 311 to form image light, and a driver that drives the liquid crystal panel according to an input drive signal. Configured. Note that the light modulation device 312 is not limited to a configuration including a liquid crystal panel, and a configuration other than liquid crystal, such as a device using a micromirror, may be employed.
The projection lens 313 enlarges and projects the image light formed by the light modulation device 312 in the projection area of the projection surface 4, thereby displaying a projection image on the projection surface 4.

The imaging means 32 is a camera (imaging device) that can detect the emission position of infrared light emitted from the electronic pen 5 with respect to the projection image Po, and can be constituted by a CCD (Charge Coupled Device). A device provided with another imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) device can also be configured.
Although not shown, the imaging unit 32 is provided on the front side (front side in the projection direction) of the exterior housing 30, and the imaging area of the imaging unit 32 is an image projection area (display) by the projection lens 313. Area).

Specifically, in such an imaging unit 32, for example, a point M1 is instructed by infrared light emitted from the electronic pen 5 with respect to a projection image Po as shown in FIG. Then, the captured image Px shown in FIG.
Then, the imaging unit 32 detects the coordinate position (indicated position) of the point M1 (indicated in FIG. 6 which will be described later) by the designated point M1 in the captured image Px, and transmits the position detection data to the PC 2. .

The storage means 33 stores various programs and data necessary for the control means 35 to control the entire projector 3.
The image receiving unit 34 outputs image information (including an image signal and a packet) received from the PC 2 to the control unit 35. As such image information, for example, an input image Pi that is a calibration image including a point M generated by an image generation unit 211 of the PC 2 described later is included.

  The control unit 35 includes a CPU (Central Processing Unit), a video processing processor, and the like, and executes the program stored in the storage unit 33 to control the entire projector 3. The control unit 35 includes an image processing unit 351, a frame memory 352, and a drive control unit 353 configured by the video processing processor.

The image processing unit 351 develops an image corresponding to the image information input from the image receiving unit 34 on the frame memory 352.
The frame memory 352 is configured to be capable of storing a plurality of images (frames) for one screen formed by the light modulation device 312. In such a frame memory 352, for example, the input image Pi by the image processing unit 351 is developed as described above.
The drive control unit 353 outputs a drive signal for driving the light modulation device 312 to the light modulation device 312 based on the image data of the input image written in the frame memory 352. In addition, the drive control unit 353 outputs a control signal for controlling turning on / off of the light source device 311 and the amount of emitted light.

[PC configuration]
FIG. 4 is a block diagram showing the configuration of the PC 2 that constitutes the IW device 1.
The PC 2 generates an image to be projected on the projector 3 and transmits image information corresponding to the image. As shown in FIG. 4, the PC 2 includes an operation unit 20, a control unit 21, and a storage unit 22.
Among these, the operation means 20 includes a keyboard and a pointing device such as a mouse, and the operation means 20 outputs an operation signal corresponding to an input operation to these to the control means 21.

  The storage unit 22 stores various programs and data necessary for processing by the control unit 21. As such a program, which will be described later in detail, for example, an image generation program for generating an input image Pi that is executed by the control means 21 and is a calibration image including the point M1 shown in FIG. An estimated position calculation program for causing the estimated position calculation unit 214 (to be described later) to calculate a coordinate position (estimated position) of the estimated point N may be used.

In addition, the storage unit 22 stores in advance, as the data, a template of a calibration image, set position data of the point M included in the calibration image, and the like.
In addition, the storage means 22 stores position detection data (position detection data of the instructed point M) input from the projector 3 and coordinate position data (estimation) of the estimated point N calculated by the estimated position calculation unit 214 described later. Position data), and the upper limit value of the deviation amount between the coordinate position (instructed position) of the infrared light emission position and the coordinate position (estimated position) of the estimated point N referred to by the coordinate position determining unit 215 described later Remembered.
Here, the upper limit value of the deviation amount is set by the range setting unit 217 described later by appropriately inputting a desired value according to the usage status of the IW device 1 using the operation unit 20 described later. It is.

  The control unit 21 includes, for example, a CPU (Central Processing Unit) and the like, and executes a later-described program stored in the storage unit 22. In other words, the control unit 21 realizes various functions by processing the program and data stored in the storage unit 22. Such a control means 21 processes the program when executing the above-described calibration, and as shown in FIG. 4, an image generation unit 211, an image transmission unit 212, and a position information acquisition unit. 213, the estimated position calculation part 214, the coordinate position determination part 215, the execution part 216 as a control part, and the range setting part 217 are implement | achieved as a function.

FIG. 5 is a diagram illustrating an example of the input image Pi generated by the PC 2.
When an operation for executing a calibration process is performed from the operation unit 20, the image generation unit 211 reads an image generation program, setting position data of the points M (M1 to M9), and the like from the storage unit 22, and generates a calibration image. Generate. This calibration image includes one of points M1 to M9 as shown in FIG. These points M1 to M9 are set as calibration images in order from the point M1, and in the present embodiment, a maximum of nine calibration images each including the points M1 to M9 are generated.
The image transmission unit 212 transmits the calibration image generated by the image generation unit 211 to the projector 3.
The position information acquisition unit 213 acquires position information (position detection data indicating a position instructed by the electronic pen 5) received from the projector 3 after the calibration image is transmitted by the image transmission unit 212. The position information acquisition unit 213 stores the position information in the storage unit 22.

Here, FIGS. 6 and 7 are diagrams illustrating how the calibration is performed in the IW device 1, and the functions of the estimated position calculation unit 214 and the coordinate position determination unit 215 are described with reference to FIGS. 6 and 7. explain.
As shown in FIGS. 6A to 6C, the image projection unit 31 of the projector 3 uses the calibration image CP1 (see FIG. 6A) including the point M1 generated by the image generation unit 211 of the PC 2. The projected image PP1 (see FIG. 6B) is displayed on the projection surface 4. Then, as shown in FIG. 6C, when the point M1 displayed by the electronic pen 5 operated by the user is instructed, the imaging unit 32 detects infrared light emitted from the electronic pen 5. Then, the projection image PP1 is captured (see FIG. 6D).
Further, as shown in FIGS. 6E to 6G, similarly to FIGS. 6A to 6C, the point M2 is included at a position different from the point M1 generated by the image generation unit 211 of the PC2. The image projection means 31 of the projector 3 displays the calibration image CP2 (see FIG. 6E) to be displayed on the projection surface 4 as the projection image PP2 (see FIG. 6F). Then, as shown in FIG. 6G, when the point M2 displayed by the electronic pen 5 operated by the user is instructed, the imaging unit 32 detects the infrared light emitted from the electronic pen 5. Then, the projection image PP2 is captured (see FIG. 6H).

  The estimated position calculation unit 214 detects position detection data indicating the emission position of the infrared light included in the captured images SP1 and SP2 of the projection images PP1 and PP2 transmitted from the imaging unit 32 at the positions indicated by the points M1 and M2. Data is read from the storage means 22 as data. Then, the estimated position calculation unit 214, based on the position detection data and the coordinate positions of the points M1 and M2 in the calibration images CP1 and CP2, the estimated position (estimated position) of the point M3 to be displayed next. ) Is calculated, and the coordinate position of the estimation point N (illustrated by a broken line in FIG. 7K) is calculated.

As shown in FIGS. 7 (I), (L), and (M), a calibration image CP3 including a point M3 that is not on a straight line connecting the points M1 and M2 generated by the image generation unit 211 of the PC 2 (FIG. 7). (I) is displayed on the projection surface 4 by the image projection means 31 of the projector 3 as the projection image PP3 (see FIG. 7L). Then, as shown in FIG. 7M, when the point M3 displayed by the electronic pen 5 operated by the user is instructed, the imaging unit 32 detects the infrared light emitted from the electronic pen 5. Then, the projection image PP3 is captured (see FIG. 7N).
As shown in FIG. 7 (O), the coordinate position determination unit 215 uses the coordinate position of the estimated point N calculated by the estimated position calculation unit 214 (shown by a broken line) and the position detection data of the designated position corresponding to the point M3. The corresponding position (illustrated by a cross) is compared. Then, the coordinate position determination unit 215 refers to the upper limit value of the shift amount stored in the storage unit 22 and inputs a determination result as to whether or not it is larger than the upper limit value of the shift amount to the image generation unit 211.

The execution unit 216 displays the subsequent calibration image by the projector 3 when it is determined that the subsequent calibration image display is unnecessary based on the comparison result (determination result) by the coordinate position determination unit 215. Stop. The execution unit 216 performs the above-described calibration based on the position detection data of each indicated position that has already been detected and the set position data of the point M. For example, the execution unit 216 calculates a coefficient for converting coordinates corresponding to each position detection data into coordinates corresponding to each set position data. On the other hand, when the execution unit 216 determines that the subsequent calibration image needs to be displayed based on the comparison result (determination result) by the coordinate position determination unit 215, the subsequent calibration image by the projector 3 is displayed. Continue to display.
The range setting unit 217 sets a threshold value of the deviation amount based on the numerical value input by the operation unit 20 and stores the threshold value in the storage unit 22.

[IW device calibration execution processing]
Next, the calibration execution processing of the IW device 1 will be described with reference to the flowchart shown in FIG.
In the IW device 1 of the present embodiment, as described above, the PC 2 needs to perform calibration execution processing before executing the interactive function. In the calibration execution process, when a predetermined operation is performed on the operation unit 20 by the user, the control unit 21 of the PC 2 reads and processes the calibration execution program stored in the storage unit 22 to process the IW device. This is performed by using each configuration of 1.
In this calibration execution process, as shown in FIG. 8, first, the image generation unit 211 generates a calibration image CP1 (see FIG. 6A) including the point M1, and the image transmission unit 212 performs the image transmission. CP1 is transmitted (step S1: image generation step).

Next, the image projection means 31 of the projector 3 displays the calibration image CP1 as the input image on the projection surface 4 as the projection image PP1 (see FIG. 6B) (step S2: image generation step).
Then, as shown in FIG. 6C, when the point M1 displayed by the electronic pen 5 operated by the user is instructed, the imaging unit 32 detects infrared light emitted from the electronic pen 5. Then, the projection image PP1 is captured (see FIG. 6D). Then, the imaging unit 32 detects the emission position of the infrared light included in the captured image SP1 of the projection image PP1, and transmits position detection data indicating the emission position to the PC 2 as position detection data of the designated position ( Step S3: designated position detecting step).
The position detection data is stored in the storage unit 22 by the position information acquisition unit 213 as described above.

Next, the image generation unit 211 generates a calibration image CP2 (see FIG. 6E) including the point M2 set at a position different from the point M1, and the image transmission unit 212 transmits the image CP2. (Step S4: Image generation step)
Then, the projector 3 displays the projection image PP2 (see FIG. 6F) corresponding to the calibration image CP2 on the projection surface 4 by the image projection unit 31 (step S5: image generation step).
Next, as shown in FIG. 6G, when the point M2 included in the projection image PP2 is instructed by the electronic pen 5, as described above, the imaging unit 32 detects infrared light and projects the infrared light. A captured image SP2 obtained by capturing the image PP2 is acquired. Then, as in the case described above, the imaging unit 32 transmits position detection data indicating the emission position of infrared light in the captured image SP2 to the PC 2 as position detection data of the indicated position corresponding to the point M2 (step S2). S6: indicated position detecting step).

Next, the estimated position calculation unit 214 reads out the position detection data of the designated position corresponding to the points M1 and M2 from the storage unit 22, and based on these points M1 and M2, the third point to be displayed next is displayed. An estimated position (coordinate position of the estimated point N) of the point M3 (see FIG. 7K) is calculated (step S7: estimated position calculating step).
Then, the image generation unit 211 generates a calibration image CP3 (see FIG. 7I) including the point M3 that is not on the straight line connecting the points M1 and M2 (step S8: image generation process).

Further, the projector 3 causes the image projection means 31 to display the calibration image CP3 generated in step S8 on the projection surface 4 as the projection image PP3 (see FIG. 7L) (step S9: image generation process).
Next, as in the case described above, when the point M3 is instructed by the electronic pen 5 (see FIG. 7M), the imaging unit 32 detects the infrared light and captures the projected image Po. An image SP3 is acquired (see FIG. 7N). Then, the imaging unit 32 transmits the position detection data of the infrared light emission position in the captured image SP3 to the PC 2 as the position detection data of the designated position (step S10: designated position detection step).

Thereafter, the coordinate position determination unit 215 compares the position corresponding to the position detection data of the designated position corresponding to the point M3 with the estimated position of the estimated point N3 calculated in step S9 (see FIG. 7 (O)). Step S11). Then, the coordinate position determination unit 215 determines whether the displacement amount at the time of comparison is greater than or equal to the above-described threshold value or less than the threshold value, whereby the next display order point M (the fourth point at this time) It is determined whether or not a calibration image including the point M4) is necessary (step S12: coordinate position determination step).
Specifically, in step S12, if the amount of deviation between the two is less than the threshold value, it is determined that the point M4 is unnecessary, and the execution unit 216 displays the calibration image including the point M4 by the projector 3. Is stopped, and calibration is executed based on the instructed points M1 to M3 (step S13: control process). After step S13, the calibration execution process ends.

On the other hand, if it is determined in step S12 that the amount of deviation between the two is greater than or equal to the threshold, it is determined that point M4 is necessary, and the process proceeds to step S7. Thereby, based on the already detected indication position, the estimated position calculation unit 214 calculates the expected indication position (position of the estimation point N) of the point M in the next display order. Then, a calibration image including the next display order point M4 is newly generated by the image generation unit 211, and the projector 3 displays the calibration image and detects the designated position. In this manner, steps S7 to S12 are repeated until the deviation amount becomes less than the threshold value.
If it is determined in step S12 that the amount of deviation is less than the threshold, the process proceeds to step S13, and the execution unit 216 is based on the indicated position corresponding to the detected point M. Then, calibration is executed (step S13: control process).
Thus, the calibration execution process ends.

The IW device 1 according to the first embodiment described above has the following effects.
According to the present embodiment, when the displayed points M1 and M2 are respectively indicated by the electronic pen 5, the imaging unit 32 responds to the points M1 and M2 based on the emission position of the infrared light. When the indicated position is detected, the estimated position calculation unit 214 calculates the estimated position of the estimated point N estimated to be instructed to indicate the next display order point M3 based on the detected indicated position. Then, the coordinate position determination unit 215 compares the coordinate position (estimated position) of the estimated point N with the indicated position of the point M3 detected by the imaging unit 32, and the deviation amount is equal to or greater than a predetermined threshold value. It is determined whether or not there is. If the amount of deviation is greater than or equal to a predetermined threshold, a calibration image that further includes a point (point M4) is generated. If the amount of deviation is less than the threshold, display of the calibration image is stopped and already detected. Calibration is executed based on the designated position.

  According to this, when the calibration image CP3 including the third point M3 in the display order is displayed, the indication position corresponding to the point M3 and the estimated point N calculated based on the position of the point M3 are displayed. If the estimated position is within a predetermined threshold range, calibration can be appropriately performed based on the designated positions of the first to third points M1 to M3. For this reason, the image display can be completed up to the display of the calibration image CP3 including the third point M3, and it is not necessary to display the calibration image including the fourth point M4. The number of point indication operations can be reduced. Therefore, the calibration processing time can be shortened compared to the case where the calibration is executed using the above-described five conventional points.

Here, depending on the usage environment, the amount of deviation between the indicated position where the position is detected and the estimated position of the estimated point N to be calculated may not affect the usage situation. That is, there may be no problem even when the upper limit value set as the threshold is large.
Therefore, in the present embodiment, the threshold value can be appropriately set by the range setting unit 217 depending on the use environment or the application. Therefore, for example, even when the deviation amount between the designated position and the estimated position of the estimated point N is large, the calibration process can be executed only by displaying the minimum necessary point M. The number of instruction operations for point M can be reduced.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
The IW device of the present embodiment has the same configuration as that of the IW device 1 described above. However, in the IW device 1, for example, based on the amount of deviation between the indicated position corresponding to the point M3 and the estimated position of the estimated point N. The coordinate position determination unit 215 determines whether or not it is necessary to display a calibration image including the point M4 in the next display order. On the other hand, in the IW device of the present embodiment, the end button is displayed in the calibration image including the point M in the display order after the point M3, and the user can display the calibration image including the next point M. If the end button is instructed by determining that it is unnecessary, the calibration is executed based on the already detected indication position and the estimation position of the estimation point, and the calibration image is not displayed.
In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

FIG. 9 is a block diagram showing the IW device 1A in the present embodiment. In FIG. 9, since the configuration of the projector 3 is the same as that of the first embodiment, a detailed configuration diagram of the projector 3 is omitted. FIG. 10 is a diagram illustrating a state in which the end button B is displayed on the projection surface 4 in a state where the three points M1 to M3 are displayed.
As shown in FIG. 9, the IW device 1 </ b> A according to the present embodiment has the same configuration as the above-described IW device 1 except that the PC 2 </ b> A is provided instead of the PC 2. The PC 2A has the same configuration as the PC 2 except that the end button display unit 218 is newly functionally added to the configuration of the control means 21.
When a calibration image including the above-described points M3 to M9 is generated by the image generation unit 211, the end button display unit 218 synthesizes an end button B with each image as shown in FIG.

The image generation unit 211 in this embodiment combines the estimated point N calculated by the estimated position calculation unit 214 with a calibration image including the third and subsequent points M in the display order of the points M. As shown in FIG. 10, the projector 3 that receives the image information of such a calibration image includes a guess point N, a point M (point M <b> 3 in the example of FIG. 10), and a calibration image including an end button B. Is displayed as the projected image Po.
When the user designates the end button B using the electronic pen 5, the designated position is detected by the imaging means 32, and it is detected that the end button B has been designated. Accordingly, the execution unit 216 of the PC 2A executes calibration without generating and displaying a calibration image including the point M4.

FIG. 11 is a flowchart showing calibration execution processing of the IW device 1A of the present embodiment.
In the following description, description of the same processing as in the first embodiment will be omitted.
In the calibration execution process in this embodiment, as shown in FIG. 11, the above-described steps S1 to S6 are sequentially executed.
After step S6, the estimated position calculation unit 214, based on the position detection data of the indicated position corresponding to the points M1 and M2, estimates the indicated position (the estimated point N) of the third point M3 to be displayed next. Coordinate position) is calculated (step S7: estimated position calculation step).
Thereafter, the image generation unit 211 includes a point M3, a display indicating the position of the estimated point N calculated by the estimated position calculation unit 214, and an end button B generated by the end button display unit 218. A calibration image is generated (step S21: image generation step).
Then, the projector 3 receives the image information of the calibration image generated in step S21 and displays the calibration image as the projection image Po (step S22: image generation process).
When this calibration image is displayed, the user compares the position of the point M3 with the display position indicating the estimated point N. When the user determines that the difference between these positions is within the predetermined range, the end button B included in the calibration image is instructed, and when the user is outside the predetermined range, Point M3 is instructed. For this reason, the position of the point M3 and the position of the end button B are set sufficiently apart so that these indicated positions can be clearly distinguished.

  Next, when the projection image is instructed by the electronic pen 5, the imaging unit 32 captures the projection image in response to detection of infrared light from the electronic pen 5, and based on the captured image, the imaging unit 32 responds to the point M 3. The indicated position (the emission position of the infrared light) is detected. Then, position detection data indicating the designated position is transmitted to the PC 2A (step S23: designated position detecting step).

The PC 2A determines whether or not the end button B has been instructed based on the acquired position detection data (step S24). If it is determined that the instruction has been instructed, the execution unit 216 detects the detected point M. The calibration is executed based on the indicated position corresponding to (step S13: control process). Thereby, the calibration execution process ends.
If it is determined that the end button B is not instructed, the same processing as in step S11 of the first embodiment is performed, and the coordinate position determination unit 215 determines that the amount of deviation when compared is equal to or greater than a threshold value. It is determined whether or not it is less than the threshold value (step S12: coordinate position determination step).

In the determination process (coordinate position determination process) in step S12, when it is determined that the deviation amount is equal to or greater than the threshold value, it is determined that the point M4 is necessary, and the process proceeds to step S7. Thereby, based on the already detected indication position, the estimated position calculation unit 214 calculates the expected indication position (position of the estimation point N) of the point M in the next display order (step S7: estimation position). Calculation step). Then, a calibration image including a display M indicating the next display order point M, an estimated point N, and an end button B is newly generated by the image generation unit 211, and the projector 3 displays the calibration image. (S21, S22: image generation process) and the detection of the designated position are performed (S23: designated position detection process). In this manner, steps S7, S21 to S24, S11, and S12 are repeated until the deviation amount becomes less than the threshold value.
On the other hand, if it is determined in step S12 that the amount of deviation is less than the threshold value, the process proceeds to step S13, and the execution unit 216 is based on the indicated position corresponding to the detected point M. Then, calibration is executed (step S13: control process). Thereby, the calibration execution process ends.

According to the IW device 1A of the second embodiment described above, the same effects as the IW device 1 described above can be obtained, and the following effects can be obtained.
In the present embodiment, when the third point M3 is displayed, an end button B is displayed in the display area of the calibration image. According to this, the user can obtain the position of the third point M3 and the estimated position corresponding to the third point when necessary (for example, when the estimated position of the estimated point N is displayed). When the end button B is pressed, the processing can be ended only by displaying the calibration image including the third point, and the user points to the point M. The number of operations can be further reduced. Therefore, the calibration processing time can be further shortened.

[Modification of Embodiment]
The best configuration, method, and the like for carrying out the present invention have been disclosed above, but the present invention is not limited to this.
In each of the above embodiments, the projector is exemplified as the image display means or the image display device, but the present invention is not limited to this. That is, any device including an image output means, an image display means or an image display device, and a position detection means for detecting an indicated position by an electronic pen or the like in an image displayed by the image display means or the image display device. Good. In this case, the image output means is not limited to the above-described PC, but may be other image output means. The image display means or the image display device is not limited to the projector, and various displays can be employed. Further, the position detection means is not limited to the image pickup apparatus having the image pickup device, and may be an apparatus having another configuration as long as the indicated position in the display image can be detected. Similarly, the electronic pen is not limited to the configuration in which the indication position is indicated by the emission of infrared light, but may have another configuration. In addition, a position detection unit may be provided in the image display unit that employs various displays, and the designated position instructed with respect to the image displayed by the image display unit may be detected by the position detection unit.

  In each of the embodiments described above, the control unit 21 of the PC 2 performs image generation of the input image Pi, comparison of coordinate positions, calculation of the coordinate position of the estimated point, and the like. You may perform the process of. That is, the configuration of the control unit 21 of the PC 2 may be included in the control unit 35 of the projector 3. In this case, the projector 3 performs a series of processes described as being executed by the PC 2.

  In each of the above-described embodiments, the projector 3 includes the imaging unit 32. However, as described above, the projector 3 may include an imaging device including the imaging unit. In this case, the imaging device detects an instruction position by the electronic pen 5 from a captured image obtained by imaging an imaging area including the projection area of the image, and transmits position information indicating the instruction position to the PC or the projector. Alternatively, the captured image may be transmitted to a PC or a projector, and the PC or projector may acquire position information indicating the position indicated by the electronic pen 5.

In each of the above embodiments, the point M1 and the point M2 are displayed at different timings. However, the point M1 and the point M2 may be displayed at the same time, and the coordinate positions of the points M1 and M2 may be detected at the same time. Further, the points M1 to M3 may be displayed at the same time, and these coordinate positions may be detected simultaneously.
Moreover, in each said embodiment, although the point M1 and the point M2 were displayed on the diagonal line in the input image Pi, it is not limited to this, It does not need to be on a diagonal line. At this time, it is preferable that the three most recent points M displayed in order are not on the same straight line, and it is preferable that the respective points M are sufficiently separated.

In each of the above-described embodiments, the electronic pen 5 that emits infrared light is used to indicate a point. However, the present invention is not limited to this, and it is sufficient that the indicated position on the projection surface 4 can be clearly indicated. It may have a function of emitting ultrasonic waves. Moreover, although the electronic pen 5 has a shape imitating a pen, the electronic pen 5 may be configured with a long shape such as a pointer or a laser pointer.
Further, when the position detection unit is configured to detect a predetermined object (for example, a human finger), the position detection unit can detect the indicated position by detecting the position of the object. The instruction means exemplified in the electronic pen is not always necessary.

In each of the above embodiments, as the position detection means, the imaging means 32 that can detect infrared light and images the area of the projection surface 4 including the projection area of the projection image Po is described. Not limited to. In other words, the position detection unit of the present invention may detect the position of the detection target with a configuration different from that of the imaging unit 32. For example, infrared light and ultrasonic waves are emitted from the electronic pen 5, and the position detection unit includes an infrared light detection unit that detects infrared light, and an ultrasonic detection unit that detects ultrasonic waves. The indicated position of the electronic pen 5 may be detected based on the detection result of the detection means.
In each of the above embodiments, the electronic pen 5 that emits infrared light is used. However, the present invention is not limited to this, and an electronic pen that emits colored light in the visible light range may be used.

  In each of the above embodiments, the projector 3 including the light modulation device 312 such as a liquid crystal panel is illustrated, but the present invention is not limited to this. That is, as long as it is a light modulation device that forms an image by modulating an incident light beam according to image information, a light modulation device having another configuration may be adopted. For example, the present invention can be applied to a projector using a light modulation device other than liquid crystal, such as a device using a micromirror.

  The present invention can be suitably used for an interactive whiteboard device including an image output unit, an image display unit, and a position detection unit, and an image display device constituting the device.

  DESCRIPTION OF SYMBOLS 1,1A ... Interactive white board apparatus, 2, 2A ... PC (image output means), 3 ... Projector (image display means or image display apparatus), 31 ... Image projection means (image display means), 32 ... Imaging means (position) Detection means), 211... Image generation unit, 214... Estimated position calculation unit, 215... Coordinate position determination unit, 216... Execution unit (control unit), 217.

Claims (7)

Image output means for outputting an image;
Image display means for displaying the image;
An interactive whiteboard device comprising position detection means for detecting a designated position designated in the display area of the image,
Either of the image output means and the image display means,
An image generation unit that generates a plurality of calibration images each including points with different display positions;
Based on the pointed position of each of the points included in the plurality of calibration images already detected by the position detecting unit, the point is indicated according to the point included in the next image that is the calibration image to be displayed next. An estimated position calculation unit for calculating an estimated position estimated to be
A coordinate position determination unit that determines whether or not an indicated position of a point included in the next image detected by the position detection unit is within a predetermined range with respect to the estimated position;
When the coordinate position determination unit determines that the position is not within the predetermined range, the image display means displays the calibration image further next to the next image,
An interactive whiteboard device, comprising: a control unit that stops display of the calibration image after the next calibration image when the coordinate position determination unit determines that the position is within the predetermined range. The interactive whiteboard device according to claim 1, wherein
An end button display unit for displaying an end button when the next image is displayed;
When the position detecting unit detects that the end button has been instructed, the control unit stops the display of the calibration image after the next calibration image. . The interactive whiteboard device according to claim 1 or 2,
An interactive whiteboard device comprising: a range setting unit for setting the predetermined range. Image display means for displaying an image;
An image display device comprising: position detection means for detecting an indicated position designated in the display area of the image,
An image generation unit that generates a plurality of calibration images each including points with different display positions, and outputs them to the image display means;
Based on the pointed position of each point included in the plurality of calibration images already detected by the position detecting means, the direction is indicated according to the point included in the next image which is the calibration image displayed next. An estimated position calculating unit that calculates an estimated position estimated to be performed;
A coordinate position determination unit that determines whether or not an indicated position of a point included in the image detected by the position detection unit is within a predetermined range with respect to the estimated position;
When the coordinate position determination unit determines that the position is not within the predetermined range, the image display means displays the calibration image further next to the next image,
An image display device comprising: a control unit that stops displaying the calibration image after the next calibration image when the coordinate position determination unit determines that the position is within the predetermined range. The image display device according to claim 4,
An end button display unit for displaying an end button when the next image is displayed;
The control unit stops display of the calibration image after the next calibration image when the position detecting unit detects that the end button has been instructed. In the image display device according to claim 4 or 5,
An image display device comprising: a range setting unit that sets the predetermined range. A calibration method executed by the image display device according to any one of claims 4 to 6,
An image generation step of generating a plurality of calibration images each including points with different display positions, and outputting them to the image display means;
An indicated position detecting step of detecting an indicated position indicated in the display area of the calibration image;
Based on the indicated position of each point included in the plurality of calibration images already detected by the indicated position detecting step, according to the point included in the next image which is the calibration image displayed next An estimated position calculating step for calculating an estimated position estimated to be instructed;
A coordinate position determination step for determining whether or not an indicated position of a point included in the next image detected by the indicated position detection step is within a predetermined range with respect to the estimated position;
When it is determined by the coordinate position determination step that the position is not within the predetermined range, the image display means displays the calibration image further next to the next image,
A calibration method comprising: a control step of stopping display of the calibration image after the next calibration image when the coordinate position determination step determines that the position is within the predetermined range.

Images