JP5157973B2 - Projection input / output system and program thereof - Google Patents

Description

  The present invention relates to a projection type input / output system including a screen, an electronic pen, a computer apparatus, and a projector, and in particular, executes a process of aligning (calibrating) the position coordinates of the screen and the position coordinates of the display screen of the computer apparatus. Regarding processing.

  In recent years, electronic pens that digitize written information have been developed, and “Anoto pen” developed by Swedish company Anoto is known as a representative example. Anotopen is used together with special paper on which a dot pattern indicating a position coordinate patterned by a predetermined algorithm is printed. Anotopen has a small camera for imaging the dot pattern printed on the dedicated paper at the pen tip, a processor for calculating the position coordinates on the dedicated paper from the captured dot pattern, and the calculated position coordinates etc. It is equipped with a data communication unit that transmits to. When the user writes characters on the special paper with an anotopen or puts a check mark on the image designed on the special paper, the small camera captures the dot pattern printed on the special paper as the pen moves. The entry information such as characters and images written by the user is recognized from the continuous position coordinates calculated by the processor. Then, this entry information is transmitted from the Anotopen by the data communication unit to a nearby terminal device such as a personal computer or a mobile phone (for example, see Patent Document 1).

  In addition, using such an electronic pen, a computer, a projector, and a screen, when the screen is filled in or scanned with the electronic pen, the computer performs a predetermined process accordingly and projects the computer display screen onto the screen with the projector. The system which performs is proposed (for example, patent documents 2-5). The electronic pen reads the dot pattern formed on the screen, and the computer analyzes the position coordinates based on the information transmitted from the electronic pen and performs processing according to the position coordinates. Even if an electronic pen without ink is used, if you trace the screen with an electronic pen, you can project an image with lines drawn from the projector, control the image, and operate software. It is supposed to be possible.

  Also, the computer screen is projected on the whiteboard by the projector, and the coordinates entered with the markers on the whiteboard are read with the coordinate reader using ultrasonic or infrared rays and sent to the computer, so that the user clicks on the computer screen. It is also proposed to calibrate the coordinates of the whiteboard and the coordinates of the personal computer screen by first projecting a calibration point with a projector and pointing the point with a marker in a system that can do this (patent) Reference 6).

Japanese translation of PCT publication No. 2003-511761 JP 2008-152755 A JP 2002-149331 A JP 2008-155421 A Special table 2003-508831 gazette JP 2007-233999 A

  However, even if the alignment process is executed once, the relative positional relationship between the projector and the screen is caused by excessive pressing of the electronic pen on the screen or displacement of the table on which the projector is arranged. It may shift while the system is in use.

  Therefore, the present invention provides a projection type input / output system capable of performing, as necessary, a position adjustment (calibration) process between a screen position coordinate and a position coordinate on a display screen in a display unit of a computer device, and a program thereof The purpose is to provide.

A projection type input / output system according to the present invention receives a screen on which a coding pattern indicating position coordinates is formed, an electronic pen for reading the coding pattern, and information on the coding pattern transmitted from the electronic pen. A projection type input / output system comprising: a computer device that performs processing according to the information; and a projector that receives an image signal from the computer device and projects an image on the screen,
When the computer device receives information on the coding pattern in an area outside the range defined by the alignment process between the position coordinates on the screen and the position coordinates on the display screen in the display means of the computer device, The positioning process is started.

  According to the projection type input / output system, when the coded pattern formed on the screen is read by the electronic pen, the electronic pen transmits information on the coded pattern to the computer device. The computer device transmits an image signal to a projector that projects an image on a screen, and receives information related to a coding pattern from an electronic pen. In such a projection-type input / output system, when the computer apparatus receives information on an area outside the range defined by the alignment process from the electronic pen, the relative positional relationship between the screen and the projector is shifted. Since there is a possibility, alignment can be performed by starting and executing the alignment process.

  In the projection type input / output system, the computer device may be configured to allow selection of whether to perform the process before the alignment process is started. With this configuration, the user can select to perform the alignment process when necessary, and not to perform the process when unnecessary.

  Further, in the projection input / output system, the computer device receives information on a coding pattern in an area outside the range of the screen defined by the alignment process, and after the predetermined time has elapsed, It may be configured to start the matching process. With this configuration, even if the relative positional relationship between the screen and the projector is considerably shifted and the user cannot select the alignment process, the alignment process can be started.

Further, the program according to the present invention receives a screen on which a coding pattern indicating position coordinates is formed, an electronic pen that reads the coding pattern, and information on the coding pattern transmitted from the electronic pen, A program executed by the computer apparatus in a projection type input / output system comprising: a computer apparatus that performs processing according to the information; and a projector that receives an image signal from the computer apparatus and projects an image on the screen. There,
In the state in which the position adjustment processing between the position coordinates on the screen and the position coordinates on the display screen of the display means of the computer apparatus is executed on the computer device, the position is specified by the alignment processing from the electronic pen. When the information about the coding pattern in the area outside the range of the screen is received, the alignment process is started.

  In the above program, the computer apparatus may be configured to allow selection of whether to perform the processing before the positioning processing is started.

  Further, in the above program, the computer apparatus receives information related to a coding pattern in an area outside the range of the screen defined by the alignment process, and performs the alignment process after a predetermined time has elapsed. It can be configured to start.

  By installing the program in the computer device, the computer device can be realized and a projection input / output system can be configured.

  According to the present invention, it is possible to perform alignment (calibration) processing between the position coordinates of the screen and the position coordinates on the display screen of the display unit of the computer device as necessary.

1 is a system configuration diagram of a projection type input / output system according to a first embodiment. It is explanatory drawing which shows the relationship between the arrangement | positioning of the dot in a dot pattern, and the value converted. (A) shows a dot pattern typically, and (b) is a figure showing an example of information corresponding to it. It is the schematic which shows the structure of an electronic pen. It is a functional block diagram of a computer apparatus. It is a flowchart of the calibration starting process in 1st Embodiment. It is an execution selection image of the calibration process projected on the screen. It is a flowchart of a calibration process. It is an image which shows the display reference point projected on the screen. It is a figure which shows the position coordinate of the display reference point on the display screen of a computer apparatus. It is a figure which shows the position coordinate of the recognition reference point on a screen. It is a figure which shows the stroke drawing in the screen after a calibration process. It is a system block diagram of the projection type input / output system in 2nd Embodiment. It is a flowchart of the calibration starting process in 2nd Embodiment.

  Hereinafter, a preferred projection input / output system of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
[System configuration of projection input / output system]
FIG. 1 is a system configuration diagram of a projection type input / output system 10 according to the first embodiment. As shown in FIG. 1, a projection type input / output system 10 according to the present embodiment includes an electronic pen 1 used by a user, a computer device 2 that receives and processes entry information from the electronic pen 1, and a computer device 2. The projector 3 is configured to project a display (display means) 26 onto the screen 4.

[screen]
First, the screen 4 will be described. The screen 4 has a sheet 402 having a dot pattern (coded pattern), which will be described later, formed on the entire surface by printing on the front surface of a magnet board (white board) 401 that is hard enough to apply sufficient pressure to the electronic pen 1 from above. It is pressed with a magnet or pasted with glue or adhesive. A wall surface of a room may be used instead of the magnet plate 401. Further, instead of the paper 402, a sheet on which a dot pattern is printed may be used, or the dot pattern may be directly formed on the whiteboard 40. When the user writes a character or the like on the screen 4 with the electronic pen 1, the electronic pen 1 reads the dot pattern printed on the paper 402 locally and continuously along the movement locus of the pen tip portion. The coordinates of the local position are calculated, and the electronic data (entry information) is transmitted to the computer device 2. On the other hand, the projector 3 receives the image signal from the computer apparatus 2 and projects the same image as the image displayed on the display unit 26 onto the area 403 of the screen 4.

  An input setting area (positioning (calibration) instruction area) 404 is formed on the sheet 402 of the screen 4. The input setting area 404 aligns the position coordinates defined by the dot pattern (coded pattern) on the paper 402 of the screen 4 and the position coordinates on the display screen (display) in the display means 26 of the computer apparatus 2 ( This is an area for causing the computer apparatus 2 to execute (calibration) processing. Of course, a dot pattern is also printed in the input setting area 404. The input setting area 404 is provided outside the area 403 where an image is projected onto the screen 4 by the projector 3 so as not to be mistaken for the entry (coordinate input) on the display screen of the computer apparatus 2. . Then, the user can arbitrarily start the alignment (calibration) process by instructing the input setting area 404 of the screen 4 with the electronic pen 1. When the calibration process is executed in the computer apparatus 2, a coordinate input area 403 in which position coordinates corresponding to the display screen of the computer apparatus 2 can be accurately input on the screen 4 by the electronic pen 1 is defined. Definition information is stored in the storage means 25.

[Dot pattern]
Next, an Anoto dot pattern (coded pattern) printed on the sheet 402 of the screen 4 will be described. FIG. 2 is a diagram illustrating the relationship between the dots of the dot pattern printed on the paper 402 and the values to which the dots are converted. As shown in FIG. 2, each dot of the dot pattern is associated with a predetermined value depending on its position. In other words, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the grid (intersection of the vertical line and horizontal line). The value of each dot can be further converted into a first bit value for the X coordinate and a second bit value for the Y coordinate. The position coordinates on the screen 4 are determined by the combination of information thus associated. This dot pattern is printed with an ink containing carbon that absorbs infrared rays.

  FIG. 3A shows an arrangement of dot patterns at a certain position. As shown in FIG. 3A, 6 × 6 dots are arranged within a range of about 2 mm in length and width so that a unique pattern can be obtained regardless of where 6 × 6 dots are taken on the screen 4. Yes. The dot pattern formed by these 36 dots holds relative position coordinates on the paper 402 and a dot pattern address (coded pattern address). FIG. 3B is a diagram in which each dot shown in FIG. 3A is converted into a value associated with the regularity shown in FIG. 3 according to the shift direction from the reference position of the lattice. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

[Electronic pen]
Next, the electronic pen 1 will be described. FIG. 4 is a schematic diagram showing the structure of the electronic pen 1. As shown in FIG. 4, the electronic pen 1 includes a pen unit 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a processor 108 including a CPU, a memory 109 such as a ROM and a RAM, A communication unit 111 including a real time clock 110, an antenna, and the like, and a battery 112 are provided. The tip of the pen unit 104 is a pen point unit 103, and the user touches the pen point unit 103 of the electronic pen 1 against the paper 402 of the screen 4 to enter characters or tap (pen point unit). 103). Since the electronic pen 1 is used for tapping or writing characters on the screen 4 projected by the projector 3, the pen unit 104 does not have to be filled with ink.

  The battery 112 is for supplying electric power to each component in the electronic pen 1. For example, the battery 112 is configured to turn on / off the electronic pen 1 itself by attaching and detaching a cap (not shown) of the electronic pen 1. May be. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects the pressure applied through the pen unit 104 from the pen tip unit 103 when the user writes or taps a character or mark on the screen 4 with the electronic pen 1, that is, the writing pressure, and the value is detected by the processor. To 108.

  The processor 108 switches on / off the switches of the LED 105 and the CMOS camera 106 based on the writing pressure data supplied from the pressure sensor 107. That is, when a user writes a character or the like on the screen 4 with the electronic pen 1, the pen pressure is applied to the pen tip portion 103, and when the pressure sensor 107 detects a writing pressure higher than a predetermined value, the processor 108 It is determined that entry has started, and the LED 105 and the CMOS camera 106 are operated.

  The LED 105 and the CMOS camera 106 are attached near the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 illuminates infrared rays toward the vicinity of the pen tip portion 103 on the screen 4. The region is slightly shifted from the position where the pen tip portion 103 contacts the screen 4. The CMOS camera 106 captures a dot pattern in the area illuminated by the LED 105 and supplies image data of the dot pattern to the processor 108. Here, since carbon absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed by the carbon contained in the dots. Therefore, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. Therefore, by photographing with the CMOS camera 106, by setting a threshold value based on the difference in the amount of reflected infrared light, it is possible to distinguish the dot region containing carbon from the other regions. Note that the shooting area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 3A, and the shooting by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called.

  The processor 108 performs X and Y coordinates (simply “position coordinates”, “position coordinates” on the screen 4 of the stroke (handwriting) to be entered by the user from the dot pattern of the image data supplied by the CMOS camera 106 during the user entry. (Also called “coordinate data”) and dot pattern addresses (coded pattern addresses) unique to the screen 4 are continuously calculated. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 3 (a) supplied by the CMOS camera 106 into the data array shown in FIG. 3 (b). The data is converted into a Y coordinate bit value, and X, Y coordinate data and a dot pattern address are calculated from the data array by a predetermined calculation method. Then, the processor 108 associates the current time (time stamp: entered time information) transmitted from the real-time clock 110, the writing pressure data, the dot pattern address, and the X and Y coordinate data. Since the 6 × 6 dot pattern on the screen 4 does not overlap in the screen 4, when the user enters characters or the like with the electronic pen 1, it indicates which position on the screen 4 the entered position corresponds to. It can be specified by coordinate calculation by the processor 108.

  The memory 109 stores property information such as a pen ID such as “pen01” for identifying the electronic pen 1, a pen manufacturer number, and a pen software version. Then, the communication unit 111 associates a pen ID (electronic pen identification information), time information (time stamp), writing pressure data, dot pattern address, and X, Y coordinate data, and enters computer information as entry information. 2 to send. Transmission to the computer apparatus 2 by the communication unit 111 is performed immediately and sequentially by wireless transmission of Bluetooth (registered trademark).

[Computer device]
Next, the computer apparatus 2 will be described. The computer device 2 includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as ROM and RAM, a display, a personal computer including a mouse and a keyboard, and the like. . FIG. 5 is a functional block diagram of the computer apparatus 2. Functionally, the computer apparatus 2 includes an input unit 21 such as a mouse and a keyboard, a reception unit 22, a processing unit 24, a storage unit 25, a display unit 26, and a transmission unit 27. Then, the computer device 2 performs a predetermined process based on the entry information received from the electronic pen 1.

  The receiving unit 22 includes an antenna, a receiving circuit, and the like, and receives entry information from the electronic pen 1 and transmits it to the processing unit 24. The transmission unit 27 transmits an image signal used for displaying an image on the display unit 26 to the projector 3 according to an instruction from the processing unit 24. A data transmission method to the projector 3 is a wireless method even if a wired method is used. There may be. The display means 26 is constituted by a display or the like, and displays the contents instructed by the processing means 24.

  The storage means 25 is configured by a memory such as a hard disk, ROM, or RAM. In the storage unit 25, there is a program for performing alignment processing between the position coordinates based on the dot pattern (coded pattern) on the screen 4 and the position coordinates on the display screen in the display unit 26 of the computer apparatus 2. It is remembered. With respect to this program, the storage unit 25 stores the coordinate range of the input setting area (positioning instruction area) 404 on the screen 4, the display coordinate range of the window 5 displayed during the calibration process, the display reference point, and the recognition reference point. The position coordinates such as are stored. The storage unit 25 also stores a coordinate input area 403 that is defined as the calibration process being executed by the processing unit 24. Further, the storage means 25 also stores a program for processing entry information generated by handwriting on the screen 4 with the electronic pen 1.

  The processing means 24 is constituted by a processor such as a CPU, and based on the entry information received from the electronic pen 1 by the receiving means 22, displays the entry contents in various forms on the display means 26, or causes the storage means 25 to display it. A stored program for performing calibration processing is activated to perform calibration processing. In particular, when the position coordinate corresponding to the dot pattern formed in the input setting area (positioning (calibration) instruction area) 404 of the screen 4 is received from the electronic pen 1, the processing unit 24 performs calibration. -Start the program and start the calibration process. Then, the processing means 24 defines a coordinate input area 403 in which the position coordinates corresponding to the display screen of the computer apparatus 2 can be accurately input on the screen 4 by the electronic pen 1 by executing the calibration process. The processing unit 24 causes the transmission unit 27 to transmit the image signal for causing the projector 3 to project the same image as the image displayed on the display unit 26 in synchronization with the projector 3.

  The projector 3 projects an image similar to the image displayed on the display unit 26 on the area 403 of the screen 4 based on the image signal received from the transmission unit 27 of the computer apparatus 2.

<Calibration startup process flow>
Next, calibration processing in the projection input / output system 10 will be described with reference to FIGS.

  FIG. 6 is a flowchart of the calibration activation process. First, the user performs writing or tapping on the paper 402 of the screen 4 using the electronic pen 1. Then, the electronic pen 1 detects contact with the paper 402 when the pressure sensor 107 detects a writing pressure equal to or higher than a predetermined value, and images a dot pattern along the handwriting with the camera 106 while irradiating infrared rays with the LED 105. Then, the processor 108 calculates position coordinates corresponding to the handwriting from the image data of the captured dot pattern, the position coordinates, the current time transmitted by the real-time clock 110, and writing pressure data (hereinafter, “stroke information”). Then, it is immediately and sequentially transmitted to the computer apparatus 2 as entry information in association with the pen ID (step S101).

  In the computer apparatus 2, when the receiving means 22 receives the entry information (stroke information and pen ID) from the electronic pen 1, the entry information is stored in the storage means 25 (step S201). The processing means 24 first checks whether the coordinate input area 403 on the screen 4 is defined (step S202). When the processing unit 24 recognizes that the coordinate input area 403 is not defined (step S202: No), the processing unit 24 starts the calibration process, assuming that the calibration process is not executed (step S207). Therefore, if the user tries to write on the screen 4 with the electronic pen 1 when the calibration process is not executed, the calibration process is started.

  On the other hand, when the processing means 24 recognizes that the coordinate input area 403 is defined (step S202: Yes), the input setting area (positioning instruction area) 404 is electronically determined from the entry information transmitted from the electronic pen 1. It is determined whether or not an instruction has been given by the pen 1 (step S203). When the processing unit 24 recognizes that the input setting area 404 has been instructed (step S203: Yes), the processing unit 24 starts the calibration process (step S207). Thus, even after the calibration process is executed once, even if the relative positional relationship between the screen 4 and the projector 3 has changed, the user sets the input setting area 404 on the screen 4 to the electronic setting. The calibration process can be started by instructing with the pen 1.

  In step S203, when the processing unit 24 recognizes that the input setting area 404 is not instructed (step S203: No), the position coordinates of the stroke information included in the received and acquired entry information are obtained by the calibration process. It is determined whether or not the position coordinates are outside the defined range of the coordinate input area 403 (step S204). When the processing unit 24 determines that the received and acquired position coordinates are out of the range of the coordinate input area 403 (step S204: Yes), an image as shown in FIG. The image is projected on the screen 4 and the user is allowed to select whether or not to start the calibration process (step S206). That is, as shown in FIG. 7, the processing unit 24 projects a window 5 for allowing the user to select whether or not to start the calibration process. Here, the processing unit 24 projects an “OK” button 501 for starting the calibration process and a “cancel” button 502 for not starting the calibration process in the window 5. Thereby, when there is a necessity for the calibration process, the process can be performed, and when it is not necessary, the user can select not to perform the process.

  The user accidentally points out of the range of the coordinate input area 403 after the calibration process with the electronic pen 1, but the relative positional relationship between the projector 3 and the screen 4 changes after the calibration process. If not, the calibration process need not be executed. In that case, the user taps the “cancel” button 502 projected on the screen 4 with the electronic pen 1. When the processing means 24 receives the entry information including the position coordinates corresponding to the dot pattern in the coordinate area on which the “cancel” button 502 is projected, the “cancel” button 502 stored in the storage means 25 will be described later. Since it is recognized that the instruction is within the coordinate area to be projected, it is recognized that the calibration process is selected not to start (step S206: No), and the process ends.

  On the other hand, in step S204, after the calibration process, if the projector 3 or the screen 4 is moved and the relative positional relationship between the projector 3 and the screen 4 is slightly shifted, “OK” projected on the screen 4 is displayed. ”Button 501 is tapped with electronic pen 1 by the user. In this case, there is a slight deviation between the coordinate area where the “OK” button 501 is projected and the coordinate area where the “OK” button 501 stored in the storage unit 25 is projected. When the processing unit 24 receives the entry information including the position coordinates corresponding to the dot pattern in the area where the “OK” button 501 is to be projected, the processing unit 24 recognizes that the selection is made to start the calibration process. Then (step S206: Yes), the calibration process is started (step S207).

  However, after the calibration process, if the projector 3 or the screen 4 is moved to a considerable extent and the relative positional relationship between the projector 3 and the screen 4 is considerably shifted, an “OK” button 501 is projected. A considerable shift occurs between the area and the area where the “OK” button 501 stored in the storage means 25 is to be projected. In this case, even if the user taps the screen 4 on which the “OK” button 501 is projected with the electronic pen 1, the processing unit 24 cannot recognize that the “OK” button 501 is selected. Therefore, when the processing unit 24 does not recognize the instruction with the electronic pen 1 to the “cancel” button 502, the processing unit 24 starts the calibration process after a predetermined time (for example, 5 seconds) has elapsed (step S206: Yes → Step S207). Here, the processing means 24 displays a countdown display of the number of seconds in the window 5 in addition to the display “Calibration will be reset in 5 seconds” as shown in FIG. 7, for example. Thereby, even if the relative positional relationship between the projector 3 and the screen 4 is considerably deviated and the user cannot select to execute the calibration process, the calibration process can be started.

  Note that the coordinate area where the “OK” button 501 and the “cancel” button 502 should be projected may be shifted after the calibration process. Accordingly, the processing unit 24 calculates the coordinate area where the “OK” button 501 and the “cancel” button 502 are to be projected each time the calibration process is executed in step S207, and updates and stores it in the storage unit 25. Let

  The “OK” button 501 and the “Cancel” button 502 can be selected by a mouse operation or a keyboard operation on the screen display of the display unit 26 of the computer device 2 without using the electronic pen 1. Good. In this case, the processing unit 24 recognizes the selection of the “OK” button 501 and the “Cancel” button 502 by operating the mouse or keyboard of the input unit 21, and determines whether to start the calibration process in step S207. I do.

  As described above, there is a possibility that the relative positional relationship between the screen 4 and the projector 3 may change after the calibration process. Therefore, the processing unit 24 may change the screen 4 defined by the calibration process from the electronic pen 1. A window for selecting the start of the calibration process as shown in FIG. 7 when the entry information outside the range of the coordinate input area 403 in which accurate coordinate input is possible is received (step S204: Yes). By projecting 5 onto the screen 4 from the projector 3 and allowing the user to select it, the calibration process can be executed as necessary.

<Calibration process>
Next, the calibration process in step S207 of FIG. 6 will be described with reference to FIGS.

  FIG. 8 is a flowchart of the calibration process. FIG. 9 is an image showing display reference points projected on the screen 4. When the processing unit 24 starts the calibration process, the processing unit 24 displays the first display reference point 601 on the display screen 201 of the display unit 26 and, as shown in FIG. One display reference point 601 is projected, and the user is prompted to tap to the position where the first display reference point 601 is projected by the electronic pen 1 (step S401). Subsequently, when the user taps the position where the first display reference point 601 is projected (the center of the concentric circle) using the electronic pen 1, the electronic pen 1 reads the dot pattern at the tap position and generates stroke information. The entry information obtained in association with the pen ID is transmitted to the computer apparatus 2 (step S301). In the computer apparatus 2, when the entry information is received from the electronic pen 1 by the receiving means 22, the processing means 24 instructs the position coordinates of the pen-up of the entry information to the position where the first display reference point 601 is projected. The recognized reference point 1 on the sheet 402 is stored in the storage unit 25 (step S402).

  Further, the processing unit 24 displays the second display reference point 602 on the display screen 201 of the display unit 26 and causes the projector 3 to project the second display reference point 602 on the screen 4 (FIG. 9). (See), the user is prompted to tap to the position where the second display reference point 602 is projected by the electronic pen 1 (step S403). Subsequently, when the user taps the position where the second display reference point 602 is projected (the center of the concentric circle) using the electronic pen 1, the electronic pen 1 reads the dot pattern at the tap position and generates stroke information. The entry information obtained in association with the pen ID is transmitted to the computer apparatus 2 (step S302). In the computer apparatus 2, when the entry information is received from the electronic pen 1 by the receiving means 22, the processing means 24 instructs the position coordinates of the pen-up of the entry information to the position where the second display reference point 602 is projected. The recognized reference point 2 on the sheet 402 is stored in the storage unit 25 (step S404).

  The processing unit 24 then uses the paper 402 based on the first and second display reference points on the paper 402 by the dot pattern calculation and the coordinate values of the recognition reference points 1 and 2 on the display screen 201 of the display unit 26. From the upper position coordinates, an arithmetic expression for calculating the position coordinates on the display screen 201 in the computer apparatus 2 is obtained and stored in the storage means 25. At the same time, as a result of the calibration process, the processing means 24 determines a coordinate input area 403 on which the position coordinates can be accurately input on the screen 4 by the electronic pen 1 corresponding to the display screen 201, and defines the coordinate area. And it memorize | stores in the memory | storage means 25 (step S405).

  Here, a method of obtaining the calculation formula for calibration will be described with reference to FIGS.

FIG. 10 is a diagram showing the position coordinates of the first and second display reference points on the display screen 201 of the computer apparatus 2. As shown in FIG. 10, on the display screen 201 of the display means (display) 26 of the computer apparatus 2, the X axis is the right direction, the Y axis is the downward direction, and the following coordinates are as follows.
Upper left point: Coordinate 202 (Xamin, Yamin),
Lower right corner point: coordinates 203 (Xamax, Yamax),
First display reference point: coordinates 204 (Xa1, Ya1),
Second display reference point: coordinates 205 (Xa2, Ya2),
However, Xamin ≤ Xa1 <Xa2 ≤ Xamax,
Yamin ≤ Ya1 <Ya2 ≤ Yamax,

FIG. 11 is a diagram illustrating the position coordinates of the recognition reference points 1 and 2 on the sheet 402 of the screen 4. The recognition reference points 1 and 2 are based on the analysis of a dot pattern (coded pattern) designated by the electronic pen 1 by projecting the first and second display reference points on the display screen 201 of the computer apparatus 2 onto the screen 4. Position coordinates. As shown in FIG. 11, in the coordinate input area 403 projected onto the screen 4 by the projector 3, the X axis is set to the right direction, the Y axis is set to the downward direction, and the following coordinates are set as follows.
Upper left point: coordinate 405 (Xb0, Yb0),
Upper left corner point of coordinate input area 403: coordinates 406 (Xbmin, Ybmin),
Lower right corner point of coordinate input area 403: coordinates 407 (Xbmax, Ybmax),
Recognition reference point 1: coordinate 408 (Xb1, Yb1),
Recognition reference point 2: coordinates 409 (Xb2, Yb2),
However, Xb0 ≦ Xbmin <Xb1 <Xb2 ≦ Xbmax,
Yb0 ≤ Ybmin <Yb1 <Yb2 ≤ Ybmax,

Then, the coordinates 204 (Xa1, Ya1) of the first display reference point and the coordinates 205 (Xa2, Ya2) of the second display reference point in the display area 201 of the computer apparatus 2 are recognized in the dot pattern on the paper 402 of the screen 4. The coordinates correspond to the coordinates 408 (Xb1, Yb1) of the reference point 1 and the coordinates 409 (Xb2, Yb2) of the recognition reference point 2, respectively. Then, the upper left corner point coordinates 406 (Xbmin, Ybmin) and the lower right corner point coordinates 407 (Xbmax, Ybmax) (each vertical angle point of the rectangular coordinate input area 403) defining the coordinate input area 403 projected on the screen 4 are as follows. It can be expressed by the following formula.

[Formula 1] Xbmin = Xb1− (Xa1−Xamin) × (Xb2−Xb1) / (Xa2−Xa1)
[Formula 2] Ybmin = Yb1− (Ya1−Yamin) × (Yb2−Yb1) / (Ya2−Ya1)

[Formula 3] Xbmax = Xb2 + (Xamax−Xa2) × (Xb2−Xb1) / (Xa2−Xa1)
[Formula 4] Ybmax = Yb2 + (Yamax−Ya2) × (Yb2−Yb1) / (Ya2−Ya1)

A coordinate input area 403 in step S204 can be defined from a rectangular area having the coordinate value as the vertical angle point.

Then, from the position coordinates (Xbn, Ybn) defined by the dot pattern on the paper 402 of the screen 4, the position coordinates (Xan, Yan) on the display screen 201 in the display means 26 of the computer device 2 are expressed by the following equations. Can be sought.

[Formula 5] Xan = Xamin + [Xbn− (Xbmin−Xb0)] × (Xa2−Xa1) / (Xb2−Xb1)
[Formula 6] Yan = Yamin + [Ybn− (Ybmin−Yb0)] × (Ya2−Ya1) / (Yb2−Yb1)

Therefore, the processing means 24 converts the position coordinates (Xbn, Ybn) on the screen 4 to the position coordinates (Xan, Yan) on the display screen 201 of the computer device 2 according to the above formula, and performs various processes. Go. Note that a predetermined coefficient for correcting distortion may be applied to the term of the arithmetic expression depending on the angle of the projector 3 with respect to the screen 4.

  After the calibration process, the user writes characters or the like using the electronic pen 1 in the coordinate input area 403 that is defined on the screen 4 so that the position coordinates can be accurately input corresponding to the display screen 201. Then, the electronic pen 1 generates stroke information corresponding to the stroke of the handwriting, and transmits entry information associated with the pen ID to the computer device 2 (step S101). In the computer apparatus 2, when the entry information is received by the receiving means 22, the processing means 24 stores the entry information in the storage means 25 (step S201), and the position coordinates can be accurately input corresponding to the display screen 201. Although it is determined that the coordinate input area 403 is defined (step S202: Yes), it is determined that the received entry information is not information in the input setting area (positioning instruction area) 401 (step S203: No). . Further, since the processing means 24 determines that the acquired entry information is not outside the coordinate input area 403 (step S204: No), the position coordinates (Xbn, Ybn) included in the stroke information of the entry information are calibrated. The calculation formula obtained by the processing is converted into the position coordinates (Xan, Yan) on the display screen 201 of the computer apparatus 2 to draw a stroke and perform various processes.

  That is, as shown in FIG. 12, when the user enters a character or the like with the electronic pen 1 in the coordinate input area 403 of the screen 4, the computer device 2 draws a stroke line segment corresponding to the handwriting, and the screen 4 The line segment of the stroke is projected by the projector 3 without deviating from the above handwriting. Further, since the position coordinates on the screen 4 and the position coordinates on the display screen 201 in the display unit 26 of the computer apparatus 2 are aligned by the calibration process by the coordinate conversion by the arithmetic expression, the coordinate input area of the screen 4 By selecting a menu or icon projected in 403 with the electronic pen 1, the application can be operated or started.

<Operational effect of the first embodiment>
In the projection type input / output system 10 according to the first embodiment, since an input setting area (positioning instruction area) 404 is provided on the screen 4, the user can input an input setting area (positioning instruction) on the screen 4. By instructing the (area) 404 with the electronic pen 1, it is possible to arbitrarily start the alignment (calibration) process. Since this input setting area (positioning instruction area) 404 is provided outside the area 403 projected on the screen 4 by the projector 3, entry (coordinate input) in the display screen 201 of the computer apparatus 2 is performed. Do not confuse.

  In step S202, since the calibration process is not executed, the coordinate input area 403 that can be accurately input corresponding to the display screen 201 is not defined on the screen 4 (step S202: No). Even if the user does not notice it and tries to write on the screen 4 with the electronic pen 1, the calibration process is started (step S207). Therefore, it is possible to prevent entry with the electronic pen 1 when the calibration process is not executed.

  Further, when the processing unit 24 recognizes that the input setting area (positioning instruction area) 404 is instructed by the electronic pen 1 even if the coordinate input area 403 defined by the calibration process is defined (Step S24). S203: Yes), the calibration process is started (step S207). Therefore, once the calibration process is executed, even if the relative positional relationship between the screen 4 and the projector 3 is deviated, the user consciously inputs the input setting area (positioning instruction area) of the screen 4. ) 404 can be started with the electronic pen 1 to start the calibration process (step S207).

  Further, when the processing unit 24 receives the entry information outside the range of the coordinate input area 403 defined by the calibration process (step S204: Yes), the relative relationship between the screen 4 and the projector 3 is obtained after the calibration process. Since there is a possibility that the positional relationship has shifted, the calibration process can be started (step S207) by allowing the user to select the execution of the calibration process as necessary (step S206: Yes). At that time, if the relative positional relationship between the projector 3 and the screen 4 is considerably deviated, there is a possibility that the “OK” button 501 shown in FIG. 7 cannot be selected. Even if not, the processing means 24 automatically starts the calibration process after a predetermined time (step S207).

  As described above, the projection input / output system 10 of the present embodiment can perform the calibration process as necessary. After the calibration process, the position coordinates can be corrected by converting the position coordinates using an arithmetic expression.

[Modification]
The first embodiment may be configured as follows.

  In step S204 of the first embodiment, when the processing unit 24 receives entry information outside the range of the coordinate input area 403 defined by the calibration process (step S204: Yes), without passing through step S206, The calibration process may be started immediately (step S207). This is because when the user enters the electronic pen 1 outside the coordinate input area 403, the relative positional relationship between the projector 3 and the screen 4 is likely to be considerably shifted.

  Alternatively, in step S204 of the above-described embodiment, as a reference for determining that the processing unit 24 has received entry information outside the range of the coordinate input area 403 defined by the calibration process, the position coordinates of the received entry information, A distance (a distance obtained by extending a perpendicular from the position coordinate to the side) from the position coordinates of the received entry information among the sides constituting the coordinate input area 403 is obtained, and the distance is a predetermined length (for example, 1 cm) or more. If the distance of the received position coordinate from the coordinate input area 403 is short, the fact that the entry information outside the range of the coordinate input area 403 has been received is not caused by a shift in the relative positional relationship between the projector 3 and the screen 4. This is because the possibility is high.

  Further, the selection process for starting the calibration process shown in FIG. 7 may be performed before starting the calibration process from steps S202 and S203.

Second Embodiment
Next, the projection type input / output system 11 in the second embodiment will be described. In the description of the second embodiment, the configuration different from the first embodiment will be mainly described. In the second embodiment, the same reference numerals as those in the first embodiment are the same as those in the first embodiment unless otherwise specified.

  FIG. 13 is a system configuration diagram of the projection input / output system 11 in the second embodiment. In the projection input / output system 10 of the first embodiment, the input setting area 404 is formed on the sheet 402 of the screen 4, whereas in the projection input / output system 11 of the second embodiment, the input setting is made. The area 704 is formed on the operation sheet 7 on which a dot pattern indicating the position coordinates is printed, and is different in that the operation sheet 7 is a sheet different from the sheet 402 of the screen 4. Other configurations are the same as those of the first embodiment. When the user has the operation sheet 7 at hand, it is easy to tap the input setting area 704 with the electronic pen 1.

<Calibration startup process flow>
Next, with reference to FIG. 14, a calibration process in the projection input / output system 11 will be described.

  As described above, in the second embodiment, the sheet 402 and the operation sheet 7 are different sheets. Therefore, the flow of the calibration activation process in the second embodiment (see FIG. 14) is compared with the flow of the calibration activation process in the first embodiment (see FIG. 6). After storing the stroke information and the pen ID) in the storage unit 25 (step S601), the processing unit 24 determines whether the entry information is written on the sheet 402 of the screen 4 (step S602). When it is determined that the entry information is entered on the sheet 402 of the screen 4 (step S602: Yes), it is determined whether or not the coordinate input area 403 is defined (step S603), and the entry information is the sheet of the screen 4 sheet. When it is determined that the information is not entered in 402 (step S602: No), the entry is made. Distribution determines whether or not entered in the input setting area 704 of the operation seat 7 (step S607). In addition, when it is determined that the entry information has been entered in the input setting area 704 of the operation sheet 7 (step S607: Yes), the processing unit 24 starts a calibration process (step S608), and the entry information Is determined not to be entered in the input setting area 704 of the operation sheet 7 (step S607: No), the process is terminated.

  Other processes are the same as those in the first embodiment, and the processes in steps S603, S604, S605, S606, and S608 shown in FIG. 14 are the same as those in steps S202, S204, S205, S206, and S207 in FIG. This is the same as the process.

<Operational effects of the second embodiment>
The projection input / output system 11 in the second embodiment also has the same effects as the projection input / output system 10 in the first embodiment, and the operation sheet 7 is a sheet different from the sheet 402 of the screen 4. Therefore, the user can easily tap the input setting area 704 with the electronic pen 1 by holding the operation sheet 7 at hand.

[Modification]
Also in the second embodiment, a modified configuration similar to that of the first embodiment can be employed.

  In the first and second embodiments, the calibration process is performed based on the position coordinates of two points. However, the calibration process may be performed based on the position coordinates of three or more points. Moreover, in the said embodiment, although the front type projector was mentioned as an example as the projector 3, it is applicable also to a rear projector. Further, the dot pattern (coded pattern) is not limited to the Anoto method.

  In the first and second embodiments, the electronic pen 1 does not have an ink cartridge. However, it is possible to write characters or the like directly on the paper 402 with the electronic pen 1 using the pen unit 104 as an ink cartridge. You may do it. In this case, in the computer device 2, the stroke drawn with the electronic pen 1 is displayed on the display unit 26, but an image signal that is not projected by the projector 3 may be output from the computer device 2 to the projector 3.

  This projection input / output system can be used in school classes, company conference rooms, and presentations.

DESCRIPTION OF SYMBOLS 1 ... Electronic pen 2 ... Computer apparatus 21 ... Input means 22 ... Reception means 24 ... Processing means 25 ... Memory | storage means 26 ... Display means 27 ... Transmission means 201 ... Display screen 204 ... 1st display reference point 205 ... 2nd display reference point 3 ... Projector 4 ... Screen 401 ... Magnet plate (white board)
402: paper 403 ... image projection area / coordinate input area 404 ... input setting area (positioning (calibration) instruction area)
408 ... Recognition reference point 1
409 ... Recognition reference point 2
5 ... Window 501 ... "OK" button 502 ... "Cancel" button 601 ... Projection position of first display reference point, recognition reference point 1
602: Projection position of second display reference point, recognition reference point 2
10, 11 ... Projection input / output system

Claims (6)

A screen on which a coding pattern indicating position coordinates is formed;
An electronic pen for reading the coded pattern;
A computer device that receives information on the coding pattern transmitted from the electronic pen and performs processing according to the information;
A projection input / output system comprising: a projector that receives an image signal from the computer device and projects an image on the screen;
When the computer device receives information on the coding pattern in an area outside the range defined by the alignment process between the position coordinates on the screen and the position coordinates on the display screen in the display means of the computer device, A projection type input / output system characterized by starting the alignment process.   The projection type input / output system according to claim 1, wherein the computer device can input a selection as to whether or not to perform the processing before starting the alignment processing.   The computer apparatus receives information on a coding pattern in an area outside the range of the screen defined by the alignment process, and starts the alignment process after a predetermined time has elapsed. The projection type input / output system according to claim 2. A screen on which a coding pattern indicating position coordinates is formed, an electronic pen that reads the coding pattern, and a computer that receives information on the coding pattern transmitted from the electronic pen and performs processing according to the information A program executed by the computer apparatus in a projection type input / output system comprising: an apparatus; and a projector that receives an image signal from the computer apparatus and projects an image on the screen;
In the state in which the position adjustment processing between the position coordinates on the screen and the position coordinates on the display screen of the display means of the computer apparatus is executed on the computer device, the position is specified by the alignment processing from the electronic pen. When the information regarding the coding pattern in the area outside the range of the screen is received, the alignment process is started.   The program according to claim 4, wherein the computer apparatus is configured to allow selection of whether or not to perform the processing before inputting the alignment processing.   The computer apparatus receives information related to a coding pattern in an area outside the range of the screen defined by the alignment process, and starts the alignment process after a predetermined time has elapsed. The program according to claim 5.

Images