JP3151886U - Information processing system - Google Patents

Abstract

Provided is an information processing system that guides the position of a dot pattern to be read by an electronic pen when performing calibration processing in an easy-to-understand manner. An information processing system receives a screen on which a dot pattern indicating position coordinates is formed, an electronic pen that reads the dot, and entry information transmitted from the electronic pen, and stores the information in the information. The computer apparatus which performs the process according to this, and the projector which receives the image signal from the computer apparatus and projects an image on the screen 4 are provided. The computer apparatus has a calibration processing function, displays an image image indicating an image projection area on the screen 4, and guides a position to be tapped with the electronic pen 1. [Selection] Figure 8

Description

  The present invention relates to an information processing system capable of inputting information using a coded pattern indicating position coordinates, and more particularly, to a calibration process between the position coordinates of a coded pattern and the position coordinates of a display screen of a computer device.

  In recent years, electronic pens that digitize written information have been developed, and “Anoto pen” developed by Swedish company Anoto is known as a representative example. Anotopen is used together with special paper on which a dot pattern indicating a position coordinate patterned by a predetermined algorithm is printed. Anotopen has a small camera for imaging the dot pattern printed on the dedicated paper at the pen tip, a processor for calculating the position coordinates on the dedicated paper from the captured dot pattern, and the calculated position coordinates etc. It is equipped with a data communication unit that transmits to. When the user writes characters on the special paper with an anotopen or puts a check mark on the image designed on the special paper, the small camera captures the dot pattern printed on the special paper as the pen moves. The entry information such as characters and images written by the user is recognized from the continuous position coordinates calculated by the processor. Then, this entry information is transmitted from the Anotopen by the data communication unit to a nearby terminal device such as a personal computer or a mobile phone (for example, see Patent Document 1).

  In addition, using such an electronic pen, a computer, a projector, and a screen, when the screen is filled in or scanned with the electronic pen, the computer performs a predetermined process accordingly and projects the computer display screen onto the screen with the projector. The system which performs is proposed (for example, patent documents 2-5). The electronic pen reads the dot pattern formed on the screen, and the computer analyzes the position coordinates based on the information transmitted from the electronic pen and performs processing according to the position coordinates. Even if an electronic pen without ink is used, if you trace the screen with an electronic pen, you can project an image with lines drawn from the projector, control the image, and operate software. It is supposed to be possible.

  Also, the computer screen is projected on the whiteboard by the projector, and the coordinates entered with the markers on the whiteboard are read with the coordinate reader using ultrasonic or infrared rays and sent to the computer, so that the user clicks on the computer screen. In a system capable of calibrating, it is also proposed to calibrate the coordinates of the whiteboard and the coordinates of the personal computer screen by first projecting a calibration point by a projector and instructing the point with a marker (patent) Reference 6).

  In addition, it has been proposed that a transparent sheet on which a dot pattern is printed is pasted on a computer display and handwritten on the transparent sheet with an electronic pen so that the dot pattern can be read with the electronic pen and handwritten on the computer. (For example, Patent Document 7). Furthermore, in order to calibrate the position coordinates on the computer display and the position coordinates on the dot pattern of the transparent sheet attached to the display, it is also proposed to read the dot pattern near the four corners of the display with an electronic pen. (Patent Document 8).

Japanese translation of PCT publication No. 2003-511761 JP 2008-152755 A JP 2002-149331 A JP 2008-155421 A Special table 2003-508831 gazette JP 2007-233999 A JP 2003-256122 A JP 2008-165384 A

  For the calibration process, the operation of causing the electronic pen or the like to read the dot pattern may be performed by the manufacturer at the time of manufacturing the system or by the user after shipment. For this reason, it is necessary to guide in an easy-to-understand manner what position the dot pattern is read by the electronic pen or the like when any person performs the calibration process.

  Therefore, an object of the present invention is to provide a system that guides the position of a coded pattern (dot pattern) to be read by a reading unit such as an electronic pen when performing calibration processing.

An information processing system according to the present invention receives a screen on which a coding pattern indicating position coordinates is formed, a reading unit that reads the coding pattern, and information on the coding pattern transmitted from the reading unit, An information processing system comprising a computer device that performs processing according to the information, and a projector that receives an image signal from the computer device and projects an image on the screen,
The computer device includes:
Storage means for displaying an image image projected on the screen and storing a guide image for guiding a position designated by the reading means;
The guide image is read out and displayed on the display means, and the position coordinates on the screen calculated by the coded pattern of the indicated position read by the reading means are displayed on the display screen in the display means of the computer device. And processing means for obtaining a coordinate conversion function for converting to position coordinates.

  According to the information processing system, when the coded pattern formed on the screen is read by the reading unit, the reading unit transmits information related to the coded pattern to the computer device. The computer device transmits an image signal to a projector that projects an image on a screen, and receives information on a coding pattern from a reading unit. In such an information processing system, an image image projected on a screen as a guide image is read from the storage means by the processing means and displayed on the display means of the computer device. If the user points the screen with the electronic pen in accordance with the guidance image, the electronic pen reads the coded pattern at the designated position and transmits information related to the coded pattern to the computer device. The computer apparatus receives the information, and obtains a coordinate conversion function for converting the position coordinates on the screen into the position coordinates on the display screen in the display means of the computer apparatus by the processing means.

Alternatively, an information processing system according to the present invention receives a transparent sheet on which a coding pattern indicating position coordinates is formed, reading means for reading the coding pattern, and information on the coding pattern transmitted from the reading means. An information processing system including a computer device that performs processing according to the information,
The transparent sheet is affixed to the display screen of the display means of the computer device,
The computer device includes:
Storage means for displaying an image on the display screen and storing a guide image for guiding a position designated by the reading means;
The guide image is read and displayed on the display means, and the position coordinates on the transparent sheet calculated by the coded pattern of the indicated position read by the reading means are displayed on the display screen of the display means of the computer device. And a processing means for obtaining a coordinate conversion function for converting into the position coordinates.

  According to the information processing system, when the coded pattern formed on the transparent sheet is read by the reading unit, the reading unit transmits information on the coded pattern to the computer device. The computer device receives information about the coding pattern from the reading means. In such an information processing system, the image of the display screen is read from the storage means by the processing means and displayed as a guide image on the display means of the computer device. In response to the guidance image, if the user points the display screen on the transparent sheet with the electronic pen, the electronic pen reads the coded pattern at the instructed position and sends information related to the coded pattern to the computer device. Send. The computer apparatus receives the information, and obtains a coordinate conversion function for converting the position coordinates on the transparent sheet into the position coordinates on the display screen in the display means of the computer apparatus by the processing means.

  In the information processing system described above, the processing unit displays the guide image in the center in a range smaller than the display screen, and the pointing position specified by the reading unit is a position corresponding to a corner of the display screen. It is good to do. With this configuration, the guidance image is not displayed at the designated position designated by the reading unit, so that the instruction is not hindered, and the accuracy of the coordinate conversion function is improved by setting the designated position as the corner of the display screen. be able to.

  Furthermore, in the information processing system described above, it is preferable that the indicated position indicated by the reading unit is indicated by an arrow. With this configuration, it is possible to indicate the position indicated by the reading unit more easily.

  According to the present invention, in performing the calibration process, the position of the coded pattern read by the reading means such as an electronic pen can be guided in an easy-to-understand manner.

It is a system configuration figure of the information processing system in a 1st embodiment. It is explanatory drawing which shows the relationship between the arrangement | positioning of the dot in a dot pattern, and the value converted. (A) shows a dot pattern typically, and (b) is a figure showing an example of information corresponding to it. It is the schematic which shows the structure of an electronic pen. It is a functional block diagram of a computer apparatus. It is a flowchart of the calibration starting process in 1st Embodiment. It is a flowchart of a calibration process. It is a figure which shows the state by which the guidance image of the recognition reference point 1 was projected on the screen. It is a figure which shows the state by which the guidance image of the recognition reference point 2 was projected on the screen. It is a figure which shows the position coordinate on the display screen of a computer apparatus. It is a figure which shows the position coordinate of the recognition reference point on a screen. It is a figure which shows the stroke drawing in the screen after a calibration process. It is a figure which shows the modification of the guidance image in 1st Embodiment. It is a flowchart of the calibration starting process in 2nd Embodiment. It is a figure which shows the state by which the guidance image of the recognition reference point 1 was projected on the screen. It is a figure which shows the state by which the guidance image of the recognition reference point 2 was projected on the screen. It is a figure which shows the position coordinate on the display screen of a computer apparatus. It is a figure which shows the position coordinate of the recognition reference point on a screen. It is a system configuration figure of an information processing system in other embodiments. It is a figure which shows the state by which the guidance image was displayed on the display screen of a computer apparatus.

  Hereinafter, an information processing system according to the present invention will be described in detail with reference to the drawings.

<First Embodiment>
[System configuration of information processing system]
FIG. 1 is a system configuration diagram of an information processing system 10 in the first embodiment. As shown in FIG. 1, an information processing system 10 according to the first embodiment includes an electronic pen (reading unit) 1 used by a user, a computer device 2 that receives and processes entry information from the electronic pen 1, The projector 3 is configured to project an image displayed on the display unit 26 of the computer device 2 onto the screen 4.

[screen]
First, the screen 4 will be described. The screen 4 has a sheet 402 having a dot pattern (coded pattern), which will be described later, formed on the entire surface by printing on the front surface of a magnet board (white board) 401 that is hard enough to apply sufficient pressure to the electronic pen 1 from above. It is pressed with a magnet or pasted with glue or adhesive. A wall surface of a room may be used instead of the magnet plate 401. Further, instead of the paper 402, a sheet on which a dot pattern is printed may be used, or the dot pattern may be directly formed on the whiteboard 40. When the user writes a character or the like on the screen 4 with the electronic pen 1, the electronic pen 1 reads the dot pattern printed on the paper 402 locally and continuously along the movement locus of the pen tip portion. The coordinates of the local position are calculated, and the electronic data (entry information) is transmitted to the computer device 2. On the other hand, the projector 3 receives the image signal from the computer apparatus 2 and projects the same image as the image displayed on the display unit 26 onto the image projection area 403 of the screen 4.

[Dot pattern]
Next, an Anoto dot pattern (coded pattern) printed on the sheet 402 of the screen 4 will be described. FIG. 2 is a diagram illustrating the relationship between the dots of the dot pattern printed on the paper 402 and the values to which the dots are converted. As shown in FIG. 2, each dot of the dot pattern is associated with a predetermined value depending on its position. In other words, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the grid (intersection of the vertical line and horizontal line). The value of each dot can be further converted into a first bit value for the X coordinate and a second bit value for the Y coordinate. The position coordinates on the screen 4 are determined by the combination of information thus associated. This dot pattern is printed with an ink containing carbon that absorbs infrared rays.

  FIG. 3A shows an arrangement of dot patterns at a certain position. As shown in FIG. 3A, 6 × 6 dots are arranged within a range of about 2 mm in length and width so that a unique pattern can be obtained regardless of where 6 × 6 dots are taken on the screen 4. Yes. The dot pattern formed by these 36 dots holds relative position coordinates on the paper 402 and a dot pattern address (coded pattern address). FIG. 3B is a diagram in which each dot shown in FIG. 3A is converted into a value associated with the regularity shown in FIG. 3 according to the shift direction from the reference position of the lattice. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

[Electronic pen]
Next, the electronic pen 1 will be described. FIG. 4 is a schematic diagram showing the structure of the electronic pen 1. As shown in FIG. 4, the electronic pen 1 includes a pen unit 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a processor 108 including a CPU, a memory 109 such as a ROM and a RAM, A communication unit 111 including a real-time clock 110, an antenna, and the like, and a battery 112 are provided. The tip of the pen unit 104 is a pen point unit 103, and the user touches the pen point unit 103 of the electronic pen 1 against the paper 402 of the screen 4 to enter characters or tap (pen point unit). 103). Since the electronic pen 1 is used for tapping or writing characters on the screen 4 projected by the projector 3, the pen unit 104 does not have to be filled with ink.

  The battery 112 is for supplying electric power to each component in the electronic pen 1. For example, the battery 112 is configured to turn on / off the electronic pen 1 itself by attaching and detaching a cap (not shown) of the electronic pen 1. May be. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects the pressure applied through the pen unit 104 from the pen tip unit 103 when the user writes or taps a character or mark on the screen 4 with the electronic pen 1, that is, the writing pressure, and the value is detected by the processor. To 108.

  The processor 108 switches on / off the switches of the LED 105 and the CMOS camera 106 based on the writing pressure data supplied from the pressure sensor 107. That is, when a user writes a character or the like on the screen 4 with the electronic pen 1, the pen pressure is applied to the pen tip portion 103, and when the pressure sensor 107 detects a writing pressure higher than a predetermined value, the processor 108 It is determined that entry has started, and the LED 105 and the CMOS camera 106 are operated.

  The LED 105 and the CMOS camera 106 are attached near the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 illuminates infrared rays toward the vicinity of the pen tip portion 103 on the screen 4. The region is slightly shifted from the position where the pen tip portion 103 contacts the screen 4. The CMOS camera 106 captures a dot pattern in the area illuminated by the LED 105 and supplies image data of the dot pattern to the processor 108. Here, since carbon absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed by the carbon contained in the dots. Therefore, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. Therefore, by photographing with the CMOS camera 106, by setting a threshold value based on the difference in the amount of reflected infrared light, it is possible to distinguish the dot region containing carbon from the other regions. Note that the shooting area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 3A, and the shooting by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called.

  The processor 108 performs X and Y coordinates (simply “position coordinates”, “position coordinates” on the screen 4 of the stroke (handwriting) to be entered by the user from the dot pattern of the image data supplied by the CMOS camera 106 during the user entry. (Also called “coordinate data”) and dot pattern addresses (coded pattern addresses) unique to the screen 4 are continuously calculated. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 3 (a) supplied by the CMOS camera 106 into the data array shown in FIG. 3 (b). The data is converted into a Y coordinate bit value, and X, Y coordinate data and a dot pattern address are calculated from the data array by a predetermined calculation method. Then, the processor 108 associates the current time (time stamp: entered time information) transmitted from the real-time clock 110, the writing pressure data, the dot pattern address, and the X and Y coordinate data. Since the 6 × 6 dot pattern on the screen 4 does not overlap in the screen 4, when the user enters characters or the like with the electronic pen 1, it indicates which position on the screen 4 the entered position corresponds to. It can be specified by coordinate calculation by the processor 108.

  The memory 109 stores property information such as a pen ID such as “pen01” for identifying the electronic pen 1, a pen manufacturer number, and a pen software version. Then, the communication unit 111 associates a pen ID (electronic pen identification information), time information (time stamp), writing pressure data, dot pattern address, and X, Y coordinate data, and enters computer information as entry information. 2 to send. Transmission to the computer apparatus 2 by the communication unit 111 is performed immediately and sequentially by wireless transmission such as Bluetooth (registered trademark).

[Computer device]
Next, the computer apparatus 2 will be described. The computer device 2 includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as ROM and RAM, a display, a personal computer including a mouse and a keyboard, and the like. . FIG. 5 is a functional block diagram of the computer apparatus 2. Functionally, the computer apparatus 2 includes an input unit 21 such as a mouse and a keyboard, a reception unit 22, a processing unit 24, a storage unit 25, a display unit 26, and a transmission unit 27. Then, the computer device 2 performs a predetermined process based on the entry information received from the electronic pen 1.

  The receiving unit 22 includes an antenna, a receiving circuit, and the like, and receives entry information from the electronic pen 1 and transmits it to the processing unit 24. The transmission unit 27 transmits an image signal used for displaying an image on the display unit 26 to the projector 3 according to an instruction from the processing unit 24. A data transmission method to the projector 3 is a wireless method even if a wired method is used. There may be. The display means 26 is constituted by a display or the like, and displays the contents instructed by the processing means 24.

  The storage means 25 is configured by a memory such as a hard disk, ROM, or RAM. The storage means 25 includes a program for performing alignment processing between the position coordinates based on the dot pattern (coded pattern) on the screen 4 and the position coordinates on the display screen of the display means 26 of the computer device 2. A guide image is stored. With respect to this program, the storage means 25 stores the display coordinate range of the window 6 displayed on the display means 26 during the calibration process, the position coordinates of the display reference point, the recognition reference point, and the like. Further, the storage means 25 also stores a program for processing entry information generated by handwriting on the screen 4 with the electronic pen 1.

  The processing means 24 is constituted by a processor such as a CPU, and based on the entry information received from the electronic pen 1 by the receiving means 22, displays the entry contents in various forms on the display means 26, or causes the storage means 25 to display it. A stored calibration program is activated to perform calibration processing. The processing unit 24 causes the transmission unit 27 to transmit the image signal for causing the projector 3 to project the same image as the image displayed on the display unit 26 in synchronization with the projector 3.

  The projector 3 projects an image similar to the image displayed on the display unit 26 on the image projection area 403 of the screen 4 based on the image signal received from the transmission unit 27 of the computer apparatus 2.

<Calibration startup process flow>
Next, calibration processing in the information processing system 10 will be described with reference to FIGS.

  FIG. 6 is a flowchart of the calibration activation process. First, the user performs writing or tapping on the paper 402 of the screen 4 using the electronic pen 1. Then, the electronic pen 1 detects contact with the paper 402 when the pressure sensor 107 detects a writing pressure equal to or higher than a predetermined value, and images a dot pattern along the handwriting with the camera 106 while irradiating infrared rays with the LED 105. Then, the processor 108 calculates position coordinates corresponding to the handwriting from the image data of the captured dot pattern, the position coordinates, the current time transmitted by the real-time clock 110, and writing pressure data (hereinafter, “stroke information”). Then, it is immediately and sequentially transmitted to the computer apparatus 2 as entry information in association with the pen ID (step S101).

  In the computer apparatus 2, when the receiving means 22 receives the entry information (stroke information and pen ID) from the electronic pen 1, the entry information is stored in the storage means 25 (step S201). Then, the processing unit 24 checks whether or not a coordinate conversion function for performing the calibration process is defined (step S202). When the processing unit 24 recognizes that the coordinate transformation function is not defined (step S202: No), the processing unit 24 starts the calibration process, assuming that the calibration process is not executed (step S204). Therefore, if the user tries to write on the screen 4 with the electronic pen 1 when the calibration process is not executed, the calibration process is started.

  On the other hand, when the processing unit 24 recognizes that the coordinate conversion function is defined (step S202: Yes), the position coordinates of the received entry information by the dot pattern on the screen 4 are displayed using the coordinate conversion function. The position is converted into the position coordinates on 201, and the stroke is drawn (step S203).

<Calibration process>
Next, the calibration process in step S204 of FIG. 6 will be described with reference to FIGS.

  FIG. 7 is a flowchart of the calibration process. 8 and 9 show the state of the screen 4 on which the image of the display screen 201 is projected during the calibration process. When the calibration process is started, the processing unit 24 reads a guide image for guiding the calibration process from the storage unit 25 and displays it on the window 6 as shown in FIG. Guide the user to tap the upper left corner with the electronic pen (operation guide A1: step S401). Here, an image image simulating the screen 4 and the image projection area 403 is displayed as the guide image, and the tapping position (the upper left corner 601 of the image projection area 403) is guided in an easy-to-understand manner. That is, in the guidance image, a message such as “Please touch the upper left corner of the projected image with a pen” and a human figure indicating the position corresponding to the tap position are displayed together with the image image projected on the screen 4. The Note that the processing unit 24 displays the guide image in the center of the window 6 in a range smaller than the display screen 201 so as not to obstruct the tap with the electronic pen 1. This guide image may be a moving image or a still image. In addition, since the shape of the screen varies depending on the product, the image image displayed as the guide image may be a general shape of the screen.

  When the user taps the upper left corner 601 of the image projection area 403 using the electronic pen 1 according to the image, the electronic pen 1 reads the dot pattern at the tap position to generate stroke information, and corresponds to the pen ID. Then, the entry information obtained is transmitted to the computer apparatus 2 (step S301).

  When the computer apparatus 2 receives the entry information from the electronic pen 1 by the receiving means 22, the processing means 24 stores the pen-up position coordinates in the entry information in the storage means 25 as the recognition reference point 1 on the paper 402. Store (step S402). Further, as shown in FIG. 9, the processing unit 24 displays a guide image on the window 6, and guides the user to tap the lower right corner of the image projection area 403 of the screen 4 with the electronic pen according to the guide image (operation). Guide A2: Step S403). Here, as in step S401, as a guide image, an image simulating the screen 4 and the image projection area 403, a message such as “Please touch the lower right corner of the projected image with a pen”, and a tap position are displayed. A human figure indicating the corresponding position is displayed, and the tapping position (the lower right corner 602 of the image projection area 403) is guided in an easy-to-understand manner. When the user taps the lower right corner 602 of the image projection area 403 using the electronic pen 1 according to the image, the electronic pen 1 reads the dot pattern at the tap position to generate stroke information, and corresponds to the pen ID. Then, the entry information obtained is transmitted to the computer apparatus 2 (step S302).

  In the computer apparatus 2, when the entry information is received from the electronic pen 1 by the receiving means 22, the processing means 24 stores the pen-up position coordinates in the entry information in the storage means 25 as the recognition reference point 2 on the paper 402. Store (step S404). Then, the processing means 24 calculates the position on the display screen 201 in the computer apparatus 2 from the position coordinates on the sheet 402 of the screen 4 based on the coordinate values of the recognition reference points 1 and 2 on the sheet 402 by the dot pattern calculation. A coordinate conversion function for calculating the coordinates is obtained and stored in the storage means 25. At the same time, as a result of the calibration process, the processing unit 24 determines a coordinate input area (image projection area) 403 on which the position coordinates can be accurately input on the screen 4 by the electronic pen 1 corresponding to the display screen 201. The coordinate area is defined and stored in the storage unit 25 (step S405).

  Here, how to obtain the coordinate conversion function in the calibration process will be described with reference to FIGS.

FIG. 10 is a diagram illustrating position coordinates on the display screen 201 of the computer apparatus 2. As shown in FIG. 10, on the display screen 201 of the display means (display) 26 of the computer apparatus 2, the X axis is the right direction, the Y axis is the downward direction, and the following coordinates are as follows.
End point of upper left corner: Coordinate 202 (Xamin, Yamin),
Bottom right corner: coordinate 203 (Xamax, Yamax),
Where Xamin <Xamax,
Yamin <Yamax,

FIG. 11 is a diagram illustrating the position coordinates of the recognition reference points 1 and 2 on the sheet 402 of the screen 4. The recognition reference points 1 and 2 are obtained by writing the end points of the upper left corner and the lower right corner of the coordinate input area 403 defined by projecting the display screen 201 of the computer apparatus 2 onto the screen 4 with the electronic pen 1. It is a position coordinate based on the analysis of the obtained dot pattern (coding pattern). As shown in FIG. 11, in the coordinate input area (image projection area) 403 projected onto the screen 4 by the projector 3, the X axis is the right direction and the Y axis is the downward direction, and the following coordinates are as follows.
End point of upper left corner of paper 402: coordinates 405 (Xb0, Yb0),
End point of the upper left corner of the coordinate input area 403 (recognition reference point 1): coordinates 406 (Xbmin, Ybmin),
End point of the lower right corner of the coordinate input area 403 (recognition reference point 2): coordinates 407 (Xbmax, Ybmax),
However, Xb0 ≤ Xbmin <Xbmax,
Yb0 ≤ Ybmin <Ybmax,

  Then, the coordinates 202 (Xamin, Yamin) of the upper left corner and the coordinates 203 (Xamax, Yamax) of the lower right corner on the display screen 201 of the computer apparatus 2 are the recognition reference points in the dot pattern on the paper 402 of the screen 4. 1 corresponding to the coordinates 406 (Xbmin, Ybmin) and the coordinates 407 (Xbmax, Ybmax) of the recognition reference point 2 respectively. That is, the coordinate input area 403 in step S203 can be defined from a rectangular area having the coordinate values of the recognition reference point 1 and the recognition reference point 2 as diagonal vertices.

  Then, from the position coordinates (Xbn, Ybn) defined by the dot pattern on the paper 402 of the screen 4, the position coordinates (Xan, Yan) on the display screen 201 in the display means 26 of the computer apparatus 2 are converted into the following coordinates. It can be obtained with a function.

[Formula 1] Xan = Xamin + {Xbn− (Xbmin−Xb0)} × (Xamax−Xamin) / (Xbmax−Xbmin)
[Formula 2] Yan = Yamin + {Ybn− (Ybmin−Yb0)} × (Yamax−Yamin) / (Ybmax−Ybmin)

  Therefore, the processing means 24 converts the position coordinates (Xbn, Ybn) on the screen 4 to the position coordinates (Xan, Yan) on the display screen 201 of the computer device 2 according to the above formula, and performs various processes. Go. Note that a predetermined coefficient for correcting distortion may be applied to the term of the coordinate conversion function depending on the angle of the projector 3 with respect to the screen 4.

  After the calibration process, the user writes characters or the like using the electronic pen 1 in the coordinate input area 403 that is defined on the screen 4 so that the position coordinates can be accurately input corresponding to the display screen 201. Then, the electronic pen 1 generates stroke information corresponding to the stroke of the handwriting, and transmits entry information associated with the pen ID to the computer device 2 (step S101). In the computer apparatus 2, when the entry information is received by the receiving means 22, the processing means 24 stores the entry information in the storage means 25 (step S201), and determines that a coordinate conversion function is defined (step S202: Yes), the position coordinates (Xbn, Ybn) included in the stroke information of the entry information are used as the position coordinates (Xan, Yan) on the display screen 201 of the computer apparatus 2 using the coordinate conversion function obtained by the calibration process. Conversion is performed to draw strokes and perform various processes.

  That is, as shown in FIG. 12, when the user enters a character or the like with the electronic pen 1 in the coordinate input area 403 of the screen 4, the computer device 2 draws a stroke line segment corresponding to the handwriting, and the screen 4 The line segment of the stroke is projected by the projector 3 without deviating from the above handwriting. Further, since the position coordinates on the screen 4 and the position coordinates on the display screen 201 of the display unit 26 of the computer apparatus 2 are aligned by the coordinate conversion by the coordinate conversion function by the calibration process, the coordinate input of the screen 4 is performed. By using the electronic pen 1 to select a menu or icon projected in the area 403, the application can be operated or started.

<Operational effect of the first embodiment>
In the information processing system 10 according to the first embodiment, the processing unit 24 displays a guide image on the window 6 and guides the tap position for the calibration process. The tap by 1 can be performed. In addition, since the tap positions for the calibration process are the upper left corner and the lower right corner of the image projection area 403 of the screen 4, it is easy for the user to understand as the tap positions, and moreover, the tap position is closer to the center. The accuracy of the coordinate transformation function by calibration can be improved. Further, since the window 6 on which the guide image is displayed is displayed in the center in a range smaller than the display screen 201, it does not become an obstacle to tapping with the electronic pen 1.

[Modification]
The first embodiment may be configured as follows.

  In the first embodiment, the tap position for the calibration process is the upper left corner and the lower right corner of the image projection area 403 of the display screen 201 projected on the screen 4, but instead the lower left corner and the upper right corner. A corner may be used, or three or all four corners of the four image projection regions 403 may be used. At this time, the coordinate conversion function may be modified as appropriate. If the coordinate conversion function is set to three or more corners of the image projection area 403, the coordinate conversion function is printed on the paper 402, which is generated when the screen 4 is manufactured or pasted on a wall surface. The rotation angle (inclination) between the coordinate axis based on the dot pattern and the coordinate axis based on the image projection area 403 of the display screen 201 projected on the screen 4 is calculated, and the accuracy of the calibration process is further increased.

  Further, in steps S401 and S403 in the flow chart of the calibration process shown in FIG. 7, the processing means 24 displays an arrow 701 indicating the tap position on the display means 26 together with the guidance image of the window 6, as shown in FIG. You may let them. At this time, the processing means 24 may be made conspicuous by blinking the arrow 701 or displaying an animation such that the arrow 701 approaches the tap position. Further, although the projector 3 of the information processing system 10 is a front projector type, it may be a rear projector type, and in this case, an image is formed on the screen 4 on which a dot pattern is printed. It should be transparent.

Second Embodiment
Next, the information processing system 11 in the second embodiment will be described. In the description of the second embodiment, the configuration different from the first embodiment will be mainly described. In the second embodiment, the same reference numerals as those in the first embodiment are the same as those in the first embodiment unless otherwise specified.

  In the second embodiment, the tap position, the guide image, and the coordinate conversion function in the calibration process are different. This point will be described with reference to FIGS.

  FIG. 14 is a flowchart of calibration processing in the information processing system 11 according to the second embodiment. In the first embodiment, for the recognition reference point 1, the upper left corner of the image projection area 403 of the screen 4 is set as the tap position (see FIG. 7: step S401, FIG. 8). In the second embodiment, as shown in FIG. 15, the processing means 24 displays the circular mark 8 in the center of the display screen 201 in the window 6 and also displays the center (display) of the mark 8 in the window 6. A message image such as “Please touch the center of the circle with the pen” is displayed to guide the user to tap the reference point) with the electronic pen 1 (operation guide B1: step S601). When the user taps the center of the mark 8 using the electronic pen 1 according to the image, the electronic pen 1 reads the dot pattern at the tap position to generate stroke information, and is obtained in association with the pen ID. The entered information is transmitted to the computer apparatus 2 (step S501).

  When the computer apparatus 2 receives the entry information from the electronic pen 1 by the receiving means 22, the processing means 24 stores the pen-up position coordinates in the entry information in the storage means 25 as the recognition reference point 1 on the paper 402. Store (step S602). Further, as shown in FIG. 16, the processing means 24 displays a guide image on the window 6, and guides the user to tap the lower right corner of the image projection area 403 of the screen 4 with the electronic pen according to the guide image (operation). Guide B2: Step S603). Here, as in the first embodiment, the guidance image includes an image imitating the screen 4 and the image projection area 403 and a message such as “Please touch the lower right corner of the projected image with a pen”. A human figure indicating the position corresponding to the tap position is displayed, and the tapping position (the lower right corner 602 of the image projection area 403) is guided in an easy-to-understand manner. When the user taps the lower right corner 602 of the image projection area 403 using the electronic pen 1 according to the image, the electronic pen 1 reads the dot pattern at the tap position to generate stroke information, and corresponds to the pen ID. Then, the entry information obtained is transmitted to the computer apparatus 2 (step S502).

  In the computer apparatus 2, when the entry information is received from the electronic pen 1 by the receiving means 22, the processing means 24 stores the pen-up position coordinates in the entry information in the storage means 25 as the recognition reference point 2 on the paper 402. Store (step S604). Then, the processing means 24 calculates the position coordinates on the display screen 201 in the computer apparatus 2 from the position coordinates on the sheet 402 based on the coordinate values of the recognition reference points 1 and 2 on the sheet 402 by the dot pattern calculation. A coordinate conversion function for performing the calculation is obtained and stored in the storage means 25. At the same time, as a result of the calibration process, the processing unit 24 determines a coordinate input area (image projection area) 403 on which the position coordinates can be accurately input on the screen 4 by the electronic pen 1 corresponding to the display screen 201. The coordinate area is defined and stored in the storage unit 25 (step S605).

  Here, how to obtain the coordinate conversion function for calibration will be described with reference to FIGS.

FIG. 17 is a diagram illustrating position coordinates on the display screen 201 of the computer apparatus 2. As shown in FIG. 17, on the display screen 201 of the display means (display) 26 of the computer apparatus 2, the X axis is the right direction, the Y axis is the downward direction, and the following coordinates are as follows.
End point of upper left corner: Coordinate 202 (Xamin, Yamin),
Bottom right corner: coordinate 203 (Xamax, Yamax),
Display reference point (center of mark 8): coordinate 204 (Xa1, Ya1),
However, Xamin ≤ Xa1 ≤ Xamax, (Xamin ≠ Xamax)
Yamin ≤ Ya1 ≤ Yamax, (Yamin ≠ Yamax)

FIG. 18 is a diagram illustrating the position coordinates of the recognition reference points 1 and 2 on the sheet 402 of the screen 4. The recognition reference point 1 is a position coordinate based on the analysis of a dot pattern (coded pattern) indicated by the electronic pen 1 by projecting the display reference point mark 8 on the display screen 201 of the computer apparatus 2 onto the screen 4. . The recognition reference point 2 is used to analyze the dot pattern obtained by writing the end point of the lower right corner on the coordinate input area 403 defined by the projection of the display screen 201 of the computer device 2 onto the screen 4 with the electronic pen 1. The position coordinates based on. As shown in FIG. 18, in the coordinate input area (image projection area) 403 projected onto the screen 4 by the projector 3, the X axis is in the right direction, the Y axis is in the downward direction, and the following coordinates are as follows.
End point of upper left corner of paper 402: coordinates 405 (Xb0, Yb0),
End point of upper left corner of coordinate input area 403: coordinates 406 (Xbmin, Ybmin),
End point of the lower right corner of the coordinate input area 403 (recognition reference point 2): coordinates 407 (Xbmax, Ybmax),
Display reference point (center of mark 8) (recognition reference point 1): coordinates 408 (Xb1, Yb1),
However, Xb0 ≤ Xbmin <Xb1 ≤ Xbmax,
Yb0 ≤ Ybmin <Yb1 ≤ Ybmax,

  Then, the coordinates 204 (Xa1, Ya1) of the display reference point and the coordinates 203 (Xamax, Yamax) of the lower right end point on the display screen 201 of the computer apparatus 2 are the recognition reference point 1 in the dot pattern on the paper 402 of the screen 4. The coordinates correspond to the coordinates 408 (Xb1, Yb1) and the coordinates 407 (Xbmax, Ybmax) of the recognition reference point 2, respectively. The upper left corner point coordinates 406 (Xbmin, Ybmin) that define the coordinate input area 403 projected on the screen 4 can be expressed by the following formula.

[Formula 3] Xbmin = Xb1− (Xa1−Xamin) × (Xbmax−Xb1) / (Xamax−Xa1)
[Formula 4] Ybmin = Yb1− (Ya1−Yamin) × (Ybmax−Yb1) / (Yamax−Ya1)

  Therefore, the processing unit 24 defines the coordinate input area 403 in step S203 from the coordinates 406 calculated by Equations 3 and 4 and the rectangular area having the coordinate values of the recognition reference point 2 as diagonal vertices. Can do.

  Then, from the position coordinates (Xbn, Ybn) defined by the dot pattern on the paper 402 of the screen 4, the position coordinates (Xan, Yan) on the display screen 201 in the display means 26 of the computer apparatus 2 are converted into the following coordinates. It can be obtained with a function.

[Formula 5] Xan = Xamin + {Xbn− (Xbmin−Xb0)} × (Xamax−Xa1) / (Xbmax−Xb1)
[Formula 6] Yan = Yamin + {Ybn− (Ybmin−Yb0)} × (Yamax−Ya1) / (Ybmax−Yb1)

  Therefore, the processing means 24 converts the position coordinates (Xbn, Ybn) on the screen 4 to the position coordinates (Xan, Yan) on the display screen 201 of the computer device 2 according to the above formula, and performs various processes. Go. Note that a predetermined coefficient for correcting distortion may be applied to the term of the coordinate conversion function depending on the angle of the projector 3 with respect to the screen 4.

<Operational effects of the second embodiment>
In the information processing system 11 of the second embodiment, the tap position for the recognition reference point 1 is not the upper corner of the image projection area 403 of the display screen 201 projected on the screen 4 but the center of the image projection area 403. Therefore, even if the user does not reach the tap with the electronic pen 1 at the upper corner, the user can easily reach the center of the image projection area 403 lower than the upper corner. Moreover, the mark 8 displayed in the center of the image projection area 403 is easy to understand because it is guided that the position is tapped by a balloon arrow or the like of a message displayed together. When the electronic pen 1 taps the lower right corner 602 of the image projection area 403, the guide image is displayed in the center of the window 6 in a range smaller than the display screen 201, so that there is no hindrance to the tap.

[Modification]
The second embodiment may be configured as follows.

  In the second embodiment, the display reference point is displayed at the center of the display screen 201 and the tap position for the recognition reference point 1 is set at the center of the image projection area 403. However, the present invention is not limited to this. Further, the tap position for the recognition reference point 2 is not limited to the lower right corner of the image projection area 403 but may be another corner. Further, the calibration tap positions may be three or more. Further, an arrow indicating the tap position for the recognition reference point 2 may be displayed as shown by an arrow 701 in FIG. 13 as in the modification of the first embodiment. Further, although the projector 3 of the information processing system 11 is of the front projector type, it may be of the rear projector type, and the screen 4 on which the dot pattern is printed may be made transparent so that an image is formed.

[Other Embodiments]
The present invention is not limited to the above embodiment.

  In the first and second embodiments, the information processing systems 10 and 11 are configured using the projector 3, but as shown in FIG. 19, in the information processing system 100, a dot pattern (readable by the electronic pen 1) ( The encoding pattern) may be printed on the transparent sheet 9 and the transparent sheet 9 may be pasted on the display screen 201 of the computer apparatus 2. At this time, when performing the calibration process, as shown in FIG. 20, the processing means 24 causes the display screen 201 to display an image image of the display screen 201 and its frame as a guide image, and the guide image. To guide the position of tapping with the electronic pen 1. The user taps the guided tap position on the display screen 201 from the transparent sheet 9 using the electronic pen 1 to read the dot pattern.

  In this case, a dot pattern is printed on the transparent sheet 9 with ink having a characteristic of selectively reflecting infrared rays. In the electronic pen 1, the processor 108 has a strong infrared reflection from image data captured by the CMOS camera 106. A determination may be made by setting a threshold value so that the partial area is recognized as a dot. The threshold value is set so that the infrared rays transmitted through the transparent sheet 9 from the display can be distinguished from the infrared rays reflected by the dots.

  In the above embodiment, the electronic pen 1 does not have an ink cartridge. However, the pen unit 104 may be used as an ink cartridge so that characters can be directly written on the paper 402 with the electronic pen 1. . In this case, the computer apparatus 2 displays the stroke drawn with the electronic pen 1 on the display screen 201, but the image signal that is not projected by the projector 3 may be output from the computer apparatus 2 to the projector 3. Further, in order to make it easy to recognize dots from the image data captured by the CMOS camera 106 of the electronic pen 1, an infrared transmission filter is provided in the opening of the electronic pen 1 so that only the infrared light is irradiated to the CMOS camera 106. You may provide near 102. Further, the dot pattern (coded pattern) is not limited to the Anoto method.

  The information processing system can be used in school classes, company conference rooms, lectures, presentations, and the like.

DESCRIPTION OF SYMBOLS 1 ... Electronic pen 2 ... Computer apparatus 21 ... Input means 22 ... Reception means 24 ... Processing means 25 ... Memory | storage means 26 ... Display means 27 ... Transmission means 201 ... Display screen 3 ... Projector 4 ... Screen 401 ... Magnet board (white board)
402: Paper 403 ... Coordinate input area (image projection area)
6 ... Window 9 ... Transparent sheet 10, 11, 100 ... Information processing system

Claims (4)

A screen on which a coding pattern indicating position coordinates is formed;
Reading means for reading the coded pattern;
A computer device that receives information on the coding pattern transmitted from the reading unit and performs processing according to the information;
An information processing system comprising: a projector that receives an image signal from the computer device and projects an image on the screen;
The computer device includes:
Storage means for displaying an image image projected on the screen and storing a guide image for guiding a position designated by the reading means;
The guide image is read out and displayed on the display means, and the position coordinates on the screen calculated by the coded pattern of the indicated position read by the reading means are displayed on the display screen in the display means of the computer device. An information processing system comprising processing means for obtaining a coordinate conversion function for converting to position coordinates. A transparent sheet on which a coded pattern indicating position coordinates is formed;
Reading means for reading the coded pattern;
An information processing system comprising: a computer device that receives information related to the coding pattern transmitted from the reading unit and performs processing according to the information;
The transparent sheet is affixed to the display screen of the display means of the computer device,
The computer device includes:
Storage means for displaying an image on the display screen and storing a guide image for guiding a position designated by the reading means;
The guide image is read and displayed on the display means, and the position coordinates on the transparent sheet calculated by the coded pattern of the indicated position read by the reading means are displayed on the display screen of the display means of the computer device. An information processing system comprising: processing means for obtaining a coordinate conversion function for converting into the position coordinates. The processing means displays the guide image in the center in a range smaller than the display screen,
The information processing system according to claim 1, wherein the designated position designated by the reading unit is a position corresponding to a corner of the display screen.   The information processing system according to any one of claims 1 to 3, wherein the guide image indicates the indicated position indicated by the reading unit with an arrow.

Images