JP2005092435A - Pen type data input device and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic pen comprising an acceleration sensor and a camera, in which the position on a paper can be specified by calculating, in a case of writing onto a general paper, moving information of the electronic pen on the paper by the acceleration sensor when the electronic pen leaves the paper and from a plurality of images taken by the camera when the pen is moving on the paper, and by reading, in a case of writing to an exclusive paper, a code pattern displayed on the surface of the paper, and a program therefor. <P>SOLUTION: This electronic pen comprises a three-axial acceleration sensor, the function of acquiring trace information showing the moving trace of the electronic pan based on the acceleration detected by the sensor, an ink jet for printing the content drawn by a writer to a paper, a camera for taking the image of the printed content, and the function of acquiring trace information showing the printed trace based on a plurality of images taken by the camera. A code on the paper surface taken by the camera is read, and the position on the paper is specified based on it. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an apparatus and a program for capturing characters or figures written by a writer into a computer.

  Currently, an input device including a two-dimensional sensor array, such as an LCD (Liquid Crystal Display) tablet or a digitizer tablet, is widely used for inputting pen writing contents into a personal portable terminal or a computer application device. Since such an input device requires a two-dimensional sensor array having a relatively large area, a separate sensing plane is required. Therefore, it is difficult to carry, occupies a predetermined space, and has a disadvantage in terms of cost.

  Therefore, conventionally, in order to solve such a problem, only a single electronic pen is used on a general plane without a physical tablet. At this time, if this single electronic pen can input a document, it is very effective because a natural input can be made because a wide input space is provided compared to a conventional pen input device.

  In order to input a document or a picture using such a self-motion sensing type electronic pen, the position coordinate of the electronic pen tip with respect to a certain reference coordinate system must be continuously obtained. However, most writing operations are performed in a down state in which the pen is in contact with the writing surface, and when moving, the pen is in an up state in which the pen is not in contact with the writing surface. In order to obtain a continuous coordinate value of a pen, a means capable of accurately measuring the position value even in a contact or non-contact state is required.

Here, there are three types of conventional electronic pen-like input devices: an external coordinate measurement method, an internal coordinate measurement method, and a paper surface coordinate printing method.
The external coordinate measurement method is a method of measuring the coordinates of the pen tip outside the pen. For example, a triangulation method (Patent Document 1), an electromagnetic wave (Patent Document 2), or an ultrasonic wave (Patent Document 3). There are methods that use the time difference of flight.

However, since the above method is such that a transmission signal is transmitted from the pen and received from the outside, in a device such as a portable terminal, the receiver must be attached to the main body of the terminal, so There are inconvenient disadvantages.
The internal coordinate measurement method is a built-in method for measuring the coordinates of the pen tip inside the pen, and is a method for sensing the movement of the pen tip inside the pen. Initially, a 2-axis or 3-axis acceleration sensor mounted inside the pen. The system (patent document 4, patent document 5, patent document 6) which calculates | requires the position motion of an electronic pen through double integration using this was proposed.

  However, there is a problem that it is difficult to mount the acceleration sensor on the pen tip. When the acceleration sensor is mounted at a certain height, the influence on the tilt angle of the pen center axis is not taken into consideration, which may cause a large position error. In addition, since the accumulated error increases due to double integration of the acceleration signal, there is a disadvantage that it is difficult to measure accurate motion.

  In order to correct the influence on the tilt angle of the pen, A.TCross (Patent Document 7) proposes a method of moving the acceleration sensor element of two or more axes to the pen tip and moving the signal processing unit to the upper part of the pen. However, since the sensor element and the signal processing unit are separated and the influence of electrical noise is large, there is a problem that ink cannot be attached to the tip of the pen.

  On the other hand, Seiko Co., Ltd. (Patent Document 8) uses biaxial acceleration and 2-gyro to correct the tilt angle, finds the position of the acceleration by double integration, and calculates the tilt angle of the pen by simply integrating the angular velocity of the pen. A method of measuring is proposed. In addition, Richo (Patent Document 9 and Patent Document 10) presents a method for determining the position of a pen tip that performs a general three-dimensional writing motion by incorporating a three-axis acceleration sensor and a three-axis gyro sensor inside the pen. Yes.

  However, since the input plane must always be perpendicular to the direction of gravity, there is a limitation in use. In the method using the inertial sensor (acceleration sensor, gyro), acceleration is pen-integrated through double integration and angular velocity is pen-integrated through single integration. The position and angle of the sensor are estimated, but due to noise and drift of the sensor signal, the accumulative error increases in proportion to the square of the time in the acceleration system and proportional to the time in the case of the gyro to estimate the precise pen tip movement. There are difficult problems.

  In order to reduce the accumulated error, Intersense recently added an ultrasonic sensor to a pen device using a three-axis acceleration sensor and a three-axis gyro to reduce the position accumulated error generated from the inertial sensor (acceleration, angular velocity). Presenting technology. However, there is a problem that the addition of such an external sensor is not portable.

  In addition, as a result of these improvements, in Patent Document 11, in addition to a three-axis acceleration sensor, an optical three-dimensional measurement for measuring the tilt angle of the center axis of the electronic pen with respect to the writing surface and the height with respect to the writing surface. Some have improved the accuracy by providing a measuring device and correcting the position coordinates from the triaxial acceleration sensor using the height of the writing surface.

However, this method systematically detects the position in the height direction, but the amount of movement in contact with the writing surface must be measured with an acceleration sensor. Not suitable.
The paper coordinate printing method is a method in which coordinates are recorded with fine points on one surface of the paper, the coordinates are read by a two-dimensional image sensor (camera) mounted on the pen, and the locus of the pen is calculated based on the coordinates. And patent document 12 etc. are proposed. This method is accurate because the locus of the pen is detected by reading the coordinates written on the paper.

However, this method can operate only on the paper on which the coordinates are written in advance, and there is a problem that the locus of the pen cannot be taken freely on ordinary ordinary paper.
US Pat. No. 5,166,668 US Pat. No. 5,977,958 U.S. Pat. No. 4,478,674 US Pat. No. 5,247,137 International Publication WO 94/09447 Pamphlet US Pat. No. 5,587,558 US Pat. No. 5,434,371 JP-A-6-67799 US Pat. No. 5,902,968 US Pat. No. 5,981,884 JP 2003-029915 A International Publication No. WO92 / 17859 Pamphlet

As described above, the conventional electronic pen has various problems and is not always satisfactory.
Therefore, in the present invention, in order to solve the above problems, a pen-type data input device (electronic pen) provided with an acceleration sensor and a camera, when the pen-type data input device is separated from the paper surface. Calculate the pen movement information on the paper by using the acceleration sensor or multiple images taken by the camera when writing on the paper, or read the approximate code from the paper on which the data code including the coordinate position is written. A pen-type data input device and a program for calculating the position of a pen on a paper surface are provided.

According to the first aspect of the present invention, in the pen-type data input device for acquiring characters or figures written on a writing target medium as electronic data, according to the invention of claim 1,
Acceleration detecting means for detecting acceleration generated when the pen-type data input device is moved, and trajectory information indicating the trajectory of the pen-type data input device based on the acceleration detected by the acceleration detecting means. First trajectory information acquisition means for acquiring; writing means for writing the character or the figure on the writing target medium using ink; and imaging means for imaging the character or the figure written by the writing means; Second trajectory information acquisition means for acquiring trajectory information indicating a trajectory of the character or the graphic written by the writing means based on a plurality of images of the character or the graphic imaged by the imaging means; Reading means for reading the code pattern from a sheet on which a predetermined code pattern is arranged, and based on the code pattern read by the reading means. There are, can be achieved by providing a pen-type data input device characterized by comprising a third trajectory information acquisition means for acquiring track information of the character or the figure is written by said writing means.

With this configuration, the dedicated paper on which the code pattern is arranged can be used, so that it is possible to easily specify whether the electronic pen is at that position on the dedicated paper.
According to the second aspect of the present invention, the pen-type data input device is further configured based on whether the code pattern is read by the reading unit. 2. The apparatus according to claim 1, further comprising: a first selection unit that selects one of the second trajectory information acquisition unit and the third trajectory information acquisition unit and acquires the trajectory information. This can be achieved by providing a pen type data input device.

With this configuration, both normal paper and dedicated paper can be used.
Further, according to the invention described in claim 3 of the claim, the reading means also serves as the imaging means. The pen-type data input according to claim 1 or 2, This can be achieved by providing a device.

With this configuration, one camera can be used for a plurality of purposes.
In addition, according to the invention described in claim 4, the above-mentioned problem is that the pen-type data input device further includes a pressure detection unit that detects a pressure change with respect to the writing target medium, and the pressure detection unit. A second trajectory information is acquired by selecting one of the first trajectory information acquisition unit and the second trajectory information acquisition unit based on the information indicating the pressure change detected by the second step. The pen-type data input device according to claim 1, wherein the pen-type data input device is provided.

With such a configuration, when the electronic pen is away from the paper surface, the trajectory of the electronic pen is analyzed using the triaxial acceleration sensor, and when the electronic pen is in contact with the paper surface, the electronic pen is used. The trajectory of the pen can be analyzed.
Further, according to the invention described in claim 5, the second trajectory information acquisition unit is configured such that the second trajectory information acquisition unit includes the image captured this time by the imaging unit and the image captured last time. 2. The pen-type data input device according to claim 1, wherein the pen-type data input device generates a trajectory information indicating a trajectory of the character or the graphic written by the writing means based on the comparison result. Can be achieved.

By configuring in this way, the trajectory information can be obtained from the difference between continuously captured images.
According to the sixth aspect of the present invention, when the second trajectory information acquisition means is writing by the writing means, the first trajectory information acquisition means This can be achieved by providing the pen-type data input device according to claim 5, wherein the comparison range is determined based on the trajectory information acquired this time and the trajectory information acquired last time.

By configuring in this way, it is possible to speed up the processing and reduce the processing load by limiting the comparison range of two images to the vicinity of the portion where writing is performed.
According to the seventh aspect of the present invention, the pen-type data input device further includes the trajectory information acquired by the first or second trajectory information acquisition means. It can be achieved by providing a pen-type data input device according to claim 1, further comprising transmission means for transmitting to an external device.

With this configuration, the content written with the electronic pen can be transmitted to the host computer via wireless communication or the like.
Further, according to the invention described in claim 8 of the present invention, the pen-type data input device is configured to obtain the trajectory information by the transmitting means when communication with the external device is connected. The pen-type data input device according to claim 7, wherein the trajectory information is retained when the transmission is performed and the communication with the external device is disconnected.

  By configuring in this way, when the communication with the host computer is possible, the content written with the electronic pen can be transmitted to the host computer via wireless communication or the like in real time, and the communication is impossible. In such a case, the contents can be stored in a memory built in the electronic pen.

  According to the invention described in claim 9, the above-described problem is obtained by sending the electronic data to a computer in a pen-type data input device that acquires, as electronic data, characters or figures written on a writing target medium. In a data input program for executing processing to be acquired, an acceleration detection process for detecting an acceleration generated when the pen-type data input device is moved, and the pen based on the acceleration detected by the acceleration detection process. A first trajectory information acquisition process for acquiring trajectory information indicating a trajectory of movement of the type data input device; a writing process for writing the character or the figure on the writing target medium using ink; and writing by the writing process An imaging process for imaging the character or the figure that has been made, and an image of the plurality of characters or the figure that has been captured by the imaging process A second trajectory information acquisition process for acquiring trajectory information indicating the trajectory of the character or the graphic written by the writing process, and a reading process for reading the code pattern from a sheet on which a predetermined code pattern is arranged, Data input for causing a computer to execute a third trajectory information acquisition process for acquiring trajectory information of the character or the graphic written by the writing process based on the code pattern read by the reading process This can be achieved by providing a program.

With this configuration, the dedicated paper on which the code pattern is arranged can be used, so that it is possible to easily specify whether the electronic pen is at that position on the dedicated paper.
Further, according to the invention described in claim 10 of the claim, the second trajectory information acquisition process is performed between the image captured this time by the imaging process and the image captured last time. The data input program according to claim 9, wherein the computer executes a process of comparing and generating a trajectory information indicating a trajectory of the character or the graphic written by the writing process based on the comparison result. Can be achieved by providing.

By configuring in this way, the trajectory information can be obtained from the difference between continuously captured images.
According to the invention described in claim 11, the data input program further acquires the trajectory information acquired by the first or second trajectory information acquisition process from an external device. This can be achieved by providing a data input program according to claim 9 for causing a computer to execute transmission processing to be transmitted to the computer.

  With this configuration, the content written with the electronic pen can be transmitted to the host computer via wireless communication or the like.

When the present invention is used, the locus of the pen can be analyzed satisfactorily using a single image sensor, whether it is a paper surface on which a code is written or a paper surface on which a code is not written.
In addition, a sensor that can detect whether the pen is down on the paper surface is provided, and a method for switching between two pen locus analysis methods according to the determination of whether a code is written from the read image and the value of the pen down sensor. By providing the above, it is possible to automatically and appropriately switch the paper surface on which the code is written and the paper surface on which the code is not written.

(Embodiment 1)
The feature of the electronic pen in this embodiment is that it has the following four basic functions.
(1) The amount of ink can be controlled electronically according to the writing pressure, and the thickness of the pen character can be adjusted.
(2) The pen trajectory and writing pressure of characters and figures written on plain paper can be stored.
(3) The pen locus can be transferred to a personal computer (hereinafter referred to as PC) by wireless communication.
(4) It is possible to accurately detect the pen trajectory by reading out the code from the paper on which the specific code is written.

Furthermore, in addition to the above four functions, information of a specific code can be read out and sent to a PC to realize various functions. The present embodiment will be described below.
FIG. 1 is a configuration diagram of the electronic pen 1. The inkjet 17 and the inkjet nozzle 2 for spraying ink onto the paper surface are provided so that the thickness of the pen character can be adjusted according to the writing pressure. A pen tip 3 is mounted next to the nozzle 2 so that the nozzle 2 does not directly contact the paper surface. Ink does not come out from the pen tip 3, and a material that does not feel uncomfortable when a figure is written with the pen 1 is selected and mounted.

  A pressure sensor 4 is mounted at the base of the pen tip 3, and the pressure applied to the pen tip 3 is detected by the sensor 4. As a result, the thickness of the pen character can be adjusted according to the pressure. Further, the electronic pen 1 can be equipped with a color ink cartridge 5, and the ink cartridge 5 is made of CMYK (cyan, magenta, yellow, black) so that not only black ink but also a full color can be created. Includes four different colors. The color of the pen 1 can be selected and controlled so that the ink and the amount of ink corresponding to the color of the pen 1 are selected by being sprayed onto the writing surface via the nozzle 2 by the inkjet 17 described above. Drawn.

  In addition, a triaxial acceleration sensor 6 is provided to detect the locus of the pen 1, and a lens 7 and a CMOS image sensor 8 for reading a figure drawn on a paper surface are also provided to compensate for the insufficient accuracy of the acceleration sensor. Have. In addition, a processor 9 for processing the read image or calculating a pen locus and the like and a memory 10 for storing the calculated locus and writing pressure are also provided. A battery 11 for operating the electronic circuit is also mounted.

  Further, the image sensor 8 has an infrared LED 14 used for reading out coordinate points written on a special paper surface and an infrared filter 15 for transmitting only infrared rays. The filter 15 is driven so that only infrared rays can be transmitted to the CMOS sensor 8. The RF communication circuit 13 is a circuit for transmitting information such as a pen locus to the host computer. Next, the drawing operation of the electronic pen 1 will be described with reference to FIGS.

  FIG. 2 shows a control block diagram of the ink jet. FIG. 2 is a detailed circuit diagram of the partial block of FIG. 1, which is a configuration diagram of the electronic pen 1 in the present embodiment, with the addition of a signal flow. When drawing on paper (written material) using the electronic pen 1, drawing may be performed by sliding the pen tip 3 against the written material in the same manner as a normal pen.

  The pen tip 3 is directly connected to the pressure sensor 4 and is mounted so that the pressure applied to the pen tip 3 can reach the pressure sensor 4 as it is. Therefore, when the user (writer) uses the pen 1, the writing pressure can be detected by the pressure sensor 4. The pressure sensor 4 in the present embodiment uses a piezoresistive pressure sensor, and uses a material whose resistance value changes according to the pressure.

  A constant current is passed through the sensor 4, and the voltage at the end of the sensor 4 is obtained by the AD converter 902 in the processor 9 in accordance with the pressure value. This digital pressure value is read by the CPU 901. The CPU 901 adjusts the amount of ink sent from the color ink cartridge 5 to the inkjet 17 in accordance with this numerical value. The ink amount is adjusted by the following method.

First, assuming that the amount of black ink (K) used for drawing black is 1, the ink mixing ratio of CMYK for creating the current color (C1) to be drawn is set to Kc, Km, Let Ky, Kk.
As for this mixing ratio, the ink mixing ratio for obtaining the color closest to the color to be drawn is examined in advance, and programming of a method for deriving the mixing ratio is performed, or the mixing ratio of the ink corresponding to each color is performed. Is stored in a memory or the like, so that the mixing ratio can be easily obtained.

  Next, assuming that the maximum pressure value applied to the pen tip 3 is Pmax and the current pen pressure is P, the ink mixing ratios are Kc * P / Pmax, Km * P / Pmax, Ky * P / Pmax, respectively. Kk * P / Pmax. In accordance with these mixing ratios, the CPU 901 controls each valve 103 (103a, 103b, 103c, 103d) so as to flow from the ink cartridge 5 to the inkjet 17. Since a small valve cannot normally control the intermediate value of the outflow amount in an analog manner, the intermediate amount is created by controlling the valve open time (open) and OFF time (close) at high speed.

FIG. 3 shows pen line thickness control by duty modulation. This figure shows duty modulation for changing the diameter of ink on the paper (thickness at the time of pen) according to three types of opening and closing times.
FIG. 3A shows the duty modulation when the writing pressure is low. In this case, since the amount of outflow of ink is small, a thin line (the right figure in FIG. 3A) can be drawn. FIG. 3B shows duedy modulation when the writing pressure is intermediate. In this case, since the amount of ink flowing out is medium, an intermediate thickness line (the right figure in FIG. 3B) is drawn. Can do. FIG. 3C shows duty modulation when the writing pressure is large. In this case, since the amount of ink flowing out is large, a thick line can be drawn (the right diagram in FIG. 3C).

By controlling the opening / closing time of the valve 103 by duty modulation in this way, it becomes possible to eject ink from the inkjet nozzle 2 in accordance with the pressure of the pen, so that the thickness of the pen of a desired color can be controlled. It is.
Next, when drawing is performed on plain paper using the electronic pen 1 in the present embodiment, a trajectory analysis of the pen 1 is performed in order to record the drawing contents.

  FIG. 4 shows a configuration of a position detection circuit using a triaxial acceleration sensor, and is a block diagram showing an apparatus for performing a basic trajectory analysis of the pen 1. Although the drawing operation on the paper (written material) using the electronic pen 1 of the present embodiment has been described above, the locus of the pen 1 when the drawing is performed can be analyzed and acquired as data.

The triaxial acceleration sensor 6 is a device that detects acceleration according to the movement of the pen 1. In order to specify the coordinates of the pen 1 using this, the position of the pen 1 can be calculated by performing double integration from the acceleration.
The triaxial acceleration sensor 6 has been developed using a piezoresistor that can detect acceleration in each of the three axial directions of the space. Since the voltage values corresponding to the accelerations are output, the respective voltage values are configured so that the AD converter 902 in the processor 9 can continuously read the acceleration values in a time division manner via the selector 903. The

  In the present embodiment, the image sensor 8 is also provided as another means for analyzing the locus of the pen 1. What is shown in the lower left of FIG. In the electronic pen 1, the image on the writing surface 20 is condensed on the CMOS image sensor 8 by the lens 7 and converted into image data by the image sensor 8. In this embodiment, this image data is sent to the memory 10 by a DMA (Direct Memory Accessing) circuit 904. As a result, the image data can be operated from the CPU 901.

Incidentally, the CMOS sensor 8 is configured to sample the image data at a cycle of 100 sheets per second (at intervals of 10 milliseconds) and sequentially send them to the memory. The infrared LED 14 and the switch 905 will be described later.
Next, the trajectory analysis operation by the triaxial acceleration sensor 6 will be described with reference to FIGS.

FIG. 5 shows the movement of the pen on the paper surface and the change in the tilt of the pen axis, that is, the operation when the pen draws on the paper surface. FIG. 6 shows the movement of the electronic pen 1 from the pen up to the down in the present embodiment.
The movement of the pen 1 includes a two-dimensional movement along the paper surface 20 and a movement of a pen axis change that changes the tilt of the pen axis with respect to the paper surface 20. Further, the pen 1 leaves the paper surface 20 (at the point P1) (pen-up), and changes the tilt of the pen (P2), while touching (down) the paper surface 20 at another point (P3).

FIG. 7 shows an algorithm for obtaining the coordinate position from the acceleration sensor by the processor 9. The processor 9 is programmed to execute a plurality of processes called a task. This figure is a flowchart of one task process.
In this task, first, data detected by the triaxial acceleration sensor 6 is read (S1). The triaxial acceleration is obtained from these data (S2). The pen velocity v (three-dimensional vector) is calculated from the acceleration excluding these gravity components (S3). Further integration is performed to calculate a three-axis absolute position Px (three-dimensional vector) (S4).

  Next, it is determined whether or not the pen 1 is down on the paper surface 20 (S5). When it is not down to the page 20 (proceeding to "No" in S5), the process proceeds to S9 described later. When it is down to the paper surface 20 (proceed to “Yes” in S5), it is determined whether or not there is a request for three-axis parameter correction (S6). If there is no correction request in S6, the process proceeds to S8 described later.

If there is a correction request in S6, this correction request resets the pen speed variable when the pen-down occurs in another task (Task 2) for calculating coordinates on the paper, which will be described later, because the pen speed at that time is zero. Request to do so (S7).
This is because the speed and angular velocity of the pen 1 were calculated by integrating the acceleration sensor, and there was a possibility that error accumulation occurred and did not match the actual speed. Can be cleared. Next, a paper basis vector is created (S8). This will be described with reference to FIG.

  FIG. 8 is a diagram showing the relationship between the triaxial absolute space and the paper surface coordinates, and shows the triaxial absolute space obtained from the acceleration sensor 6 and the coordinates on the paper surface. In the figure, the X, Y, and Z axes are absolute coordinate systems calculated from triaxial acceleration. The square area representing the paper surface 20 indicates the position of the paper surface 20 in the absolute space. Eu and Ev vectors on the paper surface 20 are basis vectors that form the coordinate system of the paper surface 20.

  An example in which a line between points P1 and P2 is drawn on the paper surface 20 is shown. It is assumed that this vector is measured as Px in the XYZ coordinate system and as Pu in the coordinate system on the paper surface 20. Since the relationship between the X, Y, and Z axes and the U and V axes is linear, the following relationship is established.

  If the corresponding points of the two coordinates are (x1, y1, z1) → (u1, v1), (x2, y2, z2) → (u2, v2) at two points,

The relationship holds. Than this,

As a result, the basis vectors Eu and Ev are respectively set to (1, 0) and (0, 1) in (u, v) of the equation (B-1) using the coefficients obtained in the equation (B-3). Since it is obtained by substituting, it becomes as follows.

FIG. 9 shows a detailed algorithm for creating the basis vector of the page in S8. First, the pen position Pu of the current page coordinate is obtained. Since this is calculated in another task Task2, it is assumed that the latest Pu that has already been calculated is read (S10).
Next, the sample number Ns of the corresponding number of points of two types of coordinates is examined (S11). It is assumed that this is initialized to “0” at the start of Task 1 in advance. When this process is performed for the first time after initialization, since Ns = 0, the process branches to S12. Here, the pen position Pu of the current paper surface coordinate system is stored in (u1, v1), the coordinate value of Px in the triaxial acceleration coordinate system is stored in (x1, y1, z1) (S12), and the corresponding number of points Ns is set to 1. (S13), this flow ends.

  Thereafter, when this process is called again with the pen 1 still down on the paper surface (S8 in FIG. 7), after obtaining Pu in S10, Ns is “1”. The process branches to S14. For the corresponding points of the current coordinate, Pu is stored in (u2, v2) and Px is stored in (x2, y2, z2) (S14), and Δ in the equation (B-5) is calculated (S15).

This Δ is an outer product of two vectors (u1, v1) and (u2, v2), and it is desirable that this value be larger in order to improve the accuracy of calculation of a basis vector to be obtained later. Therefore, in S16, it is determined whether or not Δ is larger than a predetermined value (S16).
. If this Δ does not reach the predetermined value or is equal to or smaller than the predetermined value, it is determined that the second sample is not suitable for obtaining the basis vector, and the process is terminated without performing the subsequent processing.

On the other hand, if Δ exceeds a predetermined amount, the basis vector can be calculated with high accuracy, so that the current sample is valid and the number of samples Ns is set to “2” (S17). Then, the base vectors Eu and Ev are calculated by the equation (B-4) (S18), and the process ends.

Thereafter, when this processing is called again with the pen 1 still down on the paper surface (S8 in FIG. 7), after obtaining Pu in S10, Ns is “2”. It is determined that Ns> 1, and the process ends.
As described above, in S8, the coordinates Pu on the paper surface calculated from the current absolute coordinates Px and Task2 are referred to, these two coordinates are stored in pairs, and Eu, Ev from the past several coordinate pairs. Can be requested. In the present embodiment, calculation of Eu and Ev is performed with Task1, but of course, it may be performed with Task2. As a result, the basis vector can be calculated while writing on the paper surface 20 with the pen 1.

Returning to the flow of FIG. When S8 is completed or when pen-down is not performed in S5, the process waits for a predetermined time (T seconds) (S9). This is because if only this task is processed, other tasks cannot be processed.
However, when the trajectory of the pen 1 is obtained only by this method, an acceleration detection interval time of a normal acceleration sensor is several milliseconds, and an error in acceleration change between detection times occurs. This error is also integrated to perform the integration to obtain the coordinates, resulting in a fairly large error. With the current acceleration sensor, it is difficult to obtain a trajectory with a resolution of 100 DPI (Dot Per Inch).

For this reason, the image sensor 8 is used to solve the problem while the pen 1 is drawing on the paper. As described above, when the writer is drawing a character graphic on the paper surface 20, it can be detected by the pressure sensor.
FIG. 10 shows a writing example of characters read from the camera in the present embodiment. Here, the camera refers to an imaging system including the lens 7 and the image sensor 8. FIG. 10A and FIG. 10B show two images that are taken continuously (at intervals of 10 milliseconds) while the writer is drawing the character “A” with the electronic pen 1. From these two images, the pen moving distance and the pen coordinate position can be calculated by the algorithm shown in FIGS.

FIG. 11 shows the flow of the pen coordinate position analysis task. In order to distinguish from the coordinate systems X, Y, and Z calculated by the three-axis acceleration described above, a two-dimensional coordinate system on the paper surface calculated from the camera image is set as U and V coordinates.
The calculation of the moving distance by the image sensor 8 is performed in another task (Task 2). After the start of the task, the variable FirstDown flag indicating that the pen has never been down on the paper surface 20 is set to Yes, and the variable PenStatus indicating the pen state is initialized to the pen-up state PEN_UP. (S20). Then, the image matching flow is repeatedly called (S21).

  FIG. 12 shows a pen coordinate analysis flowchart by image matching in S21. First, the state of the pen pressure is checked by the pressure sensor 4, and it is determined whether or not the pen 1 is writing (pen down state) (S30). When the pen 1 is in the down state, the previous pen state PenStatus is checked (S31). PenStatus for the first pen down is PEN_UP.

  If “PenStatus = PEN_UP” in S31, PenStatus is set to PEN_DOWN (S32), and the 3-axis absolute coordinates of the position on the pen-down paper surface 20 are read (S33). At this time, the coordinates stored in the three-axis absolute coordinates Px are stored in Pxd. Next, as described above, in order to reset the speed error, a correction request for the parameter (speed) of the three-axis absolute coordinates is made (S34).

  Then, it is checked whether or not the pen is down for the first time on the paper surface 20 by referring to the FirstDown flag. If it is the first time (the process proceeds to “Yes” in S35), the FirstDown flag is cleared (FirstDown = No) (S40) to initialize the paper plane coordinates (Pu = 0) (S41). Then, the paper surface coordinate position down on the paper surface is stored (Pu_down = Pu) (S39).

  Thereafter, the process is looped again. While the pen is being drawn on the paper surface, the process passes through S30 and S31, and since “PenStatus = PEN_DOWN”, the process branches to S42. Next, it is determined whether an image has been input (S42). As described above, image image data is taken periodically (every 10 milliseconds) (see FIG. 10), and loops until an image is input and enters a standby state (proceed to “No” in S42). ). When a new image is input (proceed to "Yes" in S42), the number of pixels is too large to perform matching calculation between the input images as they are, so a reduced image is created from the input image (S43) ( The process of S33 will be described later).

  Next, an image matching processing routine is called in S44 and S45, and a movement vector Du is obtained (detailed operation will be described later). The coordinate position is updated by adding the difference vector Du to the past pen coordinates Pu (S46). The obtained position coordinates and the current writing pressure are read from the pressure sensor 4, and both are sequentially stored in the memory 10 (S47).

As described above, while the writer is drawing on the paper surface 20, it is possible to track the pen trajectory and the pen pressure by performing image matching. When the writer performs the pen-up, the pen pressure disappears. As a result, the determination of the pen pressure is determined to be up in S30, and the process branches to S48.
It is determined based on PenStatus whether or not it is the first pen-up state (S48). If it is the first time, PenStatus is set to PEN_UP (S49), and the three-axis absolute coordinate Px at that time is stored in Pxu (S50). Further, the paper plane coordinate Pu at that time is stored in Pu_up (S51). After pen-up, the amount of movement of the pen 1 in the air is measured by Task 1.

  When the pen 1 is lowered onto the paper surface 20 again, it is determined in S35 that the second time or later, and the movement vector (du, dv) is determined in S36- using the previous pen-up coordinate and the current absolute coordinate value. Calculated in S38. First, the airborne absolute movement vector Dx is obtained by subtracting the 3-axis absolute coordinate Pxu acquired in S51 from the 3-axis absolute coordinate Pxd acquired in S33 (S36).

  Next, a process for converting the aerial absolute movement vector Dx into coordinates on the paper is performed. Eu and Ev are vectors corresponding to the XYZ coordinate systems of the U and V component unit vectors of the U and V coordinates, respectively. Now, converting the X-Y-Z system absolute air movement Dx to the U-V coordinate system is nothing other than decomposing the Dx vector into Eu and Ev. Therefore, if Dx = (x, y, z), Eu = (xu, yu, zu), Ev = (xv, iv, zv), the component (du, dv) of the UV coordinate system of Dx to be obtained is

Can be obtained.
The paper surface coordinate position Pu when pen-down is performed again may be obtained by adding (du, dv) obtained in S37 to the coordinates when pen-up is performed in S51 (S38). This Pu is stored as Pu_down (S39). In this way, the movement vector of the paper surface coordinates can be calculated from the absolute coordinates of the three axes.

  FIG. 13 is an example of an image matching algorithm. In this algorithm, matching is performed by calculating the sum of differences in pixel values between corresponding pixels for two images to be compared. The range of the distance between images to be matched (the amount of deviation between two images) is passed as an argument. In the flow of FIG. 12, this corresponds to the process of S45.

  The range can be specified by Du_Begin to Du_End on the U-axis side and Dv_Begin to Dv_End on the V-axis side. Normally, -Range to + Range are specified for both du and dv so that matching can be performed within the range of the maximum distance RANGE that can be moved at normal pen speed between image samplings.

  First, when du = Du_Begin (S60), dv = Dv_Begin is set (S61). In S60 and S61, the movement distances du and dv between the two images designated by this argument are designated in order. Next, two images are matched according to the following formula (S62).

  p (n, u, v) is the pixel value of the coordinates u, v of the current image n, and p (n-1, u-du, v-dv) is a coordinate point in the previous image n-1. This indicates the value of a pixel separated from (u, v) by (du, dv). The pixel value is represented by 8 bits, and black is 0 and white has a maximum value of 255. MU and MV are image sizes in the U and V axis directions of these images, and indicate the range of images to be matched. D (du, dv) represents the result of summing the pixel value differences between the overlapping pixels in a state where the two images are shifted by the movement distances du and dv. Since the difference is 0 when the values of the corresponding pixels are the same, the larger the calculated D (the closer to 0), the more similar the two images.

dv is incremented (S63), and the above processing is repeated until dv> Dv_End. Next, Du is incremented (S64), and the above processing is repeated until du> Du_End.
Next, the maximum value of D (du, dv) is searched (S65). This is the range of du and dv shown below

Among these, the maximum value is obtained, and the obtained du and dv are set as the movement value Du = (du, dv) (S66).
As described above, the matching of the two images is calculated in S62. The larger the assumed D, the more similar the two images. The maximum similarity is searched for in S65 from the similarities obtained in S60 to S64, and since it can be said that these two images are most similar at this time, du and dv at this time are two images. It can be estimated in S66 that the vector the pen has moved between.

  The coordinates Pu of the pen position can be sequentially calculated from the captured image. In the present embodiment, a device for further reducing the amount of calculation for matching is made. The process performed in S44 of FIG. 12 is as follows. The current 3-axis absolute coordinate position Px is read from Task 1, the difference between the 3-axis absolute coordinate position Px when this process was performed last time and the current coordinate position Px read this time is calculated, and the moved absolute coordinate vector Dx is obtained. obtain.

  By calculating the inner product in the same manner as S36 and S37 from this value, the paper surface movement distance Du = (du, dv) in the paper surface coordinate system (U-V axis) can be calculated. Since this value is an approximate movement vector moved between the two images, if the image matching is performed in the vicinity, the movement vector can be obtained efficiently.

For example, when calling an algorithm, arguments of Du_Begin = du−δ, Du_End = du + δ, Dv_Begin = dv−δ, Dv_End = dv + δ (δ is a sufficiently small value compared to RANGE in the vicinity) may be passed.
In the present embodiment, the matching is performed without considering the rotation of the shaft. However, when the matching is performed in consideration of the rotation of the shaft from the above-described angular velocity, a more accurate movement amount is required.

  Next, the resolution of the camera in this embodiment will be described. FIG. 13 described above shows an algorithm for calculating the amount of movement of the pen 1 by an image. Using the algorithm shown in the figure, the moving distance can be calculated in units of pixels from the input image. Therefore, it is sufficient if the resolution of the camera is more than necessary for storing the pen trajectory. Usually, it is said that about 100 DPI is desirable to reproduce the trajectory of the pen. Therefore, the image sensor and the optical design may be designed so that there are four or more scanning lines per mm.

  In the present embodiment, the image sensor is 480 × 480 with an angle of view of 12 mm. The reason why the resolution is higher than necessary here is because it is used for other operations described later. Since this number of images is over-spec for performing matching, the number of pixels of the image to be matched is reduced to about 128 × 128 resolution in S43 in the algorithm of FIG.

  Next, transfer of trajectory information of the electronic pen 1 to the host will be described. When the pen 1 is used to draw on the paper surface 20 by the method described above, the pen trajectory and writing pressure are stored in the memory in real time. The contents of the memory are read out in real time when the radio is being serviced, and sent out to a host such as a PC by radio. When the radio is not being serviced, it is stored in the memory 10 until the radio is serviced. The data stored when the wireless service is provided is called at once and transferred to a host such as a PC.

  This is because the wireless nature makes communication unstable, so it is designed with that in mind. Of course, if the host does not operate for some reason, for example, you can take out only the pen without having the host PC at the business trip and write the proceedings on the business trip in the pen. Data can be saved and read out on the office PC after returning to work. In the present embodiment, data is transferred wirelessly by a PC. However, the present invention is not limited to this and may be wired.

  Next, pen trajectory detection when using dedicated paper in this embodiment will be described. The method of analyzing the trajectory by performing the image matching described above is effective in a considerable number of cases, but is drawn in an unstable state when the paper surface moves relatively or when the paper is held by hand. When the pen 1 is down, there is a disadvantage that there is an error in the coordinate position when the pen 1 is down.

  As a technique for solving this problem, the electronic pen 1 can read the data from the paper surface on which data including the coordinate position and the like is recorded. This data is efficiently recorded on paper with a small area and a small number of information points. An example of encoding using this method is shown in FIGS.

FIG. 14 shows allocation of information symbols and codes in the present embodiment. In the figure, a point is recorded in a glance area corresponding to a data bit (1) or not recorded (0) (the data encoding / decoding operations are not related to the present invention and are therefore omitted). .) Assuming that 4 枡 is one code symbol, when a point is recorded at the upper left, a code of “00” is expressed. When a point is recorded in the upper right, a code “01” is expressed. When a point is recorded in the lower left, a code “10” is expressed. When a point is recorded in the lower right, the code “11” is expressed. Thus, the information symbol represents 2-bit information.

  FIG. 15 shows the symbol arrangement of the blocks recorded on the dedicated paper. A hook-shaped symbol in the upper left corner is called a sync symbol, and a 9 × 9 unit in which the sync symbol is arranged in the upper left corner is called a block. The entire paper is composed of this block. In the block, there are eight information symbols described in FIG. 14 in addition to the synchronization symbols. In order to represent a coordinate position using this block, as shown in FIG. 15, coordinate data x0 to x5 in the X-axis direction is assigned by three information symbols on the upper right side of the block, and three information symbols on the lower left side of the block are assigned. To assign coordinate data y0 to y5 in the Y direction. The remaining two information blocks can be assigned z0 to z3 (option data) for other purposes.

  By detecting the hook-shaped synchronization symbol alone, the direction of the block can be known, and at the same time, a square in which the synchronization symbols of four adjacent blocks are associated with the vertices can be detected. By dividing the synchronization symbols into nine equal parts in the x and y directions to form a virtual lattice, and detecting where each of the eight 2 × 2 lattices has a dot, x, Each information of y and z can be restored.

  Actually, due to limitations such as imaging resolution, the synchronization symbol detection position (for example, the center of gravity) includes an error depending on the inclination and brightness of the subject. In order to reduce this influence, as a method of increasing the reference points, the number of synchronization symbols serving as reference points is increased by expanding the image capture range, information symbols are incorporated into the reference points (use of deviation between information symbols and grids), etc. Is possible.

Also, the size of 1 bit can be used as the resolution of the pen trajectory. If this glance size is set to 100 DPI (Dot Per Inch), the resolution of the pen trajectory (identifying the coordinate position) can be equal or higher.
This symbol (symbol) is recorded in advance by a printer or printing. In addition, it is desirable to use a material having a component different from that of the normal ink used for the pen 1 as the ink used for recording the code. This is because when drawing letters that intersect, the pen may be overwritten on the code, and then the pen may be overwritten at the same position. In such a case, the code is once overwritten with the pen ink. But this code must be readable. Therefore, in this embodiment, the ink of the pen 1 uses an ink of a material that transmits infrared rays, and the ink that records a code uses a material that absorbs infrared rays.

Next, code reading of the electronic pen 1 will be described.
FIG. 16 shows an example in which symbols (code data) are written on the entire page. This symbol is read out by the same apparatus used in FIG. On the writing surface in FIG. 4, a code as shown in FIG. 15 is recorded on the entire surface of the paper. That is, this sheet is a dedicated sheet in the present embodiment. This code is printed with an infrared absorbing material as described above. When reading the code, the CPU 901 controls the switch 905 to turn on the infrared LED 14. The infrared transmission filter 15 is rotated from the vertical direction to the horizontal direction by the electromagnetic spring 16 about the rotation axis, and the filter 15 is inserted between the lens 7 and the image sensor 8 (indicated by a dotted line). )become.

  FIG. 17 illustrates the operation of the electromagnetic spring in an easily understandable manner. The electromagnetic spring 16 has a rotation axis, and an infrared filter 15 is attached along the rotation axis. The infrared filter 15 is stabilized in either the sleeping state or the standing position in FIG. 16 by the operation of either the electromagnet A (30) or the electromagnet B (31).

  When reading the character, the electromagnet A (30) is turned on and the filter 15 is made to stand so that all light including infrared rays is allowed to pass through and irradiate the sensor light receiving unit 32 of the image sensor 8. On the other hand, when reading the code, only the infrared ray is allowed to pass, so that the electromagnet B (31) is turned on to operate the infrared transmission filter 15 (visible light filter). When reading the code in this way, only the infrared rays reach the image sensor, so that only the code printed on the paper surface can be read regardless of the upper body to which the ink of the pen is applied.

Next, switching between code reading and image matching will be described. In other words, a description will be given of the trajectory analysis switching operation of the electronic pen 1 when using normal paper and using special paper.
FIG. 18 shows a flow of a trajectory analysis task by switching between the encoding method and the image matching method. The method of obtaining the coordinate position by reading the code from the paper on which the code is printed does not matter how the pen moves off the paper surface. This is because the pen 1 can always specify the position of its own pen by reading the coordinates on the paper. This method can fully track the pen trajectory.

  However, the author must always prepare a special paper with a sign. Therefore, the electronic pen 1 analyzes the pen trajectory by reading the code when the code is written on the paper surface 20, and if the code is not written, the electronic pen 1 matches the written character. The movement vector is obtained, and as a result, the pen locus on the paper surface is analyzed.

  An example of the switching method is shown in FIG. This figure is obtained by adding a pen analysis function based on the sign method to Task 2 in FIG. In the task, first, “Matching” is substituted for the PenMode flag for determining which method the pen locus is to be analyzed for, and initialization is performed (S70). As described above, it waits until an image that is periodically sampled is captured (S71). For the captured image, it is first determined whether or not a code is written on the paper (S72).

Here, the determination in S72 is performed regardless of whether or not the pen is down on the paper surface. When the code is written on the paper surface, the code can be read before the pen 1 approaches the paper surface and touches it.
Therefore, in this embodiment, when the code is ready to be read (proceed to “Yes” in S72), it is assumed that the code is immediately written on this page, and “Code” is substituted into the PenMode flag. The sign mode is set (S73). Thereafter, the paper position Pu is obtained from the read image from the code analysis and the code (S74, S75), the coordinate value and the current pen pressure are stored in the memory, and the locus is recorded (S76). When the pen mode is changed to the sign mode, the FirstDown flag is reset with Yes (S77).

  While the pen is drawing on the paper, if the code cannot be read because of dust or the like, the process proceeds to the “No” direction in S71, and the PenMode flag is determined (S78). Here, since “Code” is stored in PenMode as described above, the pen pressure is determined (S79). While the pen 1 is down, it branches in the “down” direction in S79, so that the pen mode maintains the code mode. In this manner, while the pen 1 is drawing on the paper with the code written thereon, the locus can be recorded by repeating the loop described above.

  When the pen 1 is far away from the paper and cannot read the code, the pen pressure determination in S79 branches to “up”, “Matching” is substituted for the pen mode (S80), and the image matching mode is set. The matching process is called (S81). This image matching process is the same as the flow of FIG.

  Thereafter, when the pen 1 is drawn on plain paper on which no code is written, the processes of S71, S72, S78, S79, S80, and S81 are repeated regardless of whether the pen 1 is up or down. Therefore, this image matching process (S81) is called repeatedly. Therefore, the pen trajectory can be recorded by image matching by the method described above.

  As described above, according to the present embodiment, the trajectory analysis and recording of the pen 1 can always be performed regardless of the paper surface on which the code is written or the paper surface on which the code is not written by switching the method of analyzing the trajectory of the two types of pens 1. become. In addition, since these two types of analysis methods quickly determine the paper surface and perform dynamic switching, the two analysis processes do not need to be performed in parallel, and can be realized without using an expensive processor.

In addition, when sending pen trace data to the host via communication, it is possible to devise the difference in the accuracy of the pen trace in the application on the host side by sending in which mode the trace was calculated It is.
(Embodiment 2)
In the first embodiment, two readings are realized by switching the infrared transmission filter 15 between the case where the code is read and the case where the character written in normal ink is read. In the present embodiment, the image sensor is used. In addition, the image sensor is shared by a method of alternately arranging a filter that transmits only infrared rays and a filter that absorbs the macro filter.

As a result, the resolution at the time of reading the code is lowered, but there is a feature that the infrared transmission filter 15 and the electromagnetic spring 16 can be omitted. This can be dealt with by increasing the number of pixels of the image sensor as the resolution decreases.
As another method, since the pen ink and the code ink used in the first embodiment need only be exclusive, the pen ink uses an ultraviolet transmitting material, and the code ink uses an ultraviolet absorbing material. The filter 15 may be replaced with an ultraviolet transmission filter, and the infrared LED may be replaced with an ultraviolet LED.

(Embodiment 3)
In the first embodiment, an electronic pen equipped with one acceleration sensor has been described. In the present embodiment, an electronic pen equipped with two acceleration sensors will be described.
The writing trajectory only needs to be able to specify how the pen tip 3 is moving on the paper surface 20, so that only the two-dimensional movement along the paper surface 20 should be originally detected. However, when the method using the acceleration sensor is used, if the pen axis changes in inclination as shown in FIG.

  As shown in FIG. 6, the three-dimensional acceleration sensor only estimates auxiliary pen movement when the pen moves away from the paper surface. Therefore, in the first embodiment, the movement distance is obtained on the assumption that the pen tilt or the like when the pen 1 floats and moves does not change. However, even if such a restriction is provided, the movement distance is long. Unless there is no problem in practical use. Of course, by adding another three-axis gyro to the electronic pen 1 in this embodiment as in Patent Document 9, it is possible to obtain pen position coordinates excluding the influence of the tilt of the pen axis. This embodiment can further improve the accuracy of the trajectory calculation of the electronic pen in consideration of the influence of the tilt of the pen shaft.

1 is a configuration diagram of an electronic pen in Embodiment 1. FIG. FIG. 2 is a control block diagram of inkjet in Embodiment 1. 5 is a diagram illustrating pen line thickness control by duty modulation in Embodiment 1. FIG. FIG. 3 is a diagram illustrating a configuration of a position detection circuit using a triaxial acceleration sensor in the first embodiment. It is a figure which shows the operation | movement at the time of the pen in Embodiment 1 drawing on the paper surface. It is a figure which shows the mode of the movement of the electronic pen from pen-up in Embodiment 1 to down. It is a figure which shows the algorithm which acquires a coordinate position from the acceleration sensor in the processor 9 in Embodiment 1. FIG. FIG. 3 is a diagram illustrating a relationship between a three-axis absolute space and paper plane coordinates in the first embodiment. FIG. 6 is a diagram illustrating an algorithm for creating a paper basis vector in the first embodiment. It is a figure which shows the example of writing of the character read from the camera in Embodiment 1. FIG. FIG. 6 is a diagram illustrating a flow of a pen coordinate position analysis task in the first embodiment. It is a figure which shows the pen coordinate analysis flowchart by the image matching in S41 in Embodiment 1. FIG. It is a figure which shows the example of the matching algorithm which does not consider the rotation of the axis | shaft in Embodiment 1. FIG. 6 is a diagram illustrating assignment of information symbols and codes according to Embodiment 1. FIG. It is a figure which shows the symbol arrangement | positioning of the block in Embodiment 1. FIG. FIG. 4 is a diagram illustrating an example of symbols (code data) on the entire paper surface according to the first embodiment. FIG. 5 is a diagram illustrating an outline of an operation of an electromagnetic spring in the first embodiment. It is a figure which shows the locus | trajectory analysis task by switching of the encoding method and image matching method in Embodiment 1.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Electronic pen 2 Inkjet nozzle 3 Pen tip 4 Pressure sensor 5 Color ink cartridge 6 Triaxial acceleration sensor 7 Lens 8 CMOS image sensor 9 Processor 10 Memory 11 Battery 13 RF communication circuit 14 Infrared LED
15 Infrared filter 16 Electromagnetic spring 17 Inkjet 20 Paper 30 Electromagnet A
31 Electromagnet B
32 sensor light-receiving part 103 (103a, 103b, 103c, 103c) Valve 901 CPU
902 AD converter 903 selector 904 DMA

Claims (11)

In a pen-type data input device that acquires characters or figures written on a writing target medium as electronic data,
Acceleration detecting means for detecting acceleration generated when the pen-type data input device is moved;
First trajectory information acquisition means for acquiring trajectory information indicating the trajectory of movement of the pen-type data input device based on the acceleration detected by the acceleration detection means;
Writing means for writing the character or the figure on the writing target medium using ink;
Imaging means for imaging the character or the figure written by the writing means;
Second trajectory information acquisition means for acquiring trajectory information indicating a trajectory of the character or the graphic written by the writing means based on a plurality of images of the character or the graphic imaged by the imaging means;
Reading means for reading the code pattern from the paper on which the predetermined code pattern is arranged;
Based on the code pattern read by the reading means, third trajectory information acquisition means for acquiring trajectory information of the character or the graphic written by the writing means;
A pen-type data input device. The pen-type data input device further includes:
Based on whether or not the code pattern is read by the reading means, the trajectory information is acquired by selecting one of the second trajectory information acquisition means and the third trajectory information acquisition means. The pen-type data input device according to claim 1, further comprising: a first selection unit that performs the selection. The pen-type data input device according to claim 1, wherein the reading unit also serves as the imaging unit. The pen-type data input device further includes:
Pressure detecting means for detecting a pressure change with respect to the writing target medium;
Based on the information indicating the pressure change detected by the pressure detection means, one of the first trajectory information acquisition means and the second trajectory information acquisition means is selected to obtain the trajectory information. A second selection means to obtain;
The pen-type data input device according to claim 1, further comprising: The second trajectory information acquisition unit compares the image captured this time by the imaging unit with the previously captured image, and based on the comparison result, the character or the character written by the writing unit The pen-type data input device according to claim 1, wherein trajectory information indicating a trajectory of the figure is generated. The second trajectory information acquisition means, when writing by the writing means, a range for the comparison based on the trajectory information acquired this time by the first trajectory information acquisition means and the trajectory information acquired last time. The pen-type data input device according to claim 5, wherein the pen-type data input device is determined. The pen-type data input device further includes:
The pen-type data input device according to claim 1, further comprising: a transmission unit configured to transmit the trajectory information acquired by the first or second trajectory information acquisition unit to an external device. The pen-type data input device transmits the trajectory information by the transmission means when communication with the external device is connected, and the trajectory information when the communication with the external device is disconnected. The pen-type data input device according to claim 7, wherein the pen-type data input device is held. In a data input program for causing a computer in a pen-type data input device that acquires characters or figures written on a writing target medium as electronic data to execute processing for acquiring the electronic data,
An acceleration detection process for detecting an acceleration generated when the pen-type data input device is moved;
First trajectory information acquisition processing for acquiring trajectory information indicating the trajectory of movement of the pen-type data input device based on the acceleration detected by the acceleration detection processing;
A writing process for writing the character or the figure on the writing target medium using ink;
An imaging process for imaging the character or the graphic written by the writing process;
A second trajectory information acquisition process for acquiring trajectory information indicating a trajectory of the character or the graphic written by the writing process based on a plurality of images of the character or the graphic captured by the imaging process;
A reading process of reading the code pattern from a sheet on which a predetermined code pattern is arranged;
A third trajectory information acquisition process for acquiring trajectory information of the character or the graphic written by the writing process based on the code pattern read by the reading process;
Data input program for causing a computer to execute. The second trajectory information acquisition process includes:
The image captured this time by the imaging process is compared with the previously captured image, and the trajectory information indicating the trajectory of the character or the graphic written by the writing process is generated based on the comparison result. The data input program according to claim 9, for causing a computer to execute the processing to be performed. The data input program further includes:
The data input program according to claim 9 for causing a computer to execute a transmission process of transmitting the trajectory information acquired by the first or the second trajectory information acquisition process to an external device.

Images