JP2006079391A - Trajectory acquiring apparatus and trajectory acquiring program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire an excellent writing trajectory information even when an image such as a document is superimposed on a position detection code, thereby the code can not be decoded. <P>SOLUTION: A trajectory acquiring apparatus 1 for acquiring a trajectory of a pointer on a medium P with the position detection code formed on its surface includes comprises an image input part 12 for serially reading partial images on the medium P, a position information acquiring part 13 for acquiring the position information of the partial images on the medium P by decoding the position detection code included in the partial image read by the image input part 12, a movement information acquiring part 14 for acquiring movement information from the plurality of partial images serially read by the image input part 12, and a trajectory information storing part 16 for storing the information acquired by the position information acquiring part 13 and the movement information acquiring part 14. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a trajectory acquisition apparatus and a trajectory acquisition program for acquiring a trajectory on a medium when an object is added to the surface of a medium on which a machine-readable code is formed.

  In recent years, a new user interface has been proposed in which electronic information can be accessed from a paper surface by embedding address information capable of specifying the position on the paper surface in advance and reading the address information on the paper surface.

  For example, in Patent Document 1 and Patent Document 2, a position detection code (glyph code pattern) in which address information formed in the vicinity of a perceptual display such as an icon graphic formed on a medium is read by a pen-type scanner or the like. A new user interface that detects the position of the perceptual display on the medium by reading with a device is proposed.

  Also, by using the same type of technology, a position detection code in which address information that can specify the position on the paper surface is embedded in advance on the paper surface is printed, and the position detection code on the paper surface is read by the image sensor attached to the writing instrument. There is a technique in which writing information for a document can be acquired electronically by detecting embedded address information.

  For example, the technique described in Patent Document 3 digitizes written information added on paper together with operations on written information (e-mail transmission, input to an electronic calendar, input to an electronic task list, etc.). This enables the fusion of paper information and electronic information.

  In Patent Document 4, a method of using information such as a reading position, distortion, and inclination of a two-dimensional code as a position detection code for correcting address information, a correction code different from the two-dimensional code for detecting address information A technique for improving the position detection accuracy of the address information obtained from the read two-dimensional code by a method of correcting the address information using a correction code and the like is printed near the two-dimensional code. Yes.

JP-A-10-187911 JP 2000-99257 A Special table 2003-508843 gazette JP 2000-293303 A

  Here, in order to electronically obtain additional information on the printed electronic document, it is necessary to superimpose and print the electronic document and a position detection code in which address information is embedded. However, if the document image overlaps the position detection code for detecting the address information, a part of the position detection code cannot be read, so that the position detection code cannot be correctly decoded and the address information cannot be detected. Problems arise.

  In addition, the retouching information on the position detection code that does not overlap the document image can be acquired by the conventional technology, but when rewriting the retouched position detection code, a part of the position detection code can be read by the retouching information. Therefore, the position detection code cannot be correctly decoded, and the address information cannot be detected.

  In the conventional technique, since the writing address information of the writing tool cannot be calculated in such a case, the trajectory information of the writing tool cannot be reproduced. To avoid this, the position detection code is printed with a special color material that absorbs light of a specific wavelength, and the color material used for the document printed on paper and the color material used for the ink of the writing instrument are detected. Position detection by using a color material that absorbs light of a different wavelength from the color material used to print the code and irradiating the position detection code printed on the paper with light of a wavelength that matches the special color material. The code detection accuracy is increased so that trajectory information on the paper can be acquired.

  However, in such a configuration, it is necessary to prevent the absorption wavelength of the printing color material of the position detection code and the printing color material of the document to be printed on the paper surface, the coloring material used for the ink of the writing instrument from interfering with each other, Since a special color material is used, the cost is increased, and the color material that can be used is limited, so that a color range that can be reproduced is limited.

  Even if the color material to be used for printing is set in this way, if a third party writes on the paper with a color material that interferes with the print color material of the position detection code, the area around the area to be written will be written. There arises a problem that the trajectory information cannot be acquired.

  The present invention has been made in view of the above-described circumstances, and a moving trajectory that can successfully acquire writing trajectory information even when an image such as a document is superimposed on the position detection code and the position detection code cannot be decoded. It is an object of the present invention to provide an acquisition device and a movement trajectory acquisition program.

  The present invention has been made to achieve such an object. That is, the present invention is a trajectory acquisition apparatus for acquiring a trajectory of an indicator on a medium having a machine-readable code formed on a surface thereof, an image input means for continuously reading partial images on the medium, and an image input Position information acquisition means for acquiring the position information of the partial image on the medium by decoding the machine-readable code included in the partial image read by the means, and movement information from a plurality of partial images read continuously by the image input means The trajectory acquisition apparatus includes movement information acquisition means to be acquired and storage means for accumulating information acquired by the position information acquisition means and the movement information acquisition means.

  The present invention also provides a trajectory acquisition program for acquiring a trajectory of an indicator on a medium having a machine-readable code formed on a surface, the step of continuously reading a partial image on the medium, and the read partial image Decoding the machine-readable code included in the information to acquire position information of partial images on the medium, acquiring movement information from a plurality of consecutively read partial images, and storing the position information and movement information A trajectory acquisition program having steps.

  In the present invention, the partial image having the machine-readable code formed on the medium is read, and the position information of the partial image and the movement information are acquired from the plurality of continuously read partial images. Therefore, even when the machine-readable code included in the partial image cannot be read, the trajectory of the indicator can be acquired using the movement information.

  Therefore, according to the present invention, even when an image such as a document is superimposed on a machine-readable code for detecting a position, it is possible to acquire writing trajectory information satisfactorily. Accordingly, it is not necessary to limit the material of the ink, and the configuration of the user interface system that reads the position detection code with a pen-type scanner or the like can be simplified.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic configuration diagram illustrating a trajectory acquisition apparatus according to the present embodiment. That is, the trajectory acquisition apparatus 1 according to the present embodiment includes a writing unit (indicator) 10 for writing on a medium (paper) P, and a writing pressure detection unit for detecting a writing operation at the writing unit 10. 11, an image input unit 12 for capturing an image on the paper near the writing point, and a position detection code image input from the image input unit 12 is decoded to calculate position information (address information) on the paper. Information obtained from the position information acquisition unit 13, the movement information acquisition unit 14 that calculates the movement direction and movement distance of the image from the two-dimensional image input from the image input unit 12, and the information obtained from the position information acquisition unit 13 and the movement information acquisition unit 14 A trajectory information calculation unit 15 for calculating the writing trajectory information in the writing unit 10, a trajectory information storage unit 16 for storing and storing the calculated trajectory information, and for transmitting the stored trajectory information to a terminal such as an external PC. Communication interface And a Esu 17.

  The apparatus main body (housing) of the trajectory acquisition apparatus 1 is a portable pen type or the like, and can be written by the user on the medium P by hand. For this reason, the writing unit 10 is provided with a pen tip from which ink is drawn from the tip. The writing pressure detection unit 11 is a part that detects whether or not the user is writing a character or the like with the writing unit 10 based on the writing pressure. When a character or the like is written in the writing unit 10, a predetermined writing pressure is applied. Therefore, whether or not a character or the like is written can be detected by detecting this. In addition, although it detects whether the character etc. are written by writing pressure here, even if it detects by other methods (For example, the method of detecting the change of the conductivity by a pen tip touching paper) good.

  The image input unit 12 is a part that images a machine-readable code on the medium P (in this embodiment, a two-dimensional code is taken as an example of the position detection code) with a two-dimensional image sensor such as a CCD. The data of the two-dimensional code captured by the image input unit 12 is sent to the position information acquisition unit 13 and the movement information acquisition unit 14. The position information acquisition unit 13 obtains position information indicating where the pen tip is on the medium P from the read two-dimensional code. Here, the position information is X and Y coordinates from the determined origin on the medium P. The movement information acquisition unit 14 obtains movement information indicating in which direction the pen tip is moving. Here, the movement information is the movement direction and movement amount of the pen tip.

  The trajectory information calculation unit 15 obtains the pen tip trajectory information from the information obtained by the position information acquisition unit 13 and the movement information acquisition unit 14. Here, the trajectory information is curve (straight line) information obtained from a plurality of coordinate data (plot) obtained from position information and movement information. The trajectory information is accumulated in the trajectory information storage unit 16 and transmitted to a terminal such as an external personal computer via the communication interface 17.

  FIG. 2 is a block configuration diagram of the trajectory acquisition apparatus according to the present embodiment. Here, the configuration illustrated in FIG. 1 is a specific example, and the same description among the block configurations illustrated in FIG. 2 indicates the same configuration. 1 is a specific example of the motion detection unit 21 shown in FIG. 2, the communication interface 17 shown in FIG. 1 is a specific example of the communication unit 27 shown in FIG. 2, and is not shown in FIG. The control part 20 which comprises each part is shown by FIG.

  As a form of the trajectory acquisition apparatus of the present embodiment, a pen-type configuration as shown in FIG. 1 is provided, or the image input unit 12, the motion detection unit 21, and the communication unit 27 are provided in the main body housing, and the position information acquisition unit 13 is provided. The movement information acquisition unit 14, the trajectory information calculation unit 15, and the trajectory information storage unit 16 may be all or partly provided outside.

  FIG. 3 is a flowchart for explaining the operation procedure of the trajectory acquisition program according to the present embodiment. The trajectory acquisition program is executed by the control unit of the trajectory acquisition device described above, partially executed by an external processing device (computer or the like), or stored as software on a recording medium such as a CD-ROM. It is realized in the form of being provided in a state or distributed via a network.

  First, it is detected whether or not a writing operation is being performed by the writing pressure detection unit (step S1), and if it is a writing operation, an image is input from the image input unit (step S2). Next, the two-dimensional code is detected and decoded from the image input by the position information acquisition unit to detect position information on the paper (step S3). Next, it is determined whether or not the position information has been successfully detected (step S4). If the position information has failed, the movement information acquisition unit detects the movement information of the image (step S5) and stores the movement information in the memory (step S5). Step S6).

  On the other hand, if the position information is successfully detected, the memory storing the movement information is checked (step S7). If the information is stored (Yes in step S8), the newly detected position information is used. The movement information stored in the memory is corrected to position information (step S9), the movement information is cleared (step S10), and the corrected position information and detected position information are stored in the memory (step S10). Step S11).

  That is, when the position information can be detected from the two-dimensional code, it is determined whether or not the movement information detected before is stored in the memory. If the position information is stored, the movement information (movement direction) is determined. And the movement distance) and the position information (coordinate position on the medium), the position information when the two-dimensional code cannot be detected is back-calculated, and the obtained position information is stored in the memory. Thereby, when the two-dimensional code can be detected, the pen tip locus information can be obtained by calculating the pen tip position information on the medium by the position information acquisition unit.

  On the other hand, when the two-dimensional code cannot be detected (for example, when the two-dimensional code is added), the movement information acquisition unit detects the movement information (movement direction and movement distance) of the pen tip and stores it in the memory. When the position information can be detected from the two-dimensional code, the position information of the pen tip so far can be calculated backward from the movement information stored in the memory and the detected position information to obtain trajectory information. It becomes like this. As a result, it is possible to obtain pen tip trajectory information even when a two-dimensional code cannot be detected due to printing or writing on the medium.

  Note that in the above-described trajectory acquisition program, movement information is acquired when position information cannot be detected from the two-dimensional code, and back calculation is performed from movement information acquired in advance when position information can be detected next time. The position information that could not be detected is obtained, but if the position information is detected first and the position information cannot be detected next, the movement information is acquired and the position information detected earlier is obtained. Position information that could not be detected by adding the movement information to the information may be calculated.

  FIG. 4 is a diagram illustrating a medium (paper) on which the trajectory acquisition apparatus according to the present embodiment performs a writing operation. On the medium (paper) P, a document and a two-dimensional code in which address information on the paper is embedded are printed on the entire background of the document. As an example of the two-dimensional code, one constituted by a “\”-shaped pattern image representing bit 0 and a “/”-shaped pattern image representing bit 1 can be cited.

  The size of each pattern image is about 4 dots × 4 dots at 600 dpi (dot per inch), and the interval between the pattern images is about 6 dots at 600 dpi. Therefore, it is difficult to identify the structure of the pattern, and it appears that a gray image with a uniform density (when the pattern image is printed in black) is superimposed on the document background.

  FIG. 5 is a diagram illustrating a configuration example of a two-dimensional code. The two-type bit pattern images described above are combined to form a two-dimensional code in which the address information on the page is embedded. For example, if the two-dimensional code is an 8 × 8 matrix in which two types of bit pattern images are combined, address information can be embedded on the paper surface at intervals of about 2 mm (6 dots / bit pattern × 8 dots). Although the two-dimensional code is described as a hatched pattern shape, it is not limited to this.

  FIG. 6 is a schematic diagram for explaining address information formed on a medium by a two-dimensional code. On the paper surface, the two-dimensional code shown in FIG. 5 is formed on the entire paper surface, and address information represented by each two-dimensional code (A1, A2,..., B1, B2,... In FIG. 6). Are configured so as not to overlap on the paper surface.

  FIG. 7 is a diagram for explaining a state in which writing is performed on a medium. Writing can be performed on the medium P using the dedicated pen (trajectory acquisition device) shown in FIG. On the medium P, the two-dimensional code described above is printed on the entire background. In the present embodiment, the locus of the pen tip can be calculated even when the two-dimensional code cannot be detected by printing or writing on the two-dimensional code.

  FIG. 8 is a diagram illustrating an image input by the image input unit of the dedicated pen (trajectory acquisition apparatus) illustrated in FIG. 1. The image input unit is configured to capture an area (imaging area R) that includes two or more two-dimensional code images. With such a configuration, it is possible to include at least one two-dimensional code image in the imaging region R of the image input unit. The position information acquisition unit can acquire address information embedded in the two-dimensional code in the imaging region by decoding the two-dimensional code image input from the image input unit. Furthermore, the position of the trajectory acquisition device can be corrected to more accurate position information by detecting the shift amount between the captured two-dimensional code image and the imaging region by the movement information acquisition unit.

  FIG. 9 is a diagram illustrating an example of a detected trajectory. The image input unit captures a plurality of partial images continuously in a predetermined sampling period. By detecting the two-dimensional code from the captured partial image and calculating the position of the pen tip on the medium, the trajectory of the pen tip moving on the medium is obtained by performing the process on the captured partial image continuously. Can do. The plot in the figure indicates the calculated pen tip position, and the locus of the pen tip is obtained by calculating a curve connecting a plurality of plots.

  FIG. 10 is a diagram illustrating an example of writing on characters included in a print document. When writing on a character or ruled line included in a printed document, the character or ruled line is on the two-dimensional code, so the address information cannot be decoded by correctly decoding the two-dimensional code, only the information obtained from the position information acquisition unit Then, a correct locus cannot be acquired.

  FIG. 11 is a diagram illustrating an example in which a trajectory is calculated using information only from the position information acquisition unit. Since the two-dimensional code cannot be decoded in the print portion (print portion of “ABC”) in the drawing, correct address information cannot be acquired, and trajectory information is lost. Therefore, since the trajectory is calculated so as to connect the portions where the address information can be acquired, the trajectory is short-circuited different from the original trajectory.

  FIG. 12 is a diagram for explaining a movement information acquisition method of the movement information acquisition unit. The moving direction and moving distance of the image are calculated from the two images (a) and (b) input with a predetermined time interval from the image input unit. That is, it is possible to acquire movement information by detecting an attention image in each image and obtaining how the attention image has moved in the imaging region R, that is, a moving direction and a moving distance of the attention image. it can. As the attention image, a fiber image of a medium (paper) can be cited. Since the fiber image is unique, movement information can be obtained by obtaining from the two captured images the same fiber image as the fiber image captured as the target image.

  FIG. 13 is a block diagram illustrating the configuration of each information acquisition unit, where (a) is the configuration of the position information acquisition unit, and (b) is the configuration of the movement information acquisition unit. In the position information acquisition unit illustrated in FIG. 13A, the image shaping unit 131 performs image inclination correction and distortion correction. The pattern detection unit 132 detects a bit pattern image from the input image. The digital information detection unit 133 detects a two-dimensional code from the bit pattern image and decodes it to extract the embedded information. The address decoding unit 134 calculates address information on the paper surface from information extracted from the two-dimensional code.

  In the movement information acquisition unit shown in FIG. 13B, the image shaping unit 141 performs preprocessing for detecting difference information such as density normalization, resolution reduction, and minute noise removal for the input image. The difference information detection unit 142 generates difference information between the image already stored in the image storage unit 143 and the newly input image, and the movement information detection unit 144 calculates the moving direction and movement distance of the image from the difference information. . In this embodiment, the example in which the moving direction and distance of the image are calculated from the difference information is shown. However, the present invention is not limited to this. For example, the cross-correlation value between two images is calculated, and the most cross-correlation value is obtained. A method of obtaining the moving direction and moving distance of the image from the position where the height increases is also possible.

  FIG. 14 is a diagram showing the detection result of the trajectory according to the present embodiment. Even if there is printing on the two-dimensional code and the two-dimensional code cannot be detected, since the movement information of the pen tip is acquired, a new information is obtained using the position information and movement information detected so far. Position information can be calculated, and the locus of the pen tip can be accurately detected.

  In this embodiment, the two-dimensional code is described as an example of the position detection code. However, the present invention is not limited to this, and another position detection code (for example, an M series) as in the prior art may be used. .

It is a schematic block diagram explaining the locus | trajectory acquisition apparatus which concerns on this embodiment. It is a block block diagram of the locus | trajectory acquisition apparatus which concerns on this embodiment. It is a flowchart explaining the operation | movement procedure of a locus | trajectory acquisition program. It is a figure explaining the medium (paper) with which the locus | trajectory acquisition apparatus which concerns on this embodiment performs writing operation | movement. It is a figure which shows the structural example of a two-dimensional code. It is a schematic diagram explaining the address information which a two-dimensional code comprises on a medium. It is a figure explaining a mode that writing was performed on the medium. It is a figure explaining the image which the image input part of the exclusive pen (trajectory acquisition apparatus) shown in FIG. 1 inputs. It is a figure which shows the example of the detected locus | trajectory. It is a figure which shows the example which wrote on the character contained in a printed document. It is a figure which shows the example which calculated the locus | trajectory only with the information of a positional information acquisition part. It is a figure explaining the movement information acquisition method of a movement information acquisition part. It is a block diagram explaining the structure of each information acquisition part. It is a figure which shows the detection result of the locus | trajectory by this embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Trajectory acquisition apparatus, 10 ... Writing part, 11 ... Writing pressure detection part, 12 ... Image input part, 13 ... Position information acquisition part, 14 ... Movement information acquisition part, 15 ... Trajectory information calculation part, 16 ... Trajectory information storage , 17 ... Communication interface, 20 ... Control unit, 21 ... Motion detection unit, 27 ... Communication unit, P ... Medium, R ... Imaging region

Claims (11)

A trajectory acquisition device for acquiring a trajectory of an indicator on a medium having a machine-readable code formed on a surface thereof,
Image input means for continuously reading partial images on the medium;
Position information acquisition means for acquiring a position information of the partial image on a medium by decoding a machine-readable code included in the partial image read by the image input means;
Movement information acquisition means for acquiring movement information from a plurality of partial images continuously read by the image input means;
A trajectory acquisition device comprising: storage means for accumulating information acquired by the position information acquisition means and the movement information acquisition means. Trajectory calculation means for calculating trajectory information of the indicator on the medium from at least one of position information acquired by the position information acquisition means and movement information acquired by the movement information acquisition means. The trajectory acquisition apparatus according to claim 1. The locus calculation means calculates the locus information based on the position information when the position information can be acquired, and calculates the locus information based on the position information obtained from the movement information when the position information cannot be acquired. The trajectory acquisition apparatus according to claim 2. The trajectory acquisition apparatus according to any one of claims 1 to 3, wherein the image input unit reads an invisible partial image on the medium. The image input means reads a machine-readable code on the medium and a fiber image of the medium as the partial image,
The movement information acquisition unit acquires the movement information from a fiber image acquired from a plurality of the partial images continuously read by the image input unit. The trajectory acquisition device according to item. 6. The apparatus according to claim 1, wherein the apparatus main body including at least the image input unit includes a pen type having portability, and includes a writing instrument capable of writing on the medium. Trajectory acquisition device. A trajectory acquisition program for acquiring a trajectory of an indicator on a medium having a machine-readable code formed on a surface,
Continuously reading partial images on the medium;
Decoding machine-readable code contained in the read partial image to obtain position information of the partial image on a medium;
Acquiring movement information from a plurality of partial images read continuously;
And a step of accumulating the position information and the movement information. The trajectory acquisition program according to claim 7, further comprising: calculating trajectory information of the indicator on the medium from at least one of the position information and the movement information. When the position information can be acquired, the trajectory information is calculated based on the position information, and when the position information cannot be acquired, the trajectory information is calculated based on position information obtained from the movement information. The trajectory acquisition program according to claim 8. The trajectory acquisition program according to any one of claims 7 to 9, wherein the partial image on the medium includes an invisible image. Reading the machine-readable code on the medium and the fiber image of the medium as the partial image;
The trajectory acquisition program according to any one of claims 7 to 10, wherein the movement information is acquired by capturing a fiber image from the plurality of partial images read continuously.

Images