JP4159948B2 - Two-dimensional code reading device, two-dimensional code reading method, two-dimensional code reading program, and storage medium - Google Patents

Description

  The present invention relates to image reading, and more particularly, to a barcode reading apparatus, a barcode reading method, and a barcode reading program for reading a barcode.

  With the digitization of business and the spread of personal computers, opportunities to digitize documents and handle them as so-called electronic documents are increasing. On the other hand, since the document display quality of a display such as a CRT or LCD is often inferior to the print quality on paper, there are many opportunities to print electronically created electronic documents on paper as paper documents.

  Although it is a daily routine to manually write on such a printed paper document, the original electronic document is not associated with the added content at all, and the content added to the printed paper document Is reflected in the electronic document, it is necessary to open the electronic document on a personal computer or the like and edit it again. Therefore, there is a need for means for digitizing the added information in a form associated with the original electronic document so that the handwriting work performed on the printed paper document can be used effectively.

  On the other hand, there is an invention in which optically readable code symbols (two-dimensional codes) are arranged in a matrix on a paper surface, and the two-dimensional code is read with a camera provided on the pen at the same time as writing on the paper surface with a pen. (For example, refer to Patent Document 1). According to the invention described in Patent Document 1, it is possible to acquire coordinate information of a position where writing is performed using a two-dimensional code read by the camera, and associate the page information before writing with the writing information while reading the writing information in real time. be able to. In the invention described in Patent Document 1, document-specific information for specifying a document and coordinate positions on paper are simultaneously converted into a monochrome pattern and printed on paper. Even if reading with an optical pen is unstable using a two-dimensional code that has an error correction function for black and white patterns, it has a function that can acquire document-specific information and coordinate information, so reading is unstable Even in such cases, reading is possible.

  Also disclosed is an invention relating to digitization of writing information using a two-dimensional code using a X coordinate, a Y coordinate, a pattern indicating the direction of the code, and a pattern larger than other patterns arranged at the center of the code called a homing pattern. (For example, see Patent Document 2). In the invention described in Patent Document 2, since the two-dimensional code is printed with invisible ink, it cannot be seen with the naked eye, but the two-dimensional code can be read at the same time as writing on the paper surface using a pen equipped with a predetermined camera. it can. Therefore, it is possible to associate the page information before the writing with the writing information while reading the writing information in real time without impairing the determination of the document content.

Further, an invention relating to digitization of retouched information using a two-dimensional code that can have information of 2 bits or more in one dot by shifting a dot from a predetermined position as a two-dimensional code is disclosed (for example, , Patent Document 3). The two-dimensional code has horizontal and vertical coordinate information, and dots are continuously printed on the entire paper. In the invention described in Patent Document 3, it is possible to associate the page information before the writing with the writing information while reading the writing information in real time by reading the two-dimensional code at the same time as the writing with the pen provided with the small camera. it can.
JP 2000-293303 A USP 5661506 WO 00/73981 A1

  However, since the two-dimensional code described in Patent Document 1 has a large size, a wide range of paper must be photographed with an optical pen. For this reason, since the quality of the image read by problems, such as depth of field, falls, it becomes difficult to read a two-dimensional code continuously. In addition, the two-dimensional code is not captured and decoded by the camera during writing, but because a general-purpose two-dimensional code is used, the decoding process takes time and is difficult to perform in real time. . If the two-dimensional code is read and the coordinate position cannot be acquired continuously while acquiring continuous writing information, discontinuous points of the writing information are generated, so that correct writing information cannot be obtained.

  In the invention described in Patent Document 2, since the two-dimensional code is printed with invisible ink, it is practically impossible to print the two-dimensional code using a general printing apparatus. That is, in order to use this two-dimensional code, it is necessary to print the two-dimensional code in advance using a non-visible ink in a special printing apparatus. Therefore, in order to embed the information of the document that the user wants to print in the print medium at the time of printing, not only a means such as printing with another barcode superimposed is necessary, but the barcode must be added before adding to the document. It is necessary to read, and convenience is impaired. If you forget to read the barcode and add another document, the link between the added information and the electronic document will be inconsistent. Although it is conceivable to print a two-dimensional code with visible ink, the two-dimensional code described in Patent Document 2 uses a pattern larger than other patterns, so that the readability of the document and the added content is impaired. .

  Further, in the invention described in Patent Document 3, a two-dimensional code that makes it possible to place coordinate information of 2 bits or more on one dot by arranging dots slightly shifted from two-dimensionally equidistant points. Disclosure. However, since the amount of position shift is as small as 30 μm, it is difficult to reproduce a correct shift amount due to accuracy of paper feeding, unevenness of the rotation speed of the photosensitive member, and the like in a widely used laser printer (resolution of about 1200 dpi). . Therefore, in the case of the two-dimensional code described in Patent Document 3, it is necessary to prepare a paper on which the two-dimensional code is printed by using a printing technique such as offset printing in advance.

  In view of the above problems, an object of the present invention is to provide a barcode reading apparatus, a barcode reading method, and a barcode reading program that are fast and versatile.

To solve the above problems, is surrounded by the code frame rectangle formed by a plurality of dots, the projection image two-dimensional code corner dots are arranged in a rectangular vertex is photographed, dots constituting the two-dimensional code In a two-dimensional code reading device that has dot detection means for detecting a dot and that reads information represented by a two-dimensional code based on the dots detected by the dot detection means, it is within a predetermined range from the target dot N detected by the dot detection means When a plurality of other dots are searched and two pairs of dots that are point-symmetric with respect to the target dot N are detected among the plurality of dots existing around the target dot N, the target dot N becomes the corner dot. The corner dot detection means for detecting the equivalent and the corner dot detected by the corner dot detection means A corner dot that estimates the position of another corner dot from the position of a fixed first corner dot based on the position of a pair of dots that are point-symmetric with respect to the first corner dot and a preset number of dots between corners A plurality of corner dot candidates in the vicinity of the estimated position of the other corner dot estimated by the position estimation means and the corner dot position estimation means are searched for, and two of the plurality of corner dot candidates are point-symmetric with respect to the corner dot candidate. Corner dot determination means for determining that a corner dot candidate corresponds to another corner dot when a pair of dots is detected, and a two-dimensional code from the positions of the first corner dot and the other three corner dots And a code frame detecting means for detecting the code frame .

According to the present invention, to detect the corner dot code frame of the two-dimensional code, it is possible to detect the code frame based on the corner dot, it can be read fast two-dimensional code.
Also, the first corner dot detection result is verified by estimating the position of the first corner dot from a predetermined corner dot among other corner dots detected based on the first corner dot. can do. If the position of the first corner dot is verified, the positions of the other corner dots are also verified. Since the code frame of a two-dimensional code can be detected by connecting corner dots with straight lines, even if the dots that make up the code frame are difficult to detect due to characters, ruled lines, dirt, etc., the two-dimensional code can be detected. Can be detected.

In one embodiment of the present invention, the position of the first corner dot estimated by the corner dot position estimating means based on the position of the corner dot determined by the corner dot determining means to be a corner dot, and the corner dot detecting means When the position of the first corner dot detected by (1) coincides with the first corner dot , it has a corner dot coincidence judging means for judging that the corner dot forming the code frame has been detected .

According to the present invention, if the first corner dot is detected, the remaining corner dots are detected in the vicinity of the estimated position after the position is estimated. Therefore, the dots constituting the code frame of the two-dimensional code are detected. all may even without it detects.

Further, in one aspect of the present invention, the corner dot position estimating means sets the position of the corner dot detected by the corner dot detecting means and the position of a pair of dots that are point-symmetric with respect to the corner dot, starting from an appropriate origin. The position of the other corner dot is estimated in the direction obtained by subtracting the second vector connecting the origin and the other dot of the pair from the first vector connecting the origin and the one of the pair of dots. Features.

According to the present invention, it is possible to determine the direction of another corner dot in the extending direction of a straight line generated by the corner dot constituting the corner and the dot constituting the code frame adjacent to the corner dot.

In one embodiment of the present invention, the corner dot position estimating means divides a vector obtained by subtracting the second vector from the first vector by 2, and multiplies the preset number of dots between the corners to obtain a corner by that distance. The positions of other corner dots are estimated at positions away from the corner dots detected by the dot detection means.

According to the present invention, it is possible to determine the direction of another corner dot in the extending direction of a straight line generated by the corner dot constituting the corner and the dot constituting the code frame adjacent to the corner dot.

Further, in one aspect of the present invention, the corner dot determination unit is configured to center a plurality of corner dot candidates near the estimated position of the other corner dots estimated by the corner dot position estimation unit, with the target pixel at the estimated position as the center. The predetermined range is detected by scanning in a spiral manner from the estimated position as a starting point .

According to the present invention, a dot can be detected at high speed by scanning from a predetermined dot. In addition, corner dots can be detected by detecting dots that are point-symmetric with respect to dots estimated as corner dots.

In one embodiment of the present invention, the corner dot coincidence determining unit is configured to determine the first corner dot estimated by the corner dot position estimating unit based on the position of the corner dot determined to be a corner dot by the corner dot determining unit. When the distance between the position and the position of the first corner dot detected by the corner dot detection means is within a predetermined threshold, it is determined that the corner dot matches.

According to the present invention, if the distance between the detected position of the first corner dot and the position of the first corner dot estimated from the other corner dots is within a predetermined threshold, the corner frame is It can be determined that it has been detected.

  A high-speed and versatile barcode reader, barcode reading method and barcode reading program could be provided.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[2D barcode]
First, a two-dimensional barcode (hereinafter simply referred to as a two-dimensional code) used in the present embodiment will be described. FIG. 1 shows an example in which a part of a predetermined area of a printed document used in this embodiment is enlarged. Each of the regions 2a, 2b, 2c, and 2d surrounded by the dotted line is one two-dimensional code. The two-dimensional code is composed of a plurality of optically readable black dots, and each two-dimensional code is arranged in a matrix form vertically and horizontally. The lines of H1, H2, and H3 and the lines of V1, V2, and V3 that are composed of continuous dots in FIG. 1 are the vertical and horizontal boundaries of each two-dimensional code. The intersection of the vertical line H1 and the horizontal line V1, for example, the dot 15 is referred to as a corner (corner) of the two-dimensional code.

  A two-dimensional code and a boundary (hereinafter simply referred to as a two-dimensional pattern) are printed simultaneously with the contents of the document when the document is printed. The two-dimensional code 2a and the like are encoded with identification (ID) information of an electronic document that is an original of a printed document and information indicating coordinates where each two-dimensional code is located.

  For example, “horizontal coordinate = 95, vertical coordinate = 10, document ID = 10” is encoded in the two-dimensional code 2a, and “horizontal coordinate = 96, vertical coordinate = 10, document ID = 10” is encoded in the two-dimensional code 2b. The two-dimensional code 2c is encoded with “horizontal coordinates = 95, vertical coordinates = 11, document ID = 10”, and the two-dimensional code 2d is encoded with “horizontal coordinates = 96, vertical coordinates = 11, document ID = 10”. Yes.

  Here, the dots constituting the two-dimensional code and the size of the two-dimensional code will be described with reference to FIG. The dot 201 is an example of one dot constituting the two-dimensional pattern, and the cell 202 is an example of the size of the minimum dot that can be printed by the printer to be used.

  The dots 201 constituting the two-dimensional pattern are printed in 2 × 2 units of the minimum dots 202 of the printer to be used. For example, in the case of a 1200 dpi printer, the minimum dot diameter of the printer is about 21 μm, so the dot diameter of the two-dimensional code is ideally about 42 μm. Actually, the diameter is a little larger due to the dot gain. In this embodiment, the dots are arranged in the horizontal and vertical directions with an interval 6 times the diameter of the dots. In this case, even when all dots are present at the dot arrangement position, the area ratio occupied by the dots is ideally 2.8%, and even if a dot gain of about 50% is expected, the area ratio is less than 5%. At a black occupancy rate of about 5%, it looks light gray to the human eye, so there is no problem that it is difficult to read a document or a written character by being disturbed by dots.

  When the dot arrangement is determined as described above, in the present embodiment, each two-dimensional code is composed of eleven vertical dots and seven horizontal dots, so the size of each two-dimensional code is horizontal (horizontal). It becomes 2 mm and a vertical (vertical) 3 mm square. Each position of the two-dimensional code is represented by using, for example, the position of the upper left corner of the page as the origin. For example, the horizontal coordinate = 95 means that it is the 95th two-dimensional code in the horizontal direction and is at a position of 190 mm in the horizontal direction from the origin. Similarly, the vertical coordinate = 10 is the tenth two-dimensional code in the vertical direction, and means that the position is 30 mm in the vertical direction from the upper left origin on the paper surface.

  Although details will be described later, the size of the horizontal 2 mm and vertical 3 mm square is similar to that of the A4 size paper, and the horizontal / vertical coordinate information can be stored effectively.

  The document ID will be described. The document ID is unique information of the original electronic document, for example, an identification ID of a database in which the electronic document is stored. That is, since the document ID of the two-dimensional code 2a is 10, the upper left corner (the intersection of the vertical line H1 and the horizontal line V1) of the two-dimensional code 2a is “position of 190 mm horizontally and 30 mm vertically from the upper left of the page. , The printed document is a document whose ID is 10. "

  Next, details of data arranged in the two-dimensional code will be described with reference to FIG. FIG. 3 is a diagram showing the two-dimensional code 2a of FIG. The two-dimensional code 2a has 7 × 11 cells surrounded by H1, H2, V1, and V2 constituting a code frame. A cell refers to a place where dots can be placed. Therefore, the two-dimensional code of this embodiment can include a maximum of 77 dots.

  In this embodiment, the two-dimensional code is divided into several areas, and predetermined information is arranged in each area. An area 401 is an area for arranging data representing horizontal coordinates, and has 4 × 2 cells. Since it corresponds to 1 bit per cell, it has a capacity of 1 byte. An area 402 is a place where data representing vertical coordinates is arranged. Similar to the horizontal coordinate area, it has a capacity of 1 byte.

Since A4 size paper is 210mm horizontally and 297mm vertically, one size of the two-dimensional code is 2mm horizontally and 3mm vertically, so 105 in the horizontal direction and 99 in the vertical direction. Can be arranged. Since the number that can be expressed in 1 byte is 2 ⁸ = 256, the position information on A4 size paper can be displayed in 1 byte. When the paper size is A3, the size is 297 mm horizontally and 420 mm vertically, so 148 codes are arranged in the horizontal direction and 120 two-dimensional codes in the vertical direction. In this case, a data capacity of 1 byte is sufficient. In the case of the A2 size, the size is 420 mm in the horizontal direction and 594 mm in the vertical direction, so 210 two-dimensional codes can be arranged in the horizontal direction and 198 in the vertical direction. In this case, a data capacity of 1 byte is sufficient. .

  In this way, if the vertical and horizontal lengths of the two-dimensional code are appropriately determined according to the vertical and horizontal lengths of the paper, a wide range of sizes can be achieved even if the number of bits representing the horizontal and vertical coordinates (data length) is fixed to a fixed size. Can be used with any paper.

  Next, an area 403 is a place where data representing a document ID is arranged. It has a capacity of 4 × 6 24 bits (= 3 bytes). Regions 404 to 407 are places where codes for error correction are arranged. Each has a capacity of 1 byte, and the error correction code is composed of a total of 4 bytes. Regions 408 and 409 are patterns for representing the vertical direction of the two-dimensional code, the region 408 is a 3 × 1 black dot, and the region 409 is a pattern without a 2 × 1 dot, and the upper and lower sides of the two-dimensional code are discriminated. Used for.

  Next, the bit string arranged in each area will be described. FIG. 4 shows how bit strings of data such as horizontal coordinates are arranged in each area. In the areas 401 to 407, 1 indicates the MSB (most significant bit) and 8 indicates the LSB (least significant bit).

[Document Management System]
Next, a document management system using a two-dimensional code will be described. FIG. 5 shows a schematic configuration diagram of a document management system using a two-dimensional code. The document management system in FIG. 5 includes a printer 601, a copier 602, a scanner 603, information processing devices 604 and 609 such as a personal computer, a storage device 605, a portable information terminal 608, and a pen-type coordinate input device 607 via a network. Each is connected. The storage device 605 includes a document management database 606 and a document to be printed. The document management database 606 is a database for managing each document such as adding an ID for identification to each document.

  In this embodiment, the two-dimensional code is read, and the ID number and position of the currently handwritten document are output to an information processing apparatus or the like. A pen-type coordinate input device 607 in FIG. 5 will be described as a device for reading this two-dimensional code.

  The pen-type coordinate input device 607 includes a writing instrument-like device main body 807 that can be held by a person and perform a writing operation. A writing instrument 813, that is, a tip end portion of a ballpoint pen, a mechanical pencil, or the like is attached to the leading end portion 805 of the apparatus main body 807 so that writing can be performed on a document. An image reading device 806 provided on the side of the apparatus main body 807 is configured by a photoelectric conversion element 806a such as a CCD and an optical system 806b including a lens, and reads an image on a print document. Note that the image reading device 806 may be provided with illumination as necessary.

  A microcomputer 808 is mounted on the apparatus main body 807, and an image reading apparatus 806 is electrically connected to the microcomputer 808. Various processes based on the image of the printed document read by the image reading device 806 are performed by the microcomputer 808. That is, the read two-dimensional code is decoded and the coordinates of the two-dimensional code on the paper surface and the document ID are detected. A method for reading the two-dimensional code will be described later. Further, the microcomputer 808 can be connected to an information processing apparatus 604 such as an external personal computer other than the apparatus main body 807, and data stored in the microcomputer 808 can be output to the information processing apparatus 604. In FIG. 5, illustrations of a power source for supplying power to the image reading device 806, the microcomputer 808, and the interface between the microcomputer 808 and the information processing device 604 are omitted.

FIG. 6 shows a configuration diagram of the microcomputer 808. In the microcomputer 808, a CPU (central processing unit) 881, a ROM (read only memory) 882, a RAM (random access memory) 883, and a two-dimensional code reading device 884 are electrically connected via a bus 880. Various external devices are electrically connected via a bus 880. The ROM 882 contains a program for controlling the operation of the pen-type coordinate input device 607 and a program for operating the microcomputer 808 in advance. The RAM 883 temporarily stores images read from the image reading device 806, intermediate data generated during code reading, and document IDs and coordinates obtained when decoding a two-dimensional code. Although the two-dimensional code reading device 884 will be described later, the two-dimensional code is detected from the image stored in the RAM 883 and the read document ID and coordinates are detected.
Further, the microcomputer 808 is connected to an LCD display device 809, an LED 810, or a buzzer 811. When the information received from the information processing device 604 is displayed on the LCD display device 809 or specific information is received, the LED 810 is displayed. Can be flashed or a buzzer 811 can be sounded to notify the outside.

The apparatus main body 807 is provided with a pressure sensor 812 that detects whether the tip portion 805 is in contact with the writing surface. That is, the pressure applied to the tip 805 when the tip 805 contacts the writing surface is transmitted to the pressure sensor 812 via the writing tool 813. The pressure sensor 812 senses this pressure and transmits the sensed information to the microcomputer 808.

By using the pen-type coordinate input device 607, if the position of the leading end 805 on the printed document is continuously detected, the movement locus of the leading end 805 on the printed document can be obtained. Further, as described above, the writing trajectory when writing on the paper surface can be obtained faithfully by the pressure sensor that detects whether or not the tip portion 805 is in contact with the writing surface.

  If a document that does not have a two-dimensional code is added to the document, the code cannot be read although writing is in progress, and an unreadable display is made using the LCD 809 or the LED 810 to notify the user.

[Printing a document with a two-dimensional code printed]
Next, an example of printing a document on which a two-dimensional code is printed will be described with reference to FIGS. 5 and 7. FIG. 7 shows a flowchart of processing from creating a two-dimensional code, superimposing the generated two-dimensional code on a document to be printed, and printing the document.

  The two-dimensional code is created by a printer driver included in the information processing apparatus 604 or 609 when a document print command is executed in the information processing apparatus 604 or 609. First, the printer driver makes an inquiry to the document management database 606 in the storage device 605 and issues a document ID for each page of a document to be printed (S501).

  Next, coordinate data indicating coordinate information is determined according to the size of the paper (S502). When the document ID and coordinate data are issued, the document ID and the coordinate information on the document are combined to create data to be encoded. For example, the document ID is 123456, the coordinate information is in mm units (24, 123), and so on.

  Next, the data is encoded. Data encoding processing (S503) and error correction code addition (S504) processing will be described with reference to FIG. The document ID is converted from a 6-digit number to a binary value of 3 bytes. Further, since the horizontal coordinate is 24 mm when expressed in mm from the origin at the upper left of the paper surface, it is divided by the horizontal length 2 mm of the two-dimensional code 24/2 = 12, and the vertical coordinate is 123 mm in the coordinate information. Therefore, it is 123/3 = 41 divided by the horizontal length of 3 mm of the two-dimensional code. As described above, the horizontal and vertical coordinates each fit in one byte.

  The coordinate information is thus converted into a 2-byte binary value and is combined with the document ID into a total of 5 bytes of data (S503 in FIGS. 7 and 8).

  After the data is encoded, an error correction code is added based on the encoded data (S504 in FIGS. 7 and 8). In the example of FIG. 8, a 4-byte error correction code is added to 5-byte data. A Reed-Solomon code is used for this error correction code. The Reed-Solomon code is a powerful error correction method that can correct an error in byte units, and can correct an error that is less than half of the error correction code length. The details of Reed-Solomon error correction codes are described in many books such as Shogodo "Code Theory (Computer Fundamental Course 18)" by Miyagawa, Iwabuchi, and Imai. In the case of the present invention, when the error correction code length is 4 bytes, error correction of up to 2 bytes is possible.

  The encoded data and error correction code data created in this way are assigned to each area of the two-dimensional code described with reference to FIG. 3, and a matrix image in which the two-dimensional code is arranged on the entire page is created (S505). Since the processing for one page is completed (Yes in S506), the processing is repeated until all pages are completed (S507).

  Thereafter, printing is performed using the printer unit of the printer 601 or the copying machine 602. When the printing is normally completed, the printer driver registers information of the document, the number of pages, and the document ID that are normally printed in the document management database 606.

  FIG. 9 is a flowchart showing a flow in which an operator prints a document on which a two-dimensional code is printed against the background of the process of FIG. First, the operator edits a document stored in the storage device 605 in FIG. 5 as necessary (S901). When the operator desires to print the document, it issues a print command (S902). At this point, the processing described with reference to FIG. 7 is performed, and the document edited in S901 is superimposed on the sheet on which the two-dimensional code is arranged, and the document is printed by the printer 601 or the copier 602. If printing has been completed normally (Yes in S903), the printer driver registers the document ID that has been successfully printed in the document management database 606 (S904).

  FIG. 10 shows an example of a document on which a two-dimensional code is superimposed and printed. A print document 1001 is an example of a form document having a predetermined format, and a two-dimensional pattern is printed on the background. The enlarged part 1003 is an enlarged part 1002 of the print document 1001. In the enlarged portion 1003, the dot 1004 corresponds to the region 408 in FIG. Black dots other than the dot 1004 are examples of code frames, and white dots are examples of data dots.

  Up to this point, since the document on which the two-dimensional code is printed has been printed, the reading of the two-dimensional code will be described in detail.

[2D code reader]
FIG. 11 is a functional configuration diagram of the two-dimensional code reading device 884 described in FIG. First, a paper image (monochrome 8-bit image) read by the image reading device 806 in FIG. 5 is input, and a code position detector 1101 selects one two-dimensional code frame from a plurality of two-dimensional codes in the image. Is detected. Details will be described later. An example of the photographed image is shown in FIG. In FIG. 12, a plurality of two-dimensional codes composed of dots are photographed.

  If the position of the two-dimensional code can be detected, the data acquisition unit 1102 acquires 0 or 1 data according to each monochrome cell (position where the dot is to be arranged) of the two-dimensional code, and the data arrangement of the two-dimensional code Data is sorted according to the rules. Next, the data replacement unit 1106 responds to the signal indicating whether or not the continuous writing is output from the continuous writing detector 1105 with respect to the acquired data. A known document ID is read from the memory 1108, and the data replacer replaces a portion corresponding to the document ID of the data acquired from the two-dimensional code with the known document ID. Here, although not shown in the pen-type coordinate input device 807, the continuous writing detector 1105 inputs an output signal of a pressure sensor that detects pressure felt at the pen tip when writing, and determines whether writing is in progress. The output of the writing detector is input, and if the writing state continues for a predetermined time or longer, it is determined that continuous writing is in progress, and otherwise it is determined that continuous writing is not in progress, and the signal is output.

  An example in which data is replaced based on the document ID is shown in FIG. In the table of FIG. 13, the first row is correct data, that is, data that has been encoded and added with an error correction code before being read. In the second line, a two-dimensional code is extracted from the image and data is reconstructed from the code dots. There is an error in the first and second Y coordinates and ID information. In this case, since there are three errors, error correction is impossible. However, as shown in the third line, by replacing the document ID as the known information, the error of the ID portion is eliminated, and the error is only 1 byte of the Y coordinate. Is obtained.

  Subsequently, the data replacer 1106 outputs the data replaced by the error corrector 1107 and performs correction processing. The error corrector 1107 outputs determination information as to whether error correction has been successful and data after error correction. This error-corrected data is a document ID and coordinate information. The output error correction determination information is input to the data decoder 1109, the nib coordinate calculator 1110, and the known information memory 1108, and is used to control each device. The data decoder 1109 operates when the error correction determination information indicates successful error correction. If error correction fails, the data decoder does not operate.

  The nib coordinate calculator 1110 calculates and outputs nib coordinates when the error correction determination information indicates that the error correction is successful and the output of the writing detector is writing. If the error correction is unsuccessful or not being written, a coordinate value ((-1, -1), etc.) that is impossible in reality is output. Further, in the known information memory 1108, when the error correction is successful, a portion corresponding to the document ID of the error-corrected information is newly stored.

  Data that has been successfully corrected is decoded by the data decoder 1109 into coordinate information and document ID on the page. In the example of FIG. 13, the ID on the paper is horizontal coordinate = 24 mm, vertical coordinate 123 mm, and document ID = 23. Using the coordinates and the image of the two-dimensional code that has been successfully decoded, the pen tip coordinate calculator calculates coordinates on the paper surface of the pen tip (details will be described later). As a result, the position of the tip 805 of the pen-type coordinate input device 607 is determined. As described above, the document ID and the sitting amount of the nib on the paper surface are acquired.

(Code position detector)
Details of the code position detector 1101 will be described. An example of a functional configuration diagram of the code position detector 1101 is shown in FIG. As shown in FIG. 14, the code position detector 1101 detects a dot and its position from the input image, and detects a dot constituting the code frame from the detected dot image, and determines the code position. Code frame detector 1402 for

・ Dot detector 1401
A method in which the dot detector 1401 detects dots will be described with reference to FIG. A predetermined pixel is determined from the photographed image, and the pixel is set as a target pixel Z. Whether or not a dot exists in the target pixel Z is determined as follows.

There are no dots already detected in the surrounding pixels of the pixel of interest Z, the pixels (A to H) indicated by hatching in FIG. 15, and the pixel value of the pixel of interest Z is any of the surrounding pixels (I to X). When the pixel value is smaller than a predetermined value (Th), it is detected that there is a two-dimensional code dot in the target pixel Z (the pixel value is represented by black = 0 and white = 255). That is,
There is no already detected dot in pixel (AH)
and
Pixel value of Z <peripheral pixel (I to X) −predetermined value (Th)
In this case, it is detected that there is a two-dimensional code dot in the target pixel Z.

  The setting of the predetermined value (Th) will be described. The smaller Th is, the easier it is to detect dots, but at the same time, noise is also detected as dots, which may cause errors in the detected data. If Th is large, the probability of detecting dots without detecting noise increases, but it becomes difficult to detect a code frame in which dots are continuous, and the coordinate acquisition rate is reduced. FIG. 16 shows the relationship between the magnitude of Th and the coordinate acquisition rate. Both the data when the document ID is replaced and the case where the document ID is not replaced are described.

Code frame detector 1402
Next, the code frame detector 1402 will be described. The code frame detector 1402 is configured as shown in FIG. FIG. 17 shows an example of a functional configuration diagram of the code frame detector 1402.

  The code frame detector 1402 inputs a dot image and a dot position and detects whether a dot in the image is a corner or not, and forms one end of the short side of the code frame together with the first corner. A second corner position estimator 1702 for estimating the position of the second corner, a second corner detector 1703 for detecting a true second corner in the vicinity of the estimated second corner position, and the second corner of the long side of the code frame A third corner position estimator 1704 that estimates the position of the third corner that forms one end, a third corner detector 1705 that detects a true third corner in the vicinity of the estimated third corner position, and a code frame together with the third corner A fourth corner position estimator 1706 for estimating the position of the fourth corner forming one end of the other short side, near the estimated fourth corner position A fourth corner detector 1707 for detecting the true fourth corner, and a first corner position estimator 1708 for estimating the position of the first corner that forms one end of the other long side of the code frame together with the fourth corner. The first corner coincidence determination unit 1709 determines whether the first corner position is in the vicinity of the first detected first corner and each detected corner is one code frame.

  For explanation, the first to fourth names are given to the corners, but the first corner dot may be any corner of the code frame of the two-dimensional code.

  FIG. 18 is a diagram illustrating a procedure in which the code frame detector 1402 detects a corner. The first corner detector 1701 determines whether or not a certain dot (X) is a corner of a two-dimensional code in an image in which dots are detected (details will be described later). The second corner position estimator 1702 estimates the position of a dot that is 8 dots away from the corner determined as the first corner in the direction of the code frame composed of dots (X → B, X → D). And the vicinity position of the 2nd corner dot (B, D) is detected among the estimated positions. The second corner detector 1703 detects whether or not there is a second corner in the vicinity of the second corner position estimated by the second corner position estimator 1702. The third corner position estimator 1704 estimates a position 12 dots away from the second corner B or D in the direction of the code frame composed of dots (B → G, D → E) in the clockwise direction. Then, the position near the third corner dot (G, E) is detected. The third corner detector 1705 detects whether there is a third corner in the vicinity of the third corner position estimated by the third corner position estimator 1704. The fourth corner position estimator 1706 tracks the position of the code frame 8 dots away from the third corner (G or E) in the clockwise direction, and determines the position near the fourth corner dot (G → C, E → A). To detect. The fourth corner detector 1707 detects whether there is a fourth corner in the vicinity of the fourth corner position estimated by the fourth corner position estimator 1706. The first corner position estimator 1708 estimates the position 12 dots away from the fourth corner (C or A) of the code frame composed of dots in the clockwise direction (C → X, A → X), A position near the corner dot (X) is detected. The first corner coincidence determination unit 1709 detects whether the true first corner detected by the first corner detector 1701 and the position estimated by the first corner position estimator 1708 are in the vicinity. It is determined whether or not the four corners form one code frame.

  Through the above procedure, the code frame detector 1402 in FIG. 14 detects the code frame of the two-dimensional code of either (1) or (2) in FIG. When the code frame is detected, the coordinates of the dots constituting the code frame can be detected. Which of (1) and (2) is to be processed earlier is determined by the direction having the dot closest to the dot X. If one of the code frames can be detected, the code frame detection ends. If one of the code frames cannot be detected, the other code frame is detected.

  Next, the first corner detection operation by the first corner detector 1701 will be described with reference to FIG. FIG. 19A shows an example of a functional configuration diagram of the first corner detector 1701, and FIG. 19B shows a diagram showing 17 × 17 pixels centered on the target pixel. As shown in FIG. 19A, the first corner detector 1701 detects surrounding dots in the image in which the two-dimensional code is photographed to detect whether there are four or more dots around the dot of interest. And a point-symmetric pair detector 1902 for detecting two pairs of dots that are point-symmetric with respect to the target dot from the detected surrounding dots.

・ Ambient dot detector 1901
The surrounding dot detector 1901 and the point symmetric pair detector 1902 will be described with reference to FIG. FIG. 19B is an image in which dots are detected, and the dots are detected in the pixel N. A dot of the pixel N is referred to as a noticed dot N, and other dots exist in the pixels A to F.

In surrounding dot detection, a predetermined length of Len centered on the target dot N
Among the x Len pixels, other dots are searched in a spiral shape from the target dot N, and a maximum of six dots existing around the target dot N are detected. A typical value for Len is 17. Since noise may be erroneously detected as a dot, not only the four dots necessary to create two pairs are detected, but a maximum of six dots are detected. If only three or less dots can be detected, it is determined that the dot N is not a corner dot, and the process is terminated. In the example of FIG. 19B, 6 dots A to F exist. This is detected as a surrounding dot.

Point symmetrical pair detector 1902
Subsequently, the point-symmetric pair detector 1902 detects two pairs of dots A to F that are close to the target dot N, have an intermediate point close to N, and have a small distance between the two dots. . In the case of FIG. 19B, two sets of (A, C) and (B, D) correspond to it. E and F are determined not to be noise or corner nearest dots. When the target dot N and two surrounding point symmetrical pairs are detected in this way, the target dot N is detected as the first corner.

-2nd-4th and 1st corner position estimator Next, the detail of the 2nd corner position estimator 1702 of FIG. 17 is demonstrated based on FIG. Here, the first, second, third, and fourth corner position estimators only change the corner name, and have the same configuration except for some parameters. Description is omitted. FIG. 20 is a diagram illustrating dots that form the vicinity of a corner of a two-dimensional code. X is a first corner dot detected by the first corner detector 1701, and A to I are dots constituting a code frame. Among these, A to D are known because they are detected by the point-symmetric pair detector 1902 of FIG. The dot Y is the estimated position of the second corner dot, and the dot Y ′ is the true second corner dot.

  The second corner position estimator 1702 estimates the position of a corner in the direction from X to the dot DEF. For the description of the corner position estimation processing, a coordinate vector starting from an appropriate origin and ending with dots A to I and X is used.

Because the second corner position is 8 dots away from the first corner,
In the second corner position estimator,
Y = (D−B) 8/2 + X
Calculate By subtracting vector B from vector D, a vector pointing in the Y direction from X can be obtained. Since this vector has a length of 2 dots, it is divided by 2, and the vector X is added to this to estimate the second corner dot Y separated by 8 dots from X. Therefore, the vector Y is an estimated vector of the second or fourth corner dot Y.

In the first and third corner position estimators, using the number of dots 12 in the vertical direction,
Y = (D−B) 12/2 + X
Calculated by Y is an estimated vector of the first or third corner dot Y.

Second to Fourth Corner Detector Since the position of the second corner has been estimated, the second corner detector 1703 will be described next. FIG. 21 shows an example of a functional configuration diagram of the second corner detector 1703. The second corner detector 1703 is a corner for detecting one or more dots around the pixel position estimated by the second corner position estimator (therefore, the pixel having the dot Y whose position is estimated). It comprises a dot candidate detector 2101 and a corner determiner 2102 for determining whether or not the detected corner dot candidate is a corner. The third and fourth corner detectors are the same as those of the second corner detector 1703, and thus description thereof is omitted.

  Details of the corner dot candidate detector 2101 will be described with reference to FIG. FIG. 22 shows an area having a size of Slen × Slen pixels centered on a pixel having a corner dot Y whose position is estimated. The typical size of Slen is 12. A pixel represented by diagonal lines in the center of the region in FIG. 22 is an estimated corner position. A dot is searched for in a spiral shape (dotted line in FIG. 22) outward from this center. In this case as well, six dots (numbers 1 to 6) in the region are detected in the same manner as the surrounding dot detection in FIG.

  Next, the corner determination unit 2102 in FIG. 21 determines whether or not the dot detected by the corner dot candidate detector 2101 is a corner dot. FIG. 23 shows an example of a functional configuration diagram of the corner determiner 2102. A corner determination unit 2102 is a FIFO storage device 2301 for storing the positions of dot candidates detected by the corner dot candidate detector 2101 and a corner for determining whether or not a dot at a pixel position where a corner dot candidate is a corner dot. A dot determination unit 2302. If there is a dot determined as a corner by the corner dot determination unit 2302, the position is output and the dot position information remaining in the FIFO storage device 2301 is discarded. If the corner dot determination unit 2302 does not determine that the corner is a corner, the next dot position is read from the FIFO, and the corner dot determination unit 2302 operates to determine whether the dot position is a corner. The corner dot determiner 2302 is performed with the same configuration as the first corner detector 1701 in FIG. That is, when the dot 1 in FIG. 22 is detected by the corner dot candidate detector 2101, the surrounding dot detection and the point-symmetric pair detection are performed with the dot 1 as the target dot N in FIG. 19. If the result of determination by the corner dot determiner 2302 is a corner dot, the position information of the second corner is output.

  When the positions of the second to fourth corners are sequentially detected as a result of the same processing, the first corner position estimator 1708 in FIG. 17 estimates the position of the first corner from the position of the fourth corner. The first corner coincidence determination unit 1709 shown in FIG. 17 has a predetermined first corner position estimated by the first corner position estimator 1708 and a first corner first detected by the first corner detector 1701. It is determined whether each corner determined this time is a corner that forms one code frame.

・ First corner coincidence determination unit 1709
To determine whether or not the first corner estimated by the first corner position estimator 1708 and the first corner detected in advance by the first corner detector 1701 are at a predetermined distance, the equation (1) or Equation (2) is used.

| Y'-Y |
<Dth1 (1)
Or
| Y'x -Yx |
<Dth1 and | Y'y-Yy | <Dth2 (2)
However, | Y′−Y | is the estimated distance between the first corner vector Y and the first corner vector Y ′. Y'x, Y'y
Are the x component and y component of the vector Y ′, respectively, and Yx and Yy are the x component and y component of the vector Y, respectively. Dth1 and DTH2 represent predetermined threshold values. A typical value is 10. Therefore, if the first corner estimated by the first corner position estimator 1708 and the first corner detected in advance by the first corner detector 1701 are closer than a predetermined threshold (distance), the first corner coincides. The determiner 1709 determines that the code frame of the two-dimensional code has been detected.

  Thus, the code position of the two-dimensional code is detected by the code position detector 1101 of FIG. Next, the data acquisition unit 1102 of FIG. 11 will be described with reference to FIG.

(Data acquisition device)
FIG. 24A shows a two-dimensional code composed of dots and a code frame. FIG. 24B shows a diagram representing the position of each dot of the two-dimensional code as a virtual locus. FIG. 24C is a flowchart illustrating an example of processing for acquiring two-dimensional code data. FIG. 24D is a flowchart illustrating another example of processing for acquiring two-dimensional code data.

  Hereinafter, a method for acquiring data of a two-dimensional code will be described with reference to FIG. In FIG. 24 (a), the corners of the code frame are a0, b0, c0, and d0, and the dots excluding the four corners of the code frame are a1 to a7, b1 to b11, c1 to c7, and d1. d11. In order to acquire data of the two-dimensional code, a vertical line connecting a1 to a7 and c1 to c7 and a horizontal line connecting b1 to b11 and d1 to d11 are drawn, and an intersection of the vertical line and the horizontal line is detected. The intersection is the position where the data dot of the two-dimensional code exists.

Projective transformation is used as one method for detecting the intersection of a vertical line and a horizontal line. FIG. 24B shows virtual coordinates of each dot of the two-dimensional code. Since the positions of the four corners are known, the coordinates corresponding to them are a0 and Av0 (0, 0), b0 and Bv0 (8, 0), c0 and Cv0 (8, 12), d0 and Dv0 (0, 12) is known. The projective transformation will be described in detail later. From these four corresponding points, a0 = f (Av0),
An 8-ary linear simultaneous equation that satisfies b0 = f (Bv0), c0 = f (Cv0), and d0 = f (Dv0) is created and solved to obtain the image coordinates of each data dot (S2401).

  Since f (x) is a function having eight unknown parameters, by solving the simultaneous equations, conversion from virtual coordinates to image coordinates in FIG. 24B becomes possible. When conversion from the virtual coordinates to the image coordinates becomes possible, projective conversion from the virtual coordinates (1, 1) to (7, 11) to the image coordinates becomes possible, and the presence position of the data dot can be detected ( S2402). However, since the position where the data dot exists is an ideal position, the actual dot does not necessarily exist at that position, and the data dot may be shifted by about one pixel. Therefore, the detection of data dots is performed by searching for a 3 × 3 pixel area centering on the existence position of the data dots obtained by projective transformation (S2403). If there is a dot in this area, the data at that position is “1”, otherwise it is “0”. This is performed for 77 virtual seats in the two-dimensional code to obtain code data.

  When the data of the two-dimensional code is acquired, the data is reconfigured according to the data arrangement rule described with reference to FIG. 4 (S2404). At this time, the determination of the vertical direction of the two-dimensional code refers to the areas 408 and 409 in FIG. 4, all the data existing in the area 408 is “1”, and all the data existing in the area 409 is “0”. Based on the assumption that it should be, it matches the direction with the high degree of coincidence with the acquired data.

  There are other methods for detecting the existence position of the data dot besides performing projective transformation. For example, as shown in the flowchart of FIG. 24D, points a1 to a7 are divided into 8 equally between a0 and b0, and points b1 to b11 and d0 to c0 are divided into 12 equally between b0 and c0. The points that divide the space into eight equal parts are c1 to c7, the points that divide a0-d0 into 12 parts are respectively d1 to d11, and the positions of the data dots are the line segment axcx and the line segment bydy (where x = 1,..., 7, y = 1,. If it is detected whether or not a dot exists in a 3 × 3 pixel area around the coordinates of the intersection, a data dot of a two-dimensional code can be obtained.

(Calculation of nib coordinates)
Since the data dot of the two-dimensional code photographed with the camera provided in the pen was obtained, calculation of the coordinates of the pen tip will be described next.

  FIG. 25 shows a functional block diagram of the nib coordinate calculator 1110 described in FIG. The pen tip coordinate calculator 1110 includes a projection parameter calculator 2501 and a pen tip coordinate converter 2502 as shown in FIG. First, the two-dimensional code corner coordinates on the image and the corresponding two-dimensional code corner coordinates on the paper surface are input to the projection parameter calculator 2501. Projective parameters are calculated from them. A method for calculating the projection parameters will be described in detail with reference to FIG.

  FIG. 26 is a diagram illustrating a method for calculating projection parameters. FIG. 26A shows the corners (As to Ds) of the two-dimensional code and the coordinates (Ps) of the tip end portion 805 of the pen-type coordinate input device 607 in the photographed image. In the pen-type input device 607, since the image reading device 806 and the tip portion 805 are fixed, the coordinate Ps is always constant. Depending on the configuration of the pen-type coordinate input device 607, the distal end portion 805 may be inside the image captured by the image reading device 806 or may be outside the image. If it is outside the image, its coordinates will exceed a negative value or maximum image coordinate.

  FIG. 26B shows the corners (Ar to Dr) of the two-dimensional code on the paper surface and the coordinates (Pr) of the tip 805 of the pen-type coordinate input device 607. Since Ar is determined by decoding the two-dimensional code, Br to Dr which are adjacent corners are automatically determined. Here, when the coordinates As to Ds and the coordinates Ar to Dr are obtained, a conversion formula is obtained from the relationship between the two.

変換係数ｂ１〜ｂ８は未知数であるが、コーナー座標の関係により８つの一次方程式が作られる。この連立方程式を解く事によりｂ１〜ｂ８のパラメータが算出される。このパラメータを用いることで、ペンの先端部８０５の画像座標Ｐｓを紙面上の座標Ｐｒに変換する。

Although the conversion coefficients b1 to b8 are unknown numbers, eight linear equations are created according to the relationship of the corner coordinates. The parameters b1 to b8 are calculated by solving the simultaneous equations. By using this parameter, the image coordinates Ps of the pen tip 805 are converted into coordinates Pr on the paper.

  To summarize the above, the corner coordinates As to Ds of the two-dimensional code on the image are calculated by the code position frame detector 1101 in FIG. 11, and the pen point coordinates Ps are known because they are fixed. The code corner coordinates Ar are obtained by decoding the two-dimensional code, and Br to Dr can be calculated because they are adjacent to Ar. Pr is obtained by Equation 1.

[Modification of the above embodiment]
Next, a modification of the two-dimensional code reader will be described. FIG. 27 is a functional configuration diagram of the two-dimensional code reading device in the present modification. The two-dimensional code reader of FIG. 27 differs from FIG. 11 in that there are two error correctors (first error corrector 1107a and second error corrector 1107b) and a selector 1111 accordingly. In addition, the same code | symbol is attached | subjected to the same component as FIG. 11, and the description is abbreviate | omitted. The functions of the first error corrector 1107a and the second error corrector 1107b are the same as those of the error corrector 1107 in FIG.

  The function of the selector 1111 will be described. When the continuous writing detector 1105 determines that continuous writing is in progress, the output of the second error corrector 1107b is always selected, and the same operation as that of the two-dimensional code reader of FIG. 11 is performed. When the output of the writing detector is writing and not writing continuously, that is, at the start of writing, the selector 1111 performs the following operation. When the error correction determination information output from the first error corrector 1107a is error correction success, the output of the first error corrector is selected. When the error correction determination information output from the first error corrector is an error correction failure, the output of the second error corrector is selected.

  When such processing is performed, when new writing is performed, if the same document is added, the error correction rate as the device is improved, which is more advantageous than the device of FIG.

[2D code reading by software]
The two-dimensional code reader 884 in FIG. 6 described so far reads the code by hardware, but it may be performed by software. In this case, a program that realizes the two-dimensional code reading method is stored in the ROM 882 of the microcomputer 808, and instructions of the program are sequentially loaded into the CPU and executed to execute the two-dimensional code reading process. .

  FIG. 28 is a flowchart showing a processing procedure by software. FIG. 28A shows a process for decoding a two-dimensional code, and FIG. 28B shows a process for an error correction procedure. First, after inputting an image, the position of the two-dimensional code is detected in S2801.

  Details of the code position detection step of S2801 are shown in the flowchart of the code position detection process of FIG. The flowchart in FIG. 29 includes a dot detection step (S2901) for detecting dots from an image and a code frame detection step (S2902) for detecting a code frame from the detected dots. The dot detection method in the dot detection step (S2901) is the same as the dot detection method described in FIG. FIG. 30 is a flowchart showing the process of the code frame detection step (S2902). Each step in the flowchart of FIG. 30 is configured to realize a function corresponding to each block of the code frame detector shown in FIG. That is, S3001 implements the function of the first corner detector 1701, S3002 implements the function of the second corner position estimator 1702, S3003 implements the function of the second corner detector 1703, and S3004 implements the third corner. The function of the position estimator 1704 is realized, S3005 realizes the function of the third corner detector 1705, S3006 realizes the function of the fourth corner position estimator 1706, and S3007 realizes the function of the fourth corner detector 1707. In step S3008, the function of the first corner position estimator 1708 is realized. In step S3009, the function of the first corner coincidence determination unit 1709 is realized.

  When the processing of FIGS. 29 and 30 is finished and the detection of the code position (S2801) of FIG. 28 is finished, in the data acquisition step (S2802), 0 or 1 data is acquired according to the black and white of the two-dimensional code. Sort the data. The data acquisition step (S2802) is executed according to the flowchart shown in FIG. 24 (c) or (d). Therefore, the processing content is equivalent to that realized by the function of the data acquisition unit 1102 in FIG.

  Subsequently, error correction of the data acquired in S2802 is performed (S2803). First, it is determined whether or not the acquired data is being continuously written (the pen-type coordinate input device tip 805 in FIG. 5 is always pressed against the paper surface) (S2811). ), Data replacement is performed with the known information (document ID) in the data replacement step (S2812). After data replacement, error correction is performed in an error correction step (S2813). If the error correction is successful (Yes in S2814), the known information (document ID) decoded in the step of saving the document ID (known information saving step S2815) is saved.

  Subsequently, the original data (coordinate information) is restored in the data restoration step (S2805), and the coordinates on the paper surface of the tip 805 of the pen-type coordinate input device 607 are obtained in the pen tip coordinate calculation step (S2806).

  In S2811, if the acquired data is not in continuous writing (the first addition) (No in S2811), the data is not replaced and error correction is performed in the error correction step (S2813). If the error correction is successful (Yes in S2814), the document ID (known information) is saved in the step of saving the document ID (known information saving step S2815), and the original data (coordinate information) is saved in the data restoring step (S2805). ) And the coordinates on the paper surface of the tip end portion 805 of the pen-type coordinate input device 607 are obtained in the pen tip coordinate calculation step (2806).

  With the above processing, the steps of code position detection (S2801), data acquisition (S2802), error correction (S2803), coordinate / document ID restoration (S2805), and nib coordinate calculation (S2806) are performed as shown in FIG. The functions of the position detector, data acquisition unit, error correction unit, data decoder, and nib coordinate calculator were realized by software.

[Other examples of 2D code reading by software]
FIG. 31 is a flowchart showing another example of two-dimensional code reading by software. In FIG. 31, the same steps as those in FIG. 28 are denoted by the same reference numerals, and the description thereof is omitted. The difference from the flowchart of FIG. 28 is that, when writing is in progress and continuous detection is not being performed (Yes in S3101 and No in S2811), error correction is performed without replacing the document ID at the start of writing (S2813b). . If error correction cannot be performed (No in S2814b), processing for error correction is performed by replacing the document ID portion (S2812, S2813). By this process, when adding to the same document, the error correction rate is improved by using the method of FIG. 31 compared to the method of FIG.

[Acquisition of added information, superimposing method on original document]
So far, the apparatus, system, method, and software for acquiring the added coordinates by reading the two-dimensional code have been described. An apparatus for acquiring the added information and superimposing it on the original document will be described. The system used is the one shown in FIG. 5, and the control flow is shown in the flowchart of FIG.

  Assume that a predetermined document is printed with a two-dimensional code. The pen-type coordinate input device 607 is used to write on the document. Then, as described above, the two-dimensional code on the document is read, and the document ID and coordinate information are acquired by the pen-type coordinate input device 607 (S3201). The document ID and coordinate information read in real time with the addition are transferred to the information processing apparatus 604 in FIG. 5 (S3202). When the information processing apparatus 604 receives the document ID and the coordinate information, the information processing apparatus 604 transfers the document ID to the information processing apparatus 609 that manages the document management database 606. Based on the received document ID, the information processing apparatus 609 identifies the currently added document (page) from the document management database 606 (S3203).

  FIG. 33 shows an example of registered items in the document management database 606. The database in FIG. 33 includes items of identification number (ID), document name, path name, creator, update date, page, total number of pages, and coordinate string file. As an example, it is assumed that the document specified by the identification number is page 1 of “PATENT.DOC” having the identification number (ID) 123456 of FIG. This document information is transmitted to the pen-type coordinate input device 607, and the received document information is displayed on the LCD display device 809 of the pen-type input device 607.

  Since the entity of the electronic document (file) such as the file name and page number has been specified, the information processing apparatus 604 opens “PATENT.DOC” with an application (for example, word processing software) associated with the information processing apparatus 604, Is displayed on a display (not shown) connected to the screen, and the editing state is set (S3204). The coordinate information sequentially transmitted from the pen-type coordinate input device 607 is drawn on the document “PATENT.DOC” in the edited state (S3205 to S3208). A drawing method can be realized by opening a new drawing object in a document window and drawing the received coordinates by connecting them with a line. At the time of completion of writing (Yes in S3209), the pen-type coordinate input device 607 transmits a signal for completion of writing to the information processing device 604, and the information processing device 604 that has received the signal for completion of writing draws in the window. Then, the update information and the corresponding document are updated / saved (S3210). In this way, one object is added to the document “PATENT.DOC”.

  In the embodiment described above, the added information is displayed on the computer display in real time, and it can be confirmed immediately whether the actually added information is processed correctly, so it is necessary to correct the contents added later. This is a great advantage for the user.

  Also, the detection method of the code frame of the two-dimensional code in the present embodiment is to detect corner dots and connect the corner dots with straight lines, so that the character code, ruled lines, dirt, image quality is poor, etc. Even if it is difficult to detect dots on the frame, the code frame cannot be detected immediately. Therefore, the probability of detecting the two-dimensional code (acquisition of coordinate data etc.) increases.

  In addition, the information processing apparatus 604 stores the received coordinate information as a number sequence text and at the same time as a separate file in the coordinate sequence file of FIG. This is useful because it is possible to display and check only the added data when the user wants to check the added data by opening the application later, such as when the document is huge, or when it is impossible to check it quickly. is there.

It is an example of the two-dimensional code in a present Example. It is a figure which shows the magnitude | size of the dot which comprises the two-dimensional code in a present Example, and a two-dimensional code. It is a figure which shows one two-dimensional code. It is a figure which shows the data array of a two-dimensional code. It is a schematic block diagram of a document management system. It is an example of the block diagram of a microcomputer. It is a flowchart figure which shows the process which prints the document on which the two-dimensional code was superimposed. It is a figure which shows the process of data encoding and error correction code addition. It is a flowchart figure which shows the flow which an operator prints the document on which the two-dimensional code was superimposed. It is an example of the document on which the two-dimensional code was superimposed and printed. It is a functional block diagram of a two-dimensional code reader. It is an example of the image by which the two-dimensional code was image | photographed. It is a figure which shows the example by which data is replaced based on document ID. It is an example of a functional block diagram of a code position detector. It is a figure for demonstrating the method of a dot detection. It is a figure which shows the relationship between Th (threshold) and a coordinate acquisition rate. It is an example of a functional block diagram of a code frame detector. It is a figure which shows the procedure in which a code frame detector detects a corner. It is a figure for demonstrating the detection method of a 1st corner. It is a figure for demonstrating the method of estimating the position of a 2nd corner. It is an example of the functional block diagram of a 2nd corner detector. It is a figure which shows the area | region of the magnitude | size of SlenxSlen pixel centering on the pixel which has a corner dot. It is an example of the functional block diagram of a corner determination device. It is a figure for demonstrating the data acquisition method of a data acquisition device. It is an example of a functional block diagram of a nib coordinate calculator. It is a figure for demonstrating the calculation method of a projection parameter. It is an example of the function block diagram of the two-dimensional code reader in a modification. It is a flowchart figure which shows the process of the two-dimensional code reading by software. It is a flowchart figure of a code position detection process. It is a flowchart figure which shows the process of a code frame detection step. It is a flowchart figure which shows the other process of the two-dimensional code reading by software. It is a flowchart figure which shows the update process of the document at the time of document addition and a document management database. It is a figure which shows an example of the registration item of a document management database.

Explanation of symbols

2a 2D code 15 corner dot 604 information processing device 606 document management database 607 pen type coordinate input device 806 image reading device 808 microcomputer 884 2D code reading device 1101 code position detector 1102 data acquisition device 1103 pressure sensor 1104 writing detector 1105 Continuous writing detector 1106 Data replacer 1107 Error corrector 1108 Known information memory 1109 Data compounder 1110 Nib coordinate detector 1111 Selector 1401 Dot detector 1402 Code frame detector 1701 First corner detector 1702 Second corner position estimation 1709 First corner coincidence determination unit 1901 Surrounding dot detector 1902 Point symmetrical pair detector 2101 Corner dot candidate detector 2102 Corner determination unit 2301 F FO memory device 2302 corner dot judgment unit 2501 projection parameter calculator 2502 pen coordinate converter

Claims (14)

A dot detection means for detecting dots constituting a two-dimensional code from a projected image obtained by photographing a two-dimensional code surrounded by a rectangular code frame composed of a plurality of dots and having corner dots arranged at the vertices of the rectangle A two-dimensional code reading device that reads information represented by a two-dimensional code based on the dots detected by the dot detection means;
A plurality of other dots within a predetermined range are searched for from the target dot N detected by the dot detection means, and two sets that are point-symmetric with respect to the target dot N among the plurality of dots existing around the target dot N Corner dot detection means for detecting that the dot of interest N corresponds to a corner dot when a pair of dots is detected ;
Among the corner dots detected by the corner dot detection means, the positions of other corner dots from the position of the predetermined first corner dot are set in advance and the position of a pair of dots that are point-symmetric with respect to the first corner dot. Corner dot position estimating means for estimating based on the number of dots between the corners ,
The plurality of corner dot candidates in the vicinity of the estimated position of the other corner dots estimated by the corner dot position estimating means are searched, and among the plurality of corner dot candidates, two sets of dots that are point-symmetric with respect to the corner dot candidate Corner dot determination means for determining that a corner dot candidate corresponds to another corner dot when a pair is detected;
Code frame detection means for detecting a code frame of a two-dimensional code from the positions of the first corner dot and the other three corner dots;
A two-dimensional code reading device comprising: Based on the position of the corner dot determined by the corner dot determination means to be a corner dot, the position of the first corner dot estimated by the corner dot position estimation means;
A corner dot match determination unit that determines that a corner dot forming a code frame is detected when the position of the first corner dot detected by the corner dot detection unit matches ;
The two-dimensional code reader according to claim 1. The corner dot position estimating means includes
The position of the corner dot detected by the corner dot detection means, the position of a pair of dots that are point-symmetric with respect to the corner dot, is represented by a vector starting from an appropriate origin.
Estimating the position of another corner dot in a direction obtained by subtracting the second vector connecting the other dot of the pair and the origin from the first vector connecting one of the pair of dots and the origin;
The two-dimensional code reader according to claim 1 or 2. The corner dot position estimating means includes
Dividing the vector obtained by subtracting the second vector from the first vector by 2 and multiplying it by the number of dots between the corners set in advance, the distance away from the corner dot detected by the corner dot detection means, Estimate the position of other corner dots,
The two-dimensional code reader according to claim 3. The corner dot determination means includes
A plurality of corner dot candidates in the vicinity of the estimated position of the other corner dots estimated by the corner dot position estimating means are spirally shaped starting from a position where a predetermined range centered on the target pixel of the estimated position is estimated. Scan to detect
The two-dimensional code reading device according to claim 2. The corner dot coincidence determining means is
Based on the position of the corner dot determined by the corner dot determination means to be a corner dot, the position of the first corner dot estimated by the corner dot position estimation means;
When the distance between the first corner dot position detected by the corner dot detection means is within a predetermined threshold, it is determined that the corner dot matches.
The two-dimensional code reading device according to claim 2 . A dot detection means for detecting dots constituting a two-dimensional code from a projected image obtained by photographing a two-dimensional code surrounded by a rectangular code frame composed of a plurality of dots and having corner dots arranged at the vertices of the rectangle A two-dimensional code reading method of a two- dimensional code reading device for reading information represented by a two-dimensional code based on the dots detected by the dot detection means;
The corner dot detection unit searches for a plurality of other dots within a predetermined range from the target dot N detected by the dot detection unit , and among the plurality of dots existing around the target dot N, a point related to the target dot N Detecting that the target dot N corresponds to a corner dot when a pair of two symmetric dots is detected ;
A dot in which the corner dot position estimating means is point-symmetric with respect to the first corner dot from the predetermined first corner dot position among the corner dots detected by the corner dot detecting means. Estimating based on the position of the pair of and the number of dots between preset corners ;
The corner dot determination means searches for a plurality of corner dot candidates in the vicinity of the estimated position of the other corner dots estimated by the corner dot position estimation means , and among the plurality of corner dot candidates, the corner dot candidate is point symmetric. Determining that a corner dot candidate corresponds to another corner dot when two pairs of dots are detected;
A step of detecting a code frame of a two-dimensional code from the positions of the first corner dot and the other three corner dots;
A two-dimensional code reading method comprising: The corner dot match determination means
Based on the position of the corner dot determined by the corner dot determination means to be a corner dot, the position of the first corner dot estimated by the corner dot position estimation means;
A step of determining that a corner dot forming a code frame has been detected when the position of the first corner dot detected by the corner dot detection means coincides ;
The two-dimensional code reading method according to claim 7, further comprising : The corner dot position estimating means includes
The position of the corner dot detected by the corner dot detection means, the position of a pair of dots that are point-symmetric with respect to the corner dot, is represented by a vector starting from an appropriate origin.
Estimating the position of another corner dot in a direction obtained by subtracting the second vector connecting the other dot of the pair and the origin from the first vector connecting one of the pair of dots and the origin;
9. The two-dimensional code reading method according to claim 7, wherein the two-dimensional code is read. The corner dot position estimating means includes
Dividing the vector obtained by subtracting the second vector from the first vector by 2 and multiplying it by the number of dots between the corners set in advance, the distance away from the corner dot detected by the corner dot detection means, Estimate the position of other corner dots,
The two-dimensional code reading method according to claim 9 . The corner dot determination means includes
A plurality of corner dot candidates in the vicinity of the estimated position of the other corner dots estimated by the corner dot position estimating means are spirally shaped starting from a position where a predetermined range centered on the target pixel of the estimated position is estimated. Scan to detect
The two-dimensional code reading method according to claim 8 . The corner dot coincidence determining means is
Based on the position of the corner dot determined by the corner dot determination means to be a corner dot, the position of the first corner dot estimated by the corner dot position estimation means;
When the distance between the first corner dot position detected by the corner dot detection means is within a predetermined threshold, it is determined that the corner dot matches.
The two-dimensional code reading method according to claim 8 . A dot detection means for detecting dots constituting a two-dimensional code from a projected image obtained by photographing a two-dimensional code surrounded by a rectangular code frame composed of a plurality of dots and having corner dots arranged at the vertices of the rectangle A computer having a two-dimensional code reading device that reads information represented by the two-dimensional code based on the dots detected by the dot detection means;
The corner dot detection unit searches for a plurality of other dots within a predetermined range from the target dot N detected by the dot detection unit , and among the plurality of dots existing around the target dot N, a point related to the target dot N Detecting that the target dot N corresponds to a corner dot when a pair of two symmetric dots is detected ;
A dot in which the corner dot position estimating means is point-symmetric with respect to the first corner dot from the predetermined first corner dot position among the corner dots detected by the corner dot detecting means. Estimating based on the position of the pair of and the number of dots between preset corners ;
The corner dot determination means searches for a plurality of corner dot candidates in the vicinity of the estimated position of the other corner dots estimated by the corner dot position estimation means , and among the plurality of corner dot candidates, the corner dot candidate is point symmetric. Determining that a corner dot candidate corresponds to another corner dot when two pairs of dots are detected;
A step of detecting a code frame of a two-dimensional code from the positions of the first corner dot and the other three corner dots;
A two-dimensional code reading program characterized in that A storage medium storing the two-dimensional code reading program according to claim 13.

Images