JP4397866B2 - Two-dimensional pattern reading device, two-dimensional pattern reading method - Google Patents

Description

  The present invention relates to a two-dimensional pattern reading apparatus and a two-dimensional pattern reading method for reading a two-dimensional pattern including document unique information and document position information.

A technique is known in which a two-dimensional code including encoded coordinate information is arranged on a print medium such as paper to specify the position and content of image information formed on the print medium. The conventional one-dimensional barcode has information only in the horizontal (horizontal) direction, whereas the two-dimensional code has information in two directions, horizontal and vertical, and therefore can represent more complicated information.
As a prior document, in Patent Document 1, optically readable code symbols are arranged in a matrix on the paper surface, and the code symbols are read by the camera at the same time as writing on the paper surface with a pen equipped with a small camera. A technique related to a system that reads writing information in real time by associating it with each other and associates the paper information before the writing with the writing information is disclosed.
Patent Document 2 is composed of a pattern representing the X-coordinate, Y-coordinate, and code direction, and a pattern that is larger than other patterns called homing patterns and arranged at the center of the code. Techniques have been proposed to prevent it from being visible.
Patent Document 3 discloses a two-dimensional code that can have information of 2 bits or more per dot by shifting the dot from a predetermined position. Such a two-dimensional code has coordinate information in the horizontal and vertical directions, and dots are continuously printed on the entire paper. By reading this, it is possible to determine what coordinate information the two-dimensional code has. The technology to acquire is disclosed.
Further, in Patent Document 4, a coordinate and document information are encoded, and a two-dimensional dot pattern printed on paper is read while writing with a pen with a camera. A technique for acquiring whether or not there is a medium is disclosed.
JP 2000-293303 A US Pat. No. 5,661,506 WO00 / 73981 JP 2004-94907 A

By the way, in Patent Document 1, information unique to a document and coordinate positions are converted into a monochrome pattern at the same time, printed on paper, and information added to the document on paper with an optical pen is read from the monochrome pattern. It acquires the information unique to the document and the coordinate position on the paper, and adds the additional information to the original document. Such a technique of Patent Document 1 uses a two-dimensional code having an error correction function for a black and white pattern, and has a function for acquiring document-specific information and coordinate information even when reading with an optical pen is unstable. Therefore, even when reading is impossible in the technique of Japanese Patent Laid-Open No. 7-141104, reading is possible.
However, in this case, since the size of the two-dimensional code becomes large, it is necessary to photograph a wide range of paper with an optical pen, and the quality of an image read due to problems such as depth of field is not good. It becomes difficult to read the code continuously. In addition, the two-dimensional code is not created to capture and decode the paper surface being written with a camera, but is a general-purpose two-dimensional code. Therefore, the decoding process takes time and can be processed in real time. difficult. One of the purposes of this technology is to acquire continuous writing information, so if the two-dimensional code is read and the coordinate position cannot be acquired continuously, discontinuities in the writing information will occur, and correct writing will occur. There was a drawback that information could not be obtained.
Further, in Patent Document 2 described above, it is practically impossible to print a code using a general printing apparatus, and it is necessary to print a code using invisible ink in a special printing apparatus in advance. In order to embed information on the document that the user wants to print in the print medium, it is necessary not only to have another means of printing with another barcode superimposed, but also to read the barcode before adding it to the document. This is inconvenient for the user.
If the user forgets to read the barcode and adds another document, the link between the added information and the electronic document causes a contradiction and becomes a serious problem. Moreover, even if it is printed with visible ink, a pattern (mark) larger than the other patterns is used, which is easily noticeable to humans, and has the disadvantage that the readability of documents and writings is impaired.

Further, in Patent Document 3, two-dimensional information having a code position information of 6 × 6 dots is obtained by placing two or more bits of information on one dot by disposing dots from two-dimensionally spaced points. The code is disclosed. With this technology, the amount of position shift is as small as 30 μm, and with the resolution (1200 dpi) of a laser printer that is generally popular, it is possible to reproduce the correct amount of shift due to paper feeding accuracy, unevenness of the rotational speed of the photoreceptor, and the like. difficult.
Therefore, even in the case of this two-dimensional code, it is necessary to prepare a paper on which the two-dimensional code is printed by using a printing technique such as offset printing in advance, so that there is a drawback that a problem equivalent to that of Patent Document 3 occurs. there were.
Patent Document 4 discloses a method for reading a two-dimensional code pattern without using the peripheral portion of the read image. Although a certain degree of improvement can be expected in speeding up, the processing speed is dramatically improved. Cannot be achieved.
In addition, there was a problem in the performance of the reading algorithm for reproducing the writing. Further, regarding the dot detection method, a 5 × 5 pixel window is used for the detection, but in the 5 × 5 pixel window, it is easy to make an error that a noise component is determined as a dot. Further, in the method of estimating the pen tip position, the coordinates of the corner dot detected from the read image do not match the center of the dot, and the accuracy of estimating the accurate position of the pen tip is not good. was there.
Therefore, the present invention has been made in view of the above-described points, and obtains information obtained by handwriting on a document printed on paper at higher speed and more accurately, and reflects the information in the original electronic document. An object of the present invention is to provide a two-dimensional pattern reading apparatus and a two-dimensional pattern reading method that can be performed.

To achieve the above object, an invention according to claim 1, the two-dimensional pattern composed of a plurality of dots inputs the captured image captured, the dot detecting means for detecting the dots in the dot detecting means code frame detection means for detecting a code frame from the detected dots, in a two-dimensional pattern reading apparatus and a data decoding means for decoding data read dot of the code frame, the dot detecting unit, the pixel of interest When a difference in luminance between the pixel and its neighboring pixels is equal to or greater than a predetermined value, a dot candidate is detected using a 7 × 7 pixel region centered on the pixel of interest among dots and noise included in the image. a dot candidate detecting means, in an area smaller than the area where the dot candidate is detected, a dot candidate and neighboring image detected by the dot candidate detecting means And the minimum luminance pixel determination means for determining a dot having the minimum luminance by comparing the preparative, no pixels which previously determined the dots adjacent pixels Dots candidates that have a minimum brightness in said minimum brightness pixel determining means In this case, it is characterized by comprising surrounding dot searching means for determining the dot candidate as a dot pixel .
The invention according to claim 2 is characterized in that the minimum dot luminance pixel determining means uses a 5 × 5 pixel area centered on the target pixel.
According to a third aspect of the invention, a two-dimensional pattern composed of a plurality of dots enter the captured image captured, the dot detecting means for detecting the dot code frame from the detected dots in the dot detecting means code frame detection means for detecting, in a two-dimensional pattern reading apparatus and a data decoding means for decoding data read dot of the code frame, the dot detecting means, a predetermined small area is searched dot A dot search area determination unit that determines whether or not the pixel is an area to be performed, and the luminance of the pixel of interest and its neighboring pixels for each pixel in the area determined as the dot search area by the dot search area determination unit if the difference is equal to or greater than a predetermined value, the out of dots and noise included in the image, using the area of 7 × 7 pixels around the target pixel of dot candidate Minimum determining a dot candidate detection means for detecting, in an area smaller than the area where the dot candidate is detected, the dot having the minimum luminance by comparing the detected dot candidate and neighboring pixels by the dot candidate detecting means a luminance pixel determination unit, when no pixels which have already been determined that a dot in the adjacent pixel of dots candidates that have a minimum brightness in said minimum brightness pixel determining means, surrounding dots determines the dot candidate dot pixel And a search means.

The invention according to claim 4 is the two-dimensional pattern reading device according to claim 3 , wherein the dot search area determination means sets a small area every other horizontal pixel and vertical pixel.
The invention according to claim 5, wherein the dot searching area is characterized dimensional pattern reading apparatus according to claim 3, wherein a small region consisting of 3 × 3 pixels.
The invention according to claim 6, wherein the dot minimum luminance pixel determination means is characterized dimensional pattern reading apparatus according to claim 3, wherein the use of the region of 5 × 5 pixels centered on the pixel of interest.
According to a seventh aspect of the present invention, in the two-dimensional pattern reading device according to any one of the first to sixth aspects, the position information of the data decoded by the data decoding means and the coordinates of the code frame in the image are used. includes a coordinate calculation unit for performing an operation on a given coordinate in the image coordinate Te, the coordinate calculation unit includes: a dot center detecting means for estimating the exact coordinates of the dots present in a given position of the code frame, the dot Projection parameter calculation means for calculating a projection transformation parameter from the center coordinates of the dot detected by the center detection means and the decoded position information, and using the projection parameter calculated by the projection parameter calculation means, And a coordinate conversion means for calculating a predetermined coordinate.

According to an eighth aspect of the present invention, the dot center detecting means has a dot coordinate correcting means for detecting a pixel having a minimum luminance in a small region including a dot, and a pixel detected by the dot coordinate correcting means. 8. The two-dimensional pattern reading device according to claim 7 , further comprising dot position estimating means for estimating an accurate position of a dot by performing a predetermined calculation in a small area.
The invention according to claim 9, the two-dimensional pattern composed of a plurality of dots enter the captured image captured, the dot detection step of detecting the dot code frame from the detected dots in the dot detecting step In a two-dimensional pattern reading method comprising: a code frame detection step for detecting an image; and a data decoding step for reading the dots in the code frame and decoding the data, the dot detection step includes the luminance of the pixel of interest and its neighboring pixels if the difference is equal to or greater than a predetermined value, among the dots and noise included in the image, and the dot candidate detection step of detecting a candidate of dots using an area of 7 × 7 pixels centered on the pixel of interest, the in an area smaller than the area where the dot candidate is detected, the dot candidate detection candidate dots and near detected by the step And the minimum luminance pixel determination step of determining a dot having the minimum luminance by comparing the pixel, the pixel that has already been determined that the dot in the adjacent pixel of Dots candidates that have a minimum brightness in said minimum brightness pixel determination step If no, the surrounding dots search step of determining the dot candidate dot pixel, characterized in that it consists of.
The invention according to claim 10, wherein the dot minimum luminance pixel determination step is characterized dimensional pattern reading method of claim 9 wherein using an area of 5 × 5 pixels centered on the pixel of interest.
The invention according to claim 11, the two-dimensional pattern composed of a plurality of dots enter the captured image captured, the dot detection step of detecting the dot code frame from the detected dots in the dot detecting step In a two-dimensional pattern reading method comprising a code frame detecting step for detecting the data and a data decoding step for reading the dots in the code frame and decoding the data, the dot detecting step is configured to search for a dot in a predetermined small region. A dot search region determination step for determining whether or not the region is to be performed, and a pixel of interest and its neighboring pixels for each pixel in the region determined as the dot search region determined in the dot search region determination step If the difference in luminance between is equal to or greater than a predetermined value, among the dots and noise included in the image, centered on the pixel of interest 7 A dot candidate detecting step of detecting a dot candidate using an area of 7 pixels in an area smaller than the area where the dot candidate is detected, the detected dot candidate and neighboring pixels in the dot candidate detecting step and the minimum luminance pixel determination step of determining a dot having the minimum luminance comparison, if no pixel is already determined dot adjacent pixels dots candidates that have a minimum brightness in said minimum brightness pixel determination step And a surrounding dot searching step for determining the dot candidate as a dot pixel.

The invention according to claim 12, the dot search area determination step is characterized dimensional pattern reading method according to claim 11, wherein setting a small region in the horizontal one pixel vertically every other pixel.
The invention according to claim 13, wherein the dot searching area is characterized dimensional pattern reading method according to claim 11, wherein a small region consisting of 3 × 3 pixels.
The invention according to claim 14 is characterized by the two-dimensional pattern reading method according to claim 11 , wherein the dot minimum luminance pixel determination step uses a 5 × 5 pixel area centered on the target pixel.
A fifteenth aspect of the invention is the two-dimensional pattern reading method according to any one of the ninth to fourteenth aspects, based on position information of data decoded by the data decoding step and coordinates of a code frame in an image. A coordinate calculation step for performing an operation on a predetermined coordinate in the image coordinates, the coordinate calculation step including a dot center detection step for estimating an accurate coordinate of a dot present at a predetermined position of the code frame, A projection parameter calculation step for calculating a projection transformation parameter from the center coordinates detected in the dot center detection step and the decoded position information, and a predetermined parameter in the image coordinates using the projection parameter calculated in the projection parameter calculation step. And a coordinate conversion step for adding a calculation to the coordinates.
According to a sixteenth aspect of the present invention, the dot center detection step includes a dot coordinate correction step for detecting a pixel having a minimum luminance in a small region including a dot, and a predetermined region in the small region centered on the detected pixel. 16. The two-dimensional pattern reading method according to claim 15, further comprising a dot position estimating step for estimating an accurate position of the dot by performing an operation.

  According to the present invention, it is possible to read a two-dimensional pattern with high accuracy, efficiency, and high speed, so that the reproduction quality of written information can be improved.

FIG. 1 is a plan view of a print document according to an embodiment of the present invention.
In FIG. 1, a two-dimensional code 22 is an optically readable code composed of a plurality of black dots, arranged in a matrix with black dots regularly arranged in the vertical and horizontal directions as a boundary. When the document 21 is printed, the pattern of the document and the two-dimensional code 22 are printed simultaneously. The two-dimensional code 22 includes, for example, identification (ID) information of an electronic document that is the original of the printed document on the surface of the printed document 21 on which the two-dimensional code 22 is printed, and information that indicates the coordinates of the two-dimensional code 22 Is encoded. In FIG. 1, for the sake of explanation, the document is not shown, and only a two-dimensional code such as a line spacing or a margin is printed.
For example, “horizontal coordinate = 95, vertical coordinate = 10, document ID = 10” is encoded in the upper left code symbol (two-dimensional code) 22a in FIG. 1, and “horizontal coordinate = 96, vertical” is encoded in the two-dimensional code 22b. “Coordinate = 10, document ID = 10”, the two-dimensional code 22c has “horizontal coordinate = 95, vertical coordinate = 11, document ID = 10”, and the two-dimensional code 22d has “horizontal coordinate = 96, vertical coordinate = 11, “Document ID = 10” is encoded.

FIG. 2 is a diagram showing the dot size and the dot interval of the two-dimensional code. The two-dimensional code dot and code size will be described with reference to FIG. The two-dimensional code 22 and the dots 23 constituting the boundary are printed in 2 × 2 units of the minimum dots 24 of the printer. In the case of a 1200 dpi printer, since the minimum dot diameter of the printer is 21 μm, the dot diameter of the two-dimensional code 22 is ideally 42 μm. Actually, the diameter is a little larger due to the dot gain. The positions where the dots 23 are arranged in the horizontal and vertical directions are determined with an interval of 6 times the dot diameter. In this case, even when all the dots are present at the dot arrangement position, the area ratio occupied by the dots is ideally 2.8%, and even if a dot gain of about 50% is expected, it is less than 5%. This area ratio looks light gray to the human eye, so there are no problems such as being difficult to see the document due to the dots being obstructed or being difficult to see the added text.
When the code arrangement is determined as described above, the two-dimensional code 22 has a horizontal size of 2 mm and a vertical size of 3 mm, and the horizontal coordinate = 95 means that the upper left corner of the two-dimensional code has the upper left corner of the page as the origin. It means that it is at a position of 190 mm in the horizontal direction. As will be described in detail later, the horizontal 2 mm and vertical 3 mm square sizes are similar to A4 size paper, and the data length for storing the horizontal / vertical coordinate information can be used effectively.
Similarly, the vertical coordinate = 10 means that the upper left corner of the two-dimensional code is at a position of 30 mm in the vertical direction with the upper left corner of the page as the origin.
The document ID is unique information of the original electronic document, for example, an identification ID of a database in which the electronic document is stored. That is, in the two-dimensional code 22, the upper left corner (the intersection of the vertical line H1 and the horizontal line V1) is located at a position of 190 mm horizontally and 30 mm vertically from the upper left of the page, and the ID of the printed document is “10”. Information.

FIG. 3 is a diagram showing a data arrangement area of a two-dimensional code, and details of data arranged in the two-dimensional code 22 will be described using this figure. The two-dimensional code 22a has 7 × 11 cells surrounded by H1, H2, V1, and V2 constituting a code frame. A cell is a unit in which dots can be hit, and includes a maximum of 77 dots.
An area 31 is an area for arranging data representing horizontal coordinates, and has 4 × 2 cells. Since one cell corresponds to one bit, the area 31 has a capacity of 1 byte.
The area 32 is an area in which data representing vertical coordinates is arranged, and has a capacity of 1 byte like the horizontal coordinates. A4 size paper has a horizontal size of 210 mm and a vertical size of 297 mm. One size of the two-dimensional code 22 is horizontal 2 mm and vertical 3 mm, so that data in 105 levels in the horizontal direction and 99 levels in the vertical direction are required, but 1 byte has sufficient capacity. Yes. When the paper size is A3, the size is 297 mm horizontally and 420 mm vertically, and therefore, data in 148 steps in the horizontal direction and 120 steps in the vertical direction are required. In this case, a data capacity of 1 byte is sufficient. Similarly, even in the case of the A2 size, since the size is 420 mm in the horizontal direction and 594 mm in the vertical direction, data in 210 steps in the horizontal direction and 198 steps in the vertical direction are necessary. In this way, when the vertical and horizontal lengths of the two-dimensional code are determined according to the difference in vertical and horizontal lengths of the paper, even if the data length representing the horizontal and vertical coordinates is fixed to the same fixed size, a wide range of paper sizes can be handled. It is like that.
The area 33 is an area for arranging data representing a document ID, and has a capacity of 4 × 6 24 bits (= 3 bytes). Regions 34 to 37 are regions for arranging codes for error correction and each have a capacity of 1 byte. Therefore, the error correction code is composed of a total of 4 bytes. The areas 38 and 39 are patterns for representing the vertical direction of the code. The area 38 is used to determine the top and bottom of the code with a pattern of 3 × 1 black dots and the area 39 is a pattern without 2 × 1 dots.

FIG. 4 is a diagram showing how the bit strings of each data are arranged. Note that “1” shown in the areas 31 to 37 in FIG. 3 indicates the MSB, and “8” indicates the LSB.
FIG. 5 is a diagram showing the configuration of the document management system as the present embodiment.
The document management system shown in this figure includes a printer 1, a copier 2, a scanner 3, an information processing device 4, a storage device 5, a document management database 6, a pen-type coordinate input device 7, a portable information terminal 8, an information processing device 9, The two-dimensional code creation device 10 is configured and connected via a network.
The printer 1 is a printing device that performs printing, the scanner 3 is a reading device that reads an image, and the copier 2 is a multifunction device having functions such as a copy, a scanner, a fax machine, and a printer. The information processing device 4 controls the pen-type coordinate input device 7 and the portable information terminal 8, and the information processing device 9 controls the storage device 5. The configuration of the pen-type coordinate input device 7 will be described later.
Next, in the document management system shown in FIG. 5, the above-described two-dimensional code 22 is created and used as print data to be superimposed on the document to be printed, and the procedure until the document is printed will be described with reference to FIG. The creation of the two-dimensional code 22 is executed by the printer driver in the information processing apparatuses 4 and 9 when the information processing apparatuses 4 and 9 shown in FIG.
In this case, the printer driver first asks the document management database 6 in the storage device 5 to issue a document ID for each page of the document 11 to be printed (S1). When the document ID is issued, the data to be encoded is acquired by combining it with the coordinate information on the document (S2). For example, the document ID shown in FIG. 7A is “123456”, and the coordinate information (X, Y) is in mm units (24, 123). Next, the data is encoded (S3). As a result, the document ID converts a 6-digit number into a 3-byte binary value as shown in FIG. Also, the horizontal coordinate is 24/2 = 12, the vertical coordinate is 123/3 = 41, and conversion is performed so that each horizontal and vertical can be accommodated by 1 byte. The coordinate information is thus converted into a 2-byte binary value and combined with the document ID, for a total of 5-byte data. After encoding, an error correction code is added based on the encoded data (S4). In the example shown in FIG. 7C, a 4-byte error correction code is added to 5-byte data. A Reed-Solomon code is adopted as the error correction code. The Reed-Solomon code is a powerful error correction method that can correct an error in byte units, and can correct an error that is less than half of the error correction code length. The details of Reed-Solomon error correction codes are described in many books such as Shogodo “Code Theory (Computer Fundamental Course 18)” by Miyagawa, Iwabuchi, and Imai. In the case of this embodiment, as shown in FIG. 7C, the error correction code length is 4 bytes, so that 2-byte error correction is possible.

The encoded data and error correction code data created in this way are assigned to each cell of the two-dimensional code 22 (S5), and in the subsequent step S6, it is determined whether or not the creation of the two-dimensional code for one page has been completed. I do. When one page is completed, it is determined in the next step S7 whether or not all pages have been completed. When all pages have been completed, the process is terminated. If it is determined in step S6 that the creation of the two-dimensional code for one page has not been completed, the process returns to step S2 to perform processing. If it is determined in step S7 that the creation of two-dimensional codes for all pages has not been completed, the process returns to step S1 to perform processing.
As a result, a matrix image in which the two-dimensional code is arranged on the entire page can be created. Thereafter, printing is performed using the printer unit of the printer 1 or the copying machine 2. When printing is completed normally, information such as a document, page, document ID, etc., that has been normally printed is registered in the ID management table in the document management database 6.
FIG. 8 is a flow of creating a two-dimensional code and using the print data superimposed on the document to be printed as viewed from the operator until a document is printed.
In this case, first, the document 11 stored in the storage device 5 of the document management system shown in FIG. 5 is edited (S11), a print command is issued, and printing of the document is executed by the printer 1 or the copying machine 2 (S11). S12). If printing is completed normally (S13), the document ID that has been successfully printed is registered in the ID management table in the document management database 6 (S14).
FIG. 9 is a diagram showing an example of a document printed with a two-dimensional code. A print document 21 shown in FIG. 9A is a form document having a predetermined format, for example, and a two-dimensional background. The code 22 is printed. FIG. 8B is an enlarged view of a part of the print document 21, which shows alignment dots that also serve as a code frame of the two-dimensional code 22 and data dots that are arranged in the frame. Two-dimensional codes 22 are regularly arranged.

Now that the document with the two-dimensional code has been printed, the schematic configuration of the pen-type coordinate input device 7 that performs writing next, captures an image of the paper, decodes the two-dimensional code, and obtains coordinate information is shown in FIG. This will be described with reference to the block diagram shown.
The pen-type coordinate input device 7 shown in FIG. 5 includes a writing instrument-like device main body 61 that can be held by a person and perform a writing operation. A writing tool 63, that is, a tip of a ballpoint pen, a mechanical pencil, or the like is attached to the tip 62 of the apparatus main body 61 so that the document can be added. The image reading device 64 provided on the apparatus main body 61 side includes a photoelectric conversion element 64a such as a CCD and an optical system 64b including a lens, and reads an image on the print document 21. The image reading device 64 can be provided with illumination as necessary. The reading resolution of the CCD is 320 × 240 pixels.
A microcomputer 65 is mounted on the apparatus main body 61, and an image reading device 64 is connected to the microcomputer 65. The microcomputer 65 performs various processes based on the image on the print document 21 read by the image reading device 64. That is, the read two-dimensional code 22 is decoded, and coordinates on the paper surface of the two-dimensional code are detected (details will be described later). The microcomputer 65 can be connected to the information processing apparatus 4 such as a PC outside the apparatus main body 61, and can output data stored in the microcomputer 65 to the information processing apparatus 4. In FIG. 6, the image reading device 64, a power source that supplies power to the microcomputer 65, and the interface between the microcomputer 65 and the information processing device 4 are not shown.

As shown in FIG. 10, the microcomputer 65 is connected to a CPU 71, a ROM 72, a RAM 73, and a two-dimensional code reader 74 via a bus 70. That is, the CPU 71 is connected to various external devices via the bus 70. In the ROM 72, a program for controlling the operation of the pen-type coordinate input device 7 shown in FIG. The RAM 73 temporarily stores an image read from the image reading device 64, intermediate data generated during code reading, a document ID and coordinates obtained when a two-dimensional code is decoded, and the like.
The two-dimensional code reading device 74 is configured as shown in FIG. 11, and detects the two-dimensional code from the image stored in the RAM 73 and reads it to detect the document ID and coordinates. The two-dimensional code reader 74 will be described later.
Further, an LCD display device 66, an LED 67, or a buzzer 68 is connected to the microcomputer 65 shown in FIG. Thereby, the microcomputer 65 displays the information received from the information processing device 4 on the LCD display device 66, or when specific information is received, blinks the LED 67 or sounds the buzzer 68 to the outside. You can be notified.
The apparatus main body 61 is provided with a pressure sensor 69 that detects whether or not the tip end portion 62 of the pen-type coordinate input device 7 is in contact with the writing surface. The pressure sensor 69 senses the pressure transmitted through the writing tool 63 when the tip end portion 62 comes into contact with the writing surface, and transmits the sensed information to the microcomputer 65.
If the position of the front end 62 on the print document 21 is continuously detected using the pen-type coordinate input device 7 configured as described above, the movement locus of the front end 62 on the print document 21 can be obtained. it can. Moreover, the writing locus when writing on the paper surface can be obtained faithfully by the pressure sensor 69 that detects whether the tip 62 of the pen-type coordinate input device 7 is in contact with the writing surface. If a document that does not have a two-dimensional code is added to a document that is not to be read by the pen-type coordinate input device 7, an unreadable display is displayed on the LCD display device because the code cannot be read despite writing. 66 or LED 67 is used to notify the user.

Next, the operation of reading the two-dimensional code will be described in detail.
FIG. 11 is a diagram showing the configuration of the two-dimensional code reader 74. As described above, the two-dimensional code reader 74 is built in the microcomputer 65 in the pen-type coordinate input device 7 shown in FIG. 5 (see FIG. 10).
In the two-dimensional code reading device 74 shown in FIG. 11, the image (8-bit image) on the paper surface read by the image reading device 64 is input to the dot detector 81, and the dot detector 81 detects the dot of the two-dimensional code 22. To do. The code frame detector 82 detects a frame of one two-dimensional code 22 and its position from a plurality of two-dimensional codes 22 in the image from the dots detected by the dot detector 81. An example of the read image is shown in FIG.
When the code frame detector 82 can detect the position of the two-dimensional code 22, the data acquisition unit 83 acquires “0” or “1” data according to each monochrome cell of the two-dimensional code 22. The data is rearranged according to the data arrangement rule of the dimension code 22.
The data replacer 84 reads the known document ID from the known information memory 88 based on the signal indicating whether or not the continuous writing is output from the continuous writing detector 91 with respect to the acquired data. The part corresponding to the document ID of the data acquired from the two-dimensional code 22 is replaced with a known document ID.
Detection information of the writing detector 92 is input to the continuous writing detector 91. The writing detector 92 receives an output signal of a pressure sensor 69 that detects a pressure felt at the pen tip when writing by the pen-type coordinate input device 7, and determines whether writing is in progress. When the writing state continues for a predetermined time or more, it is determined that continuous writing is being performed, and otherwise, it is determined that continuous writing is not being performed, and the detection result is output to the continuous writing detector 91.

The error corrector 85 performs error correction on the output of the data replacer 84. The error corrector 85 outputs determination information as to whether or not error correction has been successful and data after error correction. This error-corrected data is a document ID and coordinate information.
The error correction determination information output from the error corrector 85 is input to the data decoder 86, the nib coordinate calculator 87, and the known information memory 88, and is used to control each device.
In the data decoder 86, if the error correction determination information is error correction success, the data decoder 86 operates. If the error correction fails, the data decoder 86 does not operate. The nib coordinate calculator 87 calculates and outputs the nib coordinates if the error correction determination information indicates that the error correction is successful, and the output of the writing detector 92 is in writing. If it is not in writing, a coordinate value (-1, -1) or the like which is impossible in reality is output. Further, in the known information memory 88, when the error correction is successful, a part corresponding to the document ID of the error-corrected information is newly stored.
Here, a specific example of data replacement in the data replacer 84 is shown in FIG.
In this case, the first line shows data when correct data is encoded to generate an error correction code. The second line shows observation data obtained by extracting the two-dimensional code 22 from the image and reconstructing the data from the two-dimensional code dots. Here, if there is an error in the Y coordinate of the observation data and the first and second ID information, there are three errors in this case, and the error corrector 85 in the subsequent stage cannot perform error correction. End up.
Therefore, in order to prevent the error corrector 85 from performing error correction, the document ID is replaced with known information stored in the known information memory 88 as shown in the third row. As a result, there is no error in the part of the ID information, and only one byte of the Y coordinate becomes an error, so that error correction is possible and correct coordinate information and ID information can be obtained.
The data that has been successfully corrected is decoded by the data decoder 86 into coordinate information and document ID on the paper. The ID on the paper is horizontal coordinate = 24 mm, vertical coordinate 123 mm, and document ID = 23. Using the coordinates and the coordinates on the image of the two-dimensional code 22 that has been successfully decoded, the pen tip coordinate calculator 87 calculates the coordinates of the pen tip on the paper (details will be described later). Thereby, the position of the front-end | tip part 62 of the pen-type coordinate input device 7 is decided.

Next, the operation of the dot detector will be described with reference to FIG.
The area used for dot detection is a 7 × 7 pixel window. It is possible to make a determination that is more resistant to noise than the conventional 5 × 5 pixels. An area where dot detection is performed is an area 102 in FIG. Area 101 is the entire image area (320 × 240 pixels), and area 102 is an area of 280 × 220 pixels. In the outermost area, the image quality is not good and it may be difficult to perform correct dot detection, and we do not want to perform as wasted processing as possible for real-time processing. Dot detection is performed.
FIG. 15 is a diagram showing the configuration of the dot detector.
The dot detector shown in FIG. 15 includes a dot candidate detector (dot candidate detection unit) 101, a minimum luminance pixel determination unit (minimum luminance pixel determination unit) 102, a peripheral dot searcher (surrounding dot search unit) 103, And dot detection is performed in a 7 × 7 pixel area centered on the pixel Z.
The dot candidate detector 101 detects a dot candidate when the difference in luminance between the target pixel Z and its neighboring pixels is greater than or equal to a predetermined value.
The minimum luminance pixel determination unit 102 determines whether the dot candidate detected by the dot candidate detector 101 has the minimum luminance as compared with the neighboring pixels.
The peripheral dot search unit 103 checks whether there is a pixel already determined to be a dot around the dot candidate determined to be the minimum luminance pixel by the minimum luminance pixel determination unit 102, and if there is no dot in the periphery, the dot candidate is a dot It is finally determined that
The dot candidate detector 101 determines that the pixel Z is a dot candidate when the following eight expressions (1) to (8) are satisfied simultaneously. Th is a threshold value having a predetermined value, and for the sake of simplicity, each alphabet indicating the pixel position in FIG. 16 represents a pixel value. The input image represents black = 0 and white = 255.
Z <A-Th (1)
Z <E-Th (2)
Z <L-Th (3)
Z <P-Th (4)
Z <Q-Th (5)
Z <R-Th (6)
Z <S-Th (7)
Z <T-Th (8)

Next, the dot candidate satisfying the above eight expressions is compared with surrounding pixels by the minimum luminance pixel determination unit 102 to determine whether or not it has the minimum luminance. When the determination criteria satisfy all the following 12 formulas (9) to (20), it is determined that the dot candidate has the minimum luminance.
Z <B (9)
Z <C (10)
Z <D (11)
Z <F (12)
Z <G (13)
Z <H (14)
Z <I (15)
Z <J (16)
Z <K (17)
Z <M (18)
Z <N (19)
Z <O (20)
Next, as for the pixel Z determined as the minimum luminance pixel by the minimum luminance pixel determination unit 102, whether or not there is a pixel already determined to be a dot by the dot detector in the surroundings U, V, W, and X. Is detected by the surrounding dot searcher 103, and if there is no dot in the surroundings U, V, W, and X, it is first determined that the pixel Z is a dot.
Although the predetermined value (Th) is set, the smaller the Th is, the easier it is to detect dots, but at the same time, noise is also detected as dots, and errors may occur in the detected data. Also, if Th is large, errors can be reliably reduced without detecting noise, but it is difficult to detect code frames, and the coordinate acquisition rate is reduced. FIG. 17 shows an example of the coordinate acquisition rate according to the size of Th. Both the data when the document ID is replaced and the case where the document ID is not replaced are described.

FIG. 18 shows another configuration of the dot detector.
The dot detector shown in FIG. 18 is configured to perform the dot detection at a higher speed.
A dot search area determination device (dot search area determination means) 104 sets a small area 3 × 3 pixels for searching for dots. A dot candidate detector 101 for a 7 × 7 pixel window centered on each pixel in a 3 × 3 pixel region, and a pixel Z detected as a dot candidate by the dot candidate detector 101 is compared to surrounding pixels. A minimum luminance pixel determining unit 102 that determines whether or not the luminance is low, and a peripheral dot searcher 103 that checks whether there is a pixel that has already been determined to be a dot around a dot candidate that has been determined to be the minimum luminance pixel; The dot candidate is finally determined to be a dot if there is no dot in the vicinity by the peripheral dot searcher 103.
The dot search area determination unit 104 determines whether or not a dot search area is to be searched every other pixel in the horizontal and vertical directions. As shown in FIG. 16, when all the expressions (1) to (8) are satisfied with 7 × 7 pixels centered on the Z pixel, an area of 3 × 3 pixels centered on the Z pixel is obtained. It is determined that the region is to search for dots.
The dot candidate detector 101 and the minimum luminance pixel determiner 102 perform processing on each pixel in the 3 × 3 pixel region centered on the Z pixel. This processing has already been described.
Finally, the peripheral dot searcher 103 searches for a pixel already determined to be a dot around the pixel Z (U, V, W, X, Y). In the method of this embodiment, since there are pixels that are determined whether or not they are dots, it is necessary to determine whether or not dots have already been detected for the pixel Y as well.

Next, the configuration of the code frame detector will be described with reference to FIGS.
FIG. 19 is a diagram showing the configuration of the code frame detector. FIG. 20 is a diagram showing a detection path of the code frame detector.
As shown in FIG. 19, the code frame detector 82 includes a first corner detector 111 that determines whether a certain dot X is a corner of a code from an image in which dots are detected, and the first corner detector 111. A first dot tracker 112 that detects eight pixels of a code frame composed of dots in four directions and detects a second corner candidate pixel (B, D). Then, a second corner detector 113 for detecting whether or not the second corner candidate pixel detected by the first dot tracker 112 is the second corner, and 12 of a code frame composed of dots from the second corner. A second dot tracker 114 that detects the third corner candidate pixel (G, E) by tracking pixels in the dot tracking direction (clockwise direction) of the first dot tracker 112; Further, a third corner detector 115 for detecting whether or not the third corner candidate pixel detected by the second dot tracker 114 is the third corner, and the eight pixels of the code frame from the third corner are converted into the first number. A third dot tracker 116 that detects the fourth corner candidate pixel (C, A) by tracking in the dot tracking direction (clockwise direction) of the two-dot tracker 114, and the third dot tracker 116 detected by the third dot tracker 116. A fourth corner detector 117 for detecting whether or not the four corner candidate pixels are the fourth corner pixels, and a fourth dot tracking for detecting a dot constituting a code frame between the first corner and the fourth corner. Instrument 118. By this code frame detector 82, a maximum of two code frames of the two-dimensional code 22 can be detected, and the coordinates of the dots constituting the code frame on the image can be detected.

Here, the operation of the first corner detector 111 will be described with reference to FIG.
As shown in FIG. 21A, the first corner detector 111 includes a surrounding dot detector 121 that detects whether or not there are four or more dots around a pixel of interest where dots are present, and the detected surroundings. It is composed of a point-symmetric pair detector 122 for detecting two sets of dot pairs whose target is a target pixel from the dots.
The surrounding dot detector 121 detects dots existing around the pixel of interest N among 17 × 17 pixels centering on the pixel of interest N shown in FIG. In the case of FIG. 21B, there are six pixels A to F. This is detected as a surrounding dot pixel. Among these A to F, two sets are detected that are close to N and whose intermediate point is close to N. For example, in the case of FIG. 21B, two sets of (B, D) and (C, F) correspond to it. A and E are judged to be noise and are removed. When the dot N of the target pixel and two surrounding point-symmetric pairs are detected in this way, the target pixel N is detected as the first corner. The area to be detected by the first corner detector 111 is an area 103 of 180 × 120 pixels shown in FIG. 14, and has approximately half the size of the image area 101 in the vertical and horizontal directions. Further, the direction of searching for dots is not normal raster scanning, but spiral scanning from the inside to the outside as shown in FIG. The reason for this is that the vicinity of the center of the image has better image quality than the surroundings and the probability of reliable dot detection is high, so detecting the code pattern from or near the center of the image takes less time. The possibility that it can be detected is increased. Therefore, it is very advantageous for real-time processing.
The reason why the area 103 in which the first corner detector 111 operates is smaller than the area 101 of the entire image or the operation area 102 of the dot detector 81 is that, when the code pattern exists in the area 102, the innermost part of the code pattern inside the image This is because the area 103 is the existence range of the corners existing in FIG.

Next, details of the dot tracker will be described with reference to FIG. First, the details of the first dot tracker 112 will be described with reference to FIG. FIG. 23 shows the dots constituting the vicinity of the corner of the two-dimensional code. X is a corner dot, and other A to I are dots constituting a code frame. First, a dot is traced from X to DEF. For simplicity, A to I and X are regarded as coordinate vectors, and Y = 2D-X is calculated.
Y is an estimated vector of dot E. A to D are known because they are already detected by the first corner detector 111. The dot E is searched for in 5 × 5 pixels around Y. If E is present, calculate Y = 2E-D. This time, Y is an estimated vector of dot F. This tracking is repeated seven times to detect second corner candidate pixels.
There are four tracking directions (FIG. 20: X → A, X → B, X → C, and X → D), and the information of all the detected second corner candidate pixels is transferred to the second corner detector.
The basic operation of the second corner detector 113 is the same as that of the first corner detector 111. However, there is a possibility that a plurality of dots (up to 4) serving as the target pixel to be input exist. The maximum number of second corners detected is 2 (FIG. 20: B, D).
The basic operation of the second dot tracker 114 is the same as that of the first dot tracker 112. The difference is that there are two dots to start tracking. When tracking in one direction for each dot (FIG. 21: B → G, D → E), the dot tracking operation is repeated 11 times for each tracking. is there. In this way, the third corner candidate pixel (FIG. 20: G, E) is detected.
The third corner detector 115 detects whether or not the third corner candidate pixel (FIG. 20: G, E) tracked by the second dot tracker 114 is the third corner. The basic operation is the same as that of the first and second corner detectors 111 and 113. Assume that the third corner detector 115 detects G and E shown in FIG. 20 as being the third corner.

The third dot tracker 116 performs dot tracking in two directions (FIG. 20: G → C, E → A). The number of times of tracking is the same as that of the first dot tracker 112, and FIG. 20: C and A are detected as fourth corner candidate pixels.
The fourth dot tracker 118 has the same configuration as that of the second dot tracker 114. The fourth dot tracker 118 tracks 11 times in the directions of C → X and A → X shown in FIG. Since X is already known to be the first corner, the code frame is detected by the tracking path (1) and tracking path (2) shown in FIG. In this way, the code frame of the two-dimensional code is detected. As for which code is used when decoding a two-dimensional code, the one with the larger horizontal coordinate of the code center on the image is given priority. If this code cannot be decoded, the other code is used for decoding.
The image area tracked by the first dot tracker 112, the second dot tracker 114, the third dot tracker 116, and the fourth dot tracker 118 is the area 104 (260 × 200 pixels) in FIG. An area used for corner detection in the first, second, third, and fourth corner detectors 111, 113, 115, and 117 is slightly smaller than the area 102 where the dots are detected. This is because a point-symmetric pair dot is detected in the range using the pixel region.

Next, the data acquisition unit 83 shown in FIG. 11 will be described with reference to FIG.
In FIG. 24, it is set as a1-a7, b1-b11, c1-c7, d1-d11 except for the four points of the corner of a code frame. In order to acquire code data, the intersection of a horizontal line and a vertical line connecting them is detected. That is, the coordinates of the intersection of the straight line connecting a1 and c1 and the straight line connecting b2 and d2 are detected. Then, 3 × 3 detects the presence / absence of a dot in the area around the coordinates of the intersection. In the case of FIG. 24, there is no dot, and the acquired data is “0”. This is performed for all the code data to obtain the code data, and the data is reconstructed according to the data arrangement rule (FIG. 4).
Next, the nib coordinate calculator 87 shown in FIG. 11 will be described.
The nib coordinate calculator 87 includes a dot center detector 130, a projection parameter calculator 131, and a nib coordinate converter 132, as shown in FIG. The code center coordinates on the image, the corresponding code corner coordinates on the paper surface, and the captured gray scale image are input to the dot center detector 130, and the center position of the dot used for projection parameter calculation is estimated from them.
FIG. 26 is a diagram showing the configuration of the dot center detector 130.
As shown in FIG. 26, the dot center detector 130 includes a dot coordinate correction unit 133 that detects a pixel having the minimum luminance in a small region including dots, and a pixel detected by the dot coordinate correction unit 133 as a center. The dot position estimation unit 134 estimates the exact position of the dot by performing a predetermined calculation in the small area.
FIG. 27 is a diagram for explaining a projection parameter calculation method. The projection parameter calculation method will be described later.
In FIG. 27A, the coordinates of the pattern corners (As to Ds) and the pen tip (Ps) in the image are not integers representing the position of each pixel of the image, but are in contact with the center of the dot in the image and the pen tip paper. Represents the coordinates of the center of the part. That is, as shown in FIG. 28, the center of the dot does not necessarily come to the center of the pixel position, and the position is shifted. Here, by estimating the exact center of the dot, the accuracy of estimating the coordinates on the paper surface of the pen tip is improved.

（式２２）

上記式（２１）、式（２２）の二つの式を利用してＡｓ〜Ｄｓの４ドットの中心座標を決定する。そして、先に求めたＡｓ〜Ｄｓから射影パラメータを算出する。
図２７を用いて射影パラメータの算出方法について詳述する。
図２７（ａ）は画像におけるコードのコーナー（Ａｓ〜Ｄｓ）とペン型座標入力装置７の先端部６２（Ｐｓ）の座標である。Ｐｓの座標は常に一定である。なぜならばペン型座標入力装置７では画像読取装置６４と先端部６２は固定されているからである。先端部６２はペン型座標入力装置７の構成により画像読取装置６４で撮像される画像内にある場合もあれば画像外である場合もありうる。画像外にある場合は、その座標は負の値あるいは画像の座標の最大値を越えることになる。
図２７（ｂ）は紙面におけるコードのコーナー（Ａｒ〜Ｄｒ）とペン型座標入力装置７の先端部６２（Ｐｒ）の座標である。コードのデコードによりＡｒが決定されるので、その隣接コーナーであるＢｒ〜Ｄｒは自動的に決まる。算出したいのはＰｒの座標である。射影変換式は図２７（ｃ）に示す式により画像座標から紙面座標への変換が行われる。変換係数ｂ１〜ｂ８は未知数であるが、コーナー座標の関係により８つの一次方程式が作られる。この連立方程式を解く事によりｂ１〜ｂ８のパラメータが算出される。
ペン先座標変換器はこのパラメータを用いてＰｓをＰｒに変換して紙面上の先端部６２の座標を決定する。
ペン先座標変換器１３２はこのパラメータを用いてＰｓをＰｒに変換して紙面上の先端部６２の座標を決定する。 Next, a method for estimating an accurate dot center will be described.
As shown in FIG. 29A, a pixel position having a minimum pixel value in the vicinity of a 3 × 3 pixel centered on the pixel position of the dot detected by the dot detector 11 is detected in the original grayscale input image. To do. This pixel position is assumed to be (i, j). Further, the value of each pixel in the 3 × 3 pixel window centered on the pixel position (i, j) is represented by v (x, y) (where i−1 ≦ x) as shown in FIG. ≦ i + 1, j−1 ≦ y ≦ j + 1), and the center coordinates (ic, jc) are estimated using the following calculation formula.
(Formula 21)

(Formula 22)

The center coordinates of the four dots of As to Ds are determined using the two formulas (21) and (22). Then, a projection parameter is calculated from As to Ds obtained previously.
A method for calculating projection parameters will be described in detail with reference to FIG.
FIG. 27A shows the coordinates of the code corners (As to Ds) in the image and the tip 62 (Ps) of the pen-type coordinate input device 7. The coordinates of Ps are always constant. This is because in the pen-type coordinate input device 7, the image reading device 64 and the tip 62 are fixed. Depending on the configuration of the pen-type coordinate input device 7, the distal end portion 62 may be inside an image captured by the image reading device 64 or may be outside the image. If it is outside the image, its coordinates will exceed a negative value or maximum image coordinate.
FIG. 27B shows the coordinates of the code corners (Ar to Dr) on the paper surface and the tip 62 (Pr) of the pen-type coordinate input device 7. Since Ar is determined by decoding the code, Br to Dr which are adjacent corners are automatically determined. What is to be calculated is the coordinates of Pr. The projection conversion formula is converted from image coordinates to paper plane coordinates by the formula shown in FIG. Although the conversion coefficients b1 to b8 are unknown numbers, eight linear equations are created according to the relationship of the corner coordinates. The parameters b1 to b8 are calculated by solving the simultaneous equations.
The nib coordinate converter uses this parameter to convert Ps to Pr to determine the coordinates of the tip 62 on the paper surface.
The nib coordinate converter 132 uses this parameter to convert Ps to Pr to determine the coordinates of the tip 62 on the paper surface.

Next, another embodiment of the two-dimensional code reader is shown in FIG. In addition, the same code | symbol is attached | subjected to the same site | part as FIG. 11, and description is abbreviate | omitted. The two-dimensional code reader shown in FIG. 30 is different from the two-dimensional code reader shown in FIG. 14 in that a first error corrector 141, a second error corrector 142, and a selector 143 are provided. It is said.
The functions of the first error corrector 141 and the second error corrector 142 have the same functions as the error corrector 85 shown in FIG. The selector 143 always selects the output of the second error corrector 142 when the continuous writing detector 91 determines that continuous writing is in progress, and performs the same operation as the two-dimensional code reader shown in FIG. I do.
On the other hand, when it is determined from the output of the writing detector 92 that writing is in progress but not continuous writing, that is, at the start of writing, the selector 143 performs the following operation. When the error correction determination information of the first error corrector 141 is error correction success, the output of the first error corrector 141 is selected. When the error correction determination information of the first error corrector 141 is error correction failure, the output of the second error corrector 142 is selected.
When such a process is performed, the error correction rate as an apparatus is improved if the same document is added when a new writing is performed, which is advantageous over the two-dimensional code reading apparatus shown in FIG.
The two-dimensional code reading device 74 described so far reads the code by hardware, but it may be performed by software. In this case, a program for realizing the two-dimensional code reading method is a microcomputer 65 (see FIG. 5). ) ROM 72 (see FIG. 10). Then, the instructions of the program are sequentially loaded into the CPU 71, the instructions are executed, and the reading process of the two-dimensional code 22 is performed. The processing procedure will be described with reference to the flowcharts shown in FIGS.

First, after inputting an image, as shown in FIG. 31A, dots in the image are detected (S21). Next, after detecting a code frame for extracting the position of the two-dimensional code (S22), data “0” or “1” is acquired and rearranged according to the black and white of the two-dimensional code. (S23). Thereafter, error correction is performed in step S24, and if it is determined in step S25 that the error correction has been successful, the process proceeds to step S26. If it is not determined that the error correction has been successful, it is determined that error correction has failed. In some cases, the two-dimensional decoding process ends.
The processes in steps S24 and S25 surrounded by a broken line are as shown in FIG.
First, in step S31, it is determined whether or not the acquired data is being continuously detected, that is, whether the tip 62 of the pen-type coordinate input device 7 is always pressed against the paper surface. Data replacement is performed using (document ID). After data replacement, error correction is performed in step S33. If it is determined in step S34 that the error correction has been successful, the known information (document ID) decoded in step S35 is stored and the error correction process is completed. Thereafter, the process proceeds to step S26 shown in FIG. 31A, the original data (coordinate information) is restored, and the coordinates on the paper surface of the tip end portion 62 of the pen-type coordinate input device 7 are calculated in the subsequent step S27.
On the other hand, if the acquired data is not being continuously detected in step S31 of FIG. 31B, that is, if it is the first writing, the data in step S32 is not replaced, and error correction is performed in step S33. If it is determined in step S34 that the error correction has been successful, the document ID (known information) is stored in step S35, and the original data (coordinate information) is stored in step S26 shown in FIG. In the subsequent step S27, the coordinates on the paper surface of the tip end portion 62 of the pen-type coordinate input device 7 may be calculated.
The steps of dot detection, code frame detection, data acquisition, error correction, coordinate / document ID reconstruction, and pen tip coordinate calculation are the dot detector 81, code frame detector 82, and data acquirer shown in FIG. 83, the error corrector 85, the data decoder 86, and the nib coordinate calculator 87 are realized by software.

FIG. 32 is a flowchart showing dot detection processing when the detection method of the dot detector shown in FIG. 15 is realized by software.
In this case, first, in step S71, a dot candidate is detected for a 7 × 7 pixel region centered on the pixel Z, and in step S72, the pixel Z detected as a dot candidate is more than the surrounding pixels. It is determined whether or not the luminance is low. Thereafter, in step S73, it is determined whether there is a pixel already determined to be a dot around the dot candidate determined to be the minimum luminance pixel.
FIG. 33 is a flowchart showing dot detection processing when the detection method of the dot detector shown in FIG. 18 is realized by software.
In this case, first, in step S70, it is determined whether or not it is a region where dots are to be searched every other pixel in the horizontal and vertical directions, and a small region 3 × 3 pixels for searching for dots is set. Hereinafter, in the same manner as described above, in step S71, dot candidates are detected for a 3 × 3 pixel area centered on the pixel Z, and in subsequent step S72, the pixel Z detected as a dot candidate is more than the surrounding pixels. It is determined whether or not the luminance is low. Thereafter, in step S73, it may be determined whether there is a pixel already determined to be a dot around the dot candidate determined to be the minimum luminance pixel.

FIG. 34 is a flowchart showing dot center detection processing when the dot center detection method in the pen point coordinate calculator 87 shown in FIG. 25 is realized by software.
In this case, first, in step S81, the accurate coordinates of the dot existing at a predetermined position of the code frame are estimated. In subsequent step S82, projective transformation parameters are calculated from the center coordinates detected in the dot center detecting step in step S81 and the decoded position information, and predetermined projections in the image coordinates are calculated using the projection parameters calculated in step S83. The coordinate conversion process of the pen tip may be performed by adding a calculation to the coordinates of.
As described above, when the dot detection method is realized by software, dots can be detected at a very high speed.
Further, another processing procedure of the two-dimensional code reading process will be described with reference to the flowcharts shown in FIGS.
The flowchart shown in FIG. 35 differs from the flowchart shown in FIG. 31 in that writing is not being performed and continuous detection is not being performed, that is, error correction is not performed without replacing the document ID at the start of writing, and error correction cannot be performed. In this case, error correction is performed by replacing the document ID portion. In other words, if it is determined in step S41 shown in FIG. 35B that writing is in progress and it is determined in step S42 that continuous detection is not being performed, error correction is performed without replacing the document ID at the start of writing in step S43. In step S44, if the error cannot be corrected, the document ID portion is replaced in step S45 to correct the error. As a result, when adding to the same document, the error correction rate can be improved by performing the two-dimensional code reading process as shown in FIG.
Note that the processing shown in FIG. 35A and the processing in steps S45 to S48 shown in FIG. 35B are the processing shown in FIG. 31A and the processing in steps S32 to S35 shown in FIG. Since it is the same as the processing, the description is omitted.

So far, an apparatus and a system for acquiring a modified coordinate by reading a two-dimensional code have been described. Therefore, a system and method for acquiring the newly added information and superimposing it on the original document will be described with reference to the document management system shown in FIG. 5 and the flowchart shown in FIG. Here, it is assumed that the document is printed with a two-dimensional code.
Here, if the pen-type coordinate input device 7 is used for writing on the document, as described above, the two-dimensional code 22 on the document is read, and the document ID and coordinate information are acquired by the pen-type coordinate input device 7. (S51). Details of the code symbol reading will be described later. The document ID and coordinate information read in real time along with the writing are transmitted to the information processing apparatus 4. When the information processing apparatus 4 receives the document ID and the coordinate information, it transfers the document ID to the information processing apparatus 9 that manages the document management database 6 (S52).
Based on the received document ID, the information processing apparatus 9 identifies the page of the document currently being added from the document management database 6 (S53). Here, for example, it is assumed that the specified document is page 1 of “patent.doc” having the identification number (ID) “123456” shown in FIG. This document information is transmitted to the pen-type coordinate input device 7, and the received document information is displayed on the LCD display device 66 of the pen-type coordinate input device 7. Since the file entity can be identified in this way, the information processing apparatus 4 opens “patent.doc” with an application (for example, word processing software) associated with the file, and for example, is connected to the information processing apparatus 4 (not shown). It is displayed on the display to be in an editing state (S54).

The coordinate information sequentially transmitted from the pen-type coordinate input device 7 is drawn on the document “patent.doc” in the edited state. The drawing method can be realized by opening a new drawing object in a document window and drawing the received coordinates by connecting them with a line (S55 to S58).
When the writing is finished (S59 “Y”), the pen-type coordinate input device 7 transmits a signal for finishing writing to the information processing device 4, and the information processing device 4 that has received the signal for finishing writing draws on the window. Exit. In this way, one object is added to the document “patent.doc” (S60). In the present embodiment, the added information is displayed on a computer display in real time, and it is possible to immediately check whether the actually added information is correctly processed, which is a great merit for the user. The information processing apparatus 4 stores the received coordinate information as a separate text file as a sequence of text. This is useful because it is possible to display and check only the added data when it is impossible to check it quickly when the user wants to check the added data by opening the application later, such as when the document is huge. is there.
In the above-described embodiment, the case where a two-dimensional code is printed on paper has been described. However, this method is not limited to paper, and a paper-like display, film, whiteboard, or the like can be added. Needless to say, it works effectively.

FIG. 3 is a plan view of a printed document according to the embodiment of the present invention. The figure which shows the magnitude | size of the dot of a two-dimensional code, and the space | interval of a dot. The figure which shows the data arrangement | positioning area | region of a two-dimensional code. The figure which shows the arrangement | positioning order of the data to a two-dimensional code. 1 is a configuration diagram of a document management system according to an embodiment of the present invention. The flowchart which showed the encoding process of the two-dimensional code. Explanatory drawing of the encoding process of data, and the addition process of an error correction code. The figure which shows the document printing and the registration method to a database. The figure which shows an example of the document which attached and printed the two-dimensional code. The figure which shows the structure of a microcomputer. The block diagram which showed the structure of the two-dimensional code reader. The figure which shows an example of the image which read the two-dimensional code. The figure explaining the possibility of error correction by document ID replacement. The figure which shows the area | region of the whole image, and the process area | region of each detector. The figure which showed the structure of the dot detector. Explanatory drawing of a dot detector. The figure which shows the change of the coordinate acquisition rate by replacement of document ID. The figure which showed the other structure of the dot detector. Explanatory drawing of a code frame detector. Explanatory drawing of the tracking route of a code frame. Explanatory drawing of a corner detector. The figure which shows an example of a spiral scan. Explanatory drawing of a dot tracker. Explanatory drawing of a data acquisition device. The figure which shows the structure of a nib coordinate calculator. The figure which shows the structure of a dot center detector. Explanatory drawing of nib coordinate calculation. The figure which showed the relationship between the position of a dot center, and a pixel. Explanatory drawing of a pixel position. The block diagram which showed the other structure of the two-dimensional code reader. The flowchart which showed an example of the reading process of a two-dimensional code. The flowchart which showed the dot detection process. The flowchart which showed the other example of the dot detection process. The flowchart which showed the dot center detection process. The flowchart which showed the other example of the reading process of a two-dimensional code. The figure which shows the document update at the time of document addition, and the update method of a database. The figure which shows the item registered into the document management database.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Printer, 2 ... Copy machine, 3 ... Scanner, 4 ... Information processing apparatus, 5 ... Memory | storage device, 6 ... Document management database, 7 ... Pen type coordinate input device, 8 ... Portable information terminal, 9 ... Information processing apparatus, DESCRIPTION OF SYMBOLS 10 ... Two-dimensional code production apparatus, 21 ... Print document, 22 ... Two-dimensional code, 23 ... Dot, 24 ... Minimum dot, 61 ... Apparatus main body, 62 ... Tip part, 63 ... Writing instrument, 64 ... Image reader, 64a ... Photoelectric conversion element, 64b ... optical system, 65 ... microcomputer, 66 ... LCD display device, 67 ... LED, 68 ... buzzer, 69 ... pressure sensor, 70 ... bus, 71 ... CPU, 72 ... ROM, 73 ... RAM, 74 ... Two-dimensional code reader 81 ... dot detector 82 ... code frame detector 83 ... data acquisition device 84 ... data replacement device 85 ... error corrector 86 ... data decoder 87 ... nib coordinate calculator , DESCRIPTION OF SYMBOLS 8 ... Known information memory, 91 ... Continuous writing detector, 92 ... Writing detector, 101 ... Dot candidate detector, 102 ... Minimum brightness pixel determination device, 103 ... Peripheral dot search device, 104 ... Dot search area determination device, 11 ... dot detector, 111 ... first corner detector, 112 ... first dot tracker, 113 ... second corner detector, 114 ... second dot tracker, 115 ... third corner detector, 116 ... third dot Tracker, 117 ... fourth corner detector, 118 ... fourth dot tracker, 121 ... ambient dot detector, 122 ... point symmetry pair detector, 130 ... dot center detector, 131 ... projection parameter calculator, 132 ... Nib coordinate converter, 133 ... dot coordinate correction means, 134 ... dot position estimation means, 141 ... first error corrector, 142 ... second error corrector, 143 ... selector

Claims (16)

The two-dimensional pattern composed of a plurality of dots enter the captured image captured, the dot detecting means for detecting the dots, the code frame detection means for detecting a code frame from the detected dots in the dot detecting means; In a two-dimensional pattern reading device comprising data decoding means for reading the dots in the code frame and decoding the data,
The dot detection means uses a 7 × 7 pixel region centered on the pixel of interest among the dots and noise included in the image when the difference in luminance between the pixel of interest and a neighboring pixel is a predetermined value or more. a dot candidate detection means for detecting a candidate of dots and, in the region smaller than the region where the dot candidate is detected, the minimum luminance by comparing the detected dot candidate and neighboring pixels by the dot candidate detecting means dots and minimum luminance pixel determination means for determining a dot, if no pixel is already determined dot adjacent pixels dots candidates that have a minimum brightness in said minimum brightness pixel determining means, said dot candidate having A two-dimensional pattern reading device comprising: surrounding dot searching means for determining a pixel.   2. The two-dimensional pattern reading apparatus according to claim 1, wherein the dot minimum luminance pixel determination unit uses a 5 × 5 pixel area centered on the target pixel.   A dot detection unit that inputs a captured image obtained by capturing a two-dimensional pattern composed of a plurality of dots, detects the dot, a code frame detection unit that detects a code frame from the dots detected by the dot detection unit, In a two-dimensional pattern reading device comprising data decoding means for reading the dots in the code frame and decoding the data,
  The dot detection means includes a dot search area determination means for determining whether or not a predetermined small area is an area for searching for dots, and an area within the area determined as a dot search area by the dot search area determination means. When the difference in luminance between the target pixel and its neighboring pixels is greater than or equal to a predetermined value for each pixel, a 7 × 7 pixel area centered on the target pixel is used among the dots and noise included in the image The dot candidate detection means for detecting the dot candidate and the dot candidate detected by the dot candidate detection means in the area smaller than the area where the dot candidate is detected are compared with the neighboring pixels to obtain the minimum luminance. The minimum luminance pixel determining means for determining the dot having, and when there is no pixel already determined to be a dot in the adjacent pixels of the dot candidate having the minimum luminance by the minimum luminance pixel determining means, Tsu preparative candidate dot pixel and the surrounding dots searching means for determining the two-dimensional pattern reading apparatus characterized by comprising a.   4. The two-dimensional pattern reading apparatus according to claim 3, wherein the dot search area determination means sets a small area every other horizontal pixel and vertical pixel.   4. The two-dimensional pattern reading apparatus according to claim 3, wherein the dot search area is a small area composed of 3 × 3 pixels.   4. The two-dimensional pattern reading apparatus according to claim 3, wherein the dot minimum luminance pixel determination means uses a 5 × 5 pixel area centered on the target pixel.   The two-dimensional pattern reading device according to any one of claims 1 to 6,
  Coordinate calculation means for calculating a predetermined coordinate in the image coordinates based on the position information of the data decoded by the data decoding means and the coordinates of the code frame in the image,
  The coordinate calculation means includes a dot center detection means for estimating an accurate coordinate of a dot present at a predetermined position of the code frame, a dot center coordinate detected by the dot center detection means, and decoded position information A projection parameter calculation means for calculating a projection conversion parameter from the above, and a coordinate conversion means for calculating a predetermined coordinate in the image coordinates using the projection parameter calculated by the projection parameter calculation means. A two-dimensional pattern reading device.   The dot center detection means performs a predetermined calculation in a dot coordinate correction means for detecting a pixel having the minimum luminance in a small area including a dot, and a small area centered on the pixel detected by the dot coordinate correction means. The two-dimensional pattern reading device according to claim 7, further comprising: a dot position estimating unit that estimates an accurate position of the dot.   A captured image obtained by capturing a two-dimensional pattern composed of a plurality of dots, a dot detection step for detecting the dot, and a code frame detection step for detecting a code frame from the dots detected in the dot detection step; In a two-dimensional pattern reading method comprising a data decoding step of reading the dots in the code frame and decoding the data,
  The dot detection step uses a 7 × 7 pixel area centered on the pixel of interest among the dots and noise included in the image when the difference in luminance between the pixel of interest and a neighboring pixel is a predetermined value or more. A dot candidate detection step for detecting a dot candidate, and comparing the dot candidate detected by the dot candidate detection step with a neighboring pixel in an area smaller than the area where the dot candidate is detected, A minimum luminance pixel determining step for determining a dot having a dot, and a dot candidate is determined to be a dot pixel when there is no pixel already determined to be a dot in an adjacent pixel of a dot candidate having the minimum luminance in the minimum luminance pixel determining step A two-dimensional pattern reading method comprising: a surrounding dot search step.   10. The two-dimensional pattern reading method according to claim 9, wherein the dot minimum luminance pixel determination step uses a 5 × 5 pixel area centered on the target pixel.   A captured image obtained by capturing a two-dimensional pattern composed of a plurality of dots, a dot detection step for detecting the dot, and a code frame detection step for detecting a code frame from the dots detected in the dot detection step; In a two-dimensional pattern reading method comprising a data decoding step of reading the dots in the code frame and decoding the data,
  The dot detection step includes a dot search region determination step for determining whether or not the predetermined small region is a region for searching for dots,
  The dots included in the image when the difference in luminance between the pixel of interest and its neighboring pixels is greater than or equal to a predetermined value for each pixel in the area determined as the dot search area determined in the dot search area determination step And a dot candidate detection step of detecting a dot candidate using a 7 × 7 pixel area centered on the target pixel, and the dot candidate in an area smaller than the area where the dot candidate is detected A minimum luminance pixel determination step for comparing the dot candidate detected in the detection step with neighboring pixels to determine a dot having the minimum luminance, and a dot already adjacent to the dot candidate having the minimum luminance in the minimum luminance pixel determination step And a surrounding dot search step for determining the dot candidate as a dot pixel when there is no pixel determined as a two-dimensional pattern. Over down reading method.   12. The two-dimensional pattern reading method according to claim 11, wherein the dot search area determination step sets a small area every other horizontal pixel and vertical pixel.   The two-dimensional pattern reading method according to claim 11, wherein the dot search area is a small area composed of 3 × 3 pixels.   12. The two-dimensional pattern reading method according to claim 11, wherein the dot minimum luminance pixel determination step uses a 5 × 5 pixel area centered on the target pixel.   The two-dimensional pattern reading method according to any one of claims 9 to 14,
  A coordinate calculation step for calculating a predetermined coordinate in the image coordinates based on the position information of the data decoded in the data decoding step and the coordinates of the code frame in the image;
  The coordinate calculation step includes a dot center detection step for estimating an accurate coordinate of a dot existing at a predetermined position of the code frame, a center coordinate detected in the dot center detection step, and decoded position information. A projection parameter calculation step for calculating a projection conversion parameter; and a coordinate conversion step for calculating a predetermined coordinate in the image coordinates using the projection parameter calculated in the projection parameter calculation step. Dimensional pattern reading method.   The dot center detection step includes a dot coordinate correction step for detecting a pixel having a minimum luminance in a small area including a dot, and an accurate dot calculation by performing a predetermined calculation in the small area centered on the detected pixel. The two-dimensional pattern reading method according to claim 15, further comprising a dot position estimating step for estimating a position.

Images