JP4793868B2 - Writing medium, writing information detection device, writing information detection method, program, and recording medium - Google Patents

Description

  The present invention relates to a writing medium used for adding information to an electronic document, and an apparatus and method for accurately detecting information added to the writing medium using an ordinary writing tool such as a pencil or a ballpoint pen.

  In Patent Document 1, by adding a code symbol that is optically readable to a matrix using a pen equipped with a small camera, and simultaneously reading the code symbol with the camera to obtain coordinate information, A system is described in which retouching information is read in real time to associate preprinted paper information with retouching information.

  Patent Document 2 is composed of a pattern representing the X-coordinate, Y-coordinate, and code direction, and a pattern that is larger than another pattern called a homing pattern and arranged at the center of the code, and these are not visible to human eyes. A two-dimensional code characterized by this is described.

  Patent Document 3 describes a two-dimensional code that gives information of 2 bits or more per dot by shifting the dot from a predetermined position. This two-dimensional code has horizontal and vertical coordinate information, and the coordinates of the two-dimensional code can be detected by reading the coordinate information.

  In Patent Document 4, when writing on paper with a special pen equipped with a camera, the two-dimensional dot pattern encoded on the paper and encoded with coordinates and document information is read by the camera, which document of which A technique for detecting whether a position is being written is disclosed.

  In FIG. 11 of Patent Document 5, a paper document on which a two-dimensional code encoded with a document ID is encoded and timing marks are printed at four corners is written with an ordinary writing tool such as a pencil or a ballpoint pen. Is read out using a scanner and input as an image, and a writing locus is extracted from the image.

JP 2000-293303 A US Pat. No. 5,661,506 International Publication No. 00/73981 Pamphlet JP 2006-85679 A JP 2005-122682 A

  The techniques described in Patent Documents 1 to 4 have a restriction that it is essential to use a special writing device in which a camera, a CPU, and the like are attached to a pen.

  In the technique described in Patent Document 3, a dot pattern is printed on a paper document using a black material, and other document contents are printed using a color material, and a ballpoint pen and an infrared camera that transmit infrared light are used. By writing, it was possible to easily obtain the writing without seeing the writing trajectory or the document contents in the camera image. However, even if such a paper is written with an ordinary writing tool such as a pencil and then the paper is read with a scanner or the like, the writing cannot be distinguished from the document content, and writing information cannot be extracted.

  In Patent Document 5, an image obtained by scanning a paper document written with an ordinary writing instrument with a scanner is input, and this image and the original image are aligned using timing marks at four corners, and a difference between the images is obtained. Is described as a writing. However, it is difficult to accurately extract writing information with high positional accuracy due to the influence of local paper feed unevenness, drum rotation unevenness, and scanner running unevenness at the time of printing a paper document. In addition, in the application where a golfer's hitting ball trajectory is entered with a pencil on a golf course map printed by a copying machine and the hitting ball trajectory is extracted, colors such as fairway, green, rough, etc. on the map are displayed. There is also a problem that a pencil slips in a dark area and it becomes difficult to fill in a locus.

  The present invention has been made in view of the above-described problems, and the object of the present invention is a writing medium that is most suitable for use for inputting information to be added to an electronic document, and the writing medium has been rewritten. An object of the present invention is to provide an apparatus and a method for detecting information accurately and with high accuracy.

  The writing medium according to the first aspect of the present invention is a writing medium associated with a specific electronic document, and is a two-dimensional code that is a dot pattern of a predetermined size on a medium of a material that can be written using an ordinary writing instrument. The patterns are arranged and printed in a matrix, and each two-dimensional code pattern is encoded with coordinate information indicating the position of the two-dimensional code pattern on the medium, and constitutes each two-dimensional code pattern. The dots are color-coded to represent area information of the electronic document.

  A writing medium according to a second aspect of the present invention is a writing medium associated with a specific electronic document, and is a two-dimensional code pattern that is a dot pattern of a predetermined size on a medium of a material that can be written using an ordinary writing instrument. Are arranged and printed in a matrix, and each two-dimensional code pattern is encoded with coordinate information indicating the position of the two-dimensional code pattern on the medium and identification information of the electronic document. The dots constituting the dimensional code pattern are color-coded to represent area information of the electronic document.

  According to a third aspect of the present invention, in the writing medium according to the first or second aspect of the present invention, an error correction code is also encoded in each of the two-dimensional code patterns.

  According to a fourth aspect of the present invention, in the writing medium according to the first, second, or third aspect, the area ratio of the dots of the two-dimensional code pattern to the entire medium is less than 5 percent. .

In the handwritten information detection apparatus according to the fifth aspect of the present invention, two-dimensional code patterns are arranged and printed in a matrix, and dots constituting each of the two-dimensional code patterns represent region information of a specific electronic document. Color-coded, a retouched information detection device for detecting information rewritten using a writing instrument on a writing medium with a color image obtained by reading a writing medium associated with the electronic document as an input,
First processing means for binarizing the color image to generate a binarized image in which a background pixel is a white pixel and a non-background pixel is a black pixel;
Based on the color image, among the pixels of the color image, only pixels corresponding to the dots of the two-dimensional code pattern printed in a matrix on the writing medium are represented by black pixels, and the other pixels are represented. Generates a dot image represented by white pixels, and detects and outputs dot coordinates representing the position of the dot of the two-dimensional code pattern on the dot image and a dot color representing the color of the dot of the two-dimensional code pattern Second processing means to:
Based on the dot coordinates output from the second processing means, by removing black pixels corresponding to the dots of the two-dimensional code pattern from the binarized image, it corresponds to the part of the writing medium that has been rewritten. Third processing means for generating a binary image representing the black pixel (hereinafter referred to as an added pixel),
The coordinates representing the position of the two-dimensional code pattern on the writing medium by detecting the two-dimensional code pattern from the dot image and decoding each detected two-dimensional code pattern (denoted as coordinates on the writing medium) And a fourth processing means for outputting the coordinates on the writing medium in association with the coordinates representing the position of the two-dimensional code pattern on the dot image (coordinates on the image);
Based on the coordinates representing the position of each of the retouched pixels on the binary image and the coordinates on the writing medium and the coordinates on the image of the two-dimensional code pattern output from the fourth processing unit. A fifth processing means for obtaining coordinates indicating the position of the writing pixel on the writing medium and outputting it as writing information;
Sixth processing means for generating area information representing an area represented by a group of dots of the same color based on the dot coordinates and the dot color output from the second processing means;
It is characterized by having.

Invention according to claim 6, in revision information detecting apparatus according to the invention of claim 5, wherein said fourth processing means, by decoding the detected two-dimensional code pattern, also the identification information of the electronic document It is characterized by acquiring and outputting.

  According to a seventh aspect of the present invention, in the handwritten information detection apparatus according to the fifth aspect of the invention, the third processing unit performs thinning processing on the binary image, and the fifth processing unit performs the thinning processing. The added pixel in the binary image after processing is a processing target.

  According to an eighth aspect of the present invention, in the handwritten information detection device according to the fifth aspect of the invention, the second processing means detects a dot for each color component of the color image, and based on the detection result, The dot image is generated and the coordinates and color of the dot of the two-dimensional code pattern are detected.

  According to a ninth aspect of the present invention, in the retouched information detecting apparatus according to the fifth aspect of the invention, the fourth processing unit sets a plurality of small areas in the dot image, and the two-dimensional area is set in each of the set small areas. It is characterized by detecting a code pattern.

According to a tenth aspect of the present invention, there is provided an additional information detection method in which two-dimensional code patterns are arranged and printed in a matrix, and the dots constituting each two-dimensional code pattern represent region information of a specific electronic document. Color-coded, a rewritten information detection method for detecting information added using a writing instrument on a writing medium, with a color image obtained by reading a writing medium associated with the electronic document as an input,
A first processing step of binarizing the color image to generate a binarized image in which a background pixel is a white pixel and a non-background pixel is a black pixel;
Based on the color image, among the pixels of the color image, only pixels corresponding to the dots of the two-dimensional code pattern printed in a matrix on the writing medium are represented by black pixels, and the other pixels are represented. Generates a dot image represented by white pixels, and detects and outputs dot coordinates representing the position of the dot of the two-dimensional code pattern on the dot image and a dot color representing the color of the dot of the two-dimensional code pattern A second processing step,
Based on the dot coordinates output from the second processing step, by removing black pixels corresponding to the dots of the two-dimensional code pattern from the binarized image, it corresponds to the part of the writing medium that has been rewritten A third processing step of generating a binary image representing the black pixel (hereinafter referred to as an added pixel);
The coordinates representing the position of the two-dimensional code pattern on the writing medium by detecting the two-dimensional code pattern from the dot image and decoding each detected two-dimensional code pattern (denoted as coordinates on the writing medium) A fourth processing step of outputting the coordinates on the writing medium in association with coordinates (coordinates on the image) representing the positions on the dot image of the two-dimensional code pattern;
Based on the coordinates representing the position of each of the retouched pixels on the binary image and the coordinates on the writing medium and the coordinates on the image of the two-dimensional code pattern output from the fourth processing unit. A fifth processing step of obtaining coordinates indicating the position of the writing pixel on the writing medium and outputting it as writing information;
A sixth processing step for generating region information representing a region represented by a group of dots of the same color based on the dot coordinates and the dot color output from the second processing step;
It is characterized by having.

Invention of claim 11, wherein, in the revision information detecting method according to the invention of claim 10, wherein the fourth process step, by decoding the detected two-dimensional code pattern, also the identification information of the electronic document It is characterized by acquiring and outputting.

  According to a twelfth aspect of the present invention, in the handwritten information detection method according to the tenth aspect, the third processing step performs thinning processing on the binary image, and the fifth processing step includes the thinning processing. The added pixel in the binary image after processing is a processing target.

  According to a thirteenth aspect of the present invention, in the retouching information detection method according to the tenth aspect of the present invention, the second processing step detects dots for each color component of the color image, and based on the detection result, The dot image is generated and the coordinates and color of the dot of the two-dimensional code pattern are detected.

  According to a fourteenth aspect of the present invention, in the handwritten information detection method according to the tenth aspect of the invention, the fourth processing step sets a plurality of small areas in the dot image, and the two-dimensional area is set in each of the set small areas. It is characterized by detecting a code pattern.

A program according to a fifteenth aspect of the present invention is a program that causes a computer to function as each means of the additional information detecting device according to the fifth aspect of the present invention.

A recording medium according to a sixteenth aspect of the present invention is a computer-readable recording medium on which a program that causes a computer to function as each means of the additional information detecting device according to the fifth aspect of the invention is recorded.

(1) The writing medium according to the first to fourth aspects of the present invention can easily add information such as lines and characters using an ordinary writing instrument such as a pencil or a pole pen. Then, by reading the rewritten writing medium with a scanner or the like and inputting it as a color image, it is possible to detect the rewriting information accurately and with high accuracy by the retouching information detecting apparatus or method according to the inventions of claims 5 to 14. it can. In the writing medium according to the second aspect of the invention, the identification information of the electronic document associated therewith can also be acquired, so that the electronic document associated with the identification information is opened and the additional information is added to the electronic document. Becomes easier. Since the error correction code is also encoded in the writing medium according to the third aspect of the invention, more accurate writing information can be detected. In the writing medium according to the first to fourth aspects of the present invention, since the area information of the related electronic document is expressed by the color coding of the dots of the two-dimensional code pattern, the writing medium is viewed without looking at the electronic document itself. By simply recognizing the contents of the electronic document, for example, the outline of areas such as fairways, bunker and rough in the golf course map, it becomes easy to add a golfer's hitting ball trajectory. Further, it is possible to perform processing such as detecting area information based on dot color coding and printing it out together with the additional information. Since the writing medium according to the fourth aspect of the present invention has a dot area ratio of less than 5%, the writing medium appears to be a bright color to the human eye, so the written lines and characters are two-dimensional codes. The pattern does not become difficult to see, and the color material used for dot printing does not make the writing tool slip easily, so that the effect of smooth writing can be obtained.
(2) According to the handwritten information detection apparatus or method according to the inventions of claims 5 to 14, information added to the writing medium according to the invention of claims 1 to 4 is accurately and accurately detected, Furthermore, it is possible to obtain an effect that the identification information and area information of the electronic document associated with the writing medium can be detected.
(3) The computer according to any one of claims 5 to 9 is loaded by loading the program according to the invention according to claim 15 or the program recorded on the recording medium according to the invention according to claim 15 into the computer. It can be operated as a handwritten information detection device.

  First, an outline of what the writing medium of the present invention is will be described.

  The writing medium of the present invention is not simply a blank sheet or a normal print document. The writing medium of the present invention is intended to be used for adding information to the contents of an electronic document, and is therefore associated with a specific electronic document.

  Specifically, the writing medium of the present invention is a medium of a material that can be written using a normal writing instrument such as a pencil or a ballpoint pen, such as paper, and a two-dimensional code pattern that is a dot pattern of a predetermined size in a matrix. Printed side by side. Each two-dimensional code pattern is encoded with coordinate information indicating a position on the medium. In a preferred embodiment, identification information of an electronic document associated with the writing medium is encoded in addition to the coordinate information. The dots constituting the two-dimensional code pattern are color-coded so as to represent the area information of the associated electronic document. That is, an area composed of a group of dots of the same color represents a certain area on the related electronic document.

  For example, consider a writing medium associated with an electronic document whose content is a map of a golf course. There are fairways, bunker, rough, green, water hazard, etc. on the golf course, but the dot color corresponding to the fairway area is yellow, the dot color corresponding to the bunker area is magenta, and the dot corresponding to the water hazard area The color of the dots is different for each region, such as cyan. In the case of a writing medium associated with an electronic document containing a form, for example, a blue dot represents a character area, and a magenta dot represents an entry frame area such as letters and numbers. The dots are color coded.

  Thus, when the area information of the electronic document is expressed by the dot color, writing becomes easy. For example, in the case of a writing medium associated with an electronic document whose content is a map of a golf course, it is possible to grasp the area such as the fairway of the golf course without referring to the print of the electronic document. There is an advantage that the trajectory of the hit ball of each player can be easily added while watching. Further, by detecting the area information from the writing medium without opening the associated electronic document, it is possible to reproduce the outline of the contents of the associated electronic document and draw the additional information on it.

  Hereinafter, embodiments of the writing medium of the present invention will be described in detail with reference to the drawings.

  As described above, a two-dimensional code pattern of a predetermined size is printed in a matrix on the writing medium. An example of such a two-dimensional code pattern is shown in FIG. 2a, 2b, 2c, and 2d in FIG. 1 are two-dimensional code patterns. Here, four two-dimensional code patterns are shown, but the same two-dimensional code patterns are regularly printed in a matrix arrangement vertically and horizontally on almost the entire surface of the writing medium. The vertical dot rows (vertical lines) H1, H2, and H3 and the horizontal dot rows (horizontal lines) V1, V2, and V3 form a frame (boundary) of the two-dimensional code pattern. Then, as described above, the dots constituting the two-dimensional code pattern (including the frame or the boundary dot) are color-coded to represent the area information on the associated electronic document and optically read. Printed using possible black or color colorants. It should be noted that the color coding of the dots can be performed regardless of the boundary of the two-dimensional code pattern.

  As described above, each two-dimensional code pattern on the writing medium includes the identification information (document ID information) of the electronic document associated with the writing medium and the horizontal and vertical indicating the position of the two-dimensional code pattern on the medium. Coordinates are encoded. The document ID information is identification information unique to the electronic document, for example, ID information in a database in which the electronic document is stored.

  In the two-dimensional code pattern 2a shown in FIG. 1, "horizontal coordinates = 95, vertical coordinates = 10, document ID = 10" is encoded, and in the two-dimensional code pattern 2b, "horizontal coordinates = 96, vertical coordinates = 10, “Document ID = 10” is encoded, “Horizontal coordinate = 95, Vertical coordinate = 11, Document ID = 10” is encoded in the two-dimensional code pattern 2c, and “Horizontal coordinate = 96” is encoded in the two-dimensional code pattern 2d. , Vertical coordinate = 11, document ID = 10 ”are encoded.

  Here, the dot size of the two-dimensional code pattern, the dot interval, and the size of the two-dimensional code pattern will be described with reference to FIG. The dots 2702 of the two-dimensional code pattern (including the frame) are printed in 2 × 2 units of the minimum dots 2701 of the printer used for printing the writing medium. For example, in the case of a 1200 dpi printer, since the minimum dot diameter is 21 μm, the dot diameter of the two-dimensional code pattern is ideally 42 μm (in reality, the dot diameter is a little larger due to the dot gain). It is assumed that such dots are arranged in the horizontal and vertical directions with an interval of 6 times the diameter. In this case, even when dots are printed at all the dot arrangement positions, the area ratio occupied by dots with respect to the entire writing medium is 2.8% if there is no dot gain, and even if a dot gain of about 50% is expected, the area ratio Is less than 5%. With such an area ratio, since the two-dimensional code pattern looks bright to the human eye, characters, lines, etc. added by the two-dimensional code pattern do not become difficult to see. In addition, the color material used for printing does not interfere with writing.

  As described above, if the arrangement positions of the dots of the two-dimensional code pattern are determined, the two-dimensional code pattern has a size of 2 mm (horizontal) × 3 mm (vertical). Therefore, the information of horizontal coordinate = 95 and vertical coordinate = 10 encoded in the two-dimensional code pattern is that the upper left corner of the two-dimensional code pattern is 190 mm (= 95 × 2) in the horizontal direction with the upper left corner of the medium as the origin. It means that the position is 30 mm (= 10 × 3) in the vertical direction. As will be described in detail later, the 2 mm (horizontal) × 3 mm (vertical) size of the two-dimensional code pattern is similar to that of A4 size paper, and the data length for storing horizontal / vertical coordinate information can be used effectively. It becomes possible.

  Thus, the two-dimensional code pattern 2a shown in FIG. 1 has an upper left corner (intersection of the vertical line H1 and the horizontal line V1) at a position of 190 mm horizontally and 30 mm vertically from the upper left of the medium, and an electronic document having a document ID = 10. Information associated with the.

  The data arrangement in the two-dimensional code pattern will be described by taking the two-dimensional code pattern 2a in FIG. 1 as an example. FIG. 3 is an enlarged view of the two-dimensional code pattern 2a in FIG. The two-dimensional code pattern 2a as a whole has 7 × 11 cells surrounded by dot arrays H1, H2, V1, and V2 that form a frame. A cell is a unit in which dots representing information can be placed. Reference numeral 401 denotes an area in which data representing horizontal coordinates is arranged, and has 4 × 2 cells. Since information is expressed by the presence or absence of dots, and one cell corresponds to one bit, the region 401 has a capacity capable of encoding 1-byte information. Reference numeral 402 denotes an area in which data representing vertical coordinates is arranged, and has a capacity of 4 × 2 cells and 1 byte. The A4 size paper is 210 mm (horizontal) x 297 mm (vertical) size, and the 2D code pattern is 2 mm (horizontal) x 3 mm (vertical) size. Although 105 in the horizontal direction and 99 in the vertical direction can be arranged, the horizontal and vertical coordinates of such a two-dimensional code pattern can be expressed by 1 byte. Since the size of the A3 size paper is 297 mm (horizontal) × 420 mm (vertical), a maximum of 148 horizontal dimension patterns and 120 vertical dimensions can be arranged on the A3 size paper. The horizontal and vertical coordinates of such a two-dimensional code pattern can be expressed by 1 byte. In the case of A2 size paper having a size of 420 mm (horizontal) × 594 mm (vertical), a maximum of 210 two-dimensional code patterns can be arranged in the horizontal direction and 198 in the vertical direction. The horizontal and vertical coordinates of the pattern can also be expressed by 1 byte. In this way, when the aspect ratio of the paper and the aspect ratio of the two-dimensional code pattern are equal, even if the data lengths of the regions 401 and 402 representing the horizontal and vertical coordinates of the two-dimensional code pattern are fixed to the same size, a wide variety of paper sizes Can be supported.

  In FIG. 3, reference numeral 403 denotes an area for arranging data representing a document ID, which has 4 × 6 cells and a capacity of 24 bits (3 bytes). Four areas 404 to 407 are areas where error correction codes are arranged. Each area has a capacity of 1 byte and has a capacity of 4 bytes in total of the 4 areas. Reference numerals 408 and 409 denote areas for representing the vertical direction of the two-dimensional code pattern. The area 408 has 3 × 1 cells, and black dots are applied to all the cells. The area 409 has 2 × 1 cells, and no dot is placed in any of the cells.

  FIG. 4 is a diagram showing in what order the data bits are arranged in the regions 401 to 407 of the two-dimensional code pattern. In FIG. 4, “1” means the most significant bit (MSB) of data, “2” means a bit lower by one, “3” means a bit lower by one, and so on. Larger numbers mean lower bits. Therefore, “8” in the area 401 or the like means the least significant bit (LSB), and “24” in the area 403 means the least significant bit.

  FIG. 5 is a diagram showing the configuration of the document management system according to the embodiment of the present invention. This document management system has a configuration in which a printer 601, a copier 602, a scanner 603, information processing devices 604 and 609 such as a personal computer and a workstation, and a storage device 605 such as a hard disk device are connected via a network. However, such a system configuration is merely an example. The storage device 605 stores the electronic document 1, the document management database 606, and the like.

  Next, a procedure for creating a print image of a writing medium as described above and printing it in the document management system shown in FIG. 5 will be described.

  FIG. 6 is a flowchart for explaining a procedure for creating a print image of a writing medium. The print image creation process of the writing medium is performed by the printer driver in the information processing apparatus when the information processing apparatus 609 (or 604) executes the print instruction of the associated electronic document 1 (however, other Naturally, it is possible to execute the program by the application software of the above).

  First, the printer driver inquires of the document management database 606 in the storage device 605 and issues a document ID for each page of the electronic document 1 (step 1).

  Hereinafter, in the processing loop of steps 2 to 7, processing for sequentially creating a print image of a writing medium corresponding to the document page in units of two-dimensional code patterns is performed.

  That is, the printer driver acquires coordinate information indicating the position on the writing medium of the two-dimensional code pattern to be encoded into one two-dimensional code pattern (step 2). For example, as shown in FIG. 7, a document ID “123456” and coordinate information (24, 123) in mm units are obtained.

  Next, the printer driver executes an encoding process of the acquired document ID and mm-unit coordinate information (step 3). That is, as shown in FIG. 7, the 6-digit document ID is converted into a 3-byte binary value. In addition, the horizontal and vertical coordinate information (24, 123) in units of mm is divided by the horizontal size 2 and vertical size 3 of the two-dimensional code pattern, respectively, and converted into values (12, 41) that fit in 1 byte, respectively. Are converted into binary values of 1 byte each. Thus, with respect to one two-dimensional code pattern, 2 bytes of coordinate information and 3 bytes of document ID, a total of 5 bytes of data are obtained.

  Next, the printer driver adds an error correction code to the 5-byte data (step 4). In the example of FIG. 7, a 4-byte error correction code is added to 5-byte data. A Reed-Solomon code is adopted as this error correction code. The Reed-Solomon code is a powerful error correction code that can correct errors in byte units, and can correct errors that are less than half the code length. In this example, since the error correction code length is 4 bytes, 2-byte error correction is possible. Reed-Solomon error correction codes are described in a number of books such as “Miyakawa, Iwatari, Imai,“ Code Theory (Computer Fundamental Course 18) ”, Shosodo”, and so on, so further explanation is omitted.

  The page of the electronic document 1 includes, for example, a rectangular area from coordinates (x0, y0) to (x1, y1) that is a tee ground area, and an area surrounded by a coordinate point sequence (xi, yi) is a bunker area. Assume that there is area information such as an area. The printer driver acquires such area information corresponding to the position of the two-dimensional code pattern that is currently processed (step 5).

  The printer driver assigns the generated encoded data and error correction code data to the corresponding cells of the two-dimensional code pattern, and creates a two-dimensional code pattern in which colors are assigned to the respective dots based on the area information acquired. The two-dimensional code pattern is arranged at the position indicated by the mm-unit coordinate information on the medium print image page secured in the memory (step 6). A character (character area) can also be expressed by color coding of dots of a two-dimensional code pattern. When four color materials of cyan, magenta, yellow, and black are used in the printer used for printing, it is appropriate to select the dot color from seven colors of cyan, magenta, yellow, red, green, blue, and black. It is.

  By repeating the same processing, a writing medium print image in which a two-dimensional code pattern is arranged in a matrix on the entire page is created.

  When the electronic document 1 is composed of a plurality of pages, a writing medium print image is created by the same processing for each page.

  In this way, when a writing medium print page in which a two-dimensional code pattern is arranged in a matrix for all pages of the electronic document 1 is created, the printer driver prints the created writing medium print, which is not shown in FIG. The image is printed using the printer unit of the printer 601 or the copier 602. If this printing is normally completed, each page of the print image of the writing medium and the document ID information are registered in the document management database 606.

  FIG. 8 shows a flow of the above series of operations. First, the operator edits each page of the electronic document 1 stored in the storage device 605 with application software as necessary, and issues a print command (step 11). A writing medium print image corresponding to each page of the electronic document 1 is created by the printer driver according to the procedure shown in FIG. 6 and printed by the printer 601 or the copier 602 (step 12). If the printing is normally completed (step 13, Yes), the printer driver performs the registration process in the document management database 606 as described above (step 14).

  Now, the writing medium printed as described above can be added using an ordinary writing instrument such as a pencil or a ballpoint pen. Then, it is possible to perform processing or the like of reading an image of a writing medium, extracting additional information from the image, and adding the extracted information to the associated electronic document.

  Next, extraction of information added to the writing medium will be described.

  When extracting the information added to the writing medium, the writing medium is read as a color image by using the scanner unit 603 of the document management system (FIG. 5) or the scanner unit of the copier 602, and the writing information is obtained by processing the color image. Extract. This processing is executed by the retouching information detection device (not shown in FIG. 5) of the present invention provided in the scanner 603 or the copier 602, or by software in the information processing device 604 or 609. The In the case of the latter software processing, the color image read from the writing medium is temporarily stored in the storage device 605.

  First, a writing information detection apparatus provided inside the scanner 603 or the copier 602 will be described.

  FIG. 9A is a block diagram of the writing information detection apparatus. The retouching information detection apparatus shown here includes a binarization unit 200, dot detection units 202R, 202G, and 202B, an image composition / dot color detection unit 204, a retouching extraction unit 206, a two-dimensional code decoding unit 208, and a region information extraction unit. 210 and an added coordinate specifying unit 212. Each part will be described sequentially.

  First, the binarization unit 200 converts color image data (RGB data) obtained by reading an added writing medium into grayscale image data, and binarizes the grayscale image data using a binarization threshold value. Thus, this is means for generating binarized image data in which the pixel corresponding to the rewritten portion on the writing medium and the high-density dot is black, and the other pixels are white. Although a preset fixed value can be used as the binarization threshold, the binarization threshold may be adaptively determined according to grayscale image data. As a method for adaptively determining the binarization threshold value, for example, a method described in JP 2001-8032 A can be used. Briefly describing this, grayscale image data is divided into blocks, and pixels are sampled with a predetermined sampling period for each block, and pixels whose luminance value exceeds the low luminance threshold among the sampled pixels are sampled. An average luminance value is calculated, and a binarization threshold for the block is calculated using a predetermined calculation formula using the average luminance value. Although the low luminance threshold value can be fixed, the low luminance threshold value may be determined by calculation based on the average luminance value calculated for adjacent blocks.

  The dot detection units 202R, 202G, and 202B receive color image data (RGB data) obtained by reading a writing medium, divided into RGB planes, and input a two-dimensional code pattern (including its frame) on the R, G, and B planes. ) Is detected. Each dot detection unit has the same configuration.

  FIG. 10A is a block diagram illustrating a first example of each dot detection unit 202. Dot detection is performed while moving the target pixel from the upper left in the order of raster scanning. The region referred to when detecting dots is a 7 × 7 pixel region centered on the target pixel Z as shown in FIG.

  The dot detection unit 202 according to the present embodiment refers to a 7 × 7 pixel area centered on the target pixel Z, a dot candidate determination unit 3101 that determines whether the target pixel Z is a dot candidate, A minimum luminance pixel determining unit 3102 that determines whether or not the luminance of the target pixel Z that is a dot candidate by the dot candidate determining unit 3101 is lower than the luminance of surrounding pixels, and the minimum luminance pixel determining unit 3102 is the minimum luminance pixel. It is checked whether there is a pixel already detected as a dot around the dot candidate determined to be, and if there is no dot already detected in the periphery, the dot candidate is finally detected as a dot. Composed. Each part will be described in more detail.

  The dot candidate determination unit 3101 determines the target pixel Z as a dot candidate when all the conditions of the following formulas (1) to (8) are satisfied. However, Th is a predetermined threshold value, and A, B, C,. . . The alphabets such as indicate the value of the pixel at the corresponding position in FIG. One pixel on each surface of R, G, B is expressed by 8 bits and takes a value from 0 to 255.

Z <A-Th (1)
Z <E-Th (2)
Z <L-Th (3)
Z <P-Th (4)
Z <Q-Th (5)
Z <R-Th (6)
Z <S-Th (7)
Z <T-Th (8).

  The minimum luminance pixel determination unit 3102 determines that the pixel of interest Z, which is a dot candidate, has the minimum luminance when all the conditions of the following formulas (9) to (20) are satisfied.

Z <B (9)
Z <C (10)
Z <D (11)
Z <F (12)
Z <G (13)
Z <H (14)
Z <I (15)
Z <J (16)
Z <K (17)
Z <M (18)
Z <N (19)
Z <O (20).

  The surrounding dot searcher 3103 is the U, V, W, X around the pixel of interest Z determined as a dot candidate by the dot candidate determination unit 3101 and determined as the minimum luminance pixel by the minimum luminance pixel determination unit 3102. If there is no pixel detected as a dot at the position, the target pixel Z is finally determined as a dot (the target pixel Z is detected as a dot). Since the dot detection process is performed in the raster scan order from the upper left, the peripheral dot searcher 3103 does not need to examine the pixel at the Y position.

  In addition, it is necessary to set the threshold value Th in the above formulas (1) to (8) to an appropriate value. The smaller the threshold value Th, the easier it is to detect dots, but at the same time, noise is also detected as dots, and the possibility of errors in detection data increases. On the other hand, if the threshold value Th is large, the false detection of noise can be reduced, but the original dot detection rate is lowered and the reading probability of the two-dimensional code is lowered.

  FIG. 12A is a block diagram illustrating a second embodiment of the dot detector 202. The dot detector 202 according to this embodiment is different from the first embodiment in that a dot search area determination device 3100 is added in the forefront stage in order to speed up the dot detection process.

  The dot search area determination unit 3100 selects a pixel every other pixel in the horizontal and vertical directions on the R, G, or B plane of the color image data read from the writing medium, and selects the pixel as the target pixel Z in FIG. When all the conditions of the above formulas (1) to (8) are satisfied, an area of 3 × 3 pixels centered on the pixel is determined as an area for searching for a dot (dot search area).

  Only each pixel in the dot search area becomes a target pixel for processing by the dot candidate determination unit 3101. Except for this point, the processing content of the dot candidate determination unit 3101 is the same as in the case of the first embodiment. Thus, since the dot search area is narrowed down in advance, the processing speed is increased compared to the first embodiment in which dot search is performed for all pixels. The processing contents of the dot candidate determination unit 3101 for each target pixel are the same as those in the first embodiment. The processing content of the minimum luminance pixel determination unit 3102 is the same as that in the first embodiment.

  The processing contents of the peripheral dot searcher 3402 are the same as in the case of the first embodiment. However, in this embodiment, U and V around the target pixel Z determined to be the minimum luminance pixel of the target pixel Z. , W, and X positions, the Y position is checked for the presence or absence of pixels that have been detected as dots, and when no pixel that has been detected as a dot exists at any position, the target pixel Z is finally determined to be a dot. The reason for checking the Y position in this way is processed for each dot search area in this embodiment, so that the pixel at the Y position is located in the dot search area processed before the dot search area including the target pixel Z. This is because if it is included, the pixel at the Y position may have already been detected as a dot.

  Returning now to FIG. The image composition / dot color detection unit 204 synthesizes a dot image based on the detection results of the dot detection units 202R, 202G, and 202B for each RGB image plane, and detects the coordinates and color of the dot on the dot image. To register in the dot coordinate list. More specifically, the position of the detected dot on each RGB image plane may be shifted by one pixel between the RGB planes due to a dot position shift when printing on a writing medium, a position shift when detecting a dot, or the like. is there. Therefore, when a plurality of dots exist in a 3 × 3 pixel area centered on each dot detected on each RGB plane, these dots are collectively displayed as one dot on the dot image. In a dot image, dots are represented as black pixels, and pixels other than dots are represented as white pixels. For example, if a dot is detected in the entire RGB plane in a small area of 3 × 3 pixels centered on a certain pixel position, the dot color is black and the dot is detected only on the R plane. If the dot color is cyan and the dot is detected only on the G plane, the dot color is magenta. If the dot is detected only on the B plane, the dot color is detected on the yellow and GB planes. For example, if the dot color is red and a dot is detected on each RB plane, the dot color is determined to be green, and if a dot is detected on each RG plane, the dot color is determined to be blue. The image composition / dot color detection unit 204 outputs a dot image and a dot coordinate list (a list of dot coordinates and dot colors).

  The area information extraction unit 210 classifies dot coordinates having the same color and forming one area based on the dot coordinates and dot colors included in the dot coordinate list output from the image composition / dot color detection unit 204. I do. That is, if the noticed dot and the adjacent dot have the same color, it is determined that the dots form one area. By repeating this process until all the areas are determined, an area represented by a group of dots of the same color is extracted.

  Based on the binarized image data output from the binarization unit 200 and the dot coordinates in the dot coordinate list output from the image composition / dot color detection unit 204, the retouching extraction unit 206 performs writing on the writing medium. Extract the part. That is, in the binarized image data, black in a small area of 3 × 3 pixels (or 5 × 5 pixels) centered on each dot coordinate (the coordinates of the dots constituting the two-dimensional code pattern) in the dot coordinate list. Change the pixel to a white pixel. In the binarized image data after this processing (hereinafter referred to as binary image data), only black pixels corresponding to the added part remain. Next, the retouching extraction unit 206 performs thinning processing on the binary image data. The thinning process is a process for setting the line thickness to one pixel width. However, since there is a paint other than a line in addition, thinning is not performed for a line having a certain width (for example, a width of 48 pixels (= 2 mm) or more in the case of an input image of 600 dpi). For such thinning processing, various known algorithms may be used. Note that the thinning process can be omitted, and such an embodiment is also included in the present invention.

  Thus, the writing position after thinning is only the position on the binarized image data, that is, the position on the input image data obtained by reading the writing medium, and not the position on the writing medium. Therefore, the coordinate on the writing medium at the writing position is detected by the writing coordinate specifying unit 212. To that end, the two-dimensional code decoding unit 208 decodes a plurality of two-dimensional code patterns in the dot image, It is necessary to acquire coordinate information representing the position on the writing medium encoded in the three-dimensional code pattern, and associate this with the position of the two-dimensional code pattern on the input image data.

  The two-dimensional code / decoding unit 208 will be described. FIG. 13A is a block diagram illustrating an example of the two-dimensional code decoding unit 208. As shown in the figure, the two-dimensional code / decoding unit 208 according to this embodiment includes a small region setting unit 300, a code frame detector 302, a data acquisition unit 304, an error correction unit 306, and a data decoder 308.

If all the two-dimensional code patterns on the input image data are to be detected, not only the processing time increases but also useless information is obtained. Therefore, a small area is set in the input image by the small area setting device 300. Then, for each small area, one two-dimensional code pattern is decoded. The two-dimensional code pattern may not be normally decoded. In this case, the coordinate information on the writing medium encoded in the two-dimensional code pattern in the small area and the position of the two-dimensional code pattern on the input image Although the association cannot be made, there is no problem because the coordinate information obtained by other small areas in the vicinity of the small area can be substituted. As a specific example, when an A4 size writing medium is read at 600 dpi and an image is input, a small region is set for each 256 × 256 pixels in the input image. Since the size of the input image is 4960 × 7016 pixels, 20 small regions are arranged in the short side direction and 28 small regions are arranged in the long side direction.
Next, the code frame detector 302 detects one two-dimensional code pattern frame and its position for each small region, and outputs an image and position information within the detected frame. Next, the data acquisition unit 304 acquires “1” or “0” data indicating the presence / absence of a dot in each cell of the two-dimensional code pattern from the in-frame image input from the code frame detector 302. The data is rearranged according to the data arrangement rule (FIG. 4) in the two-dimensional code pattern and output.

  Next, error correction unit 306 performs error correction on the data output from data acquisition unit 304. The error corrector 306 outputs determination information as to whether or not error correction has been successful and data after error correction. The data after error correction is the document ID encoded in the two-dimensional code pattern and the coordinate information on the writing medium of the two-dimensional code pattern.

  If the error correction determination information is “error correction successful”, the data decoder 308 decodes the document ID and the coordinate information on the writing medium from the error-corrected data. However, when the error correction determination information is “error correction failure”, the data decoder 308 outputs a numerical value that cannot be used as a document ID, for example (−1), and a numerical value that cannot be used as coordinate information on the writing medium, for example (−1, -1) is output.

  Data decoding will be described with reference to FIG. The data encoded in the two-dimensional code pattern consists of 2 bytes of coordinate information (1 byte in the X direction, 1 byte in the Y direction), 3 bytes of the document ID, and 4 bytes of the error correction code. It can be corrected. When the data acquisition unit 304 obtains “correct data” shown in FIG. 14, the horizontal coordinate = 24 mm (= 12 × 2), the vertical coordinate = 123 mm (= 41 × 3), and the document ID = 23 is decoded. Is done. Further, when data that has an error in 2 bytes shaded as in “observation data 1” shown in FIG. 14 is acquired by the data acquisition unit 304, the error can be corrected. The vertical coordinates and the document ID can be correctly decoded. However, in the case of data with an error of 3 bytes or more shaded as “observation data 2” shown in FIG. 14, the error cannot be corrected correctly, and normal decoding is impossible.

  Then, the coordinates on the writing medium of the two-dimensional code pattern decoded by the data decoder 308 (the coordinates of the upper left corner of the two-dimensional code pattern) and the image of the upper left corner of the two-dimensional code pattern detected by the code frame detector 302 The position information list associated with the upper coordinates and the document ID information decoded by the data decoder 302 are output as output data of the two-dimensional code decoding unit 208.

  Next, each part of the two-dimensional code decoding unit 208 will be described in detail.

  FIG. 15 is a block diagram of the code frame detector 302. Code frame detection is performed by tracking dots in a small area of a dot image. FIG. 16 shows a dot tracking path.

  The code frame detector 302 is a first corner detector 4000 for detecting the first corner (X) of the two-dimensional code pattern, and the second corner candidate is tracked for eight pixels in the vertical and horizontal directions from the first corner (X). A first dot tracker 4001 for detecting pixels, a second corner detector 4002 for determining whether the second corner candidate pixel detected by the first dot tracker 4001 is the second corner (B, D), From the second corner (B, D) detected by the second corner detector 4002, the code frame 12 constituted by dots is extended so that the dot tracking direction from the first corner to the second corner extends in the clockwise direction. A second dot tracker 4003 that tracks pixels and detects a third corner candidate pixel, and a third corner candidate pixel detected by the second dot tracker 4003 is a third corner (G E) a third corner detector 4004 for determining whether or not the third corner detector 4004, a third dot tracker 4005 for detecting a fourth corner candidate pixel by tracking eight pixels of the code frame in the clockwise direction from the third corner, A fourth corner detector 4006 for determining whether or not the fourth corner candidate pixel is the fourth corner (C, A), a code frame between the fourth corner (C, A) and the first corner (X) And a fourth dot tracker 4007 for detecting dots constituting the.

  According to such a code frame detector 302, in each small region, a code frame (XBGC and EAXD) of a maximum of two two-dimensional code patterns is detected and the code position (the upper left corner of the two-dimensional code pattern, that is, The coordinates on the dot image of X or E) can be detected.

  The first corner detector 4000 will be further described. FIG. 17A is a block diagram of the first corner detector 4000, and FIG. 17B is a diagram for explaining the operation thereof.

  As illustrated in FIG. 17A, the first corner detector 4000 includes a surrounding dot detector 4011 and a point symmetrical pair detector 4012.

  As shown in FIG. 17-2, the surrounding dot detector 4011 includes four or more dots around the pixel of interest (dot) N in a 17 × 17 pixel region centered on the pixel of interest N where the dot exists. Find out if exists. In the example of FIG. 17-2, six surrounding dots A to F are detected. The point-symmetric pair detector 4012 is a pair of dots that are point-symmetric with respect to the target pixel N from the peripheral dots detected by the peripheral dot detector 4011 (the distance from the target pixel N is short, and the intermediate point is the target pixel N). Near dot pair) is detected. In the example of FIG. 17-2, a pair of dots B and D and a pair of dots C and F are detected. A and E are judged as noise and are removed. When such two pairs of dots are detected, the target pixel (dot) N is detected as the first corner. When two pairs of dots are not detected, the target pixel (dot) N is not detected as the first corner, but the first corner is detected again by moving the target pixel.

  In such first corner detection, dots are not traced by normal raster scanning, but dots are traced spirally from the inside to the outside as shown in FIG. The reason is that a complete two-dimensional code pattern without a defect is immediately found near the center of the image area, and thus efficient processing is possible. On the other hand, since a part of the two-dimensional code is missing in the portion near the image end, useless processing increases and processing time increases.

  Next, the first dot tracker 4001 will be further described with reference to FIG. FIG. 19 shows dots constituting the code frame near the first corner of the two-dimensional code pattern, where X is a corner dot at the first corner, and A to I are other dots constituting the code frame.

Now, the dots are traced from X to DEF. For simplicity, A to I and X are regarded as coordinate vectors. First Y = 2D-X
Calculate Y is an estimated vector of dot E. A to D are known because they are detected by the first corner detector 4000. A dot E is traced in 5 × 5 pixels around Y. If E exists,
Y = 2E-D
Calculate Y is an estimated vector of dot F. This tracking is repeated seven times to detect second corner candidate pixels. There are four tracking directions (in FIG. 16, X → A, X → B, X → C, and X → D). Information on all the detected second corner candidate pixels is transferred to the second corner detector 4002.

  The basic operation of the second corner detector 4002 is the same as that of the first corner detector 4000. However, in the case of the second corner detector 4002, there is a possibility that a plurality of dots (maximum 4) to be the target pixel exist, and the maximum number of second corners detected in the second corner detector 4002 (B in FIG. 16). D) is different from the first corner detector 4000.

  The basic operation of the second dot tracker 4003 is the same as that of the first dot tracker 4001. However, in the case of the second dot tracker 4003, there are two dots for starting tracking, and tracking in one direction is performed for each dot (B → G, D → E in FIG. 16), and each tracking is performed. The difference from the first dot tracker 4001 is that the third corner candidate pixel (G, E in FIG. 16) is detected by repeating the hit dot tracking operation 11 times.

  The third corner detector 4004 determines whether the third corner candidate pixel (G, E in FIG. 16) tracked by the second dot tracker 4003 is the third corner and detects the third corner. . The basic operation is the same as that of the first and second corner detectors 4000 and 4002.

  If the third corner detector 4004 detects G and E in FIG. 16 as the third corner, the third dot tracker 4005 performs dot tracking in two directions (G → C and E → A in FIG. 16). Do. The number of times of tracking is the same as that of the first dot tracker 4001, and C and A in FIG. 16 are detected as fourth corner candidate pixels.

  The fourth dot tracker 4007 has the same configuration as that of the second dot tracker 4003. The fourth dot tracker 4007 traces 11 times in each of the C → X and A → X directions in FIG. Since X is already known to be a dot in the first corner, code frames of two two-dimensional code patterns are detected by the tracking path (1) and tracking path (2) shown in FIG. . Which of the two two-dimensional code patterns is to be decoded, the two-dimensional code pattern having the larger horizontal coordinate at the center on the image is given priority. If this two-dimensional code pattern cannot be decoded, the other two-dimensional code pattern is decoded.

  The code frame detector 302 (FIG. 13) has been described above.

  Next, the data acquisition unit 304 (FIG. 13-1) will be described with reference to FIGS. 20-1 and 20-2. FIG. 20A shows an example of one two-dimensional code pattern and a dot image in the vicinity thereof. FIG. 20B is a flowchart for explaining the operation of the data acquisition unit 304.

  As shown in FIG. 20A, attention is focused on dots a1 to a7, b1 to b11, c1 to c7, and d1 to d11 other than the corner dots of the frame (code frame) of the two-dimensional code pattern. The data acquisition unit 304 connects a straight line connecting the corresponding dots a1 to a7 and the dots c1 to c7 arranged in the horizontal direction, and a straight line connecting the corresponding dots b1 to b11 and the dots d1 to d11 arranged in the vertical direction. The coordinates of the intersections (cells) are detected (step 21). Then, for each intersection, the presence or absence of a dot in the 3 × 3 pixel area centered on the coordinates is detected (step 22). When a dot is detected in a 3 × 3 pixel area centered on the coordinates of the intersection, data “1” is acquired for the intersection, and no dot is detected in the 3 × 3 pixel area. Sometimes, data “0” is acquired for the intersection. The data acquisition unit 304 reconstructs the data acquired in this way into code data according to the data arrangement rule of the two-dimensional code pattern (FIG. 4) and outputs the code data.

  Next, the operation of the handwritten coordinate specifying unit 212 (FIG. 9-1) will be described in detail. The input coordinate specifying unit 212 receives binary image data representing only a part to be added output from the writing extraction unit 206 and a position information list output from the two-dimensional code decoding unit 208. As described above, this position information list is obtained by decoding the coordinates of the upper left corner of the two-dimensional code pattern (coordinates on the writing medium) and the image of the upper left corner of the two-dimensional code pattern, which are decoded from each two-dimensional code pattern. It is a list in which coordinates representing coordinates (coordinates on the image) are associated. Since the image on the writing medium is input by the scanner, the distortion of the image can be ignored. The additional coordinate specifying unit 212 can determine the position of the additional portion on the writing medium by linear interpolation using the coordinates on the writing medium and the coordinates on the image for a plurality of two-dimensional code patterns in the position information list. .

  A specific process will be described. First, binary image data representing only the retouched portion is scanned to detect the coordinates (coordinates on the image) of pixels (retouched pixels) of the retouched portion. For each detected additional pixel, the coordinates on the writing medium of three two-dimensional code patterns associated with the on-image coordinate information close to the coordinates are extracted from the position information list.

  This will be described with reference to FIG. In FIG. 21, X is a retouched pixel whose position is to be specified, and A to I are images on the upper left corner of the two-dimensional code pattern that can be decoded within each small region by the two-dimensional code decoding unit 208. Coordinates. What we want now is the coordinates on the writing medium of the additional pixel X. In the example of FIG. 21, since the three points closest to the retouched pixel X among A to I are E, F, and H, the coordinates on the writing medium of the two-dimensional code pattern corresponding to E, F, and H are positioned. It is taken out from the information list, and using this, the coordinates on the writing medium of the additional pixel X are calculated as follows.

The relationship between the coordinates on the image (xi.yi) of E, F, and H and the coordinates on the writing medium (xp, yp) is: xp = a1 * xi + a2 * yi + a3
yp = a4 * xi + a5 * yi + a6
It can be expressed as. a1 to a6 are unknown numbers. Since the coordinates on the image of E, F, and H and the coordinates on the writing medium are known, the unknowns a1 to a6 are obtained by substituting them into the above equation, whereby the coordinates on the writing medium of the writing pixel X (xp, yp) ) Can be specified.

  As described above, the same processing as that of the additional information detection apparatus described above may be executed by software on the information processing apparatus 604 or 609. In this case, a program for realizing the additional information detection process is stored in the storage device 605 of the information processing device 609, for example. Then, this program is loaded into the information processing device 609 or 604, and the command is executed, whereby the additional information detection process is performed. It is assumed that color image data read by scanning a writing medium by the scanner unit of the scanner 603 or the copying machine 602 is stored in the storage device 605 in advance.

  A process procedure for detecting writing information by software will be described with reference to a flowchart shown in FIG.

  First, the color image data of the writing medium is read into the memory from the storage device 605 (step 101). The binarization processing similar to the binarization unit 200 (FIG. 9-1) is performed on the color image data of the writing medium. Only the added portion and the high-density dot are black pixels, and the other pixels are white pixels. The binarized image data is generated (step 102). In addition, dot detection processing similar to that of the dot detection units 202R, 202G, and 202B (FIG. 9-1) is performed for each of the RGB planes of the color image data of the writing medium to detect dots constituting the two-dimensional code pattern. (Step 103). Then, based on the dot detection result, the same dot image composition and dot color detection processing as the image composition / dot color detection unit 204 (FIG. 9-1) is performed, and the dot coordinates and color are stored in the dot coordinate list. Register (step 104).

  Next, on the basis of the binarized image data generated in step 102 and the dot coordinates in the dot coordinate list obtained in step 104, processing similar to that of the retouching extraction unit 206 (FIG. 9-1) is executed, and two-dimensional Binary image data from which the dots of the code pattern are removed is generated (step 105). Further, based on the dot position list generated in step 104, processing similar to that of the region information extraction unit 210 (FIG. 9-1) is executed to extract a region that is a collection of the same color, and region information is acquired ( step 107). Further, the same processing as that of the two-dimensional code decoding unit 208 (FIG. 9-1) is performed based on the dot image data generated in step 104, and the coordinates on the writing medium and the coordinates on the image at the upper left corner of the two-dimensional code pattern are processed. Is generated (step 106). Finally, based on the binary image data remaining in the retouched part generated in step 105 and the position information list generated in step 106, the same process as that of the retouched coordinate specifying unit 212 (FIG. 9-1) is performed. The coordinates on the writing medium of the added pixel are obtained (step 108).

  FIG. 10-2 shows an example of the processing flow in step 103. In the figure, step 201, step 202, and step 203 are steps for performing processing corresponding to the dot candidate determination unit 3101, the minimum luminance pixel determination unit 3102, and the peripheral dot search unit 3103 shown in FIG.

  FIG. 12-2 shows another example of the processing flow in step 103. In the figure, step 200, step 201, step 202, and step 203 perform processing corresponding to the dot search area determination unit 3100, dot candidate determination unit 3101, minimum luminance pixel determination unit 3102, and peripheral dot search unit 3103 shown in FIG. It is a step.

  FIG. 13-2 shows a processing flow in step 106. In the figure, step 301, step 302, step 303, and step 304 are steps corresponding to the small area setting unit 300, code frame detector 302, data acquisition unit 304, and error correction unit 306 shown in FIG. Step 305 and step 306 are processing steps corresponding to the data decoder 308 in FIG.

  As described above, the writing information detection procedure by the writing information detection apparatus or software of the present invention has been described. However, it is obvious that the above description also serves as the description of the writing information detection method of the present invention. The description of the writing information detection procedure by software is also a description of a program for realizing the writing information detection device or the writing information detection method by software on the computer. Such a program, that is, a program that causes a computer to function as each component of the additional information detection apparatus, or a program that causes a computer to execute the processing steps of the additional information detection method is also encompassed by the present invention. Further, various recording (storage) media that can be read by a computer, such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor storage element, on which such a program is recorded are also included in the present invention.

  Here, correspondence between the description of the claims and the embodiment will be described. Referring to FIG. 9A, the binarization unit 200 corresponds to “first processing means”, and the combination of the dot detection units 202R, 202G, and 202B and the image composition / dot color detection unit 204 is “second processing”. Corresponding to “means”, the retouching extraction unit 206 corresponds to “third processing means”, the two-dimensional code decoding unit 208 corresponds to “fourth processing means”, and the retouched coordinate specifying unit 212 corresponds to “fifth processing means”. The area information extraction unit 210 corresponds to “sixth processing means”. Referring to FIG. 9B, step 102 corresponds to “first processing step”, the combination of step 103 and step 104 corresponds to “second processing step”, and step 105 corresponds to “third processing step”. , Step 106 corresponds to the “fourth processing step”, step 108 corresponds to the “fifth processing step”, and step 107 corresponds to the “sixth processing step”.

  This completes the description of the extraction of additional information on the writing medium. Next, utilization of the retouching information extracted from the writing medium will be described. FIG. 22 is a flowchart for explanation.

  The writing medium is added using a writing instrument such as a pencil or a ballpoint pen. The written writing medium is read as color image data by the scanner unit of the scanner 603 or the copying machine 602. Then, the same processing as that of the retouching information detection apparatus operating on the information processing apparatus 609 or 604 is executed by the retouching information detection apparatus shown in FIG. 9A provided inside the scanner 603 or the copier 602. The program acquires additional information (coordinate information), region information, and document ID information of the writing medium based on the color image data of the writing medium (step 401).

  Based on the document ID information, the information processing apparatus 609 specifies the document (or page) associated with the writing medium using the document management database 606, and acquires the document (or page) (step 402). For example, if an item as shown in FIG. 23 is registered in the document management database 606 and the document ID extracted from the writing medium is “123456”, the document is the page 1 of the document name “golf.doc”. Identified as being.

  The information processing apparatus 609 opens the identified document “golf.doc” with an application associated with the document “golf.doc”, displays the document on an unillustrated display, and sets the editing state (step 403). Then, the rewriting information extracted from the writing medium is drawn on the document “golf.doc” in the editing state. Specifically, for example, a new drawing object is opened in a document window (step 404), and the retouching information extracted from the writing medium, that is, the coordinates of the retouched portion is connected by a line (step 405). Update / save processing of “golf.doc” is performed (step 406). In this way, one object as additional information is added to the document “golf.doc”.

  Note that the coordinate information, which is the additional information, is saved as a separate file (coordinate string file) (FIG. 23 shows a case where such a coordinate string file is saved). If you save the retouching information as a separate file in this way, even if the user opens the application later and wants to check the retouched information, even if the document has a huge size that cannot be opened quickly, the separate file There is an advantage that only the added information stored as can be quickly displayed and confirmed.

  In addition to the additional information, area information is extracted from the writing medium. Therefore, for example, in the case of a writing medium associated with an electronic document whose content is a golf course map, the golf course map can be substantially restored by the extracted area information, and the additional information can be drawn on it. That is, without opening the electronic document associated with the writing information, it is possible to create an image in which the contents of the electronic document are almost reproduced and to which the additional information is added, and to print the image. .

It is a figure which shows the example of the two-dimensional code pattern printed on the writing medium of this invention. It is explanatory drawing of the magnitude | size of a dot of a two-dimensional code pattern, and a dot space | interval. It is explanatory drawing of the area | region of a two-dimensional code pattern. It is explanatory drawing of the bit arrangement in each area | region of a two-dimensional code pattern. 1 is a configuration diagram of a document management system according to an embodiment of the present invention. FIG. It is a flowchart for demonstrating the preparation procedure of the printing image of a writing medium. It is explanatory drawing of the data encoded by a two-dimensional code pattern. It is a flowchart for demonstrating the procedure which opens an electronic document and prints the writing medium linked | related with it. It is a block diagram of the retouching information detection apparatus concerning the embodiment of the present invention. It is a flowchart which shows a writing information detection process flow. It is a block diagram which shows the 1st Example of a dot detection part. It is a flowchart which shows a dot detection process flow. It is a figure which shows the surrounding pixel of the attention pixel referred by dot detection. It is a block diagram which shows the 2nd Example of a dot detection part. It is a flowchart which shows a dot detection process flow. It is a block diagram which shows one Example of a two-dimensional code decoding part. It is a flowchart which shows a two-dimensional code decoding process flow. It is a figure for explanation of decoding and error correction of data encoded in a two-dimensional code pattern. It is a block diagram of a code frame detector. It is explanatory drawing of code frame detection. It is a block diagram of a 1st corner detector. It is explanatory drawing of point symmetrical pair detection. It is explanatory drawing of spiral-shaped dot detection. It is explanatory drawing of dot tracking. It is explanatory drawing of the data dot detection of a two-dimensional code pattern. It is a flowchart which shows the processing flow of a data acquisition device. It is explanatory drawing of the method of calculating | requiring the coordinate on the medium of a writing pixel by interpolation. It is a flowchart for demonstrating the process which detects additional information etc. from a writing medium, and utilizes it. It is a figure which shows the example of the registration item of a document management database.

Explanation of symbols

2, 2a to 2d Two-dimensional code pattern 200 Binary unit 202R, 202G, 202B Dot detection unit 204 Image composition / dot color detection unit 206 Retouching extraction unit 208 Two-dimensional code / decoding unit 210 Region information extraction unit 212 Retouched coordinate specification Unit 300 small area setting unit 302 code frame detector 304 data acquisition unit 306 error correction unit 308 data decoder 400 first corner detector 401 first dot tracker 402 second corner detector 403 second dot tracker 404 third Corner detector 405 Third dot tracker 406 Fourth corner detector 407 Fourth dot tracker 411 Surrounding dot detector 412 Point symmetrical pair detector 601 Printer 602 Copying machine 603 Scanner 604, 609 Information processing device 605 Storage device 606 Database 3100 Dot search area determination unit 3101 Dot candidate determination unit 3102 Minimum luminance pixel determination unit 3103 Peripheral dot search unit

Claims (16)

A writing medium associated with a specific electronic document,
A two-dimensional code pattern, which is a dot pattern of a predetermined size, is printed in a matrix on a medium of a material that can be written using an ordinary writing instrument, and each of the two-dimensional code patterns includes the two-dimensional code pattern. Coordinate information indicating a position on the medium is encoded, and dots constituting each of the two-dimensional code patterns are color-coded so as to represent area information of the electronic document. A writing medium associated with a specific electronic document,
A two-dimensional code pattern that is a dot pattern of a predetermined size is printed in a matrix on a medium of a material that can be written using an ordinary writing instrument, and each medium of the two-dimensional code pattern is printed on each two-dimensional code pattern. The coordinate information indicating the position above and the identification information of the electronic document are encoded, and the dots constituting each two-dimensional code pattern are color-coded so as to represent the area information of the electronic document. To write.   The writing medium according to claim 1, wherein an error correction code is also encoded in each two-dimensional code pattern.   4. The writing medium according to claim 1, wherein an area ratio of dots of the two-dimensional code pattern to the whole medium is less than 5%. A writing medium associated with the electronic document, in which two-dimensional code patterns are printed side by side in a matrix, and dots constituting each of the two-dimensional code patterns are color-coded to represent area information of a specific electronic document Is a retouched information detecting device for detecting information rewritten using a writing instrument on the writing medium,
First processing means for binarizing the color image to generate a binarized image in which a background pixel is a white pixel and a non-background pixel is a black pixel;
Based on the color image, among the pixels of the color image, only pixels corresponding to the dots of the two-dimensional code pattern printed in a matrix on the writing medium are represented by black pixels, and the other pixels are represented. Generates a dot image represented by white pixels, and detects and outputs dot coordinates representing the position of the dot of the two-dimensional code pattern on the dot image and a dot color representing the color of the dot of the two-dimensional code pattern Second processing means to:
Based on the dot coordinates output from the second processing means, by removing black pixels corresponding to the dots of the two-dimensional code pattern from the binarized image, it corresponds to the part of the writing medium that has been rewritten. Third processing means for generating a binary image representing the black pixel (hereinafter referred to as an added pixel),
The coordinates representing the position of the two-dimensional code pattern on the writing medium by detecting the two-dimensional code pattern from the dot image and decoding each detected two-dimensional code pattern (denoted as coordinates on the writing medium) And a fourth processing means for outputting the coordinates on the writing medium in association with the coordinates representing the position of the two-dimensional code pattern on the dot image (coordinates on the image);
Based on the coordinates representing the position of each of the retouched pixels on the binary image and the coordinates on the writing medium and the coordinates on the image of the two-dimensional code pattern output from the fourth processing unit. A fifth processing means for obtaining coordinates indicating the position of the writing pixel on the writing medium and outputting it as writing information;
Sixth processing means for generating area information representing an area represented by a group of dots of the same color based on the dot coordinates and the dot color output from the second processing means;
A retouching information detecting apparatus characterized by comprising: The fourth processing means, by decoding the detected two-dimensional code pattern, revision information detecting apparatus according to claim 5, characterized in that also outputs the acquired identification information of the electronic document. The third processing means performs a thinning process on the binary image,
6. The writing information detection apparatus according to claim 5, wherein the fifth processing unit sets a processing pixel to a writing pixel in the binary image after the thinning process.   The second processing means detects dots for each color component of the color image, generates the dot image based on the detection result, and detects the coordinates and colors of the dots of the two-dimensional code pattern. The handwritten information detection apparatus according to claim 5.   6. The retouched information detection apparatus according to claim 5, wherein the fourth processing unit sets a plurality of small areas in the dot image and detects the two-dimensional code pattern in each set small area. A writing medium associated with the electronic document, in which two-dimensional code patterns are printed side by side in a matrix, and dots constituting each of the two-dimensional code patterns are color-coded to represent area information of a specific electronic document Is a retouched information detection method for detecting information rewritten using a writing tool on the writing medium,
A first processing step of binarizing the color image to generate a binarized image in which a background pixel is a white pixel and a non-background pixel is a black pixel;
Based on the color image, among the pixels of the color image, only pixels corresponding to the dots of the two-dimensional code pattern printed in a matrix on the writing medium are represented by black pixels, and the other pixels are represented. Generates a dot image represented by white pixels, and detects and outputs dot coordinates representing the position of the dot of the two-dimensional code pattern on the dot image and a dot color representing the color of the dot of the two-dimensional code pattern A second processing step,
Based on the dot coordinates output from the second processing step, by removing black pixels corresponding to the dots of the two-dimensional code pattern from the binarized image, it corresponds to the part of the writing medium that has been rewritten A third processing step of generating a binary image representing the black pixel (hereinafter referred to as an added pixel);
The coordinates representing the position of the two-dimensional code pattern on the writing medium by detecting the two-dimensional code pattern from the dot image and decoding each detected two-dimensional code pattern (denoted as coordinates on the writing medium) A fourth processing step of outputting the coordinates on the writing medium in association with coordinates (coordinates on the image) representing the positions on the dot image of the two-dimensional code pattern;
Based on the coordinates representing the position of each of the retouched pixels on the binary image and the coordinates on the writing medium and the coordinates on the image of the two-dimensional code pattern output from the fourth processing unit. A fifth processing step of obtaining coordinates indicating the position of the writing pixel on the writing medium and outputting it as writing information;
A sixth processing step for generating region information representing a region represented by a group of dots of the same color based on the dot coordinates and the dot color output from the second processing step;
A retouching information detection method characterized by comprising: The fourth treatment step, by decoding the detected two-dimensional code pattern, revision information detecting method according to claim 10, characterized in that also outputs the acquired identification information of the electronic document. The third processing step performs thinning processing on the binary image,
The retouching information detection method according to claim 10, wherein the fifth processing step targets a retouched pixel in the binary image after the thinning process.   The second processing step detects dots for each color component of the color image, generates the dot image based on the detection result, and detects the coordinates and colors of the dots of the two-dimensional code pattern. The rewritten information detection method according to claim 10.   11. The additional information detection method according to claim 10, wherein the fourth processing step sets a plurality of small areas in the dot image, and detects the two-dimensional code pattern in each of the set small areas. The program which makes a computer function as each means of the retouching information detection apparatus of Claim 5 . A computer-readable recording medium on which a program for causing a computer to function as each means of the additional information detecting apparatus according to claim 5 is recorded.

Images