JP2008299508A - Position detection device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine a pointed position on a medium accurately even if the pointed position on the medium, an image read from the medium and an image used for detecting the position on the medium are misaligned. <P>SOLUTION: A position detection device comprises an image reading part 21 for reading an image, a dot array generation part 22 for generating a dot array from the image, a block detection part 23 for detecting a block on the dot array to generate a code array, a synchronization code detection part 24 for detecting a synchronization code on the code array, an identification code detection part 30, etc. for extracting identification information from the code array, an x-coordinate code detection part 40, a y-coordinate code detection part 45, etc. for extracting coordinate information from the code array, and a coordinate correction part 50 for correcting the coordinate information. The coordinate information correction uses the position of a nib relative to the image read by the image reading part 21, and the position of the image read by the image reading part 21 relative to the image used in the x-coordinate code detection part 40, the y-coordinate code detection part 45, etc. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a position detection device and a program.

  A method for providing a position code for encoding a plurality of positions on a surface is known (see, for example, Patent Document 1). In this patent document 1, a cyclic numeral sequence is printed a plurality of times along the surface. At that time, using different rotations of the cyclic numeral sequences so that a predetermined deviation occurs between the adjacent numeric strings, a plurality of code windows divided on the surface include at least three cyclic numeral sequences and adjacent codes. It has one numeric string that overlaps one numeric string of the window, and the position of the code window is encoded by a shift between the cyclic numeric strings belonging to the code window.

  There is also known a technique that enables spatially asynchronous reading of encoding from a recording medium that records machine-readable encoding of digital quantum that is logically ranked (see, for example, Patent Document 2). In this document, multiple copies of an encoding that are essentially identical can be formed, and a machine-recognizable spatial synchronization index can be combined with the print position in each of the encoding copies to synchronize the encoding spatially. A large number of cases are provided, and these cases are written in a spatially periodic center grid in the recording medium according to the layout rules.

  Systems and methods for determining the position of captured images from larger images are also known (see, for example, Patent Document 3). In this Patent Document 3, a non-repetitive sequence is folded into a non-repetitive sequence unique to each sub-window of a predetermined size, and the captured position is determined for each sub-window in the non-repetitive sequence. Thus, the position of the captured image is determined from the image of the larger subwindow.

  A technique that enables long-time recording and repeated reproduction of multimedia information that is optically readable is also known (see, for example, Patent Document 4). In this patent document 4, a recording apparatus uses a printer system or a printing plate making system as a dot code that can optically read so-called multimedia information including audio information, video information, digital code data, etc. And record on a medium such as paper. The pen-type information reproducing device sequentially captures the dot code according to the manual scanning of the dot code, and the original audio information is output by the audio output device, the original video information is displayed by the display device, and the original digital code data Is output by a page printer or the like.

  There is also known a technique that enables precise detection of coordinates on a medium without using a tablet (see, for example, Patent Document 5). In this patent document 5, a pen-type coordinate input device optically reads a code symbol formed on a medium and indicating the coordinates on the medium, decodes the read code symbol, and reads the code symbol in the read image. The position, orientation, and amount of distortion are detected. Then, the coordinates of the position of the tip on the medium are detected based on the decoded information and the position, orientation, and distortion amount of the code symbol.

JP-T-2003-511762 JP-A-9-185669 JP 2004-152273 A JP-A-6-231466 JP 2000-293303 A

Conventionally, when there is a deviation between the indicated position on the medium, the partial image read from the medium, and the partial image used for detecting the position on the medium, the indication on the medium is indicated. There was a problem that it was not possible to accurately determine the position.
The object of the present invention is to indicate the position on the medium even if there is a deviation between the indicated position on the medium, the partial image read from the medium, and the partial image used for detecting the position on the medium. This is to make it possible to obtain the determined position with high accuracy.

According to the first aspect of the present invention, there is provided an instruction unit for indicating a position on a medium, a reading unit for reading a first partial image included in an image printed on the medium, and the first read by the reading unit. Extracting means for extracting a second partial image representing its own position on the medium from one partial image, and using the second partial image extracted by the extracting means, Detection means for detecting the position of the second partial image; first information indicating a relative position of the position indicated by the indication means with respect to the first partial image; and the first information of the first partial image. Acquisition means for acquiring second information indicating a relative position with respect to the second partial image; and the position of the second partial image detected by the detection means; the first information and the second information; Correction means for correcting using It is a position detecting apparatus according to claim.
According to a second aspect of the present invention, the acquisition unit acquires, as the first information, information that specifies a distance and direction from the center of the first partial image at the position indicated by the instruction unit. The position detecting device according to claim 1.
According to a third aspect of the present invention, the extraction unit extracts the second partial image from the third partial image after extracting the third partial image from the first partial image, and acquires the second partial image. The means includes the second information, third information indicating a relative position of the first partial image with respect to the third partial image, and a relative position of the third partial image with respect to the second partial image. The position detection device according to claim 1, wherein the position detection device is acquired separately from the fourth information.
According to a fourth aspect of the present invention, the acquisition unit acquires, as the third information, information that specifies a distance and a direction of the center of the first partial image from the center of the third partial image. The position detecting device according to claim 3.
The invention according to claim 5 is characterized in that the acquisition means further acquires, as the third information, information specifying a rotation angle of the first partial image with respect to the third partial image. A position detecting device according to claim 4.
According to a sixth aspect of the present invention, the second partial image and the third partial image form a rectangle sharing a vertical axis and a horizontal axis, and the acquisition means The information specifying the displacement in the vertical axis direction and the horizontal axis direction from one vertex of the second partial image of one vertex is acquired as the fourth information. This is a position detection device.
In the seventh aspect of the invention, the second partial image includes a pattern image in which unit images are arranged at n (2 ≦ n <m) of m (m ≧ 4) in the vertical axis direction. Ax (Ax ≧ 2) and Ay (Ay ≧ 2) images are arranged in the horizontal axis direction, and the extracting unit extracts a section having the same size as the pattern image in the vertical axis direction. Ay frames arranged in the horizontal axis direction are moved on the third partial image to obtain a moving position where the number of sections including n unit images is a predetermined number or more. The image in the frame is extracted as the second partial image, and the acquisition means uses the movement amount in the vertical axis direction and the horizontal axis direction of the frame to the movement position as information for specifying the displacement. The position detection device according to claim 6, wherein the position detection device acquires the position detection device.
The invention according to claim 8 extracts a second partial image representing its position on the medium from the first partial image of the image read from the image printed on the medium, to the computer. A function, a function of detecting the position of the second partial image on the medium using the extracted second partial image, and the first of the position on the medium instructed by the instruction unit A first information indicating a relative position of the first partial image and a second information indicating a relative position of the first partial image with respect to the second partial image, and the detected second information It is a program for realizing the function of correcting the position of a partial image using the first information and the second information.
In the invention according to claim 9, in the function of extracting, after extracting the third partial image from the first partial image, the second partial image is extracted from the third partial image, In the function to obtain, the second information includes the third information indicating a relative position of the first partial image with respect to the third partial image, and the second partial image of the third partial image. 9. The program according to claim 8, wherein the program is acquired separately from fourth information indicating a relative position.

The invention of claim 1 has this configuration even if there is a deviation between the indicated position on the medium, the partial image read from the medium, and the partial image used for detecting the position on the medium. As compared with the case where the recording is not performed, the instructed position on the medium can be obtained with high accuracy.
The invention of claim 2 has an effect that the calculation for obtaining the designated position on the medium from the partial image read from the medium becomes easier than in the case where the present configuration is not provided.
The invention of claim 3 does not have this configuration even if it is necessary to extract a part of a partial image read from the medium and extract a partial image used for detecting a position on the medium therefrom. Compared to the case, the instructed position on the medium can be obtained with high accuracy.
The invention of claim 4 has an effect that the calculation for obtaining the partial image read from the medium from the partial image extracted therefrom becomes easier than in the case where the present configuration is not provided.
According to the fifth aspect of the present invention, even if there is a difference in angle between the partial image read from the medium and the partial image extracted from the partial image, the partial image read from the medium is extracted from the partial image. It has the effect that it can obtain | require from.
The invention according to claim 6 has an effect that calculation for obtaining a partial image of the partial image read from the medium from the partial image used for detecting the position on the medium is facilitated.
According to the seventh aspect of the present invention, a partial image used for detecting a position on a medium is obtained from a partial image read from the medium by using a result in a decoding process when the mCn method is used. It has the effect of being able to.
The invention of claim 8 has this configuration even if there is a deviation between the indicated position on the medium, the partial image read from the medium, and the partial image used for detecting the position on the medium. As compared with the case where the recording is not performed, the instructed position on the medium can be obtained with high accuracy.
The invention of claim 9 does not have this configuration even if it is necessary to extract a part of a partial image read from the medium and extract a partial image used for detecting a position on the medium therefrom. Compared to the case, the instructed position on the medium can be obtained with high accuracy.

The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described in detail below with reference to the accompanying drawings.
First, a code pattern constituting a code image to be read will be described.
In the present embodiment, mCn (= m) by a pattern image (hereinafter referred to as “unit code pattern”) in which unit images are arranged at n (1 ≦ n <m) locations selected from m (m ≧ 3) locations. ! / {(Mn)! × n!}) Expresses the information as shown. That is, instead of associating one unit image with information, a plurality of unit images are associated with information. Assuming that one unit image is associated with information, there is a drawback in that erroneous information is expressed when the unit image is lost or noise is added. On the other hand, for example, if two unit images are associated with information, it can be easily understood that there is an error when the number of unit images is one or three. Furthermore, in the method of expressing 1 bit or at most 2 bits in one unit image, it is not possible to express a synchronous pattern that controls reading of the information pattern with a pattern visually similar to the pattern that represents information. . For this reason, the present embodiment employs the above encoding method. Hereinafter, such an encoding method is referred to as an mCn method.
Here, a unit image having any shape may be used. In the present embodiment, a dot image (hereinafter simply referred to as “dot”) is used as an example of a unit image, but an image having another shape such as a hatched pattern may be used.

FIG. 1 shows an example of a unit code pattern in the mCn system.
In the figure, a black area and a hatched area are areas where dots can be arranged, and a white area between them is an area where dots cannot be arranged. Of the areas where dots can be arranged, dots are arranged in black areas and dots are not arranged in hatched areas. That is, FIG. 1 shows an example in which an area where a total of 9 dots of 3 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is provided. FIG. FIG. 6B shows a unit code pattern, and FIG. 5B shows a unit code pattern in the 9C3 method in which 3 dots are arranged in an area where dots can be arranged.

  By the way, the dots (black areas) arranged in FIG. 1 are only for information representation, and do not necessarily match the dots (minimum squares in FIG. 1) which are the minimum units constituting the image. In this embodiment, when “dot” refers to the former dot and the latter dot is referred to as “pixel”, the dot has a size of 2 pixels × 2 pixels at 600 dpi. . Since the size of one pixel at 600 dpi is 0.0423 mm, one side of the dot is 84.6 μm (= 0.0423 mm × 2). The larger the dot, the more likely it will be noticeable, so it is preferable to make it as small as possible. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. However, the above value 84.6 μm is a numerical value to the last, and is about 100 μm in the actually printed toner image.

FIG. 2 shows another example of the unit code pattern. In this figure, spaces between dots are omitted.
FIG. 2A shows all unit code patterns in the 9C2 system shown in FIG. In the 9C2 system, 36 (= ₉ C ₂ ) types of information are expressed using these unit code patterns. FIG. 2B shows all unit code patterns in the 9C3 system shown in FIG. In the 9C3 system, 84 (= ₉ C ₃ ) types of information are expressed using these unit code patterns.

The unit code pattern is not limited to m = 9. For example, a value such as 4, 16 may be adopted as the value of m. Also, any value may be adopted as long as n satisfies 1 ≦ n <m.
Further, as the value of m, not only a square number (the square of an integer) but also a product of two different integers may be adopted. That is, the area in which dots can be arranged is not limited to a square area as in the case of arranging 3 dots × 3 dots, but may be a rectangular area as in the case of arranging 3 dots × 4 dots. In this specification, the rectangle means a rectangle in which the lengths of two adjacent sides are not equal.

By the way, all unit code patterns in the mCn system are assigned pattern values that are numbers for identifying each unit code pattern.
FIG. 3 shows the correspondence between unit code patterns and pattern values in the 9C2 system. Since there are 36 unit code patterns in the 9C2 system, 0 to 35 are used as pattern values. However, the correspondence shown in FIG. 3 is merely an example, and any pattern value may be assigned to any unit code pattern. In addition, although not shown in the figure for the 9C3 system, each unit code pattern is assigned a pattern value of 0 to 83.

  In this way, mCn types of unit code patterns are prepared by selecting n locations from m locations. In this embodiment, a specific pattern is used as an information pattern among these unit code patterns. The rest is used as a synchronization pattern. Here, the information pattern is a pattern representing information to be embedded in the medium. The synchronization pattern is a pattern used for taking out an information pattern printed on a medium. For example, it is used for specifying the position of the information pattern or detecting the rotation of the image. Any medium may be used as long as an image can be printed. Since paper is a representative example, the medium will be described as paper below, but metal, plastic, fiber, or the like may be used.

Here, the synchronization pattern will be described.
FIG. 4 is an example of a synchronization pattern in the 9C2 system. Thus, when m in the mCn system is a square number, it is necessary to prepare four types of unit code patterns as synchronization patterns in order to detect image rotation. Here, the unit code pattern of the pattern value 32 is an upright synchronization pattern. In addition, the unit code pattern of pattern value 33 is rotated 90 degrees to the right, the synchronization pattern of unit value pattern 34 is rotated 180 degrees to the right, and the unit code pattern of pattern value 35 is rotated 270 degrees to the right. The synchronization pattern is used. In this case, 5 bits of information may be expressed by using the remainder obtained by removing these four types of synchronization patterns from the 36 types of unit code patterns as an information pattern. However, the method of assigning 36 types of unit code patterns to information patterns and synchronization patterns is not limited to this. For example, five sets of synchronization patterns that constitute one set of four types may be prepared, and the remaining 16 types of information patterns may represent 4-bit information.

  Although not shown, when m in mCn is a product of two different integers, two types of unit code patterns may be prepared as synchronization patterns in order to detect image rotation. For example, when an area in which 3 dots in the vertical direction and 4 dots in the horizontal direction should be detected but an area in which the 4 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is detected, the image is 90 degrees or 270 degrees at that time. It is because it turns out that it is rotating.

Next, a method for expressing identification information and coordinate information by arranging such code patterns will be described.
First, a code block composed of the above-described unit code pattern will be described.
FIG. 5 shows an example of a code block. In the figure, it is also shown that a unit code pattern in the 9C2 system is used. That is, 36 types of unit code patterns are divided into, for example, 4 patterns used as synchronization patterns and 32 patterns used as information patterns, and each pattern is arranged according to a layout.
In the figure, a layout in which 3 areas × 3 dots area (hereinafter referred to as “block”) 25 × 5 × 5 are arranged is used. Of the 25 blocks, a synchronization pattern is arranged in the upper left block. In addition, an information pattern representing X coordinate information that specifies coordinates in the X direction on the paper surface is arranged in the four blocks to the right of the synchronization pattern, and the Y direction coordinates on the paper surface are specified in the four blocks below the synchronization pattern. An information pattern representing coordinate information is arranged. Further, an information pattern representing the identification information of the document printed on the paper surface or the paper surface is arranged in 16 blocks surrounded by the information pattern representing the coordinate information.
The unit code pattern in the 9C2 system is merely an example, and the unit code pattern in the 9C3 system may be used as shown in the lower left of the figure.

Next, a layout for expressing identification information and coordinate information in this embodiment will be described.
FIG. 6 shows a part of such a layout.
In this layout, the code block shown in FIG. 5 is used as a basic unit, and this code block is periodically arranged on the entire sheet. In the figure, the synchronization pattern is represented by “S”. Also, the coordinate information is expressed in M series over the vertical and horizontal sides of the page. In the figure, patterns representing the X coordinate are represented by “X1”, “X2”,..., And patterns representing the Y coordinate are represented by “Y1”, “Y2”,. Furthermore, although some methods can be used for encoding the identification information, RS encoding is suitable in this embodiment. This is because the RS code is a multi-level coding method, and in this case, the block representation should correspond to the multi-value of the RS code. In the figure, the pattern representing the identification information is represented by “IXY” (X = 1 to 4, Y = 1 to 4).

Here, the M series used for expressing the coordinate information will be described.
The M sequence is a sequence whose partial sequence does not match other partial sequences. For example, the eleventh order M-sequence is a bit string of 2047 bits. The same sequence as the partial sequence of 11 bits or more extracted from the 2047-bit bit string does not exist in the 2047-bit bit string other than itself. In this embodiment, 16 unit code patterns are associated with 4 bits. That is, a bit string of 2047 bits is expressed in decimal notation for every 4 bits, a unit code pattern is determined according to the correspondence of FIG. 3, and is expressed as a coordinate code across the paper in the horizontal and vertical directions. Therefore, at the time of decoding, the position on the bit string is specified by specifying three consecutive unit code patterns.

FIG. 7 shows an example of encoding coordinate information using an M sequence.
(A) shows a bit string “0111000101011010000110010...” As an example of the 11th order M-sequence. In the present embodiment, this is divided into 4 bits, and the first partial sequence “0111” is a unit code pattern of pattern value 7 and the second partial sequence “0001” is a unit code pattern of pattern value 1. As a result, the third partial sequence “0101” is arranged as a unit code pattern with a pattern value of 5, and the fourth partial sequence “1010” is arranged as a unit code pattern of a pattern value of 10, respectively.

In addition, if the unit code pattern is divided by 4 bits and assigned to the unit code pattern in this way, as shown in (b), all pattern strings can be expressed in 4 cycles. That is, since the 11th order M-sequence is 2047 bits, if this sequence is cut out every 4 bits and expressed by a unit code pattern, 3 bits are added at the end. The first 1 bit of the M sequence is added to these 3 bits to form 4 bits, which are represented by a unit code pattern. Further, the unit code pattern is cut out every 4 bits from the second bit of the M sequence. Then, the next cycle starts from the third bit of the M sequence, and the next cycle starts from the fourth bit of the M sequence. Furthermore, the fifth cycle starts from the fifth bit, which coincides with the first cycle. Therefore, if four cycles of the M sequence are cut out every 4 bits, it is possible to use all 2047 unit code patterns. Since the M sequence is 11th order, the three consecutive unit code patterns do not match the continuous code pattern at any other position. Therefore, decoding can be performed by reading three unit code patterns at the time of reading. However, in the present embodiment, coordinate information is expressed by four unit code patterns in consideration of the occurrence of errors.
Note that the M sequence of four periods is divided and stored in 2047 blocks. The length of one side of one block is 0.508 mm when the printing conditions described above, i.e., the dot has a size of 2 pixels × 2 pixels at 600 dpi. When the layout of FIG. 6 is adopted, a synchronization block is inserted every 4 blocks of 2047 consecutive blocks. Therefore, the length is 1299.5 mm because the total is about 2558 blocks multiplied by 5/4. That is, a length of 1299.5 mm will be encoded.

Next, the image generation apparatus 10 that generates such an image will be described.
FIG. 8 is a block diagram illustrating a configuration example of the image generation apparatus 10.
As shown in the figure, the image generation apparatus 10 includes an information acquisition unit 11, an identification code generation unit 12, an X coordinate code generation unit 13, a Y coordinate code generation unit 14, a code array generation unit 15, and a pattern image storage. Unit 16 and a code image generation unit 17.

  The information acquisition unit 11 acquires the identification information of the paper or the document printed on the paper and the coordinate information on the paper. Then, the former is output to the identification code generation unit 12, and among the latter, the X coordinate information is output to the X coordinate code generation unit 13 and the Y coordinate information is output to the Y coordinate code generation unit 14.

The identification code generation unit 12 encodes the identification information acquired from the information acquisition unit 11 and outputs the identification code that is the encoded identification information to the code array generation unit 15. The coding of the identification information includes block division and RS coding processing.
Among these, the block division is a process of dividing the bit string constituting the identification information into a plurality of blocks in order to perform RS encoding. For example, when an information pattern capable of expressing 5-bit information in the 9C2 system is used, the 60-bit identification information is divided into 12 blocks having a block length of 5 bits.
RS encoding is a process of performing RS encoding on the divided blocks and adding redundant blocks for error correction. If an RS code capable of correcting an error of 2 blocks is adopted in the previous example, the code length is 16 blocks.

The X coordinate code generation unit 13 encodes X coordinate information. The Y coordinate code generation unit 14 encodes the Y coordinate information. And these output the coordinate code | cord | chord which is the encoded coordinate information to the code arrangement | sequence production | generation part 15. FIG. The encoding of the coordinate information includes M-sequence encoding and block division processing.
Among these, the M sequence encoding is a process of encoding coordinate information using the M sequence. For example, a necessary M-sequence order is obtained from the length of coordinate information to be encoded, and a coordinate code is generated by dynamically generating the M-sequence. However, when the length of the coordinate information to be encoded is known in advance, the M series may be stored in the memory or the like of the image generation apparatus 10 and read out when generating the image.
Block division is processing for dividing an M sequence into a plurality of blocks. For example, if 16 types of unit code patterns are selected as the information pattern, 4-bit information is stored in each block in the code block. Therefore, 16 bits of X coordinate information are stored for a code block having a layout as shown in FIG. Then, the coordinate information is represented by the unit code pattern by the method described with reference to FIG.

The code array generation unit 15 reads the identification code generated by the identification code generation unit 12, the coordinate code generated by the X coordinate code generation unit 13 and the Y coordinate code generation unit 14, and the identification code and the coordinate code. Are arranged on a two-dimensional plane to generate a two-dimensional code array. Here, the synchronization code is a code corresponding to the synchronization pattern.
The pattern image storage unit 16 stores, for example, a unit code pattern in the mCn system shown in FIG. Here, the pattern value is attached to the unit code pattern as described above, and each unit code pattern can be read using the pattern value as a key. In the present embodiment, the pattern image storage unit 16 may store at least unit code patterns in the mCn system corresponding to predetermined m and n.
The code image generation unit 17 refers to the two-dimensional code array generated by the code array generation unit 15 and selects a unit code pattern corresponding to each code value to generate a code image.

  The code image is transferred to an image forming unit (not shown), and the image forming unit forms a code image on the sheet. At this time, the image forming unit may form a superimposed image in which the document image of the electronic document and the code image are superimposed on the paper surface. In addition, when a superimposed image is formed in this way, the correspondence between the position on the electronic document and the position on the paper is managed so that the writing data by the pen device on the paper is reflected at an appropriate position on the electronic document. It is desirable to do.

The image forming unit forms the code image with K toner (infrared light absorbing toner containing carbon) or special toner, for example, using an electrophotographic system.
Here, as the special toner, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm) is exemplified. . Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished depending on whether or not an image formed on a printed medium can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.

  These functions are realized by cooperation between software and hardware resources. Specifically, the CPU 91 (see FIG. 26) of the image generation device 10 includes the information acquisition unit 11, the identification code generation unit 12, the X coordinate code generation unit 13, the Y coordinate code generation unit 14, the code array generation unit 15, and the code. For example, the program for realizing the image generation unit 17 is realized by reading the program from the magnetic disk device 93 (see FIG. 26) into the main memory 92 (see FIG. 26) and executing it. The pattern image storage unit 16 may be realized using, for example, a magnetic disk device 93 (see FIG. 26). Furthermore, the program and data stored in the magnetic disk device 93 (see FIG. 26) may be loaded from a recording medium such as a CD, or may be downloaded via communication means such as the Internet.

When the pen device indicates a predetermined position on the image on which the unit code pattern described above is printed, an image sensor provided in the pen device reads a part of the image. Then, a dot pattern is read from this image and a decoding process is performed. However, in this case, the decoded coordinate information represents the position of the image read by the image sensor, and does not strictly represent the position designated by the pen device.
The occurrence of this deviation will be specifically described below.
FIG. 9 is a schematic view of a pen device for this explanation.
As shown in the figure, between the point where the pen center line passing through the center of the pen intersects the paper surface (center of the pen tip) and the point where the image sensor center line passing through the center of the image sensor intersects the paper surface (center of the imaging area) Has a deviation IP unique to the pen device.

The deviation between the decoded coordinate information and the position designated by the pen device includes not only this fixed deviation but also a deviation that is dynamically determined according to the result of the image processing. This will be described using the coordinates on the paper.
FIG. 10 shows an area (hereinafter referred to as “imaging area”) captured by the imaging device of the pen device, a center point of the area (hereinafter referred to as “imaging area center”), and the pen device for decoding in The area to be read (hereinafter referred to as “read area”), the center point of the area (hereinafter referred to as “read area center”), and the point indicated by the pen tip of the pen device (hereinafter referred to as “pen point center”) It is the figure which showed the relationship.

Here, as shown in (a), the relative coordinates of the center of the pen tip with respect to the center of the imaging region are (IPX, IPY). The coordinates vary depending on the tilt of the pen device, but a standard tilt is determined, and a value when the pen device is placed at the standard tilt may be set. In this specification, the X axis is parallel to the horizontal frame of the imaging area, the right direction is the positive X direction, the Y axis is parallel to the vertical frame of the imaging area, and the downward direction is the positive Y direction. The coordinates on the captured image are represented.
Further, the vertical and horizontal direction of the imaging region does not always coincide with the direction of the dot row due to the rotation of the paper. Therefore, an angle from the direction of the dot row to the vertical direction of the imaging region is defined as IRD. Although there are two dot row directions, the direction is closest to the vertical direction of the image sensor. At this time, the IRD takes an angle from −90 ° to 90 °. In the present specification, the angle is clockwise and the positive direction.

Now, the pen device reads an area of 17 dots × 17 dots necessary for decoding from the imaging area. This is a reading area, but a reading area appropriate for decoding does not necessarily exist near the center of the imaging area. There may be a case where an appropriate reading area is shifted from the center due to the influence of illumination or the like.
FIG. 4B is an enlarged view of a part including the center of the imaging region and the center of the reading region in FIG. Here, the relative coordinates of the center of the imaging region with respect to the center of the reading region are (IRX, IRY).

Next, the image processing apparatus 20 that reads and processes the code image formed on the paper surface will be described.
FIG. 11 is a block diagram illustrating a configuration example of the image processing apparatus 20.
As illustrated, the image processing apparatus 20 includes an image reading unit 21, a dot array generation unit 22, a block detection unit 23, a synchronization code detection unit 24, a rotation determination unit 25, and a code array rotation unit 26. Prepare. In addition, an identification code detection unit 30, an identification code restoration unit 31, an identification code decoding unit 32, an identification code error detection unit 33, and an identification code error correction unit 34 are provided. Furthermore, an X coordinate code detection unit 40, an X coordinate code decoding unit 42, an X coordinate code error detection unit 43, an X coordinate code error correction unit 44, a Y coordinate code detection unit 45, and a Y coordinate code decoding unit 47. A Y coordinate code error detection unit 48, a Y coordinate code error correction unit 49, a coordinate correction unit 50, and an information output unit 55.

  The image reading unit 21 reads a code image printed on a paper surface using an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). In the present embodiment, an image reading unit 21 is provided as an example of a reading unit that reads the first partial image.

  The dot array generation unit 22 detects dots from the read code image, and generates a dot array with reference to the dot positions. Here, dot detection is performed as follows. That is, first, an area where dots are easily detected is specified from the read code image. Then, in that area, the dot image portion and other background image portions are separated by binarization processing, and the dot position is detected from each binarized image position. Also, the dot array is generated by replacing the dot detected as an image with digital data, such as “1” for the position where the dot is present and “0” for the position where there is no dot on the two-dimensional array. Do. In the present embodiment, a dot array generation unit 22 is provided as an example of a function of extracting the third partial image of the extraction unit.

  The block detection unit 23 detects a block corresponding to the unit code pattern in the code block on the dot array. In other words, a rectangular block delimiter having the same size as the unit code pattern is appropriately moved on the dot array, the position where the number of dots in the block is equal is the correct block delimiter position, and the code that stores the pattern value in each block Create an array. In the present embodiment, the block detection unit 23 is provided as an example of the function of extracting the second partial image of the extraction unit.

The synchronization code detector 24 refers to the type of each unit code pattern detected from the dot array and detects the synchronization code.
The rotation determination unit 25 determines image rotation based on the detected synchronization code. For example, when a square unit code pattern is used, there is a possibility of rotating in units of 90 degrees. Therefore, the direction is detected depending on which of the four types of synchronization patterns the detected synchronization code corresponds to. Further, when a rectangular unit code pattern is used, there is a possibility that the unit code pattern is rotated by 180 degrees. Therefore, the direction is detected depending on which of the two types of synchronization patterns the detected synchronization code corresponds to.
The code array rotation unit 26 rotates the code array by the rotation angle detected by the rotation determination unit 25 and sets the code array in the correct direction.

The identification code detection unit 30 detects the identification code from the code array whose angle is corrected with reference to the position of the synchronization code. In the present embodiment, the identification code detection unit 30 is provided as an example of a detection unit that detects a bit string and a determination unit that determines a bit string.
The identification code restoration unit 31 rearranges the detected identification codes in the correct code order.
The identification code decoding unit 32 decodes the identification code using the same parameters (such as the number of blocks) used in the RS code encoding process described with reference to FIG. 8, and outputs identification information.
The identification code error detection unit 33 detects an error of the decoded identification code, and the identification code error correction unit 34 corrects the error when the detected error is a correctable error.

The X coordinate code detection unit 40 detects the X coordinate code from the code array whose angle is corrected with reference to the position of the synchronization code.
The X coordinate code decoding unit 42 extracts the M series partial sequence from the detected X coordinate code, refers to the position of this partial sequence in the M sequence used for image generation, and corrects this position with the offset by the synchronization code The value is output as X coordinate information.
The X coordinate code error detection unit 43 detects an error in the decoded X coordinate code, and the X coordinate code error correction unit 44 corrects the error when the detected error is a correctable error.

The Y coordinate code detection unit 45 detects the Y coordinate code from the code array whose angle is corrected with reference to the position of the synchronization code.
The Y coordinate code decoding unit 47 extracts the M series partial sequence from the detected Y coordinate code, refers to the position of this partial sequence in the M sequence used for image generation, and corrects this position with the offset by the synchronization code. The value is output as Y coordinate information.
The Y coordinate code error detection unit 48 detects an error of the decoded Y coordinate code, and the Y coordinate code error correction unit 49 corrects the error when the detected error is a correctable error.

  In the present embodiment, as an example of detection means for detecting the position of the second partial image, an X coordinate code detection unit 40, an X coordinate code decoding unit 42, an X coordinate code error detection unit 43, an X coordinate code error A correction unit 44, a Y coordinate code detection unit 45, a Y coordinate code decoding unit 47, a Y coordinate code error detection unit 48, and a Y coordinate code error correction unit 49 are provided.

The coordinate correction unit 50 corrects the coordinate information composed of the X coordinate information output from the X coordinate code decoding unit 42 and the Y coordinate information output from the Y coordinate code decoding unit 47 to obtain the coordinate information of the nib center. get. Here, the correction is performed using a relative coordinate with respect to the center of the imaging area given from the center of the pen tip, a relative coordinate with respect to the center of the imaging area given from the image reading unit 21, or the like. In the present embodiment, as an example of an acquisition unit that acquires information used to correct the position of the second partial image, and as an example of a correction unit that corrects the position of the second partial image, a coordinate correction unit 50 is provided.
The information output unit 55 outputs the identification information obtained from the identification code decoding unit 32 and the coordinate correction unit 50, and the X coordinate information and Y coordinate information of the nib center.

  These functions are realized by cooperation between software and hardware resources. Specifically, the CPU 91 (see FIG. 26) of the image processing apparatus 20 includes a dot array generation unit 22, a block detection unit 23, a synchronization code detection unit 24, a rotation determination unit 25, a code array rotation unit 26, and an identification code detection unit. 30, an identification code restoration unit 31, an identification code decoding unit 32, an identification code error detection unit 33, an identification code error correction unit 34, an X coordinate code detection unit 40, an X coordinate code decoding unit 42, an X coordinate code error detection unit 43, An X coordinate code error correction unit 44, a Y coordinate code detection unit 45, a Y coordinate code decoding unit 47, a Y coordinate code error detection unit 48, a Y coordinate code error correction unit 49, a coordinate correction unit 50, and an information output unit 55 are realized. For example, the program is realized by reading the program from the magnetic disk device 93 (see FIG. 26) into the main memory 92 (see FIG. 26) and executing it. Further, the program and data stored in the magnetic disk device 93 (see FIG. 26) may be loaded from a recording medium such as a CD, or may be downloaded via communication means such as the Internet.

Next, an outline of the operation of the image processing apparatus 20 will be described.
First, the image reading unit 21 reads a code image printed on a paper surface with an imaging element.
Then, the dot array generation unit 22 cuts out a region suitable for dot detection from the read code image. This region may be cut out by known image processing. For example, the following method can be considered. That is, first, the imaging area is divided into a plurality of small areas. In this case, each small area is larger than one dot. Next, for each small region, the contrast between the dot portion and the portion other than the dot is evaluated. Then, the surrounding small areas having the next highest contrast are connected with the small area having the highest contrast as the center. This process is repeated, and the area is expanded until it is large enough to cover the size of 17 dots × 17 dots, and the finally obtained area is cut out from the imaging area.

When an area suitable for dot detection is cut out in this way, the dot array generation unit 22 detects a reading area having a size of 17 dots × 17 dots from the area. Further, the inclination of the dot row in this dot image is also obtained.
In this case, the reading area may be selected from the central part of the cut-out area, but it is preferable to use the evaluation information of the contrast between the dot part and the part other than the dot.
There are various methods for obtaining the inclination of the dot row. For example, the following method may be used. That is, first, a plurality of dot pairs composed of two dots are detected from the dots in the cut-out area. In this case, the dot pair is selected such that the distance between the two dots is equal to a predetermined distance. In the present embodiment, since information is expressed by a unit code pattern of the mCn system as shown in FIG. 1, the predetermined distance here is two dots at adjacent positions on the left and right or top and bottom in FIG. Adopt the distance when placed. Thereafter, the plurality of dot pairs are divided into two groups whose inclinations are orthogonal to each other, and the inclinations of the dot rows are obtained by obtaining the respective inclinations.

  In the dot array generation unit 22, for example, the reading area is specified from the imaging area by performing the image processing as described above, and at this time, the relative position (IRX, IRY) of the imaging area center with respect to the reading area center is known. Therefore, this is stored in a memory (not shown). Further, the inclination of the dot row obtained above is also stored in a memory (not shown) as an IRD. However, IRD is an angle in the vertical direction of the imaging region from the inclination of the dot row. The direction of the dot row is closest to the vertical direction of the image sensor. Therefore, the IRD takes an angle from -90 ° to 90 °.

Next, generation of a dot array in the dot array generator 22 will be described.
FIG. 12 is a diagram illustrating conversion from a dot image to a dot array by the dot array generation unit 22. As shown in the figure, the dot array generation unit 22 converts the image in the reading area into a 17 × 17 dot array. Each element of this array stores “1” if there is a dot at the corresponding position, and “0” if there is no dot at the corresponding position.

Further, the block detection unit 23 detects a block which is the minimum unit of information expression on the dot array corresponding to the reading area.
FIG. 13 is a diagram specifically showing processing when detecting a block by moving a block break. Here, it is known that encoding has been performed by the 9C2 system, and a case of decoding by the 9C2 system is shown.
First, the block detection unit 23 acquires a dot array from the dot array generation unit 22. The size of the dot arrangement acquired here is preset, and is (number of blocks necessary for decoding × number of dots on one side of block + number of dots on one side of block−1) ² . However, since this dot arrangement corresponds to an arbitrarily selected area of the image, the position of the block delimiter is unknown. Therefore, first, block division is performed with reference to the end of the dot array. In this example, since m = 9, block delimiters composed of blocks each having a size of 3 dots × 3 dots are overlapped. Next, the dots in each block are counted. In this example, since n = 2, the position of the block delimiter when there are two dots in each block is the correct delimiter position, but it can be seen that the number of dots varies at this position and is not correct. Therefore, the number of dots in the block is counted by shifting the block delimiter. That is, the same operation is performed for the start position in the right direction, the position moved by one dot, and the position moved by two dots. In addition, the same operation is performed for each of these positions with respect to the start position in the downward direction, the position moved by 1 dot, and the position moved by 2 dots. As a result, the number of dots in all the blocks is “2” at a position where the dot has moved one dot to the right and moved two dots downward. Therefore, this position is set as a correct separation position.

FIG. 14 shows the correct delimiter positions determined by the block detection unit 23.
Here, when the block delimiter is overlapped at the position indicated by the bold line, two “1” s are included in all the blocks, so this position is the correct block delimiter position. The block detection may be started from any position. Here, the starting position (upper left) of the dot arrangement is used as a starting point, and the amount of movement in the X direction from this starting point to the correct block break position is MX, The amount of movement in the Y direction is MY. In the figure, MX = 0 and MY = 1. Hereinafter, an area of 5 blocks × 5 blocks obtained by overlapping block delimiters at the correct delimiter positions is referred to as a “decoding area”.

Further, the block detection unit 23 obtains a pattern value from the dot positions (arrangement of “0” and “1”) in each block. At this time, the correspondence between the dot position and the pattern value shown in FIG. 3 is referred to.
FIG. 15 is a diagram illustrating an example when pattern values are assigned to each block.
For example, since the upper left block indicates that there are dots at the upper left and lower right in the block, the pattern value is 0 from the correspondence in FIG. Further, in the right block, since it is shown that there are dots at the upper left and right center in the block, the pattern value is 12 from the correspondence in FIG.

Next, the synchronization code detection unit 24 searches for the pattern value of the synchronization code from among them. In the present embodiment, the unit code pattern having pattern values of 32 to 35 is used as the synchronization pattern, so this number is searched. In the example of FIG. 15, the pattern value 35 is a synchronization code.
Referring to FIG. 4, the synchronization code of the pattern value 35 is a synchronization code obtained by rotating the synchronization code of the pattern value 32, which is an upright synchronization code, to the right by 270 degrees. Therefore, in order to make the code array upright, the code array needs to be rotated 270 degrees to the left. In that case, it is necessary to change the pattern value of each element in the code array to the pattern value when rotated 270 degrees to the left.

  FIG. 16 shows the pattern value of the unit code pattern at the erect position, the pattern value of the unit code pattern obtained by rotating the unit code pattern at the erect position 90 degrees to the right, and the unit code pattern at the erect position 180 degrees to the right. It is a correspondence table | surface of the pattern value of the unit code pattern which rotated, and the pattern value of the unit code pattern which rotated the unit code pattern of the erect position 270 degree | times to the right. Such a correspondence table may be stored in a memory (not shown), and the synchronization code detection unit 24 may refer to this correspondence table and change pattern values.

FIG. 17A specifically shows the pattern value change at this time. For example, a block with a pattern value of 0 at the upper left turns into a block with a pattern value of 1 at the upper right by rotating 270 degrees to the left. Further, the block of the pattern value 12 on the right is rotated to the left by 270 degrees to be the second block of pattern values 13 from the top of the rightmost column.
In addition, when the code array is rotated, values used for conversion from the decoding area to the reading area and conversion from the coordinates on the captured image to the coordinates on the code image are also changed. The former is the relative coordinates (RDX, RDY) of the upper left point of the reading area with respect to the upper left point of the decoding area, and the latter is the angle RDD of the imaging area with respect to the reading area.
Therefore, as shown in FIG. 17B, the synchronization code detection unit 24 obtains these pieces of information for each synchronization pattern in addition to the rotation angle, and stores them in a memory (not shown) in association with each other.

Thereafter, the X coordinate code detection unit 40, the X coordinate code decoding unit 42, the X coordinate code error detection unit 43, and the X coordinate code error correction unit 44 are other than the block in which the synchronization code of the row in which the synchronization code is arranged is arranged. The position in the X direction of each block is obtained from 4 blocks. The Y coordinate code detection unit 45, the Y coordinate code decoding unit 47, the Y coordinate code error detection unit 48, and the Y coordinate code error correction unit 49 are other than the block in which the synchronization code of the column in which the synchronization code is arranged is arranged. The position in the Y direction of each block is obtained from 4 blocks.
In FIG. 18, the right figure is an example in which the synchronization codes are arranged in the second block from the left and the second block from the top. The blocks sandwiching the synchronization code in the row where the synchronization code is arranged are “X1”, “X2”, “X3”, and “X4”. By comparing the combination of the pattern values of the unit code patterns arranged in these blocks with the pattern string shown in FIG. 7, it is possible to determine what number block in the pattern string is “X1”. Then, by correcting the number of blocks in which the synchronization pattern is arranged, the block position BX in the X direction of “X1” is obtained. Similarly, for the Y direction, the block position BY in the Y direction of “Y1” is obtained.

When the block positions in the X direction and the Y direction are obtained in this way, the coordinate correction unit 50 starts a series of processes for converting the coordinate information in the decoding area into the coordinate information of the nib center.
In FIG. 18, in the left diagram, the pixel position (DSX, DSY) of the start point of the decoding area is shown in gray. That is, the leftmost pixel position of “X1” is DSX, and the uppermost pixel position of “Y1” is DSY. Since one block is 12 pixels, (DSX, DSY) is DSX = (BX−1) × 12 + 1
DSY = (BY-1) × 12 + 1
Is required.

As a result of the above processing, the upper left pixel position of the decoding area is obtained, but as described above, the decoding area and the reading area are not the same position, so correction is necessary to obtain the position of the reading area. It is. In this correction, (RDX, RDY) shown in FIG. 17B is used. From this value, the upper left coordinates (RSX, RSY) of the reading area are
RSX = DSX + RDX
RSY = DSY + RDY
Is required.

In addition, the width and height of the reading area are predetermined design values. Therefore, if these are DAW and DAH, respectively, the reading area center (CRX, CRY) is
CRX = RSX + DAW / 2
CRY = RSY + DAH / 2
Is required. The coordinates so far are the coordinates on the code image.

Furthermore, it is necessary to obtain the center of the imaging region. The relative coordinates (IRX, IRY) of the center of the imaging region with respect to the center of the reading region are already given by the dot array generation unit 22. Further, the relative coordinates (IPX, IPY) of the pen tip center with respect to the center of the imaging region are determined from the specifications of the pen device. Thereby, the relative coordinates of the nib center with respect to the reading area center are (IRX + IPX, IRY + IPY). However, since these coordinates are coordinates on the captured image, it is necessary to convert them to coordinates on the code image. These two coordinates are rotated to the right by a certain angle about the reading area center. Therefore, the coordinates on the code image are converted by rotating coordinates.
((IRX + IPX) cosRDD + (IRY + IPY) sinRDD,
− (IRX + IPX) sinRDD + (IRY + IPY) cosRDD)
It becomes.

When these are combined, the coordinates (PPX, PPY) on the code image of the nib center are as follows:
PPX = (IRX + IPX) cosRDD + (IRY + IPY) sinRDD + DSX + RDX + DAW / 2
PPY = − (IRX + IPX) sinRDD + (IRY + IPY) cosRDD + DSY + RDY + DAH / 2
Is required.

  FIG. 19 shows the positional relationship between the coordinates used to determine the coordinates of the nib center. In the present embodiment, as described above, the reading area start point is obtained from the decoding area start point, and the reading area center is obtained from the reading area start point. Further, the center of the imaging area is obtained from the center of the reading area, and the center of the pen tip is obtained from the center of the imaging area. FIG. 19 also shows the upper direction of the code image and the upper direction of the captured image. As described above, since there is an angle difference between the code image and the captured image, this angle difference is taken into consideration when obtaining the center of the imaging region and the center of the pen tip from the center of the reading region.

The above procedure is organized as follows.
FIG. 20 is a diagram for explaining this procedure. Note that the numbers of the following procedures correspond to the numbers in the figure.
(1) The coordinates (DSX, DSY) of the starting point (upper left point) of the decoding area are obtained.
(2) The coordinates (DSX + RDX, DSY + RDY) of the start point (upper left point) of the reading area are obtained using (RDX, RDY) obtained in advance.
(3) The coordinates of the reading area center are obtained from the area size (DAW, DAH) necessary for decoding.
(4) On the captured image, the coordinates of the center of the imaging region and the center of the pen tip with respect to the reading region center are obtained.
(5) The coordinates obtained in (4) are converted into coordinates on the code image, and added to the coordinates of the center of the reading area obtained in (3), so that the coordinates (PPX) of the nib center on the code image are obtained. , PPY).

In the present embodiment, an imaging region is used as an example of the first partial image read from the medium, and a decoding region is used as an example of the second partial image that represents its own position on the medium. Yes. A reading area is used as an example of a third partial image that is a partial image extracted from the first partial image and is used for extracting the second partial image.
In addition, (IPX, IPY) is used as an example of information specifying the distance and direction from the center of the first partial image at the position indicated by the instruction means. Furthermore, (IRX, IRY) is used as an example of information for specifying the distance and direction of the center of the first partial image from the center of the third partial image. Furthermore, IRD is used as an example of information for specifying the rotation angle of the first partial image with respect to the third partial image. In addition, (RDX, RDY) is used as an example of information specifying the displacement in the vertical axis direction and the horizontal axis direction from one vertex of the second partial image of one vertex of the third partial image.

Next, the operation of the image processing apparatus 20 will be described in more detail. In the description of this operation, it is assumed that the unit code pattern of the 9C2 system is arranged in the layout shown in FIG.
First, the operation of the block detector 23 will be described.
FIG. 21 is a flowchart illustrating an operation example of the block detection unit 23.
First, the block detector 23 acquires a dot array from the dot array generator 22 (step 201). The size of this dot array is (number of blocks necessary for decoding × number of dots on one side of block + number of dots on one side of block−1) ² as described above. In this embodiment, the number of blocks necessary for decoding is 5 × 5, and the number of dots on one side of the block is 3, so a 17 × 17 dot array is acquired.

Next, a block break is superimposed on the acquired dot array (step 202). In this embodiment, a 5 × 5 block delimiter is used. Then, “0” is substituted into the counters I and J, and “0” is substituted into MaxBN (step 203).
Here, I and J count the number of steps in which the block break is moved from the initial position. The block break is moved for each line of the image, and the number of lines moved at that time is counted by counters I and J. MaxBN records the maximum count value when counting the number of blocks in which the number of dots detected in the block is “2” while moving the block delimiter.

  Next, the block detector 23 moves the block delimiter I in the X direction and J in the Y direction (step 204). Since I and J are “0” in the initial state, the block break does not move. Then, the number of dots included in each block of the block delimiter is counted, and the number of blocks whose dot number is “2” is counted. The counted number of blocks is stored in IB [I, J] (step 205). In I and J of IB [I, J], values of I and J indicating the movement amount of the block break are recorded, respectively.

  Next, the block detection unit 23 compares IB [I, J] with MaxBN (step 206). Since MaxBN has an initial value “0”, in the first comparison, IB [I, J] is larger than MaxBN. In this case, the value of IB [I, J] is substituted for MaxBN, the value of MX is substituted for I, and the value of MY is substituted for J (step 207). When IB [I, J] is MaxBN or less, the values of MaxBN, MX, and MY are left as they are.

Thereafter, the block detection unit 23 determines whether I = 2 (step 208).
If I = 2 is not satisfied, “1” is added to I (step 209). Then, the processing in steps 204 and 205 is repeated to compare IB [I, J] with MaxBN (step 206).
If IB [I, J] is larger than MaxBN, which is the maximum value of IB [I, J] up to the previous time, the value of IB [I, J] is assigned to MaxBN, and the value of MX is substituted for I at that time. , J is substituted for the value of MY (step 207). If MaxBN is larger than IB [I, J], it is determined whether I = 2 (step 208). When I = 2, it is next determined whether J = 2 or not (step 210). If J = 2 is not satisfied, “0” is substituted for I, and “1” is added to J (step 211). Such a procedure is repeated to detect a case where (I, J) is (0, 0) to (2, 2) and IB [I, J] is maximum.

When the processing up to I = 2 and J = 2 is completed, the block detection unit 23 compares the stored MaxBN with the threshold value TB (step 212). The threshold value TB is a threshold value for determining whether the number of blocks having a dot number of “2” is such that decoding is possible.
Here, when MaxBN is larger than the threshold value TB, the block break is fixed at the positions MX and MY, and the pattern value of each block is detected at that position. Then, the detected pattern value is recorded in the memory as a code array PA [X, Y] together with variables X and Y for identifying each block (step 213). At this time, if the pattern value cannot be converted, “99” which is not used as a pattern value is recorded. Then, the block detection unit 23 outputs MX and MY and the code array PA [X, Y] to the synchronous code detection unit 24 (step 214).
On the other hand, if MaxBN is equal to or less than the threshold value TB, it is determined that decoding is impossible due to large noise in the image, and decoding is impossible (step 215).

Next, the operation of the synchronization code detection unit 24 will be described.
FIG. 22 is a flowchart illustrating an operation example of the synchronization code detection unit 24.
First, the synchronization code detection unit 24 acquires MX and MY and the code array PA [X, Y] from the block detection unit 23 (step 251).
Next, the synchronous code detection unit 24 substitutes “1” for K and L (step 252). K is a counter indicating the number of blocks in the X direction, and L is a counter indicating the number of blocks in the Y direction.

Next, the synchronization code detection unit 24 determines whether the pattern value of PA [K, L] is 32 (step 253).
If the pattern value of PA [K, L] is 32, it is determined that the rotation of the code array PA [X, Y] is not necessary, and K is substituted for the X coordinate SX of the block having the synchronization code, and the Y coordinate SY Substitute L for. Further, MX is substituted for the movement amount ShiftX in the X direction of the block delimiter, and MY is substituted for the movement amount ShiftY in the Y direction (step 254).
In this embodiment, at this time, as shown in FIG. 17B, the correction amounts RDX and RDY from the decoding area start point to the reading area start point and the coordinates on the captured image are displayed on the code image. The amount of rotation RDD when converting to the coordinates is stored. Specifically, each value is obtained and stored by RDX = −MX × 4, RDY = −MY × 4, and RDD = IRD. Here, since the interval of 1 dot is 4 pixels, 4 is multiplied to convert from the coordinates counted in dots to the coordinates counted in pixels.

Next, the synchronization code detection unit 24 determines whether the pattern value of PA [K, L] is 33 (step 255).
If the pattern value of PA [K, L] is 33, the code array PA [X, Y] is rotated 90 degrees to the left (step 256). As shown in FIG. 4, the unit code pattern of the pattern value 33 is an image obtained by rotating the unit code pattern of the pattern value 32 by 90 degrees in the right direction, so that the image is erected by rotating 90 degrees in the reverse direction. Yes. At this time, all pattern values in the code array PA [X, Y] are converted into pattern values when rotated 90 degrees to the left.
In accordance with this rotation, L is substituted for the X coordinate SX of the block having the synchronization code, and 6-K is substituted for the Y coordinate SY. Also, MY is substituted for the movement amount ShiftX in the X direction of the block delimiter, and 2-MX is substituted for the movement amount ShiftY in the Y direction (step 257).
In this embodiment, at this time, as shown in FIG. 17B, the correction amounts RDX and RDY from the decoding area start point to the reading area start point and the coordinates on the captured image are displayed on the code image. The amount of rotation RDD when converting to the coordinates is stored. Specifically, each value is obtained and stored by RDX = −MY × 4, RDY = − (2−MX) × 4, RDD = 270 + IRD. The RDD is converted into a rotation angle from the vertical direction on the code image.

Next, the synchronization code detection unit 24 determines whether or not the pattern value of PA [K, L] is 34 (step 258).
If the pattern value of PA [K, L] is 34, the code array PA [X, Y] is rotated 180 degrees to the left (step 259). As shown in FIG. 4, since the unit code pattern of the pattern value 34 is an image obtained by rotating the unit code pattern of the pattern value 32 by 180 degrees, the unit code pattern of the pattern value 34 is rotated 180 degrees and the image is erected. I am letting. At this time, all the pattern values in the code array PA [X, Y] are converted into pattern values when rotated 180 degrees.
In accordance with this rotation, 6-K is substituted for the X coordinate SX of the block having the synchronization code, and 6-L is substituted for the Y coordinate SY. Further, 2-MX is substituted for the movement amount ShiftX in the X direction of the block delimiter, and 2-MY is substituted for the movement amount ShiftY in the Y direction (step 260).
In this embodiment, at this time, as shown in FIG. 17B, the correction amounts RDX and RDY from the decoding area start point to the reading area start point and the coordinates on the captured image are displayed on the code image. The amount of rotation RDD when converting to the coordinates is stored. Specifically, each value is obtained and stored by RDX = − (2-MX) × 4, RDY = − (2-MY) × 4, RDD = 180 + IRD.

Next, the synchronization code detection unit 24 determines whether or not the pattern value of PA [K, L] is 35 (step 261).
If the pattern value of PA [K, L] is 35, the code array PA [X, Y] is rotated 270 degrees to the left (step 262). As shown in FIG. 4, since the unit code pattern of the pattern value 35 is an image obtained by rotating the unit code pattern of the pattern value 32 to the right by 270 degrees, it is rotated 270 degrees in the reverse direction to erect the image. . At this time, all the pattern values in the code array PA [X, Y] are converted into pattern values when rotated 270 degrees in the left direction.
In accordance with this rotation, 6-L is substituted for the X coordinate SX of the block having the synchronization code, and K is substituted for the Y coordinate SY. Further, 2-MY is substituted for the movement amount ShiftX in the X direction of the block delimiter, and MX is substituted for the movement amount ShiftY in the Y direction (step 263).
In this embodiment, at this time, as shown in FIG. 17B, the correction amounts RDX and RDY from the decoding area start point to the reading area start point and the coordinates on the captured image are displayed on the code image. The amount of rotation RDD when converting to the coordinates is stored. Specifically, each value is obtained and stored by RDX = − (2-MY) × 4, RDY = −MX × 4, RDD = 90 + IRD.

When the values are assigned to SX, SY, ShiftX, and ShiftY in steps 254, 257, 260, and 263, the synchronization code detection unit 24 assigns these values to the identification code detection unit 30 and the X coordinate code detection unit 40. , And output to the Y coordinate code detector 45 (step 264).
If PA [K, L] is not any of the pattern values 32-35, the synchronization code detector 24 determines whether K = 5 (step 265). If K = 5 is not satisfied, “1” is added to K (step 266), and the process returns to step 253. If K = 5, it is determined whether L = 5 (step 267). If L = 5 is not satisfied, “1” is substituted for K, “1” is added to L (step 268), and the process returns to step 253. That is, the processing of steps 253 to 264 is repeated while changing the values of K and L until a block having pattern values of 32 to 35 is detected. If a block having pattern values of 32 to 35 cannot be detected even when K = 5 and L = 5, an undecodable determination signal is output (step 269).

  Note that the processing of steps 253, 255, 258, and 261 may be performed by the rotation determination unit 25 under the control of the synchronization code detection unit 24. Further, the processing of steps 254, 256, 257, 259, 260, 262, and 263 may be performed by the code array rotation unit 26 under the control of the synchronization code detection unit 24.

Next, the operation of the X coordinate code detection unit 40 will be described.
FIG. 23 is a flowchart illustrating an operation example of the X coordinate code detection unit 40.
First, the X coordinate code detection unit 40 acquires the code arrays PA [X, Y], SX, and SY from the synchronization code detection unit 24 (step 401).
Next, the X coordinate code detection unit 40 initializes all elements of the X coordinate code array XA [X, 0] with “99” (step 402). The “99” is a number that is not used as a pattern value. Then, “1” is substituted into counters IX and IY for identifying each block in the code block. Here, IX is a counter indicating the number of blocks in the X direction, and IY is a counter indicating the number of blocks in the Y direction. Furthermore, the X coordinate code detection unit 40 substitutes “1” into a counter KX for identifying each element in the X coordinate code array (step 403).

Further, the X coordinate code detection unit 40 determines whether IY-SY is divisible by “5” (step 404). That is, it is determined whether or not the synchronization code is arranged in the block specified by IY.
Here, if IY-SY is not divisible by “5”, that is, if no synchronization code is arranged in this block, the X coordinate code is not extracted, so “1” is added to IY ( Step 405), the process proceeds to Step 404.
On the other hand, when IY-SY is divisible by “5”, that is, when a synchronization code is arranged in this block, the X coordinate code detection unit 40 determines whether IX-SX is divisible by “5”. (Step 406). That is, it is determined whether or not the synchronization code is arranged in the block specified by IX.

Here, when IX-SX is divisible by “5”, that is, when a synchronous code is arranged in this block, since the X coordinate code is not extracted, “1” is added to IX (Step 1 407), go to step 406.
On the other hand, when IX-SX is not divisible by “5”, that is, when a synchronization code is not arranged in this block, the X coordinate code detection unit 40 adds PA [IX, IY] to XA [KX, 0]. Is substituted (step 408).

Then, it is determined whether or not IX = 5 (step 409).
If IX = 5 is not satisfied, “1” is added to KX (step 410), “1” is added to IX (step 407), and the processing of steps 406 to 408 becomes IX = 5. Repeat until When IX = 5, it is next determined whether IY = 5 (step 411). If IY = 5 is not satisfied, “1” is substituted into IX and KX (step 412), “1” is added to IY (step 405), and the processing of steps 404 to 410 is performed until IY = 5. Repeat. When IY = 5, the X coordinate code detector 40 outputs the X coordinate code array XA [X, 0] to the X coordinate code decoder 42 (step 413).
Although only the operation of the X coordinate code detection unit 40 has been described here, the Y coordinate code detection unit 45 performs the same operation.
Thus, the detailed operation description of the image processing apparatus 20 in the present embodiment is finished.

Next, the operation of the coordinate correction unit 50 will be described.
FIG. 24 is a flowchart illustrating an operation example of the coordinate correction unit 50.
First, the coordinate correction unit 50 acquires the relative coordinates (IPX, IPY) of the pen tip center with respect to the imaging area center and the relative coordinates (IRX, IRY) of the imaging area center with respect to the reading area center (step 501). Here, since (IPX, IPY) is stored in advance in a memory (not shown) as the specification of the pen device, it is read out. Further, (IRX, IRY) is stored in the memory when the dot array generation unit 22 detects the dot image, and is read out.

Next, the coordinate correction unit 50 acquires the number of pixels (RDX, RDY) of the movement amount at the block break and the rotation angle RDD with respect to the reading area of the imaging area (step 502). Here, since these values are stored in the memory when the rotation is detected by the synchronous code detection unit 24, they are read out.
Further, the coordinate correction unit 50 obtains the position BX of the leftmost block of the code block from the X coordinate code decoding unit 42 and obtains the position BY of the uppermost block of the code block from the Y coordinate code decoding unit 47 (step 503).

  Thereafter, the coordinate correction unit 50 obtains the pixel position (DSX, DSY) of the decoding area start point (step 504), and further converts it to the pixel position (RSX, RSY) of the reading area start point (step 505). . Finally, the pixel position (CRX, CRY) at the center of the reading area is obtained (step 506), and the pixel position (PPX, PPY) at the center of the pen tip is obtained (step 507). Note that the pixel position conversion formulas in these steps have been described in detail in the above description, so description thereof will be omitted here.

This is the end of the description of the operation of the present embodiment.
By the way, in this embodiment, first, the position of the decoding area is obtained, then the position of the reading area is obtained, and further, the position of the imaging area is obtained to obtain the position of the pen tip. However, not all of these positions need to be determined. For example, when an encoding method other than the mCn method is employed, it may be considered that the decoding area and the reading area may be identified. In such a case, the imaging area may be obtained directly from the decoding area.

Next, a specific hardware configuration of the image processing apparatus 20 in the present embodiment will be described.
First, the pen device 60 that implements the image processing apparatus 20 will be described.
FIG. 25 is a view showing the mechanism of the pen device 60.
As illustrated, the pen device 60 includes a control circuit 61 that controls the operation of the entire pen. In addition, the control circuit 61 includes an image processing unit 61a that processes a code image detected from an input image, and a data processing unit 61b that extracts identification information and position information from the processing result.
The control circuit 61 is connected to a pressure sensor 62 that detects the writing operation by the pen device 60 by the pressure applied to the pen tip 69. Further, an infrared LED 63 that irradiates infrared light onto the medium and an infrared CMOS 64 that inputs an image are also connected. Further, an information memory 65 for storing identification information and position information, a communication circuit 66 for communicating with an external device, a battery 67 for driving the pen, and pen identification information (pen ID) are stored. A pen ID memory 68 is also connected.

  The image reading unit 21 shown in FIG. 11 is realized by, for example, the infrared CMOS 64 in FIG. The dot array generation unit 22 is realized by, for example, the image processing unit 61a in FIG. Further, the block detection unit 23, the synchronization code detection unit 24, the rotation determination unit 25, the code array rotation unit 26, the identification code detection unit 30, the identification code restoration unit 31, the identification code decoding unit 32, and the identification code error illustrated in FIG. Detection unit 33, identification code error correction unit 34, X coordinate code detection unit 40, X coordinate code decoding unit 42, X coordinate code error detection unit 43, X coordinate code error correction unit 44, Y coordinate code detection unit 45, Y coordinate The code decoding unit 47, the Y coordinate code error detection unit 48, the Y coordinate code error correction unit 49, the coordinate correction unit 50, and the information output unit 55 are realized by, for example, the data processing unit 61b in FIG.

  In the present embodiment, the pen tip 69 of the pen device 60 is provided as an example of the instruction unit that indicates the position on the medium. However, the present invention is not limited to this. Instead of writing on the medium, a structure for simply pointing to a predetermined position on the medium may be regarded as the instruction means.

The processing realized by the image processing unit 61a or the data processing unit 61b in FIG. 25 may be realized by a general-purpose computer, for example. Accordingly, assuming that such processing is realized by the computer 90, the hardware configuration of the computer 90 will be described.
FIG. 26 is a diagram illustrating a hardware configuration of the computer 90.
As shown in the figure, the computer 90 includes a CPU (Central Processing Unit) 91 as a calculation means, a main memory 92 as a storage means, and a magnetic disk device (HDD: Hard Disk Drive) 93. Here, the CPU 91 executes various types of software such as an OS (Operating System) and applications to realize the above-described functions. The main memory 92 is a storage area for storing various software and data used for execution thereof, and the magnetic disk device 93 is a storage area for storing input data for various software, output data from various software, and the like. .
Further, the computer 90 includes a communication I / F 94 for performing communication with the outside, a display mechanism 95 including a video memory and a display, and an input device 96 such as a keyboard and a mouse.

  The program for realizing the present embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

It is the figure which showed an example of the unit code pattern in 9Cn system. It is the figure which showed the other example of the unit code pattern in 9Cn system. It is the figure which showed the correspondence of the unit code pattern and pattern value in 9C2 system. It is the figure which showed the example of the synchronous pattern in 9C2 system. It is the figure which showed the example of the basic layout of a code block. It is the figure which showed the example of the layout on the paper surface of a code block. It is a figure for demonstrating the expression of the coordinate by M series. It is the block diagram which showed the function structure of the image generation apparatus in this Embodiment. It is a figure for demonstrating the shift | offset | difference of a nib center and an imaging area center. It is a figure for demonstrating the shift | offset | difference of a nib center, an imaging area, and a reading area. It is the block diagram which showed the function structure of the image processing apparatus in this Embodiment. It is a figure for demonstrating the process at the time of producing | generating a dot array. It is a figure for demonstrating the process at the time of detecting a block on a dot arrangement | sequence. It is a figure for demonstrating the state after detecting a block on a dot arrangement | sequence. It is a figure for demonstrating the process at the time of converting from a dot arrangement to a code arrangement. It is the figure which showed the response | compatibility of a rotation angle and a pattern number. It is a figure for demonstrating the memory | storage of the information accompanying rotation of a code arrangement | sequence and rotation. It is a figure for demonstrating the process at the time of calculating | requiring a decoding area | region start point. It is the figure which showed the relationship of each coordinate until it calculates | requires the nib center from a decoding area | region start point. It is the figure put together about conversion of coordinates. It is the flowchart which showed the operation example of the block detection part in this Embodiment. It is the flowchart which showed the operation example of the synchronous code detection part in this Embodiment. It is the flowchart which showed the operation example of the X coordinate code detection part in this Embodiment. It is the flowchart which showed the operation example of the coordinate correction | amendment part in this Embodiment. It is the figure which showed the mechanism of the pen device which can implement | achieve the image processing apparatus in this Embodiment. It is a hardware block diagram of the computer which can apply this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image generation apparatus, 20 ... Image processing apparatus, 21 ... Image reading part, 22 ... Dot array generation part, 23 ... Block detection part, 24 ... Synchronous code detection part, 25 ... Rotation determination part, 26 ... Code arrangement rotation part , 30 ... identification code detection unit, 31 ... identification code restoration unit, 32 ... identification code decoding unit, 33 ... identification code error detection unit, 34 ... identification code error correction unit, 40 ... X coordinate code detection unit, 42 ... X coordinate Code decoding unit, 43 ... X coordinate code error detection unit, 44 ... X coordinate code error correction unit, 45 ... Y coordinate code detection unit, 47 ... Y coordinate code decoding unit, 48 ... Y coordinate code error detection unit, 49 ... Y Coordinate code error correction unit 50... Coordinate correction unit 55 55 Information output unit

Claims (9)

Indicating means for indicating a position on the medium;
Reading means for reading a first partial image included in an image printed on the medium;
Extraction means for extracting a second partial image representing its own position on the medium from the first partial image read by the reading means;
Detecting means for detecting the position of the second partial image on the medium using the second partial image extracted by the extracting means;
First information indicating a relative position of the position instructed by the instruction means with respect to the first partial image, and second information indicating a relative position of the first partial image with respect to the second partial image. Acquisition means for acquiring;
A position detection apparatus comprising: a correction unit that corrects the position of the second partial image detected by the detection unit using the first information and the second information.   2. The information acquisition unit according to claim 1, wherein the acquisition unit acquires, as the first information, information specifying a distance and direction from a center of the first partial image at a position specified by the instruction unit. Position detector. The extraction means extracts the second partial image from the third partial image after extracting the third partial image from the first partial image,
The acquisition unit includes the second information, the third information indicating a relative position of the first partial image with respect to the third partial image, and the second partial image of the third partial image. The position detection device according to claim 1, wherein the position detection device is acquired separately from fourth information indicating a relative position.   The said acquisition means acquires the information which specifies the distance and direction from the center of the said 3rd partial image of the center of the said 1st partial image as said 3rd information. Position detector.   The position detection apparatus according to claim 4, wherein the acquisition unit further acquires, as the third information, information specifying a rotation angle of the first partial image with respect to the third partial image. The second partial image and the third partial image have a rectangular shape sharing a vertical axis and a horizontal axis,
The acquisition means uses the fourth information to specify information specifying a displacement in the vertical axis direction and the horizontal axis direction from one vertex of the second partial image of one vertex of the third partial image. The position detection device according to claim 3, acquired as: The second partial image includes Ax (Ax ≧ 2) pattern images in which the unit images are arranged at n (2 ≦ n <m) of m (m ≧ 4), An image formed by arranging Ay (Ay ≧ 2) in the horizontal axis direction,
The extraction means moves a frame formed by arranging Ax pieces of sections having substantially the same size as the pattern image in the vertical axis direction and Ay pieces in the horizontal axis direction on the third partial image, and n Obtaining a moving position where the number of sections including the unit images is a predetermined number or more, and extracting an image in the frame at the moving position as the second partial image;
The position detection apparatus according to claim 6, wherein the acquisition unit acquires the amount of movement of the frame to the movement position in the vertical axis direction and the horizontal axis direction as information for specifying the displacement. On the computer,
A function of extracting a second partial image representing its own position on the medium from the first partial image of the image read from the image printed on the medium;
A function of detecting the position of the second partial image on the medium using the extracted second partial image;
First information indicating a relative position of the position on the medium instructed by the instruction unit with respect to the first partial image, and a second information indicating a relative position of the first partial image with respect to the second partial image The ability to get information and
A program for realizing a function of correcting the position of the detected second partial image using the first information and the second information. In the extracting function, after extracting the third partial image from the first partial image, the second partial image is extracted from the third partial image,
In the function to obtain, the second information includes third information indicating a relative position of the first partial image with respect to the third partial image, and the second partial image of the third partial image. 9. The program according to claim 8, wherein the program is acquired separately from fourth information indicating a relative position with respect to.

Images