JP2009181347A - Image-processing device, pen device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to synchronize unit images constituting a pattern image even if there is distortion in a read-out image. <P>SOLUTION: An image-processing device is provided with: a detection part which scans an image taken by an image pickup device and detects unit images to identify positions; a line synchronization part 233 which draws grid lines, following to the axis of coordinates sequentially from a synchronous origin point on the basis of the unit image position information detected by the detection part, to generate grids for synchronization; and a bit sequence-generating part 234 which generates a bit sequence, according to the presence or absence of the unit images at prescribed positions on the image, on the basis of the grids generated by the line synchronization part 233. The line synchronization part 233 makes each grid line adaptable to the unit image position correlated with the grid line. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, a pen device, and a program.

  Embed a pattern image representing information coded on a medium such as printing paper or a pattern image with a specific meaning, read it with a reading device, decode the code information, and control the device according to the given meaning It has been performed conventionally. Information represented by the pattern image includes identification information of the medium itself in which the pattern image is embedded, identification information of a document recorded on the medium, position information indicating a position on the medium, and the like.

  In order to decode the code information described by the pattern image, it is necessary to synchronize the pattern image detected from the read image with the bit string. In the prior art disclosed in Patent Document 1, synchronization is performed by determining the coordinates of a mark and the coordinates of a raster point while sequentially tracing the mark according to a virtual raster.

Japanese translation of PCT publication No. 2003-511863

In a reading device or the like that a user operates with a hand such as a pen device, the read image may be distorted depending on the direction and angle of the reading device with respect to the medium.
The present invention has been made in view of the above problems, and an object thereof is to enable synchronization of unit images constituting a pattern image even when the read image has distortion.

The invention according to claim 1 is based on a detection unit that scans an image captured by an imaging apparatus and detects a unit image to identify a position, and based on position information of the unit image detected by the detection unit. A line synchronization unit that generates a lattice for synchronization by sequentially drawing lattice lines according to the coordinate axes from the origin, and the unit image at a predetermined position on the image based on the lattice generated by the line synchronization unit. An image processing apparatus comprising: a bit array generation unit that generates a bit array according to presence or absence.
The invention according to claim 2 is characterized in that the line synchronization unit individually adapts each grid line to the position of the unit image associated with the grid line. Device.
The invention according to claim 3 is characterized in that the line synchronization unit corrects the inclination of each grid line in accordance with the position of the unit image associated with the grid line. It is a processing device.
The invention according to claim 4 is characterized in that the line synchronization unit corrects the interval between the grid lines in accordance with the position of the unit image associated with the grid line. An image processing apparatus.
The invention according to claim 5 is the position of the two unit images in the combination of the two unit images in which the initial values of the grid line inclination and interval used by the line synchronization unit are closest to each other among the unit images. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined based on a relationship.
In the invention according to claim 6, the angle formed by the two unit images in the combination of the unit images is corrected based on the positional relationship of the unit images in a certain range including the synchronization origin. A reference angle determination unit that determines a reference angle that serves as a reference for the inclination is further provided, wherein the line synchronization unit uses the reference angle determined by the reference angle determination unit as an initial value of the inclination of the grid line. The image processing apparatus according to claim 5.
According to a seventh aspect of the present invention, there is provided an imaging unit that images the surface of a medium on which a unit image is formed, a detection unit that detects the unit image from the image captured by the imaging unit, and identifies the position, and the detection unit A line synchronization unit that generates a synchronization grid by sequentially drawing grid lines according to the coordinate axis from the synchronization origin based on the position information of the unit image detected by the step, and the grid generated by the line synchronization unit. And a bit array generation unit that generates a bit array in accordance with the presence / absence of the unit image at a predetermined position on the image.
The invention according to claim 8 is characterized in that the line synchronization unit individually adapts each grid line to the position of the unit image associated with the grid line. It is a device.
The invention according to claim 9 is characterized in that the line synchronization unit corrects the inclination of each grid line in accordance with the position of the unit image associated with the grid line. It is a pen device.
The invention according to claim 10 is characterized in that the line synchronization section corrects the interval between the respective grid lines in accordance with the position of the unit image associated with the grid line. It is a pen device.
According to an eleventh aspect of the present invention, a computer scans an image picked up by an image pickup device, detects a unit image and specifies a position, and based on the detected position information of the unit image, the synchronization origin is used. A function for generating a synchronization grid by sequentially drawing grid lines according to coordinate axes, and generating a bit array based on the presence of the unit image at a predetermined position on the image based on the generated grid This is a program for realizing the functions.
The invention according to claim 12 is a function of generating the grid, and causes the computer to execute a process of drawing each grid line individually adapted to the position of the unit image associated with the grid line. The program according to claim 11.
The invention according to claim 13 is a function for generating the grid, and causing the computer to execute a process of correcting the inclination of each grid line according to the position of the unit image associated with the grid line. The program according to claim 12, wherein:
The invention according to claim 14 is a function for generating the grid, and causing the computer to execute a process of correcting an interval between the grid lines in accordance with a position of the unit image associated with the grid line. The program according to claim 12, wherein:

According to the first aspect of the present invention, by generating a grid while sequentially drawing grid lines, it is possible to flexibly adapt to image distortion compared to the case of using a fixed grid.
According to the second aspect of the present invention, it is possible to appropriately cope with image distortion by adapting each grid line to the actual position of the unit image.
According to the third aspect of the present invention, it is possible to individually adjust the inclination of the lattice lines and adapt to the distortion of the image.
According to the fourth aspect of the present invention, it is possible to individually adjust the interval between the lattice lines and adapt to the distortion of the image.
According to the fifth aspect of the present invention, since the initial values of the inclination and interval of the grid lines are determined based on the actual position of the unit image, it is easy to cope with image distortion.
According to the invention which concerns on Claim 6, the inclination of a grid line can be determined still more accurately using the position of a some unit image.
According to the invention of claim 7, by generating a lattice while sequentially drawing lattice lines in the pen device, it is possible to flexibly adapt to image distortion as compared with the case of using a fixed lattice. .
According to the invention according to claim 8, in the pen device, it is possible to appropriately cope with the distortion of the image by adapting each grid line to the actual position of the unit image.
According to the invention of claim 9, in the pen device, the inclination of the grid lines can be individually corrected to adapt to the distortion of the image.
According to the tenth aspect of the present invention, in the pen device, the interval between the grid lines can be individually corrected to adapt to the distortion of the image.
According to the eleventh aspect of the invention, in the computer that realizes the function of the image processing apparatus, the lattice is generated while sequentially drawing the lattice lines, so that the distortion of the image is more flexible than when a fixed lattice is used. Can be adapted to.
According to the twelfth aspect of the present invention, in the computer that realizes the function of the image processing apparatus, it is possible to appropriately cope with image distortion by adapting each grid line to the actual position of the unit image.
According to the thirteenth aspect of the present invention, in the computer that realizes the function of the image processing apparatus, the inclination of the grid lines can be individually corrected to adapt to the image distortion.
According to the fourteenth aspect of the present invention, in the computer that realizes the function of the image processing apparatus, the lattice line interval can be individually corrected to adapt to the image distortion.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The present embodiment relates to a process of reading a pattern image printed on a medium such as paper and decoding information (code information or the like) recorded by the pattern image. As a premise of the present embodiment, first, a configuration example of a pattern image will be described.

<Configuration example of pattern image>
In the present embodiment, an information block representing specific information is arranged in the entire paper surface as a medium or in a specific area on the paper surface. An information block is a minimum unit of a pattern expressing meaningful information. This information block is composed of a set of a plurality of pattern blocks, and a unit code pattern is arranged in this pattern block. The unit code pattern represents a code for recording information and is the minimum unit of the pattern image.

FIG. 1 is a diagram showing the relationship between unit code patterns, information blocks, and paper (medium).
In FIG. 1, an information block P2 is configured by a set of unit code patterns P1. Information is recorded in a region P3 surrounded by a thick frame on the paper surface by a set of information blocks P2. Next, a specific configuration example of the unit code pattern and the information block will be described in detail.

In the present embodiment, mCn (= m! / {(Mn)! ×) by a pattern image in which unit images are arranged at n (1 ≦ n <m) locations selected from m (m ≧ 3) locations. n!}) Represents street information. This pattern image is a unit code pattern. That is, instead of associating one unit image with information, a plurality of unit images are associated with information. Assuming that one unit image is associated with information, there is a drawback in that erroneous information is expressed when the unit image is lost or noise is added. On the other hand, for example, if two unit images are associated with information, it can be easily understood that there is an error when the number of unit images is one or three. Further, in the method of expressing 1 bit or 2 bits at most in one unit image, it is not possible to express a synchronous pattern that controls reading of the information pattern with a pattern visually similar to the pattern that represents information. . For this reason, in the present embodiment, the above encoding method is adopted. Hereinafter, such an encoding method is referred to as an mCn method.
Here, the unit image may have any shape, but in the present embodiment, a dot image (hereinafter simply referred to as “dot”) is used as an example of the unit image.

FIG. 2 is a diagram illustrating an example of a unit code pattern in the mCn system.
In the example shown in FIG. 2, the black area and the hatched area are areas where dots can be arranged. The white area between the dot-placeable areas is a non-dot-placeable area that forms a gap (blank) for identifying individual dots. Of the areas where dots can be arranged, dots are arranged in black areas and dots are not arranged in hatched areas. That is, FIG. 2 shows an example in which a region where a total of 9 dots of 3 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is provided. FIG. FIG. 6B shows a unit code pattern, and FIG. 5B shows a unit code pattern in the 9C3 system in which 3 dots are arranged in a dot arrangementable area.

  By the way, the dots (black areas) arranged in FIG. 2 are only for information representation, and do not necessarily match the dots (minimum squares in FIG. 2) which are the minimum units constituting the image. In the present embodiment, when “dot” refers to the former dot and the latter dot is referred to as “pixel”, the dot has a size of 2 pixels × 2 pixels at 600 dpi. Since the size of one pixel at 600 dpi is 0.0423 mm, one side of the dot is 84.6 μm (= 0.0423 mm × 2). The larger the dot, the more likely it will be noticeable, so it is preferable to make it as small as possible. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. However, the above value 84.6 μm is a numerical value to the last, and is about 100 μm in the actually printed toner image.

FIG. 3 is a diagram comprehensively showing the types of unit code patterns shown in FIG. In this figure, a space between dots is omitted for simplification of display.
FIG. 3A shows all unit code patterns in the 9C2 system shown in FIG. In the 9C2 system, 36 (= ₉ C ₂ ) types of information are expressed using these unit code patterns. FIG. 3B shows all unit code patterns in the 9C3 system shown in FIG. In the 9C3 system, 84 (= ₉ C ₃ ) types of information are expressed using these unit code patterns.

Note that the unit code pattern is not limited to m = 9, and a value such as 4, 16 may be employed as the value of m. Any value of n may be adopted as long as 1 ≦ n <m is satisfied.
Further, as the value of m, not only a square number (the square of an integer) but also a product of two different integers may be adopted. That is, the area in which dots can be arranged is not limited to a square area as in the case of arranging 3 dots × 3 dots, but may be a rectangular area as in the case of arranging 3 dots × 4 dots. In the present specification, the rectangle means a rectangle in which the lengths of two adjacent sides are not equal.

By the way, all unit code patterns in the mCn system are assigned pattern values that are numbers for identifying each unit code pattern.
FIG. 4 shows the correspondence between unit code patterns and pattern values in the 9C2 system. Since there are 36 unit code patterns in the 9C2 system, 0 to 35 are used as pattern values. However, the correspondence shown in FIG. 4 is merely an example, and any pattern value may be assigned to any unit code pattern. In addition, although not shown in the figure for the 9C3 method, each unit code pattern is assigned a pattern value of 0 to 83.

  In this way, mCn types of unit code patterns are prepared by selecting n locations from m locations, but in this embodiment, a specific pattern is used as an information pattern among these unit code patterns, Use the rest as a synchronization pattern. Here, the information pattern is a pattern representing information to be embedded in the medium. The synchronization pattern is a pattern used for extracting an information pattern embedded in the medium. For example, it is used for specifying the position of the information pattern or detecting the rotation of the image. Any medium may be used as long as an image can be printed. Since paper is most commonly used, the medium will be described as paper in the following, but it may be metal, plastic, fiber, or the like.

Here, the synchronization pattern will be described.
FIG. 5 is an example of a synchronization pattern in the 9C2 system.
As shown in FIG. 5, when the unit code pattern is a square (the same number of vertical and horizontal dots), it is necessary to prepare four types of unit code patterns as synchronization patterns in order to detect image rotation. Here, the unit code pattern of the pattern value 32 is an upright synchronization pattern. In addition, the unit code pattern of the pattern value 33 is rotated 90 degrees to the right, the synchronization pattern of the pattern value 34 is rotated 180 degrees to the right, and the unit code pattern of the pattern value 35 is rotated 270 degrees to the right. The synchronization pattern is used. In this case, 5-bit information may be expressed by using the remainder obtained by removing these four types of synchronization patterns from 36 types of unit code patterns as an information pattern. However, the method of assigning 36 types of unit code patterns to information patterns and synchronization patterns is not limited to this. For example, five sets of synchronization patterns that constitute one set of four types may be prepared, and the remaining 16 types of information patterns may represent 4-bit information.

  On the other hand, although not particularly illustrated, when the unit code pattern is a rectangle (different in the number of vertical and horizontal dots), two types of unit code patterns may be prepared as synchronization patterns in order to detect image rotation. For example, when an area in which 3 dots in the vertical direction and 4 dots in the horizontal direction should be detected but an area in which 4 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is detected, the image is 90 degrees or 270 degrees at that time. It is because it turns out that it is rotating.

Next, a method for expressing the identification information and the coordinate information by arranging the above code patterns will be described.
First, an information block composed of unit code patterns will be described.
FIG. 6 is a diagram illustrating a configuration example of an information block.
In the example shown in FIG. 6, it is shown at the same time that the unit code pattern in the 9C2 system is used. That is, 36 types of unit code patterns are divided into, for example, 4 patterns used as synchronization patterns and 32 patterns used as information patterns, and each pattern is arranged according to a layout.

Further, in the example shown in FIG. 6, a layout in which 25 pattern blocks in which 5 × 5 unit code patterns of 3 × 3 dots can be arranged is arranged is used as the information block layout. Of the 25 pattern blocks, a synchronization pattern is arranged in one block at the upper left. In addition, an information pattern representing X coordinate information that specifies coordinates in the X direction on the paper surface is arranged in the four blocks to the right of the synchronization pattern, and the Y direction coordinates on the paper surface are specified in the four blocks below the synchronization pattern. An information pattern representing coordinate information is arranged. Furthermore, an information pattern representing the identification information of the document printed on the paper surface or the paper surface is arranged in 16 blocks surrounded by the information pattern representing the coordinate information.
The unit code pattern in the 9C2 system described above is merely an example, and the unit code pattern in the 9C3 system may be used as shown in the lower left of the figure.

Next, a layout for expressing identification information and coordinate information in the present embodiment will be described.
FIG. 7 shows a part of such a layout.
In the layout shown in FIG. 7, the information block shown in FIG. 6 is used as a basic unit, and this information block is periodically arranged on the entire sheet. In the figure, the synchronization pattern is represented by “S”, the pattern representing the X coordinate is represented by “X1”, “X2”,..., And the pattern representing the Y coordinate is represented by “Y1”, “Y2”,. The coordinate information is expressed in M series over, for example, the vertical and horizontal directions of the page. Further, although some methods can be used for encoding the identification information, RS encoding is suitable in this embodiment. This is because the RS code is a multi-level coding method, and in this case, the block representation should correspond to the multi-value of the RS code. In FIG. 7, the pattern representing the identification information is represented by “IXY” (X = 1 to 4, Y = 1 to 4).

Here, the M series used for expressing the coordinate information will be described.
The M sequence is a sequence whose partial sequence does not match other partial sequences. For example, the eleventh order M-sequence is a bit string of 2047 bits. The same sequence as the partial sequence of 11 bits or more extracted from the 2047-bit bit string does not exist in the 2047-bit bit string other than itself. In the present embodiment, 16 unit code patterns are associated with 4 bits. That is, a 2047-bit bit string is represented in decimal notation for every 4 bits, a unit code pattern is determined according to the numerical value correspondence as shown in FIG. 4, and is expressed as a coordinate code across the paper in the horizontal and vertical directions. Therefore, at the time of decoding, the position on the bit string is specified by specifying three consecutive unit code patterns.

FIG. 8 shows an example of encoding coordinate information using an M sequence.
(A) shows a bit string “0111000101011010000110010...” As an example of the 11th order M-sequence. In the present embodiment, this is divided into 4 bits, and the first partial sequence “0111” is a unit code pattern of pattern value 7 and the second partial sequence “0001” is a unit code pattern of pattern value 1. As a result, the third partial sequence “0101” is arranged as a unit code pattern with a pattern value of 5, and the fourth partial sequence “1010” is arranged as a unit code pattern of a pattern value of 10, respectively.

  As described above, when 4 bits are divided and assigned to the unit code pattern, all pattern strings can be expressed in 4 cycles as shown in FIG. That is, since the 11th order M-sequence is 2047 bits, if this sequence is cut out every 4 bits and expressed by a unit code pattern, 3 bits will be added at the end. The first 1 bit of the M sequence is added to these 3 bits to form 4 bits, which are represented by a unit code pattern. Further, the unit code pattern is cut out every 4 bits from the second bit of the M sequence. Then, the next cycle starts from the third bit of the M sequence, and the next cycle starts from the fourth bit of the M sequence. Furthermore, the 5th cycle starts from the 5th bit, which coincides with the first cycle. Therefore, if four cycles of the M sequence are cut out every 4 bits, it is possible to use all 2047 unit code patterns. Since the M sequence is 11th order, the three consecutive unit code patterns do not match the continuous code pattern at any other position. Therefore, decoding can be performed by reading three unit code patterns at the time of reading. However, in the present embodiment, coordinate information is expressed by four unit code patterns in consideration of occurrence of errors.

  By the way, when such encoding is performed, the M sequence of 4 periods is divided into 2047 blocks and stored. In the layout as shown in FIG. 7, since the blocks where the synchronization pattern is arranged are sandwiched between them, the length of 2558 blocks (≈2047 × 5/4) is covered with the M sequence of 4 cycles. become. Since the length of one side of one block is 0.508 mm (12 pixels at 600 dpi), the length of 2558 blocks is 1299.5 mm. That is, a length of 1299.5 mm is encoded. Since the long side of the A0 size is 1188.0 mm, in this embodiment, the coordinates on the A0 size paper are encoded.

The pattern image configured as described above is formed on a sheet of paper, which is a medium, using K toner (infrared light absorbing toner containing carbon) or special toner, for example, using an electrophotographic system.
Here, as the special toner, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm) is exemplified. . Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished based on whether or not an image formed on the printed paper can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.

<Configuration and Operation of Image Processing Device>
Next, an image processing apparatus that reads a pattern image printed on a paper surface (formed on a medium) will be described.
FIG. 9 is a diagram illustrating a functional configuration example of the image processing apparatus.
As illustrated in FIG. 9, the image processing apparatus 10 includes an imaging unit 100 that captures a pattern image, a dot analysis unit 200 that detects dots forming the pattern image, and a unit code pattern and an information block based on the detected dots. A decoding unit 300 for restoring and decoding the information.

  The imaging unit 100 is configured using an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and reads a pattern image formed on a paper surface as a continuous image frame. Then, the read pattern image is sent to the dot analysis unit 200 as 8-bit image data. The size of the image read by the imaging unit 100 is not particularly limited, but is 156 pixels × 156 pixels here.

  The dot analysis unit 200 is realized by a program-controlled CPU, memory, and the like, detects dots from the image captured by the imaging unit 100, and generates a two-dimensional bit string. The dot analysis unit 200 includes a position detection unit 210, an angle detection unit 220, and a synchronization unit 230 as functions for analyzing image data.

  The position detection unit 210 detects a dot position from the 8-bit image received from the imaging unit 100. Specifically, dots are detected from the acquired image frame, and position information (coordinate values) of the detected dots on the image frame is calculated. Therefore, the position detection unit 210 is a unit image detection unit that detects dots that are unit images.

  The angle detection unit 220 detects the alignment angle of dots on the image detected by the position detection unit 210. The dot alignment angle is an inclination of the dot alignment direction in the image, and is a rotation angle with respect to the original arrangement of dots. The original arrangement of dots is, for example, an arrangement of vertical and horizontal dots in a unit code pattern as shown in FIG.

  The synchronization unit 230 converts the dot arrangement obtained as a detection result of the position detection unit 210 and the angle detection unit 220 into a two-dimensional bit arrangement. This conversion is called synchronization, and the arrangement of each bit in the obtained bit arrangement is called a synchronization line. The obtained bit arrangement and information on the angle and interval of the synchronization line are sent to the decoding unit 300.

  Decoding section 300 is realized by a program-controlled CPU, memory, and the like. The decoding unit 300 detects a pattern block of 3 dots × 3 dots based on the bit arrangement received from the dot analysis unit 200, the synchronization line angle and interval information. The obtained pattern block is rotationally corrected to restore the information block, and then the information recorded by the information block is decoded. Further, when the image processing apparatus 10 is a pen device that reads a pattern image while writing on a sheet on which the pattern image is formed (the detailed configuration of the pen device will be described later), the position of the pen tip and the imaging unit There is a certain deviation from the position imaged by 100. Therefore, the decoding unit 300 converts the coordinates obtained from the image into the coordinates of the pen tip based on the information on the angle and interval of the synchronization line.

  Also, as shown in FIG. 9, information about the success or failure of decoding in the decoding unit 300 is fed back to the position detection unit 210, angle detection unit 220, and synchronization unit 230 of the dot analysis unit 200. The position detection unit 210, the angle detection unit 220, and the synchronization unit 230 determine parameters to be used in each process based on the information on the success or failure of the decoding. Specifically, as a result of processing using predetermined parameters, if decoding is successful (a correct decoding result is obtained), processing is performed using the same parameters for the next image frame. I will do it. On the other hand, when decoding fails, the parameter with the highest success rate is selected and used for the next image frame with reference to the past processing history.

  When the image processing apparatus 10 is a pen device to be described later, the image processing apparatus 10 is provided with a function for recognizing whether writing with a pen is performed. The position detection unit 210, the angle detection unit 220, the synchronization unit 230, and the decoding unit 300 of the dot analysis unit 200 are supplied with a stroke start signal indicating that writing with a pen has started. When a stroke start signal is supplied, these function blocks initialize their settings. Furthermore, when the image processing apparatus 10 is a pen device, the decoding unit 300 has relative distance information (pen-tip distance information) from the pen tip to the imaging surface for correcting the pen tip from the decoding result. Supplied.

FIG. 10 is a flowchart showing the flow of processing performed by the dot analysis unit 200 and the decoding unit 300.
As an initial operation, it is assumed that the imaging unit 100 has captured a sheet of paper as a medium.
As shown in FIG. 10, first, the position detection unit 210 initializes the dot coordinate storage area in the memory (step 1001), and detects the dot position (step 1002). If the position detection is successful (Yes in Step 1003), the angle detection unit 220 then initializes the dot alignment angle storage area in the memory (Step 1004) and detects the dot alignment angle (Step 1005). . If the angle detection is successful (Yes in Step 1006), the synchronization unit 230 then initializes the bit array storage area of the memory (Step 1007) and synchronizes the dot array with the bit array (Step 1008). If the dot array synchronization is successful (Yes in Step 1009), the decoding unit 300 then initializes the decoding result storage area of the memory (Step 1010) and performs the decoding process (Step 1011). If the decoding is successful (Yes in Step 1012), the decoding unit 300 transmits a signal indicating that the decoding is successful to the position detection unit 210, the angle detection unit 220, and the synchronization unit 230 (Step 1013). The result is output (step 1014).

  When detection of the position of the dot by the position detection unit 210, detection of the dot alignment angle by the angle detection unit 220, or synchronization of the dot arrangement by the synchronization unit 230 fails (No in steps 1003, 1006, and 1009) No processing after the failed step is performed. In these cases, and when decoding by the decoding unit 300 fails (No in step 1012), the position detection unit 210, the angle detection unit 220, and the synchronization unit transmit signals indicating that the decoding unit 300 has failed in decoding. 230 (step 1015) and output as a decoding result (step 1014).

<Influence of image distortion in angle detection and synchronization>
Here, the influence of image distortion in angle detection and synchronization will be described.
11 to 13 are diagrams illustrating examples of images captured by the imaging unit 100 (hereinafter referred to as captured images).
FIG. 11A is an example of a captured image when the dot alignment angle is 0 degree, and FIG. 11B is an example of a captured image when the dot alignment angle is 45 degrees. In these figures, an XY coordinate system orthogonal to the captured image is set, and the horizontal direction is the X axis and the vertical direction is the Y axis. The case where the dot alignment direction is along the X axis and the Y axis is defined as an angle of 0 degree.

  In FIGS. 11A and 11B, the imaging surface of the imaging unit 100 and the paper surface on which dots are formed are parallel. In the state shown in these drawings, the dots are arranged in two directions orthogonal to each other (in the drawings, the directions indicated by the solid line and the broken line are the dot alignment directions). A quadrangle formed by four adjacent dots forms a square. In these cases, it is easy to specify the alignment direction of the dots by examining the positional relationship between two adjacent dots and to detect the alignment angle with respect to the original arrangement of the dots.

  As a method of synchronizing the detected dots with the bit arrangement, a grid along the dot arrangement is applied to the image frame, and a bit value (0 or 1) is specified for each grid cell according to the presence or absence of dots. A method can be considered. As shown in FIGS. 11A and 11B, when the dots are arranged along two orthogonal directions, an orthogonal lattice is applied. In the case of FIG. 11B, the lattice may be rotated and applied in accordance with the dot alignment angle.

  When the image processing apparatus 10 is a pen device or the like, which will be described later, the imaging surface of the imaging unit 100 is inclined with respect to the paper surface depending on the angle of the pen device with respect to the paper surface, and the captured image is distorted. In this case, as in FIGS. 11A and 11B, it is not easy to detect the dot alignment angle or to synchronize using a fixed grid.

  FIG. 12 is a captured image in a state in which the upper part of the image is inclined to the back and the lower part is inclined to the front with respect to FIG. In this state, the interval between the dots in the captured image is short at the top and long at the bottom, so that a quadrangle formed by four adjacent dots becomes a trapezoid or the like. When the captured image is distorted as shown in FIG. 12, the dot interval and the dot alignment angle vary depending on the location, and thus a fixed lattice such as an orthogonal lattice cannot be applied.

  FIG. 13 is a captured image in a state in which the upper part of the image is inclined to the back and the lower part is inclined to the front with respect to FIG. In this state, since the arrangement in the two directions of the dots in the captured image is not orthogonal, a quadrangle formed by four adjacent dots is a rhombus or a parallelogram. When the captured image is distorted as shown in FIG. 13, in a quadrangle formed by four adjacent dots, the length of a short diagonal line approaches the length of two sides, making it difficult to detect the correct dot alignment direction. Become. In the example shown in FIG. 13, the dots are recognized as being arranged in three directions: a direction indicated by a solid line, a direction indicated by a broken line, and a direction indicated by a one-dot chain line.

  Therefore, in the present embodiment, the angle detection unit 220 first detects a neighboring dot pair for each dot detected from the captured image by the position detection unit 210. The proximity dot pair is composed of a dot and a dot closest to the dot when a predetermined dot is noticed in many cases. Next, three angles are extracted based on the detected frequency distribution of the angles of adjacent dot pairs, and the dot alignment angle is specified based on the relationship between the extracted three angles and the like.

  In addition, the synchronization unit 230 specifies a synchronization origin near the center of the image frame, and draws a grid line (synchronization line) while performing correction in accordance with the actual dot position in order from the synchronization origin to the outside. This sets the grid for synchronization.

<Function of angle detector>
Next, the function of the angle detection unit 220 will be described in more detail.
FIG. 14 is a diagram illustrating a functional configuration of the angle detection unit 220.
The angle detection unit 220 includes a proximity dot pair detection unit 221 that detects a proximity dot pair, and a frequency distribution generation unit 222 that generates frequency distribution data of the detected proximity dot pair. Then, a first axis angle extracting unit 223, a second axis angle extracting unit 224, and a third axis angle extracting unit 225 that extract three angles based on the generated frequency distribution data, and dots from the extracted three angles And an angle specifying unit 226 for selecting and specifying the alignment angle. In the configuration shown in the figure, the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, the third axis angle extraction unit 225, and the angle specification unit 226 (the part surrounded by a broken line in the figure). Is an angle specifying means for specifying the dot alignment angle.

  The proximity dot pair detection unit 221 acquires position information (coordinate values) on the image frame of the dots detected by the position detection unit 210, and detects a proximity dot pair. In other words, the proximity dot pair detection unit 221 is means for detecting a combination of dot coordinate values that are closest to the dot coordinate value that is the output of the position detection unit 210. A detailed method for detecting the adjacent dot pair will be described later.

  The frequency distribution generation unit 222 generates frequency distribution data of the angle formed by the adjacent dot pair detected by the adjacent dot pair detection unit 221. Here, the angle formed by the adjacent dot pair is a predetermined direction (for example, in the captured image) in which a straight line connecting the two dots constituting the adjacent dot pair serves as a reference for the dot position information output from the position detection unit 210. This is an angle formed with respect to the set XY coordinate (X-axis direction).

  The first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 are candidates for dot alignment directions based on the frequency distribution data generated by the frequency distribution generation unit 222. The direction angle (first axis angle, second axis angle, third axis angle) is extracted. A detailed method of detecting each angle will be described later.

  The angle specifying unit 226 is based on the first axis angle, the second axis angle, and the third axis angle extracted by the first axis angle extracting unit 223, the second axis angle extracting unit 224, and the third axis angle extracting unit 225. The angle (alignment angle) of the dot alignment direction corresponding to the direction of the original dot arrangement is specified and output. For example, in the case of the unit code pattern as shown in FIG. 3, since the dots are arranged in two vertical and horizontal directions, the alignment angles in two directions corresponding to the two vertical and horizontal directions are specified. In FIG. 14, these two directions are described as an X axis and a Y axis. A detailed method for specifying the alignment angle will be described later.

  Further, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 are based on the distance between dots of adjacent dot pairs arranged along the extracted angle in this direction. A reference interval between dots is calculated. Then, the angle specifying unit 226 specifies a reference interval between dots at two alignment angles. In other words, the first axis angle extracting unit 223, the second axis angle extracting unit 224, the third axis angle extracting unit 225, and the angle specifying unit 226 serve as an interval specifying unit that specifies the dot interval in the dot arrangement detected from the captured image. Also works. A method for detecting the reference interval will be described later.

<Detection method of adjacent dot pairs>
Next, a method for detecting a proximity dot pair by the proximity dot pair detection unit 221 will be described in detail.
FIG. 15 is a diagram illustrating a state in which a pair of adjacent dots is detected from an image in which dots are detected. FIG. 15A shows an example of the dot position detected by the position detection unit 210, and FIG. 15B shows a state in which the proximity dot pair is detected with respect to FIG. 15A. In FIG. 15B, adjacent dot pairs are represented by connecting lines. In this case, when the dot B closest to a certain dot A forms a close dot pair with another dot C, the dot A does not form a close dot pair with any other dot (A, B, C is not particularly shown).

FIG. 16 is a diagram illustrating a specific example of a method for detecting a neighboring dot pair.
In FIG. 16, for example, it is assumed that adjacent dot pairs are detected for dots numbered 1 to 4.
The proximity dot pair detection unit 221 first recognizes the positions of dots numbered 1 to 4 detected by the position detection unit 210. Then, the distance to the dots of other numbers is scanned based on the dot of number 1. As a result, the dot closest to the number 1 dot is the number 3 dot, and the distance (PD) between them is recognized as 20. Then, the proximity dot pair detection unit 221 has information “Pair [1] = 3” indicating that the number 3 dot is a proximity dot for the number 1 dot, and the distance to the number 3 dot is 20. Information “MPD [1] = 20” is generated. Similarly, information (hereinafter, attribute information) of “Pair [3] = 1” and “MPD [3] = 20” is generated for the dot of number 3.

  Here, the number in [] of Pair [] indicates the number of its own dot, and the value defined by Pair [] = indicates the number of the partner dot that is currently a close dot pair. Further, the number in [] of MPD [] indicates the number of its own dot, and the value of MPD [] indicates the distance to the partner dot that is the adjacent dot pair at the present time.

Next, the proximity dot pair detection unit 221 scans the distance to dots other than number 1 with reference to the number 2 dot. When the dot number 1 is scanned first, it is determined that the dot number 1 is not the closest dot to the number 2 dot, so the dot number 1 is excluded from the scan target.
As a result of this scan, the dot closest to the number 2 dot is the number 3 dot, and the distance (PD) between them is recognized as 15. This distance is shorter than the dot distance (20) between number 1 and number 3. Therefore, the position detection unit 210 generates attribute information “Pair [2] = 3” and “MPD [2] = 15” for the dot of number 2 and “Pair [3] = 2” for the dot of number 3. And attribute information “MPD [3] = 15” is generated. At this time, since the dot of number 1 is deprived of the dot of number 3 by the dot of number 2, the attribute information is changed to “Pair [1] = − 1” and “MPD [1] = − 1”. , The dot of number 1 is in a state of not forming a close dot pair with any number of dots.

  Similarly, the proximity dot pair detection unit 221 scans the distance to dots other than the numbers 1 and 2 with the number 3 as a reference. As a result, the dot closest to the number 3 dot is the number 4 dot, and the distance (PD) between them is recognized as 10. This distance is shorter than the dot distance (15) between number 2 and number 3. Therefore, the position detection unit 210 generates attribute information “Pair [3] = 4” and “MPD [3] = 10” for the dot of number 3, and “Pair [4] = 3” for the dot of number 4. And “MPD [4] = 10” attribute information is generated. At this time, since the dot of number 2 is deprived of the dot of number 3 by the dot of number 4, the attribute information is changed to “Pair [2] = − 1” and “MPD [2] = − 1”. , The dot of number 2 is in a state that does not form a close dot pair with any of the dots of any number.

  At this time, the dot that has not been scanned is only the dot with the number 4, but since this dot has already formed a close dot pair with the dot with the number 3, the process is finished without being scanned again. Although the case where the number of dots is four is taken as an example here, even when the number of dots is five or more, adjacent dot pairs are sequentially detected by repeating the scanning procedure described above.

FIG. 17 is a flowchart for explaining the flow of proximity dot pair detection processing by the proximity dot pair detection unit 221.
As described with reference to FIG. 16, the proximity dot pair is detected by sequentially examining each of the plurality of dots as a reference and examining the positional relationship with other dots. This reference dot is called a reference dot, and the other dots are called target dots. As shown in FIG. 17, the proximity dot pair detection unit 221 performs the following loop processing while sequentially setting each dot as a reference dot.

  First, the proximity dot pair detection unit 221 acquires the coordinate value of the dot selected as the reference dot (step 1701), generates a dot pair number (PairIndex), and assigns it to the reference dot (step 1702). Here, the coordinate value of the reference dot is (Xs, Ys). Next, the proximity dot pair detection unit 221 executes the following loop processing while sequentially setting each dot other than the reference dot as a target dot.

  First, the proximity dot pair detection unit 221 acquires the coordinate value of the dot selected as the target dot (step 1703). Here, the coordinate value of the target dot is (Xe, Ye). Next, the proximity dot pair detection unit 221 calculates the distance ((Xs, Ys) − (Xe, Ye)) between the reference dot and the target dot (step 1704). When the calculated inter-dot distance is the minimum value among the inter-dot distances regarding the current reference dot (Yes in step 1705), dot pair information (for example, the relationship between the target dot and the reference dot) , Pair [] and MPD [] shown in FIG. 16 are generated and stored (step 1706). At this time, if there is dot pair information that uses other dots as target dots, the dot pair information is deleted.

  On the other hand, when the inter-dot distance calculated in step 1704 is not the minimum value among the inter-dot distances regarding the current reference dot (No in step 1705), in other words, dot pair information regarding other dots with shorter inter-dot distances. Is already stored, the already stored dot pair information is retained as it is.

  When the loop processing of steps 1703 to 1706 is performed with all the dots other than the reference dots as the target dots, the relationship between the target dots and the reference dots is determined by setting one dot closest to the reference dot as the target dot. The dot pair information to be represented is saved. Thereafter, the proximity dot pair detection unit 221 performs processing for determining whether the reference dot and the target dot are proximity dot pairs based on the dot pair information stored in Step 1706 (Step 1707). FIG. 18 is a flowchart for explaining the details of the proximity dot pair discrimination processing shown in step 1707.

  Referring to FIG. 18, the proximity dot pair detection unit 221 first checks whether or not the target dot is linked to another dot. That is, it is checked with reference to the attribute information (Pair [] and MPD []) described with reference to FIG. 16 whether or not the target dot is already paired with another dot. If the attribute information is set for the target dot, the target dot is linked to another dot. If the attribute information is not set for the target dot, the target dot is not linked to another dot. If there is a link between the target dot and another dot (Yes in step 1801), the adjacent dot pair detection unit 221 next determines the inter-dot distance between the target dot and the reference dot, and the target dot and link. The inter-dot distance between the other dots is compared. If the inter-dot distance between the reference dot and the reference dot is shorter (No in step 1802), the proximity dot pair detection unit 221 deletes the link between the target dot and another dot (step 1803).

  When there is no link between the target dot and another dot (No in Step 1801), or after deleting the link between the target dot and another dot in Step 1803, the proximity dot pair detection unit 221 Check whether the reference dot is linked to other dots. If there is a link between the reference dot and another dot (Yes in step 1804), the adjacent dot pair detection unit 221 then determines the inter-dot distance between the reference dot and the target dot, and the reference dot and link. The inter-dot distance between the other dots is compared. If the inter-dot distance to the target dot is shorter (No in Step 1805), the proximity dot pair detection unit 221 deletes the link between the reference dot and other dots (Step 1806).

Thereafter, the proximity dot pair detection unit 221 links the reference dot and the target dot (step 1807). Thereby, the reference dot and the target dot form a close dot pair.
On the other hand, when the inter-dot distance between the target dot and the other linked dots is shorter than the inter-dot distance between the reference dot and the target dot (Yes in step 1802), or When the inter-dot distance between other linked dots is shorter (Yes in step 1805), the proximity dot pair detection unit 221 does not link the reference dot and the target dot. Therefore, these combinations do not form adjacent dot pairs.

<Method for extracting first axis angle, second axis angle, and third axis angle>
Next, a method for extracting the first axis angle, the second axis angle, and the third axis angle by the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 will be described in detail.
First, as preprocessing, the frequency distribution generation unit 222 generates frequency distribution data of angles formed by adjacent dot pairs detected by the adjacent dot pair detection unit 221. The frequency distribution data is data obtained by integrating the appearance frequency for each angle value formed by the adjacent dot pair.

FIG. 19 is a histogram that visually represents an example of the frequency distribution of angles formed by pairs of adjacent dots generated by the frequency distribution generation unit 222.
The frequency distribution data is generated in a range of 180 degrees (in the example of FIG. 19, from −90 degrees to +90 degrees). Using this frequency distribution data, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 extract the first axis angle, the second axis angle, and the third axis angle, respectively. To do.

The first axis angle extraction unit 223 extracts the angle of the maximum frequency as the first axis angle based on the frequency distribution data generated by the frequency distribution generation unit 222. The angle of the maximum frequency is obtained as follows.
The first axis angle extraction unit 223 first detects an angle having the maximum appearance frequency. Then, an average angle based on the appearance frequency is calculated in a certain range centered on the detected angle (hereinafter, target range). The calculated angle is the angle with the highest frequency and is the first axis angle. The average angle is a value obtained by adding a value obtained by multiplying the appearance frequency for each angle of the target range and dividing by the total appearance frequency (a value obtained by adding all the appearance frequencies in the target range).

For example, assuming that the angle having the highest appearance frequency is 10 degrees and the target range is a range from −2 degrees to +2 degrees, the average angle is calculated in the range of 8 degrees to 12 degrees. Also, for example, in this range, if the appearance frequency of 8 degrees is 10, the appearance frequency of 9 degrees is 20, the appearance frequency of 10 degrees is 100, the appearance frequency of 11 degrees is 50, and the appearance frequency of 12 degrees is 40, the average The angle is 10.4 degrees according to the following calculation.

(8 × 10 + 9 × 20 + 10 × 100 + 11 × 50 + 12 × 40) / (10 + 20 + 100 + 50 + 40) = 10.4

  The second axis angle extraction unit 224 assumes an angle (hereinafter referred to as a reference angle) obtained by making a transition (offset) by a certain angle from the first axis angle extracted by the first axis angle extraction unit 223, and determines the reference angle. The angle with the highest frequency is extracted as the second axis angle within a certain range as the center. Then, similarly to the calculation of the first axis angle, an average angle based on the appearance frequency is calculated in a predetermined target range, and is set as the second axis angle. Here, the transition angle formed by the first axis angle and the reference angle is set based on the original arrangement of dots. For example, the unit code pattern shown in FIG. 3 is 90 degrees. The calculation method of the maximum frequency angle is the same as in the case of the first axis angle. The fixed range including the reference angle may be arbitrarily determined, but may be determined based on whether or not the second axis angle can be detected and the history of decoding success / failure by the decoding unit 300.

  The third axis angle extraction unit 225 extracts the angle with the highest frequency as the third axis angle within a certain range centered on an angle symmetrical to the second axis angle with respect to the reference angle. Then, an average angle based on the appearance frequency is calculated in a predetermined target range, and is set as the third axis angle. For example, when the reference angle is 100 degrees and the second axis angle is 80 degrees, the angle symmetrical to the second axis angle with respect to the reference angle is 120 degrees. The fixed range including this angle may be arbitrarily determined, but may be determined based on whether or not the second axis angle can be detected and the history of decoding success / failure by the decoding unit 300. The calculation method of the maximum frequency angle is the same as in the case of the first axis angle.

FIG. 20 illustrates extraction of the first axis angle, the second axis angle, and the third axis angle by the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225. It is a flowchart explaining the flow of a process.
As shown in FIG. 20, first, the frequency distribution generation unit 222 executes the following loop processing for each adjacent dot pair sequentially.

First, the frequency distribution generation unit 222 acquires the coordinates (X1, Y1) and (X2, Y2) of the two dots of the adjacent dot pair (step 2001), and calculates the angle θ formed by the adjacent dot pair by the following equation: Calculate (step 2002). Then, 1 is added to the frequency of the frequency distribution data of the obtained angle θ (step 2003).

θ = tan ⁻¹ {(Y1-Y2) / (X1-X2)}

  Once the processing of steps 2001 to 2003 has been performed for all adjacent dot pairs, the first axis angle extraction unit 223 next extracts the first axis angle based on the generated frequency distribution data (step 2004). ). Further, a reference interval between dots along the first axis angle is calculated (step 2005). The details of the reference interval and its detection method will be described later.

  Next, the second axis angle extraction unit 224 extracts the second axis angle based on the frequency distribution data and the first axis angle (step 2006), and calculates a reference interval between dots along the second axis angle. (Step 2007). Further, the third axis angle extraction unit 225 extracts the third axis angle based on the frequency distribution data, the reference angle, and the second axis angle (step 2008), and the reference interval between dots along the third axis angle. Is calculated (step 2009).

FIG. 21 is a diagram showing an image of the extracted angles in the three directions.
Assume that the dots arranged as shown in FIG. 21A are detected and the first axis angle is extracted as shown in FIG. The second axis angle and the third axis angle are extracted with a reference angle offset by 90 degrees with respect to the first axis angle.

<Identification method of alignment angle>
Next, a method for specifying the alignment angle by the angle specifying unit 226 will be described in detail.
The angle specifying unit 226 includes a first axis angle, a second axis angle, a first axis angle extracted by the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225. Based on the triaxial angle, the dot alignment angle is specified and output. Specifically, the angles of the two dot alignment directions (referred to as X axis and Y axis in FIG. 14) corresponding to the original dot arrangement direction are specified. This alignment angle is specified by the following procedure.

  First, the angle specifying unit 226 acquires the first axis angle, the second axis angle, the third axis angle, and information on the success or failure of decoding by the decoding unit 300 with respect to the immediately preceding image frame (hereinafter referred to as the previous frame). When decoding for the previous frame is successful, the angle specifying unit 226 selects two of the first axis angle, the second axis angle, and the third axis angle that are close to the alignment angle specified in the previous frame as the alignment angle. . Since the previous frame and the current frame are read continuously, it is considered that the appropriate alignment angle does not change greatly. However, when the image processing apparatus 10 is a pen device, the positional relationship between the imaging unit 100 and the paper surface changes as needed. Therefore, a close angle is selected that is not the same as the alignment angle specified in the previous frame.

When decoding for the previous frame fails, the angle specifying unit 226 specifies the alignment angle in two directions as follows, for example.
If all of the first axis angle, the second axis angle, and the third axis angle have been extracted, select the first axis angle and the second axis angle or the third axis angle that is closer to the reference angle, The alignment angle.
When the third axis angle is not extracted, the angle specifying unit 226 selects the first axis angle and the second axis angle and sets it as the alignment angle. The case where the third axis angle is not extracted means that, in the frequency distribution data, when there is no angle having a high appearance frequency within a certain range centered on an angle symmetrical to the second axis angle with respect to the reference angle, This is the case when the shaft angle and the reference angle are the same.
When the second axis angle is not extracted, the angle specifying unit 226 sets the first axis angle and a reference angle offset by a certain angle with respect to the first axis angle as the alignment angle. The case where the second axis angle is not extracted is a case where, in the frequency distribution data, there is no angle having a high appearance frequency in a certain range centered on the reference angle.

  Furthermore, when decoding fails in a predetermined image frame, the angle specifying unit 226 may select an angle different from that of the image frame that has failed in decoding as the alignment angle in the next image frame. Specifically, for example, when decoding fails using the first axis angle and the second axis angle as the alignment angles, in the next image frame, the first axis angle and the third axis angle are set as the alignment angles. The axis angle and the third axis angle are set as the alignment angle, and so on.

<Calculation method of reference interval>
Next, a method for calculating a reference interval by the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 will be described.
The reference interval is an average value of inter-dot distances in adjacent dot pairs arranged along a certain direction. However, in order to eliminate the proximity dot pairs with a large inter-dot distance protruding, a certain threshold value (maximum inter-dot distance) is provided, and an average value is calculated for the proximity dot pairs whose inter-dot distance is equal to or less than this threshold value. This threshold is determined as follows, for example.

  First, the frequency distribution generation unit 222 generates frequency distribution data of the inter-dot distances of the adjacent dot pairs detected by the adjacent dot pair detection unit 221. The frequency distribution data is data obtained by integrating the appearance frequency for each value of the inter-dot distance of the adjacent dot pair. A distance slightly longer than the inter-dot distance where the appearance frequency is the maximum is set as a threshold value. FIG. 22 shows a histogram that visually represents an example of the frequency distribution of the inter-dot distance. In FIG. 22, the threshold is set to a distance slightly longer than the inter-dot distance with the maximum appearance frequency.

  The first axis angle extraction unit 223 extracts a pair of adjacent dots arranged along the first axis angle that has an inter-dot distance equal to or less than the threshold value obtained as described above. Then, the average value of the inter-dot distance is calculated and set as the reference interval of the dots in the first axis direction. In consideration of various errors, the angle formed by the adjacent dot pair includes a certain width (−α degrees to + α degrees) with respect to the first axis angle. That is, the adjacent dot pair whose angle formed by the adjacent dot pair is the first reference angle ± α is the adjacent dot pair arranged along the first axis angle.

  Similarly, the second axis angle extraction unit 224 extracts a pair of adjacent dots arranged along the second axis angle, in which the inter-dot distance is equal to or less than the threshold obtained as described above, and the second axis direction The dot reference interval is calculated. Further, the third axis angle extraction unit 225 extracts a pair of adjacent dots arranged along the third axis angle that has a dot-to-dot distance equal to or less than the threshold value obtained as described above. The reference interval is calculated.

  The angle specifying unit 226 acquires a reference interval in each direction from the first axis angle extracting unit 223, the second axis angle extracting unit 224, and the third axis angle extracting unit 225. Then, a reference interval in the direction (X axis and Y axis in FIG. 14) specified as the alignment angle is selected and output together with the alignment angle. The alignment angle and the information on the reference interval at the alignment angle are used by the synchronization unit 230 to specify a synchronization line for synchronization.

<Function of the synchronization unit>
Next, the function of the synchronization unit 230 will be described in more detail.
FIG. 23 is a diagram illustrating a functional configuration of the synchronization unit 230.
The synchronization unit 230 performs synchronization processing based on the determined synchronization origin and synchronization reference angle, and the synchronization origin determination unit 231 and the synchronization reference angle determination unit 232 that determine the synchronization origin and the synchronization reference angle as preprocessing. And a synchronization unit 233. The synchronization unit 230 includes a bit array generation unit 234 that generates a two-dimensional bit array based on the synchronization result. As illustrated in FIG. 23, the synchronization unit 230 includes position information (XY coordinate values) on the image frame of the dots detected by the position detection unit 210 and the dot alignment angle detected by the angle detection unit 220. (X axis and Y axis) and a reference interval are supplied.

  The synchronization origin determining unit 231 determines a synchronization origin for starting synchronization based on the position information (XY coordinate values) of the dots detected by the position detection unit 210 on the image frame. One of the actual dots detected by the position detector 210 is assigned to the synchronization origin. The synchronization origin is the center of the two-dimensional bit array. A specific method for determining the synchronization origin will be described later.

  The synchronization reference angle determination unit 232 determines the synchronization reference angle based on the dot alignment angles (X axis and Y axis) detected by the angle detection unit 220 and the reference interval. The synchronization reference angle is the rotation angle of the X axis and the Y axis that intersect at the synchronization origin. The directions of the X axis and the Y axis ideally coincide with the dot alignment angle detected by the angle detection unit 220. However, the angle detection unit 220 detects the angle using a minute distance that is a dot interval between adjacent dot pairs. Therefore, the synchronization reference angle determination unit 232 performs correction on a macroscopic scale using the positional relationship of more dots with respect to the alignment angle. Thereby, the accuracy of the rotation angle of the X axis and the Y axis is improved. A specific method for determining the rotation angle of the X axis and the Y axis as the synchronization reference angle will be described later.

  The line synchronization unit 233 generates a synchronization grid by sequentially drawing grid lines while synchronizing each line (lattice line) from the synchronization origin to the outside. As a result, not a fixed grid prepared in advance, but a grid that individually corresponds to the distortion of the captured image is dynamically generated. A specific method for generating the lattice will be described later.

  The bit array generation unit 234 generates a two-dimensional bit array based on the lattice generated by the line synchronization unit 233. Specifically, the bit arrangement is generated by arranging 1 when there is a dot on the grid point and 0 when there is no dot. In the optical system of the imaging unit 100, the captured image may be inverted (so-called mirror image inversion) depending on the number of mirrors that guide light. In this case, the bit array generation unit 234 inverts the generated bit array and outputs the result.

FIG. 24 is a flowchart showing the overall flow of processing by the synchronization unit 230.
As shown in FIG. 24, in the synchronization unit 230, the synchronization origin determining unit 231 first determines the synchronization origin (step 2401). Next, based on the determined synchronization origin, the synchronization reference angle determination unit 232 determines synchronization reference angles in two directions (X-axis direction and Y-axis direction) (steps 2402 and 2403). Note that the order of determining the synchronization reference angles in the two directions is not limited to the order shown in FIG.

Next, on the basis of the determined synchronization origin and synchronization reference angle, the line synchronization unit 233 performs line synchronization in four directions (X-axis positive and negative directions, Y-axis positive and negative directions) (steps). 2404-1407). Note that the execution order of line synchronization in the four directions is not limited to the order shown in FIG.
Finally, the bit array generation unit 234 generates a two-dimensional bit array from the dot array using the synchronization grid generated by line synchronization (step 2408).

<Determining the synchronization origin>
Next, a method for determining the synchronization origin by the synchronization origin determination unit 231 and a method for determining the synchronization reference angle by the synchronization reference angle determination unit 232 will be described in detail.
FIG. 25 is a diagram for explaining a method of determining the synchronization origin by the synchronization origin determination unit 231.
First, the synchronization origin determination unit 231 calculates the coordinates of the barycentric positions of a plurality of detected dots based on the position information acquired from the position detection unit 210. The coordinates of the barycentric position are obtained by adding the coordinate values of all the dots for the X coordinate and the Y coordinate, respectively, and dividing by the number of dots. In FIG. 25A, the location indicated by a white circle (◯) is the center of gravity.

  Next, the synchronization origin determining unit 231 calculates the distance from the center of gravity position to each dot, and sets the dot closest to the center of gravity position as the synchronization origin. In FIG. 25B, the dot indicated by the solid line arrow is present at a position closest to the center of gravity indicated by a white circle (◯). Therefore, this dot becomes the synchronization origin.

FIG. 26 is a flowchart for explaining the synchronization origin determination processing by the synchronization origin determination unit 231.
Referring to FIG. 26, the synchronization origin determining unit 231 first calculates the coordinates of the barycentric positions of a plurality of dots (step 2601). Then, the following loop processing is performed for each dot sequentially.

  The synchronization origin determination unit 231 calculates the distance (dist) from the center of gravity position to the dot (step 2602). Then, the minimum distance (dist_min) from the center of gravity position calculated so far to the dot is compared with the distance (dist) calculated in step 2602. If the distance (dist) is smaller (Yes in step 2603), the synchronization origin determination unit 231 updates the minimum distance (dist_min) to the value of the distance (dist) and stores the dot coordinates (step). 2604). On the other hand, when the distance (dist) is larger (No in step 2603), the current minimum distance (dist_min) is maintained as it is.

<Determination of synchronization reference angle>
FIG. 27 is a diagram for explaining a method for determining the synchronization reference angle by the synchronization reference angle determination unit 232.
First, the synchronization reference angle determination unit 232 converts the coordinate value of the dot into a relative coordinate having the synchronization origin as the origin (see FIG. 27A). Then, the converted coordinate value is rotated according to the alignment angle with respect to the Y axis among the alignment angles detected by the angle detection unit 220 (see FIG. 27B). Next, the synchronization reference angle determination unit 232 generates five vertical lattice lines centered on a line passing through the synchronization origin, and superimposes them on the coordinate space obtained in FIG. 27B (see FIG. 27C). ). Here, the interval between the vertical grid lines is a reference interval of dots detected by the angle detection unit 220. The reason for using five vertical grid lines is to obtain a sample with a certain width in the X-axis direction. Note that the number of vertical grid lines to be generated is not limited to five.

  Next, the synchronization reference angle determination unit 232 calculates the amount of deviation from the vertical grid lines for the dots existing within the range of the five vertical grid lines. Then, the variance is calculated for each vertical grid line, and the calculated values are integrated (see FIG. 27D). Further, the synchronization reference angle determination unit 232 performs the same processing for each predetermined angle while rotating the coordinate value of each dot within a certain range around the synchronization origin. Specifically, for example, the alignment angle used in FIG. 27B is rotated by 0.5 degrees within a range of −8 degrees to +8 degrees, and the dispersion for each vertical grid line is obtained at each angle. Accumulate. Then, the angle at which the integrated value of the dispersion becomes the minimum is set as the rotation angle of the Y axis as the synchronization reference angle.

  The synchronization reference angle determination unit 232 performs the same process on the X axis, and determines the rotation angle of the X axis as the synchronization reference angle. Note that either the Y-axis rotation angle or the X-axis rotation angle may be obtained first, or the processes may be performed in parallel. In addition, when processing for either the X axis or the Y axis is performed, if 90 degrees is added to the alignment angle in advance, the coordinate space in which the processing is performed can be made common, and the processing becomes light.

FIG. 28 is a flowchart for explaining the synchronization reference angle determination processing by the synchronization reference angle determination unit 232. Here, the processing operation for one of the alignment angles detected by the angle detection unit 220 will be described. As described above, the synchronization reference angle is determined by the same operation for other alignment angles.
The synchronization reference angle determination unit 232 performs the loop process shown in FIG. 28 for each angle (θ) while rotating within a certain range with respect to the predetermined alignment angle detected by the angle detection unit 220. In this loop processing, the synchronization reference angle determination unit 232 first performs dot information acquisition processing (step 2801). Based on the obtained dot information, a dispersion integration process is performed (step 2802).

FIG. 29 is a flowchart illustrating details of the dot information acquisition process.
The synchronization reference angle determination unit 232 executes the loop process shown in FIG. 29 for each dot sequentially. In this loop process, the synchronization reference angle determination unit 232 first obtains the X coordinate (RotX) of the dot rotated by the angle (θ) about the synchronization origin (step 2901). Next, a shift amount (diff) with respect to the vertical lattice line of the X coordinate (RotX) is acquired (step 2902). Then, the obtained shift amount (diff) is stored (step 2903), and 1 is added to the number of dots to be processed (dots existing within the range of five vertical grid lines in the example of FIG. 27). (Step 2904).

FIG. 30 is a flowchart illustrating the details of the dispersion integration process.
The synchronization reference angle determination unit 232 executes the loop process shown in FIG. 30 for each vertical grid line sequentially. In this loop process, the synchronization reference angle determination unit 232 first acquires the number of dots (cnt) that is finally obtained by addition in step 2904 in FIG. 29 (step 3001). If the number of dots (cnt) is less than 2, the process for the vertical grid line ends (No in step 3002).

  On the other hand, when the number of dots (cnt) is 2 or more (Yes in step 3002), the synchronization reference angle determination unit 232 executes the next loop process while sequentially targeting each dot corresponding to the vertical grid line. To do. First, the dot shift amounts (diff) obtained in steps 2902 and 2903 in FIG. 29 are acquired (step 3003) and integrated (step 3004). Further, a value obtained by squaring the deviation amount (diff) is integrated (step 3005).

After the loop processing in steps 3003 to 3005 is completed, the synchronization reference angle determination unit 232 calculates the variance (var) of the deviation amount related to the vertical grid line by the following equation (step 3006) and integrates (step 3007). .

var = (cnt × Sdiff ² −Sdiff × Sdiff) / (cnt × cnt)

However, Sdiff ² is an integrated value of the square of the deviation amount (diff) obtained in Step 3005, and Sdiff is an integrated value of the deviation amount (diff) obtained in Step 3004.

  Referring to FIG. 28 again, the synchronization reference angle determination unit 232 includes the integrated value (Svar) of the dispersion of the dot shift amount with respect to the vertical grid line obtained by the dispersion integration process in step 2802 and the minimum value obtained so far. The dispersion integrated value (Svar_min) is compared. If the dispersion integrated value (Svar) is smaller (Yes in step 2803), the value of the minimum dispersion integrated value (Svar_min) is updated to the value of the dispersion integrated value (Svar) (step 2804), and the synchronization reference angle (Θest) is updated to the angle (θ) at this time (step 2805). On the other hand, when the variance integrated value (Svar) is equal to or greater than the minimum variance integrated value (Svar_min) (No in step 2803), the current minimum variance integrated value (Svar_min) and the synchronization reference angle (θest) are held as they are. .

<Line synchronization and grid generation>
Next, a line synchronization and lattice generation method by the line synchronization unit 233 will be described in detail.
FIG. 31 is a diagram for explaining the concept of line synchronization by the line synchronization unit 233.
As shown in FIG. 31, the line synchronization unit 233 performs synchronization for each line (lattice line) from the synchronization origin in a total of four directions including the plus and minus directions of the X axis and the plus and minus directions of the Y axis. A grid for synchronization is generated by drawing grid lines in order.

FIG. 32 is a diagram illustrating a state in which synchronization dots are superimposed on an image frame.
When there is no distortion in the image as shown in FIG. 32A, since the grid of the orthogonal lattice matches the position of each dot, synchronization can be performed normally. On the other hand, when the image is distorted as shown in FIG. 32 (b), the grid cannot correspond to the position of each dot in the fixed grid set in advance, so that synchronization cannot be performed.
Therefore, the line synchronization unit 233 according to the present embodiment performs grid synchronization while correcting the inclination and the interval in accordance with the actual dot position in order from the synchronization origin, so that it is possible to cope with a case where the image is distorted. Is generated dynamically.

FIG. 33 is a diagram for explaining a method of generating a lattice line by the line synchronization unit 233.
First, the line synchronization unit 233 converts the coordinate value of the dot into a relative coordinate having a predetermined synchronization reference point as the origin (see FIG. 33A). Then, the converted coordinate values are rotated in accordance with the synchronization line angle with respect to the Y axis (see FIG. 33B). Next, the line synchronization unit 233 detects dots existing inside the lattice boundary (see FIG. 33C), and stores the positions of the lattice lines and the coordinate values of the dots within the lattice boundary (FIG. 33 ( d)).

  Here, the synchronization reference point is a point serving as a position reference when generating each lattice line. Initially, the synchronization origin determined by the synchronization origin determination unit 231 becomes the synchronization reference point, and as the grid line is generated, the synchronization reference point for the next grid line is generated. For example, when a grid line along the Y axis is generated, the synchronization reference point is generated on the X axis passing through the synchronization origin (that is, the synchronization reference point is an intercept of the grid line). The position of the newly generated synchronization reference point is determined according to the final position of the grid line generated immediately before.

  The distance between the newly generated synchronization reference point and the synchronization reference point in the lattice line generated immediately before is called a synchronization interval. Initially, the reference interval calculated by the angle detector 220 is used as the synchronization interval. Each time a grid line is generated, the interval is updated to the interval between the synchronization reference point in the last generated grid line and the synchronization reference point in the grid line generated immediately before.

  The baseline angle is an inclination of a grid line generated with respect to the current synchronization reference point. Initially, the synchronization reference angle determined by the synchronization reference angle determination unit 232 becomes the same baseline angle, and when a new grid line is generated, the final inclination of the grid line generated immediately before is newly updated. This is the same baseline angle of the generated grid line.

  The grid boundary is a virtual line that divides a dot corresponding to the generated grid line from other dots. As shown in FIGS. 33 (c) and 33 (d), the dots present in the band-like region sandwiched between the lattice boundaries are the dots corresponding to the lattice lines. The distance from the lattice line to the lattice boundary is determined according to the lattice line generated immediately before and the position of the lattice line generated before that.

  After generating the grid lines as described above, the line synchronization unit 233 corrects the tilt of the generated grid lines and the position of the synchronization reference point based on the positions of the dots within the grid boundaries. By correcting the synchronization reference point, the synchronization interval is also changed. Further, based on the corrected inclination of the final grid line and the position of the synchronization reference point (grid line position), as described above, the same baseline angle and synchronization interval for the next generated grid line, and the grid A boundary is determined.

  The line synchronization unit 233 calculates a regression line based on the position of the grid line and the position of the dot within the grid boundary, and corrects the grid line. In the present embodiment, RMA (Reduced Major Axis) regression is used for calculating the regression line. According to RMA regression, the regression line is obtained by the following equation.

ここで、σ_xは格子境界内のドットにおけるＸ座標の標準偏差であり、σ_yはＹ座標の標準偏差である。この回帰直線の傾きの符号は、相関関数の符号によって判定される。相関関数Ｒ_xyは、次式によって求められる。

Here, σ _x is the standard deviation of the X coordinate at the dots in the lattice boundary, and σ _y is the standard deviation of the Y coordinate. The sign of the slope of the regression line is determined by the sign of the correlation function. The correlation function R _xy is obtained by the following equation.

なお、実際には、Ｒ_xyの分母（Ｓ_xＳ_y）は常に正なので、分子（Ｓ_xy）の符号によって回帰直線の傾きが判定されることとなる。

In practice, since the denominator (S _x S _y ) of R _xy is always positive, the slope of the regression line is determined by the sign of the numerator (S _xy ).

FIG. 34 is a diagram illustrating how the synchronization reference point is changed by correcting the grid line and the synchronization interval is corrected. FIG. 34 also shows a method for determining a lattice boundary.
First, the synchronization reference point A1 is generated at the position of the synchronization interval P0 from the synchronization reference point A0 in the lattice line generated immediately before, and the lattice line and the lattice boundary B1 passing through the synchronization reference point A1 are set (FIG. 34 ( a)). The grid lines are corrected based on the positions of the dots within the grid boundaries. Next, when the grid line is corrected, the synchronization reference point A1 moves to become the synchronization reference point A1 ′ (FIG. 34 (b)). At this time, the distance between the synchronization reference point A0 and the corrected synchronization reference point A1 ′ is the corrected new synchronization interval P1 (FIG. 34 (c)).

  Next, an intermediate point M1 between the synchronization reference point A0 and the uncorrected synchronization reference point A1 is calculated, and the distance from the corrected synchronization reference point A1 ′ to the point M1 is set as a new grid boundary B2 (FIG. 34 ( d)). Then, the next synchronization reference point A2 is generated at the position of the synchronization interval P1 from the synchronization reference point A1 ′ (FIG. 34 (e)), and the lattice line and the lattice boundary B2 passing through the synchronization reference point A2 are set ( FIG. 34 (f)).

FIG. 35 is a flowchart showing the operation of the line synchronization unit 233.
Each time the line synchronization unit 233 draws a new grid line, the line synchronization unit 233 executes a loop process shown in FIG. 35 for the new grid line. In this loop processing, the line synchronization unit 233 first executes synchronization processing of dots within the lattice boundary (step 3501). If a dot is detected within the lattice boundary, the synchronization line angle and the synchronization interval are corrected, and the lattice boundary in the next lattice line is set (steps 3502 and 3503).

FIG. 36 is a flowchart for explaining details of the dot synchronization processing.
The line synchronization unit 233 executes the loop process shown in FIG. 36 for each dot sequentially. In this loop process, the line synchronization unit 233 first calculates the distance (diffX, diffY) between the synchronization reference point (coordinates: (BaseX, BaseY)) and the dot (step 3601). Next, the line synchronization unit 233 obtains the position of the dot (X-axis direction synchronization: RotX, Y-axis direction synchronization: RotY) rotated by the synchronization line angle around the synchronization reference point (step 3602).

  Next, the line synchronization unit 233 determines whether the position of the dot after rotation is within the lattice boundary. If within the grid boundary (Yes in step 3603), the line synchronization unit 233 stores the grid line number (grating line identification information) and the coordinates (x, y) of the dot before rotation (steps 3604 and 3605). Then, the number of dots entering the grid boundary is counted up (step 3606). On the other hand, when the position of the dot after rotation is outside the grid boundary (No in step 3603), the line synchronization unit 233 excludes the dot from the process related to the current grid line.

FIG. 37 is a flowchart for explaining the details of the synchronization line angle and synchronization interval correction processing and the lattice boundary setting processing.
First, the line synchronization unit 233 obtains the average (AveX, AveY) and variance (VarX, VarY) of the coordinates before the rotation of the dots in the lattice boundary (step 3701). Then, the synchronization line angle (LineSlope) that is the inclination of the lattice line is corrected by the following equation (step 3702).

LineSlope = sqrt (VarY / VarX)) (sqrt is the square root)

If the correlation coefficient is negative, the sign of the slope of the synchronization line angle is inverted to negative (steps 3703 and 3704).

Next, the line synchronization unit 233 corrects the synchronization reference point (LineIntercept) that is the intercept of the lattice line by the following equation (step 3705).

LineIntercept = AveY-LineSlope · AveX

Next, the line synchronization unit 233 calculates a rotation angle (see step 3602 in FIG. 36) in the line synchronization process for the next lattice line by tan ⁻¹ (LineSlope) (step 3706). Further, the absolute value (IfAbsLineSlope) of the synchronization line angle is obtained (step 3707). The line synchronization unit 233 corrects the X coordinate of the synchronization reference point if the current process is synchronization in the X axis direction, and corrects the Y coordinate of the synchronization reference point if synchronization is in the Y axis direction. (Steps 3708, 3709, 3710). The correction of the synchronization reference point is based on the following calculation.

X coordinate: BaseX _new = (y-LineIntercept) / LineSlope
Y coordinate: BaseY _new = LineSlope · X + LineIntercept

Next, the line synchronization unit 233 calculates the corrected displacement (diffX, diffY) of the synchronization reference point as follows (step 3711).

diffX = BaseX _new -BaseX
diffY = BaseY _new −BaseY

Further, the line synchronization unit 233 updates the synchronization interval (Pitch) (step 3712).

Pitch _new = sqrt (diffX ² + diffY ² )

In addition, the lattice boundary (BoundaryLimit) in the line synchronization processing for the next lattice line is corrected (step 3713).

BoundaryLimit = Pitch _new -Pitch / 2

Referring to FIG. 35 again, the line synchronization unit 233 corrects the position of the synchronization reference point (BaseX _next , BaseY on the _next grid line) using each value corrected and updated in step 3503 described in detail in FIG. _next ) is calculated (step 3504).

BaseX _next = BaseX + (Pitch _new · Cosθ _y + Pitch _new · Sinθ _y )
BaseY _next = BaseY− (Pitch _new · Sinθ _x −Pitch _new · Cosθ _y )

<Configuration of pen device>
Next, a specific hardware configuration of the image processing apparatus 10 in the present embodiment will be described. Here, a configuration example of a pen device that realizes the image processing apparatus 10 will be described.
FIG. 38 shows the mechanism of the pen device.
As illustrated, the pen device 60 includes a control circuit 61 that controls the operation of the entire pen. The control circuit 61 includes an image processing unit 61a that processes a code image detected from an input image, and a data processing unit 61b that extracts identification information and coordinate information from the processing result there. Although not particularly shown, the control circuit 61 includes a non-volatile memory that stores programs for operating the image processing unit 61a and the data processing unit 61b. The control circuit 61 is connected to a pressure sensor 62 that detects a writing operation by the pen device 60 by a pressure applied to the pen tip 69. Further, an infrared LED 63 that irradiates infrared light onto the medium and an infrared CMOS 64 that inputs an image are also connected. Further, an information memory 65 for storing identification information and coordinate information, a communication circuit 66 for communicating with an external device, a battery 67 for driving the pen, and pen identification information (pen ID) are stored. A pen ID memory 68 is also connected.

  In the image processing apparatus 10 illustrated in FIG. 9, the imaging unit 100 is realized by, for example, the infrared CMOS 64 in FIG. 38. The dot analysis unit 200 is realized by, for example, the image processing unit 61a in FIG. Furthermore, the decoding unit 300 is realized by, for example, the data processing unit 61b in FIG. In addition to the pen device 60 as shown in FIG. 38, the image processing apparatus 10 may realize functions realized by the image processing unit 61a or the data processing unit 61b by, for example, a general-purpose computer.

  Programs that realize the functions of the image processing unit 61a and the data processing unit 61b can be provided via communication means or stored in a recording medium such as a CD-ROM. Further, it may be provided by being stored in the above-described nonvolatile memory included in the control circuit 61.

It is a figure which shows the relationship between the unit code pattern used by this embodiment, an information block, and a paper surface (medium). It is a figure which shows the example of the unit code pattern in a mCn system. FIG. 3 is a diagram comprehensively showing types of unit code patterns shown in FIG. 2. It is a figure which shows a response | compatibility with the unit code pattern and pattern value in 9C2 system. It is a figure which shows the example of the synchronous pattern in 9C2 system. It is a figure which shows the structural example of an information block. It is a figure which shows the example of the layout for expressing the identification information and coordinate information in this embodiment. It is a figure which shows the example of the encoding of the coordinate information using M series. It is a figure which shows the function structural example of the image processing apparatus of this embodiment. It is a flowchart which shows the flow of the process by the dot analysis part and decoding part of this embodiment. It is a figure which shows the example of a captured image, and is an example of a captured image in case the alignment angle of a dot is 0 degree | times. It is a figure which shows the example of a captured image, and is an example of a captured image in case the alignment angle of a dot is 45 degree | times. It is a figure which shows the picked-up image of the state where the upper part of the image leaned back and the lower part leaned forward. It is a figure which shows the picked-up image of the state in which the upper part of the image leans back and FIG. It is a figure which shows the function structure of the angle detection part of this embodiment. It is a figure which shows a mode that the proximity | contact dot pair was detected from the image from which the dot was detected. It is a figure which shows the specific example of the detection method of the proximity | contact dot pair in this embodiment. It is a flowchart explaining the flow of the detection process of the proximity dot pair by the proximity dot pair detection part of this embodiment. FIG. 18 is a flowchart illustrating details of proximity dot pair determination processing shown in step 1707 of FIG. 17. FIG. It is the histogram which expressed visually the example of the frequency distribution of the angle which the close dot pair produced | generated by the frequency distribution production | generation part of this embodiment makes. Flow of extraction processing of the first axis angle, the second axis angle, and the third axis angle by the frequency distribution generation unit, the first axis angle extraction unit, the second axis angle extraction unit, and the third axis angle extraction unit of the present embodiment. It is a flowchart to explain. It is a figure which shows the image of the angle of 3 directions extracted by this embodiment. It is the histogram which expressed the example of frequency distribution of the distance between dots in this embodiment visually. It is a figure which shows the function structure of the synchronization part of this embodiment. It is a flowchart which shows the whole flow of the process by a synchronizer. It is a figure explaining the determination method of the synchronous origin by the synchronous origin determination part of this embodiment. It is a flowchart explaining the determination process of the synchronous origin by a synchronous origin determination part. It is a figure explaining the determination method of the synchronous reference angle by the synchronous reference angle determination part of this embodiment. It is a flowchart explaining the determination process of the synchronous reference angle by a synchronous reference angle determination part. It is a flowchart explaining the detail of the dot information acquisition process by a synchronous reference angle determination part. It is a flowchart explaining the detail of the dispersion | distribution integration process by a synchronous reference angle determination part. It is a figure explaining the concept of the line synchronization by the line synchronization part of this embodiment. It is a figure which shows a mode that the dot for a synchronization was piled up with respect to the image frame. It is a figure explaining the production | generation method of the lattice line by a line synchronizing part. It is a figure which shows a mode that a synchronous reference point is changed by correction | amendment of a lattice line, and a synchronous space | interval is correct | amended. It is a flowchart which shows operation | movement of a line synchronizer. It is a flowchart explaining the detail of a dot synchronization process. It is a flowchart explaining the detail of the correction process of a synchronous line angle and a synchronous space | interval, and the setting process of a lattice boundary. It is the figure which showed the mechanism of the pen device which is an image processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 100 ... Imaging part, 200 ... Dot analysis part, 210 ... Position detection part, 220 ... Angle detection part, 230 ... Synchronization part, 231 ... Synchronization origin determination part, 232 ... Synchronization reference angle determination part, 233: Line synchronization unit, 234: Bit array generation unit, 300: Decoding unit

Claims (14)

A detection unit that scans an image captured by the imaging device, detects a unit image, and identifies a position;
Based on the position information of the unit image detected by the detection unit, a line synchronization unit that generates a synchronization grid by sequentially drawing grid lines from the synchronization origin according to the coordinate axis;
An image processing apparatus comprising: a bit array generation unit configured to generate a bit array based on the presence or absence of the unit image at a predetermined position on the image based on the lattice generated by the line synchronization unit.   The image processing apparatus according to claim 1, wherein the line synchronization unit individually adapts each grid line to a position of the unit image associated with the grid line.   The image processing apparatus according to claim 2, wherein the line synchronization unit corrects the inclination of each grid line according to the position of the unit image associated with the grid line.   The image processing apparatus according to claim 2, wherein the line synchronization unit corrects the interval between the grid lines in accordance with the position of the unit image associated with the grid line.   The initial values of the grid line inclination and interval used by the line synchronization unit are determined based on the positional relationship between the two unit images in the combination of the two unit images located closest to the unit image. The image processing apparatus according to claim 1. By correcting the angle formed by the two unit images in the unit image combination based on the positional relationship of the unit images in a certain range including the synchronization origin, a reference angle serving as a reference for the inclination of the grid line A reference angle determination unit for determining
The image processing apparatus according to claim 5, wherein the line synchronization unit uses the reference angle determined by the reference angle determination unit as an initial value of the inclination of the grid line. An imaging unit for imaging the surface of the medium on which the unit image is formed;
A detection unit that detects the unit image from the image captured by the imaging unit and identifies the position;
Based on the position information of the unit image detected by the detection unit, a line synchronization unit that generates a synchronization grid by sequentially drawing grid lines from the synchronization origin according to the coordinate axis;
A pen device comprising: a bit array generation unit that generates a bit array in accordance with the presence or absence of the unit image at a predetermined position on the image based on the lattice generated by the line synchronization unit.   The pen device according to claim 7, wherein the line synchronization unit individually adapts each grid line to a position of the unit image associated with the grid line.   The pen device according to claim 8, wherein the line synchronization unit corrects the inclination of each grid line in accordance with the position of the unit image associated with the grid line.   The pen device according to claim 8, wherein the line synchronization unit corrects the interval between the grid lines in accordance with the position of the unit image associated with the grid line. On the computer,
A function of scanning an image picked up by an image pickup device, detecting a unit image, and specifying a position;
Based on the detected position information of the unit image, a function of generating a grid for synchronization by sequentially drawing grid lines from the synchronization origin according to the coordinate axis;
A program for realizing a function of generating a bit arrangement according to the presence or absence of the unit image at a predetermined position on the image based on the generated lattice.   12. The function of generating the grid causes the computer to execute a process of drawing each grid line by individually adapting the grid line to the position of the unit image associated with the grid line. The listed program.   The function for generating the grid causes the computer to execute a process of correcting the inclination of each grid line according to the position of the unit image associated with the grid line. The listed program.   The function of generating the grid causes the computer to execute a process of correcting the interval between the grid lines in accordance with the position of the unit image associated with the grid line. The listed program.

Images