JP2009181346A - Image processor, pen device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect the direction of a pattern image by specifying a direction in which unit images configuring the pattern image are arranged. <P>SOLUTION: An image processor includes: a position detection part for imaging a recording medium in which unit images arrayed along at least two directions crossing at a prescribed angle are recorded by an imaging part, and for detecting unit images (dots) from the acquired image; a near dot pair detection part 221 for detecting a pair of near dots being the combination of the most closely positioned unit images as regards the unit images detected from the image by the position detection part; a first axial angle extraction part 223, a second axial angle extraction part 224, a third axial angle extraction part 225 and an angle specification part 226 for specifying an angle showing the inclination of the array of the unit images on the basis of the appearance frequency of the inclination of a straight line connecting the respective unit images configuring the pair of near dots detected by the near dot pair detection part 221. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, a pen device, and a program.

  Embed a pattern image representing information coded on a medium such as printing paper or a pattern image with a specific meaning, read it with a reading device, decode the code information, and control the device according to the given meaning It has been performed conventionally. Information represented by the pattern image includes identification information of the medium itself in which the pattern image is embedded, identification information of a document recorded on the medium, position information indicating a position on the medium, and the like.

  In this type of technology, it is necessary to detect the orientation of the pattern image in the read image in order to decode the code information described in the pattern image. In particular, it is very important to properly detect the orientation of a pattern image in a reading device such as a pen device that is held and operated by a user, because the orientation and angle of the reading device with respect to a medium change from time to time. .

  Patent Document 1 discloses a conventional technique in which, in a two-dimensional code including a code frame and a cell surrounded by the code frame, a pattern indicating the direction of the code together with data or the like is recorded in the cell.

JP 2006-85679 A

The pattern image embedded in the medium is desired to reduce redundant patterns in order to increase the information density.
The present invention has been made in view of the above problems, and an object of the present invention is to identify the direction in which unit images constituting a pattern image are arranged and to detect the direction of the pattern image. It is another object of the present invention to specify the direction in which unit images are arranged even when the read image is distorted.

The invention according to claim 1 is a unit image for detecting a unit image from an image obtained by imaging a recording medium on which unit images arranged along at least two directions intersecting at a predetermined angle are recorded by an imaging device. A combination of a detection unit, a combination detection unit that detects a combination of unit images that are closest to each other with respect to a unit image detected from the image by the unit image detection unit, and a combination of unit images detected by the combination detection unit And an angle specifying means for specifying an angle representing the inclination of the arrangement of the unit images on the basis of the appearance frequency of the inclination of the straight line connecting the unit images constituting the unit image.
According to a second aspect of the present invention, the interval specification for specifying the reference interval of the unit images in the array in which the angle is specified by the angle specifying means based on the distance between the unit images constituting the combination of the unit images. The image processing apparatus according to claim 1, further comprising means.
According to a third aspect of the present invention, the angle specifying means sets the angle having the highest appearance frequency among the inclinations of the straight lines connecting the unit images as a first angle, and only a predetermined angle with respect to the first angle. The angle having the highest appearance frequency in a certain range including the transitioned reference angle is defined as a second angle, and the first angle and the second angle are defined as angles in the two directions. The image processing apparatus described in the above.
According to a fourth aspect of the present invention, the angle specifying means sets the angle having the highest appearance frequency as a third angle in a certain range including an angle that is symmetrical to the second angle with respect to the reference angle. 4. The image processing according to claim 3, wherein two of the first angle, the second angle, and the third angle are selected based on a reference and set as the angles in the two directions. 5. Device.
The invention according to claim 5 is the information encoded and recorded by the unit image based on the unit image detected by the unit image detecting unit and the angle of the arrangement of the unit images specified by the angle specifying unit. The angle specifying means includes an angle of the arrangement of the unit images when the decoding by the decoding means is successful among the first angle, the second angle, and the third angle. The image processing apparatus according to claim 4, wherein two angles close to are selected.
According to a sixth aspect of the present invention, there is provided an imaging means for imaging a recording medium on which unit images arranged along at least two directions intersecting at a predetermined angle are recorded, and an image obtained by imaging by the imaging means. A unit image detecting unit for detecting a unit image, a combination detecting unit for detecting a combination of unit images located closest to the unit image detected from the image by the unit image detecting unit, and the combination detecting unit. A pen comprising: an angle specifying means for specifying an angle representing an inclination of the arrangement of the unit images based on an appearance frequency of a slope of a straight line connecting the unit images constituting the combination of the detected unit images; -It is a device.
The invention according to claim 7 is the interval specification that specifies the reference interval of the unit images in the array in which the angle is specified by the angle specifying unit based on the distance between the unit images constituting the combination of the unit images. The pen device according to claim 6, further comprising means.
According to an eighth aspect of the invention, a unit image is detected from an image obtained by imaging a recording medium on which a unit image arranged along at least two directions intersecting at a predetermined angle is recorded by an imaging device. A function for detecting the combination of unit images located closest to the unit image detected from the image, and the appearance of a slope of a straight line connecting the unit images constituting the combination of detected unit images And a function for specifying an angle representing the inclination of the arrangement of the unit images based on the frequency.
The invention according to claim 9 further includes a function of specifying, in the computer, a reference interval of the unit images in the array in which the angles are specified based on a distance between the unit images constituting the unit image combination. The program according to claim 8 to be realized.

According to the first aspect of the present invention, it is possible to specify the inclination of the arrangement of unit pixels based on the combination of adjacent unit pixels.
According to the second aspect of the present invention, the unit pixel reference interval can be specified along with the inclination of the unit pixel array.
According to the third aspect of the present invention, it is possible to specify the inclination in two directions in the unit pixel array.
According to the fourth aspect of the present invention, even when the arrangement of unit images is detected in three directions, the inclination in two directions in the arrangement of unit pixels can be specified.
According to the invention which concerns on Claim 5, the inclination in the arrangement | sequence of a unit pixel can be specified more appropriately based on the success or failure of decoding of information.
According to the invention of claim 6, in the pen device, the inclination of the arrangement of the unit pixels can be specified based on the combination of the adjacent unit pixels.
According to the invention of claim 7, in the pen device, the reference interval of the unit pixels can be specified together with the inclination of the arrangement of the unit pixels.
According to the eighth aspect of the present invention, in the computer that realizes the function of the image processing apparatus, the inclination of the arrangement of unit pixels can be specified based on the combination of adjacent unit pixels.
According to the ninth aspect of the present invention, in the computer that implements the function of the image processing apparatus, the unit pixel reference interval can be specified along with the inclination of the unit pixel array.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The present embodiment relates to a process of reading a pattern image printed on a medium such as paper and decoding information (code information or the like) recorded by the pattern image. As a premise of the present embodiment, first, a configuration example of a pattern image will be described.

<Configuration example of pattern image>
In the present embodiment, an information block representing specific information is arranged in the entire paper surface as a medium or in a specific area on the paper surface. An information block is a minimum unit of a pattern expressing meaningful information. This information block is composed of a set of a plurality of pattern blocks, and a unit code pattern is arranged in this pattern block. The unit code pattern represents a code for recording information and is the minimum unit of the pattern image.

FIG. 1 is a diagram showing the relationship between unit code patterns, information blocks, and paper (medium).
In FIG. 1, an information block P2 is configured by a set of unit code patterns P1. Information is recorded in a region P3 surrounded by a thick frame on the paper surface by a set of information blocks P2. Next, a specific configuration example of the unit code pattern and the information block will be described in detail.

In the present embodiment, mCn (= m! / {(Mn)! ×) by a pattern image in which unit images are arranged at n (1 ≦ n <m) locations selected from m (m ≧ 3) locations. n!}) Represents street information. This pattern image is a unit code pattern. That is, instead of associating one unit image with information, a plurality of unit images are associated with information. Assuming that one unit image is associated with information, there is a drawback in that erroneous information is expressed when the unit image is lost or noise is added. On the other hand, for example, if two unit images are associated with information, it can be easily understood that there is an error when the number of unit images is one or three. Further, in the method of expressing 1 bit or 2 bits at most in one unit image, it is not possible to express a synchronous pattern that controls reading of the information pattern with a pattern visually similar to the pattern that represents information. . For this reason, in the present embodiment, the above encoding method is adopted. Hereinafter, such an encoding method is referred to as an mCn method.
Here, the unit image may have any shape, but in the present embodiment, a dot image (hereinafter simply referred to as “dot”) is used as an example of the unit image.

FIG. 2 is a diagram illustrating an example of a unit code pattern in the mCn system.
In the example shown in FIG. 2, the black area and the hatched area are areas where dots can be arranged. The white area between the dot-placeable areas is a non-dot-placeable area that forms a gap (blank) for identifying individual dots. Of the areas where dots can be arranged, dots are arranged in black areas and dots are not arranged in hatched areas. That is, FIG. 2 shows an example in which a region where a total of 9 dots of 3 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is provided. FIG. FIG. 6B shows a unit code pattern, and FIG. 5B shows a unit code pattern in the 9C3 system in which 3 dots are arranged in a dot arrangementable area.

  By the way, the dots (black areas) arranged in FIG. 2 are only for information representation, and do not necessarily match the dots (minimum squares in FIG. 2) which are the minimum units constituting the image. In the present embodiment, when “dot” refers to the former dot and the latter dot is referred to as “pixel”, the dot has a size of 2 pixels × 2 pixels at 600 dpi. Since the size of one pixel at 600 dpi is 0.0423 mm, one side of the dot is 84.6 μm (= 0.0423 mm × 2). The larger the dot, the more likely it will be noticeable, so it is preferable to make it as small as possible. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. However, the above value 84.6 μm is a numerical value to the last, and is about 100 μm in the actually printed toner image.

FIG. 3 is a diagram comprehensively showing the types of unit code patterns shown in FIG. In this figure, a space between dots is omitted for simplification of display.
FIG. 3A shows all unit code patterns in the 9C2 system shown in FIG. In the 9C2 system, 36 (= ₉ C ₂ ) types of information are expressed using these unit code patterns. FIG. 3B shows all unit code patterns in the 9C3 system shown in FIG. In the 9C3 system, 84 (= ₉ C ₃ ) types of information are expressed using these unit code patterns.

Note that the unit code pattern is not limited to m = 9, and a value such as 4, 16 may be employed as the value of m. Any value of n may be adopted as long as 1 ≦ n <m is satisfied.
Further, as the value of m, not only a square number (the square of an integer) but also a product of two different integers may be adopted. That is, the area in which dots can be arranged is not limited to a square area as in the case of arranging 3 dots × 3 dots, but may be a rectangular area as in the case of arranging 3 dots × 4 dots. In the present specification, the rectangle means a rectangle in which the lengths of two adjacent sides are not equal.

By the way, all unit code patterns in the mCn system are assigned pattern values that are numbers for identifying each unit code pattern.
FIG. 4 shows the correspondence between unit code patterns and pattern values in the 9C2 system. Since there are 36 unit code patterns in the 9C2 system, 0 to 35 are used as pattern values. However, the correspondence shown in FIG. 4 is merely an example, and any pattern value may be assigned to any unit code pattern. In addition, although not shown in the figure for the 9C3 method, each unit code pattern is assigned a pattern value of 0 to 83.

  In this way, mCn types of unit code patterns are prepared by selecting n locations from m locations, but in this embodiment, a specific pattern is used as an information pattern among these unit code patterns, Use the rest as a synchronization pattern. Here, the information pattern is a pattern representing information to be embedded in the medium. The synchronization pattern is a pattern used for extracting an information pattern embedded in the medium. For example, it is used for specifying the position of the information pattern or detecting the rotation of the image. Any medium may be used as long as an image can be printed. Since paper is most commonly used, the medium will be described as paper in the following, but it may be metal, plastic, fiber, or the like.

Here, the synchronization pattern will be described.
FIG. 5 is an example of a synchronization pattern in the 9C2 system.
As shown in FIG. 5, when the unit code pattern is a square (the same number of vertical and horizontal dots), it is necessary to prepare four types of unit code patterns as synchronization patterns in order to detect image rotation. Here, the unit code pattern of the pattern value 32 is an upright synchronization pattern. In addition, the unit code pattern of the pattern value 33 is rotated 90 degrees to the right, the synchronization pattern of the pattern value 34 is rotated 180 degrees to the right, and the unit code pattern of the pattern value 35 is rotated 270 degrees to the right. The synchronization pattern is used. In this case, 5-bit information may be expressed by using the remainder obtained by removing these four types of synchronization patterns from 36 types of unit code patterns as an information pattern. However, the method of assigning 36 types of unit code patterns to information patterns and synchronization patterns is not limited to this. For example, five sets of synchronization patterns that constitute one set of four types may be prepared, and the remaining 16 types of information patterns may represent 4-bit information.

  On the other hand, although not particularly illustrated, when the unit code pattern is a rectangle (different in the number of vertical and horizontal dots), two types of unit code patterns may be prepared as synchronization patterns in order to detect image rotation. For example, when an area in which 3 dots in the vertical direction and 4 dots in the horizontal direction should be detected but an area in which 4 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is detected, the image is 90 degrees or 270 degrees at that time. It is because it turns out that it is rotating.

Next, a method for expressing the identification information and the coordinate information by arranging the above code patterns will be described.
First, an information block composed of unit code patterns will be described.
FIG. 6 is a diagram illustrating a configuration example of an information block.
In the example shown in FIG. 6, it is shown at the same time that the unit code pattern in the 9C2 system is used. That is, 36 types of unit code patterns are divided into, for example, 4 patterns used as synchronization patterns and 32 patterns used as information patterns, and each pattern is arranged according to a layout.

Further, in the example shown in FIG. 6, a layout in which 25 pattern blocks in which 5 × 5 unit code patterns of 3 × 3 dots can be arranged is arranged is used as the information block layout. Of the 25 pattern blocks, a synchronization pattern is arranged in one block at the upper left. In addition, an information pattern representing X coordinate information that specifies coordinates in the X direction on the paper surface is arranged in the four blocks to the right of the synchronization pattern, and the Y direction coordinates on the paper surface are specified in the four blocks below the synchronization pattern. An information pattern representing coordinate information is arranged. Furthermore, an information pattern representing the identification information of the document printed on the paper surface or the paper surface is arranged in 16 blocks surrounded by the information pattern representing the coordinate information.
The unit code pattern in the 9C2 system described above is merely an example, and the unit code pattern in the 9C3 system may be used as shown in the lower left of the figure.

Next, a layout for expressing identification information and coordinate information in the present embodiment will be described.
FIG. 7 shows a part of such a layout.
In the layout shown in FIG. 7, the information block shown in FIG. 6 is used as a basic unit, and this information block is periodically arranged on the entire sheet. In the figure, the synchronization pattern is represented by “S”, the pattern representing the X coordinate is represented by “X1”, “X2”,..., And the pattern representing the Y coordinate is represented by “Y1”, “Y2”,. The coordinate information is expressed in M series over, for example, the vertical and horizontal directions of the page. Further, although some methods can be used for encoding the identification information, RS encoding is suitable in this embodiment. This is because the RS code is a multi-level coding method, and in this case, the block representation should correspond to the multi-value of the RS code. In FIG. 7, the pattern representing the identification information is represented by “IXY” (X = 1 to 4, Y = 1 to 4).

Here, the M series used for expressing the coordinate information will be described.
The M sequence is a sequence whose partial sequence does not match other partial sequences. For example, the eleventh order M-sequence is a bit string of 2047 bits. The same sequence as the partial sequence of 11 bits or more extracted from the 2047-bit bit string does not exist in the 2047-bit bit string other than itself. In the present embodiment, 16 unit code patterns are associated with 4 bits. That is, a 2047-bit bit string is represented in decimal notation for every 4 bits, a unit code pattern is determined according to the numerical value correspondence as shown in FIG. 4, and is expressed as a coordinate code across the paper in the horizontal and vertical directions. Therefore, at the time of decoding, the position on the bit string is specified by specifying three consecutive unit code patterns.

FIG. 8 shows an example of encoding coordinate information using an M sequence.
(A) shows a bit string “0111000101011010000110010...” As an example of the 11th order M-sequence. In the present embodiment, this is divided into 4 bits, and the first partial sequence “0111” is a unit code pattern of pattern value 7 and the second partial sequence “0001” is a unit code pattern of pattern value 1. As a result, the third partial sequence “0101” is arranged as a unit code pattern with a pattern value of 5, and the fourth partial sequence “1010” is arranged as a unit code pattern of a pattern value of 10, respectively.

  As described above, when 4 bits are divided and assigned to the unit code pattern, all pattern strings can be expressed in 4 cycles as shown in FIG. That is, since the 11th order M-sequence is 2047 bits, if this sequence is cut out every 4 bits and expressed by a unit code pattern, 3 bits will be added at the end. The first 1 bit of the M sequence is added to these 3 bits to form 4 bits, which are represented by a unit code pattern. Further, the unit code pattern is cut out every 4 bits from the second bit of the M sequence. Then, the next cycle starts from the third bit of the M sequence, and the next cycle starts from the fourth bit of the M sequence. Furthermore, the 5th cycle starts from the 5th bit, which coincides with the first cycle. Therefore, if four cycles of the M sequence are cut out every 4 bits, it is possible to use all 2047 unit code patterns. Since the M sequence is 11th order, the three consecutive unit code patterns do not match the continuous code pattern at any other position. Therefore, decoding can be performed by reading three unit code patterns at the time of reading. However, in the present embodiment, coordinate information is expressed by four unit code patterns in consideration of occurrence of errors.

  By the way, when such encoding is performed, the M sequence of 4 periods is divided into 2047 blocks and stored. In the layout as shown in FIG. 7, since the blocks where the synchronization pattern is arranged are sandwiched between them, the length of 2558 blocks (≈2047 × 5/4) is covered with the M sequence of 4 cycles. become. Since the length of one side of one block is 0.508 mm (12 pixels at 600 dpi), the length of 2558 blocks is 1299.5 mm. That is, a length of 1299.5 mm is encoded. Since the long side of the A0 size is 1188.0 mm, in this embodiment, the coordinates on the A0 size paper are encoded.

The pattern image configured as described above is formed on a sheet of paper, which is a medium, using K toner (infrared light absorbing toner containing carbon) or special toner, for example, using an electrophotographic system.
Here, as the special toner, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm) is exemplified. . Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished based on whether or not an image formed on the printed paper can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.

<Configuration and Operation of Image Processing Device>
Next, an image processing apparatus that reads a pattern image printed on a paper surface (formed on a medium) will be described.
FIG. 9 is a diagram illustrating a functional configuration example of the image processing apparatus.
As illustrated in FIG. 9, the image processing apparatus 10 includes an imaging unit 100 that captures a pattern image, a dot analysis unit 200 that detects dots forming the pattern image, and a unit code pattern and an information block based on the detected dots. A decoding unit 300 for restoring and decoding the information.

  The imaging unit 100 is configured using an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and reads a pattern image formed on a paper surface as a continuous image frame. Then, the read pattern image is sent to the dot analysis unit 200 as 8-bit image data. The size of the image read by the imaging unit 100 is not particularly limited, but is 156 pixels × 156 pixels here.

  The dot analysis unit 200 is realized by a program-controlled CPU, memory, and the like, detects dots from the image captured by the imaging unit 100, and generates a two-dimensional bit string. The dot analysis unit 200 includes a position detection unit 210, an angle detection unit 220, and a synchronization unit 230 as functions for analyzing image data.

  The position detection unit 210 detects a dot position from the 8-bit image received from the imaging unit 100. Specifically, dots are detected from the acquired image frame, and position information (coordinate values) of the detected dots on the image frame is calculated. Therefore, the position detection unit 210 is a unit image detection unit that detects dots that are unit images.

  The angle detection unit 220 detects the alignment angle of dots on the image detected by the position detection unit 210. The dot alignment angle is an inclination of the dot alignment direction in the image, and is a rotation angle with respect to the original arrangement of dots. The original arrangement of dots is, for example, an arrangement of vertical and horizontal dots in a unit code pattern as shown in FIG.

The synchronization unit 230 converts the dot arrangement obtained as a detection result of the position detection unit 210 and the angle detection unit 220 into a two-dimensional bit arrangement. This conversion is called synchronization, and the arrangement of each bit in the obtained bit arrangement is called a synchronization line. The dot alignment angle detected by the angle detector 220 is the angle of the bit string (synchronization line).
The obtained bit arrangement and information on the angle and interval of the synchronization line are sent to the decoding unit 300.

  Decoding section 300 is realized by a program-controlled CPU, memory, and the like. The decoding unit 300 detects a pattern block of 3 dots × 3 dots based on the bit arrangement received from the dot analysis unit 200, the synchronization line angle and interval information. The obtained pattern block is rotationally corrected to restore the information block, and then the information recorded by the information block is decoded. Further, when the image processing apparatus 10 is a pen device that reads a pattern image while writing on a sheet on which the pattern image is formed (the detailed configuration of the pen device will be described later), the position of the pen tip and the imaging unit There is a certain deviation from the position imaged by 100. Therefore, the decoding unit 300 converts the coordinates obtained from the image into the coordinates of the pen tip based on the information on the angle and interval of the synchronization line.

  Also, as shown in FIG. 9, information about the success or failure of decoding in the decoding unit 300 is fed back to the position detection unit 210, angle detection unit 220, and synchronization unit 230 of the dot analysis unit 200. The position detection unit 210, the angle detection unit 220, and the synchronization unit 230 determine parameters to be used in each process based on the information on the success or failure of the decoding. Specifically, as a result of processing using predetermined parameters, if decoding is successful (a correct decoding result is obtained), processing is performed using the same parameters for the next image frame. I will do it. On the other hand, when decoding fails, the parameter with the highest success rate is selected and used for the next image frame with reference to the past processing history.

  When the image processing apparatus 10 is a pen device to be described later, the image processing apparatus 10 is provided with a function for recognizing whether writing with a pen is performed. The position detection unit 210, the angle detection unit 220, the synchronization unit 230, and the decoding unit 300 of the dot analysis unit 200 are supplied with a stroke start signal indicating that writing with a pen has started. When a stroke start signal is supplied, these function blocks initialize their settings. Furthermore, when the image processing apparatus 10 is a pen device, the decoding unit 300 has relative distance information (pen-tip distance information) from the pen tip to the imaging surface for correcting the pen tip from the decoding result. Supplied.

FIG. 10 is a flowchart showing the flow of processing performed by the dot analysis unit 200 and the decoding unit 300.
As an initial operation, it is assumed that the imaging unit 100 has captured a sheet of paper as a medium.
As shown in FIG. 10, first, the position detection unit 210 initializes the dot coordinate storage area in the memory (step 1001), and detects the dot position (step 1002). If the position detection is successful (Yes in Step 1003), the angle detection unit 220 then initializes the dot alignment angle storage area in the memory (Step 1004) and detects the dot alignment angle (Step 1005). . If the angle detection is successful (Yes in Step 1006), the synchronization unit 230 then initializes the bit array storage area of the memory (Step 1007) and synchronizes the dot array with the bit array (Step 1008). If the dot array synchronization is successful (Yes in Step 1009), the decoding unit 300 then initializes the decoding result storage area of the memory (Step 1010) and performs the decoding process (Step 1011). If the decoding is successful (Yes in Step 1012), the decoding unit 300 transmits a signal indicating that the decoding is successful to the position detection unit 210, the angle detection unit 220, and the synchronization unit 230 (Step 1013). The result is output (step 1014).

  When detection of the position of the dot by the position detection unit 210, detection of the dot alignment angle by the angle detection unit 220, or synchronization of the dot arrangement by the synchronization unit 230 fails (No in steps 1003, 1006, and 1009) No processing after the failed step is performed. In these cases, and when decoding by the decoding unit 300 fails (No in step 1012), the position detection unit 210, the angle detection unit 220, and the synchronization unit transmit signals indicating that the decoding unit 300 has failed in decoding. 230 (step 1015) and output as a decoding result (step 1014).

<Dot alignment angle detection method>
Here, the influence of image distortion on the arrangement of dots and an angle detection method by the angle detection unit 220 of the present embodiment will be described.
11 to 13 are diagrams illustrating examples of images captured by the imaging unit 100 (hereinafter referred to as captured images).
FIG. 11 is an example of a captured image when the dot alignment angle is 0 degree, and FIG. 12 is an example of a captured image when the dot alignment angle is 45 degrees. In these figures, an XY coordinate system orthogonal to the captured image is set, and the horizontal direction is the X axis and the vertical direction is the Y axis. The case where the dot alignment direction is along the X axis and the Y axis is defined as an angle of 0 degree.

  11 and 12, the imaging surface of the imaging unit 100 and the paper surface on which dots are formed are parallel. In the state shown in these drawings, the dots are arranged in two directions orthogonal to each other (in the drawings, the directions indicated by the solid line and the broken line are the dot alignment directions). A quadrangle formed by four adjacent dots forms a square. In these cases, it is easy to specify the alignment direction of the dots by examining the positional relationship between two adjacent dots and to detect the alignment angle with respect to the original arrangement of the dots.

  When the image processing apparatus 10 is a pen device described later, the imaging surface of the imaging unit 100 is inclined with respect to the paper surface depending on the angle of the pen device with respect to the paper surface, and the captured image is distorted. FIG. 13 is a captured image in a state in which the upper side of the image is inclined to the back and the lower side is inclined to the front with respect to FIG. In this state, since the arrangement in the two directions of the dots in the captured image is not orthogonal, a quadrangle formed by four adjacent dots is a rhombus or a parallelogram.

  When the captured image is distorted as shown in FIG. 13, in a quadrangle formed by four adjacent dots, the length of a short diagonal line approaches the length of two sides, making it difficult to detect the correct dot alignment direction. Become. In the example shown in FIG. 13, the dots are recognized as being arranged in three directions: a direction indicated by a solid line, a direction indicated by a broken line, and a direction indicated by a one-dot chain line. Therefore, the angle detection unit 220 according to the present embodiment specifies an appropriate dot alignment direction in such a case and detects the alignment angle by the method described below.

  First, the angle detection unit 220 detects adjacent dot pairs for each dot detected from the captured image by the position detection unit 210. The adjacent dot pair is composed of a dot and a dot closest to the dot when attention is paid to a predetermined dot. Next, three angles are extracted based on the detected frequency distribution of the angles of adjacent dot pairs. Then, the alignment angle of the dots is specified based on the relationship between the extracted three angles.

<Function of angle detector>
Next, the function of the angle detection unit 220 will be described in more detail.
FIG. 14 is a diagram illustrating a functional configuration of the angle detection unit 220.
The angle detection unit 220 includes a proximity dot pair detection unit 221 that detects a proximity dot pair, and a frequency distribution generation unit 222 that generates frequency distribution data of the detected proximity dot pair. Then, a first axis angle extracting unit 223, a second axis angle extracting unit 224, and a third axis angle extracting unit 225 that extract three angles based on the generated frequency distribution data, and dots from the extracted three angles And an angle specifying unit 226 for selecting and specifying the alignment angle. In the configuration shown in the figure, the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, the third axis angle extraction unit 225, and the angle specification unit 226 (the part surrounded by a broken line in the figure). Is an angle specifying means for specifying the dot alignment angle.

  The proximity dot pair detection unit 221 acquires position information (coordinate values) on the image frame of the dots detected by the position detection unit 210, and detects a proximity dot pair. In other words, the proximity dot pair detection unit 221 is means for detecting a combination of dot coordinate values that are closest to the dot coordinate value that is the output of the position detection unit 210. A detailed method for detecting the adjacent dot pair will be described later.

  The frequency distribution generation unit 222 generates frequency distribution data of the angle formed by the adjacent dot pair detected by the adjacent dot pair detection unit 221. Here, the angle formed by the adjacent dot pair is a predetermined direction (for example, in the captured image) in which a straight line connecting the two dots constituting the adjacent dot pair serves as a reference for the dot position information output from the position detection unit 210. This is an angle formed with respect to the set XY coordinate (X-axis direction).

  The first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 are candidates for dot alignment directions based on the frequency distribution data generated by the frequency distribution generation unit 222. The direction angle (first axis angle, second axis angle, third axis angle) is extracted. A detailed method of detecting each angle will be described later.

  The angle specifying unit 226 is based on the first axis angle, the second axis angle, and the third axis angle extracted by the first axis angle extracting unit 223, the second axis angle extracting unit 224, and the third axis angle extracting unit 225. The angle (alignment angle) of the dot alignment direction corresponding to the direction of the original dot arrangement is specified and output. For example, in the case of the unit code pattern as shown in FIG. 3, since the dots are arranged in two vertical and horizontal directions, the alignment angles in two directions corresponding to the two vertical and horizontal directions are specified. In FIG. 14, these two directions are described as an X axis and a Y axis. A detailed method for specifying the alignment angle will be described later.

  Further, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 are based on the distance between dots of adjacent dot pairs arranged along the extracted angle in this direction. A reference interval between dots is calculated. Then, the angle specifying unit 226 specifies a reference interval between dots at two alignment angles. In other words, the first axis angle extracting unit 223, the second axis angle extracting unit 224, the third axis angle extracting unit 225, and the angle specifying unit 226 serve as an interval specifying unit that specifies the dot interval in the dot arrangement detected from the captured image. Also works. A method for detecting the reference interval will be described later.

<Detection method of adjacent dot pairs>
Next, a method for detecting a proximity dot pair by the proximity dot pair detection unit 221 will be described in detail.
FIG. 15 is a diagram illustrating a state in which a pair of adjacent dots is detected from an image in which dots are detected. FIG. 15A shows an example of the dot position detected by the position detection unit 210, and FIG. 15B shows a state in which the proximity dot pair is detected with respect to FIG. 15A. In FIG. 15B, adjacent dot pairs are represented by connecting lines. In this case, when the dot B closest to a certain dot A forms a close dot pair with another dot C, the dot A does not form a close dot pair with any other dot (A, B, C is not particularly shown).

FIG. 16 is a diagram illustrating a specific example of a method for detecting a neighboring dot pair.
In FIG. 16, for example, it is assumed that adjacent dot pairs are detected for dots numbered 1 to 4.
The proximity dot pair detection unit 221 first recognizes the positions of dots numbered 1 to 4 detected by the position detection unit 210. Then, the distance to the dots of other numbers is scanned based on the dot of number 1. As a result, the dot closest to the number 1 dot is the number 3 dot, and the distance (PD) between them is recognized as 20. Then, the proximity dot pair detection unit 221 has information “Pair [1] = 3” indicating that the number 3 dot is a proximity dot for the number 1 dot, and the distance to the number 3 dot is 20. Information “MPD [1] = 20” is generated. Similarly, information (hereinafter, attribute information) of “Pair [3] = 1” and “MPD [3] = 20” is generated for the dot of number 3.

  Here, the number in [] of Pair [] indicates the number of its own dot, and the value defined by Pair [] = indicates the number of the partner dot that is currently a close dot pair. Further, the number in [] of MPD [] indicates the number of its own dot, and the value of MPD [] indicates the distance to the partner dot that is the adjacent dot pair at the present time.

Next, the proximity dot pair detection unit 221 scans the distance to dots other than number 1 with reference to the number 2 dot. When the dot number 1 is scanned first, it is determined that the dot number 1 is not the closest dot to the number 2 dot, so the dot number 1 is excluded from the scan target.
As a result of this scan, the dot closest to the number 2 dot is the number 3 dot, and the distance (PD) between them is recognized as 15. This distance is shorter than the dot distance (20) between number 1 and number 3. Therefore, the position detection unit 210 generates attribute information “Pair [2] = 3” and “MPD [2] = 15” for the dot of number 2 and “Pair [3] = 2” for the dot of number 3. And attribute information “MPD [3] = 15” is generated. At this time, since the dot of number 1 is deprived of the dot of number 3 by the dot of number 2, the attribute information is changed to “Pair [1] = − 1” and “MPD [1] = − 1”. , The dot of number 1 is in a state of not forming a close dot pair with any number of dots.

  Similarly, the proximity dot pair detection unit 221 scans the distance to dots other than the numbers 1 and 2 with the number 3 as a reference. As a result, the dot closest to the number 3 dot is the number 4 dot, and the distance (PD) between them is recognized as 10. This distance is shorter than the dot distance (15) between number 2 and number 3. Therefore, the position detection unit 210 generates attribute information “Pair [3] = 4” and “MPD [3] = 10” for the dot of number 3, and “Pair [4] = 3” for the dot of number 4. And “MPD [4] = 10” attribute information is generated. At this time, since the dot of number 2 is deprived of the dot of number 3 by the dot of number 4, the attribute information is changed to “Pair [2] = − 1” and “MPD [2] = − 1”. , The dot of number 2 is in a state that does not form a close dot pair with any of the dots of any number.

  At this time, the dot that has not been scanned is only the dot with the number 4, but since this dot has already formed a close dot pair with the dot with the number 3, the process is finished without being scanned again. Although the case where the number of dots is four is taken as an example here, even when the number of dots is five or more, adjacent dot pairs are sequentially detected by repeating the scanning procedure described above.

FIG. 17 is a flowchart for explaining the flow of proximity dot pair detection processing by the proximity dot pair detection unit 221.
As described with reference to FIG. 16, the proximity dot pair is detected by sequentially examining each of the plurality of dots as a reference and examining the positional relationship with other dots. This reference dot is called a reference dot, and the other dots are called target dots. As shown in FIG. 17, the proximity dot pair detection unit 221 performs the following loop processing while sequentially setting each dot as a reference dot.

  First, the proximity dot pair detection unit 221 acquires the coordinate value of the dot selected as the reference dot (step 1701), generates a dot pair number (PairIndex), and assigns it to the reference dot (step 1702). Here, the coordinate value of the reference dot is (Xs, Ys). Next, the proximity dot pair detection unit 221 executes the following loop processing while sequentially setting each dot other than the reference dot as a target dot.

  First, the proximity dot pair detection unit 221 acquires the coordinate value of the dot selected as the target dot (step 1703). Here, the coordinate value of the target dot is (Xe, Ye). Next, the proximity dot pair detection unit 221 calculates the distance ((Xs, Ys) − (Xe, Ye)) between the reference dot and the target dot (step 1704). When the calculated inter-dot distance is the minimum value among the inter-dot distances regarding the current reference dot (Yes in step 1705), dot pair information (for example, the relationship between the target dot and the reference dot) The dot number of the target dot and the distance between the reference dot and the target dot are stored (step 1706). At this time, if there is dot pair information that uses other dots as target dots, the dot pair information is deleted.

  On the other hand, when the inter-dot distance calculated in step 1704 is not the minimum value among the inter-dot distances regarding the current reference dot (No in step 1705), in other words, dot pair information regarding other dots with shorter inter-dot distances. Is already stored, the already stored dot pair information is retained as it is.

  When the loop processing of steps 1703 to 1706 is performed with all the dots other than the reference dots as the target dots, the relationship between the target dots and the reference dots is determined by setting one dot closest to the reference dot as the target dot. The dot pair information to be represented is saved. Thereafter, the proximity dot pair detection unit 221 performs processing for determining whether the reference dot and the target dot are proximity dot pairs based on the dot pair information stored in Step 1706 (Step 1707). FIG. 18 is a flowchart for explaining the details of the proximity dot pair discrimination processing shown in step 1707.

  Referring to FIG. 18, the proximity dot pair detection unit 221 first checks whether or not the target dot is linked to another dot. That is, it is checked with reference to the attribute information (Pair [] and MPD []) described with reference to FIG. 16 whether or not the target dot is already paired with another dot. If the attribute information is set for the target dot, the target dot is linked to another dot. If the attribute information is not set for the target dot, the target dot is not linked to another dot. If there is a link between the target dot and another dot (Yes in step 1801), the adjacent dot pair detection unit 221 next determines the inter-dot distance between the target dot and the reference dot, and the target dot and link. The inter-dot distance between the other dots is compared. If the inter-dot distance between the reference dot and the reference dot is shorter (No in step 1802), the proximity dot pair detection unit 221 deletes the link between the target dot and another dot (step 1803).

  When there is no link between the target dot and another dot (No in Step 1801), or after deleting the link between the target dot and another dot in Step 1803, the proximity dot pair detection unit 221 Check whether the reference dot is linked to other dots. If there is a link between the reference dot and another dot (Yes in step 1804), the adjacent dot pair detection unit 221 then determines the inter-dot distance between the reference dot and the target dot, and the reference dot and link. The inter-dot distance between the other dots is compared. If the inter-dot distance to the target dot is shorter (No in Step 1805), the proximity dot pair detection unit 221 deletes the link between the reference dot and other dots (Step 1806).

Thereafter, the proximity dot pair detection unit 221 links the reference dot and the target dot (step 1807). Thereby, the reference dot and the target dot form a close dot pair.
On the other hand, when the inter-dot distance between the target dot and the other linked dots is shorter than the inter-dot distance between the reference dot and the target dot (Yes in step 1802), or When the inter-dot distance between other linked dots is shorter (Yes in step 1805), the proximity dot pair detection unit 221 does not link the reference dot and the target dot. Therefore, these combinations do not form adjacent dot pairs.

<Method for extracting first axis angle, second axis angle, and third axis angle>
Next, a method for extracting the first axis angle, the second axis angle, and the third axis angle by the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 will be described in detail.
First, as preprocessing, the frequency distribution generation unit 222 generates frequency distribution data of angles formed by adjacent dot pairs detected by the adjacent dot pair detection unit 221. The frequency distribution data is data obtained by integrating the appearance frequency for each angle value formed by the adjacent dot pair.

FIG. 19 is a histogram that visually represents an example of the frequency distribution of angles formed by pairs of adjacent dots generated by the frequency distribution generation unit 222.
The frequency distribution data is generated in a range of 180 degrees (in the example of FIG. 19, from −90 degrees to +90 degrees). Using this frequency distribution data, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 extract the first axis angle, the second axis angle, and the third axis angle, respectively. To do.

The first axis angle extraction unit 223 extracts the angle of the maximum frequency as the first axis angle based on the frequency distribution data generated by the frequency distribution generation unit 222. The angle of the maximum frequency is obtained as follows.
The first axis angle extraction unit 223 first detects an angle having the maximum appearance frequency. Then, an average angle based on the appearance frequency is calculated in a certain range centered on the detected angle (hereinafter, target range). The calculated angle is the angle with the highest frequency and is the first axis angle. The average angle is a value obtained by adding a value obtained by multiplying the appearance frequency for each angle of the target range and dividing by the total appearance frequency (a value obtained by adding all the appearance frequencies in the target range).

For example, assuming that the angle having the highest appearance frequency is 10 degrees and the target range is a range from −2 degrees to +2 degrees, the average angle is calculated in the range of 8 degrees to 12 degrees. Also, for example, in this range, if the appearance frequency of 8 degrees is 10, the appearance frequency of 9 degrees is 20, the appearance frequency of 10 degrees is 100, the appearance frequency of 11 degrees is 50, and the appearance frequency of 12 degrees is 40, the average The angle is 10.4 degrees according to the following calculation.

(8 × 10 + 9 × 20 + 10 × 100 + 11 × 50 + 12 × 40) / (10 + 20 + 100 + 50 + 40) = 10.4

  The second axis angle extraction unit 224 assumes an angle (hereinafter referred to as a reference angle) obtained by making a transition (offset) by a certain angle from the first axis angle extracted by the first axis angle extraction unit 223, and determines the reference angle. The angle with the highest frequency is extracted as the second axis angle within a certain range as the center. Then, similarly to the calculation of the first axis angle, an average angle based on the appearance frequency is calculated in a predetermined target range, and is set as the second axis angle. Here, the transition angle formed by the first axis angle and the reference angle is set based on the original arrangement of dots. For example, the unit code pattern shown in FIG. 3 is 90 degrees. The calculation method of the maximum frequency angle is the same as in the case of the first axis angle. The fixed range including the reference angle may be arbitrarily determined, but may be determined based on whether or not the second axis angle can be detected and the history of decoding success / failure by the decoding unit 300.

  The third axis angle extraction unit 225 extracts the angle with the highest frequency as the third axis angle within a certain range centered on an angle symmetrical to the second axis angle with respect to the reference angle. Then, an average angle based on the appearance frequency is calculated in a predetermined target range, and is set as the third axis angle. For example, when the reference angle is 100 degrees and the second axis angle is 80 degrees, the angle symmetrical to the second axis angle with respect to the reference angle is 120 degrees. The fixed range including this angle may be arbitrarily determined, but may be determined based on whether or not the second axis angle can be detected and the history of decoding success / failure by the decoding unit 300. The calculation method of the maximum frequency angle is the same as in the case of the first axis angle.

FIG. 20 illustrates extraction of the first axis angle, the second axis angle, and the third axis angle by the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225. It is a flowchart explaining the flow of a process.
As shown in FIG. 20, first, the frequency distribution generation unit 222 executes the following loop processing for each adjacent dot pair sequentially.

First, the frequency distribution generation unit 222 acquires the coordinates (X1, Y1) and (X2, Y2) of the two dots of the adjacent dot pair (step 2001), and calculates the angle θ formed by the adjacent dot pair by the following equation: Calculate (step 2002). Then, 1 is added to the frequency of the frequency distribution data of the obtained angle θ (step 2003).

θ = tan ⁻¹ {(Y1-Y2) / (X1-X2)}

  Once the processing of steps 2001 to 2003 has been performed for all adjacent dot pairs, the first axis angle extraction unit 223 next extracts the first axis angle based on the generated frequency distribution data (step 2004). ). Further, a reference interval between dots along the first axis angle is calculated (step 2005). The details of the reference interval and its detection method will be described later.

  Next, the second axis angle extraction unit 224 extracts the second axis angle based on the frequency distribution data and the first axis angle (step 2006), and calculates a reference interval between dots along the second axis angle. (Step 2007). Further, the third axis angle extraction unit 225 extracts the third axis angle based on the frequency distribution data, the reference angle, and the second axis angle (step 2008), and the reference interval between dots along the third axis angle. Is calculated (step 2009).

FIG. 21 is a diagram showing an image of the extracted angles in the three directions.
Assume that the dots arranged as shown in FIG. 21A are detected and the first axis angle is extracted as shown in FIG. The second axis angle and the third axis angle are extracted with a reference angle offset by 90 degrees with respect to the first axis angle.

<Identification method of alignment angle>
Next, a method for specifying the alignment angle by the angle specifying unit 226 will be described in detail.
The angle specifying unit 226 includes a first axis angle, a second axis angle, a first axis angle extracted by the frequency distribution generation unit 222, the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225. Based on the triaxial angle, the dot alignment angle is specified and output. Specifically, the angles of the two dot alignment directions (referred to as X axis and Y axis in FIG. 14) corresponding to the original dot arrangement direction are specified. This alignment angle is specified by the following procedure.

  First, the angle specifying unit 226 acquires the first axis angle, the second axis angle, the third axis angle, and information on the success or failure of decoding by the decoding unit 300 with respect to the immediately preceding image frame (hereinafter referred to as the previous frame). When decoding for the previous frame is successful, the angle specifying unit 226 selects two of the first axis angle, the second axis angle, and the third axis angle that are close to the alignment angle specified in the previous frame as the alignment angle. . Since the previous frame and the current frame are read continuously, it is considered that the appropriate alignment angle does not change greatly. However, when the image processing apparatus 10 is a pen device, the positional relationship between the imaging unit 100 and the paper surface changes as needed. Therefore, a close angle is selected that is not the same as the alignment angle specified in the previous frame.

When decoding for the previous frame fails, the angle specifying unit 226 specifies the alignment angle in two directions as follows, for example.
If all of the first axis angle, the second axis angle, and the third axis angle have been extracted, select the first axis angle and the second axis angle or the third axis angle that is closer to the reference angle, The alignment angle.
When the third axis angle is not extracted, the angle specifying unit 226 selects the first axis angle and the second axis angle and sets it as the alignment angle. The case where the third axis angle is not extracted means that, in the frequency distribution data, when there is no angle having a high appearance frequency within a certain range centered on an angle symmetrical to the second axis angle with respect to the reference angle, This is the case when the shaft angle and the reference angle are the same.
When the second axis angle is not extracted, the angle specifying unit 226 sets the first axis angle and a reference angle offset by a certain angle with respect to the first axis angle as the alignment angle. The case where the second axis angle is not extracted is a case where, in the frequency distribution data, there is no angle having a high appearance frequency in a certain range centered on the reference angle.

  Furthermore, when decoding fails in a predetermined image frame, the angle specifying unit 226 may select an angle different from that of the image frame that has failed in decoding as the alignment angle in the next image frame. Specifically, for example, when decoding fails using the first axis angle and the second axis angle as the alignment angles, in the next image frame, the first axis angle and the third axis angle are set as the alignment angles. The axis angle and the third axis angle are set as the alignment angle, and so on.

<Calculation method of reference interval>
Next, a method for calculating a reference interval by the first axis angle extraction unit 223, the second axis angle extraction unit 224, and the third axis angle extraction unit 225 will be described.
The reference interval is an average value of inter-dot distances in adjacent dot pairs arranged along a certain direction. However, in order to eliminate the proximity dot pairs with a large inter-dot distance protruding, a certain threshold value (maximum inter-dot distance) is provided, and an average value is calculated for the proximity dot pairs whose inter-dot distance is equal to or less than this threshold value. This threshold is determined as follows, for example.

  First, the frequency distribution generation unit 222 generates frequency distribution data of the inter-dot distances of the adjacent dot pairs detected by the adjacent dot pair detection unit 221. The frequency distribution data is data obtained by integrating the appearance frequency for each value of the inter-dot distance of the adjacent dot pair. A distance slightly longer than the inter-dot distance where the appearance frequency is the maximum is set as a threshold value. FIG. 22 shows a histogram that visually represents an example of the frequency distribution of the inter-dot distance. In FIG. 22, the threshold is set to a distance slightly longer than the inter-dot distance with the maximum appearance frequency.

  The first axis angle extraction unit 223 extracts a pair of adjacent dots arranged along the first axis angle that has an inter-dot distance equal to or less than the threshold value obtained as described above. Then, the average value of the inter-dot distance is calculated and set as the reference interval of the dots in the first axis direction. In consideration of various errors, the angle formed by the adjacent dot pair includes a certain width (−α degrees to + α degrees) with respect to the first axis angle. That is, the adjacent dot pair whose angle formed by the adjacent dot pair is the first reference angle ± α is the adjacent dot pair arranged along the first axis angle.

  Similarly, the second axis angle extraction unit 224 extracts a pair of adjacent dots arranged along the second axis angle, in which the inter-dot distance is equal to or less than the threshold obtained as described above, and the second axis direction The dot reference interval is calculated. Further, the third axis angle extraction unit 225 extracts a pair of adjacent dots arranged along the third axis angle that has a dot-to-dot distance equal to or less than the threshold value obtained as described above. The reference interval is calculated.

  The angle specifying unit 226 acquires a reference interval in each direction from the first axis angle extracting unit 223, the second axis angle extracting unit 224, and the third axis angle extracting unit 225. Then, a reference interval in the direction (X axis and Y axis in FIG. 14) specified as the alignment angle is selected and output together with the alignment angle. The alignment angle and the information on the reference interval at the alignment angle are used by the synchronization unit 230 to specify a synchronization line for synchronization.

<Configuration of pen device>
Next, a specific hardware configuration of the image processing apparatus 10 in the present embodiment will be described. Here, a configuration example of a pen device that realizes the image processing apparatus 10 will be described.
FIG. 23 shows the mechanism of the pen device.
As illustrated, the pen device 60 includes a control circuit 61 that controls the operation of the entire pen. The control circuit 61 includes an image processing unit 61a that processes a code image detected from an input image, and a data processing unit 61b that extracts identification information and coordinate information from the processing result there. Although not particularly shown, the control circuit 61 includes a non-volatile memory that stores programs for operating the image processing unit 61a and the data processing unit 61b. The control circuit 61 is connected to a pressure sensor 62 that detects a writing operation by the pen device 60 by a pressure applied to the pen tip 69. Further, an infrared LED 63 that irradiates infrared light onto the medium and an infrared CMOS 64 that inputs an image are also connected. Further, an information memory 65 for storing identification information and coordinate information, a communication circuit 66 for communicating with an external device, a battery 67 for driving the pen, and pen identification information (pen ID) are stored. A pen ID memory 68 is also connected.

  In the image processing apparatus 10 illustrated in FIG. 9, the imaging unit 100 is realized by, for example, the infrared CMOS 64 in FIG. 23. The dot analysis unit 200 is realized by, for example, the image processing unit 61a in FIG. Furthermore, the decoding unit 300 is realized by, for example, the data processing unit 61b in FIG. In addition to the pen device 60 as shown in FIG. 23, the image processing apparatus 10 may realize functions realized by the image processing unit 61a or the data processing unit 61b with, for example, a general-purpose computer.

  Programs that realize the functions of the image processing unit 61a and the data processing unit 61b can be provided via communication means or stored in a recording medium such as a CD-ROM. Further, it may be provided by being stored in the above-described nonvolatile memory included in the control circuit 61.

It is a figure which shows the relationship between the unit code pattern used by this embodiment, an information block, and a paper surface (medium). It is a figure which shows the example of the unit code pattern in a mCn system. FIG. 3 is a diagram comprehensively showing types of unit code patterns shown in FIG. 2. It is a figure which shows a response | compatibility with the unit code pattern and pattern value in 9C2 system. It is a figure which shows the example of the synchronous pattern in 9C2 system. It is a figure which shows the structural example of an information block. It is a figure which shows the example of the layout for expressing the identification information and coordinate information in this embodiment. It is a figure which shows the example of the encoding of the coordinate information using M series. It is a figure which shows the function structural example of the image processing apparatus of this embodiment. It is a flowchart which shows the flow of the process by the dot analysis part and decoding part of this embodiment. It is a figure which shows the example of a captured image, and is an example of a captured image in case the alignment angle of a dot is 0 degree | times. It is a figure which shows the example of a captured image, and is an example of a captured image in case the alignment angle of a dot is 45 degree | times. It is a figure which shows the picked-up image of the state where the upper part of the image leans back and the lower part leans forward. It is a figure which shows the function structure of the angle detection part of this embodiment. It is a figure which shows a mode that the proximity | contact dot pair was detected from the image from which the dot was detected. It is a figure which shows the specific example of the detection method of the proximity | contact dot pair in this embodiment. It is a flowchart explaining the flow of the detection process of the proximity dot pair by the proximity dot pair detection part of this embodiment. FIG. 18 is a flowchart illustrating details of proximity dot pair determination processing shown in step 1707 of FIG. 17. FIG. It is the histogram which expressed visually the example of the frequency distribution of the angle which the close dot pair produced | generated by the frequency distribution production | generation part of this embodiment makes. Flow of extraction processing of the first axis angle, the second axis angle, and the third axis angle by the frequency distribution generation unit, the first axis angle extraction unit, the second axis angle extraction unit, and the third axis angle extraction unit of the present embodiment. It is a flowchart to explain. It is a figure which shows the image of the angle of 3 directions extracted by this embodiment. It is the histogram which expressed the example of frequency distribution of the distance between dots in this embodiment visually. It is the figure which showed the mechanism of the pen device which is an image processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 100 ... Imaging part, 200 ... Dot analysis part, 210 ... Position detection part, 220 ... Angle detection part, 221 ... Proximity dot pair detection part, 222 ... Frequency distribution generation part, 223 ... 1st axis angle Extraction unit, 224 ... second axis angle extraction unit, 225 ... third axis angle extraction unit, 226 ... angle identification unit, 230 ... synchronization unit, 300 ... decoding unit

Claims (9)

Unit image detecting means for detecting a unit image from an image obtained by imaging a recording medium on which a unit image arranged along at least two directions intersecting at a predetermined angle is recorded by an imaging device;
A combination detection unit that detects a combination of unit images that are closest to each other with respect to the unit image detected from the image by the unit image detection unit;
Angle specifying means for specifying an angle representing the inclination of the arrangement of the unit images based on the appearance frequency of the inclination of the straight line connecting the unit images constituting the combination of the unit images detected by the combination detecting means. An image processing apparatus.   The apparatus further comprises interval specifying means for specifying a reference interval of the unit images in the array whose angles are specified by the angle specifying means based on a distance between the unit images constituting the combination of unit images. The image processing apparatus according to claim 1.   The angle specifying means includes a reference angle obtained by setting a first angle as an angle with the highest appearance frequency among inclinations of straight lines connecting the unit images, and including a reference angle that is shifted by a predetermined angle with respect to the first angle. The image processing apparatus according to claim 1, wherein an angle having the highest appearance frequency in the range is a second angle, and the first angle and the second angle are angles in the two directions.   The angle specifying means sets the angle having the highest appearance frequency as a third angle in a certain range including an angle that is symmetrical to the second angle with respect to the reference angle, and sets the first angle based on a predetermined reference. 4. The image processing apparatus according to claim 3, wherein two of the angle, the second angle, and the third angle are selected and set as the angles in the two directions. 5. The apparatus further comprises decoding means for decoding the information encoded and recorded by the unit image based on the unit image detected by the unit image detecting means and the angle of the arrangement of the unit images specified by the angle specifying means. ,
The angle specifying means selects two angles close to the array angle of the unit images when decoding by the decoding means is successful among the first angle, the second angle, and the third angle. The image processing apparatus according to claim 4. Imaging means for imaging a recording medium on which unit images arranged along at least two directions intersecting at a predetermined angle are recorded;
Unit image detection means for detecting a unit image from an image obtained by imaging by the imaging means;
A combination detection unit that detects a combination of the unit images at the closest position with respect to the unit image detected from the image by the unit image detection unit;
Angle specifying means for specifying an angle representing the inclination of the arrangement of the unit images based on the appearance frequency of the inclination of the straight line connecting the unit images constituting the combination of the unit images detected by the combination detecting means. A pen device characterized by that.   The apparatus further comprises interval specifying means for specifying a reference interval of the unit images in the array in which the angles are specified by the angle specifying means based on a distance between the unit images constituting the combination of the unit images. The pen device according to claim 6. On the computer,
A function of detecting a unit image from an image obtained by imaging a recording medium in which unit images arranged along at least two directions intersecting at a predetermined angle are recorded by an imaging device;
A function for detecting a combination of unit images located closest to each other with respect to the unit image detected from the image;
A function for specifying an angle representing the inclination of the arrangement of the unit images based on the appearance frequency of the inclination of a straight line connecting the unit images constituting the combination of the detected unit images;
A program to realize   9. The computer according to claim 8, further comprising a function of specifying a reference interval of the unit images in the array in which the angle is specified based on a distance between the unit images constituting the combination of the unit images. program.

Images