JP2009175845A - Image processor, pen device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a pattern image by processing to reduce any labor on a device. <P>SOLUTION: This image processor is provided with: a detection part 211 for scanning an image picked up by an image pickup part, and for detecting a unit image (dot); and a noise removal part 213 for detecting and removing any noise in the unit image detected by the detection part 211. The detection part 211 scans the image by applying a filter equipped with disk elements configured of a fixed region and ring elements set in the circumference of the disk elements, and determines whether or not the pixel under consideration is a pixel forming the unit image on the basis of a difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements and the pixel value of the pixel under consideration. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, a pen device, and a program.

  Embed a pattern image representing information coded on a medium such as printing paper or a pattern image with a specific meaning, read it with a reading device, decode the code information, and control the device according to the given meaning It has been performed conventionally. Information represented by the pattern image includes identification information of the medium itself in which the pattern image is embedded, identification information of a document recorded on the medium, position information indicating a position on the medium, and the like.

In this type of technology, it is very important to correctly detect the pattern image.
Patent Documents 1 and 2 disclose a technique for binarizing an optically read image by a binarization circuit and detecting a dot code recorded in the image.
In Patent Document 1, a threshold value for binarization is determined using a histogram value for each screen or each block in the screen. That is, the threshold is determined according to the stain on the dot code, paper distortion, the accuracy of the built-in clock, and the like.
Japanese Patent Application Laid-Open No. 2004-228688 discloses determining a threshold value for binarization of the current image frame based on statistical data obtained by dot code detection processing for the previous image frame.

As a method for detecting a minute image such as a dot, a method using a quat filter is known.
Patent Document 3 discloses a technique for detecting a minute image by generating a Q filter (quant filter) from a point spread function of an optical system that captures an image and applying it to the image.

JP-A-6-231466 International Publication No. 03/001450 Pamphlet JP 2005-164446 A

  In order to detect a pattern image with an image processing apparatus such as a pen device whose processing capability is limited, it is required that the pattern image detection process correctly detects the pattern image and the processor. It is desirable that the load be as light as possible.

  The present invention has been made in view of the above problems, and an object of the present invention is to make it possible to accurately detect a pattern image by processing with a light load on the apparatus.

The invention according to claim 1 is a detection unit that scans an image captured by an imaging device and detects a unit image, and a noise removal unit that detects and removes noise from the unit image detected by the detection unit. The detection unit scans the image by applying a filter including a disk element having a certain area and a ring element provided around the disk element, and averages the pixel values of the disk element. An image processing apparatus that determines whether or not a target pixel is a pixel forming a unit image based on a difference between a value and an average value of pixel values of ring elements and a pixel value of the target pixel.
In the invention according to claim 2, the detection unit is configured such that a difference between an average value of pixel values of the disk elements and an average value of pixel values of the ring elements in the filter is equal to or more than a first threshold, and the attention is paid to The image processing apparatus according to claim 1, wherein when the pixel value of the pixel is equal to or greater than a second threshold value, the target pixel is determined to be a pixel that forms a unit image.
The invention according to claim 3 is the image processing apparatus according to claim 2, further comprising a threshold value setting unit that dynamically sets the first threshold value used in the detection unit.
According to a fourth aspect of the present invention, the noise removing unit generates a grid area that covers the image, and the disk element in the filter for each area of the image corresponding to each small area of the grid area. A contrast threshold is set based on the difference between the average value of the pixel values of the pixel and the average value of the pixel values of the ring elements, the pixel threshold is set based on the difference and the pixel value of each pixel, and is detected by the detection unit For the pixel forming the unit image, the difference obtained by applying the filter using the pixel as a target pixel is lower than the contrast threshold, or the pixel value of the pixel is lower than the pixel threshold. 2. The image processing apparatus according to claim 1, wherein the detection of the unit image by the detection unit is canceled when the detection unit is low.
According to a fifth aspect of the present invention, the noise removing unit has a predetermined difference between an average value of pixel values of the disk elements and an average value of pixel values of the ring elements in the filter based on a detection result of the detection unit. The detection of the unit image by the detection unit is canceled when the pixel value of the pixel of interest is lower than the average value of the pixel values of the ring element. This is an image processing apparatus.
The invention according to claim 6 includes an imaging unit that captures the surface of the medium on which the unit image is formed, and a detection unit that scans the image captured by the imaging unit and detects the unit image. Scans the image by applying a filter including a disk element having a certain area and a ring element provided around the disk element, and an average value of pixel values of the disk element in the filter and the ring The pen device is characterized in that, based on a difference from an average value of pixel values of elements and a pixel value of a target pixel, it is determined whether or not the target pixel is a pixel forming a unit image.
In the invention according to claim 7, the detection unit is configured such that a difference between an average value of pixel values of the disk elements and an average value of pixel values of the ring elements in the filter is equal to or more than a first threshold value, and the attention is made. The pen device according to claim 6, wherein when the pixel value of the pixel is equal to or greater than a second threshold value, the target pixel is determined to be a pixel that forms a unit image.
According to an eighth aspect of the present invention, a grid area that covers the image is generated, and an average value of pixel values of the disk elements in the filter for each area of the image corresponding to each small area of the grid area. A contrast threshold is set based on the difference between the pixel value of the ring element and the average value of the ring elements, a pixel threshold is set based on the difference and the pixel value of each pixel, and the unit image detected by the detection unit is Regarding the pixel to be formed, when the difference obtained by applying the filter with the pixel as a target pixel is lower than the contrast threshold, or when the pixel value of the pixel is lower than the pixel threshold, The pen device according to claim 6, further comprising a noise removing unit that cancels the detection of the unit image by the detecting unit.
In the invention according to claim 9, based on the detection result of the detection unit, a difference between an average value of the pixel values of the disk elements and an average value of the pixel values of the ring elements in the filter is a predetermined threshold value or more, And a noise removal unit that cancels the detection of the unit image by the detection unit when the pixel value of the pixel of interest is lower than the average value of the pixel values of the ring element. The pen device described.
According to a tenth aspect of the present invention, a function of scanning a computer with an image captured by an imaging device by applying a filter including a disk element including a certain area and a ring element provided around the disk element. And the difference between the average value of the pixel values of the disk element and the average value of the pixel values of the ring element in the filter is greater than or equal to the first threshold value, and the pixel value of the target pixel is greater than or equal to the second threshold value. In this case, it is a program for realizing a function of detecting a unit image by determining that the target pixel is a pixel forming the unit image.
According to an eleventh aspect of the present invention, in the computer, a grid area that covers the image is generated, and the pixel of the disk element in the filter is generated for each area of the image corresponding to each small area of the grid area. A contrast threshold is set based on the difference between the average value of the values and the average value of the pixel values of the ring elements, a pixel threshold is set based on the difference and the pixel value of each pixel, and the detected unit image is Regarding the pixel to be formed, when the difference obtained by applying the filter with the pixel as a target pixel is lower than the contrast threshold, or when the pixel value of the pixel is lower than the pixel threshold, The program according to claim 10, further realizing a function of canceling detection of a unit image.
According to a twelfth aspect of the present invention, a difference between an average value of pixel values of the disk element and an average value of pixel values of the ring element in the filter is determined based on a detection result of the detection unit. The function of canceling the detection of the unit image by the detection unit when the pixel value of the pixel of interest is lower than the average value of the pixel values of the ring element. It is a program.

According to the first aspect of the present invention, since the unit image is detected by calculating the average value and calculating the difference in the filter, the unit image can be detected by a process with a light load.
According to the second aspect of the invention, the unit image can be accurately detected by independently evaluating the value obtained by the filter and the pixel value of the target pixel.
According to the invention of claim 3, the first threshold value can be dynamically changed.
According to the fourth aspect of the present invention, it is possible to improve the accuracy of unit image detection by canceling the detection result of the detection unit that has a high possibility of being a noise.
According to the fifth aspect of the present invention, the accuracy of unit image detection can be improved by removing regular reflection noise.
According to the invention which concerns on Claim 6, in a pen device, a unit image can be detected by a light load process.
According to the invention which concerns on Claim 7, in a pen device, a unit image can be correctly detected by the process with a light load.
According to the invention which concerns on Claim 8, the precision of a unit image detection can be improved by removing a noise in a pen device by a light load process.
According to the ninth aspect of the present invention, in the pen device, it is possible to improve the accuracy of unit image detection by removing specular reflection noise with light load processing.
According to the tenth aspect of the present invention, in a computer that realizes the function of the image processing apparatus, a unit image can be detected by a process with a light load.
According to the eleventh aspect of the present invention, it is possible to improve the accuracy of unit image detection by removing noise in a computer that implements the function of the image processing apparatus.
According to the twelfth aspect of the present invention, in the computer that realizes the function of the image processing apparatus, the accuracy of unit image detection can be improved by removing regular reflection noise.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The present embodiment relates to a process of reading a pattern image printed on a medium such as paper and decoding information (code information or the like) recorded by the pattern image. As a premise of the present embodiment, first, a configuration example of a pattern image will be described.

<Configuration example of pattern image>
In the present embodiment, an information block representing specific information is arranged in the entire paper surface as a medium or in a specific area on the paper surface. An information block is a minimum unit of a pattern expressing meaningful information. This information block is composed of a set of a plurality of pattern blocks, and a unit code pattern is arranged in this pattern block. The unit code pattern represents a code for recording information and is the minimum unit of the pattern image.

FIG. 1 is a diagram showing the relationship between unit code patterns, information blocks, and paper (medium).
In FIG. 1, an information block P2 is configured by a set of unit code patterns P1. Information is recorded in a region P3 surrounded by a thick frame on the paper surface by a set of information blocks P2. Next, a specific configuration example of the unit code pattern and the information block will be described in detail.

In the present embodiment, mCn (= m! / {(Mn)! ×) by a pattern image in which unit images are arranged at n (1 ≦ n <m) locations selected from m (m ≧ 3) locations. n!}) Represents street information. This pattern image is a unit code pattern. That is, instead of associating one unit image with information, a plurality of unit images are associated with information. Assuming that one unit image is associated with information, there is a drawback in that erroneous information is expressed when the unit image is lost or noise is added. On the other hand, for example, if two unit images are associated with information, it can be easily understood that there is an error when the number of unit images is one or three. Further, in the method of expressing 1 bit or 2 bits at most in one unit image, it is not possible to express a synchronous pattern that controls reading of the information pattern with a pattern visually similar to the pattern that represents information. . For this reason, in the present embodiment, the above encoding method is adopted. Hereinafter, such an encoding method is referred to as an mCn method.
Here, the unit image may have any shape, but in the present embodiment, a dot image (hereinafter simply referred to as “dot”) is used as an example of the unit image.

FIG. 2 is a diagram illustrating an example of a unit code pattern in the mCn system.
In the example shown in FIG. 2, the black area and the hatched area are areas where dots can be arranged. The white area between the dot-placeable areas is a non-dot-placeable area that forms a gap (blank) for identifying individual dots. Of the areas where dots can be arranged, dots are arranged in black areas and dots are not arranged in hatched areas. That is, FIG. 2 shows an example in which a region where a total of 9 dots of 3 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is provided. FIG. FIG. 6B shows a unit code pattern, and FIG. 5B shows a unit code pattern in the 9C3 system in which 3 dots are arranged in a dot arrangementable area.

  By the way, the dots (black areas) arranged in FIG. 2 are only for information representation, and do not necessarily match the dots (minimum squares in FIG. 2) which are the minimum units constituting the image. In the present embodiment, when “dot” refers to the former dot and the latter dot is referred to as “pixel”, the dot has a size of 2 pixels × 2 pixels at 600 dpi. Since the size of one pixel at 600 dpi is 0.0423 mm, one side of the dot is 84.6 μm (= 0.0423 mm × 2). The larger the dot, the more likely it will be noticeable, so it is preferable to make it as small as possible. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. However, the above value 84.6 μm is a numerical value to the last, and is about 100 μm in the actually printed toner image.

FIG. 3 is a diagram comprehensively showing the types of unit code patterns shown in FIG. In this figure, a space between dots is omitted for simplification of display.
FIG. 3A shows all unit code patterns in the 9C2 system shown in FIG. In the 9C2 system, 36 (= ₉ C ₂ ) types of information are expressed using these unit code patterns. FIG. 3B shows all unit code patterns in the 9C3 system shown in FIG. In the 9C3 system, 84 (= ₉ C ₃ ) types of information are expressed using these unit code patterns.

Note that the unit code pattern is not limited to m = 9, and a value such as 4, 16 may be employed as the value of m. Any value of n may be adopted as long as 1 ≦ n <m is satisfied.
Further, as the value of m, not only a square number (the square of an integer) but also a product of two different integers may be adopted. That is, the area in which dots can be arranged is not limited to a square area as in the case of arranging 3 dots × 3 dots, but may be a rectangular area as in the case of arranging 3 dots × 4 dots. In the present specification, the rectangle means a rectangle in which the lengths of two adjacent sides are not equal.

By the way, all unit code patterns in the mCn system are assigned pattern values that are numbers for identifying each unit code pattern.
FIG. 4 shows the correspondence between unit code patterns and pattern values in the 9C2 system. Since there are 36 unit code patterns in the 9C2 system, 0 to 35 are used as pattern values. However, the correspondence shown in FIG. 4 is merely an example, and any pattern value may be assigned to any unit code pattern. In addition, although not shown in the figure for the 9C3 method, each unit code pattern is assigned a pattern value of 0 to 83.

  In this way, mCn types of unit code patterns are prepared by selecting n locations from m locations, but in this embodiment, a specific pattern is used as an information pattern among these unit code patterns, Use the rest as a synchronization pattern. Here, the information pattern is a pattern representing information to be embedded in the medium. The synchronization pattern is a pattern used for extracting an information pattern embedded in the medium. For example, it is used for specifying the position of the information pattern or detecting the rotation of the image. Any medium may be used as long as an image can be printed. Since paper is most commonly used, the medium will be described as paper in the following, but it may be metal, plastic, fiber, or the like.

Here, the synchronization pattern will be described.
FIG. 5 is an example of a synchronization pattern in the 9C2 system.
As shown in FIG. 5, when the unit code pattern is a square (the same number of vertical and horizontal dots), it is necessary to prepare four types of unit code patterns as synchronization patterns in order to detect image rotation. Here, the unit code pattern of the pattern value 32 is an upright synchronization pattern. In addition, the unit code pattern of the pattern value 33 is rotated 90 degrees to the right, the synchronization pattern of the pattern value 34 is rotated 180 degrees to the right, and the unit code pattern of the pattern value 35 is rotated 270 degrees to the right. The synchronization pattern is used. In this case, 5-bit information may be expressed by using the remainder obtained by removing these four types of synchronization patterns from 36 types of unit code patterns as an information pattern. However, the method of assigning 36 types of unit code patterns to information patterns and synchronization patterns is not limited to this. For example, five sets of synchronization patterns that constitute one set of four types may be prepared, and the remaining 16 types of information patterns may represent 4-bit information.

  On the other hand, although not particularly illustrated, when the unit code pattern is a rectangle (different in the number of vertical and horizontal dots), two types of unit code patterns may be prepared as synchronization patterns in order to detect image rotation. For example, when an area in which 3 dots in the vertical direction and 4 dots in the horizontal direction should be detected but an area in which 4 dots in the vertical direction and 3 dots in the horizontal direction can be arranged is detected, the image is 90 degrees or 270 degrees at that time. It is because it turns out that it is rotating.

Next, a method for expressing the identification information and the coordinate information by arranging the above code patterns will be described.
First, an information block composed of unit code patterns will be described.
FIG. 6 is a diagram illustrating a configuration example of an information block.
In the example shown in FIG. 6, it is shown at the same time that the unit code pattern in the 9C2 system is used. That is, 36 types of unit code patterns are divided into, for example, 4 patterns used as synchronization patterns and 32 patterns used as information patterns, and each pattern is arranged according to a layout.

Further, in the example shown in FIG. 6, a layout in which 25 pattern blocks in which 5 × 5 unit code patterns of 3 × 3 dots can be arranged is arranged is used as the information block layout. Of the 25 pattern blocks, a synchronization pattern is arranged in one block at the upper left. In addition, an information pattern representing X coordinate information that specifies coordinates in the X direction on the paper surface is arranged in the four blocks to the right of the synchronization pattern, and the Y direction coordinates on the paper surface are specified in the four blocks below the synchronization pattern. An information pattern representing coordinate information is arranged. Furthermore, an information pattern representing the identification information of the document printed on the paper surface or the paper surface is arranged in 16 blocks surrounded by the information pattern representing the coordinate information.
The unit code pattern in the 9C2 system described above is merely an example, and the unit code pattern in the 9C3 system may be used as shown in the lower left of the figure.

Next, a layout for expressing identification information and coordinate information in the present embodiment will be described.
FIG. 7 shows a part of such a layout.
In the layout shown in FIG. 7, the information block shown in FIG. 6 is used as a basic unit, and this information block is periodically arranged on the entire sheet. In the figure, the synchronization pattern is represented by “S”, the pattern representing the X coordinate is represented by “X1”, “X2”,..., And the pattern representing the Y coordinate is represented by “Y1”, “Y2”,. The coordinate information is expressed in M series over, for example, the vertical and horizontal directions of the page. Further, although some methods can be used for encoding the identification information, RS encoding is suitable in this embodiment. This is because the RS code is a multi-level coding method, and in this case, the block representation should correspond to the multi-value of the RS code. In FIG. 7, the pattern representing the identification information is represented by “IXY” (X = 1 to 4, Y = 1 to 4).

Here, the M series used for expressing the coordinate information will be described.
The M sequence is a sequence whose partial sequence does not match other partial sequences. For example, the eleventh order M-sequence is a bit string of 2047 bits. The same sequence as the partial sequence of 11 bits or more extracted from the 2047-bit bit string does not exist in the 2047-bit bit string other than itself. In the present embodiment, 16 unit code patterns are associated with 4 bits. That is, a 2047-bit bit string is represented in decimal notation for every 4 bits, a unit code pattern is determined according to the numerical value correspondence as shown in FIG. 4, and is expressed as a coordinate code across the paper in the horizontal and vertical directions. Therefore, at the time of decoding, the position on the bit string is specified by specifying three consecutive unit code patterns.

FIG. 8 shows an example of encoding coordinate information using an M sequence.
(A) shows a bit string “0111000101011010000110010...” As an example of the 11th order M-sequence. In the present embodiment, this is divided into 4 bits, and the first partial sequence “0111” is a unit code pattern of pattern value 7 and the second partial sequence “0001” is a unit code pattern of pattern value 1. As a result, the third partial sequence “0101” is arranged as a unit code pattern with a pattern value of 5, and the fourth partial sequence “1010” is arranged as a unit code pattern of a pattern value of 10, respectively.

  As described above, when 4 bits are divided and assigned to the unit code pattern, all pattern strings can be expressed in 4 cycles as shown in FIG. That is, since the 11th order M-sequence is 2047 bits, if this sequence is cut out every 4 bits and expressed by a unit code pattern, 3 bits will be added at the end. The first 1 bit of the M sequence is added to these 3 bits to form 4 bits, which are represented by a unit code pattern. Further, the unit code pattern is cut out every 4 bits from the second bit of the M sequence. Then, the next cycle starts from the third bit of the M sequence, and the next cycle starts from the fourth bit of the M sequence. Furthermore, the 5th cycle starts from the 5th bit, which coincides with the first cycle. Therefore, if four cycles of the M sequence are cut out every 4 bits, it is possible to use all 2047 unit code patterns. Since the M sequence is 11th order, the three consecutive unit code patterns do not match the continuous code pattern at any other position. Therefore, decoding can be performed by reading three unit code patterns at the time of reading. However, in the present embodiment, coordinate information is expressed by four unit code patterns in consideration of occurrence of errors.

  By the way, when such encoding is performed, the M sequence of 4 periods is divided into 2047 blocks and stored. In the layout as shown in FIG. 7, since the blocks where the synchronization pattern is arranged are sandwiched between them, the length of 2558 blocks (≈2047 × 5/4) is covered with the M sequence of 4 cycles. become. Since the length of one side of one block is 0.508 mm (12 pixels at 600 dpi), the length of 2558 blocks is 1299.5 mm. That is, a length of 1299.5 mm is encoded. Since the long side of the A0 size is 1188.0 mm, in this embodiment, the coordinates on the A0 size paper are encoded.

The pattern image configured as described above is formed on a sheet of paper, which is a medium, using K toner (infrared light absorbing toner containing carbon) or special toner, for example, using an electrophotographic system.
Here, as the special toner, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm) is exemplified. . Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished based on whether or not an image formed on the printed paper can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.

<Configuration and Operation of Image Processing Device>
Next, an image processing apparatus that reads a pattern image printed on a paper surface (formed on a medium) will be described.
FIG. 9 is a diagram illustrating a functional configuration example of the image processing apparatus.
As illustrated in FIG. 9, the image processing apparatus 10 includes an imaging unit 100 that captures a pattern image, a dot analysis unit 200 that detects dots forming the pattern image, and a unit code pattern and an information block based on the detected dots. A decoding unit 300 for restoring and decoding the information.

  The imaging unit 100 is configured using an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and reads a pattern image formed on a paper surface as a continuous image frame. Then, the read pattern image is sent to the dot analysis unit 200 as 8-bit image data. The size of the image read by the imaging unit 100 is not particularly limited, but is 156 pixels × 156 pixels here.

The dot analysis unit 200 is realized by a program-controlled CPU, memory, and the like, detects dots from the image captured by the imaging unit 100, and generates a two-dimensional bit string. The dot analysis unit 200 includes a position detection unit 210, an angle detection unit 220, and a synchronization unit 230 as functions for analyzing image data.
The position detection unit 210 detects a dot position from the 8-bit image received from the imaging unit 100.
The angle detection unit 220 detects the alignment angle of dots on the image detected by the position detection unit 210.
The synchronization unit 230 converts the dot arrangement obtained as a detection result of the position detection unit 210 and the angle detection unit 220 into a two-dimensional bit arrangement. This conversion is called synchronization, and the arrangement of each bit in the obtained bit arrangement is called a synchronization line. By this synchronization, an 8-bit image becomes binary data. The dot alignment angle detected by the angle detector 220 is the angle of the bit string (synchronization line).
The obtained bit arrangement and information on the angle and interval of the synchronization line are sent to the decoding unit 300.

  Decoding section 300 is realized by a program-controlled CPU, memory, and the like. The decoding unit 300 detects a pattern block of 3 dots × 3 dots based on the bit arrangement received from the dot analysis unit 200, the synchronization line angle and interval information. The obtained pattern block is rotationally corrected to restore the information block, and then the information recorded by the information block is decoded. Further, when the image processing apparatus 10 is a pen device that reads a pattern image while writing on a sheet on which the pattern image is formed (the detailed configuration of the pen device will be described later), the position of the pen tip and the imaging unit There is a certain deviation from the position imaged by 100. Therefore, the decoding unit 300 converts the coordinates obtained from the image into the coordinates of the pen tip based on the information on the angle and interval of the synchronization line.

  Also, as shown in FIG. 9, information about the success or failure of decoding in the decoding unit 300 is fed back to the position detection unit 210, angle detection unit 220, and synchronization unit 230 of the dot analysis unit 200. The position detection unit 210, the angle detection unit 220, and the synchronization unit 230 determine parameters to be used in each process based on the information on the success or failure of the decoding. Specifically, as a result of processing using predetermined parameters, if decoding is successful (a correct decoding result is obtained), processing is performed using the same parameters for the next image frame. I will do it. On the other hand, when decoding fails, the parameter with the highest success rate is selected and used for the next image frame with reference to the past processing history.

  When the image processing apparatus 10 is a pen device to be described later, the image processing apparatus 10 is provided with a function for recognizing whether writing with a pen is performed. The position detection unit 210, the angle detection unit 220, the synchronization unit 230, and the decoding unit 300 of the dot analysis unit 200 are supplied with a stroke start signal indicating that writing with a pen has started. When a stroke start signal is supplied, these function blocks initialize their settings. Furthermore, when the image processing apparatus 10 is a pen device, the decoding unit 300 has relative distance information (pen-tip distance information) from the pen tip to the imaging surface for correcting the pen tip from the decoding result. Supplied.

FIG. 10 is a flowchart showing the flow of processing performed by the dot analysis unit 200 and the decoding unit 300.
As an initial operation, it is assumed that the imaging unit 100 has captured a sheet of paper as a medium.
As shown in FIG. 10, first, the position detection unit 210 initializes the dot coordinate storage area in the memory (step 1001), and detects the dot position (step 1002). If the position detection is successful (Yes in Step 1003), the angle detection unit 220 then initializes the dot alignment angle storage area in the memory (Step 1004) and detects the dot alignment angle (Step 1005). . If the angle detection is successful (Yes in Step 1006), the synchronization unit 230 then initializes the bit array storage area of the memory (Step 1007) and synchronizes the dot array with the bit array (Step 1008). If the dot array synchronization is successful (Yes in Step 1009), the decoding unit 300 then initializes the decoding result storage area of the memory (Step 1010) and performs the decoding process (Step 1011). If the decoding is successful (Yes in Step 1012), the decoding unit 300 transmits a signal indicating that the decoding is successful to the position detection unit 210, the angle detection unit 220, and the synchronization unit 230 (Step 1013). The result is output (step 1014).

  When detection of the position of the dot by the position detection unit 210, detection of the dot alignment angle by the angle detection unit 220, or synchronization of the dot arrangement by the synchronization unit 230 fails (No in steps 1003, 1006, and 1009) No processing after the failed step is performed. In these cases, and when decoding by the decoding unit 300 fails (No in step 1012), the position detection unit 210, the angle detection unit 220, and the synchronization unit transmit signals indicating that the decoding unit 300 has failed in decoding. 230 (step 1015) and output as a decoding result (step 1014).

<Function of position detector>
Next, the function of the position detection unit 210 of the dot analysis unit 200 will be described in more detail.
FIG. 11 is a diagram illustrating a functional configuration of the position detection unit 210.
The position detection unit 210 has a function of extracting a dot image from an image captured by the imaging unit 100 and a function of calculating the coordinates of the detected dot. Specifically, as shown in FIG. 11, as a dot detection function, a detection unit 211, a threshold setting unit 212, a noise removal unit 213, a first statistical data generation unit 214, and a second statistical data generation unit 215 are provided. With. In addition, a coordinate calculation unit 216 is provided as a coordinate calculation function.

  The detection unit 211 analyzes the image captured by the imaging unit 100 and detects a dot image. The image picked up by the image pickup unit 100 is a multi-valued image, but is converted into a binary image by detecting dots by the detecting unit 211. In the present embodiment, a quit filter is used for image analysis. Details of the dot detection processing by the detection unit 211 will be described later. The threshold setting unit 212 sets a threshold used for dot detection by the detection unit 211. In other words, the threshold setting unit 212 functions as a determination criterion setting unit that sets a determination criterion for detecting dots from an image. This threshold value may be a fixed value, but may be set dynamically according to the success or failure of the decoding process.

  The noise removal unit 213 detects and removes noise from the dot detection result by the detection unit 211. The first statistical data generation unit 214 and the second statistical data generation unit 215 generate statistical data used for noise removal processing by the noise removal unit 213. Details of the noise removal processing by the noise removal unit will be described later.

  The coordinate calculation unit 216 calculates a coordinate value representing the position on the dot image (the image captured by the imaging unit 100) detected by the dot detection function. Thereby, since the positional relationship of each dot is specified, a unit code pattern (see FIGS. 2 to 4) composed of a plurality of dots is recognized, and information recorded by the unit code pattern can be decoded. .

<Quit filter and dot detection processing>
Next, a quantit filter used for dot detection and coordinate calculation according to the present embodiment will be described.
The quat filter is a kind of Morphological filter, and is composed of a disk element and a ring element provided around the disk element. The difference between the calculation results of the disk element and the ring element becomes the output value of the filter process. The detection unit 211 scans the image captured by the image capturing unit 100 using this quantile filter, and evaluates the difference (height difference) between the pixel value of the disk element and the pixel value of the ring element. Judgment is made.

FIG. 12 is a diagram for explaining the concept of dot detection using a quantile filter.
As shown in FIG. 12, the image is scanned on the image by pressing the quantile filter from above. In FIG. 12, the pixel value in the image is expressed as a three-dimensional height. At this time, as shown in FIG. 12A, the pixel values of the disk element D and the ring element R are substantially constant (that is, the height difference) in a place where the pixel values are approximated in a region exceeding the disk element D. Is small). On the other hand, as shown in FIG. 12B, when the pixel value of the small area included in the disk element D protrudes compared to the surrounding pixel values, if the position of this small area and the disk element D coincides, Only the element R falls (that is, a large height difference occurs).

  In the image imaged by the imaging unit 100, the dot has a higher pixel value than the surrounding area. For this reason, when the positions of the disk element D and the dot coincide with each other, the difference in level between the pixel value of the disk element D and the pixel value of the ring element R, which is the output of the quoting filter, is large, and the pixel value of the disk element D is high. Therefore, the detection unit 211 applies a quoting filter to the pixel value obtained by scanning the image, and the difference in height between the pixel value of the disk element D and the pixel value of the ring element R is large, and the pixel value of the disk element D It is determined that a dot has been detected when an output (detection result) such that is high is obtained.

FIG. 13 is a diagram illustrating a configuration example of a quantile filter used in the present embodiment. FIG. 13A shows the disk element D, FIG. 13B shows the ring element R, and FIG. 13C shows an example of the data structure of the output of the detection unit 211.
As shown in FIG. 13A, the disk element D is a square area of 3 pixels × 3 pixels. From this disk element D, an average value (DiskAvg) and a maximum value (DiskMax) of the pixel values of nine pixels are acquired. As shown in FIG. 13B, the ring element R has 28 pixels (excluding the center 3 pixels × 3 pixels and the four corners 3 pixels (12 pixels in total) in the 7 pixel × 7 pixel square range). = 49-9-12). The center 3 pixels × 3 pixels correspond to the disk element D. From this ring element R, an average value (RingAvg) of 28 pixel values is acquired. In addition, the center pixel of the disk element D (indicated by a bold line in the figure) is a pixel of interest in the process using the quantile filter.

  As shown in FIG. 13C, the output of the detection unit 211 is 8-bit data. Of these, the difference filter (QuitDiff) between the average value (DiskAvg) of the disk element D and the average value (RingAvg) of the ring element R, which is the output of the quantile filter, is recorded from the least significant bit (LSB) to the seventh bit. To do. The most significant bit (MSB) is used as a tag (hereinafter referred to as a dot tag) indicating that the target pixel is determined to be a dot.

FIG. 14 is a diagram illustrating a state in which an image is scanned using the quantit filter illustrated in FIG.
The detection unit 211 detects dots while scanning from the upper left corner to the lower right corner of the image by aligning the center of the quantile filter with the target pixel. Dot detection is performed by a two-step process.

  First, a difference (QuitDiff) between the average values of the disk element and the ring element is calculated, and the obtained value is compared with the first threshold value. Here, a case where the average value of the disk elements is high is taken as a positive value. If the difference value is larger than the first threshold value, the value is recorded in the output data. The first threshold is set by the threshold setting unit 212. Details of the first threshold value setting method by the threshold value setting unit 212 will be described later.

  If the difference value for the target pixel is equal to or greater than the first threshold value, the pixel value of the target pixel is then compared with the second threshold value. If the pixel value of the target pixel is equal to or greater than the second threshold, the value of the bit assigned to the dot tag is a value indicating that it is a dot. The second threshold value is arbitrarily set, and may be a fixed value or a variable value. In the case of a fixed value, for example, the maximum value (DiskMax) in the disk element of the quantile filter may be used. In the case of a variable value, for example, a predetermined parameter may be subtracted from the maximum value (DiskMax) of the disk element. The parameter is set, for example, as R% (R is an arbitrary value) of the difference (QuitDiff) between the disk element and the ring element.

  In addition, when the difference value regarding the target pixel is equal to or less than the first threshold, the detection unit 211 calculates an inversion level (RevDiff) for the target pixel. Here, the inversion level is a difference obtained by subtracting the pixel value of the target pixel from the average value (RingAvg) of the ring elements. Therefore, the inversion level value is positive when the pixel value of the target pixel is lower. When the obtained inversion level is greater than the third threshold (that is, when the pixel value of the target pixel is lower than the average value of the ring element), the detection unit 211 causes regular reflection to occur in the target pixel. Judge that

FIG. 15 is a flowchart for explaining dot detection processing by the detection unit 211.
Prior to the processing by the detection unit 211 for the image frame, first, the threshold value setting unit 212 sets a first threshold value (TH (1)) (step 1501). A method for setting the first threshold will be described later (see FIG. 17). After the first threshold (TH (1)) is set, the detection unit 211 scans the image (image frame) captured by the imaging unit 100 by applying a quantile filter and performs the following loop processing. Is repeated over the entire image for each pixel.

  First, the detection unit 211 cuts out an area of 7 pixels × 7 pixels from the image, and acquires a pixel value (step 1502). The size of 7 pixels × 7 pixels is a size that can accommodate the quantile filter shown in FIG. The detection unit 211 applies a quoting filter to the acquired pixel value, and calculates an average value (DiskAvg) and a maximum value (DiskMax) of the pixel values corresponding to the disk elements (step 1503). In addition, the detection unit 211 calculates an average value (RingAvg) of pixel values of pixels corresponding to the ring element of the quantile filter (Step 1504). Then, the detection unit 211 calculates a difference (QuitDiff = DiskAvg−RingAvg) between the average value of the pixel values of the disk elements of the quoting filter and the average value of the pixel values of the ring elements (Step 1505). And the first threshold value (TH (1)).

  When the difference value (QuitDiff) is larger than the first threshold value (TH (1)) (Yes in Step 1506), the detection unit 211 detects the lower 7 bits of the 8-bit output data shown in FIG. Is used to record the difference value (QuitDiff) (step 1507). Next, the detection unit 211 compares the pixel value V of the target pixel located at the center of the quantile filter with the second threshold value (TH (2)) from the pixel value acquired in Step 1502. When the second threshold value (TH (2)) is a variable value determined based on the difference value (QuitDiff), it is calculated after the determination in step 1506.

  When the pixel value of the target pixel is equal to or greater than the second threshold (TH (2)) (Yes in Step 1508), the detection unit 211 detects the most significant bit (8 bits of output data illustrated in FIG. 13C). The dot tag is set to a value indicating that it is a dot (step 1509). Then, the 8-bit output data is transmitted to the first statistical data generation unit 214 (step 1510). The first statistical data generation unit 214 generates statistical data to be described later according to the acquired output data. If the pixel value of the pixel of interest is smaller than the second threshold value (TH (2)) (No in step 1508), it is determined that this pixel does not form a dot. It is not transmitted to the first statistical data generation unit 214.

  On the other hand, when the difference value (QuitDiff) is equal to or less than the first threshold value (TH (1)) (No in step 1506), the detection unit 211 determines the inversion level (RevDiff = RingAvg−the pixel value of the target pixel) in the target pixel. V) is calculated (step 1511). If the obtained value is larger than the third threshold (TH (3)), it is determined as regular reflection noise, and the second statistical data generation unit 215 is notified that this noise has been detected (step 1512). 1513). The second statistical data generation unit 215 generates statistical data to be described later in accordance with the acquired regular reflection noise detection notification.

  As described above, after dot detection is performed on the image frame and statistical data is generated, the process proceeds to noise removal processing by the noise removal unit 213 (step 1514). Details of the noise removal processing will be described later (see FIG. 22).

<First threshold setting method>
Here, the setting method of the 1st threshold value by the threshold value setting part 212 is demonstrated in detail.
Referring to FIG. 11, the threshold setting unit 212 is supplied with a decoding result by the decoding unit 300 and a stroke start signal. The threshold value setting unit 212 receives information on the success or failure of decoding included in the decoding result for each image frame, and dynamically sets the first threshold value described above based on the accumulated decoding history. Moreover, the threshold value setting part 212 initializes a decoding log | history, when a stroke start signal is received.

  When the image processing apparatus 10 is a pen device, which will be described later, it is assumed that the state of the image varies greatly depending on the scanning position and the scanning state when the image processing apparatus 10 scans the paper surface that is a medium. Therefore, in the present embodiment, the first threshold value used for dot detection is dynamically changed based on the success or failure of actual decoding to cope with such a change in the image state. Since the decoding history is initialized by the stroke start signal, the dynamic setting of the first threshold value is individually performed for each writing stroke using the pen device.

  Regarding the threshold value set by the threshold value setting unit 212, an upper limit (setting upper limit), a lower limit (setting lower limit), and an initial value are determined in advance. When the decoding for the immediately preceding image frame is successful, the threshold setting unit 212 uses the value used for dot detection from the immediately preceding image frame as it is as the first threshold used for dot detection from the current image frame. (Ie, do not change the threshold). On the other hand, when the decoding for the immediately preceding image frame fails, the first threshold used for dot detection from the current image frame is set to a value different from the immediately preceding image frame based on the decoding history.

  The threshold setting unit 212 generates, as a decoding history, frequency distribution data related to the successful decoding frequency with respect to the threshold based on the acquired decoding result. FIG. 16 is a histogram that visually represents an example of the frequency distribution. Here, the successful decoding frequency is not simply the number of successful decodings, but a value obtained by subtracting the number of successful decodings from the number of successful decodings at a predetermined threshold (however, the minimum value is 0).

  Referring to FIG. 16, the threshold setting unit 212 first sets the first threshold to an initial value. Then, the first threshold value is sequentially lowered for each image frame until decoding is successful. If the decoding is successful, 1 is added to the frequency of the success frequency at the first threshold value at that time. When decoding fails at a predetermined first threshold value where the frequency of success frequency is not 0, 1 is subtracted from the frequency of success frequency at the first threshold value.

  When the frequency distribution data is empty (for example, when decoding has never been successful since the start of the stroke), if the first threshold is lowered and the set lower limit is reached, then the next set upper limit value is set. Similarly, it repeats until decoding succeeds. On the other hand, if there is a first threshold whose success frequency is not 0, if the decoding for the previous image frame fails, the success frequency is used as the first threshold used for dot detection from the current image frame. Use the value where is the maximum. In addition, when decoding fails in an image frame in which dot detection has been performed using a value with the highest frequency of success frequency as the first threshold, as a first threshold used for dot detection from the next image frame, A value smaller by one than the value having the highest frequency of success frequency is used.

FIG. 17 is a flowchart for explaining the first threshold value setting operation by the threshold value setting unit 212, and shows the detailed contents of the processing in step 1501 shown in FIG.
When the image frame to be processed by the detection unit 211 is the head image frame in the writing stroke by the pen device (Yes in Step 1701), the threshold setting unit 212 performs frequency distribution data (decoding history) regarding the decoding success frequency. ) Is initialized (step 1702). Then, the first threshold value is set to an initial value (step 1703). Whether or not the image frame is the first image frame of the stroke is identified by whether or not a stroke start signal is received.

  When the image frame to be processed by the detection unit 211 is not the first image frame in the writing stroke by the pen device (No in step 1701), the threshold setting unit 212 is used for dot detection from the immediately preceding image frame. An index in the frequency distribution data of the first threshold (THprev) is acquired (step 1704). Then, based on the decoding result fed back from the decoding unit 300, it is checked whether or not the decoding for the immediately preceding image frame has succeeded.

  When decoding of the immediately preceding image frame is successful (Yes in Step 1705), the threshold setting unit 212 adds 1 to the frequency of the frequency distribution data of the acquired index (Step 1706). In this case, the first threshold value used for dot detection from the image frame is not changed, and the first threshold value used for dot detection from the immediately preceding image frame is used as it is.

  On the other hand, if the decoding of the previous image frame has failed (No in step 1705), the first threshold is set based on the frequency distribution data. The threshold setting unit 212 first checks whether the frequency of the frequency distribution data of the acquired index is 0 (step 1707), and if not, subtracts 1 from the frequency (step 1708). If the frequency is 0 in step 1707, the minimum value of the frequency is set to 0, and no subtraction is performed.

  Next, the threshold value setting unit 212 acquires the index (Idx_Max) having the highest frequency in the frequency distribution data (step 1709). If the frequency distribution data is not empty (No in step 1710), the maximum frequency index (Idx_Max) is obtained, and the threshold value (THmax) of the index is obtained (step 1711). Then, the threshold value (THmax) is compared with the first threshold value (THprev) used for dot detection from the immediately preceding image frame obtained in step 1704.

  When THmax ≠ THprev (No in step 1712), the threshold setting unit 212 sets the threshold (THmax) of the maximum frequency index to the first threshold used for dot detection from the current image frame (step 1713). . On the other hand, if THmax = THprev (Yes in step 1712), then the threshold setting unit 212 checks whether THmax is a set lower limit (lower limit). If THmax is not the setting lower limit (No in step 1714), a value (THmax-1) that is one smaller than THmax is set as the first threshold used for dot detection from the current image frame (step 1715). ). On the other hand, if THmax is the setting lower limit (Yes in step 1714), the threshold value (THmax) of the maximum frequency index is set to the first threshold value used for dot detection from the current image frame (step 1713).

  If the decoding has not succeeded since the start of the stroke and the frequency distribution data is empty (Yes in step 1710), the threshold setting unit 212 then uses the first used for dot detection from the previous image frame. The threshold value (THprev) of 1 is compared with the set lower limit (lower limit value). If THprev is larger than the set lower limit (Yes in step 1716), a value (THdet) obtained by lowering THprev by a certain value is calculated (step 1717). If the calculated value (THdet) is smaller than the set lower limit, THdet is set to the set lower limit (steps 1718 and 1719). Then, the obtained value (THdet) is set as a first threshold used for dot detection from the current image frame (step 1720). When THprev is the setting lower limit (No in step 1716), the threshold setting unit 212 sets the setting upper limit value (upper limit) as the first threshold used for dot detection from the current image frame. (Step 1721).

  In this embodiment, since a quantit filter is used for dot detection, a first for evaluating the difference between the average value of the pixel values in the disk element of the quantile filter and the average value of the pixel values in the ring element. Is set by the threshold setting unit 212. However, regardless of whether or not a quantile filter is used, in general, as a technique for detecting a specific dot from an image, a pixel corresponding to a dot or a pixel between a certain region including the pixel and another part of the image is used. Evaluation is performed by comparing a difference between values with a predetermined threshold value. Even in such other methods, it is necessary to cope with fluctuations in the image state caused by the scanning position and the scanning state. Therefore, the method of dynamically changing the threshold value used for dot detection based on the success or failure of actual decoding may be applied to other various dot detection methods.

<Noise removal processing>
In the present embodiment, normal noise removal processing and regular reflection removal processing are performed as noise removal processing. Regular reflection is a phenomenon in which, when an image is read, an image formed on a paper surface with toner or the like totally reflects irradiated light, and the image is close to a saturation level in the imaging unit 100. Total reflection occurs in units of one dot.

First, normal noise removal will be described.
In this process, a grid-like region (5 × 5 matrix) in which small regions (partial regions) each having a width of 32 pixels are arranged by 25 × 5 × 5 regions is set for the image captured by the imaging unit 100. The Further, this 5 × 5 matrix is converted into a grid-like region (10 × 10 matrix) in which 10 × 10 100 regions are arranged. A threshold is set for each partial region of the 10 × 10 matrix, and noise is detected and removed. In the present embodiment, since the size of the image captured by the imaging unit 100 is 156 pixels × 156 pixels as described above, the entire image is covered with the 10 × 10 matrix generated as described above.

  First, the first statistical data generation unit 214 sets the 5 × 5 matrix, generates three types of statistical data, and registers them in each partial region of the 5 × 5 matrix. The statistical data includes the integrated value of the contrast, the maximum value of the disk element (the maximum value of the pixel values of 9 pixels), and the number of detected dots. Here, the contrast is a difference (QuitDiff) between a disk element and a ring element by a quantile filter. The generation of the statistical data may be performed in parallel with the detection process by the detection unit 211 by sequentially receiving the output of the detection unit 211.

FIG. 18 is a diagram illustrating an example of a 5 × 5 matrix and statistical data.
The first statistical data generation unit 214 integrates the value of the difference (QuitDiff) obtained for each pixel included in the partial area for each partial area constituting the 5 × 5 matrix, and generates statistical statistical data (FIG. 18). Of Sc00 to Sc44). Further, for each partial area constituting the 5 × 5 matrix, the maximum value (DiskMax) of the disk element of the quart filter for each pixel included in the partial area is selected, and the statistical data of the maximum value (FIG. 18). Of Max00 to Max44). Further, the number of detected dots is counted for each partial area constituting the 5 × 5 matrix, and statistical data (Cnt00 to Cnt44 in FIG. 18) of the detected dot number is generated. Here, the number of detected dots is the number of output data of the quoting filter relating to each pixel included in the partial area, the value indicating that the dot tag is a dot. The generated statistical data is sent to the noise removing unit 213.

  The noise removal unit 213 uses the statistical data acquired from the first statistical data generation unit 214 to generate a contrast threshold value and a pixel value threshold value. Then, the original image (image captured by the image capturing unit 100) at the position detected as a dot by the detection unit 211 is compared with these threshold values. When the pixel value of the original image is lower than the threshold, or when the contrast (QuitDiff) when the pixel in the original image is the target pixel is lower than the threshold, the noise removing unit 213 determines that the detected dot is noise. Then, the detection of the dots by the detection unit 211 is canceled. Thereby, noise is removed.

Here, a method of generating threshold values for contrast and pixel values will be described in detail.
First, the noise removal unit 213 obtains the average value and the maximum value of the contrast for each partial region constituting the 5 × 5 matrix. The average value is obtained by dividing the integrated value of contrast acquired from the first statistical data generation unit 214 by the number of detected dots. When the number of detected dots is 0 for a predetermined partial area, linear interpolation is performed using the average value of adjacent partial areas.

  Next, the noise removing unit 213 divides each partial region of the 5 × 5 matrix into four to form a 10 × 10 matrix composed of 10 × 10 100 regions. The statistical data acquired from the first statistical data generation unit 214 and the average value and maximum value of the contrast obtained by the noise removal unit 213 are converted into data for each of the four small areas. This data can be obtained, for example, by linearly interpolating partial area data in a 5 × 5 matrix between adjacent partial areas.

FIG. 19 is a diagram showing a state of partial area division.
As shown in FIG. 19, it is assumed that the small areas A-1, A-2, A-3, and A-4 are obtained by dividing the partial area A in the 5 × 5 matrix. At this time, the data in the upper left small area A-1 is the average value of the data in the original partial area A, the left adjacent partial area B, and the upper adjacent partial area C. Similarly, the data of the upper right small area A-2 is the average value of the data of the original partial area A, the upper adjacent partial area C, and the right adjacent partial area D. The data of the lower left small area A-3 is obtained by linear interpolation based on the original partial area A, the left adjacent partial area B, and the lower adjacent partial area E data. The data in the lower right small area A-4 is the average value of the data in the original partial area A, the right adjacent partial area D, and the lower adjacent partial area E. A specific method of linear interpolation is not limited to the above method.

  Next, the noise removing unit 213 calculates a contrast threshold value of the 10 × 10 matrix. FIG. 20A shows an example of the contrast threshold (Ctr00 to Ctr99). The contrast threshold (Ctr) is determined to be, for example, 25% of the maximum value by using the maximum value of the contrast obtained for each small region of the 10 × 10 matrix. By determining based on the maximum contrast value, the contrast threshold value varies according to the sharpness of the entire image captured by the imaging unit 100.

In addition, the noise removing unit 213 calculates a pixel threshold value of a 10 × 10 matrix. FIG. 20B shows an example of the pixel threshold (Px00 to Px99). For example, the pixel threshold (Px) is obtained by using the average value Ct of the contrast and the maximum value Mx of the pixel value of the disk element respectively converted into data of a small area of a 10 × 10 matrix.

Px = (Mx−Ct) / k (where k is a value of about 2 to 4)

And so on.

Next, removal of noise due to regular reflection (hereinafter, regular reflection noise) will be described.
In this process, a grid-like region (39 × 39 matrix) in which 39 × 39 1521 regions of a small region (partial region) having a width of 4 pixels are arranged for the image captured by the image capturing unit 100 is set. The Then, regular reflection noise is detected and removed for each partial region of the 39 × 39 matrix. In the present embodiment, since the size of the image captured by the imaging unit 100 is 156 pixels × 156 pixels as described above, the entire image is covered with the 39 × 39 matrix generated as described above.

  First, the second statistical data generation unit 215 sets the 39 × 39 matrix. Then, for each partial region of the 39 × 39 matrix, the number of pixels (hereinafter referred to as density inversion pixels) determined to be regular reflection in the detection unit 211 is counted. This process may be performed in parallel with the detection process by the detection unit 211 by sequentially receiving the output of the detection unit 211. The obtained data is sent to the noise removal unit 213.

FIG. 21 is a diagram illustrating an example of a 39 × 39 matrix.
The noise removing unit 213 sequentially pays attention to each partial area of the 39 × 39 matrix, and when there is a density inversion pixel in any of the focused partial area and eight partial areas adjacent to this partial area, The detection of the dots by the detection unit 211 in the partial area is canceled. For example, in FIG. 21, when attention is paid to a gray partial area, if there is a density inversion pixel in any of the nine partial areas surrounded by a thick frame, the dot in the hatched partial area is displayed. Detection is canceled. Thereby, regular reflection noise is removed.

FIG. 22 is a flowchart for explaining the operation of noise removal by the noise removing unit 213, and shows the detailed contents of the processing of step 1514 shown in FIG.
First, the noise removal unit 213 calculates the average value and the maximum value of the contrast (QuitDiff) in each partial region of the 5 × 5 matrix based on the statistical data acquired from the first statistical data generation unit 214 (Step 2201, 2202). Next, the noise removing unit 213 converts the obtained data into 10 × 10 matrix data (step 2203). Then, a contrast threshold value is calculated (step 2204).

  Further, the noise removing unit 213 converts the statistical data of the maximum value of the disk element in the quantile filter acquired from the first statistical data generating unit 214 into 10 × 10 matrix data (step 2205). Then, a pixel threshold value is calculated (step 2206).

  Thereafter, the noise removing unit 213 repeats the following loop processing for each pixel over the entire image while scanning the image (image frame). However, since it is a process of removing noise from the detection result of dots by the detection unit 211, in actuality, the process is performed only for pixels whose dot tag in the output of the detection unit 211 has a value indicating that it is a dot. Is called. A pixel that is a processing target in each loop is referred to as a target pixel.

  The noise removing unit 213 first obtains the pixel value of the target pixel and the contrast (QuitDiff) when the target pixel is the target pixel from the result of dot detection by the detection unit 211 (step 2207). When the contrast value is smaller than the contrast threshold, or when the pixel value is smaller than the pixel threshold, the noise removing unit 213 changes the dot tag value in the target pixel, and the detection unit 211 detects the dot. Cancel (steps 2208, 2209, 2210).

  Next, on the basis of the statistical data acquired from the second statistical data generation unit 215, the noise removal unit 213 relates to the partial region of the 39 × 39 matrix including the target pixel and eight surrounding partial regions ( It is checked whether or not density inversion pixels exist in any of a total of nine partial areas). If there is a density inversion pixel, the noise removal unit 213 changes the value of the dot tag in the target pixel and cancels the dot detection by the detection unit 211 (steps 2211 and 2122).

  In this embodiment, statistical data is generated using 5 × 5 matrix and 39 × 39 matrix and used for noise removal. However, the size of each matrix and the size of the partial area in each matrix are the same as those in the above embodiment. It is not limited. An appropriate size may be selected according to the size and resolution of the original image captured by the imaging unit 100, the processing capability of the detection unit 210, and the like.

<Configuration of pen device>
Next, a specific hardware configuration of the image processing apparatus 10 in the present embodiment will be described. Here, a configuration example of a pen device that realizes the image processing apparatus 10 will be described.
FIG. 23 shows the mechanism of the pen device.
As illustrated, the pen device 60 includes a control circuit 61 that controls the operation of the entire pen. The control circuit 61 includes an image processing unit 61a that processes a code image detected from an input image, and a data processing unit 61b that extracts identification information and coordinate information from the processing result there. Although not particularly shown, the control circuit 61 includes a non-volatile memory that stores programs for operating the image processing unit 61a and the data processing unit 61b. The control circuit 61 is connected to a pressure sensor 62 that detects a writing operation by the pen device 60 by a pressure applied to the pen tip 69. Further, an infrared LED 63 that irradiates infrared light onto the medium and an infrared CMOS 64 that inputs an image are also connected. Further, an information memory 65 for storing identification information and coordinate information, a communication circuit 66 for communicating with an external device, a battery 67 for driving the pen, and pen identification information (pen ID) are stored. A pen ID memory 68 is also connected.

  In the image processing apparatus 10 illustrated in FIG. 9, the imaging unit 100 is realized by, for example, the infrared CMOS 64 in FIG. 23. The dot analysis unit 200 is realized by, for example, the image processing unit 61a in FIG. Furthermore, the decoding unit 300 is realized by, for example, the data processing unit 61b in FIG. In addition to the pen device 60 as shown in FIG. 23, the image processing apparatus 10 may realize functions realized by the image processing unit 61a or the data processing unit 61b with, for example, a general-purpose computer.

  Programs that realize the functions of the image processing unit 61a and the data processing unit 61b can be provided via communication means or stored in a recording medium such as a CD-ROM. Further, it may be provided by being stored in the above-described nonvolatile memory included in the control circuit 61.

It is a figure which shows the relationship between the unit code pattern used by this embodiment, an information block, and a paper surface (medium). It is a figure which shows the example of the unit code pattern in a mCn system. FIG. 3 is a diagram comprehensively showing types of unit code patterns shown in FIG. 2. It is a figure which shows a response | compatibility with the unit code pattern and pattern value in 9C2 system. It is a figure which shows the example of the synchronous pattern in 9C2 system. It is a figure which shows the structural example of an information block. It is a figure which shows the example of the layout for expressing the identification information and coordinate information in this embodiment. It is a figure which shows the example of the encoding of the coordinate information using M series. It is a figure which shows the function structural example of the image processing apparatus of this embodiment. It is a flowchart which shows the flow of the process by the dot analysis part and decoding part of this embodiment. It is a figure which shows the function structure of the position detection part of this embodiment. It is a figure explaining the concept of the dot detection using the quantum filter in this embodiment. It is a figure which shows the structural example of the quantile filter used by this embodiment. It is a figure which shows a mode that an image is scanned using the quantit filter shown in FIG. It is a flowchart explaining the dot detection process by the detection part of this embodiment. It is the histogram which expressed the example of the frequency distribution regarding the success frequency of decoding visually. It is a flowchart explaining the setting operation | movement of the 1st threshold value by the threshold value setting part of this embodiment. It is a figure which shows the example of 5 * 5 matrix and statistical data which are used by this embodiment. It is a figure which shows the mode of the division | segmentation of a partial area. It is a figure which shows the example of the contrast threshold value and pixel threshold value which are used by this embodiment. It is a figure which shows the example of the 39 * 39 matrix and statistical data which are used by this embodiment. It is a flowchart explaining the operation | movement of the noise removal by the noise removal part of this embodiment. It is the figure which showed the mechanism of the pen device which is an image processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 100 ... Imaging part, 200 ... Dot analysis part, 210 ... Position detection part, 211 ... Detection part, 212 ... Threshold setting part, 213 ... Noise removal part, 214 ... First statistical data generation part, 215 ... 2nd statistical data generation part, 216 ... Coordinate calculation part, 220 ... Angle detection part, 230 ... Synchronization part, 300 ... Decoding part

Claims (12)

A detection unit that scans an image captured by the imaging device and detects a unit image;
Among the unit images detected by the detection unit, a noise removing unit that detects and removes noise,
The detector is
The image is scanned by applying a filter including a disk element having a certain area and a ring element provided around the disk element, and an average value of pixel values of the disk element and an average value of pixel values of the ring element An image processing apparatus that determines whether or not a target pixel is a pixel that forms a unit image based on a difference from the value and a pixel value of the target pixel.   In the detection unit, a difference between an average value of pixel values of the disk element and an average value of pixel values of the ring element in the filter is equal to or greater than a first threshold value, and a pixel value of the target pixel is a second value The image processing apparatus according to claim 1, wherein the pixel of interest is determined to be a pixel that forms a unit image when the threshold is equal to or greater than a threshold value.   The image processing apparatus according to claim 2, further comprising a threshold setting unit that dynamically sets the first threshold used in the detection unit. The noise removing unit
Generating a grid area covering the image;
For each area of the image corresponding to each small area of the grid area, a contrast threshold is set based on the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter And setting a pixel threshold based on the difference and the pixel value of each pixel,
Regarding the pixel forming the unit image detected by the detection unit, the difference obtained by applying the filter using the pixel as a target pixel is lower than the contrast threshold, or the pixel value of the pixel The image processing apparatus according to claim 1, wherein the detection of the unit image by the detection unit is canceled when is lower than the pixel threshold.   The noise removing unit is based on a detection result of the detection unit, and a difference between an average value of the pixel values of the disk elements and an average value of the pixel values of the ring elements in the filter is equal to or greater than a predetermined threshold value, and The image processing apparatus according to claim 1, wherein when the pixel value of the target pixel is lower than an average value of pixel values of the ring element, the detection of the unit image by the detection unit is canceled. An imaging unit for imaging the surface of the medium on which the unit image is formed;
A detection unit that scans an image captured by the imaging unit and detects the unit image;
The detector is
The image is scanned by applying a filter including a disk element having a certain area and a ring element provided around the disk element, and an average value of pixel values of the disk element in the filter and the ring element A pen device that determines, based on a difference from an average value of pixel values and a pixel value of a target pixel, whether the target pixel is a pixel that forms a unit image.   The detection unit is configured such that a difference between an average value of pixel values of the disk element and an average value of pixel values of the ring element in the filter is equal to or greater than a first threshold value, and a pixel value of the target pixel is a second value The pen device according to claim 6, wherein the pixel of interest is determined to be a pixel that forms a unit image when the threshold is equal to or greater than a threshold value. Generating a grid area covering the image;
For each area of the image corresponding to each small area of the grid area, a contrast threshold is set based on the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter And setting a pixel threshold based on the difference and the pixel value of each pixel,
Regarding the pixel forming the unit image detected by the detection unit, the difference obtained by applying the filter using the pixel as a target pixel is lower than the contrast threshold, or the pixel value of the pixel Cancels the detection of the unit image by the detection unit when is lower than the pixel threshold,
The pen device according to claim 6, further comprising a noise removing unit.   Based on the detection result of the detection unit, the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter is equal to or greater than a predetermined threshold, and the pixel value of the target pixel is The pen device according to claim 6, further comprising: a noise removing unit that cancels the detection of the unit image by the detecting unit when the pixel value of the ring element is lower than an average value. On the computer,
A function of scanning an image picked up by the image pickup device by applying a filter including a disk element having a certain area and a ring element provided around the disk element;
When the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter is equal to or greater than a first threshold value, and the pixel value of the target pixel is equal to or greater than a second threshold value A program for realizing a function of detecting a unit image by determining that the target pixel is a pixel forming the unit image. In the computer,
Generating a grid area covering the image;
For each area of the image corresponding to each small area of the grid area, a contrast threshold is set based on the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter And setting a pixel threshold based on the difference and the pixel value of each pixel,
Regarding the pixel forming the detected unit image, if the difference obtained by applying the filter with the pixel as a target pixel is lower than the contrast threshold, or the pixel value of the pixel is the pixel threshold The program according to claim 10, further realizing a function of canceling the detection of the unit image when the value is lower. In the computer,
Based on the detection result of the detection unit, the difference between the average value of the pixel values of the disk elements and the average value of the pixel values of the ring elements in the filter is equal to or greater than a predetermined threshold, and the pixel value of the target pixel is The program according to claim 10, further realizing a function of canceling the detection of the unit image by the detection unit when the pixel value of the ring element is lower than an average value.

Images