JP5521883B2 - Information processing apparatus, imaging apparatus, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an imaging apparatus, and a program.

  A technique is known in which an encoded image representing information in an array of a plurality of dot-like images is formed on a medium, and the encoded image is decoded based on an image captured by an imaging device (for example, patent document) 1 to 3). The encoded image of such a technique represents information by the mutual positional relationship of the point images.

JP 2006-85679 A JP 2003-511863 A JP 2007-179111 A

By the way, information may not be correctly decoded from an image obtained by capturing an encoded image due to dirt on the surface of the medium on which the encoded image is formed, an imaging process of the imaging device, or the like.
An object of the present invention is to selectively use digital data generated according to an array of point images so as to suppress the possibility that information is not correctly decoded from the imaging result of an encoded image composed of point images. It is.

  In order to solve the above-described problem, in the information processing apparatus according to claim 1 of the present invention, dot-like images that are dot-like images are arranged in regularly arranged regions, and the positional relationship between the dot-like images. Based on a captured image that is an image of a medium on which an encoded image representing information is formed, an arrangement line that is a line that connects the positions corresponding to the region and indicates the arrangement of the dotted images is set as the setting unit. Generating means for generating digital data associated with the coordinates of the intersection based on the arrangement of the dotted images detected corresponding to the intersection of the array lines set by the setting means; Reliability calculation means for calculating the reliability, which is the degree of reliability of the digital data generated by the generation means, based on the correspondence relationship between the set array line and the detected dot image; and the reliability calculation Calculated by means Yoriyukido is used with priority higher the digital data, characterized by comprising a decoding means for decoding the information in accordance with the specified the positional relationship from the captured image.

  An information processing apparatus according to a second aspect of the present invention is the information processing apparatus according to the first aspect, wherein the generation means is data in which values indicated by the digital data are arranged at positions corresponding to the coordinates of the intersections. And the decoding means preferentially uses digital data included in the region with higher reliability in the array indicated by the array data, and decodes information according to the positional relationship. To do.

  The information processing apparatus according to claim 3 of the present invention is the information processing apparatus according to claim 2, wherein the correspondence relationship is a distance between the intersection and the detected point image corresponding to the intersection. The degree calculation means increases the reliability as the distance is smaller.

  An information processing apparatus according to a fourth aspect of the present invention is the information processing apparatus according to the second aspect, wherein the reliability calculation means is within a range determined on the basis of the position of the detected point image on the captured image. With the density of the image as the correspondence relationship, the reliability is increased as the density is closer to a density determined according to the array line.

  An information processing apparatus according to a fifth aspect of the present invention is the information processing device according to the third or fourth aspect, wherein the reliability calculation means includes the shape, size, density, and the point image of the detected point image. The reliability is calculated in consideration of a density difference from the medium surface or a combination of a plurality of them.

  An information processing apparatus according to a sixth aspect of the present invention is the information processing apparatus according to any one of the first to fifth aspects, wherein the decoding means detects the start point when the positional relationship is specified from the captured image. The digital data generated from a position having a smaller distance from the point image is preferentially selected, and information is decoded according to the positional relationship.

  According to a seventh aspect of the present invention, there is provided an imaging apparatus according to the information processing apparatus according to any one of the first to sixth aspects, an irradiation unit that irradiates light, and reflected light of light irradiated by the irradiation unit. Imaging means for imaging, wherein the setting means sets the array line based on a dot image detected from a captured image obtained by imaging the medium by the imaging means.

  The program according to claim 8 of the present invention is a coded image in which dot images, which are dot images, are arranged in a region where computers are regularly arranged, and information is represented by the positional relationship between the dot images. Based on the captured image that is an image of the medium on which the image is formed, the setting unit sets an array line that connects the positions corresponding to the region and indicates the array of the dotted images, and the previous setting unit sets Generating means for generating digital data associated with the coordinates of the intersection, based on the arrangement of the dotted images detected corresponding to the intersection of the arrangement lines, the set array line and the A reliability calculation unit that calculates a reliability that is a degree of reliability of the digital data generated by the generation unit based on a correspondence relationship with the detected point image, and a reliability calculated by the reliability calculation unit Is more There the digital data using with priority, the is intended to function as a decoding means for decoding the information in accordance with the specified the positional relationship from a captured image.

According to the first and eighth aspects of the invention, the digital data generated according to the arrangement of the point images is suppressed so as to suppress the possibility that information is not correctly decoded from the imaging result of the encoded image composed of the point images. Can be used selectively.
According to the second aspect of the present invention, it is possible to easily select digital data in a region where the reliability of digital data is higher.
According to the invention of claim 3, the distance between the array line and the dot image can be used as an index of the reliability of the digital data.
According to the invention which concerns on Claim 4, the density of the image around the cross | intersection part of array lines can be used as a reliability index of digital data.
According to the invention which concerns on Claim 5, the characteristic of the dotted | punctate image on a captured image can be used as a parameter | index of the reliability of digital data.
According to the invention which concerns on Claim 6, it can decode using the imaging result of the part with comparatively small distortion of a captured image.
According to the seventh aspect of the present invention, information can be decoded from a captured image obtained by imaging so as to capture an encoded image and suppress the possibility that information is not correctly decoded.

Diagram showing the overall system configuration The figure explaining the structure of an encoding image The figure which shows the composition of the unit code pattern The figure which shows the structural example of a unit code pattern Diagram showing pattern block configuration The figure explaining the layout of an encoding image Diagram showing the configuration of the electronic pen The figure which shows an example of the arrangement | sequence of the dotted | punctate image in a captured image The figure which shows an example of the arrangement | sequence of a dotted | punctate image when distortion arises in a captured image Functional block diagram showing the functional configuration of the control unit of the electronic pen Functional block diagram showing the functional configuration of the sequence information detection unit The figure for demonstrating the function of a 2nd image pair detection part. The figure explaining the contents which arrangement information shows Functional block diagram showing the functional configuration of the decoding unit Flow chart showing the processing flow of the accuracy calculation unit The figure explaining the accuracy which an accuracy calculation part calculates The flowchart which shows the processing flow of a starting point determination part. The figure explaining the starting point which a starting point determination part determines The figure explaining the outline of the processing related to the setting of array lines Flowchart showing the processing flow for array line setting Diagram showing the setting procedure of the setting reference point The figure explaining the procedure of the index value calculation of an index value calculation part Diagram showing how column spacing is updated The figure which shows an example of the array line set by the array line setting part Diagram showing the configuration of the bit array Diagram showing the structure of the confidence array Functional block diagram showing the functional configuration of the decoding unit

[Configuration of the embodiment]
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
<A. System configuration>
FIG. 1 is a diagram illustrating an overall configuration of a system according to the present embodiment. The configuration of the system according to the present embodiment is roughly divided into a personal computer (PC) 10, a medium 50, and an electronic pen 60. The electronic pen 60 is an example of an imaging apparatus according to the present invention, and includes a function of handwriting characters and figures on the medium 50 and a function of capturing an encoded image formed on the medium 50. . The encoded image formed on the medium 50 is obtained by encoding information into an image according to a predetermined encoding method. The medium 50 may be made of paper or plastic such as an OHP sheet, or other materials, or may be electronic paper whose display content is electrically rewritten. In short, the medium 50 may be any medium on which an encoded image is formed so as to be imaged with the electronic pen 60. When the information is decoded from the encoded image at the position designated by the electronic pen 60, the PC 10 acquires the decoded information and executes processing based on the acquired information. In this process, for example, the handwritten content handwritten by the user using the electronic pen 60 is digitized to generate electronic data indicating the electronic document.
Next, the configuration of the encoded image formed on the medium 50 will be described.

<B. Structure of encoded image>
FIG. 2 is a diagram illustrating the configuration of the encoded image.
The encoded image represents information by an array of dot images, which are dot images, and is composed of a set of information blocks P3 formed on the entire surface of the medium 50 or a part of the surface. However, FIG. 2 shows only one of the information blocks P3 arranged on the medium 50. The information block P3 is composed of a set of a plurality of pattern blocks P2. The pattern block P2 is composed of a set of a plurality of unit code patterns P1.

FIG. 3 is a diagram showing the configuration of the unit code pattern P1.
The unit code pattern P1 has m areas (hereinafter referred to as “arrangement areas”) in which point images are arranged one by one in accordance with the encoded information. Here, m = 9, and the unit code pattern P1 has regularly arranged arrangement regions A1 to A9. The unit code pattern P1 has mCn (= m! / {(Mn)! ×) according to the arrangement of n (1 ≦ n <m) point-like images selected from m arrangement areas. n!}) Represents street information. The unit code pattern P1 shown in FIG. 3 exemplifies an encoding method in which dot images are arranged in two of nine arrangement regions. The number of dot-like images in the number of arrangement regions affects the relationship between the mode of arrangement of dot-like images and information expressed and the number of types of information expressed by one unit code pattern P1, In the present invention, the number of dot images is not limited to a specific one. However, in the following description, a case will be described in which the number of dot-like images occupying the number of arrangement areas of the unit code pattern P1 is “2”.

In the unit code pattern P1 shown in FIG. 3, the dot images are arranged in the arrangement areas A1 and A2 filled in black, but the dot images are arranged in the arrangement areas A3 to A9 shown by hatching. It has not been. In addition, the area other than the arrangement area is an area where no dot image is arranged, and is provided, for example, to distinguish each dot image.
Note that the plurality of dot-like images constituting the encoded image each represent information by mutual positional relationship. The shape of the dot image is, for example, a square, but may be another shape such as a circle. Further, the size of the dot image is not limited to a specific size in the present invention.

FIG. 4 is a diagram illustrating a configuration example of the unit code pattern P1.
There are a plurality of types of unit code patterns P1, each having a different arrangement of dot images. Here, there are 9C2 = 36 types of unit code patterns P1. As shown in FIG. 4, pattern numbers for identifying unit code patterns P1 having different dot pattern arrangement modes are assigned, and unit code patterns P1 having pattern numbers “0” to “35” exist. To do. Among the unit code patterns P1, the dot image having a specific arrangement form is used as the first unit code pattern, and the rest is used as the second unit code pattern. The first unit code pattern represents information embedded in the medium 50.

The second unit code pattern is a pattern necessary for taking out the first unit code pattern embedded in the medium 50. The second unit code pattern is used, for example, for specifying the position of the first unit code pattern or detecting the rotation of a two-dimensional captured image that is an image of the medium 50. In short, the second unit code pattern is referred to in order to correctly interpret the information represented by the encoded image. For example, the following four types of unit code patterns P1 are used as the second unit code patterns for detecting the rotation of the captured image. Here, the unit code pattern P1 having the pattern number “32” is set as an upright second unit code pattern. The unit code pattern P1 having the pattern number “33” is obtained by rotating the pattern number “32” by 90 degrees clockwise. The unit code pattern P1 having the pattern number “34” is obtained by rotating the pattern number “32” by 180 degrees clockwise. The unit code pattern P1 having the pattern number “35” is obtained by rotating the pattern number “32” by 270 degrees clockwise. In this case, 36 unit code patterns obtained by removing these four types of second unit code patterns correspond to the first unit code patterns.
The second unit code pattern corresponds to the “synchronization pattern” disclosed in Japanese Patent Application Laid-Open No. 2009-181347. The second unit code pattern is used for the same purpose as the synchronization pattern described in the publication.

FIG. 5 is a diagram showing the configuration of the pattern block P2. FIG. 5 shows the configuration of the pattern block P2 when the second unit code pattern is upright.
In the pattern block P2, the first unit code pattern and the second unit code pattern are arranged according to a determined layout. As shown in FIG. 5, the pattern block P2 has a configuration in which a total of 25 unit code patterns P1 are arranged in a square shape, 5 in the vertical direction and 5 in the horizontal direction. Of the 25 unit code patterns P1, the second unit code pattern is arranged at the upper left corner. In four consecutive blocks located on the right side of the second unit code pattern, there is one axis (to be described later) of a coordinate plane (hereinafter referred to as “first coordinate plane”) used to define a position on the medium 50. , X-axis) is arranged as a first unit code pattern. A first unit code pattern representing the coordinates of the other axis (Y-axis, which will be described later) of the first coordinate plane is arranged in four consecutive blocks located below the second unit code pattern. In the remaining 16 blocks, for example, a first unit code pattern representing identification information of the medium 50 or identification information of a document formed on the medium 50 is arranged.
Here, the first coordinate plane is described in a two-dimensional orthogonal coordinate system defined by the X axis and the Y axis. Here, the X axis corresponds to a position in one direction in the rectangular image region of the captured image, and the Y axis corresponds to a position in the other. Here, it is assumed that the lower left corner point of the captured image is the origin of the first coordinate plane.

FIG. 6 is a diagram for explaining the layout of the information block P3.
One rectangle of the minimum size shown in FIG. 6 corresponds to one coding block P1, and the coding blocks P1 having the same alphabet and numerical code included therein are arranged in the same manner in the arrangement area A1. A dot image is arranged at A9 to A9. In FIG. 6, the second unit code pattern is represented by “S”, the pattern block P2 representing the X coordinate is represented by “X1”, “X2”,..., And the pattern block P2 representing the Y coordinate is represented by “Y1”, “Y2”. ", ... Each of the X coordinate and the Y coordinate is expressed in M series for each of the two directions on the medium 50 according to the contents of the pattern block P2, for example. For example, RS encoding is used for encoding the identification information. In FIG. 6, the pattern representing the identification information is represented by “IXY” (X = 1 to 4, Y = 1 to 4).
The encoded image having the above configuration has a configuration in which dot images are arranged in a plurality of rows in two directions orthogonal to each other.

When the encoded image having the above configuration is formed on the medium 50 using an electrophotographic method, for example, black toner (that is, infrared light absorbing toner containing carbon) or special toner is used. The special toner is, for example, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm). “Visible” and “invisible” as used herein have nothing to do with whether or not they are visually recognized. Here, “visible” and “invisible” are distinguished based on whether or not the image formed on the medium 50 is recognized based on the presence or absence of color development caused by absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes. The invisible toner preferably has an average dispersion diameter in the range of 100 nm to 600 nm in order to increase the near infrared light absorption capability necessary for machine reading of an image.
The configuration of the encoded image has been described above.

<C. Configuration of electronic pen 60>
FIG. 7 is a block diagram illustrating a configuration of the electronic pen 60.
The control unit 61 is an example of an information processing apparatus according to the present invention, and controls the operation of the electronic pen 60. The control unit 61 includes an image processing unit 61a and a data processing unit 61b. The image processing unit 61a includes an arithmetic device and a memory including a CPU (Central Processing Unit) and an ASIC (Application Specific Integrated Circuit), and performs processing based on an encoded image detected from a captured image that is an image obtained by capturing the medium 50. Run. The data processing unit 61b includes an arithmetic device including a CPU and a memory, decodes information according to the execution result of the processing of the image processing unit 61a, and extracts identification information and coordinate information.
The control unit 61 has a non-volatile memory (not shown) that stores a program for operating the image processing unit 61a and the data processing unit 61b, and implements various functions by executing the program.

The pressure sensor 62 detects the writing operation by the electronic pen 60 by the pressure applied to the pen tip 69. The imaging unit 70 includes an irradiation unit 63 and an imaging unit 64. The irradiation unit 63 is an example of the irradiation unit of the present invention, and is, for example, an LED (Light Emitting Diode), and irradiates the medium 50 with infrared light. The imaging unit 64 is an example of the imaging unit of the present invention, and captures an encoded image on the medium 50 according to the reflected light of the infrared light irradiated from the irradiation unit 63. In the electronic pen 60, the surface of the medium 50 is imaged by the imaging unit 70 at a predetermined frame rate (for example, 60 fps (frame per second)), and imaging data that is image information representing the captured image is generated. The information memory 65 stores the identification information and position information extracted by the data processing unit 61b. The communication unit 66 performs control related to communication between the electronic pen 60 and the PC 10. The battery 67 is a rechargeable battery, for example, and supplies power for driving the electronic pen 60 to each unit. The pen ID memory 68 stores identification information of the electronic pen 60. The pen tip 69 is a so-called pen shaft, and a pen tip 69a is provided at the tip. The pen tip 69a indicates the position on the medium 50 on which the encoded image to be imaged is formed when the user performs a writing operation. The irradiating unit 63 irradiates light to the imaging range R on the medium 50 indicated by the pen tip 69a when a writing operation is performed by the user. The switch 75 is used for switching various settings.
Next, a problem related to imaging of an encoded image with the electronic pen 60 will be described.

FIG. 8 is a diagram illustrating an arrangement of dot images in a captured image obtained by capturing an encoded image. FIG. 8 is a diagram in which the positions of the point images on the captured image are plotted on the first coordinate plane with “□” marks.
In the captured image shown in FIG. 8, the point images are arranged in parallel or substantially parallel to the X axis and Y axis. Here, a quadrangle formed by connecting the positions of four dot images adjacent to each other with line segments is substantially a square, and there is almost no difference with respect to the arrangement of the encoded images on the medium 50. In this case, if the positional relationship between two adjacent dot images is examined, the arrangement direction of the dot images can be detected relatively easily and accurately. In addition, as a decoding technique that is a process of converting into digital data according to the arrangement mode of the dot images, a line indicating the arrangement of dot images (hereinafter referred to as “array line”) connecting positions corresponding to the arrangement areas. ) And referring to the arrangement of the dot images corresponding to the intersections of the array lines. In the encoded image configuration described above, an array line (shown by a solid line) parallel to the X axis or the Y axis is applied to each column of the dot image array. In the captured image shown in FIG. 8, since the dot images are arranged so as to substantially match the positions of the respective intersections, the decoding is relatively easy and accurate from the correspondence between the array lines and the dot images. Easy to be done.
By the way, the electronic pen 60 may be operated with the posture with respect to the surface of the medium 50 inclined. In this case, the imaging surface of the electronic pen 60 may be tilted with respect to the surface of the medium 50, and the captured image may be distorted. In this case, as in the case of the imaging result shown in FIG. 8, it may be difficult to specify the arrangement direction of the dotted images or to set the arrangement line.

FIG. 9 is a diagram illustrating the arrangement of the dot images when the captured image is distorted. Also in FIG. 9, the positions where the point images are arranged on the captured image are plotted with “□” marks on the first coordinate plane.
In the captured image shown in FIG. 9, the interval between the point images in the X-axis direction varies, and the interval differs at each position in the Y-axis direction. Here, in FIG. 9, this interval gradually decreases in the axial direction away from the origin of the Y axis. This may occur when the electronic pen 60 is tilted with respect to the Y-axis direction. At this time, the quadrilateral formed by connecting the positions of four adjacent dot images with line segments is a trapezoid. Therefore, when the same array line as that shown in FIG. 8 is set, the intersection of the array lines and the position of the dotted image may be partially or completely shifted in the captured image, which may hinder decoding. . Further, as can be seen from FIG. 9, the distortion of such a captured image tends to increase as the distance from the central portion of the captured image approaches the end portion.

  As described above, when the captured image is distorted, the interval between the dot images and the arrangement direction of the dot images may not be accurately specified from the captured image, and information may not be correctly decoded from the imaging result of the encoded image. There is. Although not particularly illustrated, when the posture of the electronic pen 60 is tilted with respect to the X-axis direction, a quadrilateral formed by connecting the positions of four code images adjacent to each other with line segments is a parallelogram or rhombus. It becomes. In this case, it is difficult to accurately specify the arrangement direction and the interval between the point images for both the X axis and the Y axis.

Furthermore, in addition to the distortion that occurs in the captured image, information may not be correctly decoded from the captured image of the encoded image for the following reasons. For example, if dirt such as toner dropped during formation of the encoded image adheres to the surface of the medium 50, this dirt may be mistakenly recognized as a dot image constituting the encoded image. In this case, this stain is treated as a dot image, and the mutual positional relationship with other dot images is specified, which may hinder decoding and decoding. Further, when the electronic pen 60 irradiates the medium 50 with irradiation light and regular reflection occurs, the irradiation light of the irradiation unit 63 may be totally reflected on the surface of the medium 50 and captured by the imaging unit 64. . In this case, image unevenness frequently occurs within the imaging range R. At this time, there is a possibility that the portion where the brightness is high in the imaging range R and the brightness is low in the imaging range R is mistakenly recognized as a dot image constituting the encoded image.
As described above, in the technique of detecting the position of the point image from the captured image and decoding the information, the captured image may include an imaging result that hinders accurate decoding. In response to such a problem, the electronic pen 60 realizes the function described below in order to suppress the possibility that correct decoding is not realized from the imaging result of the encoded image.

[Function configuration]
<D. Functions of the electronic pen 60>
FIG. 10 is a functional block diagram showing an outline of a functional configuration of the control unit 61 of the electronic pen 60. As shown in FIG. 10, the functions realized by the control unit 61 are roughly divided into an array information detection unit 100, a decoding unit 200, and a decoding unit 300. The array information detection unit 100 detects a dot image from the captured image, and based on the position of the detected dot image on the captured image, the column direction of the array of the dot images and the interval between the columns of the array Is detected. Based on the sequence information detected by the sequence information detection unit 100, the decoding unit 200 realizes a function including decoding that converts a dotted image detected from the captured image into digital data. The decoding unit 300 decodes information represented by the encoded image based on the decoding result in the decoding unit 200.
Of the above three functions, the functions of the array information detection unit 100 and the decoding unit 200 are realized by, for example, the image processing unit 61a, and the function of the decoding unit 300 is realized by, for example, the data processing unit 61b. The

<D. 1 Sequence Information Detection Unit 100>
First, functions realized by the array information detection unit 100 will be described. FIG. 11 is a functional block diagram showing a more detailed functional configuration of the sequence information detection unit 100. As shown in FIG. 11, the array information detection unit 100 functions as an imaging data acquisition unit 101, a noise removal unit 102, a dotted image detection unit 103, a first image pair detection unit 104, and a second image pair. The detection unit 105, the column direction detection unit 106, and the column interval detection unit 107 are divided.
When the surface of the medium 50 on which the encoded image is formed is imaged by the electronic pen 60, the imaging data acquisition unit 101 acquires imaging data representing one frame of the imaging image.
The noise removal unit 102 performs a process for removing noise from the captured image represented by the imaging data. As the noise, there are noise caused by characteristics of the image sensor in the electronic pen 60 and noise caused by an electronic circuit. The noise removing unit 102 performs sharpening processing such as blurring processing and unsharp masking, for example. However, what kind of processing the noise removing unit 102 performs to remove noise may be determined according to the characteristics of the imaging system of the electronic pen 60.

The dotted image detection unit 103 detects a dotted image from the captured image based on the captured data from which noise has been removed. The dotted image detection unit 103 performs binarization processing to separate the dotted image and other images (that is, an image corresponding to the background) and detect the dotted image. When the binarized captured image includes a lot of noise components, the dotted image detection unit 103 is a dot image that forms an encoded image based on the area and shape of each image in the captured image. A filtering process for determining whether or not may be used. That is, the point image detection unit 103 is an example of the detection means of the present invention.
Note that the dot images detected by the dot image detection unit 103 include not only what constitutes an encoded image (that is, an image of the dot images constituting the encoded image), There may be a point-like noise mixed due to an imaging process of the electronic pen 60 or the like.

  The first image pair detection unit 104 detects a pair of point-like images close to each other as the first image pair with respect to the point-like images detected by the point-like image detection unit 103. The first image pair is constituted by a pair of a dot image on the captured image and a dot image at the closest position. Here, the closest position corresponds to the distance between adjacent arrangement areas such as the arrangement areas A1 and A2 and A2 and A3 with the shortest distance in the unit code pattern P1.

The second image pair detection unit 105 detects a first image pair detected by the first image pair detection unit 104 that is inclined in the column direction of the array of dot images as a second image pair. However, here, the column direction of the arrangement of dot images is the direction in which arrangement areas adjacent to each other at the shortest distance, such as arrangement areas A1 and A2 and A2 and A3, are arranged. Such an adjacent direction that is not the shortest distance is not included. That is, in the configuration of the encoded image of this embodiment, there are two different column directions of the array.
In order to narrow down what is detected as the second image pair, the second image pair detection unit 105 identifies the inclination direction of the dotted image for each of the first image pairs, and the first image is the inclination direction having a high appearance frequency. Only the pair is detected as the second image pair. Specifically, the second image pair detection unit 105 calculates a frequency distribution of angles formed by the tilt direction of the first image pair on the first coordinate plane and the X axis. And the 2nd image pair detection part 105 specifies the angle whose appearance frequency is more than a threshold value. For example, the second image pair detection unit 105 identifies an angle range included in a range determined based on the angle at which the appearance frequency is maximum. Then, the second image pair detection unit 105 detects the first image pair tilted within this angle range as the second image pair.

  FIG. 12 is a diagram for explaining the function of the second image pair detection unit 105. FIG. 12 is a diagram illustrating a captured image IMG1 that is an example of a captured image, and it is assumed that a dot image is detected at a position indicated by a “■” mark. In FIG. 12, the X axis extends along the right side of the drawing, and the Y axis extends along the upper side of the drawing. As already described, in the encoded image of this embodiment, since the dot images are arranged along two directions orthogonal to each other, the appearance frequency of the inclination direction corresponding to these two directions also appears on the captured image. Should be relatively high with respect to the other directions. Accordingly, in the example illustrated in FIG. 12, the first image pair in which the dotted images are connected by a broken line is detected as the second image pair by the second image pair detection unit 105.

  The column direction detection unit 106 detects the column direction of the arrangement of the dot images on the captured image based on the inclination direction of the second image pair detected by the second image pair detection unit 105. The column direction detection unit 106 detects the column direction of the array according to the angle range specified by the second image pair detection unit 105. For example, the column direction detection unit 106 detects, for example, an angle having the highest appearance frequency in the angle range or an average angle weighted to the appearance frequency as the column direction of the dot image array. Here, the second image pair detection unit 105 detects, for example, the column direction of the array of dot image types, that is, a tilt direction that matches or substantially matches the X axis and a tilt direction that matches or nearly matches the Y axis. Will do.

FIG. 13 is a diagram for explaining the contents indicated by the array information. Here, only the column direction of the array indicated by the array information will be described. As illustrated in FIG. 13, the column direction detection unit 106 represents the column direction of the array of dot images using the angle θ formed with the X axis. However, the arrangement direction detected by the column direction detection unit 106 is an angle formed with the X axis and represented by an angle within a range of 180 degrees. As shown in FIG. 13, when the other point-like image is inclined in the direction of the arrow on the Y-axis with respect to the point-like image located on the origin side with respect to the X-axis direction, θ becomes a positive value. On the other hand, when the other point-like image is inclined in the direction opposite to the Y-axis arrow, θ takes a negative value. Here, the column direction detection unit 106 detects two directions of angles θ1 and θ2 as the column direction of the array. Thereby, the column direction of the arrangement of the dot images on the captured image is roughly grasped.
Note that the arrangement direction detected by the column direction detection unit 106 may be represented by an angle with respect to the Y axis.

  The column interval detection unit 107 detects the interval between columns of the array of point images (that is, the column interval) based on the distance between the point images of the second image pair. Here, the column interval detection unit 107 calculates the distance between the point images in the second image pair, and calculates the average value thereof. As described above, since the interval between columns of each array is constant in the encoded image, an approximate value of the column interval on the captured image is detected from this average value. However, a certain threshold value is provided in order to eliminate a projecting large distance between point images of the second image pair, and the column interval detection unit 107 is based on the second image pair whose distance is equal to or less than the threshold value. The column spacing may be calculated. As a result, as shown in FIG. 13, the column spacing with respect to the direction represented by the angles θ1 and θ2 is obtained. Here, the column interval with respect to the arrangement direction of the angle θ1 is represented by D1, and the column interval with respect to the arrangement direction of the angle θ2 is represented by D2. In other words, the angle θ1 is an arrangement direction of a plurality of rows of dot images arranged along the direction of the angle θ2, and the angle θ2 is a plurality of rows of dot images arranged along the direction of the angle θ1. In other words, the direction.

The above is description of the function which the arrangement | sequence information detection part 100 implement | achieves. The array information detection unit 100 detects a set of array information for one frame of the captured image and performs this on the captured image of each frame. In this embodiment, since the encoded image has a configuration in which dot images are arranged in a plurality of columns along two directions orthogonal to each other, as shown in FIG. 13, the array information includes the column direction and the column interval. It is also considered as a vector containing.
However, the array information represents the column direction and the column interval when viewed in the entire captured image. However, in the case of the distorted captured image illustrated in FIG. 9, the column direction and the column interval are different at each position. However, different column directions and column intervals are not expressed for each position.

<D. 2 Decoding unit 200>
Next, functions realized by the decoding unit 200 will be described. FIG. 14 is a functional block diagram showing a more detailed functional configuration of the decoding unit 200. As shown in FIG. 14, the function of the decoding unit 200 is divided into an accuracy calculation unit 201, a start point determination unit 202, an array line setting unit 203, a bit array generation unit 204, and a reliability array calculation unit 205. It is done.

<D. 2.1 Accuracy Calculation Unit 201>
First, the function of the accuracy calculation unit 201 will be described.
The accuracy calculation unit 201 calculates the accuracy that the dot images detected by the dot image detection unit 103 constitute an encoded image. The accuracy here is an index of the probability that the detected point image captured the image constituting the encoded image, and the larger the value, the higher the possibility that the encoded image is configured. On the other hand, the accuracy calculation unit 201 lowers the accuracy of a point image that is assumed to be a point image corresponding to noise.
Here, a processing flow executed by the accuracy calculation unit 201 will be described with reference to FIG. FIG. 15 is a flowchart illustrating a processing flow in the accuracy calculation unit 201.
FIG. 15A shows the processing flow of Operation Example 1, and FIG. 15B shows the processing flow of Operation Example 2.

Operation example 1 will be described.
First, the accuracy calculation unit 201 pays attention to any of the point images detected from the captured image. The accuracy calculation unit 201 ultimately pays attention to all point-like images detected from the captured image, and therefore selects any one according to a predetermined algorithm. Then, the accuracy calculation unit 201 acquires coordinates indicating the position of the focused point image on the captured image (step SA1). Next, the accuracy calculation unit 201 acquires coordinates indicating the position on the captured image of the adjacent region with the pointed dot image as a reference (step SA2). The adjacent area corresponds to an arrangement area adjacent to the arrangement area in which the point-like image of interest is arranged in the column direction of the arrangement of the point-like images. Here, the accuracy calculation unit 201 identifies an arrangement region adjacent in four directions as an adjacent region according to the angles θ1 and θ2 indicated by the array information (see FIG. 16). For example, if the focused dot image corresponds to the arrangement area A5, the accuracy calculation unit 201 sets the areas corresponding to the arrangement areas A2, A4, A6, and A8 as the adjacent areas. Further, the accuracy calculation unit 201 determines a distance away from the position indicated by the array information as the adjacent region.
Here, the accuracy calculation unit 201 treats the arrangement areas arranged in the same column according to the angles θ1 and θ2 as adjacent areas, but includes, for example, A1, A3, A7, and A9 for the arrangement area A5. A total of eight arrangement areas may be treated as adjacent areas.

  Next, the accuracy calculation unit 201 refers to whether or not a dot image is arranged in the adjacent area (step SA3). Here, the accuracy calculation unit 201 may determine an adjacent region having coordinates in a range determined with reference to the coordinates instead of making the adjacent region correspond to a single point of coordinates. The reason for this is that there may be a case where the captured image changes from the original positional relationship when distortion occurs.

  Then, the accuracy calculation unit 201 determines whether at least one point image is arranged in the adjacent area (step SA4). Here, as shown in FIG. 16A, when the accuracy calculation unit 201 determines that there is even one point image in the adjacent region (step SA4; YES), the pointed point image is encoded. It is determined that the accuracy of the image is high, and the accuracy is set to “high accuracy” (step SA5). Here, the accuracy calculation unit 201 represents the accuracy when the accuracy is “high” by a numerical value “4”. If there is a dot image in an adjacent area as seen from a certain dot image, it is placed in a place where the dot image may be placed, so the focused dot image constitutes the encoded image It is estimated that there is a high probability of being.

  On the other hand, as shown in FIG. 16B, when the accuracy calculation unit 201 determines that no dot image is arranged in the adjacent region (step SA4; NO), the focused dot image is It is determined that the accuracy of the encoded image is low, and the accuracy is set to “low accuracy” (step SA6). Here, the accuracy calculation unit 201 represents the accuracy when the accuracy is “low” by a numerical value “0”. If there is no dot image in the adjacent area as seen from a certain dot image, it is accidentally that no dot image has been placed in the adjacent area, or because the point image in question is noise, the dot image is in the adjacent area. It is unknown whether the image does not exist. However, it is presumed that there is a low possibility that the focused point-like image constitutes the encoded image, compared to the case where the point-like image exists at least in the adjacent region. Therefore, the accuracy calculation unit 201 decreases the accuracy in this case.

  Next, the accuracy calculation unit 201 determines whether or not the accuracy calculation has been completed for all the point images (step SA7). Here, when the accuracy calculation unit 201 determines that the calculation of the accuracy has not been completed for all the point images (step SA7; NO), the accuracy calculation unit 201 returns to the process of step SA1 and focuses on another point image. The processing steps SA1 to SA6 are repeated. On the other hand, when the accuracy calculation unit 201 determines that the calculation of the accuracy has been completed for all the point images (step SA7; YES), the accuracy calculation unit 201 ends the process related to the calculation of the accuracy.

Although the above is the description of the operation example 1, the accuracy calculation unit 201 may calculate the accuracy according to the operation example 2. In the operation example 2, the accuracy calculation unit 201 calculates not only the presence / absence of the dotted image in the adjacent area but also the accuracy based on the number of the dotted images. In the following description, the same operations as those in the operation example 1 are denoted by the same reference numerals, and the description thereof is omitted.
When the accuracy calculation unit 201 acquires the coordinates indicating the position of the adjacent region in the process of step SA2, the accuracy calculation unit 201 counts the dot images arranged in the adjacent regions in the four directions (step SA8). Then, the accuracy calculation unit 201 sets the counted number of dot images as the accuracy (step SA9). For example, the accuracy calculation unit 201 sets the accuracy to “4” if the number of adjacent images is “4”, and sets the accuracy to “1” if the number of adjacent images is “1”. is there. The reason for this is that as the number of point-like images in the adjacent area increases, the frequency at which the point-like images are likely to be placed is higher. This is because it is estimated that there is a higher possibility that the encoded image is configured. Since other processes are the same as those in the first operation example, description thereof is omitted. As described above, the accuracy calculation unit 201 uses the number of adjacent dot images as an index of accuracy that the dot images on the captured image constitute an encoded image.
When the accuracy calculation unit 201 calculates the accuracy for each point image according to the procedure of either one of the operation examples 1 and 2, the accuracy indicating the position of the point image and the accuracy are stored in the memory in association with each other.

<D. 2.2 Start Point Determination Unit 202>
Next, the function of the starting point determination unit 202 will be described.
Based on the accuracy calculated by the accuracy calculation unit 201 and the position of the point image on the captured image, the start point determination unit 202 gives priority to the one with the higher accuracy, and selects one of the point images. Is determined as the starting point. This starting point is a point in the process of decoding by the decoding unit 200, which point image is used to specify the position of another point image when specifying the mutual positional relationship of the point images from the captured image. It is determined. Specifically, in the decoding unit 200 described later, each dot image on the captured image is traced using a certain dot image as a starting point, and it is referred whether there is a dot image corresponding to the arrangement area. Is done. By repeating this reference, the decoding unit 200 identifies the arrangement mode of the dot images and performs decoding. As described above, which part of the captured image is used to decode the information depends on the position of the start point determined by the start point determination unit 202. Therefore, in the captured image, a more reliable image pickup result around the start point is required.

Here, the processing flow executed by the start point determination unit 202 will be described with reference to FIGS. FIG. 17 is a flowchart illustrating a processing flow in the start point determination unit 202. FIG. 18 is a diagram illustrating an example of a captured image, and it is assumed that a dot image is detected at a position indicated by a “■” mark.
First, the start point determination unit 202 calculates the center of gravity of a point image whose accuracy is equal to or higher than a threshold value (here, “1”) on the captured image (step SB1). The start point determination unit 202 acquires the coordinates of the point images whose accuracy is equal to or greater than the threshold, and calculates the center of gravity on the captured image based on the acquired coordinates. Here, it is assumed that the start point determination unit 202 calculates the coordinates of the center of gravity G shown in FIG.
Note that if the point image is distributed over the entire captured image, the center of gravity is located near the center of the captured image.

Next, the starting point determination unit 202 searches for point-like images whose accuracy is equal to or higher than a threshold value (here, “1” or higher) in order from the center of gravity G (step SB2). Here, the start point determination unit 202 is within a predetermined distance based on the center of gravity G among the point images that are not subjected to the processing of step SB3 to be described later, as indicated by broken line arrows in FIG. The closest point image included in a certain search range T is searched.
Note that the search range T is determined because the vicinity of the end portion where the influence of distortion of the captured image is likely to appear is not determined as the start point. However, the start point determination unit 202 may search all point-like images without determining the search range T.

Next, the start point determination unit 202 calculates a weighted distance DW by giving a weight according to the accuracy to the distance from the center of gravity G to the point image searched in the process of step S2 (step SB3). Specifically, the starting point determination unit 202 calculates a weighted distance DW by giving a weight to the distance that becomes lighter as the accuracy increases. Here, the start point determination unit 202 performs a calculation of weighted distance DW = D + f (K). D is a linear distance from the center of gravity G to the point image. f (K) is a function with the accuracy K as a variable, and the value decreases as the accuracy increases. The starting point determination unit 202 uses functions such as f (K) = 1 / K and f (K) = exp (1 / K), for example. The start point determination unit 202 stores the calculated weighted distance DW in the memory in association with the coordinates of each point image.
Note that the weighting mode provided by the start point determination unit 202 may be any other mode as long as the condition that the weight becomes lighter as the accuracy increases is satisfied.

  Next, the start point determination unit 202 determines whether or not the weighted distance DW has been calculated for all point images included in the search range T (step SB4). Here, if the start point determination unit 202 determines that the weighted distance DW has not been calculated for all the point images (step SB4; NO), the process returns to the process of step SB1 and the weighted distance DW for another point image. The processing step for calculating is executed. On the other hand, when the start point determination unit 202 determines that the weighted distance DW has been calculated for all the point images (step SB4; YES), the start point determination unit 202 determines the point image having the calculated weighted distance DW as the start point. (Step SB5). For example, the start point determination unit 202 determines the point image shown in FIG.

  With the above processing, the start point determination unit 202 determines a point image that satisfies both the conditions that the distance from the center of gravity G is relatively small and the accuracy is relatively high as the start point. The reason why the start point determination unit 202 tries to determine a point image having a small distance from the center of gravity G as a start point preferentially is due to the distortion shown in FIG. This is because it is considered that the influence is often greater than the vicinity of the center of the imaging range R. In addition, the start point determination unit 202 preferentially determines a point image with high accuracy as the start point because the start point is erroneously determined as the start point of the noise, This is to suppress the possibility that the positional relationship is not accurately specified. This is because if the noise is determined as the start point, the positional relationship according to the encoded image is not obtained, and as a result, the possibility that information is not correctly decoded increases.

Note that the processing in step SB5 may be performed as follows.
For example, the start point determination unit 202 may give weighting that becomes heavier as the distance from the center of gravity G becomes smaller to the accuracy, and may determine the point image having the maximum weighted accuracy as the start point. Also in this case, the start point determination unit 202 is equivalent to determining a point-like image satisfying both conditions that the distance from the center of gravity G is small and the accuracy is relatively high as the start point. In addition, if there are a plurality of point images having the same distance from the center of gravity G, the start point determination unit 202 may use a start point with a high degree of accuracy, or if there are a plurality of point images with the maximum accuracy. The one with the smallest distance may be used as the starting point. The start point determination unit 202 may use the coordinates of the center of the captured image instead of the center of gravity G in the process of step S5, or may use another coordinate near the center. In short, the start point determination unit 202 prioritizes a point image having a small distance from a reference position on the captured image, such as the center of gravity G or the coordinates of the center of the captured image, and determines the start point. The point image with high accuracy may be determined as the start point with higher priority.
The above is the description of the procedure for determining the starting point.

<D. 2.3 Array Line Setting Unit 203>
Next, the function of the array line setting unit 203 will be described. The array line setting unit 203 is used when specifying the mutual positional relationship between the dot images from the captured image, and sets an array line indicating the array of the dot images for each column. The function of the array line setting unit 203 is divided into an array information acquisition unit 2031, a first array line setting unit 2032, an index value calculation unit 2033, a regression equation calculation unit 2034, and a second array line setting unit 2035. .
The array information acquisition unit 2031 is detected by the array information detection unit 100, and acquires array information including the column direction of the array of dot images and the interval between the columns of the array. This arrangement information is of a type corresponding to the column direction of the dot image arrangement. That is, here, the array information acquisition unit 2031 acquires two sets of array information (D1, θ1) and (D2, θ2) as shown in FIG.

  The first array line setting unit 2032 sets an array line using the start point determined by the start point determination unit 202 based on the array information acquired by the array information acquisition unit 2031. More specifically, the first array line setting unit 2032 sequentially sets array lines that pass through each point along the arrangement direction of a plurality of columns of the array of dot images indicated by the array information from the start point S. The first array line setting unit 2032 sets array lines so as to have an inclination in the column direction indicated by the array information and to make adjacent array lines have a column interval indicated by the array information.

The index value calculation unit 2033 calculates the index values of the dot images constituting the column for each column for which the array line is set by the first array line setting unit 2032. Here, the index value is a value corresponding to the distance from each dot image to the array line. Specifically, the closer the dot image is to the array line, the higher the probability that the dot image will be an encoded image and the greater the index value. On the other hand, the object far from the array line is likely to correspond to noise, and the index value becomes small.
In this case, the index value is a value corresponding to the correspondence between the dotted image on the captured image and the array line set by the first array line setting unit 2032.

  The regression equation calculation unit 2034 gives weights that become heavier as the index value calculated by the index value calculation unit 2033 is larger to the coordinates on the captured image of the dotted images, and shows the regression equation indicating the arrangement of the dotted images. Calculate for columns. The regression line indicated by this regression equation indicates the distribution of the point images on the captured image, but the results that are probable to constitute the encoded image are more important. Therefore, a regression equation indicating the distribution of point images is calculated while suppressing the influence of noise or the like as compared with the case where the regression equation calculation unit 2034 does not give weights.

  The second array line setting unit 2035 newly sets the array line of each column based on the regression line for each column calculated by the regression equation calculation unit 2034. In other words, the second array line setting unit 2035 corrects the array line set by the first array line setting unit 2032 based on the regression equation calculated by the regression equation calculation unit 2034. That is, the function corresponding to the setting means of the present invention is realized by the cooperation of the first array line setting unit 2032 and the second array line setting unit 2035.

Here, processing relating to setting of array lines in the array line setting unit 203 will be described with reference to FIGS. 19 to 24. FIG. 19 is a diagram for explaining the outline of the processing related to the setting of the array line, and is a diagram illustrating a captured image IMG2 that is an example of the captured image. As shown in FIG. 19, when the start point S determined by the start point determination unit 202 is set on the captured image IMG2, as shown by the arrow in FIG. A total of four directions indicated by the directions are set as the setting directions, and the array lines are sequentially set one by one along the setting direction. The setting direction here coincides with the column direction indicated by the arrangement information, and the coordinate axis of the coordinate plane on which the arrangement line is set is determined in this direction. Hereinafter, this coordinate plane is referred to as a “second coordinate plane”. The second coordinate plane has the starting point S as the origin, but is not necessarily an orthogonal coordinate system. FIG. 20 is a flowchart illustrating a processing flow relating to setting of array lines. FIG. 21 is a diagram for explaining an array line setting procedure. Hereinafter, a case where the array line setting unit 203 sets the direction corresponding to the angle θ as the setting direction will be described. Here, it is assumed that the column interval between the array lines set in this setting direction is D.
Here, the column direction and the column interval D, which are the angle θ, are determined based on the arrangement information. According to the example of FIG. 13, in the following description, when the setting direction corresponds to θ2, θ = θ1, D = D2, and when the setting direction corresponds to θ1, θ = θ2. , D = D1.

  First, the first array line setting unit 2032 sets the setting reference point to the start point S determined by the start point determination unit 202 (step SC1). The setting reference point is a reference point for setting the array line, and the array line set by the first array line setting unit 2032 passes through the setting reference point. Next, the first array line setting unit 2032 sets an array line that passes through the setting reference point based on the array information acquired by the array information acquisition unit 2031 (step SC2). Here, the setting reference point is the start point S, and the inclination of the array line corresponds to the column direction indicated by the array information. Here, as shown in FIG. 21A, the first array line setting unit 2032 passes through a set reference point that is the start point S, and has an inclination with an angle formed by the X axis that is θ. Set the line L.

Next, the index value calculation unit 2033 calculates the index value for each of the dot images corresponding to the column for which the array line is set by the first array line setting unit 2032 (step SC3).
FIG. 22 is a diagram for explaining the procedure for calculating the index value of the index value calculation unit 2033. In FIG. 22, what is indicated by “■” is equivalent to a dot image constituting the encoded image, and what is indicated by “□” is noise. In addition, the index value calculation unit 2033 performs the following in order to grasp which point-like image corresponds to which column of the array line.
The index value calculation unit 2033 sets a virtual line specified by the array information with reference to the array line L set by the first array line setting unit 2032. Specifically, the index value calculation unit 2033 identifies a point that is separated from the setting reference point of the array line L by the column interval D in the array direction, passes through this point, and has the same inclination as the array line L. A virtual line is set on both sides of the array line L. Then, the index value calculation unit 2033 sets a boundary line passing through the center between the array line L and the virtual line. The index value calculation unit 2033 treats the dot image located at a position closer to the boundary line with reference to the array line L as a dot image in a column common to the array line L. Therefore, the dot images d1 and d2 shown in FIG. 22A are not subject to index value calculation here.

  When the index value calculation unit 2033 specifies a point image for which the index value is to be calculated, as shown in FIG. 22B, the index value calculation unit 2033 sets the array line L to match either the X axis or the Y axis. The captured image is rotated around the reference point. The index value calculation unit 2033 rotates the captured image in order to simplify the algorithm for calculating the index value. Therefore, such rotation is not necessarily essential in the present invention. The index value calculation unit 2033 calculates the index value so that the index value increases as the dot image has a smaller distance to the array line L. Therefore, the magnitude of the index value of each point image has a tendency shown in FIG. As can be seen from FIG. 22 (b), unlike the point-like image of the encoded image, randomly mixed noise may appear at a position away from the array line L. That is, it is presumed that a dot image having a relatively large distance from the array line L is more likely to be noise than an image having a relatively small distance. Therefore, the index value calculation unit 2033 reduces the index value of the point image having a relatively large distance, thereby reducing the influence thereof.

Returning to FIG.
Next, the regression equation calculation unit 2034 gives weights that increase as the index value increases to the coordinates on the captured image of the dotted image, and calculates a regression equation of the regression line indicating the distribution of the dotted image (step) SC4). A known method may be used as the weighting method. Based on this weighting, the regression equation calculation unit 2034 relatively reduces the influence of the point image having a relatively high possibility of noise, and calculates a regression equation indicating the distribution of the point image. FIG. 21B shows an array line L (shown by a broken line) set by the first array line setting unit 2032 and a regression line Lr (shown by a solid line) indicated by the regression equation calculated by the regression equation calculation unit 2034. This shows the relationship. As shown in FIG. 21B, with respect to the array line L set by the first array line setting unit 2032, the regression line Lr indicated by the regression equation calculated by the regression equation calculation unit 2034 is the original setting standard. The point may not pass, and the slope of the straight line may not be θ (here, θa).

Then, the second array line setting unit 2035 sets an array line based on this regression equation (step SC5). Here, for example, the second array line setting unit 2035 corrects the array line L so as to match the regression line indicated by the regression equation Lr. As a result, an array line that matches the regression line Lr shown in FIG. As a result, the corrected array line passes through the new setting reference point, and the inclination also changes to θa.
Here, the second array line setting unit 2035 matches the array line with the regression line indicated by the regression equation, but at least the slope of the array line is brought close to the slope of the regression line, the regression line and the coordinate axis The array line may be moved so that the set reference point is closer to the intersection of the two.

  Next, the second array line setting unit 2035 updates the array information based on the array line L set in the process of step SC5 (step SC6). Here, since the inclination before correction is θ and the inclination after correction is θa, the second array line setting unit 2035 updates the column direction indicated by the array information from θ to θa. The second array line setting unit 2035 also updates the column spacing based on the array line L, but does not update the start point S as the set reference point.

  Next, the second array line setting unit 2035 determines whether or not the array line setting has been completed (step SC7). If the second array line setting unit 2035 determines “NO” in the process of step SC7, the process proceeds to the process of step SC8. Then, the first array line setting unit 2032 determines the next setting reference point. Here, if the setting reference point has moved in the setting direction by Δm by the process of step SC5, the first array line setting unit 2032 sets the next setting reference point at a position obtained by adding D to the setting reference point after the movement. Set a point.

  When the first array line setting unit 2032 determines the setting reference point, the first array line setting unit 2032 returns to the process of step SC2 and executes the above processing steps again. However, after that, the first array line setting unit 2032 updates the column interval indicated by the array information in the process of step SC7. Specifically, the first array line setting unit 2032 has an interval between columns indicated by the distance on the axis between the array line and the array line adjacent to the start point S (that is, the distance in the arrangement direction of a plurality of columns). As described above, the column interval indicated by the array information is updated.

FIG. 23 is a diagram illustrating a state in which the setting reference point is changed by correcting the array line and the column interval is updated.
First, a setting reference point B is set at a position of a column interval D from a setting reference point A in a certain array line, and an array line passing through the setting reference point B is set. Then, it is assumed that the set reference point B1 moves in the setting direction by Δm1 as a result of the correction of the array line to become the set reference point Ba (FIG. 23B). At this time, the second array line setting unit 2035 updates D + Δm1, which is the distance between the setting reference point A and the corrected setting reference point Ba, as a new column interval. Next, the next set reference point C is generated at the position of the column interval D + Δm1 from the set reference point Ba (FIG. 23C), and an array line passing through the set reference point C is set. Then, it is assumed that the set reference point C is moved in the setting direction by Δm2 as a result of the correction of the array line, and becomes the set reference point Ca (FIG. 23D). At this time, the second array line setting unit 2035 updates D + Δm2, which is the distance between the set reference point Ba and the corrected set reference point Ca, to a new column interval. Thereafter, the second array line setting unit 2035 updates the column spacing in the same procedure.

  The array line setting unit 203 executes the above processing steps, and sequentially sets array lines in the setting direction from the start point S. When the end of the captured image is reached, the array line setting unit 203 changes the setting direction and executes the above processing steps to set the array line. When all the array lines are set, the array line setting unit 203 stores information indicating the set array lines in the memory, and ends the processing relating to the array line setting.

According to the function of the array line setting unit 203 described above, even when the distortion of the captured image increases toward the end of the captured image, the array line setting unit 203 adjusts the array information to the distortion. It will be updated sequentially. As a result, even the array line set by the first array line setting unit 2032 is an array line adapted to some extent to the influence of this distortion. Then, the second array line setting unit 2035 corrects this array line based on the regression equation and sets a final array line, so that the dot-like image on the captured image is compared with the case where this correction is not performed. An array line that more accurately reflects the array of is set.
Here, the case where the array information is updated so as to increase the column interval is illustrated, but the column interval is updated in the same procedure when the column interval is decreased.

FIG. 24 is a diagram illustrating an example of array lines set by the array line setting unit 203. In FIG. 24, the solid line drawn so as to pass through the position of the dot image is the array line.
The second array line setting unit 2035 dynamically sets the array lines in order from the start point S while correcting the column direction and the column interval according to the actual position of the point image. Accordingly, as illustrated in FIG. 24, the array line setting unit 203 sets array lines having different column directions and column intervals at each position in the image even when the captured image is distorted. In addition, although not particularly illustrated, the second array line setting unit 2035 sets the intersection number at the intersection where the array lines intersect each other. Here, the second array line setting unit 2035 assigns intersection numbers that do not overlap each other to each intersection and stores them in the memory. This intersection number realizes not only the function of identifying each intersection, but also the function of representing the coordinates on the second coordinate plane.

By the way, the second coordinate plane on which the array line is set by the array line setting unit 203 has a different coordinate system from the first coordinate plane. The reason is that the start point determined by the start point determination unit 202 is determined, and the coordinate system extends in the column direction of the array information with the start point as the origin. Therefore, in the second coordinate plane, the X-axis is inclined by θ1 relative to the first coordinate plane and the Y-axis is inclined by θ2 relative to the first coordinate plane, and the respective axis directions coincide with the arrangement direction of the point images. Or almost match.
The function of the array line setting unit 203 has been described above.

<D. 2.4 Bit Array Generation Unit 204>
Next, the function of the bit array generation unit 204 will be described.
The bit array generation unit 204 generates digital data associated with the coordinates of the intersections based on the arrangement of the dot images corresponding to the intersections of the array lines set by the second array line setting unit 2035. Here, the bit array generation unit 204 sets “1” when the dot image exists, depending on whether the dot image exists corresponding to the intersection of the array lines. Digital data with “0” when it does not exist is generated. Further, the bit array generation unit 204 has a value (that is, a bit value) indicated by digital data corresponding to each coordinate at a position corresponding to the intersection number (that is, the coordinate on the second coordinate plane) of each intersection. Array data (hereinafter referred to as “bit array”) 400 is generated. Here, the bit array generation unit 204 generates a bit array in which values indicated by the digital data are two-dimensionally arrayed in correspondence with the coordinate system of the second coordinate plane. That is, the bit array generation unit 204 is an example of a generation unit of the present invention.

  FIG. 25 is a diagram showing the configuration of the bit array 400. As shown in FIG. In the bit array 400 shown in FIG. 25, bit values having different positions in the horizontal direction on the paper surface have different X coordinates on the second coordinate plane. In the bit arrangement, bit values having different positions in the vertical direction on the paper surface have different Y coordinates on the second coordinate plane. That is, in the bit array 400, the position where the bit value is arranged and the coordinate on the second coordinate plane, that is, the position of the intersection of the array lines has a one-to-one correspondence. That is, according to the bit array 400, it is uniquely specified from which intersection each bit value is obtained.

<D. 2.5 Reliability Array Calculation Unit 205>
Next, the function of the reliability array calculation unit 205 will be described.
The reliability array calculation unit 205 uses the bit array generation unit 204 based on the correspondence between the array line for each column set by the array line setting unit 203 and the dotted image detected corresponding to the column. The reliability that is the degree of reliability of the generated digital data is calculated. Here, the correspondence relationship is the distance between the dotted image and the array line in each column. The reliability array calculation unit 205 increases the reliability of the digital data as the distance from the dot image to the array line is smaller. Then, the reliability array calculation unit 205 calculates a reliability array 500, which is data in which values indicating reliability are arranged at positions corresponding to the coordinates of each intersection. Here, the reliability array calculation unit 205 generates a reliability array 500 in which the reliability is two-dimensionally arranged in correspondence with the coordinate system of the second coordinate plane. That is, the reliability array calculation unit 205 is an example of the reliability calculation means of the present invention.

  FIG. 26 is a diagram showing a configuration of the reliability array 500. As shown in FIG. In the reliability array 500 shown in FIG. 26, the position where the reliability value is arranged corresponds to the coordinate on the second coordinate plane, that is, the position of the intersection of the array lines on a one-to-one basis. That is, in the reliability array 500, it is uniquely specified from which intersection the value of each reliability is obtained. Here, in the reliability array 500 and the bit array 400, those having the same position in each array have the same intersection position where the values are calculated.

By the way, the reliability has five levels from “0” to “4”, and the reliability is higher as the distance between the dot-like image and the array line in each column becomes smaller. The fact that the point image is actually arranged at the place where the point image where the intersection is located is that the accuracy of the imaging result is good or the arrangement line setting is more faithful. This is because it is presumed that the reliability of the digital data corresponding thereto is high. On the other hand, it is presumed that the more point-like images that are farther from the intersection, the greater the possibility of noise, and the reliability of the corresponding digital data decreases as the distance increases.
As described above, the correspondence between the digital data obtained from the captured image and its reliability is specified by referring to the bit array 400 and the reliability array 500. In the bit array 400 and the reliability array 500, the values are two-dimensionally arranged. However, the arrangement is merely an example, and may be arranged one-dimensionally, for example.

<D. 3 Functions of Decoding Unit 300>
Next, functions of the decoding unit 300 will be described. The decoding unit 300 is an example of the decoding unit of the present invention, and realizes a function for decoding information from a captured image based on the bit array 400 and the reliability array 500. FIG. 27 is a functional block diagram showing a more detailed functional configuration of the decoding unit 300. As shown in FIG. 27, the function of the decoding unit 300 is divided into a code pattern boundary detection unit 301, a code detection unit 302, an identification information decoding unit 303, and a position code decoding unit 304.

Based on the bit arrangement 400, the code pattern boundary detection unit 301 detects the boundary of the unit code pattern that forms the pattern block P2.
The code detection unit 302 detects the second unit code pattern by referring to each pattern block P2 included in the bit array 400 based on the boundary detected by the code pattern boundary detection unit 301 from the bit array 400. The code detection unit 302 detects the direction of the bit array 400 based on the detected rotation angle of the second unit code pattern and corrects the direction.

The identification information decoding unit 303 acquires the identification information code with reference to the position of the code indicated by the second unit code pattern, and detects the identification information by decoding it. The decoding of the identification information uses the same parameters as the code parameters (RS code parameters) used for encoding.
The position code decoding unit 304 acquires the position code with reference to the position of the code indicated by the second unit code pattern, and detects the position information by decoding it. Since the position code is detected as an M-sequence partial sequence in this embodiment, the position information is acquired by specifying the position of the detected partial sequence in the same M-sequence as the image generation process.

  By the way, the decoding unit 300 gives priority to an area having higher reliability in the reliability array 500 and refers to the area of the bit array 400 corresponding to that area. Specifically, the decoding unit 300 preferentially uses and decodes data at a position determined according to the reliability in the bit array 400. For example, the decoding unit 300 selects only digital data whose reliability is equal to or higher than a threshold value (here, “3”), such as regions Rc1 and Rc2 surrounded by a rectangle in FIG. Regions Rc1 and Rc2 correspond to the bit values of regions Rb1 and Rb2 shown in FIG. Then, the decoding unit 300 decodes information from the selected digital data according to the mutual positional relationship of the point images. The decoding unit 300 may preferentially select and decode digital data generated from a position having a smaller distance from the start point S on the second coordinate plane. In the example of FIG. 26, the decoding unit 300 selects the digital data corresponding to the region Rc1 with priority over the region Rc2. The reason for adopting this mode has already been explained, but the determination of the start point S indicating the specific start point of the positional relationship is determined in order to obtain a more reliable imaging result.

  When the decoding unit 300 decodes the position information and the identification information in the above procedure, the data processing unit 61b of the electronic pen 60 transmits the decoded information to the PC 10 by controlling the communication unit 66. The PC 10 executes processing such as receiving the position information and the identification information and converting the handwritten contents into electronic data.

As described above, the electronic pen 60 mainly implements the following three functions so as to suppress the possibility that information is not correctly decoded from the imaging result of an encoded image composed of point images.
(1) Based on the accuracy of the detected point image and the position of the point image on the captured image, priority is given to the one with higher accuracy, and one of the point images is determined as the start point.
(2) At the time of setting the array line, an index value that is an index of the probability that each dot image constitutes an encoded image is calculated, and the weight of the dot image having a large index value is relatively set An array line is set using a weighted regression equation.
(3) For each piece of digital data decoded from the captured image, a degree of reliability corresponding to the correspondence between the array line and the point image is calculated, and the digital data with higher reliability is used preferentially to decode the information. .
When the electronic pen 60 realizes the above functions (1) to (3), imaging is performed due to dirt adhering to the medium 50 or an imaging process when the electronic pen 60 images the surface of the medium 50. Even if an image includes a defect, there is a possibility that information is not correctly decoded from the imaging result of an encoded image composed of dot-like images, compared to the case where the functions (1) to (3) are not realized. Is suppressed.

[Modification]
The present invention may be implemented in a form different from the above-described embodiment. Moreover, you may combine each of the modification shown below.
[Modification 1]
In the above-described embodiment, the accuracy calculation unit 201 calculates the accuracy based on the number of dot images arranged in the adjacent region, but this may be replaced from the following (a) to (d). .
(A) The accuracy calculation unit 201 may increase the accuracy as the shape of the point image is closer to a predetermined shape. As described above, since the shape of the dot image constituting the encoded image is determined, if the shape is different from the original one, it is noise or the encoded image is not formed correctly. Because it is possible.
(B) The accuracy calculation unit 201 may increase the accuracy as the size of the point image is closer to the predetermined size. As described above, since the size of the dot image constituting the encoded image is determined, if the size is different from the original size, it is noise or the encoded image is not formed correctly. Because it is possible.
(C) The accuracy calculation unit 201 may increase the accuracy as the density (for example, optical density) of the point image is closer to a predetermined density. As described above, since the density of the dot image constituting the encoded image is determined, if the density is different from the original one, it is noise or the encoded image is not formed correctly. Because it is possible.
(D) The accuracy calculation unit 201 may increase the accuracy as the density difference between the dot image and the density of the surface of the medium 50 is closer to a predetermined density difference. When the surface density of the medium 50 is different, the density difference is determined to be a certain value. However, if this deviation is large, the encoded image is not formed as expected due to insufficient toner, This is because it is considered that the surface is soiled.
Further, the accuracy calculation unit 201 may calculate the accuracy by combining a plurality of conditions according to the above (a) to (d). In addition, the accuracy calculation unit 201 may combine the accuracy calculation method of the above-described embodiment and the accuracy calculation method according to the characteristics of the point images according to the above (a) to (d).

[Modification 2]
In the above-described embodiment, the index value calculation unit 2033 converts the content of the above (a) to (d) or a plurality of combinations thereof when calculating an index value in which a point image forms an encoded image. The index value may be calculated based on the index value. This is because the index value is also an index of the probability that the dotted image constitutes the encoded image, and may be a standard equivalent to that of the accuracy. Further, the index value calculation unit 2033 may combine the calculation method of the index value according to the characteristics of these point images and the calculation method of the above-described embodiment.

[Modification 3]
In the above-described embodiment, the reliability array calculation unit 205 performs the above-described (a) to (d) or a combination of the plurality of combinations when calculating the reliability in which the dotted image constitutes the encoded image. The reliability may be calculated in consideration of the characteristics of various point images. This reliability is also an index of the probability that the dotted image constitutes the encoded image, and may be a standard equivalent to the index value. If an item common to the calculation of the index value and the reliability is included, the reliability array calculation unit 205 uses the index value calculated by the index value calculation unit 2033 as it is and calculates this as the reliability. May be.

[Modification 4]
In addition, the reliability array calculation unit 205 determines that the density of an image within a range determined based on the position of a certain point-like image on the captured image is the array line as the correspondence between the point-like image and the array line. The reliability may be increased as the density is determined accordingly. Since the dot images are arranged according to a predetermined rule, the image density within the range determined with reference to a certain dot image is originally the arrangement line setting mode (in other words, the arrangement in the encoded image). It should be decided according to the aspect of the area. If the image density is higher than necessary, it is estimated that there is a lot of noise in the surrounding area. If the image density is too low, there is a possibility that the encoded image is not correctly formed due to insufficient toner. Because.

[Modification 5]
In the embodiment described above, the electronic pen 60 may realize the functions related to the accuracy calculation unit 201 and the start point determination unit 202, and may perform decoding using other functions instead of the functions related to the array line setting unit 203 and the subsequent steps. . For example, the electronic pen 60 may set an array line by a method disclosed in Japanese Patent Application Laid-Open No. 2009-181347.
If the function related to the setting of the array line in the array line setting unit 203 is not realized, the arrangement of the dotted images constituting the encoded image may be variously modified as long as it is a regular array. It is done.

[Modification 6]
In the above-described embodiment, the array line setting unit 203 is configured to sequentially set array lines in the setting direction from the start point S. In short, the array line setting unit 203 is any of the dot images on the captured image. Compared to the conventional configuration in which at least the index value is not taken into consideration, on the second coordinate plane with such position as the origin, if an array line indicating the array of point images is set for each column based on the regression equation It is considered that the possibility of decoding failure can be suppressed. In addition, when the index value calculation unit 2033 does not hinder the calculation of the index value, the function corresponding to the first array line setting unit 2032 may be omitted.

[Modification 7]
In the above-described embodiment, an encoded image of an encoding method that generates a bit array according to the presence or absence of a dot image at each intersection of arrayed lines on a captured image is used. The system may be another encoding system. For example, an encoded image of the encoding method disclosed in Patent Document 2 may be applied to the present invention.

[Modification 8]
In the above-described embodiment, dot images are arranged along two different directions in the encoded image. However, an encoded image in which dot images are arranged along three or more directions is used in the present invention. You may apply.
In addition, some or all of the functions realized by the electronic pen 60 of the above-described embodiment may be realized by a device other than the electronic pen 60 such as the PC 10.

  In the above-described embodiment, the first coordinate plane is an orthogonal coordinate system, but may be a skew coordinate system, for example. In addition, although each axis direction is determined according to the arrangement information on the second coordinate plane, the axis setting method may be other than this, as long as the arrangement line passes through the axis. Depending on the definition of each coordinate system, when the array of dot images arranged linearly on the medium 50 is represented on the coordinate plane, the arrangement on the captured image does not become linear, and the arrangement is accordingly performed. There may be a case where the line is not straight. However, since this is only a difference in expression on the coordinate plane, a coordinate plane described in various coordinate systems may be applied to the present invention.

In the above-described embodiment, the start point determined by the start point determination unit 202 is matched with the origin of the second coordinate plane. However, the start point may be shifted from the origin on the second coordinate plane. Even in this case, the array line setting unit 203 may set the array lines sequentially in the setting direction using the start point.
Moreover, each function which the control part 61 of the electronic pen 60 mentioned above implement | achieves is implement | achieved by the combination of a some program, or may be implement | achieved by cooperation of a some hardware resource.

DESCRIPTION OF SYMBOLS 10 ... PC, 100 ... Sequence information detection part, 101 ... Imaging data acquisition part, 102 ... Noise removal part, 103 ... Point image detection part, 104 ... 1st image pair detection part, 105 ... 2nd image pair detection part, 106: Column direction detection unit, 107: Column interval detection unit, 200 ... Decoding unit, 201 ... Accuracy calculation unit, 202 ... Start point determination unit, 203 ... Array line setting unit, 2031 ... Array information acquisition unit, 2032 ... First Array line setting unit, 2033 ... index value calculation unit, 2034 ... regression equation calculation unit, 2035 ... second array line setting unit, 204 ... bit array generation unit, 205 ... reliability array calculation unit, 300 ... decoding unit, 301 ... Code pattern boundary detection unit 302 ... Code detection unit 303 ... Identification information decoding unit 304 ... Position code decoding unit 50 ... Media 60 ... Electronic pen 61 ... Control unit 61a ... Image processing unit 61b ... Data processing , 62 ... pressure sensor, 63 ... illumination unit, 64 ... imaging unit, 66 ... communication unit, 70 ... imaging unit.

Claims (8)

In a captured image that is an image of a medium in which a dot image, which is a dot image, is arranged in a regularly arranged region and an encoded image representing information is formed by the mutual positional relationship of the dot images. On the basis of the setting means, an array line that connects the positions corresponding to the region and indicates the array of the dot image,
Generating means for generating digital data associated with the coordinates of the intersection based on the arrangement of the dotted images detected corresponding to the intersection of the array lines set by the setting means;
Reliability calculation means for calculating the reliability, which is the reliability of the digital data generated by the generation means, based on the correspondence between the set array line and the detected point image;
Preferentially using the digital data with higher reliability calculated by the reliability calculation means, and decoding means for decoding information according to the positional relationship specified from the captured image. Information processing device. The generating means generates array data that is data in which values indicated by the digital data are arrayed at positions corresponding to the coordinates of the intersections,
2. The decoding means preferentially uses digital data included in a region having a higher reliability in the array indicated by the array data, and decodes information according to the positional relationship. The information processing apparatus described in 1. The correspondence is a distance between the intersection and the detected point image corresponding to the intersection,
The information processing apparatus according to claim 2, wherein the reliability calculation unit increases the reliability as the distance is smaller. The reliability calculation means uses the density of an image within a range determined with reference to the position of the detected point image on the captured image as the correspondence, and the density is determined according to the array line The information processing apparatus according to claim 2, wherein the reliability is increased as the value approaches. The reliability calculation means calculates the reliability by taking into account the shape, size, and density of the detected dot image, the density difference between the dot image and the medium surface, or a combination thereof. The information processing apparatus according to claim 3 or 4, characterized by the above. The decoding means uses the detected point-like image as a starting point when specifying the positional relationship from the captured image, and the digital data generated from a position having a smaller distance from the point-like image The information processing apparatus according to claim 1, wherein information is decoded according to the positional relationship. An information processing apparatus according to any one of claims 1 to 6,
Irradiating means for irradiating light;
Imaging means for imaging according to the reflected light of the light irradiated by the irradiation means,
The imaging device, wherein the setting unit sets the array line based on a dot image detected from a captured image obtained by imaging the medium by the imaging unit. Computer
In a captured image that is an image of a medium in which a dot image, which is a dot image, is arranged in a regularly arranged region and an encoded image representing information is formed by the mutual positional relationship of the dot images. On the basis of the setting means, an array line that connects the positions corresponding to the region and indicates the array of the dot image,
Generation means for generating digital data associated with the coordinates of the intersection based on the arrangement of the dotted images detected corresponding to the intersection of the array lines set by the setting means before;
Reliability calculation means for calculating the reliability, which is the reliability of the digital data generated by the generation means, based on the correspondence between the set array line and the detected point image;
A program for preferentially using the digital data having a higher reliability calculated by the reliability calculation means to function as a decoding means for decoding information according to the positional relationship specified from the captured image.

Images