JP4485989B2 - Information acquisition apparatus, information acquisition method, information acquisition program, and recording medium - Google Patents

Description

  The present invention relates to an information acquisition device, an information acquisition method, an information acquisition program, and a recording medium, and in particular, an information acquisition device, an information acquisition method, and an information acquisition program for acquiring association information associated with image data based on the image data. And a recording medium.

  As represented by the digital watermark technique, conventionally, a technique for embedding arbitrary information in a document image in a form that cannot be easily seen by humans has been studied. For example, in Patent Document 1, a combination of blank lengths before and after a word is used as one code unit, and the ratio of the lengths before and after is changed according to the bit value of signature data to protect copyright. A method for embedding proof data is shown.

  Patent Document 2 discloses a method of embedding information by quantizing the ratio between PS and P + S using the character spacings P and S before and after.

  Further, Patent Document 3 discloses a method of embedding information with a predetermined number of blank size patterns.

As described above, Patent Documents 1, 2, and 3 all embed and extract information using an interval of character elements or the like (hereinafter referred to as “character interval”).
JP 09-186603 A JP 2002-232679 A JP 2004-023563 A

  However, when information is embedded using the length of the character spacing, the character element may move or expand in an analog processing process such as printing of a document image in which information is embedded and scanning of the printed matter. As a result, there is a problem that the character elements are connected to each other and the length of the character interval cannot be extracted correctly.

  Such a problem can be said to general techniques for embedding and extracting information from image data regardless of the content or type of the image.

  The present invention has been made in view of the above points, and provides an information acquisition device, an information acquisition method, an information acquisition program, and a recording medium that can appropriately acquire information associated with digital content. Objective.

  Therefore, in order to solve the above-described problem, the present invention provides an information acquisition device that acquires association information associated with at least two types of associated information related to digital content based on the digital content, An associated information reliability determination unit that calculates or determines a relative first reliability of the associated information, an information acquisition unit that acquires contents of the associated information based on the digital content, and Based on the contents of the associated information, estimation means for estimating the association information for each of the associated information, the first reliability for the associated information, and the estimated information by the estimating means. Association information for determining the association information based on an estimation result for each association information And having a determining means.

  In order to solve the above problem, the present invention may be an information acquisition method in the information acquisition apparatus, an information acquisition program for causing a computer to execute the information acquisition method, or a recording medium on which the information acquisition program is recorded. .

  According to the present invention, it is possible to provide an information acquisition device, an information acquisition method, an information acquisition program, and a recording medium that can appropriately acquire information associated with digital content.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a schematic configuration example of an image processing apparatus according to an embodiment of the present invention. In FIG. 1, an image processing apparatus 10 is an apparatus having a function of associating information with image data and a function of acquiring the information from image data associated with the information. The information association unit 11, the printing unit 12, the scan Unit 13, an association information acquisition unit 14, and the like.

  The information association unit 11 associates information with “information regarding image data”. Information associated with “information related to image data” by the information association unit 11 is hereinafter referred to as “association information”. Various pieces of association information, such as character strings, numbers, and bit strings, are targeted.

  The “information about image data” corresponds to, for example, feature values such as pixel values and DCT (Discrete Cosine Transform) coefficients of the image data, image (picture) contents, or image data itself. The association of the association information with the feature amount is performed by, for example, operating (for example, quantizing) the feature amount according to a predetermined rule. Further, the association information is associated with the content of the image (picture) by, for example, operating (for example, quantizing) the content of the image. More specifically, when a character string is included in the image, the association information is associated by quantizing the distance between the characters constituting the character string according to a predetermined rule. Further, the association information is associated by synthesizing a barcode, a two-dimensional code, or the like representing the association information with the image. The association information is associated with the image data itself, for example, by associating and storing a file containing the image data and a file containing the association information.

  Hereinafter, “information relating to image data” that is a target of association information is referred to as “associated information”. The information association unit 11 associates association information with at least two types of associated information for one image data. That is, when exactly the same association information is associated with each associated information, one association information is associated with one image data in a multiple manner.

  In the following description, “associating association information with image data” or “associating association information with image data” means “associating association information with associated information of the image data”.

  The printing unit 12 corresponds to driver software for the printer 20 connected to the image processing apparatus 10 and causes the printer 20 to print the image data associated with the association information as a paper document.

  The scanning unit 13 corresponds to driver software for the scanner 30 connected to the image processing apparatus 10, and causes the scanner 30 to scan a paper document on which image data associated with association information is printed. The obtained information is output as image data.

  The association information acquisition unit 14 acquires association information from at least two types of associated information related to the image data based on the image data output by the scanner unit 13, and outputs the acquired association information.

  The information association unit 11 and the association information acquisition unit 14 will be described in more detail.

  FIG. 2 is a diagram illustrating a functional configuration example of the information association unit. In FIG. 2, the information association unit 11 includes an association information input unit 111, a first association unit 112, a second association unit 113, and the like. The association information input unit 111 acquires association information associated with image data from now on, for example, by allowing the user to input association information or reading it from a predetermined file.

  The first associating unit 112 and the second associating unit 113 associate association information with different pieces of associated information for one image data. The associated information targeted by the first association unit 112 is hereinafter referred to as “first associated information”, and the associated information targeted by the second association unit 113 is hereinafter referred to as “second associated information”.

  FIG. 3 is a diagram illustrating a functional configuration example of the association information acquisition unit. In FIG. 3, the association information acquisition unit 14 includes an associated information reliability determination unit 141, a first estimation unit 142, a second estimation unit 143, an association information determination unit 144, and the like.

  The 1st estimation part 142 and the 2nd estimation part 143 are parts which have a function which estimates correlation information based on 1st correlated information or 2nd correlated information, respectively. Here, “estimate” means that the information derived by the first estimation unit 142 and the second estimation unit 143 is not determined as association information. That is, the content of the image data associated with the association information by the information association unit 11 generally changes as a result of processing such as printing by the printer 20 or scanning by the scanner 30. Changes in the contents of the image also affect the associated information. Therefore, the contents (for example, pixel values, DCT coefficients, etc.) of the associated information when trying to acquire the association information by the association information acquisition unit 14 are related information when the association information is associated by the information association unit 11. It is not necessarily consistent with the contents of, and there is a high possibility that it contains errors. The association information acquired or calculated based on the associated information including such an error is not necessarily a correct value. Therefore, the first estimation unit 142 and the second estimation unit 143 estimate a value that may contain an error as association information for the time being.

  The first estimation unit 142 includes a first information acquisition unit 142a and a first acquisition information reliability determination unit 142b. The first information acquisition unit 142a acquires association information based on the first associated information. The first acquired information reliability determination unit 142b determines or calculates the reliability of the association information acquired by the first information acquisition unit 142a. Here, the reliability of the association information acquired by the first information acquisition unit 142a is an index that indicates the likelihood or probability that the association information is the same as the association information that is actually associated. is there. The first acquired information reliability determination unit 142b is based on the difference between the contents of the associated information acquired by the first information acquiring unit 142a and the contents that the associated information can take by associating the association information. Determine or calculate confidence.

  The second estimation unit 143 includes a second information acquisition unit 143a and a second acquisition information reliability determination unit 143b. These include the first information acquisition unit 142a and the first acquisition information in the first estimation unit 142. This corresponds to the reliability determination unit 142b.

  The associated information reliability determination unit 141 calculates or determines a relative reliability for each of the first associated information and the second associated information. The relative reliability with respect to each of the first associated information and the second associated information is when the association information is finally determined based on the association information estimated by the first estimation unit 142 and the second estimation unit 143. The weighting for each estimation result used in. That is, it is an index indicating which of the two estimation results by the first estimation unit 142 and the second estimation unit 143 is to be trusted more or to the same extent.

  The association information determination unit 144 includes a relative reliability for each of the first associated information and the second associated information calculated or determined by the associated information reliability determination unit 141, the first estimation unit 142, and the second estimation. The association information is determined based on each estimation result by the unit 143. That is, the association information determination unit 144 finally concludes that the association information was like this.

  FIG. 4 is a diagram illustrating a hardware configuration example of the image processing apparatus according to the embodiment of the present invention. 4 includes a drive device 100, an auxiliary storage device 102, a memory device 103, an arithmetic processing device 104, a display device 105, and an input device 106, which are mutually connected by a bus B. And so on.

  A program for realizing processing in the image processing apparatus 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 on which the program is recorded is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. The auxiliary storage device 102 stores the installed program and image data to be processed.

  The memory device 103 reads the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The arithmetic processing device 104 executes functions related to the image processing device 10 in accordance with a program stored in the memory device 103. The display device 105 displays a GUI (Graphical User Interface) or the like by a program. The input device 106 includes a keyboard and a mouse, and is used to input various operation instructions.

  The image processing apparatus 10 may be remotely operated from another terminal by connecting to the network. In this case, the drive device 100, the display device 105, the input device 106, and the like need only be included in the other terminals, and the image processing apparatus 10 does not necessarily have to include.

  The image processing apparatus 10 may be realized using a general-purpose computer such as a PC (Personal Computer). Further, it may be realized by an image processing apparatus called a multi-function machine or a multi-function machine that realizes a plurality of functions such as a printer function and a scanner function by a single casing. When the image processing apparatus 10 is realized by a multifunction machine, the printer 20 and the scanner 30 in FIG. 1 are built in the image processing apparatus 10.

  Hereinafter, the processing of the image processing apparatus 10 will be specifically described by dividing it into first to third embodiments. In the first embodiment, an example will be described in which association information is associated with a natural image by manipulating luminance and color difference, and the association information is acquired from the image. That is, in the first embodiment, the luminance corresponds to the first associated information, and the color difference corresponds to the second associated information.

  First, a process for associating association information with a natural image by manipulating luminance and color difference will be described. FIG. 5 is a flowchart for explaining the information association processing in the first embodiment.

  In step S <b> 101, the information association unit 11 reads image data (hereinafter referred to as “target image”) to be associated with the association information into the memory. Subsequently, the association information input unit 111 causes the user to input association information (S102). The association information may be input by, for example, displaying a dialog having a text input area. Information stored in a file or the like in advance may be used as the association information. In this case, in step S102, the association information is read from the file or the like. The association information may be input as a binary bit string, but may be input in a human-readable format such as a character string.

  Subsequently, the information association unit 11 performs YCbCr conversion on the target image (S103). Subsequently, the first association unit 112 associates association information with the Y (luminance) component of the target image (S104). Specifically, the association information is converted into a bit string, and one bit is assigned to each pixel. The order of assignment may be determined using raster scanning, for example. For example, in a pixel to which a bit value “0” is assigned, the pixel value of the Y component is quantized closest to the original value by an even multiple of a predetermined constant u. Further, in a pixel to which the bit value “1” is assigned, the Y signal is quantized to a value closest to the original value by an odd multiple of the constant u number.

  FIG. 6 is a diagram conceptually illustrating a state in which the association information is associated with the pixel value of the Y component. FIG. 6 shows a state in which the target image 502 is generated by associating the target image 501 with the association information 601 (the bit string “010010110”). Each numerical value in the target image 501 and the target image 502 indicates a pixel value of each Y component. In FIG. 6, the value of the constant u is 10.

  For example, if bit values are assigned to each pixel of the target image 501 in order from left to right and from top to bottom, “0”, “1”, and “0” are assigned to the pixel values 243, 245, and 248, respectively. It is done. Therefore, 243 and 248 assigned “0” are quantized to 240 which is an even multiple of 10 and is closest to the original value. 245 assigned “1” is quantized to a value “250” that is an odd multiple of 10 and is closest to the original value. Similar processing is performed for other pixels.

  Subsequently, the second associating unit 113 associates the association information with the Cb (color difference) component of the target image (S105). In this process, as in the case of associating with the pixel value of the Y component, the bit value of the association information is assigned to each pixel, and each pixel value is quantized to an odd number or an even multiple of a predetermined constant according to the assigned bit value. That's fine.

  Thus, the information association process is completed, and the image data associated with the association information is output from the printer 20 as a paper document by the printing unit 12.

  Next, a process for acquiring association information from image data captured by reading a paper document printed through the above information association process with the scanner 30 will be described. FIG. 7 is a flowchart for explaining the association information acquisition process in the first embodiment.

  In step S151, the association information acquisition unit 14 reads the image data read by the scanner 30 (hereinafter referred to as “target image”) onto the memory. Subsequently, the associated information reliability determination unit 141 determines whether the target image is a color image or a monochrome image (S152), and the first associated information (Y component) and the first image are determined according to the determination result. The relative reliability with respect to the second related information (Cb component) is determined (S153). That is, when the target image is determined to be a monochrome image, the associated information reliability determination unit 141 sets the reliability for the pixel value of the Y component to 1.0, and sets the reliability for the pixel value of the Cb component to 0.0. To do. This is because a monochrome image does not include a color difference component. On the other hand, when it is determined that the target image is a color image, the associated information reliability determination unit 141 sets the reliability of the pixel value of the Y component and the pixel value of the Cb component to 0.5. That is, since a color image has a luminance component and a color difference component, both values are trusted by the same amount.

  Hereinafter, the reliability for the pixel value of the Y component is denoted by Ry, and the reliability for the pixel value of the Cb component is denoted by Rcb.

  Subsequently, the first information acquisition unit 142a acquires the pixel value of the Y component in each pixel of the target image (S154). Subsequently, the first acquired information reliability calculation unit 142b calculates the reliability (high possibility) that the bit value assigned to each pixel is “0” from the pixel value of the Y component of each pixel. Thus, the association information is estimated (S155).

  Specifically, the pixel value of the Y component of each pixel is divided by twice the constant u used at the time of association, and the reliability is calculated from the difference between the quotient and the integer value closest to the quotient. The reason for dividing by twice the constant u is that the pixel value is quantized to an even multiple of the constant u when the bit value “0” is associated. That is, the difference between the quotient when the pixel value is divided by twice the constant u and the integer value closest to the quotient can be expected to be smaller as “0” is associated with the pixel value. Therefore, it can be said that the smaller the difference, the higher the reliability that the bit value associated with the pixel value is “0”.

  For example, it is assumed that the pixel value in the first row of the target image 502 in FIG. 6 changes as a result of printing and scanning, and is acquired as values 242, 251, and 238 in step S 154. Here, a quotient (12.1) obtained by dividing 242 which is a pixel value of an image to which “0” is assigned by twice a constant u (10 × 2 = 20) and an integer value (12) closest to the quotient Is 0.1. On the other hand, the difference between the quotient (12.55) obtained by dividing 251 which is the pixel value of the pixel assigned “1” by twice the constant u (20) and the integer value (13) closest to the quotient is 0. .45.

  Note that it is desirable to calculate the reliability by using a mathematical reliability calculation model, instead of simply determining the reliability based only on the difference. There are various reliability calculation models. For example, the following reliability model (function) can be used based on a Gaussian model.

Here, the meaning of each parameter is as follows.

u: Constant value during quantization μ: Average value of argument x σ: Standard deviation of argument x Confidence is calculated by giving the value obtained by multiplying the difference by u to argument x of the reliability function (1) can do. The reliability that the bit value of the i-th pixel calculated here is “0” is expressed as Ry0 (i).

  Subsequently, the second information acquisition unit 143a acquires the pixel value of the Cb component in each pixel of the target image (S156). Subsequently, the second acquired information reliability calculation unit 143b estimates the association information by calculating the reliability that the bit value associated with each pixel is “0” from the pixel value of the Cb component of each pixel. . (S157). The calculation of the reliability here may be performed in the same manner as in step S155. The reliability with which the bit value of the i-th pixel calculated here is “0” is expressed as Rcb0 (i).

  Subsequently, the association information determination unit 144 determines that the reliability (Ry) for the pixel value of the Y component, the reliability (Rcb) for the pixel value of the Cb component, and the i-th bit value related to the Y component are “0”. Based on a certain reliability Ry0 (i) and a reliability Rcb0 (i) whose i-th bit value related to the Cb component is “0”, the association information with the highest reliability is determined (S158).

  Specifically, the following formula is calculated for each pixel, and the reliability R0 (i) in which the i-th bit value of the association information is “0” is calculated.

R0 (i) = Ry × Ry0 (i) + Rcb × Rcb0 (i)
Here, the association information determination unit 144 determines that the bit value of the i-th pixel is “0” when R0 (i) is 0.5 or more. Otherwise, the association information determination unit 144 determines that the bit value of the i-th pixel is “1”. This is performed for all the pixels associated with the bit value, and the bit string with the highest reliability, that is, the association information is determined. The determination result may be displayed on a display device or stored in a file or the like.

  The association information acquisition process ends here.

  As one of the features of the processing in FIG. 7, the image type is determined in step S152. For a monochrome image, only the luminance component is used for information acquisition, and for a color image, both luminance and color difference are used. Can be mentioned. In the monochrome image, the color difference component is lost, but if the Y component is used, the association information can be acquired even from the monochrome-converted image. Technically, any of the Y component, the Cb component, and the Cr component can be used as associated information, but at least the Y component is preferably used as associated information from the above viewpoint. In addition, it is preferable to use the Cb component because the change of the Cb component tends to reduce the influence on human vision compared to the Cr component. However, the association information may be associated in triplicate using the Cr component.

  Also, the determination of the image type is not limited to monochrome or color. For example, in the case of an image format using a color index, when the number of indexes is large, the reliability for color differences is relatively increased, and when the number of indexes is small. It is also possible to relatively reduce the reliability with respect to the color difference.

  Furthermore, a specific bit string may be embedded together with the association information, the reliability of the specific bit string may be calculated when calculating the reliability of the associated information, and the reliability may be used as the reliability of each associated information. This method will be described in detail in the second embodiment.

  As described above, according to the image processing apparatus 10 in the first embodiment, the association information of the same content (value) is associated with the luminance component and the color difference component twice. Further, when obtaining the association information, the association information with the highest reliability is determined in consideration of the reliability with respect to the luminance component and the color difference component, so that the extraction accuracy of the information associated with the image data can be improved.

  Next, a second embodiment will be described. In the second embodiment, a distance between characters (hereinafter referred to as “character interval length”) with respect to an electronic document (hereinafter simply referred to as “document”) composed of a character string and an image (for example, a photograph) other than the character string. )) And the DCT coefficient will be described in association with the association information in duplicate, and the association information is acquired from the image of the document. That is, in the first embodiment, the character interval length corresponds to the first associated information, and the DCT coefficient corresponds to the second associated information. Here, the “document” may be a document file generated by word processing software or a predetermined application, or image data including a character string as an image.

  First, a process for associating association information doubly by manipulating the character spacing length and the DCT coefficient will be described. FIG. 8 is a flowchart for explaining the information association processing in the second embodiment.

  In step S201, the information association unit 11 reads a document to be associated with association information into a memory. Subsequently, the association information input unit 111 causes the user to input association information (S202). Subsequently, the information association unit 11 divides the document into a text region and other image regions (S203).

  Subsequently, the first association unit 112 associates the related information with the text area (S204). Specifically, the association information is converted into a bit string, and one bit is assigned to each character interval. In the character interval to which the bit value “0” is assigned, the character interval length is quantized to a value closest to the original value by an even multiple of a predetermined constant u. In the character interval to which the bit value “1” is assigned, the character interval length is quantized to a value closest to the original value by an odd multiple of the constant u. Note that the value of the constant u here does not have to be the same as the value of the constant u in the first embodiment.

  FIG. 9 is a diagram conceptually showing how a bit value is associated with a character interval length. FIG. 9 shows how the association information (the bit string “011”) is associated with the character interval lengths l1, l2, and l3 of each character interval of the character string “document image”. Accordingly, the value of the character interval length l1 to which “0” is assigned is quantized to 2u, and the character interval lengths l2 and l3 to which 1 is assigned are quantized to 3u.

  Moreover, the 1st correlation part 112 allocates a specific bit sequence to character space | interval length with the correlation information input by the user. Hereinafter, the specific bit string is also referred to as association information.

  FIG. 10 is a diagram illustrating an example of association information including a specific bit string. FIG. 10 shows an example in which a specific bit string (“0101”) is added to the head of the association information input by the user. As will be described later, the specific bit string is used as information for determining the reliability of the associated information (here, the character interval length).

  Subsequently, the second association unit 113 associates association information with the divided image regions (S205). Specifically, the image area is divided into blocks each composed of a plurality of pixels, and the bit value of the bit string of the association information is assigned to each block bit by bit. Further, DCT is applied to each block, and a specific coefficient among the obtained DCT coefficients is changed to a specific value according to the assigned bit value.

  FIG. 11 is a diagram conceptually showing how a bit value is associated with a DCT coefficient. In FIG. 11, when the block unit is 3 × 3 pixels and “0” is assigned to the block, the value of the DCT coefficient at the center is set to 50, and “1” is assigned. Shows an example in which the value of the DCT coefficient at the center is −50. As the DCT coefficient for changing the value, it is preferable to select a DCT coefficient having little influence on the change of the image.

  Similar to the first associating unit 112, the second associating unit 113 associates information to which the specific bit string is added as association information.

  After changing the DCT coefficient, the second associating unit 113 performs inverse DCT on each block to which the bit value is assigned, and restores the image from the DCT coefficient.

  This completes the information association process. The document associated with the association information is output from the printer 20 as a paper document by the printing unit 12.

  Next, a process for acquiring association information from image data captured by reading a paper document printed through the above information association process with the scanner 30 will be described. FIG. 12 is a flowchart for explaining association information acquisition processing in the second embodiment.

  In step S <b> 251, the association information acquisition unit 14 reads image data read by the scanner 30 (hereinafter referred to as “target image”) on the memory. Subsequently, the association information acquisition unit 14 divides the target image into two parts, a text area and an image area (S252). When a document is printed once and scanned, the entire image becomes an image. Even at this time, the image can be identified and divided using a document structure analysis technique, and divided into a text area and an image area.

  Subsequently, the first information acquisition unit 142a acquires each character interval length of the character string included as an image in the text area divided from the target image (S253). Subsequently, the first acquired information reliability calculation unit 142b estimates the association information by calculating the reliability that the bit value assigned to each character interval length is “0” from each character interval length (S254). ). The calculation of the reliability may be performed in the same manner as in the first embodiment. That is, the difference between the character interval length acquired by the first information acquisition unit 142a and the value (quantization point) that can be obtained by quantizing the character interval length to which “0” is assigned is represented by a reliability function ( What is necessary is just to give to the argument x of 1). Here, in the association information, the reliability that the i-th bit of the specific bit string is “0” is expressed as Rcsp (i), and the i-th bit value of the information input to the user is “0”. The reliability is expressed as Rcs0 (i).

  Subsequently, the second information acquisition unit 143a acquires a DCT coefficient to which a bit value is assigned from the divided image area (S255). That is, the image area divided from the target image is divided into blocks each composed of a plurality of pixels, DCT is applied to each block, and among the obtained DCT coefficients, a DCT coefficient to which a bit value is assigned (the center in FIG. 11). DCT coefficient).

  Subsequently, the second acquired information reliability calculation unit 143b estimates the association information by calculating the reliability in which the bit value assigned to each DCT coefficient is “0” from each acquired DCT coefficient ( S256). For example, when the bit value is assigned by changing the DCT coefficient to ± u / 2 (FIG. 11 corresponds to the case of u = 100), the bit value “0” is set in the argument x of the reliability function (1). The reliability may be calculated by giving the absolute difference between the value representing the value (50 in FIG. 11) and the DCT coefficient acquired by the second information acquisition unit 143a. However, when the obtained DCT coefficient exceeds u / 2, the reliability of the bit value “0” is calculated as 1, and when the acquired DCT coefficient is lower than −u / 2, the reliability of the bit value “0” is calculated as 0. Here, the reliability that the i-th bit value of the specific bit string is “0” is expressed as Rdctp (i), and the reliability that the i-th bit value in the association information input to the user is “0”. Indicated as Rdct0 (i).

  Subsequently, the associated information reliability determination unit 141 determines the reliability for each based on the specific bit string associated with each of the first associated information (character interval length) and the second associated information (DCT coefficient). Determination is made (S257). That is, the average value of the reliability with which each bit value of the specific bit string is 0 is set as the reliability for the associated information.

  Here, the reliability for the character spacing length is Rcs, and the reliability for the DCT coefficient is Rdct. For example, when “0101” is a specific bit string, Rcs can be calculated as follows.

Rcs = {Rcsp (0) + (1-Rcsp (1)) + Rcsp (2) + (1-Rcsp (3))} / 4
Rdct can be calculated similarly.

  Further, in order to set the sum of Rcs and Rdct to 1.0, each is normalized by the following calculation.

Rcs = Rcs / (Rcs + Rdct)
Rdct = Rdct / (Rcs + Rdct)
Subsequently, the association information determination unit 144 determines the reliability (Rcs) for the character interval length, the reliability (Rdct) for the DCT coefficient, and the reliability Rcs0 (i-th bit value for the character interval length is “0”). Based on i) and the reliability Rdct0 (i) whose i-th bit value related to the DCT coefficient is “0”, the association information with the highest reliability is determined (S258).

  Specifically, the following formula is calculated for each character interval length or each DCT coefficient, and the reliability R0 (i) in which the i-th bit value of the association information is “0” is calculated.

R0 (i) = Rcs × Rcs0 (i) + Rdct × Rdct0 (i)
Here, the association information determination unit 144 determines that the i-th bit value is “0” when R0 (i) is 0.5 or more. Otherwise, the association information determination unit 144 determines that the i-th bit value is “1”. This is performed for all character interval lengths and DCT coefficients associated with bit values, and the bit string with the highest reliability, that is, the association information is determined.

  This is the end of the association information acquisition process.

  As described above, according to the image processing apparatus 10 in the second embodiment, related information is associated by a method suitable for each region (for each text region or image region) constituting a document, and multiple associations are associated. Since information is integrated and association information with high reliability is determined, information extraction accuracy can be improved. Further, since the reliability of each associated information is calculated using the specific bit string, a more reliable result can be obtained.

  By the way, the number of character intervals and the number of image blocks vary depending on the document, and the number of bits that can be associated varies. However, in many applications, the association information is required to have a fixed bit length. Therefore, when the number of bits of the association information is small, the same information is repeatedly associated, and when the number of bits of the association information is large, the association is made as much as possible. It may be set to 5.

  Further, when the number of bits of the association information is larger than the number of bits of the associated information (in the second embodiment, the number of character intervals and the number of blocks for obtaining the DCT coefficient), each of the two types of associated information If the bit values of the association information are assigned in the same order, the last bit value cannot be assigned. In such a case, the associated information amount may be increased by assigning one associated information in order from the first bit value and assigning the other associated information in order from the last bit value.

  Next, a third embodiment will be described. In the third embodiment, a predetermined graphic pattern is synthesized with image data, and a file containing image data (image file) and a file containing association information are stored in association with each other in duplicate. An example in which information is associated and association information is acquired from the image will be described. Here, a two-dimensional code is used as the predetermined graphic pattern. That is, in the third embodiment, the two-dimensional code corresponds to the first associated information, and the image data itself (image file) corresponds to the second associated information. However, the predetermined graphic pattern may be other than a two-dimensional code such as a barcode as long as it can express information.

  First, association processing of association information will be described. FIG. 13 is a flowchart for explaining the information association process in the third embodiment.

  In step S <b> 301, the information association unit 11 reads image data (hereinafter referred to as “target image”) to be associated with association information into a memory. Subsequently, the association information input unit 111 causes the user to input association information (S302).

  Subsequently, the first association unit 112 generates a two-dimensional code based on the association information, and superimposes the two-dimensional code on the image of the image data (S303). For example, a QR (Quick Response) code may be used as the two-dimensional code.

  FIG. 14 is a diagram illustrating an example of generating a two-dimensional code from association information. 14A shows association information, and FIG. 14B shows a two-dimensional code generated based on the association information. In FIG. 14, the bit string of the association information (A) is two-dimensionally arranged. When the bit value is 0, the bit string is white (colorless) rectangular area (hereinafter referred to as “cell”), bit. When the value is (1), a two-dimensional code (B) is generated by arranging black cells.

  Subsequently, the second associating unit 113 stores the image data and the association information as an image file or an association information file, respectively, by a method capable of maintaining the corresponding relationship. The association between the two files may be performed based on the file names of both, or may be performed by saving both in the same folder.

  Thus, the information association process is completed, and the image data associated with the association information is output from the printer 20 as a paper document by the printing unit 12.

  Next, a process for acquiring association information from image data captured by reading a paper document printed through the above information association process with the scanner 30 will be described. FIG. 15 is a flowchart for explaining association information acquisition processing according to the third embodiment.

  In step S351, the association information acquisition unit 14 reads image data read by the scanner 30 (hereinafter referred to as “target image”) into the memory. Subsequently, the associated information reliability determination unit 141 calculates a relative reliability with respect to the first associated information (two-dimensional code) and the second associated information (image file) (S352). The reliability here may be calculated based on the type of image as in the first or second embodiment, or may be calculated by using association of a specific bit pattern. Each may be set to 0.5. Hereinafter, the reliability for the two-dimensional code is described as Rcode, and the reliability for the image file is described as Rmatch. Note that normalization is performed so that the sum of Rcode and Rmatch is 1.0.

  Subsequently, the first information acquisition unit 142a cuts out a two-dimensional code from the target image (S353). Subsequently, the first acquired information reliability calculation unit 142b has a reliability in which each bit value of the association information associated with the two-dimensional code is “0” based on information obtained by decoding the extracted two-dimensional code. Is estimated by calculating (S354). When a QR code is used as a two-dimensional code, since error correction coding is performed, the reliability is either 1.0 or 0.0 depending on whether decoding is possible. When a two-dimensional code without error correction is used, for example, the ratio of white or black pixels when the two-dimensional code is divided into cells may be used as the reliability.

  FIG. 16 is a diagram illustrating an example of calculating reliability with a bit value of 0 based on the ratio of white pixels when a two-dimensional code is divided into cells.

  In FIG. 16, (A) shows the two-dimensional code cut out from the target image by the first information acquisition unit 142a. The shape of the two-dimensional code (A) is deteriorated through processing such as printing and scanning. (B) shows a state in which the two-dimensional code (A) is divided for each cell to which a bit value is assigned. (C) shows the ratio of white pixels in each cell divided in (B) as a numerical value. Therefore, according to FIG. 16, it can be determined that the reliability of the first bit value being 0 is 0.89, the next bit value is 0.01, and thereafter, 098, 0.91,.

  Hereinafter, the reliability that the i-th bit obtained from the two-dimensional code is “0” is expressed as Rcode0 (i).

  Subsequently, the second information acquisition unit 143a acquires an image file stored in association with the association information file (S355). Here, when a plurality of image files are stored in association with each association file, the plurality of image files are acquired.

  Subsequently, the second acquired information reliability calculation unit 143b determines the reliability that each bit value of the association information associated with the image file is “0” based on the similarity between the target image and the acquired image file. The association information is estimated by calculating. Specifically, the second acquired information reliability calculation unit 143b selects an image file similar to the target image by pattern matching between the image file acquired by the second information acquisition unit 143a and the target image. The image file selected here is hereinafter referred to as “selected image”. Here, a plurality of image files may be selected. For example, a selection criterion such as one similar to the fifth one or a similarity degree equal to or higher than a threshold value may be provided. Further, the second information acquisition unit 143a acquires an association information file associated with the selected image, and the similarity between each selected image and the target image is associated with the association information included in each association information file. Reliable.

  The reliability Rmatch0 (i) having the i-th bit value of “0” can be calculated from the reliability for each association information. For example, the following may be performed.

  When the similarity of the m-th selected image is S (m) and the i-th bit of the association information associated with the m-th selected image is 0, the i-th bit value of the m-th association information is The reliability of 0 is Rmatch0 (m, i) = S (m). When the i-th bit of the m-th association information is 1, Rmatch 0 (m, i) = 0. Rmatch0 (m, i) is calculated for all selected images, and the average value for each bit is defined as Rmatch0 (i).

  The similarity S (m) is normalized in advance as follows so that the total similarity of selected images is 1.

S (m) ← S (m) / ΣS (i) {i is the selected image}
Subsequently, the association information determination unit 144 determines the reliability (Rcode) for the two-dimensional code, the reliability (Rmatch) for the image file, and the reliability Rcode0 in which the i-th bit value related to the two-dimensional code is “0”. Based on (i) and the reliability Rmatch0 (i) whose i-th bit value related to the image file is “0”, the association information with the highest reliability is determined (S357).

Specifically, first, the following equation is calculated to calculate the reliability R0 (i) in which the i-th bit value of the association information is “0”.
R0 (i) = Rcode × Rcode0 (i) + Rmatch × Rmatch0 (i)
Here, the association information determination unit 144 determines that the i-th bit value is “0” when R0 (i) is 0.5 or more. Otherwise, the association information determination unit 144 determines that the i-th bit value is “1”. This is performed for all the bit values, and the bit string with the highest reliability, that is, the association information is determined.

  This is the end of the association information acquisition flow.

  Note that a tampering determination step may be further provided after the association information is determined. The i-th bit value of the acquired association information is set to b (i), the following equation is calculated, and the reliability R of the entire acquired association information is calculated.

R = Σ {R0 (i) + b (i) × (1-2 × R0 (i))} / number of bits When the reliability R is equal to or less than a predetermined threshold value, it is determined that there has been falsification. This is effective in an environment where there is little variation in the pixel value of the image. If the reliability R is low in such an environment, it can be said that there is a high possibility that the image has been rewritten or the two-dimensional code has been overwritten.
This makes it possible to detect tampering.

  The reliability R corresponds to an average value of reliability with respect to the bit value of each bit. That is, when the i-th bit value is 0 (that is, when b (i) = 0), {R0 (i) + b (i) × (1-2 × R0 (i))} in the above formula is R0 (I), which indicates the confidence that the i-th bit value is 0. On the other hand, when the i-th bit value is 1 (that is, when b (i) = 1), {R0 (i) + b (i) × (1-2 × R0 (i))} in the above formula is 1 −R0 (i), which indicates the confidence that the i-th bit value is 1. Therefore, the average value of the reliability of the bit value of each bit being the bit value becomes the value of reliability R.

  As described above, according to the image processing apparatus 10 in the third embodiment, since the association information associated with multiple is integrated and the highly reliable association information is determined, the information extraction accuracy is increased. Can do.

  In the third embodiment, the example in which the image data itself is stored as a file has been described. However, the image data characteristics are considered in consideration of the calculation speed of image matching and the storage capacity for storing the image file. The amount may be stored in association with the association information file. As the feature amount of the image data, a pixel value statistic, a coefficient value obtained by Fourier transform, DCT, wavelet transform, or the like can be used. If the image data is converted from a document including text, the layout, the number of characters, the character spacing value, and the like can be used as the feature amount.

  As described above in the first to third embodiments, according to the present invention, the related information is not limited to one information, and the related information suitable for the image data that is the target of the related information is Since it is possible to select and associate information, the durability of the association information can be enhanced, and for example, the association information is appropriately associated with various digital contents other than image data such as audio, video, document, and program. That association information can be acquired.

  In addition to those described in the first to third embodiments, as related information, for example, combinations of RGB components, character elements (height, width, inclination, thickness, typeface, interval) , Combinations of different coefficient areas when pixel values are integrated (eg, combination of high frequency and low frequency in Fourier transform), dot pattern, combination of barcode and the element, and combination of image matching and the element It is possible to select various information depending on the situation.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to the specific embodiment which concerns, In the range of the summary of this invention described in the claim, various deformation | transformation * It can be changed.

It is a figure which shows the example of schematic structure of the image processing apparatus in embodiment of this invention. It is a figure which shows the function structural example of an information correlation part. It is a figure which shows the function structural example of an association information acquisition part. It is a figure which shows the hardware structural example of the image processing apparatus in embodiment of this invention. It is a flowchart for demonstrating the information correlation process in 1st embodiment. It is a figure which shows notionally associating association information with the pixel value of a Y component. It is a flowchart for demonstrating the correlation information acquisition process in 1st embodiment. It is a flowchart for demonstrating the information correlation process in 2nd embodiment. It is a figure which shows notionally a mode that a bit value is linked | related with a character space | interval length. It is a figure which shows the example of the correlation information containing a specific bit sequence. It is a figure which shows notionally a mode that a bit value is linked | related with a DCT coefficient. It is a flowchart for demonstrating the correlation information acquisition process in 2nd embodiment. It is a flowchart for demonstrating the information correlation process in 3rd embodiment. It is a figure which shows the example of a production | generation of the two-dimensional code from association information. It is a flowchart for demonstrating the correlation information acquisition process in 3rd embodiment. It is a figure which shows the example which calculates the reliability whose bit value is 0 by the ratio of the white pixel at the time of dividing a two-dimensional code into cells.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Information correlation part 12 Printing part 13 Scan part 14 Association information acquisition part 100 Drive apparatus 101 Recording medium 102 Auxiliary storage apparatus 103 Memory apparatus 104 Arithmetic processing apparatus 105 Display apparatus 106 Input apparatus 111 Association information input part 112 Association unit 113 Second association unit 141 Associated information reliability determination unit 142 First estimation unit 142a First information acquisition unit 142b First acquisition information reliability determination unit 143 Second estimation unit 143a Second information acquisition unit 143b Second acquisition Information reliability determination unit 144 Association information determination unit B bus

Claims (35)

An information acquisition apparatus that acquires association information associated with at least two types of associated information regarding digital content based on the digital content,
Associated information reliability determination means for calculating or determining a relative first reliability of each associated information;
Information acquisition means for acquiring the contents of the associated information based on the digital content;
Estimating means for estimating the association information for each piece of associated information based on the content of each piece of the associated information acquired;
An association information determination unit that determines the association information based on the first reliability with respect to each of the associated information and an estimation result for each of the associated information by the estimation unit. Information acquisition device. The estimation unit is configured to perform second estimation on the estimation result based on a difference between the content of the associated information acquired by the information acquisition unit and the content of the associated information associated with the association information. Calculate confidence,
The determination means is based on the first reliability with respect to each associated information, the estimation result for each associated information by the estimation means, and the second reliability with respect to the estimation result, The information acquisition apparatus according to claim 1, wherein association information is determined. 3. The information acquisition apparatus according to claim 2, wherein whether or not the digital content has been tampered with is determined based on the second reliability with respect to the estimation result calculated by the estimation unit. The associated information reliability determination unit compares the part of the association information with predetermined data for each of the association information estimated by the estimation unit, so that the first information for each associated information is obtained. The information acquisition apparatus according to claim 1, wherein the reliability is calculated. The associated information reliability determination unit compares the bit information of the association information with a predetermined bit string for each of the association information estimated by the estimation unit, thereby determining the associated information related to the associated information. 5. The information acquisition apparatus according to claim 4, wherein one reliability is calculated. The said associated information reliability determination means calculates said 1st reliability with respect to each said associated information based on the characteristic of the said digital content, The Claim 1 thru | or 5 characterized by the above-mentioned. Information acquisition device. The digital content is image data,
The information acquisition means acquires a pixel value of a luminance component and a color difference component of each pixel of the image data,
The information acquisition apparatus according to claim 1, wherein the estimation unit estimates the association information for each of the luminance component and the color difference component based on each acquired pixel value. . The information acquisition apparatus according to claim 7, wherein the luminance component and the color difference component are in a YCbCr component. The estimation means includes a difference between a pixel value of the luminance component and a pixel value of the color difference component acquired by the information acquisition means, and a pixel value that can be taken by the luminance component and the color difference component associated with the association information. The information acquisition apparatus according to claim 7, wherein a second reliability for the estimation result is calculated based on the information. The digital content is image data,
The information acquisition means acquires a character interval length in a character string included as an image in the image data and a DCT coefficient based on the image data,
The said estimation means estimates the said correlation information for every said character space | interval length and said DCT coefficient based on the acquired said character space | interval length and said DCT coefficient. The information acquisition device described in the section. The information acquisition apparatus according to claim 10, wherein the information acquisition unit calculates the DCT coefficient for each of a plurality of pixels of the image data, and acquires a predetermined DCT coefficient for each of the plurality of pixels. The estimation means is based on a difference between the character interval length and the DCT coefficient acquired by the information acquisition means, and a value that can be taken by the character interval length and the DCT coefficient associated with the association information. The information acquisition apparatus according to claim 10 or 11, wherein a second reliability for the estimation result is calculated. The digital content is image data,
The information acquisition means acquires a graphic pattern combined with the image data, stored image data stored in a predetermined storage area, or a feature amount of the stored image data,
The estimation unit is configured to determine the graphic pattern and the stored image data or the feature amount based on the acquired graphic pattern and the stored image data or the external data stored in association with the outside of the feature amount. The information acquisition apparatus according to claim 1, wherein the association information is estimated every time. The estimation unit is configured to store the stored image data or the stored image data based on a similarity between the stored image data or the feature amount acquired by the information acquisition unit and the image data or the feature amount of the image data. The information acquisition apparatus according to claim 13, wherein a second reliability for the external data associated with the feature amount is calculated. An information acquisition method for acquiring association information associated with at least two types of associated information regarding digital content based on the digital content,
An associated information reliability determination procedure for calculating or determining a relative first reliability of each associated information;
An information acquisition procedure for acquiring the contents of the associated information based on the digital content;
An estimation procedure for estimating the association information for each piece of associated information based on the content of each piece of the associated information acquired;
An association information determination procedure for determining the association information based on the first reliability with respect to each of the associated information and an estimation result for each of the associated information in the estimation procedure. Information acquisition method. The estimation procedure is based on a difference between the content of the associated information acquired in the information acquisition procedure and the content that the associated information associated with the association information can take. Calculate confidence,
The determination procedure is based on the first reliability for each associated information, the estimation result for each associated information in the estimation procedure, and the second reliability for the estimation result. The information acquisition method according to claim 15, wherein association information is determined. 17. The information acquisition method according to claim 16, wherein whether or not the digital content has been tampered with is determined based on the second reliability with respect to the estimation result calculated in the estimation procedure. In the associated information reliability determination procedure, for each association information estimated in the estimation procedure, by comparing a part of the association information with predetermined data, the first associated with each associated information The information acquisition method according to claim 15, wherein the reliability is calculated. In the associated information reliability determination procedure, for each of the association information estimated in the estimation procedure, a part of the bit string of the association information is compared with a predetermined bit string, whereby the associated information reliability determination step is performed. The information acquisition method according to claim 18, wherein one reliability is calculated. 20. The associated information reliability determination procedure calculates the first reliability for each associated information based on characteristics of the digital content. Information acquisition method. The digital content is image data,
The information acquisition procedure acquires a pixel value of a luminance component and a color difference component of each pixel of the image data,
21. The information acquisition method according to claim 15, wherein the estimation procedure estimates the association information for each of the luminance component and the color difference component based on each acquired pixel value. . The information acquisition method according to claim 21, wherein the luminance component and the color difference component are in a YCbCr component. The estimation procedure includes a difference between a pixel value of the luminance component and a pixel value of the color difference component acquired in the information acquisition procedure, and a pixel value that can be taken by the luminance component and the color difference component associated with the association information. 23. The information acquisition method according to claim 21, wherein a second reliability for the estimation result is calculated based on the estimation result. The digital content is image data,
The information acquisition procedure acquires a character interval length in a character string included as an image in the image data, and a DCT coefficient based on the image data,
The said estimation procedure estimates the said correlation information for every said character space | interval length and said DCT coefficient based on the acquired said character space | interval length and said DCT coefficient, It is any one of Claim 15 thru | or 20 characterized by the above-mentioned. Information acquisition method described in the section. The information acquisition method according to claim 24, wherein the information acquisition procedure calculates the DCT coefficient for each of a plurality of pixels of the image data and acquires a predetermined DCT coefficient for each of the plurality of pixels. The estimation procedure is based on a difference between the character interval length and the DCT coefficient acquired in the information acquisition procedure and a value that the character interval length and the DCT coefficient associated with the association information can take. 26. The information acquisition method according to claim 24 or 25, wherein a second reliability for the estimation result is calculated. The digital content is image data,
The information acquisition procedure acquires a graphic pattern combined with the image data, stored image data stored in a predetermined storage area, or a feature amount of the stored image data,
The estimation procedure is based on the acquired graphic pattern and the stored image data or the external data stored in association with the outside of the feature amount, and the graphic pattern and the stored image data or the feature amount. 21. The information acquisition method according to claim 15, wherein the association information is estimated every time. The estimation procedure includes the storage image data or the stored image data based on a similarity between the stored image data or the feature amount acquired in the information acquisition procedure and the image data or the feature amount of the image data. 28. The information acquisition method according to claim 27, wherein a second reliability for the external data associated with the feature amount is calculated. An information acquisition program for causing a computer to acquire association information associated with at least two types of associated information related to digital content based on the digital content,
An associated information reliability determination procedure for calculating or determining a relative first reliability of each associated information;
An information acquisition procedure for acquiring the contents of the associated information based on the digital content;
An estimation procedure for estimating the association information for each piece of associated information based on the content of each piece of the associated information acquired;
An association information determination procedure for determining the association information based on the first reliability with respect to each of the associated information and an estimation result for each of the associated information in the estimation procedure. Information acquisition program. The estimation procedure is based on a difference between the content of the associated information acquired in the information acquisition procedure and the content that the associated information associated with the association information can take. Calculate confidence,
The determination procedure is based on the first reliability for each associated information, the estimation result for each associated information in the estimation procedure, and the second reliability for the estimation result. 30. The information acquisition program according to claim 29, wherein association information is determined. 31. The information acquisition program according to claim 30, wherein whether or not the digital content has been tampered with is determined based on the second reliability with respect to the estimation result calculated in the estimation procedure. In the associated information reliability determination procedure, for each association information estimated in the estimation procedure, by comparing a part of the association information with predetermined data, the first associated with each associated information 32. The information acquisition program according to claim 29, wherein the reliability is calculated. In the associated information reliability determination procedure, for each of the association information estimated in the estimation procedure, a part of the bit string of the association information is compared with a predetermined bit string, whereby the associated information reliability determination step is performed. The information acquisition program according to claim 32, wherein one reliability is calculated. The said associated information reliability determination procedure calculates said 1st reliability with respect to each said associated information based on the characteristic of the said digital content, The Claim 29 thru | or 33 characterized by the above-mentioned. Information acquisition program. 35. A computer-readable recording medium on which the information acquisition program according to any one of claims 29 to 34 is recorded.