JP4746663B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention, for example, creates composite image information by embedding and synthesizing other additional sub information (security information, etc.) in the invisible state with respect to the main image information (human face image, etc.) in the visible state. The present invention relates to an image processing apparatus and an image processing method used in an image processing system that records generated composite image information on a recording medium and restores sub-information embedded therein from the recorded composite image information.

  In recent years, with the digitization of information and the spread of the Internet, techniques such as digital watermarks and digital signatures have come to be regarded as important in order to prevent forgery and alteration of images. In particular, digital watermark technology that embeds additional sub-information (sub-image information) in the main image information in an invisible state is a measure against illegal copying, counterfeiting, and falsification of personal authentication media such as ID cards and photographs embedded with copyright information. Proposed.

For example, there is known a digital watermark insertion method for embedding data in image data output to a printed matter by using characteristics of high spatial frequency components and color difference components that are difficult for humans to detect (see, for example, Patent Document 1). ).
Also, a digital watermark printing apparatus that can be confirmed with an optical filter is known (see, for example, Patent Document 2).

From the viewpoint of the recording device, conventionally, when recording face images for personal authentication on personal authentication media represented by various identification cards such as licenses and membership cards, a sublimation thermal transfer recording system is used. Is common.
In general, the sublimation type thermal transfer recording method has a drawback that the material that can be dyed with a sublimable material is limited and can be applied only to a limited recording medium. For this reason, since the degree of freedom in selecting a recording medium for recording a face image for personal authentication in the personal authentication medium is low and it is necessary to select a recording medium that is easily available as a result, security is often lowered. In general, sublimation dyes are inferior in image durability such as light resistance and solvent resistance.

On the other hand, in the melt-type thermal transfer recording system, it is possible to select a coloring material that is generally said to have good light resistance. Further, the degree of freedom of selection of the recording medium is high, and the security can be improved by using a recording medium with high specificity. However, in the melt type thermal transfer recording method, since the dot area gradation method is used in which gradation recording is performed by changing the size of the transferred dots, it is difficult to achieve gradation performance equivalent to that of sublimation thermal transfer recording.
As a countermeasure, for example, there is known a method of recording the arrangement of transfer dots in a so-called zigzag pattern (hereinafter referred to as an alternating drive recording method) (see, for example, Patent Document 3).
Japanese Patent Laid-Open No. 9-248935 JP 2001-268346 A Japanese Patent Publication No. 6-59739

  As described above, there has recently been a demand for improving the security of the face image itself by applying the digital watermark technology to the recording of a face image for personal authentication on a personal authentication medium. Because digital data is presumed to be handled, the gradation performance is high when recording with an image recording device (color printer, etc.), and the watermark information is destroyed or altered due to the recording effect. It is required that there is nothing.

  Patent Document 3 discloses a recording method for improving the gradation recording performance in the melt-type thermal transfer recording method. However, when face image information in which watermark information is embedded using digital watermark technology is recorded, a staggered pattern is obtained. Since the data is thinned out and the information in the portion is lost, the digital watermark information is destroyed.

In addition, since the digital watermark technology corresponding to printing includes digital-analog conversion and analog-digital conversion in the process from embedding of sub-information to restoration, an essential problem that image deterioration cannot be avoided. There is. Therefore, a technique for suppressing the influence of deterioration when restoring the sub information is essential.
To deal with this problem, Patent Document 1 describes a method for predicting the degree of deterioration in advance and increasing the embedding strength according to the degree, but this method has a risk of exposing sub information. There is a problem that it increases.

  In view of this, the present invention is based on the composite image information when creating composite image information in which other additional sub-information is embedded in the main image information in an invisible state for analog data to be output to a recording medium. It is an object of the present invention to provide an image processing apparatus and an image processing method capable of improving the restoration property and visibility of embedded sub information without increasing the risk of exposure of sub information.

The image processing apparatus according to the present invention includes main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state with the human naked eye, and key information used when restoring the sub-information. The composite image information created by embedding the sub-information in the main image information in an invisible state by performing an electronic watermark embedding process using a color difference modulation method using the image is recorded from the recording medium in which the composite image information is recorded in the visible state. Image input means for reading and inputting image information, and the size of the key information is expanded and contracted based on the resolution of the composite image information recorded on the recording medium and the reading resolution of the image input means, and then the key information The Fourier transform is applied to the frequency domain, and the filter passband is adjusted by referring to the expanded value. Frequency component values obtained by performing the inverse Fourier transform to the frequency filter coefficients, extracts a spatial frequency component of the key information from the composite image information input by said image input means using the frequency filter coefficients for Extraction means, and reconstruction means for reconstructing the sub information from the spatial frequency components extracted by the frequency component extraction means.

The image processing apparatus according to the present invention is used to restore main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state to the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state Image input means for reading and inputting the composite image information, color component information storage means for storing color component information, and composition input by the image input means based on the color component information stored in the color component information storage means Color component extraction means for extracting color components from the image information, and spatial frequency components specific to the key information are extracted from the color components extracted by the color component extraction means. A frequency component extracting means for, and a reconstruction means for reconstructing the sub-information from the spatial frequency component extracted by the frequency component extracting means.

The image processing apparatus according to the present invention is used to restore main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state to the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state An image input means for reading and inputting the composite image information, and a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used for embedding from the composite image information input by the image input means a region extracting means for extracting a color feature amount extraction hand for extracting a color feature in the local area from the local area extracted by the area extraction means A color composition unit that generates color component composite image information by combining color components based on the color feature amount extracted by the color feature amount extraction unit, and a color component composite image information generated by the color composition unit. Frequency component extraction means for extracting a spatial frequency component unique to the key information, and reconstruction means for reconstructing the sub-information from the spatial frequency component extracted by the frequency component extraction means.

The image processing apparatus according to the present invention is used to restore main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state to the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state An image input means for reading and inputting the composite image information, and a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used for embedding from the composite image information input by the image input means a region extracting means for extracting a color feature amount extraction hand for extracting a color feature in the local area from the local area extracted by the area extraction means A reconstruction parameter determination unit that determines a reconstruction parameter based on the color feature amount extracted by the color feature amount extraction unit, and a spatial frequency specific to the key information from the composite image information input by the image input unit Frequency component extraction means for extracting components, and reconstruction means for reconstructing the sub information from the spatial frequency components extracted by the frequency component extraction means using the reconstruction parameters determined by the reconstruction parameter determination means. It has.

The image processing method of the present invention is used when main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state with the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state An image input step of reading and inputting the composite image information by an image input means, and the size of the key information is expanded and contracted based on the resolution of the composite image information recorded on the recording medium and the reading resolution in the image input step After that, the key information is expanded in the frequency domain by performing Fourier transform, and the expanded value is referred to by referring to the expanded value. Adjust the passband of the filter, the values obtained by performing the inverse Fourier transform and the frequency filter coefficients for values after the adjustment, the synthetic image input by the image input step by using the frequency filter coefficient A frequency component extracting step for extracting a spatial frequency component specific to the key information from the information by the frequency component extracting means, and a reconfiguration for reconstructing the sub information by the reconstructing means from the spatial frequency components extracted by the frequency component extracting step. Configuration steps.

The image processing method of the present invention is used when main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state with the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state An image input step of reading and inputting the composite image information by the image input means, and a color component extraction means by the color component extraction means from the composite image information input by the image input step based on the color component information stored in the color component information storage means a color component extracting step of extracting, said by the frequency component extracting means from the color component extracted by the color component extracting step It has and a reconstruction step of reconstructing the frequency component extraction step of extracting a specific spatial frequency components, the sub-information by the reconstruction means from the spatial frequency component extracted by the frequency component extracting step to the information.

The image processing method of the present invention is used when main image information that is visible to the human naked eye, sub-information for embedding the main image information in an invisible state with the human naked eye, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state Corresponding to the fundamental wavelength of the spatial frequency component of the key information used when embedding by the region extraction means from the composite image information input by the image input step by reading and inputting the composite image information by the image input means a region extraction step of extracting as a local area regions of the number of pixels, a local area extracted by the area extracting step A color feature extraction step by al color feature extracting unit for extracting a color feature in the local area, by combining the color components by color synthesizing means based on the color feature extracted by the color feature extraction step A color composition step for generating color component composite image information, and a frequency component extraction step for extracting a spatial frequency component specific to the key information by frequency component extraction means from the color component composite image information generated by the color composition step; A reconstruction step of reconstructing the sub information from the spatial frequency component extracted by the frequency component extraction step by a reconstruction means .

Furthermore, the image processing method of the present invention is used to restore main image information visible to the naked eye of human eyes, sub-information for embedding the main image information invisible to the naked eye of human eyes, and the sub-information. From a recording medium on which composite image information created by embedding the sub-information in the main image information in an invisible state by performing digital watermark embedding processing using a color difference modulation method using key information is recorded in a visible state Corresponding to the fundamental wavelength of the spatial frequency component of the key information used when embedding by the region extraction means from the composite image information input by the image input step by reading and inputting the composite image information by the image input means a region extraction step of extracting as a local area regions of the number of pixels, a local territory extracted by the area extracting step From the color feature extraction step of extracting a color feature in the local region by the color feature extracting unit, the reconstruction parameters by the reconstruction parameter determining means based on the color feature extracted by the color feature extraction step A reconstruction parameter determination step for determining, a frequency component extraction step for extracting a spatial frequency component specific to the key information by frequency component extraction means from the composite image information input in the image input step, and the reconstruction parameter determination step A reconstruction step of reconstructing the sub information by the reconstruction means from the spatial frequency component extracted by the frequency component extraction step using the reconstruction parameter determined by

  According to the present invention, when generating composite image information in which additional sub information is embedded in the main image information in an invisible state for analog data to be output to a recording medium, the composite image information is used. It is possible to provide an image processing apparatus and an image processing method capable of improving the reconstructability and visibility of the embedded sub information without increasing the risk of exposure of the sub information.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the first embodiment will be described.
FIG. 1 schematically shows the overall configuration of the image processing system according to the first embodiment. This image processing system shows a case where it is applied to, for example, processing of a face image for personal authentication in a personal authentication medium such as an ID card. The first image processing apparatus 100 for recording the composite image information created by embedding the sub information (sub image information) in the invisible state on the recording medium M, and the first image processing apparatus 100 The second image processing apparatus 110 restores the sub-information embedded in the composite image information recorded on the recording medium M.

  The first image processing apparatus 100 captures the face image of the person who owns the personal authentication medium and converts it into digital image information. The main image input unit 101 as an image input unit and the face captured by the main image input unit 101 A first pre-processing unit 102 and a first pre-processing unit as first pre-processing means for converting image information (hereinafter also referred to as main image information) into a shape suitable for pixel formation processing of the recording unit 106 to be described later The second pre-processing unit 103 as a second pre-processing unit that converts the image information converted in 102 into a shape suitable for digital watermark embedding processing, and the image information converted by the second pre-processing unit 103 A digital watermark embedding processing unit 104 as an embedding unit that embeds digital watermark information using key information, and performs a reverse process of the second pre-processing unit 103 on the image information in which the digital watermark is embedded, and will be described later. The post-processing unit 105 as an inverse conversion means for returning to a shape suitable for the pixel formation processing of 06, and the composite image information in which the digital watermark is embedded in the recording medium M based on the image information converted by the post-processing unit 105 In the following, the function of each unit will be described in detail.

  The main image input unit 101 digitizes personal face image information by inputting a face image of the person who owns the personal authentication medium with a camera or capturing a face photograph with an image input device such as a scanner. . The main image information (face image information) at this time is composed of three planes of R (red), G (green), and B (blue).

  The first pre-processing unit 102 to the post-processing unit 105 perform embedding of a digital watermark into the main image information and conversion into a shape suitable for the pixel forming process of the recording unit 106. Hereinafter, the flow of these processes will be described with reference to the flowchart shown in FIG.

  First, the first preprocessing unit 102 performs first preprocessing corresponding to the pixel formation processing of the recording unit 106 on the main image information captured by the main image input unit 101, Pre-processed main image information is created (step S201). Here, in the first preprocessing, thinning (invalidation) processing is performed on the main image information.

  Next, the second pre-processing unit 103 performs geometric change processing on the first pre-processed main image information created in step S201 to create embedded image information (step S202). Here, a rotation process is performed on the first preprocessed main image information, and the effective image size is compressed by removing the pixel portion thinned out in the first preprocess.

  Next, the digital watermark embedding processing unit 104 performs digital watermark embedding processing on the embedded image information (second preprocessed main image information) created in step S202 (step S203). Here, composite image information is created in which the sub-information is embedded in the invisible state so that the sub-information cannot be perceived by human vision.

  Next, post-processing unit 105 performs post-processing on the composite image information created in step S203, thereby creating recorded image information (step S204). Here, reverse rotation processing is performed on the composite image information, and the pixel portion removed in the second preprocessing in step S202 is added to expand the effective image size.

  The digital watermark embedding process is not limited to R (red), G (green), and B (blue) data, and color conversion described later is performed first, and C (cyan), M (Magenta) and Y (yellow) 3 plane data may be embedded.

  The recording unit 106 creates a personal authentication medium by printing and recording the recorded image information created by the post-processing unit 105 on a recording medium M serving as a personal authentication medium. Specifically, first, R (red), G (green), and B (blue) of each pixel of the recorded image information are color-converted into C (cyan), M (magenta), and Y (yellow) for recording. As a color conversion method, a 3 × 3 or 3 × 9 color conversion matrix or LUT (Look Up Table) is used in accordance with the characteristics of the recording device. Next, a drive signal for controlling the recording device is generated from the C, M, and Y image information. For example, in the case of the melt type thermal transfer recording method, a drive voltage control signal, a drive pulse signal, etc. for the thermal head are generated. The thermal control of the thermal head is also performed here. Finally, composite image information is recorded on the recording medium M by alternately forming even-numbered pixels and odd-numbered pixels in the main scanning direction of a recording device typified by a thermal head for each recording line.

  The arrangement of dots formed on the recording medium M is as shown in FIG. Looking at the line AA ′ in FIG. 3, the dots are not arranged every other dot, but are arranged in a line in the 45 ° direction in a state where they are adjacent at a pitch d (1 / √2 of the pitch of the heating element of the thermal head). You can see that

  FIG. 4 shows an example of a personal authentication medium 401 such as an ID card created by the recording unit 106. A personal authentication face image 402 of the owner is recorded on the personal authentication medium 401. This face image 402 is created and printed and recorded by the process described with reference to FIG. In addition, personal management information 403 such as an identification number (No.), name, date of birth, and expiration date is also recorded. Since these personal management information 403 is used as sub-information for the digital watermark embedding process in step S203 in FIG. 2, the authentication face image 402 of the personal authentication medium 401 and the personal management information 403 are associated with each other. It becomes difficult to falsify or forge a part of 401, and security can be improved.

  For example, the second image processing apparatus 110 reads and inputs the composite image information 403 recorded on the personal authentication medium 401 shown in FIG. 4 and converts it into digital image information. A restoration unit 115 as restoration means for restoring digital watermark information (sub-information) from the combined image information captured by the recorded image input unit 111, and the personal authentication medium based on the digital watermark information restored by the restoration unit 115 It is comprised by the determination part 114 as a determination means to determine the authenticity of each, and hereafter, the function of each part is demonstrated in detail.

  The recorded image input unit 111 takes in the composite image information 403 recorded on the personal authentication medium 401 with an image input device such as a camera and converts it into digital composite image information. The image information at this time is composed of three planes of R (red), G (green), and B (blue) as in the case of the main image input unit 101.

  The restoration unit 115 is extracted by the frequency component extraction unit 112 as a frequency component extraction unit that extracts the spatial frequency component of the key information from the composite image information captured by the recorded image input unit 111 and the frequency component extraction unit 112. It is configured by a reconstruction unit 113 as reconstruction means for reconstructing digital watermark information (sub-information) from spatial frequency components.

  The frequency component extraction unit 112 performs frequency filtering on the composite image information captured by the recorded image input unit 111, and uses the amplitude (= intensity) information and phase information of the frequency component of the embedded key information. Take out.

  The reconstructing unit 113 performs aggregation processing and spatial filtering on the amplitude information and phase information of the spatial frequency component extracted by the frequency component extracting unit 112, and reconstructs the modulation component, that is, sub-information on the key information.

  The determination unit 114 collates the sub-information (personal management information) restored by the restoration unit 115 with the personal management information 403 on the personal authentication medium 401 captured by the recording image input unit 111, and the personal authentication medium 401 Determine the authenticity of.

  Next, an alternate drive recording method according to the present invention, specifically, a melt type thermal transfer recording method in which dots are arranged in a zigzag manner will be described. In the melt type thermal transfer recording method in which an image is formed with or without dots, when a multi-tone image is expressed, the apparent density is controlled by performing an area modulation process that changes the area of the dots. For this reason, it is required to accurately modulate the dot size, and for this purpose, an alternate drive recording method is desirable.

  The alternating drive recording method is a method in which the odd-numbered heating elements of the odd-numbered lines and the even-numbered heating elements of the even-numbered lines of the recording head (line-type thermal head) are alternately driven for each recording line. When driven in this way, the image information to be recorded is stored in a grid pattern as shown in FIG. 5A. In actual recording, the image information is staggered as shown in FIG. 5B. To form an image. Therefore, even-numbered information on odd lines and odd-numbered information on even lines of image information to be recorded are lost.

  This means that even if the sub-information is embedded in an invisible state using digital watermark processing in the image information to be recorded, only half the area of the original image information is valid and other information is lost. Therefore, it means that the digital watermark is destroyed or altered. In general, when such a digital watermark is destroyed, it is very difficult to restore the sub information, and security cannot be maintained.

  Therefore, in the present invention, when the digital watermark embedding process in step S203 is performed, the first preprocess in step S201 and the second preprocess in step S202 are performed, and further, after the digital watermark embedding process in step S203, step S204 is performed. By performing post-processing, the electronic watermark is prevented from being destroyed during alternate drive recording.

  In the first preprocessing (step S201), image information corresponding to pixels to which no energy is applied is thinned out in the alternate drive recording method. FIG. 6A shows the arrangement of the entire image information to be recorded. The black portion 601 corresponds to the pixels to be recorded (information that is not thinned out), and the white portion 602 is the pixels that are not recorded (thinned out). Information).

  In the second preprocessing (step S202), for example, a 45 ° rotation process and a process of removing the thinned information are performed on the arrangement of the image information subjected to the first preprocessing, and an effective image information size is set. Perform compression processing. As shown in FIG. 6B, when the image information array in FIG. 6A is rotated by 45 °, black portions 601 (information not thinned out) are aligned. Therefore, the white portion 602 (the portion to be thinned out) is removed and rearranged to create an array of only image information that is not affected by the alternating drive recording method.

  Furthermore, when the first pre-processing and the second pre-processing are described with reference to the specific example shown in FIG. 7, FIG. 7 (a) shows the arrangement of image information to be recorded. In the first stage, Information on aij (i = 1 to 4, j = 1 to 4) is stored. The thinning process is performed by the first preprocessing, and the even-numbered information and the odd-numbered information of the even-numbered lines in the image information array are thinned out, and the array with x shown in FIG. The element is deleted.

  Next, a 45 ° rotation process is performed in the second pretreatment, and the state shown in FIG. Further, when the elements of valid image information are rearranged after removing the portion marked with x, the state shown in FIG. 7D is obtained. Information that is not recorded (in this case, “0”) is stored in the array element of the vacant gap.

  In FIG. 7A and FIG. 7D, the arrangement size of the image information that is actually recorded or not affected by the alternating drive recording is reduced (the thick frame portion in FIG. 7D). ). By executing the digital watermark embedding process so that the sub information is contained in the thick frame portion of FIG. 7D, the sub information is completely retained.

  With respect to the post-processing (step S204), the above processing is completely reversed.

  In the present embodiment, the melt type thermal transfer recording method is shown as an example. However, the image processing of the present embodiment is applicable to any recording method in which gradation representation is performed by dot area modulation of recording pixels. Is adaptable.

FIG. 8 schematically shows the procedure described so far.
In FIG. 8, the main image information 801 is, for example, face image information for personal authentication, and the sub-information 802 is, for example, information for improving the security of the main image information 801 (this time, the number “174”). Use graphics such as birthdays as images and company logo marks. The key information 803 is key information for later restoring the sub information embedded in an invisible state by the digital watermark embedding process.

  First, embedded image information 804 is created by performing first preprocessing and second preprocessing on the main image information 801. Next, a digital watermark embedding process 805 is performed using the embedded image information 804, the sub information 802, and the key information 803, thereby creating the digital watermarked image information 806. Then, composite image information 807 is created by performing post-processing that performs inverse transformation processing of the first pre-processing and the second pre-processing. Finally, a personal authentication medium 809 is completed by performing a recording (printing) process 808 on the created composite image information 807.

  Next, the digital watermark embedding process will be described. In the present embodiment, the method of digital watermarking can be generally applied, but it is particularly compatible with the method of embedding by superimposing the sub information and the main image information.

Details of this digital watermark embedding processing method are described in, for example, Japanese Patent Application Laid-Open No. 11-168616 and Japanese Patent Application Laid-Open No. 2001-268346, and these can be applied. These methods are basically described on the premise that the main image information is a color (full color) image. However, for example, a technique described in Japanese Patent Laid-Open No. 11-355554 is added to these methods. By applying it, it is possible to embed sub-information (sub-image information) in a black and white image in an invisible state.
If it is necessary to determine the authenticity of the face image of the personal authentication medium, the sub-information recorded in an invisible state is obtained by performing the restoration process described in the above publication using the key information. Is restored.

FIG. 9 is a flowchart of digital watermark embedding processing using the color difference modulation method described in Japanese Patent Laid-Open No. 11-168616, and an application example to this embodiment will be described. In this method,
(1) Utilizing human visual characteristics
・ The gradation discrimination ability decreases as the frequency of the image increases.
-Color difference information is more difficult to distinguish than luminance information
(2) Complementary color relationship Example: Red + cyan color = achromatic color (white) (in case of additive color mixture)
(3) Applying complementary color relationships and color difference information to high-frequency carrier pattern images (color difference modulation processing)
By using the sub information, it is possible to embed the sub information in the main image information in an invisible state without causing image quality deterioration.

  In the example of (2) above, red and cyan (= green + blue) are complementary colors in the case of additive color mixing, and even if red and cyan are adjacent to each other, they are discriminated by the human eye. Difficult to see and looks achromatic.

  As in the example of (3) above, the high-frequency carrier pattern image is used to repeatedly arrange red-rich pixels and cyan-rich pixels. Human visual characteristics that cannot be identified and the color difference amount is determined to be plus or minus “0” are used. Since the composite image information (image with digital watermark) created by this method does not depend on the image format to be stored, it will be changed to a new image format in the future in addition to the currently distributed image formats such as BMP, TIFF, and JPEG. No problem at all.

  Here, the flow of the digital watermark embedding process of Japanese Patent Laid-Open No. 11-168616 shown in FIG. 9 will be briefly described. For details, refer to the description in JP-A-11-168616.

  Embedded image information (main image information) 901 is image information in which embedded information is embedded, and corresponds to an owner's face photograph (face image) in a personal authentication medium. This has 24 bits of information per pixel (8 bits for each of R, G, and B). Embedded image information (sub-information) 902 is obtained by converting embedded information into a binary image, and corresponds to, for example, an identification number or the like in a personal authentication medium. This has 1 bit of information per pixel. Mask image information (key information) 903 is image information used at the time of composition processing and restoration (reproduction) of embedded image information, and has 1-bit information per pixel.

  First, in the smoothing process step 904, the black pixel of the embedded image information 902 is set to “1” and the white pixel is set to “0” to perform the smoothing process. Here, an area of 3 × 1 pixels is extracted from both ends of the target pixel in the x direction, and a weighted average is taken. Next, in the phase modulation processing step 905, phase modulation is performed on the mask image information 903 based on the result of the smoothing processing in the smoothing processing step 904.

  Next, in color difference modulation processing step 907, color difference modulation processing is performed using the color difference amount ΔCd based on the phase modulation result in phase modulation processing step 905. In this case, three components R (red), G (green), and B (blue) are calculated separately. Next, in superimposition processing step 908, composite image information 909 is created by performing superimposition processing from the color difference modulation result in color difference modulation processing step 907 and the embedded image information 901.

  As is apparent from the above description, the embedded image information 901, the embedded image information 902, and the mask image information 903 in FIG. 9 are the main image information 801, the sub information 802, and the key in the present embodiment described in FIG. It is exactly the same as information 803. Therefore, it is obvious that the digital watermark embedding processing method shown in FIG. 9 can basically be applied to this embodiment.

  However, in this embodiment, since the first pre-processing and the second pre-processing are performed on the main image information in advance, the effective image information array size is indicated by the bold line frame in FIG. Thus, it is smaller than the original size of the main image information. Therefore, when the composite image information 909 is created by superimposing the embedded image information 901 ′ and the superimposing image information 910 obtained as a result of the color difference modulation processing, like the digital watermark embedding processing of FIG. The area of the effective portion (in this case, “174”) of the superimposing image information 910 needs to be completely included in the hatched portion of the embedded image information 901 ′.

Regarding the superimposition processing, if the embedded image information 901 ′, the superimposing image information 910, and the composite image information 909 are defined as follows,
Embedded image information: SRC _C (x, y) (A-1)
Superimposing image information: STL _C (x, y) (A-2)
Composite image information: DES _C (x, y) (A-3)
x and y are the coordinate values of the image
C = {R (red), G (green), B (blue)} plane
Each value is an integer value between 0 and 255 for 24-bit color operation.
It is expressed by the following formula.
DES _R (x, y) = SRC _R (x, y)
+ STL _R (x, y) (B-1)
DES _G (x, y) = SRC _G (x, y)
+ STL _G (x, y) (B-2)
DES _B (x, y) = SRC _B (x, y)
+ STL _B (x, y) (B-3)
In the present embodiment, R (red), G (green), and B (blue) are used as the basic primary colors for the calculation in the additive color mixture, but C (cyan) is used as the basic primary color in the calculation in the subtractive color mixture. ), M (magenta), and Y (yellow) are essentially unchanged.

  Next, a sub information restoration process for the composite image information created by the digital watermark embedding process of the above-mentioned JP-A-11-168616 will be described. The restoration of the sub information is performed by extracting a specific spatial frequency component from the synthesized image information based on the key information used in the embedding process, and reconstructing the sub information from the spatial frequency component.

  As key information, binary (monochrome) image information composed of a geometric pattern or the like can be used. For example, a black and white checkered pattern having 1 × 2 pixels as a unit rectangle, a pseudo-random pattern created based on a predetermined seed, or the like.

  As a method for extracting a specific spatial frequency component based on the key information, a spatial frequency filter can be used. The coefficient of the spatial frequency filter corresponding to the key information is calculated by the following procedures (1) to (4). The coefficient may be calculated in advance and the result may be stored, or may be calculated and used before the extraction process or each time.

(1) The size of the key information is expanded or contracted based on the resolution of the main image information, the resolution of the composite image information recorded on the recording medium, and the reading resolution of the recording image input unit 111.
(2) Perform Fourier transform and expand to the frequency domain. The conversion may be performed with an integer, or may be extended to a real number or a complex number.
(3) Refer to the expanded value and adjust the pass band of the filter.
(4) Perform inverse Fourier transform on the adjusted value, and use the obtained value as the frequency filter coefficient.

  Regarding (1) above, for example, when embedding is performed using the key information shown in FIG. 10A, the resolution of the main image information is 200 dpi, the print resolution of the composite image information and the reading of the recorded image input unit 111 If the resolution is 400 dpi, the combined image information captured by the recorded image input unit 111 is as shown in FIG.

  In FIG. 10A, a white circle 1001 represents white, a black circle 1002 represents black, 1003 represents a basic frequency waveform in the main scanning direction, and 1004 represents a basic frequency waveform in the sub-scanning direction. In FIG. 10B, a white circle 1005 represents a main color rich dot, and a black circle 1006 represents a complementary color rich dot. In this case, when the main color is red (R), the complementary color is cyan (C). Reference numeral 1007 denotes a fundamental frequency waveform in the main scanning direction, and reference numeral 1008 denotes a fundamental frequency waveform in the sub-scanning direction.

  The embedded key information has been converted into a shape 1009 shown in FIG. 10B, and the fundamental frequency is equivalent to the frequency when the size of the key information is expanded by the ratio of the reading resolution and the printing resolution. It has become. Therefore, when calculating the filter coefficient, a change in resolution in recording and reading is introduced in advance.

  In (2) to (4) above, a frequency filter that extracts the spatial frequency component of the key information from the synthesized image information is designed. Since the key information is originally binary, the edge (= white pixel and black pixel) It has a feature that the inclination of the boundary (which touches) is steep. The sharper the edge in the spatial domain, the more harmonics are included in the frequency domain. Therefore, using the frequency filter coefficient calculated using image information with many sharp edges as it is, the harmonic side The noise on the screen becomes transparent, and the S / N ratio deteriorates, which hinders the restoration of the sub information.

  For this reason, the adjustment work shown in (3) above is required, but this content depends on the individual key information and the operating environment of the system. In general, in order to suppress noise, harmonics are blocked and only frequencies close to the fundamental frequency are allowed to pass. However, in environments where noise is low, the harmonics are also allowed to pass through to actively increase the complexity of key information. There is also an approach to improve security by using it.

In order to extract the spatial frequency component of the key information from the synthesized image information captured by the recorded image input unit 111 using the frequency filter coefficient calculated in advance by the above method, the convolution integral by the following equation 1 is performed. Do.

  Here, I is the composite image information captured by the recorded image input unit 111, g is a frequency filter coefficient, and K is a spatial frequency component of the extracted key information.

  Note that the method of extracting a specific spatial frequency component is not limited to the method using the spatial frequency filter described above, and uses a well-known Fourier transform, wavelet transform, etc., and once maps to another space for processing. After applying, a method of extracting by mapping in reverse may be used.

Next, a process for reconstructing the sub information from the spatial frequency component extracted as described above will be described. The reconstruction process is performed according to the following procedures (1) to (3).
(1) Extract the zero-cross point (point where the sign changes) from the extracted spatial frequency component.
(2) Project the zero-cross point to obtain the reference phase of the spatial frequency component.
(3) Calculate the deviation from the reference phase for each coordinate of the spatial frequency component, and replace the pixel value of the coordinate that deviates by more than a predetermined threshold value with black, and replace the others with white.

  11 to 13 show schematic diagrams in each process of the reconstruction process. FIG. 11 shows the amplitude of the spatial frequency component of the extracted key information, where the white portion 1101 means the + side and the black portion 1102 means the − side. FIG. 12 shows the result of projecting the coordinates of the points where the sign of + − changes (zero cross points) and totaling the numbers, and extracting the coordinates having a number greater than or equal to a predetermined threshold TH as the zero cross points of the reference phase. Yes. FIG. 13 shows the result of calculating the deviation from the reference phase for the spatial frequency component at each coordinate and replacing the pixel value in accordance with the deviation amount.

  By performing the processing as described above, the sub information can be restored from the composite image information as black and white binary image information.

When composite image information is created by the method disclosed in Japanese Patent Laid-Open No. 2001-268346, sub information (sub image information) is restored in the following procedure.
(1) A specific spatial frequency component Fi is extracted based on predetermined key information Ki (1 ≦ i ≦ N). Here, N is the total number of key information Ki used for embedding sub-information (sub-image information).
(2) A binarization process is performed on the extraction result Fi with a predetermined threshold THi corresponding to the key information Ki.
(3) The above steps (1) to (2) are performed on Nd pieces (Nd ≦ N) of key information necessary for restoration, and the results are synthesized based on a predetermined procedure.

  The method disclosed in Japanese Patent Application Laid-Open No. 2001-268346 can use a plurality of pieces of key information when embedding sub information (sub image information), and further restore sub information (sub image information). It has a feature that a combination of key information necessary at the time can be arbitrarily selected.

  Therefore, when sub-information (sub-image information) is restored from composite image information created using the method, extraction of spatial frequency components corresponding to key information and sub-information (sub-image information) fragments The procedure of reconstruction by binarization is performed a predetermined number of times, and the procedure of synthesizing these fragments is performed.

  Key information, a threshold value, and a synthesis method necessary for restoration are determined in advance at the time of embedding. As for Nd pieces of key information Ki, all of those used at the time of embedding (all pieces of N pieces of key information) may be required, or a predetermined procedure or some randomly selected from among them (Nd with Nd ≦ N).

  The threshold THi may be a value common to all the key information Ki (i = 1,..., N), or may be an individual value for each key information Ki. The synthesis of the pieces of sub information (sub image information) may be a method of linking the pieces vertically, horizontally, or a method of synthesizing fragments by addition or exclusive OR. However, a method in which operations such as concatenation and synthesis are mixed may be used. Further, a predetermined weight may be applied to each fragment before calculation.

  By using the image processing system as described above, it is possible to create and check a personal authentication medium in which a digital watermark is embedded, and to realize a system with higher security than before.

As described above, according to the first embodiment, the additional sub-information (sub-image information) other than the main image information is invisible for analog data that is output to the personal authentication medium. Embedded image information can be created, and digital watermark information in the recorded composite image information can be maintained even after recording.
In addition, in an image recording apparatus using a fusion type thermal transfer recording method, digital watermark technology can be applied to a recorded image while maintaining high gradation performance, and the digital watermark information (sub-information) is not broken after recording. Saved and recoverable.
Next, a second embodiment will be described.
FIG. 14 shows the configuration of the second image processing apparatus 110 according to the second embodiment. The second image processing apparatus 110 includes an image input unit 11 as an image input unit, a color component information storage unit 12 as a color component information storage unit, a color component extraction unit 13 as a color component extraction unit, and a frequency component extraction unit. The frequency component extraction unit 14 as a reconstruction unit, the reconstruction unit 15 as a reconstruction unit, and the display unit 16 as an output unit. The function of each unit will be described in detail below.

  For example, the image input unit 11 takes in the composite image information 403 recorded in the personal authentication medium 401 of FIG. 4 with an image input device such as a camera, and converts it into digital composite image information. This image information is composed of three planes R, G, and B.

  The color component information storage unit 12 stores information on color components used for restoring the digital watermark. Based on the color component information read from the color component information storage unit 12, the color component extraction unit 13 extracts a color component used for restoring the digital watermark from the image information input by the image input unit 11.

  The frequency component extraction unit 104 performs frequency filtering on the color component extracted by the color component extraction unit 13, and outputs amplitude (= intensity) information and phase information about the spatial frequency component of the embedded key information. Take out.

The reconstructing unit 15 performs aggregation processing and spatial filtering on the amplitude information or phase information of the spatial frequency component extracted by the frequency component extracting unit 14, and reconstructs the modulation component, that is, sub-information on the key information.
The display unit 16 displays the sub information reconstructed by the reconstructing unit 15 (or the input image information and sub information may be displayed simultaneously).

  The color component information storage unit 12 stores information on the color component having the highest gradation in advance, for example, the color of ink used for printing such as C, M, and Y, and the optical used when capturing images such as R, G, and B. Information that can identify which color component has the highest gradation among the colors of the filter is stored. This information may be a character string such as a color name, or may be a numerical value such as the wavelength of an electromagnetic wave corresponding to the color, or a number or address assigned for management in the color component information storage unit 12. There may be.

  Hereinafter, a color component having high gradation will be described by taking printer ink as an example. In general, since the composition of ink in a printer varies from color to color, the physical characteristics of the ink also vary from color to color. Since gradation depends on physical characteristics, as a result, gradation varies depending on the ink color.

  For example, in a printer such as a sublimation type that changes the density of ink adhering to a print target by controlling the applied energy, the relationship between the applied energy and the density exhibits characteristics as shown in FIG. In general, the density does not change much between the area where the applied energy is small and the area where the applied energy is large, and the density corresponds to the applied energy in the intermediate area. Gradation is determined by the controllability of the ink density and the density range that can be realized. For the former, when the applied energy and the corresponding density are plotted, the slope becomes gentler, and for the latter the minimum density and maximum density. The larger the difference, the better the gradation.

  For example, FIG. 15 shows applied energy density characteristics of three types of ink, but ink 1 (characteristic A) has a narrow realizable density range, and ink 3 (characteristic C) has low controllability. Of these, ink 2 (characteristic B) exhibits the best gradation.

  The gradation described above is not unique only to the method exemplified as an example. For example, even in a method of changing not the density but the area of the dot as in the melt type, it is different for each ink. The same is true in that the difference in gradation is caused by the difference in physical characteristics.

  In addition, the printer ink has been described above as an example. However, the optical filters used for capturing images also have different physical characteristics such as different pass bandwidths and attenuation rates, so there is a difference in gradation. The point of going out is the same.

  Next, the relationship between digital watermark restoration and gradation will be described. The digital watermark embedded by performing the color difference modulation process can be restored through a process of color component extraction → spatial frequency component extraction → sub-information reconstruction. Since the details of the processing have already been described, if only the outline is described, the color difference is extracted from the captured image by the color component extraction process and the spatial frequency component extraction process, and the color difference is handled in the sub information reconstruction process. The digital watermark is restored by converting it into information, for example, bit on / off.

  On the other hand, information that can be expressed by color difference depends on gradation. That is, the information that can be expressed decreases if the gradation is low, and increases if it is high. Gradation changes for each color component when digital information is converted to analog information or vice versa. Therefore, by using the color component having the highest gradation, it is possible to suppress deterioration in the amount of information expressed by the color difference, and to improve the restoration property of the digital watermark.

  As described above, by using a color component with little deterioration in conversion between digital and analog, it is possible to improve the recoverability of the embedded sub information.

Next, a third embodiment will be described.
FIG. 16 shows the configuration of the second image processing apparatus 110 according to the third embodiment. The second image processing apparatus 110 includes an image input unit 21 as an image input unit, a region extraction unit 22 as a region extraction unit, a color feature amount extraction unit 23 as a color feature amount extraction unit, and a color as a color synthesis unit. The composition unit 24 includes a frequency component extraction unit 25 as a frequency component extraction unit, a reconstruction unit 26 as a reconstruction unit, and a display unit 27 as an output unit. The function of each unit will be described in detail below. To do.

  For example, the image input unit 21 takes in the composite image information 403 recorded in the personal authentication medium 401 of FIG. 4 with an image input device such as a camera, and converts it into digital composite image information. This image information is composed of three planes R, G, and B.

  The region extraction unit 22 sets a region having a predetermined size on the image information input by the image input unit 21. As the size of the area to be set, the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used at the time of embedding is used.

  The color feature amount extraction unit 23 adds up the values of pixels existing in the region set by the region extraction unit 22 for each of the R, G, and B planes, and calculates the color feature amount. The color synthesizing unit 24 calculates weights at the time of synthesizing each of the R, G, and B planes based on the total result of the color feature amount extracting unit 23, and further performs synthesis processing.

  The frequency component extraction unit 25 performs frequency filtering on the color components synthesized by the color synthesis unit 24, and extracts the amplitude (= intensity) information and phase information of the spatial frequency component of the embedded key information. .

  The reconstruction unit 26 performs aggregation processing and spatial filtering on the amplitude information or phase information of the spatial frequency component extracted by the frequency component extraction unit 25, and reconstructs a modulation component, that is, sub-information on the key information. The display unit 27 displays the sub information reconstructed by the reconstruction unit 26 (or the input image information and sub information may be displayed simultaneously).

  FIG. 17 shows an outline of the processing flow in the area extracting unit 22, the color feature amount extracting unit 23, and the color synthesizing unit 24. First, the region extraction unit 22 sets a color feature amount extraction target region S centered on the pixel of interest on the digital image information captured by the image input unit 21 (step 31).

In the present embodiment, the size of the region S is the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used at the time of embedding, that is, a value obtained by introducing a resolution change in recording and reading with respect to the fundamental wavelength. Is used. For example, assuming that the fundamental wavelength is 2 pixels, the main image and recording resolution is 200 dpi, and the reading resolution is 400 dpi, the size of the area S is 4 pixels. The target pixel moves sequentially from left to right and from the bottom to the top of the image, and the amount of movement is the size of the area.
Note that the position of the first pixel of interest is calculated in advance from the position of the personal authentication medium at the time of image input and the face image recording position in the personal authentication medium.

Next, the color feature amount for each region S is obtained using the value of the pixel inside the region for each plane (step 32). The color feature amount is a numerical value representing the color in the area. In this embodiment, the average luminance value of the pixels in the area for each plane is used as the color feature amount. For example, if the pixel values in the R, G, B planes in the m × n pixel area S are R _ij , G _ij , B _ij (i = 1... M, j = 1. The feature quantities R _F , G _F , and B _F are as shown in the following formula 2.

The color combining unit 24 determines a color combining parameter P _C corresponding to the color feature amount of each plane (step 33), on the basis of the color combining parameter P _C, the color combining processing to each pixel in the region (Step 34). In the present embodiment, when the color synthesis parameter P _C is determined from the color feature quantity, a three-dimensional array storing the color synthesis parameter P _C using the color feature quantity as an index is used.

For example, let R _ij , G _ij , B _ij (i = 1... M, j = 1... N) be the pixel values in the R, G, and B planes in the m × n pixel region S, respectively, and the color characteristics. When the amounts are R _F , G _F , and B _F , the color composition parameters P _R , P _G , and P _B are determined as follows.
P _C = A _C [R _F , G _F , B _F ] (C = R, G, B)
Here, A _C (C = R, G, B) is a three-dimensional array and stores color synthesis parameters corresponding to each plane. For example, A _R stores the color combining parameter P _R of the R plane.

The value of the color synthesis parameter P _C (C = R, G, B) is determined in advance by the following processing.
(1) Create a color patch to determine the parameters. The color patch is a printed material in which a region having a predetermined area is filled with a single color, and the region is subjected to a digital watermark embedding process.
(2) The color patch is taken in by the image input unit 21 or an image input unit having equivalent performance, and the color feature amount of each of the R, G, and B planes is calculated.
(3) The digital watermark is restored from each of the R, G, and B planes, and the average value of the amplitudes of the restored signals is calculated. At this time, color synthesis is not performed.
(4) The color synthesis parameter corresponding to the color feature amount calculated in (2) above is determined from the value calculated in (3) above. For example, if the average amplitude value of a certain plane is larger than a predetermined threshold ε, the color composition parameter of that plane is set to “1” and that of the other plane is set to “0”. Conversely, when the average amplitude value of all the planes is smaller than the threshold ε, the color synthesis parameter of all the planes is set to 1/3.

Next, color composition calculation is performed for each pixel in the region using the color composition parameter determined above. In the present embodiment, a linear sum obtained by multiplying and adding the color composition parameter corresponding to the pixel value of each plane as a weight is used as the color composition calculation. For example, let R _ij , G _ij , and B _ij (i = 1... M, j = 1... N) be the pixel values in the R, G, and B planes in the m × n pixel area S, respectively. Assuming that the color synthesis parameters in S are P _R , P _G , and P _B , the value T _ij resulting from the color synthesis is calculated as follows.
T _ij = P _R R _ij + P _G G _ij + P _B B _ij (i = 1... M, j = 1... N)
The above process is performed on the entire input image to obtain a color composite image T. Thereafter, the frequency component extraction unit 25 extracts a specific spatial frequency component from the color composite image T based on the key information used in the digital watermark embedding process, and the reconstruction unit 26 reconstructs the sub information. .

  In the above description, the position of the personal authentication medium and the recording position of the face image are used when determining the position of the first target pixel. However, the present invention is not limited to this method. A specific pattern may be recorded in advance in or around the face image, and the determination may be made using the result of detecting the position of the pattern.

  In addition, when the size of the spatial frequency component of the key information is sufficiently small with respect to the size of the face image, the color feature amount is calculated from a predetermined range centered on the target pixel without performing region division. Also good.

  Further, in the above description, the luminance average of the pixels in the region is used as the color feature amount, but it is not limited to this, and may be, for example, a median value, a mode value, These may be modified by using values indicating the distribution shape, for example, standard deviation or variance as weights. Alternatively, it may be a value resulting from applying a predetermined function or algorithm.

In the above description, the three-dimensional array is used when determining the color synthesis parameters. However, the method is not limited to this method, and may be determined based on the following functions, for example.
P _C = f _C (R _F , G _F , B _F ) (C = R, G, B)
Here, f _C (C = R, G, B) represents a color calculation function for determining a color synthesis parameter from the color feature amount. As the color calculation function f _C (C = R, G, B), for example, a linear sum P _C = W _C1 R _F + W _C2 G _F obtained by multiplying R _F , G _F , and B _F by a predetermined weight, for example. It may be + W _C3 B _F (C = R, G, B), or may be a polynomial having a high-order term such as R _F ² or a cross term such as R _F G _F. Alternatively, the calculation may be performed after logarithmic transformation such as R _F ′ = log (R _F ).

In the above description, the weighted linear sum is used in the color composition calculation. However, the present invention is not limited to this method. For example, a higher order term such as R _ij ² or R _ij G _ij is used. It may be a polynomial having a simple cross term, or may be calculated after logarithmically transforming each such as R _ij ′ = log (R _ij ). Alternatively, by using a 6-dimensional array A _{T in} which the color feature value and the color synthesis parameter are used as indexes and the color synthesis value is stored, = T _ij [R _ij , G _ij , B _ij , P _R , P _G , P _B ] Good.

  In this way, by restoring the influence of deterioration by combining the color components of the input image information, it is possible to improve the recoverability of the embedded sub information.

Next, a fourth embodiment will be described.
FIG. 18 shows the configuration of the second image processing apparatus 110 according to the fourth embodiment. The second image processing apparatus 110 includes an image input unit 41 as an image input unit, a region extraction unit 42 as a region extraction unit, a color feature amount extraction unit 43 as a color feature amount extraction unit, and a reconstruction parameter determination unit. Amplification coefficient determination unit 44, frequency component extraction unit 45 as frequency component extraction means, extraction signal amplification unit 46 as amplification means, reconstruction unit 47 as reconstruction means, and display unit 48 as output means Hereinafter, the function of each unit will be described in detail.

  For example, the image input unit 41 takes in the composite image information 403 recorded in the personal authentication medium 401 of FIG. 4 with an image input device such as a camera, and converts it into digital composite image information. This image information is composed of three planes R, G, and B.

  The area extraction unit 42 sets an area having a predetermined size on the image information input by the image input unit 41. As the size of the area to be set, the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used at the time of embedding is used.

  The color feature quantity extraction unit 43 calculates the color feature quantity by adding up the values of pixels existing in the region set by the region extraction unit 42 for each of the R, G, and B planes. The amplification coefficient determination unit 44 determines an amplification coefficient at the time of reconstruction of the restoration result based on the total result of the color feature amount extraction unit 43.

  The frequency component extraction unit 45 performs frequency filtering on the image information input by the image input unit 41, and extracts the amplitude (= strength) information and phase information of the spatial frequency component of the embedded key information. .

  The extracted signal amplification unit 46 performs amplification processing on the amplitude information or phase information of the spatial frequency component extracted by the frequency component extraction unit 45 using the amplification coefficient determined by the amplification coefficient determination unit 44.

  The reconstruction unit 47 performs aggregation processing and spatial filtering on the amplitude information or phase information of the spatial frequency component amplified by the extracted signal amplification unit 46, and reconstructs the modulation component, that is, sub-information on the key information. The display unit 48 displays the sub information reconstructed by the reconstructing unit 47 (or the input image information and sub information may be displayed simultaneously).

  FIG. 19 shows an outline of the processing flow in the extraction signal amplification unit 46. First, the color feature quantity extraction unit 43 performs the same processing as the color feature quantity extraction unit 23 described in the third embodiment, and calculates a color feature quantity (step 51).

Next, the amplification coefficient determination unit 44 determines an amplification coefficient (amplification factor) M in the region from the color feature amount for each plane (step 52). For example, if the color feature amounts in the R, G, and B planes in the m × n pixel region S are R _F , G _F , and B _F , the amplification coefficient M is determined as follows.
M = A _M [R _F , G _F , B _F ]
Here, A _M is a three-dimensional array and stores an amplification coefficient corresponding to the color feature amount.

  The extracted signal amplifying unit 46 multiplies the spatial frequency component extracted by the frequency component extracting unit 45 by the amplification coefficient M obtained by the amplification coefficient determining unit 44 for each region (step 53). By performing this processing, the extracted frequency component is corrected so that the variation in the maximum value for each region falls within a predetermined range.

In the above description, the three-dimensional array is used to determine the amplification coefficient M. However, the method is not limited to this method, and may be determined based on the following function, for example.
M = f _M (R _F , G _F , B _F )
Here, f _M represents an amplification coefficient calculation function for determining the amplification coefficient from the color feature amount. The amplification coefficient calculation function f _M, for example, R _F, G _F, respectively may be a predetermined linear sum M = W _M1 multiplied by the weight of _{R F + W M2 G F +} W M3 B F to B _F, It may be a polynomial having a high-order term such as R _F ² or a cross term such as R _F G _F. Alternatively, the calculation may be performed after logarithmic transformation such as R _F ′ = log (R _F ).

  Thus, by correcting the reconstruction process according to the color component of the input image information, the visibility of the embedded sub information can be improved.

In the fourth embodiment, a multivalued image is generated by multiplying the extracted spatial frequency component by an amplification coefficient. Instead, a binarization threshold value corresponding to a color feature amount for each region is stored. using 3-dimensional array a _B that may be configured to generate a binary image by performing binarization processing for each area. According to this method, the data amount of the restoration result can be reduced, and the contrast of the restoration result can be increased.

  As described above, according to the second to fourth embodiments, analog data that is output to a printed matter by using a color component that is less deteriorated at the time of conversion between digital and analog is targeted. When creating composite image information in which other additional sub-information is embedded in the invisible state in the main image information, it is printed without increasing the risk of the sub-information being exposed from the composite image information. The performance of restoring the sub-information embedded from the composite image information can be improved.

  In addition, by combining the color components of the input image information to reduce the effects of deterioration, the ability to restore the sub-information embedded from the printed composite image information without increasing the same risk Can be improved.

  Further, by correcting the reconstruction process according to the color components of the input image information, the visibility of the sub information restored from the printed composite image information is improved without increasing the same risk. be able to.

1 is a block diagram schematically showing an overall configuration of an image processing system according to a first embodiment of the present invention. The flowchart explaining the flow of a process from a 1st pre-processing part to a post-processing part. The figure which shows the arrangement | sequence state of the recorded dot. The top view which shows typically an example of the produced personal authentication medium. The figure explaining the image information to record and the recorded dot. The figure explaining the concept of the 1st preprocessing and the 2nd preprocessing. The figure explaining the specific example of a 1st pre-process and a 2nd pre-process. The figure which shows the flow of the whole electronic watermark embedding process. 5 is a flowchart schematically showing a procedure of digital watermark embedding processing. The schematic diagram explaining the frequency component of key information. The schematic diagram which shows the process of the reconstruction process of subinformation. The schematic diagram which shows the process of the reconstruction process of subinformation. The schematic diagram which shows the process of the reconstruction process of subinformation. The block diagram which shows schematically the structure of the 2nd image processing apparatus which concerns on the 2nd Embodiment of this invention. FIG. 6 is a characteristic diagram for explaining ink gradation. The block diagram which shows schematically the structure of the 2nd image processing apparatus which concerns on the 3rd Embodiment of this invention. The figure for demonstrating the flow of a color feature-value extraction process and a color composition process. The block diagram which shows schematically the structure of the 2nd image processing apparatus which concerns on the 4th Embodiment of this invention. The figure for demonstrating the flow of an extraction signal amplification process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... 1st image processing apparatus, 101 ... Main image input part (image input means), 102 ... 1st pre-processing part (1st pre-processing means), 103 ... 2nd pre-processing part (2nd Pre-processing means), 104 ... Digital watermark embedding processing section (embedding means), 105 ... Post-processing section (inverse conversion means), 106 ... Recording section (recording means), 110 ... Second image processing apparatus, 111 ... Recorded image Input unit (image input unit), 112, 14, 25, 45 ... frequency component extraction unit (frequency component extraction unit), 113, 15, 26, 47 ... reconstruction unit (reconstruction unit), 114 ... determination unit (determination) 115) Restoration unit (restoration means), 401 ... Personal authentication medium, 402 ... Face image, 403 ... Personal management information, 801 ... Main image information, 802 ... Sub information (sub image information), 803 ... Key information, 807 ... Composite image information, 11, 21, 4 ... Image input section (image input means), 12 ... Color component information storage section (color component information storage means), 13 ... Color component extraction section (color component extraction means), 16, 27 ... Display section, 22, 42 ... area Extraction unit (region extraction unit), 23, 43 ... color feature amount extraction unit (color feature amount extraction unit), 24 ... color synthesis unit (color synthesis unit), 44 ... amplification factor determination unit (reconstruction parameter determination unit), 46: Extraction signal amplification section (amplification means).

Claims (16)

Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . An image input for reading out and inputting the composite image information from a recording medium in which composite image information created by embedding the sub-information in the main image information in an invisible state by performing an embedded digital watermark processing Means,
The size of the key information is expanded or contracted based on the resolution of the composite image information recorded on the recording medium and the reading resolution of the image input means, and then subjected to Fourier transform on the key information to obtain a frequency domain. The value obtained by performing expansion and adjusting the pass band of the filter by referring to the expanded value and performing inverse Fourier transform on the adjusted value is defined as the frequency filter coefficient. Frequency component extraction means for extracting the spatial frequency component of the key information from the composite image information input by the image input means using
Reconstructing means for reconstructing the sub information from the spatial frequency components extracted by the frequency component extracting means;
An image processing apparatus comprising:   The reconstruction unit extracts a change point where a sign changes from the spatial frequency component extracted by the frequency component extraction unit, and obtains a reference phase of the spatial frequency component by projecting the extracted change point, For each coordinate of the spatial frequency component extracted by the frequency component extraction means, the deviation from the obtained reference phase is calculated, the pixel value of the coordinate that is deviated by a predetermined threshold or more is the first value, and the others are the second value. The image processing apparatus according to claim 1, wherein the sub-information is reconstructed by replacing with the value of   The image processing apparatus according to claim 1, further comprising a determination unit that determines authenticity of the recording medium based on the sub-information reconfigured by the reconfiguration unit. Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . An image input for reading out and inputting the composite image information from a recording medium in which composite image information created by embedding the sub-information in the main image information in an invisible state by performing an embedded digital watermark processing Means,
Color component information storage means for storing color component information;
Color component extraction means for extracting color components from the composite image information input by the image input means based on the color component information stored in the color component information storage means;
Frequency component extraction means for extracting a spatial frequency component specific to the key information from the color component extracted by the color component extraction means;
Reconstructing means for reconstructing the sub information from the spatial frequency components extracted by the frequency component extracting means;
An image processing apparatus comprising: The color component extracted from the composite image information is a color component corresponding to the color of the ink having the highest gradation characteristics at the time of recording the composite image information, and the color component information is stored in the color component information storage means. The image processing apparatus according to claim 4 . The color component extracted from the composite image information is a color component corresponding to the color of the ink having the highest gradation characteristics when the composite image information is input, and the color component information is stored in the color component information storage unit. The image processing apparatus according to claim 4 . Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . An image input for reading out and inputting the composite image information from a recording medium in which composite image information created by embedding the sub-information in the main image information in an invisible state by performing an embedded digital watermark processing Means,
A region extraction unit that extracts, as a local region, a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used in embedding from the composite image information input by the image input unit;
Color feature amount extraction means for extracting a color feature amount in the local area from the local area extracted by the area extraction means;
Color synthesis means for synthesizing color components based on the color feature values extracted by the color feature quantity extraction means to generate color component synthesized image information;
A frequency component extracting unit that extracts a spatial frequency component unique to the key information from the color component combined image information generated by the color combining unit;
Reconstructing means for reconstructing the sub information from the spatial frequency components extracted by the frequency component extracting means;
An image processing apparatus comprising: Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . An image input for reading out and inputting the composite image information from a recording medium in which composite image information created by embedding the sub-information in the main image information in an invisible state by performing an embedded digital watermark processing Means,
A region extraction unit that extracts, as a local region, a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used in embedding from the composite image information input by the image input unit;
Color feature amount extraction means for extracting a color feature amount in the local area from the local area extracted by the area extraction means;
Reconstruction parameter determination means for determining a reconstruction parameter based on the color feature amount extracted by the color feature amount extraction means;
Frequency component extraction means for extracting a spatial frequency component specific to the key information from the composite image information input by the image input means;
Reconstructing means for reconstructing the sub-information from the spatial frequency component extracted by the frequency component extracting means using the reconstruction parameter determined by the reconstruction parameter determining means;
An image processing apparatus comprising: 9. The image according to claim 8, wherein the reconstruction parameter is an amplification coefficient for amplifying the spatial frequency component, and the reconstruction means includes amplification means for amplifying the spatial frequency component using the amplification coefficient. Processing equipment. The reconfiguration parameter is a threshold value for binarizing the spatial frequency component, and the reconstruction means includes binarization means for binarizing the spatial frequency component using the threshold value. 8. The image processing apparatus according to 8 . Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . The composite image information is read by the image input means from the recording medium in which the composite image information created by embedding the sub-information in the main image information in an invisible state by performing a digital watermark embedding process that has been recorded. An image input step to input;
The size of the key information is expanded or contracted based on the resolution of the composite image information recorded on the recording medium and the reading resolution in the image input step, and then the Fourier transform is performed on the key information to obtain a frequency domain. The value obtained by performing expansion and adjusting the pass band of the filter by referring to the expanded value and performing inverse Fourier transform on the adjusted value is defined as the frequency filter coefficient. A frequency component extraction step of extracting a spatial frequency component specific to the key information by frequency component extraction means from the composite image information input by the image input step using
A reconstruction step of reconstructing the sub information by a reconstruction means from the spatial frequency component extracted by the frequency component extraction step;
An image processing method comprising: The reconstruction step extracts a change point where a sign changes from the spatial frequency component extracted by the frequency component extraction step, and obtains a reference phase of the spatial frequency component by projecting the extracted change point, For each coordinate of the spatial frequency component extracted in the frequency component extraction step, the deviation from the obtained reference phase is calculated, the pixel value of the coordinate that is deviated by a predetermined threshold or more is the first value, and the other is the second value. The image processing method according to claim 11, wherein the sub-information is reconstructed by replacing the value with a value. The image processing method according to claim 11 , further comprising a determination step of determining authenticity of the recording medium based on the sub-information reconfigured by the reconfiguration step. Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . The composite image information is read by the image input means from the recording medium in which the composite image information created by embedding the sub-information in the main image information in an invisible state by performing a digital watermark embedding process that has been recorded. An image input step to input;
A color component extraction step of extracting a color component by the color component extraction means from the combined image information input by the image input step based on the color component information stored in the color component information storage means;
A frequency component extraction step of extracting a spatial frequency component specific to the key information by frequency component extraction means from the color component extracted in this color component extraction step;
A reconstruction step of reconstructing the sub information by a reconstruction means from the spatial frequency component extracted by the frequency component extraction step;
An image processing method comprising: Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . The composite image information is read by the image input means from the recording medium in which the composite image information created by embedding the sub-information in the main image information in an invisible state by performing a digital watermark embedding process that has been recorded. An image input step to input;
A region extraction step for extracting, as a local region, a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used in embedding by the region extraction means from the composite image information input in this image input step;
A color feature amount extraction step of extracting color feature amounts in the local region from the local region extracted by the region extraction step by color feature amount extraction means ;
A color synthesizing step for synthesizing color components by color synthesizing means based on the color feature amount extracted by the color feature amount extracting step and generating color component synthesized image information;
A frequency component extracting step of extracting a spatial frequency component specific to the key information by frequency component extracting means from the color component synthesized image information generated by the color synthesizing step;
A reconstruction step of reconstructing the sub information by a reconstruction means from the spatial frequency component extracted by the frequency component extraction step;
An image processing method comprising: Using the color difference modulation method using main image information visible to the human naked eye, sub-information for embedding the main image information invisible to the human naked eye, and key information used to restore the sub-information . The composite image information is read by the image input means from the recording medium in which the composite image information created by embedding the sub-information in the main image information in an invisible state by performing a digital watermark embedding process that has been recorded. An image input step to input;
A region extraction step for extracting, as a local region, a region having the number of pixels corresponding to the fundamental wavelength of the spatial frequency component of the key information used in embedding by the region extraction means from the composite image information input in this image input step;
A color feature amount extraction step of extracting color feature amounts in the local region from the local region extracted by the region extraction step by color feature amount extraction means ;
A reconstruction parameter determination step for determining a reconstruction parameter by the reconstruction parameter determination means based on the color feature amount extracted by the color feature amount extraction step;
A frequency component extraction step of extracting a spatial frequency component specific to the key information by frequency component extraction means from the composite image information input in the image input step;
A reconstruction step of reconstructing the sub information by reconstruction means from the spatial frequency component extracted by the frequency component extraction step using the reconstruction parameter determined by the reconstruction parameter determination step;
An image processing method comprising:

Images