JP2005236517A - Imaging apparatus, data processor and processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data processor capable of lessening burden incident to certification of correctness of image data on a camera user and on the verification side. <P>SOLUTION: The encoder 13 of a digital camera 3 creates original text certification data CER by generating a random number based on seed data SEED1, adds the original text certification data CER to original text image data ORG12, and then performs encryption to create original text image data ORG. The encrypted seed data SEED is associated with the original text image data ORG and written in a recording medium 21. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus, a data processing apparatus, and a method thereof that make it possible to add certification data to image data corresponding to an imaging result and verify the validity of the image data.

For example, digital image data captured by a digital camera may be used illegally by a third party without obtaining permission from the copyright holder of the image data.
In order to prevent such unauthorized use, there is a method in which electronic signature data indicating a hash value or the like is added to image data corresponding to the imaging result output from the camera, and then the image data is disclosed on the network. is there.

However, in the above-described conventional method, the verification side of the electronic signature data needs to hold information on what the electronic signature data is, and if the number of verification target cameras is large, There is a problem that the management burden is large.
In addition, when the confidentiality of the electronic signature data of the camera is impaired, there is a problem that the user of the camera needs to notify the verification side of the change of the electronic signature data, and the burden is large.

  The present invention has been made in view of the above-described problems of the prior art, and provides an imaging apparatus, a data processing apparatus, and a method thereof that can reduce the burden on the user of the camera and the verification side accompanying the verification of the validity of the image data. For the purpose.

  In order to solve the above-described problems of the prior art and achieve the above-described object, an imaging apparatus according to a first invention is an imaging apparatus that generates digital image data according to an imaging result, and is in a matrix form. It is composed of R, G, and B pixel data obtained by imaging one of R (red), G (green), and B (blue) at each of a plurality of arranged pixel positions. Imaging means for generating first image data, and proof data generation for generating proof data used for performing predetermined proof on the first image data generated by the imaging means based on the predetermined data And means for adding the proof data generated by the proof data generating means to the first image data generated by the imaging means to generate second image data, the predetermined data, and Generated by the adding means An imaging unit that encrypts the second image data, the predetermined data encrypted by the encryption unit and the encrypted second image data are associated with each other, and the imaging apparatus And an interface for outputting to the outside.

The operation of the imaging apparatus of the first invention is as follows.
First, R, obtained by imaging one of R (red), G (green), and B (blue) light at each of a plurality of pixel positions arranged in a matrix. First image data composed of G and B pixel data is generated.
Next, proof data generation means generates proof data used for performing predetermined proof on the first image data generated by the imaging means based on the predetermined data.
Next, an adding unit adds the proof data generated by the proof data generating unit to the first image data generated by the imaging unit to generate second image data.
Next, an encryption unit encrypts the predetermined data and the second image data generated by the addition unit.
Next, the interface associates the predetermined data encrypted by the encryption unit with the encrypted second image data, and outputs the second data to the outside of the imaging apparatus.

  According to a second aspect of the present invention, there is provided a data processing device for generating proof data used for performing a predetermined proof for the first image data based on predetermined data, and the proof data generating unit generating the proof data And adding the certification data to the first image data to generate second image data, encrypting the predetermined data and the second image data generated by the adding means. And encryption means; and association means for associating the predetermined data encrypted by the encryption means with the encrypted second image data.

The operation of the data processing apparatus of the second invention is as follows.
First, proof data generation means generates proof data used for performing a predetermined proof regarding the first image data based on the predetermined data.
Next, the adding means adds the proof data generated by the proof data generating means to the first image data to generate second image data.
Next, an encryption unit encrypts the predetermined data and the second image data generated by the addition unit.
Next, an associating unit associates the predetermined data encrypted by the encrypting unit with the encrypted second image data.

  According to a third aspect of the present invention, there is provided a data processing device obtained by decrypting encrypted image data, decryption means for decrypting predetermined encrypted data associated with the image data, and decryption by the decryption means. Based on the predetermined data, proof data generating means for generating proof data for performing predetermined proof on the image data obtained by decoding by the decoding means, and the image obtained by decoding by the decoding means Correlation detection means for detecting a correlation between the data and the proof data generated by the proof data generation means, and the image data obtained by decoding by the decoding means based on the correlation detected by the correlation detection means Is generated using an imaging device that is associated with the proof data generated by the proof data generation means and adds the proof data to the image data generated according to the imaging result. And a determination unit.

The operation of the data processing apparatus of the third invention is as follows.
First, the decrypting means decrypts the encrypted image data and the predetermined encrypted data associated with the image data.
Next, proof data generating means generates proof data for performing predetermined proof on the image data obtained by decoding by the decoding means, based on the predetermined data obtained by decoding by the decoding means. To do.
Next, a correlation detecting unit detects a correlation between the image data obtained by decoding by the decoding unit and the proof data generated by the proof data generating unit.
Next, based on the correlation detected by the correlation detection unit, the determination unit associates the image data obtained by the decoding unit with the proof data generated by the proof data generation unit, It is determined whether or not the image data generated according to the imaging result is generated using an imaging device that adds the certification data.

  According to a fourth aspect of the present invention, there is provided a data processing method according to a first step of generating proof data used for performing a predetermined proof for the first image data based on the predetermined data, and generating the proof data in the first step. The second step of adding the certification data to the first image data to generate second image data, the predetermined data, and the second image data generated in the second step And a fourth step of associating the predetermined data encrypted in the third step with the encrypted second image data.

  According to a fifth aspect of the present invention, there is provided a data processing method comprising: a first step of decrypting encrypted image data; and predetermined encrypted data associated with the image data; and decrypting in the first step. A second step of generating proof data for performing a predetermined proof on the image data obtained by decrypting in the first step based on the predetermined data obtained in the step, and in the first step Based on the correlation detected in the third step and the correlation detected in the third step, the first step detects the correlation between the image data obtained by decoding and the certification data generated in the second step. The image data obtained by decrypting in the step is associated with the certification data generated in the second step, and an imaging apparatus is used to add the certification data to the image data generated according to the imaging result 4th process to determine whether or not With the door.

  According to the present invention, it is possible to provide an imaging device user, a data processing device, and a method thereof that can reduce the burden on the user of the imaging device accompanying verification of the validity of the image data and the verification side.

Embodiments of the present invention will be described below.
The first embodiment will now be described in detail the data processing system 1 of the first embodiment.
FIG. 1 is a configuration diagram of a data processing system 1 according to the present embodiment.
As shown in FIG. 1, the data processing system 1 includes, for example, a digital camera 3, a computer 4, a computer 5, and a collation device 6.
Here, the digital camera 3 corresponds to the imaging device of the first invention and the data processing device of the second invention.
Further, the computer 5 and the collation device 6 correspond to the data processing device of the third invention. Note that the computer 5 and the collation device 6 may be realized as a single device provided with both configurations.

First, the outline of the data processing system 1 will be described.
The digital camera 3 shown in FIG. 1 generates source certificate data CER (certification data of the present invention) by generating a random number based on the seed data SEED1 in the encoder 13, and generates the source image data ORG12 (first data of the present invention). 1 image data), and subsequently encrypted to generate original image data ORG (second image data of the present invention).
Further, the encoder 13 generates seed data SEED by encrypting the seed data SEED1, and writes the encrypted original image data ORG and the encrypted seed data SEED in the recording medium 21 in association with each other.
The computer 5 decrypts the seed data SEED to generate seed data SEED1, generates a random number based on the seed data SEED1, generates original proof data CER, and performs interpolation processing described later for each of R, G, and B. To generate original proof data CERa.
Further, the computer 5 decodes the original image data ORG to generate the original image data ORG1, and performs interpolation processing to be described later for each of R, G, and B to generate the original image data ORGa.
The collation device 6 determines whether the original image data ORGa is a legitimate original image (original image data) by the digital camera 3 based on the correlation between the original proof data CERa and the original image data ORGa.
Hereinafter, each component shown in FIG. 1 will be described.

[Digital camera 3]
As shown in FIG. 1, the digital camera 3 includes, for example, a lens 10, a light receiving unit 11, an A / D conversion unit 12, and an encoder 13.
Here, the light receiving unit 11 corresponds to the imaging means of the first invention.
Further, as will be described later, a configuration corresponding to the proof data generating means, the adding means, the encrypting means, and the Internet is incorporated in the encoder 13.

The lens 10 receives light from the subject and outputs it to the light receiving unit 11.
The light receiving unit 11 has a plurality of light receiving elements arranged in a matrix. The light receiving unit 11 will be described in detail later.
Light from the subject input through the lens 10 enters the light receiving element of the light receiving unit 11.
Then, the light receiving unit 11 outputs an image signal corresponding to the light reception results of the plurality of light receiving elements to the A / D conversion unit 12.
The A / D conversion unit 12 A / D converts the image signal input from the light receiving unit 11 to generate original image data ORG 12 that is digital original image data, and outputs this to the encoder 13.

FIG. 2 is a block diagram of the encoder 13 shown in FIG.
For example, as shown in FIG. 2, the encoder 13 includes a memory 14, a source certificate data generation unit 15, a superposition unit 16, an encryption unit 17, an encryption unit 18, and a writing unit 19.
Here, the original proof data generating unit 15 corresponds to the proof data generating means of the first and second inventions, the superimposing unit 16 corresponds to the adding means of the first and second inventions, and the encrypting unit 18 is the first. Corresponding to the encryption means of the first and second inventions, the writing unit 19 corresponds to the interface of the first invention and the association means of the second invention.

The memory 14 stores seed data SEED1 for generating random numbers used for generating original proof data CER. The seed data SEED1 indicates, for example, a value unique to each digital camera 3.
The source certificate data generation unit 15 generates a random number based on the seed data SEED1 read from the memory 14, and generates source certificate data CER.
As an algorithm for generating the random number, for example, a middle-square method is used.
The source certificate data CER is, for example, a signed integer matrix having a word length of about 2 to 5 bits, and the matrix size is the same as that of the source image data S12.
As will be described later, the source certificate data generation unit 15 may generate source certificate data CER for each of R, G, and B using different seed data SEED1 for each of R, G, and B.

The superimposing unit 16 generates original image data ORG1 by superimposing the original certificate data CER input from the original certificate data generating unit 15 on the original image data ORG12 input from the A / D converter 12.
Here, the original image data ORG12 is an unsigned integer matrix having a word length of about 10 to 12 bits.
The superposition unit 16 converts the source certificate data CER12 into a signed integer matrix, and then calculates the sum with the source certificate data CER to generate source image data ORG1.
Specifically, the superimposing unit 16 adds the pixel data (element data) of the original certificate data CER corresponding to each pixel data of the original image data ORG12.

The encryption unit 17 encrypts the original image data ORG1 based on the predetermined key data K2 to generate the original image data ORG.
The encryption unit 18 encrypts the seed data SEED1 based on the predetermined key data K1 to generate the seed data SEED.
Here, the encryption unit 17 and the encryption unit 18 may use the same encryption unit.
The key data K1 and K2 may be the same.

The writing unit 19 writes the original image data ORG generated by the encryption unit 17 and the seed data SEED generated by the encryption unit 18 to the recording media 20 and 21 in association with each other.
Note that only the original image data ORG may be written on the recording medium 20.

Hereinafter, the above-described light receiving unit 11 will be described in detail.
As shown in FIG. 3, the light receiving unit 11 includes a color filter 25 in which R, G, and B filters are arranged at positions (pixel positions) corresponding to the respective light receiving elements arranged in a matrix.
Accordingly, only the R component of the light from the subject passes through the color filter 25 and enters the light receiving element at the “R” position of the color filter 25 shown in FIG. Only the G component of the light from the subject passes through the color filter 25 and enters the light receiving element, and only the B component of the light from the subject enters the color filter 25 at the light receiving element at the position “B”. Through and incident.
Thereby, the light receiving unit 11 receives an image signal composed of R, G, and B pixel signals obtained by imaging any one of R, G, and B light at each of the plurality of pixel positions. Generate.

The light receiving element of the light receiving unit 11 is, for example, a CCD (Charge Coupled Device) image pickup element, and the light receiving sensitivity has an inherent variation in manufacturing. This variation is difficult to reproduce artificially.
FIG. 4 is a diagram schematically showing a cross section of the light receiving element of the light receiving unit 11 shown in FIG.
As shown in FIG. 4, the light receiving element of the light receiving unit 11 includes a silicon substrate 32, a light shielding layer 34 provided with an opening 33, and a condenser lens 35.
A photoelectric conversion region is formed in the silicon substrate 32 at a position facing the opening 33. The condensing lens 35 is formed so as to reach the silicon substrate 32 via the light shielding layer 34 from a position opposite to the silicon substrate 32 with respect to the light shielding layer 34.

The opening 33 is formed by, for example, finely processing a resist serving as an etching mask by exposure to ultraviolet rays, and then removing the light shielding layer 34 and the insulating layer 36 therebelow by etching.
The condenser lens 35 is formed in a spherical shape by subjecting the glass layer to fine processing by photolithography and then melting the glass layer at a high temperature.
In the manufacturing process of the light receiving element described above, the opening 33 is controlled so as to form a uniform shape as much as possible, but the shape varies due to various factors. For this reason, the shape of the opening 33 of each of the plurality of light receiving elements varies.
Here, due to the variation in the shape of the opening 33 of each light receiving element, the shape of light (light flux) incident on the light receiving element varies, and the amount of light incident on the photoelectric conversion region of the silicon substrate 32 varies. Accordingly, the light receiving sensitivity varies among the light receiving elements.
Such variation in light receiving sensitivity is not artificial, and is unique to each light receiving element such as a fingerprint. In addition, the influence of the variation in the light receiving sensitivity of the light receiving element occurs in the image data generated by the digital camera 3 using the light receiving element.
In the present embodiment, as described above, collation is performed using the manufacturing variation in the light receiving sensitivity of the light receiving element that occurs in the image data generated by the camera.

[Computer 4]
The computer 4 performs display decoding processing on the original image data ORG read from the recording medium 20 to generate image data S4.
Specifically, the computer 4 extracts R, G, and B pixel data from the original image data ORG1 obtained by decoding the original image data ORG to obtain image data IM_R, IM_G, and IM_B, respectively. Are subjected to display decoding processing to generate image data IM_R, IM_G, and IM_B constituting the image data S4.
That is, as shown in FIG. 5, the computer 4 interpolates the image data IM_G to generate image data IM1_G composed of pixel data at all pixel positions.
At this time, for example, as shown in FIG. 6A, the computer 4 calculates the average (= (G1 + G2 + G3 + G4) / 4) of the pixel data G1, G2, G3, and G4 around the pixel data Gx as the pixel data Gx. Interpolation processing for interpolation data is performed.

The computer 4 divides the R pixel data in the image data IM_R by the G pixel data in the image data IM1_G corresponding to the R pixel data to generate image data IM1_R.
For example, the computer 4 interpolates the image data IM1_R to generate image data IM2_R including the pixel data Hr at all pixel positions.
At this time, as shown in FIG. 6B, the computer 4 uses the pixel data R1, R2, R3, R4 to perform interpolation processing for generating surrounding pixel data Rx1, Rx2, Rx3.
That is, the pixel data Rx1 is generated by (R1 + R2) / 2, the pixel data Rx2 is generated by (R1 + R3) / 2, and the pixel data Rx3 is generated by (R1 + R2 + R3 + R4) / 4.

The computer 4 calculates (Hr * G) on the pixel data Hr and G at the corresponding pixel positions of the image data IM1_G and IM2_R to generate the image data IM3_R.
Further, the computer 4 also performs the same processing as the above-described R image data IM_R on the B image data IM_B to generate image data IM3_B.
The computer 4 uses image data IM_G as the image data IM3_G.
Then, the computer 4 transmits the image data S4 composed of the image data IM1_G, IM3_R, and IM3_B to, for example, a server device on the network and discloses it.

[Computer 5]
FIG. 7 is a block diagram of the computer 5 shown in FIG.
As illustrated in FIG. 7, the computer 5 includes, for example, a reading unit 50, a decoding unit 51, a decoding unit 52, a source certificate data generation unit 53, and an interpolation unit 54.
Here, the decrypting unit 51 and the decrypting unit 52 correspond to the decrypting means of the third invention, and the original proof data generating unit 53 corresponds to the proof data generating means of the third invention.

The reading unit 50 reads the encrypted original image data ORG and seed data SEED from the recording medium 21, outputs the original image data ORG to the decrypting unit 51, and outputs the seed data SEED to the decrypting unit 52.
The decrypting unit 51 decrypts the original image data ORG based on, for example, the key data K2 used in the encrypting unit 17 shown in FIG. 2 or the corresponding key data to generate the original image data ORG1.

For example, the decryption unit 52 decrypts the seed data SEED based on the key data K1 used in the encryption unit 18 shown in FIG. 2 or the key data corresponding to the key data K1 to generate the seed data SEED1.
The source proof data generation unit 53 generates a random number based on the seed data SEED1 generated by the decryption unit 52 to generate the source proof data CER, similarly to the source proof data generation unit 15 shown in FIG.
The interpolation unit 54 performs the interpolation processing shown in FIGS. 8 and 9 for each of the original image data ORG1 generated by the decoding unit 51 and the original certificate data CER generated by the original certificate data generating unit 53, and the original image data ORGa and original proof data CERa are respectively generated.

That is, the interpolation unit 54 interpolates the pixel data in the original image data ORG1 and the original certificate data CER for each of R, G, and B pixel data, and all the R, G, and B pixel data for all the pixel positions. Is generated.
Specifically, as shown in FIG. 8, the interpolation unit 54 extracts G pixel data from the original image data ORG1 to generate image data IM10_G, extracts R pixel data, and converts the image data IM10_R into image data IM10_R. And B pixel data is extracted to generate image data IM10_B.
Then, for example, the interpolation unit 54 performs the interpolation processing described with reference to FIG. 6A on the image data IM10_G to generate the image data IM11_G.
For example, the interpolation unit 54 performs the interpolation processing described with reference to FIG. 6B on the image data IM10_R to generate the image data IM11_R.
Further, the interpolation unit 54 performs similar interpolation processing on the image data IM10_B to generate image data IM11_B.
The interpolation unit 54 outputs original image data ORGa composed of the image data IM11_G, IM11_R, and IM11_B to the collation device 6.

Further, as shown in FIG. 9, the interpolation unit 54 extracts G pixel data from the original proof data CER to generate image data IM20_G, extracts R pixel data to generate image data IM20_R, The B pixel data is extracted to generate image data IM20_B.
Then, for example, the interpolation unit 54 performs the interpolation processing described with reference to FIG. 6A on the image data IM20_G to generate the image data IM21_G.
For example, the interpolation unit 54 performs the interpolation processing described with reference to FIG. 6B on the image data IM20_R to generate the image data IM21_R.
In addition, the interpolation unit 54 performs similar interpolation processing on the image data IM20_B to generate image data IM21_B.
The interpolation unit 54 outputs the original proof data CERa including the image data IM21_G, IM21_R, and IM21_B to the collation device 6.

[Verification device 6]
FIG. 10 is a block diagram of the collation device 6 shown in FIG.
As illustrated in FIG. 10, the collation device 6 includes, for example, an interface 41, a correlation detection unit 42, and a determination unit 43.
Here, the correlation detection unit 42 corresponds to the correlation detection unit of the third invention, and the determination unit 43 corresponds to the determination unit of the third invention.

The interface 41 inputs the above-described original image data ORGa and original proof data CERa from the computer 5.
The correlation detection unit 42 includes the image data IM11_G, IM11_R, and IM11_B shown in FIG. 8 constituting the original image data ORGa input via the interface 41 and the original certification data CERa input via the interface 41 in FIG. Correlation with image data IM21_G, IM21_R, and IM21_B to be shown is detected between the same colors of G, R, and B.
The correlation detection unit 42 receives the light received by the light receiving element of the light receiving unit 11 constituting the image data from the image data generated by the digital camera 3 in which the reflected light from the subject is incident on the light receiving element having a variation unique to the light receiving sensitivity. Processing for reflecting the variation in the light receiving sensitivity generated in the pixel data corresponding to the result in the correlation is performed.

The determination unit 43 determines whether or not the original image data ORGa is a valid original based on the correlation detected by the correlation detection unit 42.
The determination unit 43 detects the correlation between each of the image data IM11_G, IM11_R, and IM11_B constituting the original image data ORGa shown in FIG. 8 and the image data IM21_G, IM21_R, and IM21_B constituting the original certificate data CERa shown in FIG. To do.
Then, as a result of the detection, the determination unit 43, as shown in FIG. 11, forms the R image data IM11_R shown in FIG. 8 constituting the original image data ORGa and the R image shown in FIG. 9 constituting the original certification data CERa. The correlation between the data IM21_R, the correlation between the G image data IM11_G shown in FIG. 8 constituting the original image data ORGa and the G image data IM21_G shown in FIG. 9 constituting the original certification data CERa, and the original image data ORGa The correlation between the B image data IM11_B shown in FIG. 8 and the B image data IM21_B shown in FIG. 9 constituting the original certification data CERa is higher than the first level, and other correlations are less than the second level. And if so, the original image data ORG is valid original image data (original It determines that) is.
In this embodiment, the interpolation processing shown in FIGS. 8 and 9 is different from the interpolation processing in the decoding processing described with reference to FIG. Does not include processing to reflect data. Therefore, the original image data ORG is an original one generated by the digital camera 3, and the original certificate data CER added to the original image data ORG and the original certificate data generated by the computer 5 based on the seed data SEED. When the CER matches, as shown in FIG. 11, the original image data ORGa and the original proof data CERa have a correlation exceeding the first level only between pixels of the same color, and between the other pixels. The correlation is below the second level (less than the first level).
Thereby, it can be determined that the original image data ORG is an original generated by a valid digital camera 3 associated with the original certificate data CER.

Hereinafter, the configuration of the correlation detection unit 42 shown in FIG. 10 adopting the SPOMF (Symmetrical Phase Only Matched Filtering) method will be described.
SPOMF is described in the literature “Symmetric Phase-Only Matched Filtering of Fourier-Mellin Transforms for Image Registration and Recognition” IEEE Transaction on Pattern analysis and Machine Intelligence, VOL.16 No.12 December 1994, and the like.
FIG. 12 is a functional block diagram of the correlation detector 42 shown in FIG.
As shown in FIG. 12, the correlation detection unit 42 includes, for example, an FFT circuit (Fast Fourier Transforms) 121, a whitening circuit 122, an FFT circuit 123, a whitening circuit 124, a complex conjugate circuit 125, a multiplication circuit 126, and an IFFT circuit 127. Have.
The FFT circuit 121 and the FFT circuit 123 correspond to the conversion means of the present invention, the whitening circuit 122 and the whitening circuit 124 correspond to the division means of the present invention, and the complex conjugation circuit 125 corresponds to the replacement means of the present invention. The circuit 126 corresponds to the multiplication circuit of the present invention, and the IFFT circuit 127 corresponds to the inverse conversion circuit of the present invention.

The FFT circuit 121 performs, for example, Fourier transformation on the original image data ORGa input from the computer 5 to generate second frequency component data S <b> 121, and outputs this to the whitening circuit 122.
The whitening circuit 122 divides each complex number data constituting the second frequency component data S121 by the absolute value of each complex number data (that is, equalizes the absolute value of each pixel data), and the second complex number data S122. Is output to the multiplication circuit 126.

  For example, the FFT circuit 123 performs Fourier transform on the original proof data CERa input from the computer 5 to generate first frequency component data S123, and the whitening circuit 124 configures the first frequency component data S123. Each complex number data is divided by the absolute value of each complex number data to generate first complex number data S124, which is output to complex conjugate circuit 125.

The complex conjugate circuit 125 generates third complex number data S125 in which each complex number data constituting the first complex number data S124 is replaced with complex conjugate complex number data, and outputs the third complex number data S125 to the multiplication circuit 126.
The multiplication circuit 126 multiplies the second complex number data S122 and the third complex number data S125 to generate fourth complex number data S126, and outputs this to the IFFT circuit 127.
The IFFT circuit 127 performs inverse Fourier transform on the fourth complex number data S126 to generate correlation data S42, and outputs this to the determination unit 43 shown in FIG.

Hereinafter, a method for determining a value used as a criterion for determining the level of correlation described above by the determination unit 43 illustrated in FIG. 10 will be described.
The correlation detection unit 42 sets all values of correlation values S42 by cyclically shifting the relative position between the original image data ORGa and the original certificate data CERa in two dimensions.
Here, since the original image data ORGa and the original proof data CERa are uncorrelated with respect to the design, values other than the origin of the correlation data S42 indicate an accidental correlation value between the uncorrelated data.
The determination unit 43 obtains the standard deviation σ of the correlation data S42 and performs the above determination based on whether or not the origin value c00 of the correlation data S42 exceeds a predetermined number of times the standard deviation.
The reason why the value c00 is used is that when the correlation is made between the entire images, the unique pattern of the image sensor matches the origin, so that a peak appears in the output c00 of the correlator in that case. .

Each pixel data in the correlation data S42 is Cij, and the number of pixel data is n.
The determination unit 43 generates an average value mean of the values indicated by all the pixel data in the correlation data S42 based on the following formula (1).

(Equation 1)
cmean = (Σcij) / n (1)

  Moreover, the determination part 43 produces | generates standard deviation (sigma) based on following formula (2) using the said average value mean.

(Equation 2)
σ = √ {{Σ (cij-cmean) × (cij-cmean)} / n}
... (2)

  Then, based on the following equation (3), the determination unit 43 determines that the correlation level is the first level described above when the value indicated by the origin pixel data c00 in the correlation data S42 exceeds 10 times the standard deviation σ. It is determined that the level exceeds (high correlation).

(Equation 3)
c00> 10 × σ (3)

As described above, according to the data processing system 1, in the digital camera 3, the original certificate data CER is added to the original image data ORG12 to generate the original image data ORG1, and this is encrypted to obtain the original image data ORG. Generate.
Then, in the digital camera 3, the seed data SEED obtained by encrypting the seed data SEED1 used for generating the original proof data CER is written in the recording medium 21 in association with the original image data ORG.
Therefore, it is not necessary for the verification-side computer 5 and the verification device 6 to store the original proof data CER and the seed data SEED, and the management burden on the verification side is reduced.
In addition, when the confidentiality of the source certificate data CER is lost, the digital camera 3 changes the seed data SEED to be used, but there is no need to notify the verification side to that effect, and the user of the digital camera 3 In addition, the burden on the verification side is reduced.

  Further, according to the data processing system 1, as described above, in the collation device 6, as shown in FIG. 11, the correlation between the original image data ORGa and the original proof data CERa for each of the R, G, and B pixel data. Therefore, it is possible to determine with high reliability whether or not the original image data ORG is a legitimate original generated by the digital camera 3.

Second Embodiment In the present embodiment, the following configurations and functions are further added to the configurations and functions of the first embodiment described above.
In addition to the original image data ORG stored in the recording medium 21, the computer 5 shown in FIG. 1 is configured to input the determined image data OBJ (the determined image data of the present invention) based on the original certificate data CER. The validity is judged.
FIG. 13 is a diagram for explaining the data processing system 1a of the present embodiment.
The computer 5a has the same configuration as that of the first embodiment.
The computer 5 a reads at least the seed data SEED of the digital camera 3 from the recording medium 21.
Further, the computer 5a inputs the judged image data OBJ.
The computer 5a performs the processing described in the first embodiment based on the seed data SEED, generates original proof data CERa including image data IM21_G, IM21_R, and IM21_B, and outputs this to the collation device 6.
In addition, the computer 5a performs the same processing as the processing performed on the original image data ORG described above on the determination image data OBJ, and the determination image data OBJa including the image data IM11_G, IM11_R, and IM11_B. Is output to the verification device 6a.

The determination unit 43a of the collation device 6a of the present embodiment shown in FIG. 10 performs determination as shown below based on the original proof data CERa and the determination target image data OBJa input from the computer 5a.
FIG. 14 is a diagram for explaining the determination process of the determination unit 43a.
Hereinafter, each step shown in FIG. 14 will be described.
Step ST1:
As shown in FIG. 15, the determination unit 43a includes G image data IM11_G shown in FIG. 8 of the determination target image data OBJa, and R, G, B image data IM21_G shown in FIG. Based on the result of correlation with each of IM21_R and IM21_B, it is determined whether or not all of these correlations are equal to or higher than a predetermined first level. If not, the process proceeds to step ST3.

Step ST2:
The determination unit 43a determines that the determination target image data OBJa has been generated based on the image data S4 captured by the digital camera 3 and disclosed through the processing of the computer 4.
For example, as shown in FIG. 16, when the image data S4 generated and released by the computer 4 is directly input to the collation device 6a as the image data OBJa, the image data S4 is displayed as shown in FIG. Since the data IM3_R and IM3_B are generated based on the image data IM_G, the correlation between the image data IM3_R and IM3_B and the G image data IM11_G of the original image data ORGa shown in FIG.
Note that even if the image data S4 and the original image data ORGa are generated by imaging different imaging objects by the digital camera 3, the manufacturing variation caused in the manufacturing sensitivity of the light receiving element of the light receiving unit 11 described above is generated. Due to the uniqueness, the correlation shown in FIG. 15 is obtained.

In addition, as shown in FIG. 17, for example, when a large number of image data S4 obtained based on the original image data ORG captured by the digital camera 3 is disclosed, an averaging process AVR is applied to these image data S4. To generate image data S52. Thereby, a part of the image pattern information disappears in the image data S52.
Then, the original image data ORGb obtained based on the imaging result of the digital camera 3b and the image data S52 are multiplied by the pixel data unit to generate the image data S53.
In this case, when the image data S53 is input to the collation device 6a as the image data OBJ shown in FIG. 13, the correlation shown in FIG. 15 is obtained in the collation device 6a.
Thereby, the collation device 6 can determine that the image data S53 is forged based on the image data generated by the digital camera 3 instead of the original original image data.

As shown in FIG. 18, for example, when image data S4 obtained based on original image data ORG imaged by the digital camera 3 is disclosed, the color of the digital camera 3 shown in FIG. The R, G, B pixel data corresponding to the R, G, B arrangement of the filter 25 is extracted and rearranged to generate image data S58 forgering the raw data of the other digital camera.
When the forged image data S58 is falsified as the original image data of the other digital camera and input to the computer 5 as judged image data OBJ to generate the image data S58a, which is input to the collation device 6a In addition, when the correlation is detected by the collation device 6a based on the original proof data CERa of the digital camera 3, the same result as FIG. 15 is obtained as shown in FIG.
When the image data S58a is detected by the collating device 6a based on the original certificate data CERb of the other digital camera, the result shown in FIG. 19B is obtained.
Thereby, the collation device 6a can determine that the image data S58 is not the original image data of the other digital camera but the raw data forged based on the original image data ORG generated by the digital camera 3.

Also, as shown in FIG. 20, for example, when image data S4 obtained based on original image data ORG captured by the digital camera 3 is disclosed, the color of the digital camera 3 shown in FIG. The R, G, B pixel data corresponding to the R, G, B arrangement of the filter 25 is extracted and rearranged to generate the image data S58.
Then, the forgery image data S59 is generated by performing the multiplication processing MUL of the original image data ORGb obtained based on the imaging result of the digital camera 3b and the image data S58 in units of pixel data.
Then, when this forged image data S59 is falsely input as the original image data of the digital camera 3b and input to the verification device 6a, the correlation is detected by the verification device 6a based on the original image data ORGa of the digital camera 3b. The same result as FIG. 15 is obtained as shown in FIG.
When the correlation is detected by the collation device 6 based on the original proof data CERb of the digital camera 3b for the image data S59, the result shown in FIG. 21B is obtained.
Thereby, the collation device 6a can determine that the image data S58 is not the raw data of the digital camera 3 but the raw data forged based on the image data generated by the digital camera 3b.

Step ST3:
As illustrated in FIG. 22, the determination unit 43a correlates the R image data IM11_R illustrated in FIG. 8 of the determination target image data OBJa with the R image data IM21_R illustrated in FIG. The G image data IM11_G shown in FIG. 9 and the G image data IM21_G shown in FIG. 9 constituting the source certificate data CERa, the G image data IM11_B shown in FIG. 8 and the source certificate data CERa shown in FIG. It is determined whether or not all the correlations with the image data IM21_B exceed the first level. If so, the process proceeds to step ST4, and if not, the process proceeds to step ST5.

Step ST4:
The determination unit 43a determines that the determination target image data OBJ is original image data captured by the digital camera 3 corresponding to the original certification data CERa.
That is, when the determination image data OBJ is generated by the digital camera 3 and has not undergone the processing by the computer 4, the determination image data OBJa shown in FIG. The image data IM11_G of the original image proof data CERa has a first level or higher correlation only with the G image data IM21_G shown in FIG. 9, and the R image data IM11_R of the original image data ORGa shown in FIG. Only correlation with the R image data IM21_R of the certification data CERa shown in FIG. 9 occurs at the first level or higher, and the B image data IM11_B of the original image data ORGa shown in FIG. The first level or higher correlation is generated only with the B image data IM21_B shown in FIG. Thereby, the correlation shown in FIG. 22 is obtained.

Step ST5:
The determination unit 43a determines that the image data is generated based on original image data generated by a digital camera other than the digital camera 3 corresponding to the determination target image data OBJa and the original certificate data CERa.
This is the case, for example, when the correlation result in the correlation detection unit 42 is as shown in FIG.

  As described above, according to the data processing system 1, the collation apparatus 6a performs correlation detection and determination as shown in FIG. 14, so that the determination target image data OBJ is the public image data S4 of the digital camera 3, or It can be determined whether the image data is generated based on this.

Thereby, it is possible to appropriately determine whether the original image data ORG is obtained by illegally using image data obtained by another person's digital camera. At this time, it becomes possible to identify the other person's digital camera, which was not possible with the conventional method of adding digital signature information or digital watermark information based on a hash value to image data.
Further, it is possible to appropriately avoid registering forged data obtained by illegally tampering with image data of another person as original image data ORG.

The present invention is not limited to the embodiment described above.
For example, an encoder 13b as shown in FIG. 24 may be used instead of the encoder 13 shown in FIG.
In the encoder 13b shown in FIG. 24, the memory 14b stores seed data SEED1_R, SEED1_G, and SEED1_B in each of R, G, and B.
The source certificate data generation unit 15b generates source certificate data CER_R, CER_G, and CER_B based on each of the seed data SEED1_R, SEED1_G, and SEED1_B read from the memory 14b.
The superimposing unit 16b superimposes the source certificate data CER_R on the R pixel data of the source image data ORG12, superimposes the source certificate data CER_G on the G pixel data of the source image data ORG12, and sets the source certificate data CER_B on the source image data ORG12. The original image data ORGb is generated by superimposing the pixel data on the B pixel data and output to the encryption unit 17.
As a result, source image data ORG1b to which source certificate data CER_R, CER_G, and CER_B that are uncorrelated with each other between R, G, and B is generated.
The encryption unit 18 generates seed data SEED_R, SEED_G, and SEED_B by encrypting with the key data K1 of the seed data SEED1_R, SEED1_G, and SEED1_B, and outputs this to the writing unit 19.
The writing unit 19 associates the original image data ORG with the seed data SEED_R, SEED_G, and SEED_B and writes them in the recording medium 21.
In this way, by adding the original image data ORG1b to which the original certificate data CER_R, CER_G, and CER_B that are uncorrelated with each other between R, G, and B are added, FIG. 11, FIG. 15, FIG. 19, FIG. The determination based on the correlation results of R, G, and B described with reference to FIGS. 22 and 23 can be performed with higher accuracy.

  The present invention can be applied to a system in which certification data is added to image data corresponding to an imaging result and the validity of the image data is verified.

FIG. 1 is a configuration diagram of a data processing system according to the first embodiment of this invention. FIG. 2 is a block diagram of the encoder shown in FIG. FIG. 3 is a diagram for explaining a color filter used in the light receiving unit shown in FIG. FIG. 4 is a diagram schematically illustrating a cross section of a light receiving element included in the camera. FIG. 5 is a diagram for explaining interpolation processing in the computer 4 shown in FIG. FIG. 6 is a diagram for explaining the interpolation processing shown in FIG. FIG. 7 is a block diagram of the computer 5 shown in FIG. FIG. 8 is a diagram for explaining the processing of the interpolation unit shown in FIG. FIG. 9 is a diagram for explaining the processing of the interpolation unit shown in FIG. FIG. 10 is a block diagram of the collation apparatus shown in FIG. FIG. 11 is a diagram for explaining a determination method of the collation device shown in FIG. FIG. 12 is a configuration diagram of the correlation detection unit 4 shown in FIG. FIG. 13 is a configuration diagram of a data processing system according to the second embodiment of the present invention. FIG. 14 is a flowchart for explaining an operation example of the collation device shown in FIG. FIG. 15 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 16 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 17 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 18 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 19 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 20 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 21 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 22 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 23 is a diagram for explaining a determination method in the collation apparatus shown in FIG. FIG. 24 is a diagram for explaining a modification of the encoder shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Data processing system, 3 ... Digital camera, 4 ... Computer, 5, 5a ... Computer, 6, 6a ... Verification apparatus, 13, 13b ... Encoder, 14 ... Memory, 15 ... Original proof data generation part, 16 ... Superimposition part , 17 ... encryption section, 18 ... encryption section, 19 ... writing section, 25 ... color filter, 31 ... light receiving element, 32 ... silicon substrate, 33 ... opening, 34 ... light shielding layer, 35 ... condensing lens, 41 ... Interface, 42 ... Correlation detector, 43, 43a ... Determining part, 50 ... Reading part, 51 ... Decoding part, 52 ... Decoding part, 53 ... Original proof data generation part, 54 ... Interpolation part

Claims (16)

An imaging device that generates digital image data according to an imaging result,
R, G, and B pixel data obtained by imaging one of R (red), G (green), and B (blue) light at each of a plurality of pixel positions arranged in a matrix. Imaging means for generating first image data comprising:
Proof data generating means for generating proof data used for performing predetermined proof for the first image data generated by the imaging means based on predetermined data;
Adding means for generating the second image data by adding the proof data generated by the proof data generating means to the first image data generated by the imaging means;
An encryption unit for encrypting the predetermined data and the second image data generated by the addition unit;
An imaging device comprising: an interface that associates the predetermined data encrypted by the encryption unit with the encrypted second image data and outputs the second image data to the outside of the imaging device. The imaging apparatus according to claim 1, wherein the proof data generation unit generates random numbers based on the predetermined data and generates the proof data having the same size as the first image data. The proof data generation means is configured to generate a plurality of the predetermined data based on a plurality of different predetermined data corresponding to each of R pixel data, G pixel data, and B pixel data constituting the first image data. Generate proof data,
The adding means adds the proof data generated by the proof data generating means corresponding to R to the R pixel data of the first image data, and the proof data generating means generated corresponding to G The certification data is added to the G pixel data of the first image data, and the certification data generated by the certification data generation unit corresponding to B is added to the B pixel data of the first image data. The imaging device according to claim 2, wherein the second image data is generated. The imaging apparatus according to claim 1, wherein the imaging unit is provided corresponding to each of the plurality of pixel positions and forms an image of the light on a light receiving element having a variation unique to light receiving sensitivity. The imaging apparatus according to claim 1, wherein the proof data generation unit generates the proof data indicating that the first image data is generated by the imaging apparatus. Proof data generating means for generating proof data used for performing a predetermined proof for the first image data based on the predetermined data;
Adding means for generating the second image data by adding the proof data generated by the proof data generating means to the first image data;
An encryption unit for encrypting the predetermined data and the second image data generated by the addition unit;
A data processing apparatus comprising: association means for associating the predetermined data encrypted by the encryption means with the second image data encrypted. Encrypted image data, and decryption means for decrypting the encrypted predetermined data associated with the image data;
Proof data generating means for generating proof data for performing predetermined proof on image data obtained by decoding by the decoding means based on the predetermined data obtained by decoding by the decoding means;
Correlation detecting means for detecting a correlation between the image data obtained by decoding by the decoding means and the proof data generated by the proof data generating means;
Based on the correlation detected by the correlation detection means, the image data obtained by decoding by the decoding means is associated with the proof data generated by the proof data generation means and is generated according to the imaging result. A data processing apparatus comprising: a determination unit configured to determine whether the image data is generated using an imaging apparatus that adds the certification data to the image data. The correlation detection unit is configured to perform a first correlation between R pixel data constituting the image data obtained by decoding by the decoding unit and R pixel data constituting the certification data, and the decoding unit performs decoding. Second correlation between G pixel data constituting the image data obtained in the above and G pixel data constituting the certification data, and B constituting the image data obtained by decoding by the decoding means Detecting a third correlation between the pixel data and the B pixel data constituting the proof data;
The determination means is obtained by decoding by the decoding means when all of the first correlation, the second correlation and the third correlation detected by the correlation detection means are equal to or higher than a predetermined level. The data processing device according to claim 7, wherein the image data is determined to be image data generated by the imaging device. The correlation detection unit further detects a correlation between the image data to be determined and the proof data generated by the proof data generation unit,
The determination unit is configured to add the verification data to image data generated according to an imaging result in which the determination target image data is associated with the verification data based on the correlation detected by the correlation detection unit. The data processing apparatus according to claim 7, further determining whether or not the data has been generated using the data. The correlation detection means includes: a first correlation between R pixel data constituting the judged image data and R pixel data constituting the certification data; G pixel data constituting the judged image data; Detecting a second correlation with G pixel data constituting proof data, and a third correlation between B pixel data constituting the determination image data and B pixel data constituting the proof data;
The determination unit determines that the image data to be determined is detected by the imaging device when all of the first correlation, the second correlation, and the third correlation detected by the correlation detection unit are equal to or higher than a predetermined level. The data processing apparatus according to claim 9, wherein the data processing apparatus determines that the image data is generated. The correlation detection means detects the correlation between each of the R, G, and B pixel data that constitutes the image data to be judged and the G pixel data that constitutes the certification data,
The determination means is an image generated by decoding the image data generated by using the imaging device when the image data to be determined is equal to or higher than a predetermined level of all the correlations detected by the correlation detection means. The data processing device according to claim 9, wherein the data processing device is determined to be data. The correlation detection unit is configured to respond to a light reception result of the light receiving element constituting the image data on the image data generated by the imaging device in which reflected light from a subject is incident on a light receiving element having a variation unique to light receiving sensitivity. The data processing apparatus according to claim 7, wherein a process of reflecting a variation in the light receiving sensitivity generated in the pixel data in the correlation is performed. The correlation detection means includes
Transform means for orthogonally transforming the image data and the proof data obtained by the decryption means to generate first frequency component data and second frequency component data, respectively;
Each complex number data constituting the first frequency component data is divided by the absolute value of each complex number data to generate first complex number data, and each complex number data constituting the second frequency component data is converted to each complex number data constituting the second frequency component data. Dividing means for generating the second complex data by dividing by the absolute value of each complex data;
Replacement means for generating third complex number data by replacing each complex number data constituting one of the first complex number data and the second complex number data with complex conjugate complex data;
Multiplying the first complex number data or the second complex number data that has not been replaced by the replacing unit and the third complex number data generated by the replacing unit to generate fourth complex number data Multiplying means to
The data processing apparatus according to claim 7, further comprising: an inverse transform circuit that performs inverse orthogonal transform on the fourth complex number data generated by the multiplication unit to generate the correlation. A first step of generating proof data used for performing a predetermined proof on the first image data based on the predetermined data;
A second step of generating second image data by adding the certification data generated in the first step to the first image data;
A third step of encrypting the predetermined data and the second image data generated in the second step;
A data processing method comprising: a fourth step of associating the predetermined data encrypted in the third step with the second image data encrypted. A first step of decrypting the encrypted image data and the predetermined encrypted data associated with the image data;
A second step of generating proof data for performing a predetermined proof on the image data obtained by decoding in the first step based on the predetermined data obtained by decoding in the first step; When,
A third step of detecting a correlation between the image data obtained by decoding in the first step and the certification data generated in the second step;
Based on the correlation detected in the third step, the image data obtained by decoding in the first step is associated with the certification data generated in the second step, and depending on the imaging result And a fourth step of determining whether or not the image data is generated using an imaging device that adds the certification data to the image data to be generated. A fifth step of detecting a correlation between the image data to be determined and the proof data obtained by decoding in the first step;
Based on the correlation detected in the fifth step, the image data to be determined is associated with the certification data, and generated using an imaging device that adds the certification data to the image data generated according to the imaging result A data processing method further comprising: a sixth step of determining whether or not the processing has been performed.

Images