JP2006338221A - Image generation/storage device, image generation/storage method and image generation/storage program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve retrieving or authenticating precision by generating and storing image data by temporally and spatially changing predetermined image data. <P>SOLUTION: This image generation/storage device is provided with a first storage means for storing image data, a second storage means for storing a speed to decide the movement of pixels, an input means for inputting image data, a generation means for generating at least one image data by temporally and spatially changing the image data by applying a partial differential equation to the inputted image data based on a speed stored in the second storage means and a storage means for adding identical identification information associated with the image data inputted by the input means to each generated image data, and for storing the image data in the first storage means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for generating an image and storing the generated image.

  As a technique for editing image data such as a still image and generating new image data, for example, there is a technique called morphing, or a technique described in Patent Document 1.

  Morphing is to create an intermediate image to compensate for the middle in order to express a moving image of a gradual change from one shape to another. FIG. 5 is a diagram schematically showing generation of an intermediate image by morphing. In FIG. 5, corresponding points are set automatically or artificially between a cylindrical image 500 and a rectangular parallelepiped image 510. Then, by appropriately changing the combination of corresponding points, an intermediate image 520 similar to the two input images 500 and 510 is generated.

Patent Document 1 describes a technique for generating a continuous moving image from a still image using an advection equation.
JP 2004-334694 A

  Now, in the present day when various kinds of information are digitized, the importance of storing and retrieving digitized data is increasing. In addition, various types of data such as past weather data, addresses, and time-series images taken by a monitoring camera are structurally stored in the database. From the viewpoint of security, face images and palm images may be stored in a database to perform personal authentication.

  In general, data stored in a database is often discrete in time or space (ie, has a low density). For example, when accumulating facial images in a database for personal authentication, only image data in a predetermined direction such as front or side is often stored.

  However, when performing personal authentication, if the orientation of the face of the person to be authenticated is different from the orientation stored in the database, there is no corresponding face image data, so even the person fails to authenticate. May end up. Although it is conceivable that images of faces of all orientations are taken with a camera and stored in a database, the workload is large and not realistic. That is, since the shooting time and the devices used are limited, it is unrealistic to perform shooting of various variations for each person for a long time. Thus, the data stored in the database is often discrete in time or space.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to generate and accumulate image data obtained by changing predetermined image data temporally and spatially, thereby further improving the accuracy of search or authentication. There is to make it.

  In order to solve the above-described problems, the present invention provides, for example, an image generation and storage device, a first storage unit that stores image data, and a second storage unit that stores in advance a speed for determining pixel movement. And applying a partial differential equation to the input image data on the basis of the speed stored in the second storage means, the input means for inputting the image data, and the image data is temporally and spatially applied. Generating means for generating at least one image data changed to the same, and adding the same identification information associated with the image data input by the input means to each of the generated image data, Storage means for storing in the storage means.

  The present invention is also an image generation and accumulation method performed by the information processing apparatus, wherein the information processing apparatus stores a first storage unit that stores image data and a second that stores in advance a speed for determining pixel movement. A storage unit, and a processing unit, the processing unit biasing the input image data based on an input step of inputting image data and a speed stored in the second storage unit. Applying a differential equation to generate at least one image data obtained by temporally and spatially changing the image data, and associating each of the generated image data with the image data input in the input step And a storage step of adding the same identification information and storing the same in the first storage unit.

  The present invention is also an image generation and accumulation program executed by an information processing apparatus, wherein the information processing apparatus stores a first storage unit for storing image data and a speed for determining pixel movement in advance. Two storage units and a processing unit, and the processing unit is configured to input the image data based on an input step of inputting image data and a speed stored in the second storage unit. A generation step of applying partial differential equations to generate at least one image data obtained by temporally and spatially changing the image data, and corresponding to the image data input in the input step for each of the generated image data And adding the same identification information, and storing in the first storage unit.

  According to the present invention, it is possible to generate and store image data obtained by changing predetermined image data temporally and spatially, thereby further improving the accuracy of search or authentication.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a configuration diagram of an image generation / storage device 1 to which an embodiment of the present invention is applied. The illustrated image generation / storage apparatus 1 is an apparatus that generates extended image data obtained by changing predetermined image data temporally and spatially, and stores the extended image data in a database.

  The image generation / storage device 1 includes an image input unit 11 for inputting original image data, an image storage unit 12 for storing (storing) input image data, and image data stored in the image storage unit 12 to be described later. An extended image generation unit 13 that generates extended image data by applying a partial differential equation or the like, an extended image storage unit 14 that stores (stores) extended image data generated by the extended image generation unit 13, and an extended image data And a display unit 15 for displaying.

  The image storage unit 12 and the extended image storage unit 14 are so-called databases. Further, the image storage unit 12 and the extended image storage unit 14 may be integrated into one database.

  The image generation / storage device 1 described above includes a CPU, a memory, an external storage device, an input device such as a keyboard and a scanner, an output device such as a display, and a communication control device for connecting to other devices. A general-purpose computer system including a bus connecting these devices can be used. In this computer system, each function of the image generation and storage device is realized by the CPU executing a program for the image generation and storage device 1 loaded on the memory. Note that a memory or an external storage device is used for the image storage unit 12 and the extended image storage unit 14 of the image generation storage device 1.

  The program for the image generation / storage device 1 can be stored in a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, or an MO, or can be distributed via a network.

  Next, image generation processing using the image generation / storage device 1 of the present embodiment will be described.

  First, the image input unit 11 inputs predetermined image data and stores it in the image storage unit 12.

Note that the image input unit 11 may also input the image data ID (identification information) of the image data and store it in the image storage unit 12 when inputting the image data.

  Then, the extended image generation unit 13 inputs the image data stored in the image storage unit 12 into a partial differential equation or an equation corresponding to a change in illumination described later, and the extended image data is changed temporally and spatially. At least one piece of image data is generated. Then, the extended image generation unit 13 adds the same extended image data ID (identification information) as a search key to each of the generated extended image data. It is assumed that the extended image data ID is associated with the image data input by the image input unit 11. That is, it is assumed that the image data input by the image input unit 11 can be specified using the extended image data ID.

  Note that the image data ID input by the image input unit 11 may be used as the extended image data ID. The extended image data ID may be a unique ID generated by the extended image generation unit 13 using a predetermined algorithm or a unique ID input by the user.

  Then, the extended image generation unit 13 stores the extended image data set with the same extended image ID in the extended image storage unit 14. The display unit 15 displays the extended image data stored in the extended image storage unit 14 on the output device as necessary.

  Next, the partial differential equation used by the extended image generation unit 13 will be described.

  In the present embodiment, for example, the partial differential equation of Equation 1 shown below is used. The partial differential equation is described in, for example, Shunsuke Ikeda “Nonlinear phenomena in fluids—Mathematical analysis and its application”, Asakura Shoten, or Japanese Patent Application Laid-Open No. 2004-334694 (Patent Document 1).

I (x, y, n + 1) = I (x, y, n) −Δt · (U · ∇I (x, y, n) −λ∇ · ∇I (x, y, n)) Equation 1
Further, the above formula 1 can be modified and expressed by the following formula 2.

{I (x, y, n + 1) −I (x, y, n)} / Δt = −U · ∇I (x, y, n) + λ∇ · ∇I (x, y, n) 2
In Equation 2, the left side is the time term, the first term on the right side is the advection term, the second term is the diffusion term, and the image intensity I of the pixels constituting the image is a variable. If the pixel is (x, y) and the discretized time (or time) is n, I (x, y, n) is the image intensity at the discretized time n of the pixel (x, y). It is.

  Δt is the time interval, λ is the diffusion coefficient, and U is the velocity. Further, １ is a primary spatial differential operator, ∇ / ∇ is a secondary spatial differential operator, and “·” is a vector inner product. Note that the advection term in Equation 2 is represented by the product of the velocity U and the first-order spatial derivative of the image intensity I (x, y, n).

  As for the time interval Δt, the diffusion coefficient λ, and the number of integrations, predetermined values are set in advance according to the degree of deformation of the generated image. Note that it is preferable to set the time interval Δt to a relatively small value in consideration of the calculation stability condition, and to relatively increase the number of integrations accordingly. For example, “0.05 seconds” may be set as the time interval Δt, “1.0” may be set as the diffusion coefficient λ, and “100” may be set as the number of integrations.

  The speed U defines how each pixel of the image data moves in the newly generated extended image data (movement method, magnitude of movement, etc.). That is, the speed U defines movement corresponding to the future or past of the image data.

  Note that the velocity U of the present embodiment is a two-dimensional velocity vector, and a plurality of two-dimensional velocity vectors (rotation vector, translation vector, coordinate transformation vector, etc.) representing rotation, translation, diffusion, divergence, etc. are stored in the memory or It is assumed that it is stored in advance in an external storage device. Then, according to the type of input image data (or generated extended image data), a predetermined two-dimensional velocity vector is set in the partial differential equation from among the two-dimensional velocity vectors stored in the memory or the like. And

  For example, when inputting face image data as shown in FIG. 2 to be described later, a vector that rotates a two-dimensional photograph in accordance with the unevenness of the face is stored in advance in a memory or an external storage device as a speed U. I will keep it. Although it is desirable to use a vector obtained by projecting a three-dimensional velocity vector corresponding to the unevenness of the eyes and nose into a two-dimension, for simplicity, an ellipse corresponding to the size of the face is projected onto the two-dimensional velocity vector. It may be used.

  Note that the user may set (or edit) an arbitrary two-dimensional velocity vector.

  The extended image generation unit 13 discretizes the partial differential equation shown in Formula 1 or Formula 2 according to the difference method, and recursively repeats integration along the two-dimensional image plane and the discrete time axis. I (x, y, n) is an input image intensity (known amount), and I (x, y, n + 1) is an image output by calculating Equation 1 or Equation 2. It is strength. That is, by setting a predetermined number of integrations and calculating Formula 1 or Formula 2 repeatedly for the set number of integrations, desired extended image data can be generated (output).

  The generated extended image data is obtained by changing the input image data based on the speed U only at a time (past direction or future method) designated from the time corresponding to the image data.

  Further, the extended image generation unit 13 may generate corrected image data in which illumination of input image data is changed. That is, the extended image generation unit 13 adds or subtracts various luminance values F to the input screen data (n = 0) using Expression 3 shown below to obtain corrected image data corresponding to the illumination change. It may be generated.

I (x, y, n + 1) = I (x, y, n) + F Equation 3
F is a luminance value. When the illumination is a surface light source, a linear equation may be used, and when the illumination is a point light source such as a spotlight, a nonlinear equation may be used. As a nonlinear equation, for example, it is conceivable to use Equation 4 shown below for the luminance value F.

F = Ae- ^λn Formula 4
A is a predetermined constant.

  When a plurality of corrected image data in which the illumination is changed is generated in this way, the extended image generation unit 13 applies the partial differential equation of Formula 1 or Formula 2 for each corrected image data, and changes the illumination and speed. Extended image data may be generated. The extended image generation unit 13 may generate extended image data in which illumination and speed are changed by applying Expression 3 to extended image data generated from predetermined image data using a partial differential equation. .

  Next, the image generation processing of the image generation / storage device 1 will be specifically described with reference to FIGS. 2 and 3.

  FIG. 2 is an explanatory diagram when face images 210 and 220 of different viewpoints are generated from one face image 200.

  First, the image input unit 11 inputs a face image 200 facing the front as input image data, and stores the input image data in the image storage unit 12. The image input unit 11 also inputs an image data ID (such as a user ID) of the input image data and stores it in the image storage unit 12.

  Then, the extended image generation unit 13 uses the initial value of I (x, y, n) as the front-facing face image data 200, integrates the partial differential equation (Equation 1 or Equation 2) a predetermined number of times, and performs different viewpoints. A plurality (for example, about 20) of extended image data of the face is generated. Then, the extended image generation unit 13 adds the same extended image data ID as a search key to each of the generated extended image data, and stores it in the extended image storage unit 14. In this case, an image data ID (user ID) input by the image input unit 11 is used as the extended image data ID.

  In addition, when generating the image data 210 of the downward face, the extended image generation unit 13 sets a two-dimensional velocity vector for rotating the two-dimensional photograph downward according to the unevenness of the face to the velocity U of the partial differential equation. Shall be set. Further, when generating the image data 220 of the face in the horizontal direction, the extended image generation unit 13 sets a two-dimensional velocity vector for rotating the two-dimensional photograph in the horizontal direction according to the unevenness of the face to the speed U of the partial differential equation. Shall be set.

  In this way, a plurality of extended image data of different viewpoints is generated from one face image data, and the same extended image data ID is added to each of the generated extended image data and stored in the extended image storage unit 14. . By using a plurality of extended image data of the image storage unit 14, more reliable authentication processing can be performed.

  For example, in the personal authentication, the extended image data to which the extended image ID that matches the user ID (or a predetermined ID associated with the user ID) input at the time of authentication is added is extracted. The extracted extended image data is image data in which the person to be authenticated is generated from different viewpoints. Therefore, the person to be authenticated is not facing a predetermined direction (eg, front) at the time of authentication. However, you can authenticate yourself.

  FIG. 3 is a diagram illustrating an example of extended image data stored in the extended image storage unit 14.

  In FIG. 3, a plurality of extended image data (including corrected image data) generated using the partial differential equation (Equation 1 or 2) and the calculation equation (Equation 3) corresponding to the illumination change are used as a matrix database. Represents.

  For example, FIG. 3A shows a case in which image data having a predetermined pattern is input to a partial differential equation, and extended image data generated every time repeated calculation (integration) is accumulated in the vertical axis direction. FIG. 3A shows the corrected image data generated every time the image data (or extended image data) of a predetermined pattern is input to a calculation formula corresponding to the illumination change and repeatedly calculated. It has been accumulated in.

  Note that, by repeatedly setting the brightness value F of the calculation formula corresponding to the illumination change to “2.0”, for example, corrected image data in which the brightness of the image has changed linearly is generated, and sequentially in the horizontal axis direction. Accumulated. Further, the continuity of the generated corrected image data changes according to the value set as the luminance value F. 3B, 3C, and 3D are the same as FIG. 3A.

  FIG. 4 is a graph of the face recognition rate for the same face recognition algorithm. The vertical axis is the recognition rate, and the horizontal axis is the normalized and quantified (small, medium, large) face rotation change amount (face direction change amount) and illumination change amount.

  For the similarity between the face image data group stored in the database and the unknown face image data to be recognized, the normalized correlation function of both is calculated, and the larger the value, the more similar Is done. The normalized correlation function is described in Hideyuki Tamura, “Introduction to Computer Image Processing”, Soken Publishing, Japan Industrial Technology Center. Further, the face recognition algorithm uses a general face recognition algorithm distributed in the market.

  In FIG. 4, when the database of the extended image storage unit 14 storing the extended image data generated according to the present embodiment is used, the recognition rate rapidly decreases even when the rotation / illumination change amount is large. It shows that a high recognition rate can be secured. On the other hand, in the case of using the conventional database 410, there is only limited face data (face data with a predetermined direction and a predetermined illumination) due to camera photography or the like, and therefore the recognition rate when the rotation / illumination change amount is large. Shows a sharp drop.

  In the present embodiment described above, by inputting predetermined image data to the partial differential equation, extended image data of various variations in which the image data is temporally and spatially changed can be generated. Then, by accumulating the generated extended image data in order, a database having a high temporal and spatial density can be created.

  Further, by using the extended image data generated in the present embodiment, when the image that is originally the same does not match in the search or authentication (such as personal authentication) of the image data, or the image that is originally the same The case where it misses can be avoided. That is, in this embodiment, the accuracy of search or authentication can be further improved.

  In the present embodiment, corrected image data corresponding to a change in illumination is generated. Thereby, even if the illumination at the time of authentication is changing, the image data that is originally the same can be detected with higher accuracy.

  In addition, this invention is not limited to said embodiment, Many deformation | transformation are possible within the range of the summary. For example, in the present embodiment, one piece of image data is input to the partial differential equation to generate extended image data that is temporally and spatially changed. However, the present invention is not limited to this, and a plurality of time-series image data may be input, and extended image data that complements the plurality of image data may be generated.

  In this embodiment, image data is used. However, application to various data other than image data is also conceivable.

1 is a diagram illustrating an overall configuration of an image generation / storage device to which an embodiment of the present invention is applied. It is explanatory drawing in the case of producing | generating the face image data of a different viewpoint from the image data of one face. It is a figure which shows an example of a matrix-like database. It is the figure which graphed an example of the face recognition rate. It is the figure which showed typically the production | generation of the intermediate image by morphing.

Explanation of symbols

1: Image generation and storage device 11: Image input unit, 12: Image storage unit, 13: Extended image generation unit, 14: Extended image storage unit, 15: Display unit

Claims (5)

An image generation and storage device,
First storage means for storing image data;
Second storage means for storing in advance a speed for determining the movement of the pixel;
Input means for inputting image data;
Based on the speed stored in the second storage means, a partial differential equation is applied to the input image data to generate at least one image data in which the image data is temporally and spatially changed. Generating means for
Storage means for adding the same identification information associated with the image data input by the input means to each of the generated image data, and storing the same in the first storage means. Image generation and storage device. The image generation and storage device according to claim 1,
The partial differential equation includes an advection term and a time term, and has a pixel image intensity as a variable,
The advection term is a product of a speed stored in the second storage means and a first-order spatial derivative of the image intensity. The image generation and storage device according to claim 1,
Correction means for generating at least one corrected image data in which the luminance of the image data input by the input means is changed by applying a predetermined linear equation or a predetermined nonlinear equation, and storing the corrected image data in the first storage means; Further comprising
The generation unit applies the partial differential equation to each of the corrected image data, and generates image data obtained by changing the corrected image data temporally and spatially. An image generation and accumulation method performed by an information processing apparatus,
The information processing apparatus includes a first storage unit that stores image data, a second storage unit that stores in advance a speed for determining pixel movement, and a processing unit.
The processor is
An input step for inputting image data;
Based on the speed stored in the second storage unit, a partial differential equation is applied to the input image data to generate at least one image data in which the image data is temporally and spatially changed. Generating step to
Adding the same identification information associated with the image data input in the input step to each of the generated image data, and storing in the first storage unit. Image generation and accumulation method. An image generation and accumulation program executed by an information processing apparatus,
The information processing apparatus includes a first storage unit that stores image data, a second storage unit that stores in advance a speed for determining pixel movement, and a processing unit.
The processor is
An input step for inputting image data;
Based on the speed stored in the second storage unit, a partial differential equation is applied to the input image data to generate at least one image data in which the image data is temporally and spatially changed. Generating step to
Adding the same identification information associated with the image data input in the input step to each of the generated image data, and storing in the first storage unit. An image generation and storage program.