JP2001249936A - Image retrieval system, computer readable recording medium having image retrieval program recorded thereon, image generation system and computer readable recording medium having image generation program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image retrieval system capable of reducing the burdens of a user taking part in image retrieval and shortening the time until an image desired by the user is retrieved. SOLUTION: This image retrieval system for interactively retrieving the image desired by the user from an image data base is provided with a spatial frequency filter for limiting the band width of spatial frequency and a means for presenting the retrieved image to the user after limiting the band width of the spatial frequency by the spatial frequency filter.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval system, a computer-readable recording medium recording an image retrieval program, an image generation system, and a computer-readable recording medium recording an image generation program.

[0002]

2. Description of the Related Art An image retrieval system for interactively retrieving an image desired by a user from an image database,
An image generation system for interactively generating an image desired by a user has already been proposed.

[0003] In a conventional system for interactively searching or generating an image intended by a user, an image is presented by the system, and the user determines the quality of the image. Then, using the determination result, the system further searches or generates the next presentation candidate and presents it to the user. By repeating this process, the images are sequentially made to reach the images desired by the user.

In such a system, the searched or generated image is presented as it is. In that case,
Since the presented image contains all the spatial frequency components, the amount of information is very large, and the user has to make a decision on many pieces of information at once. There was difficulty.

[0005]

SUMMARY OF THE INVENTION The present invention provides an image search system and an image search program which can reduce the burden on a user involved in image search and reduce the time required for searching for an image desired by the user. It is an object of the present invention to provide a recorded computer-readable recording medium.

According to the present invention, an image generation system and a computer-readable recording medium storing an image generation program are provided which can reduce the burden on a user involved in image generation and reduce the time required to generate an image desired by the user. It is an object to provide a possible recording medium.

[0007]

SUMMARY OF THE INVENTION The present invention relates to an image retrieval system for interactively retrieving an image desired by a user from an image database, comprising: a spatial frequency filter for limiting a spatial frequency bandwidth; Means for presenting the image to the user after limiting the bandwidth of the spatial frequency with the spatial frequency filter.

[0008] It is preferable that a means is provided for switching the cutoff frequency of the spatial frequency filter stepwise from a low band to a high band in accordance with the progress of image search. Further, means for causing the user to change the cutoff frequency of the spatial frequency filter, and when the cutoff frequency of the spatial frequency filter is changed by the user, the image currently presented to the user is changed. It is preferable that the image obtained by applying a spatial frequency filter using the cutoff frequency of (i) is provided with a means for changing the presented image.

The present invention is a computer-readable recording medium storing an image retrieval program for interactively retrieving an image desired by a user from an image database, and comprising a spatial frequency band for limiting a spatial frequency bandwidth. A filter and an image search program for causing a computer to function as means for presenting the searched image to a user after limiting the spatial frequency bandwidth with a spatial frequency filter are recorded.

[0010] The recording medium stores a program for causing a computer to function as a means for switching the cutoff frequency of the spatial frequency filter stepwise from a low band to a high band in accordance with the progress of image search. Is preferred. Further, the recording medium is a means for allowing a user to change the cutoff frequency of the spatial frequency filter,
And when the cutoff frequency of the spatial frequency filter is changed by the user, the image currently presented to the user is subjected to a spatial frequency filter using the changed cutoff frequency, and the image obtained by the It is preferable that a program for causing a computer to function as means for changing a presented image be recorded.

The present invention is an image generation system for interactively generating an image desired by a user, comprising: a spatial frequency filter for limiting a spatial frequency bandwidth;
And a means for presenting the generated image to the user after limiting the spatial frequency bandwidth by the spatial frequency filter.

It is preferable that a means for switching the cutoff frequency of the spatial frequency filter from a low band to a high band stepwise in accordance with the progress of image generation is provided. Further, means for causing the user to change the cutoff frequency of the spatial frequency filter, and when the cutoff frequency of the spatial frequency filter is changed by the user, the image currently presented to the user is changed. It is preferable that the image obtained by applying a spatial frequency filter using the cutoff frequency of (i) is provided with a means for changing the presented image.

[0013] The present invention is a computer-readable recording medium recording an image generation program for interactively generating an image desired by a user, comprising: a spatial frequency filter for limiting a spatial frequency bandwidth; An image generation program for causing a computer to function as means for presenting to a user after limiting the spatial frequency bandwidth of an image by a spatial frequency filter is recorded.

The recording medium records a program for causing a computer to function as means for switching the cutoff frequency of the spatial frequency filter from low to high in accordance with the progress of image generation. Is preferred. Further, the recording medium is a means for allowing a user to change the cutoff frequency of the spatial frequency filter,
And when the cutoff frequency of the spatial frequency filter is changed by the user, the image currently presented to the user is subjected to a spatial frequency filter using the changed cutoff frequency, and the image obtained by the It is preferable that a program for causing a computer to function as means for changing a presented image be recorded.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [1] Description of the concept of the present invention First, the concept of the present invention will be described.

When a human judges something about an image,
It is easier for the user at each stage to increase the amount of information sequentially without providing the information necessary for the determination at once. That is, the amount of information presented to the user is limited to the amount necessary for the user to make a single decision, and the amount of information is increased stepwise as the number of times of the decision is repeated, so that the user's decision is smoothly performed. Can be done.

It is known that the sensitivity of a human to light and shade differs depending on the spatial frequency. That is, it shows the maximum sensitivity in the middle spatial frequency band, low spatial frequency band,
It is known that sensitivity is lower in a high spatial frequency band. The sensitivity to the shape is also determined by the relationship with the spatial frequency. For example, information such as wrinkles of the face is represented by a high spatial frequency component, and the contour of the face is represented by a low spatial frequency component. Furthermore, it is known from psychology that the spatial frequency used also varies depending on the type of facial expression and facial expression, such as a gentle face, a gentle face, and an angry face (Takahiro Kirita, Mitsuo Endo, Shigeru Akamatsu, Hideo Fukamachi, "Facial Expression Recognition and Spatial Frequency Information", Proc. Of the 58th Annual Meeting of the Japanese Psychological Association, P570, 1994
).

Using these findings, low spatial frequency information such as the outline of the face and the size of the eyes is first given, and the user is allowed to determine which of these features is desirable. By giving high spatial frequency information of details such as the situation, and making a judgment, the impression on the entire face is sequentially judged. To realize this, the cutoff frequency is automatically switched according to the progress of the search or generation. Also, a means for manually switching the cutoff frequency is provided in the case where the user cannot judge the quality of an image generated using the cutoff frequency which is automatically performed.

[2] Description of an embodiment in which the present invention is applied to a face image generation system

[2-1] Description of Configuration of Face Image Generation System

FIG. 1 shows the configuration of a face image generation system.

The face image generation system is realized by a personal computer 10. A display 21, a mouse 22, and a keyboard 23 are connected to the personal computer 10. The personal computer 10 includes a CPU 11, a memory 12, a hard disk 13, and a drive (disk drive) 14 of a removable disk such as a CD-ROM.

The hard disk 13 stores a face image generation program in addition to an OS (operating system). The face image generation program is installed on the hard disk 13 using the CD-ROM 20 in which the face image generation program is stored. Also, the hard disk 13 has
A cutoff frequency conversion table (see FIG. 3) and other data are stored.

FIG. 2 shows an example of a display screen displayed on the display 21 when the face image generating process is being executed by the face image generating program. The display screen has an area for displaying ten face images.
Further, a rating value input area for inputting a rating value is provided below the display area of each face image. The user inputs the degree of satisfaction as the face image desired by the user for each face image in the rating value input area in ten levels.

At the bottom of the screen, a start button 31,
Next candidate button 32, a button marked “small” for lowering the cutoff frequency (hereinafter referred to as frequency DOWN)
Button 33, a button (hereinafter, referred to as a frequency UP button) 34 for increasing the cutoff frequency, and an end button 35 are displayed.

On the initial screen displayed on the display 21 immediately after the start of the face image generation program, only the buttons are displayed, and no face image is displayed. When the user presses the start button 31 on this initial screen, the CPU 11
Displays ten face images. When the user presses the next candidate button 32 after the user inputs rating values for the displayed ten face images, the CPU 11 generates and displays ten new face images. After the user inputs the rating value of the newly displayed face image, the user enters the next candidate button 32
When the button is pressed, the CPU 11 further generates and displays ten new images.

A face image presented in the course of such an interactive image generation process is subjected to a spatial frequency filter, and is a blurred image, that is, an image from which high spatial frequencies have been cut. This degree of blur is automatically and gradually reduced in accordance with the progress of the interactive generation process, and the image becomes gradually clearer.

Depending on the degree of image blur that is automatically adjusted, it may be difficult for the user to evaluate the face image. Therefore, the user can manually change the degree of blur of the face image. Means for manually changing the degree of blurring of the face image are the frequency DOWN button 33 and the frequency UP button 34. When the user presses the frequency DOWN button 33, the cutoff frequency is lowered from the current value, and when the user presses the frequency UP button 34, the cutoff frequency is increased from the current value, and the face image re-filtered is displayed again. When the user presses the end button 35, the system ends.

FIG. 3 shows an example of the cutoff frequency conversion table. The cut-off frequency conversion table includes, for each predetermined cut-off frequency, the range of the variance x of the characteristic parameter value of the image (weight coefficient of the unique face),
The range of the rating value y is stored. In the cutoff frequency conversion table, a check column for storing which cutoff frequency is the currently set cutoff frequency is provided for each cutoff frequency.

The feature parameter value of the image (weight coefficient of the unique face) will be described later. The variance value of the feature parameter value of the image indicates the convergence state of the image generation, and the smaller the value, the more the convergence of the image generation. In the cutoff frequency conversion table, the variance value x of the characteristic parameter value of the image is set such that the smaller the variance of the characteristic parameter value of the image, that is, the higher the convergence of the image generation, the higher the cutoff frequency. Is set. In the cutoff frequency conversion table, the range of the rating value y is set so that the cutoff frequency increases as the rating value increases.

[2-2] Description of face image generation processing procedure

FIG. 4 shows a procedure of a face image generation process performed by the CPU 11 when the face image generation program is activated.

When the user presses the start button 31 after the initial screen is displayed (step 1), 1
Zero face images are randomly generated (step 2). Further, the initial value of the cutoff frequency, that is, the minimum value of the cutoff frequency in this example, is read from the cutoff frequency conversion table (step 3). Then, a spatial frequency filter is applied to the ten face images based on the read initial value of the cutoff frequency, and the generated images are displayed on the display 21 (step 4). After that, it is in a state of waiting for input by the user.

The user evaluates the image displayed on the display 21 and inputs a rating value. At this time, if the user feels that it is necessary to adjust the degree of blur of the image in evaluating the image, the cutoff frequency is changed to adjust the degree of blur before inputting the rating value. Button (frequency DOWN button 33 or frequency UP button 34) is pressed.

If the button 33 or 34 for changing the cutoff frequency is depressed by the user (YES in step 5), the cutoff frequency is changed and the spatial frequency filter processing is performed (step 6). .

That is, first, the current cutoff frequency is read from the cutoff frequency conversion table. Frequency D
When the OWN button 33 is depressed, a spatial frequency filter is applied to each of the currently displayed face images using a cutoff frequency one step lower than the current cutoff frequency. When the frequency UP button 34 is pressed, a spatial frequency filter is applied to each of the currently displayed face images using a cutoff frequency one step higher than the cutoff frequency.

Then, the obtained ten face images are displayed on the display 21 in place of the face images displayed so far. When the cutoff frequency is changed in this way, the check mark in the cutoff frequency conversion table is moved to a check box corresponding to the changed cutoff frequency.

[0038] When the user displays 1 on the display 21
If no face images are evaluated and the degree of satisfaction is input to the rating value input area (YES in step 7), the rating value input for each face image is stored in the memory 12 (step 8).

When the next candidate button 32 is pressed by the user after the evaluation value is input (step 9
YES), the following processing is performed.

First, a next candidate image generation process is performed (step 10). That is, ten next candidate images are generated based on the rating value for each face image stored in step 8.
Details of the next candidate image generation processing will be described later.

Thereafter, a cutoff frequency determination process is performed (step 11). The details of the cutoff frequency determination processing will also be described later. If the cutoff frequency determined by the cutoff frequency determination processing is different from the cutoff frequency that has been set, the check mark in the cutoff frequency conversion table is changed to the cutoff frequency determined by the cutoff frequency determination processing. Move to the check box corresponding to the off frequency.

When the cutoff frequency is determined by the cutoff frequency determination process, a spatial frequency filter is applied to each of the ten next candidate images generated in step 11 by using the determined cutoff frequency. One
Zero face images are displayed on the display 21 in place of the face images that have been displayed so far. Further, the rating value in the rating value input area is cleared (step 12).

After the steps 5 to 12 are repeatedly performed, if the end button 35 is pressed by the user (YES in step 13), the present face image generation processing ends.

Hereinafter, the details of the next candidate image generation processing in step 10 and the cutoff frequency determination processing in step 11 will be described.

[2-3] Explanation of Next Candidate Image Generation Processing in Step 10

As a method for generating various face images, a document
As shown in I and II, the following methods have already been proposed.

Document I: Katsunobu Isono, Masaomi Oda, Shigeru Akamatsu, "
Face Image Generation System Using Genetic Algorithm-Generation of Kansei Image ", IEICE Transactions D-II
Vol.J82-D-II No.3 pp.483-493 March 1999 Article II; Mitsuho Yamamoto, Katsunori Isono, Shigeru Akamatsu, "Face expression generation system using genetic algorithm (IEICE Technical Report, HIP98-
52,1993-3, PP.1-8)

A plurality of face image samples are separated into a shape and a texture, and a principal component analysis (PCA) method is applied to each of them. The feature of the face texture) is obtained in advance. The shape of the face is obtained by linearly combining each unique face with respect to the shape. In addition, the texture of the face is obtained by linearly combining each unique face with respect to the texture. Then, a single face image can be generated by synthesizing the obtained face shape and face texture. Various face images can be generated by changing the weight coefficient (characteristic parameter value of the image) of each unique face at the time of linear combination.

In the next candidate image generation process in step 10, the unique face for each unique face with respect to the shape is determined based on the ten face images currently displayed and the evaluation value input by the user for each face image. A new weighting factor and a new weighting factor for each unique face for the texture are separately calculated using a genetic algorithm (GA). Then, the next candidate image is generated using the obtained weighting coefficient.

A method of calculating a new weight coefficient for a unique face using a genetic algorithm (GA) is also disclosed in the above-mentioned document.
It has already been proposed by I and II.

The genetic algorithm (GA) is as follows. (1) Coding One face image is represented as one individual. The feature of each face image is represented by a weight coefficient for each unique face. Therefore, the chromosomes are obtained by arranging the respective weighting coefficients for the number of unique faces used for image generation.

(2) Fitness The GA generates a population while stochastically and preferentially allowing chromosomes with high fitness to survive, and updates the generation. In this embodiment, the rating value input by the user is used as the fitness.

(3) Selection The selection is an operation of stochastically selecting individuals more suitable for the environment according to their fitness. As a parent selection method, a roulette selection method (a method of selecting with a probability proportional to the fitness and allowing duplication) is used.

As a parent selection method, an algorithm combining a roulette selection method with an elite selection method may be used. However, the individual who is selected as an elite applies the individual who has been given the highest score by the user.

(4) Crossover Crossover is an operation for generating a new individual by rearranging chromosomes between two individuals. In this embodiment, a uniform crossover is used as the crossover. Uniform crossover is a method of applying a random mask at the time of crossover and thereby randomly determining which parent's gene is to be inherited at each locus.

(5) Mutation Mutation is an operation for stochastically changing a specific bit in an individual to another bit. The mutation can generate a pattern that cannot be obtained only by crossover, and can prevent a local solution from falling. This enables a global search. In this embodiment, one or more mutations always occur per generation. The crossover and mutation probabilities are stored in the hard disk 13 in advance.

In the next candidate image generation processing in step 10 described above, 10 images corresponding to 10 individuals obtained one generation later are obtained.
Generate images.

[2-4] Explanation of the cut-off frequency determination processing in step 11

FIG. 5 shows the procedure of the cut-off frequency determination processing in step 11 described above. First, among the ten images currently displayed (images before the next candidate image is generated in step 10), an average value of the evaluation values of five images having high evaluation values is obtained (step 21). Then, from the cutoff frequency conversion table (see FIG. 3), a cutoff frequency f1 that includes the average value of the rating values determined in step 21 as a range of the rating value y is determined (step 22).

Further, the average value of the variance of the weighting coefficients of all the unique faces with respect to the shapes of the ten next candidate images generated in step 10 is obtained (step 23). That is, first, for each eigenface with respect to the shape of the ten next candidate images generated in step 10, the average value σ ₁ ² ~ Σ
_{Find n} ² (n is the total number of unique faces). Finded value σ ₁ ^{2 to} σ
By calculating the averaging of _n ² , the average value {(σ ₁ ^{2 to} σ _n ² sum) {n} of the variances of the weight coefficients of all the unique faces for the shape is obtained. The smaller the average of the variance of the weight coefficients of all the unique faces for the shape, the more the convergence is shown. Then, from the cutoff frequency conversion table, a cutoff frequency f2 that includes, as a range of the variance x, the average of the variances of the weight coefficients of all the unique faces for the shape obtained in step 23 is obtained (step 24).

The larger of the cutoff frequency f1 obtained in step 22 and the cutoff frequency f2 obtained in step 24, f3 (= Ma
x [f1, f2]) is selected (step 25). Compare the selected cutoff frequency f3 (= Max [f1, f2]) with the currently set cutoff frequency f0 (the cutoff frequency with a check mark in the check box of the cutoff frequency conversion table) Then, the larger cutoff frequency f4 (= Max [f3, f0]) is selected (step 26). Thereby, the cutoff frequency for the next candidate image is determined.

When the cutoff frequency determined by the cutoff frequency determination processing is different from the cutoff frequency set up to that time, a check mark in the cutoff frequency conversion table is set by the cutoff frequency determination processing. Move to the check box corresponding to the determined cutoff frequency.

In step 23 of FIG. 5, the average of the variances of the weight coefficients of all the unique faces for the shapes of the ten next candidate images generated in step 10 is calculated.
The average of the variances of the weight coefficients of all the unique faces with respect to the textures of the ten next candidate images generated in step (1) may be obtained. In addition, the average of the variance of the weight coefficients of all the unique faces for the shapes of the ten next candidate images generated in step 10 and the average of the variance of the weight coefficients of all the unique faces for the texture are calculated. You may.

In the above embodiment, the information amount of the image presented to the user increases stepwise from a low spatial frequency to a high spatial frequency. Therefore, the user can easily make the determination as compared with determining all the information of the spatial frequency at once, and the image can be generated easily and quickly by repeating the determination.

[3] Description of application example to face image retrieval system

By the way, an image retrieval system for interactively retrieving a face image desired by a user from an image database has already been proposed (Masami Oda, "Image retrieval system utilizing characteristics of human image forming process",
Theory of Information Processing, vo135, no7, pp. 1449-1456, July 1994).

That is, first, the system displays a plurality of (for example, 10) face images randomly selected from the image database. The user selects as many favorite faces as desired. Thereafter, in response to a request for displaying the next screen by the user, the system extracts features common to them from the face image selected by the user.
Then, an image having the extracted characteristics and close to the average value of the images selected so far is searched from the database and presented to the user.

The present invention can be applied to such a face image search system. That is, by applying the present invention to such a face image search system, the amount of information of an image presented to a user can be increased stepwise from a low spatial frequency to a high spatial frequency.

The present invention can be applied to an image generation system for generating an image other than a face and an image search system for searching for an image other than a face.

[0070]

According to the present invention, the burden on the user involved in the image search is reduced, and the time required until the image desired by the user is searched can be shortened.

Further, according to the present invention, the burden on the user concerning the image generation is reduced, and the time until the image desired by the user is generated can be shortened.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image generation system.

FIG. 2 is a schematic diagram showing a configuration example of a display screen displayed on a display 21.

FIG. 3 is a schematic diagram illustrating an example of a cutoff frequency conversion table.

FIG. 4 is a flowchart illustrating an image generation processing procedure executed by a CPU 11;

FIG. 5 is a flowchart showing a cut-off frequency determination processing procedure in step 11 of FIG. 4;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Personal computer 11 CPU 12 Memory 13 Hard disk 14 Disk drive

Claims (8)

[Claims] 1. An image retrieval system for interactively retrieving an image desired by a user from an image database, comprising: a spatial frequency filter for limiting a bandwidth of a spatial frequency; Means for presenting to the user after limiting the bandwidth of the image retrieval system. 2. An image according to claim 1, further comprising means for switching the cutoff frequency of the spatial frequency filter from a low band to a high band stepwise according to the progress of the image search. Search system. 3. A means for causing a user to change a cutoff frequency of a spatial frequency filter, and an image currently presented to the user when the cutoff frequency of the spatial frequency filter is changed by the user. , Filtered using the changed cutoff frequency,
3. An image retrieval system according to claim 1, further comprising: means for changing a presented image to an image obtained thereby. 4. A computer-readable storage medium storing an image search program for interactively searching for an image desired by a user from an image database, comprising: a spatial frequency filter for limiting a spatial frequency bandwidth; And a means for presenting the retrieved image to a user after limiting the spatial frequency bandwidth with a spatial frequency filter, and a computer-readable recording medium recording an image retrieval program for causing a computer to function. 5. An image generating system for interactively generating an image desired by a user, wherein a spatial frequency filter for limiting a spatial frequency bandwidth and a spatial frequency filter for the generated image are limited in a spatial frequency bandwidth. Means for presenting to the user after the image is generated. 6. The image according to claim 5, further comprising means for switching a cutoff frequency of the spatial frequency filter from a low band to a high band stepwise according to the progress of image generation. Generation system. 7. A means for causing a user to change a cutoff frequency of a spatial frequency filter, and an image presently presented to the user when the cutoff frequency of the spatial frequency filter is changed by the user. , Filtered using the changed cutoff frequency,
7. An image generation system according to claim 5, further comprising: means for changing a presented image to an image obtained thereby. 8. A computer-readable recording medium storing an image generation program for interactively generating an image desired by a user, comprising: a spatial frequency filter for limiting a spatial frequency bandwidth; A computer-readable recording medium that stores an image generation program for causing a computer to function as means for presenting to a user after limiting a spatial frequency bandwidth with a spatial frequency filter.