JP2003150603A - Image processor and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor provided with a registration means of reference images or the like, capable of specifying a human body and retrieving an image by a simple operation, and a program. SOLUTION: The image processor is provided with an extraction means (15) for extracting a face image from the image picked up by using an image pickup device, a judgement means for judging whether or not the extracted face image is appropriate as the reference image to be a reference of retrieval and registration means (11a and 11b) for registering the face image judged as being appropriate as the reference image and an image retrieval means (13a) retrieves the other image with the registered reference image as the reference.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and program for accumulating and displaying digital images captured by using an electronic image capturing apparatus, and more particularly to recognizing a face of a photographed person to retrieve and extract the image. An image processing device and a program for performing

[0002]

2. Description of the Related Art In recent years, electronic image pickup devices such as digital cameras have become widespread in general households, and photographed images are recorded and stored in a storage device or storage medium provided in an image processing device such as a personal computer. ing.

A "digital album" for organizing, editing, and browsing the images thus stored.
Also called software. This software is able to arrange captured images in various layouts, and can record not only images but also audio information, so that it is not only visually enjoyable, but also provided with more information. There is.

By the way, when trying to enjoy using this "digital album", it is considered that searching for images to be attached to the album becomes a problem. For example, when a large number of images taken by the electronic image pickup device are stored for many years, it is thought that it will be difficult to find the desired image even when trying to find the desired image.

As a technique for solving this retrieval problem, a predetermined index is extracted from a photographed image and registered,
It has been proposed to search and display a photograph matching a search condition on a digital album device (Japanese Patent Laid-Open No. 10-12).
4655).

[0006]

However, since this technique is a method of indexing image information and specifying an image to be searched by a combination of a plurality of indexes, for example, when trying to search for the same person, If you are targeting images that have grown over the years, or if the same person has different color tones depending on the shooting conditions, multiple different indexes are required for each, so select the appropriate index. It is thought that the skill is required for it, and it is necessary to search repeatedly using some kinds of indexes.

When editing a digital album in a general home, most of the objects to be searched are people, and even if the search function is limited to people, it is considered that the utility value of the search technology is not impaired.

For example, searching for an image of one or more specific persons (hereinafter referred to as "specific person") to make a growth record, finding the specific person in a group photo, or other It is considered that if there is a search function for identifying a person, such as collating a person's name, it can be sufficiently used for the above-mentioned album editing work.

On the other hand, the face recognition technology in recent years is rapidly advancing, and various technologies for identifying an individual by directly collating with a photograph or the like have been proposed (for example, Japanese Patent Laid-Open No. 2000-19484).
9). Therefore, it is considered that by utilizing these techniques, the search for the specific person can be executed without performing a complicated operation.

However, how to register a reference image as a reference for identifying a person in searching for a person, and how to ensure that the reference image becomes an image suitable for the search. There is no technique disclosed regarding what kind of information should be registered as additional information at the time of registration for easier search.

The present invention has been made in view of the above circumstances, and image processing provided with a registration means for a reference image or the like that enables a person to be specified and an image to be searched for by a simple operation. The purpose is to provide a device and a program.

[0012]

SUMMARY OF THE INVENTION In order to solve the above problems, the present invention provides extraction means for extracting a face image from an image picked up by an image pickup device, and the extracted face image serves as a search reference. The image search unit includes a determination unit that determines whether or not the reference image is appropriate, and a registration unit that registers the face image determined to be appropriate as a reference image. This is an image processing device for searching.

The present invention is also the image processing apparatus according to the invention described above, which is provided with a warning means for warning that the extracted face image is not suitable as a reference image.

The present invention is also the image processing apparatus according to the above-mentioned invention, wherein the warning means further comprises a display means for displaying an inappropriate reason.

The present invention is also the image processing apparatus according to the invention described above, which is provided with template means for outputting a template for obtaining an appropriate human image to the image pickup apparatus.

According to the present invention, in the image processing apparatus according to the invention described above, the image processing apparatus is provided with an addition means for registering at least one of the name of the person in the face image and the shooting year as additional information of the reference image. Is.

Further, according to the present invention, an extraction procedure for extracting a face image from an image picked up by an image pickup device by a computer, and determining whether or not the extracted face image is appropriate as a reference image serving as a reference for retrieval. Determination procedure, registration procedure for registering a face image determined to be appropriate as a reference image, image search procedure for searching for another image based on the registered reference image,
Is a program for executing.

[0018]

1 is a block diagram showing the structure of an image processing apparatus according to the present invention.

The image processing apparatus includes an image processing unit 1 for performing image retrieval, a display device 2 for displaying processed images, which is a peripheral terminal of the image processing unit 1, and an image processing unit 1.
An operation input unit 3 for inputting various data or operation instructions to the printer, a printer 4 for printing a processed image,
An image capturing device 5 for capturing a person image of a reference image serving as a reference, and an image recording medium 6 recording an image to be subjected to a search process or the like.
It is composed of.

Then, in the image processing unit 1, an input / output control unit 10 which is an interface of each processing function of the image processing unit 1 for exchanging signals with these peripheral terminals, and a reference image which is a reference for retrieval are registered. The registration unit 11 creates the additional information related to the search of the image to be searched, and the image processing unit 1
An image accumulation processing unit 12 for accumulating inside, a retrieval unit 13 for retrieving an image in which a person of a reference image is photographed, a display unit 14 for displaying an image and also displaying information associated with the image in association with the image, An extraction / recognition unit 15 for extracting a person's face or recognizing an individual and a print processing unit 16 for collecting and printing an image and additional information are provided.

Further, the registration unit 11 extracts a person from the image input from the image recording medium 6 and creates a face image, which is an image in which the face portion of the person is extracted, as a reference image. An image designation registration unit 11a to be registered, and a new registration unit 11b that creates a face image based on the image pickup input from the image pickup apparatus and is registered as a reference image are provided, and the search unit 13 is also provided with a reference designated in the image. A reference image designation search unit 13a for checking whether a person in the image is photographed and extracting the corresponding image is provided, and the extraction recognition unit 15 identifies the face portion of the person from the image and identifies the face image. A face extraction unit 15a for extracting the face is provided, and a personal recognition unit 15b for collating and recognizing whether or not the face of the reference image is in the designated image is provided.

Further, the image processing section 1 is provided with a storage section 20 for storing information necessary for the operation of each of these functions, and the storage section 20 stores an image storage section 21 for storing an image to be searched. A reference image storage unit 22 that stores a reference image serving as a search reference is provided.

Next, the operation of each processing function of the image processing apparatus will be described. A menu screen is displayed on the display unit 2 when the operator turns on the power of the image processing apparatus or when an initial operation is performed from the operation input unit 3.

FIG. 2 is a menu screen showing the processing functions of the image processing apparatus.

The processing function of this image processing apparatus has four functions: a registration mode, an image storage mode, a search mode, and an image display mode.
It consists of one function. Then, in this menu screen, an image designation registration button 30 and a new registration button 31 are displayed in the registration mode, an image storage processing button 32 is displayed in the image storage mode, and a reference image designation search button 33 is displayed in the search mode. The image display button 34 is displayed in the image display mode.

When the operator wants to register a reference image, the image recording medium 6 on which the photographed image of the person is recorded.
When is connected to the image processing unit 1 and the image designation registration button 30 on the menu screen is operated, the image designation registration unit 11a is activated to perform reference image registration processing.

FIG. 3 is a flow chart showing a schematic operation procedure of the image designation registration section 11a.

This image designation registration function is a function of creating a reference image from an image recorded in an external image recording medium 6, and includes a function of judging whether or not it can be adopted as a reference image and a function of adding information. Have. It should be noted that the image recorded on the image recording medium 6 is the image pickup device 5.
Alternatively, it is an image captured by using another imaging device.

The operator selects an image of a person who wants to be registered as a reference image from thumbnail images of images recorded on the image recording medium 6 displayed on the display unit 2 (S1).

The image designation registration unit 11a extracts the image selected by the operator from the image recording medium 6 and displays it on the display unit 2 (S2), and at the same time, extracts the face image of the specific person from the image. The extraction unit 15a is activated. The face extraction unit 15 extracts the face image of the person photographed in the image (S3), and returns the result to the image designation registration unit 11a.

FIG. 4 is a diagram showing a face extraction result by the face extraction unit 15a.

A frame is displayed on the original image so as to surround the face of the person extracted by the face extraction unit 15. When the operator selects the person to be registered as the reference image from the operation input unit 3 (S4), the image designation registration unit 11a determines whether the face image of the selected person is appropriate as the reference image (S5). .

For example, when the face portion is covered with hair or a hat and only a small area is shown, when the eyes are closed, or the like is an inappropriate face image, and in such a case, the display unit 2 A warning to the effect that it is not suitable is displayed and the reason why it is not suitable is displayed (S7). When it is determined that the face image is appropriate (S6), the information input screen in which the size of the face is adjusted to a predetermined size is displayed on the display unit 2 (S8).

FIG. 5 is a diagram showing the structure of the information input screen.

This information input screen is provided with a resized face image and an input field for additional information such as a name, date of birth, date of photography, date of registration, and the like, which will be information at the time of retrieval, which will be described later. ing. It should be noted that the date of photographing may be configured to display the data when the information has already been input in association with the original image, and the date of registration may be preset to the date of the day. .

The operator inputs these information (S
9) If the register button is operated, the image designation registration unit 11a registers the face image and the additional information in the reference image storage unit 22 (S).
10). If there are still images to be processed (S11), the above procedure is repeated. Here, registration means storing in association with an identification code for identifying a reference image.

When the operator wants to register a reference image using the image pickup device 5, he or she connects the image pickup device 5 to the image processing section 1 and operates the new registration button 31 on the menu screen. It is activated and the reference image registration process is performed.

FIG. 6 is a flow chart showing a schematic operation procedure of the new registration unit 11b.

The new registration function is a function of creating a reference image from a person image captured from the image pickup device 5. First, the new registration unit 11b outputs a template for defining an image of a person to the imaging device 5 (S15).

FIG. 7 is a diagram showing a person image displayed on the display unit of the image pickup apparatus 5.

The photographer of the image pickup device 5 can easily obtain an appropriate face image by setting the size and orientation of the person in accordance with this template and photographing. The person image captured in this way is sent to the new registration unit 11b, and the new registration unit 11b receives the image data from the image pickup device 5 and displays it on the display unit 2 (S16), and at the same time, extracts the face image of the specific person from the image. Face extraction unit 15
Start a. The face extraction unit 15 extracts the face image of the person photographed in the image (S17) and returns the result to the new registration unit 11b.

Next, the new registration unit 11b determines whether the extracted face image is appropriate as a reference image (S1).
8). For example, if the face part is covered with hair or a hat and is small, or if the eyes are closed, etc. is an improper face image, and in such a case, the display unit 2 displays a warning that it is not appropriate. The inappropriate reason is displayed (S20). This allows the photographer to easily re-take an appropriate face image. If it is determined that the face image is appropriate (S1
9), the information input screen shown in FIG. 5 in which the size of the face is adjusted to a predetermined size is displayed on the display unit 2 (S21).

If the operator inputs additional information such as a name, date of birth, date of photography, date of registration, etc., which will be information at the time of retrieval, which will be described later (S22), and operates the register button to newly register. The unit 11b registers the face image and the additional information in the reference image storage unit 22 (S23). Then, when there are still images to be processed (S24), the above procedure is repeated.

If the reference image registration processing is configured in this way, it is possible to easily register it as a reference image without extracting a plurality of indexes that represent the characteristics of a person, and to remove an inappropriate face image for recognition. It is possible to increase the certainty of processing.

Next, a process for accumulating and storing an image input from an external image recording device or the like in the image storage unit 21 of the image processing device will be described. When the operator connects the external terminal to the image processing unit 1 and operates the image storage processing button 32 on the menu screen, the image storage processing unit 12 is activated to perform the image storage processing.

FIG. 8 and FIG. 9 are flow charts showing a schematic operation procedure of the image storage processing unit 12.

This image storage processing function, when inputting an image, extracts the search information required in the subsequent search processing in advance and stores it together with the image.

The image storage processing unit 12 receives the image data from the image pickup device 5 or the image recording medium 6 and displays it on the display unit 2 (S30), and at the same time, extracts the face image of the specific person from the image. The section 15a is activated. The face extraction unit 15a extracts the face image of the person photographed in the image (S32), and returns the result to the image accumulation processing unit 12.

The image storage processing unit 12 determines the position information (area information or center coordinates) of each face in the image data (S33).

FIG. 10 is a diagram showing a method for determining the position information of each face in the image data.

FIG. 10A shows face area information. This information represents the area where the face is present, for example, as an area surrounded by a polygon composed of a plurality of coordinate points. On the other hand, (2) in FIG. 10 shows the center coordinates of the face. This information represents the position of the face in the image data as, for example, the coordinates of one point indicating the approximate center position of the face area. The face position information is treated as information for specifying the position of the person's face in the display process.

Subsequently, the image accumulation processing section 12 activates the personal recognition section 15b. The personal recognition unit 15b is the face extraction unit 15
An individual recognition process is performed to check whether the face image extracted by a is compared with the face image of the reference image and whether the person can be recognized as the person of the reference image (S34).

When the face image can be recognized as a specific person in the reference image (S35), the recognized person in the image data is attached with the identification code of the reference image and displayed on the display unit (S3).
6) If the specific person in the reference image could not be recognized (S35), the person who could not be recognized in the image data? It is displayed with a reference numeral (S37).

The individual recognition processing described above is repeated for all the persons included in the image data, and when the matching of all faces is completed (S38), the recognition screen is displayed on the display unit 2 (S39). .

FIG. 11 is a diagram showing the structure of the recognition screen.

In this recognition screen, the above-mentioned identification code is attached to each person in the image data, and the reference image is displayed in the thumbnail form at the bottom of the screen. Each person is associated with the reference image via this identification code. By thus displaying the association, the operator can easily confirm the recognition result. The operator compares the person in the image data with the person in the reference image based on the identification code to determine whether or not the identification code needs to be changed or changed (S40).

When the change or addition is necessary, the change or addition is selected from the menu (not shown) on the image and the operation is performed (S41).

When the change is selected (S42), first, the person on the image is specified by operating the mouse or the like at the face position (or in the face area), and then the person is made to correspond. The reference image is specified using a mouse or the like (S43). By this operation, the identification code given to the person on the image data is changed to the identification code of the specified reference image.

When the addition is selected (S42), the person on the image is specified by operating the mouse or the like at the face position (or in the face area), and then the vacant space for registering the person is obtained. The corresponding reference image column (column of the identification code “0” in FIG. 11) using a mouse or the like (S4
4). By this operation, the face portion of the person on the image data is cut out and displayed for the reference image (S4).
5). Then, information such as a name is input to the additional information input field displayed at the same time to perform an additional registration operation (S46). Then, the identification code of the added reference image is displayed as a new identification code attached to the person on the image. The additional registration operation may be configured using the information input screen shown in FIG.

This change addition operation is repeatedly executed as many times as necessary, and when all the change addition processing is completed (S
47) Then, by performing the accumulation operation, the image data and the additional information such as the personal name of the person and the face position information read from the reference image are stored in the image storage unit 21 (S48). Then, the above processing is repeated for all input images (S49), and the image storage processing is ended.

FIG. 12 is a diagram showing a data structure of the image storage unit 21.

The header portion stores the file name of the image as well as additional information such as shooting date, registration date, person's name, date of birth, face position information, etc. The image data is stored, for example, compressed by the JPEG method.

Next, the search function will be described. This function is a function of extracting an image including a person in the reference image and displaying it as a thumbnail. The image to be searched at this time is recorded in an external image recording medium or in an internal recording medium. There are two cases where the image is recorded in the image recording unit.

First, the processing of the image recorded on the external image recording medium will be described. The external terminal is the image processing unit 1
Connect to and search button 33
When is operated, the reference image designation search unit 13a is activated to perform a search process.

13 and 14 show the reference image designation search section 1
It is a flowchart which shows the outline operation procedure of 3a.

First, the reference image designation search section 13a operates as shown in FIG.
A list of reference images shown in is displayed on the display unit 2 (S5).
1). The operator specifies a person to be searched on this screen by using a mouse or the like, and at this time, it is also possible to specify a plurality of images (S52). Then, only the reference images of the identified persons are displayed as a list on the display unit 2 (S5).
3).

Subsequently, the reference image designation search unit 13a extracts the shooting date from the incidental information of the reference images displayed in the list, and checks whether or not the shooting date is within a predetermined period from the present (S54). ). If it is not within the predetermined period, the reference image is an old image, so the operator is cautioned by changing the display color of only the reference image, and a warning message is displayed to re-register the new reference image. Output (S5
5). With this configuration, it is possible to prevent a problem that the old reference image is not updated as it is. Further, the predetermined period may be changed according to the age of the person. In other words, it is more appropriate to calculate the age from the date of birth and set the predetermined period to be short because the change in the face is large for children.

In the present embodiment, the shooting date of the reference image is checked when the person is specified, but the present invention is not limited to this form, and the shooting date is always displayed when the reference images are displayed in a list. The check may be performed.

Next, the reference image designation search section 13a receives the image data from the image pickup device 5 or the image recording medium 6 and displays it on the display section 2 (S56), and at the same time, extracts the face image of the specific person from the image. Then, the face extraction unit 15a is activated. The face extraction unit 15a extracts the face image of the person photographed in the image (S57), and returns the result to the reference image designation search unit 13a.

When the face image cannot be extracted (S58)
Displays a message stating that it cannot be extracted (S5
9) Then, the processing of the next image data is started.

If the face image can be extracted (S58), the face image and the reference image are compared and collated. However, if the reference image includes a plurality of the same persons, the date when the image data was photographed A reference image of the person having a date close to is selected (S60). Then, the reference image designation search unit 1
3a activates the personal recognition unit 15b. Individual recognition unit 15b
Performs a personal recognition process of checking whether or not the face image extracted by the face extraction unit 15a is compared with the face image of the reference image to recognize the person as the reference image (S61).

When the person recognized in the reference image can be recognized as a specific person (S62), the recognized person in the image data is emphasized and stored by, for example, attaching a mark (S6).
3). When the person cannot be recognized as the specific person in the reference image (S62), a message indicating that there is no person in the reference image in the image data is displayed on the display unit (S64) and the next image data is processed. .

This processing is repeated for all input image data, and when the processing for all input image data is completed (S65), the extracted image data is displayed as a list in the thumbnail format (S66). When the operator selects this thumbnail, the image data is displayed in a normal size.

With this configuration, since a face image close to the date on which the image to be searched is photographed can be searched as a reference image, it is not necessary to search based on another reference image of the same person. The search time can be shortened and the search accuracy can be improved. Further, the search result can be easily confirmed by providing a function of displaying the search result in the thumbnail form or in the enlarged form.

Next, the processing of the image recorded on the internal recording medium will be described. When the operator operates the reference image designation search button 33 on the menu screen, the reference image designation search unit 13a is activated to start the search process.

FIG. 16 is a flow chart showing a schematic operation procedure of the reference image designation search section 13a. In the search for the main internal image, since the search information has already been extracted by the above-described image storage processing, the search can be performed faster than the search for the external image.

The reference image designation / retrieval unit 13a first performs a reference image selection process. This procedure is performed by the above-described step S51.
Since this is the same as step S55, detailed description thereof will be omitted. Subsequently, the reference image designation search unit 13a retrieves the information of the header portion for each image of the designated image storage unit 21 (S70). Then, it is checked whether the name or the like described in the additional information of the reference image matches the name or the like in the header information (S71). If they match (S7
In 2), the image is extracted and stored (S73). If they do not match, it is determined that there is no person to search in the image, and the process proceeds to the next image data search process.

This processing is repeated for all input image data, and when the processing for all input image data is completed (S74), the extracted image data is displayed as a list in thumbnail format (S75). When the operator selects this thumbnail, the image data is displayed in a normal size.

As described above, by arranging to search the additional information, the search can be performed at high speed.

Next, the processing for displaying the image stored inside on the display unit 2 of the image processing apparatus and outputting it to the printer 4 will be described. When the operator operates the image display button 34 on the menu screen, the image display unit 14 is activated to perform image display processing.

FIG. 17 is a flow chart showing a schematic operation procedure of the image display unit 14.

When the operator designates an image to be displayed, the image display unit 14 reads the header information and its image data from the image storage unit 21 and displays the image on the display unit 2 (S8).
0). When the operator operates the mouse or the like on a specific person in the image data to move the pointer, the image display unit 14 reads the position of the pointer, and the face coordinates or face area information which is the face position data of the header information. (S
81).

When the face position data in the header information is associated with the personal information, personal information such as the name of the person is displayed (S82), and the operator puts the pointer on the person and clicks. Then, an enlarged image of the person is displayed (S83).

Next, when the operator performs an instruction operation to print the displayed image, the image display unit 14 passes the header information read from the image storage unit 21 and the image data to the print processing unit 16. Then, the printing operation is started (S86). Then, the print processing unit 16, together with the image,
Additional information such as the date and personal name in the header information is printed (S87).

As described above, in this function, since the additional information has already been extracted by the above-mentioned image storage processing, the name of the person can be displayed by simply pointing the pointer on the image of the person, and clicking on the person. If so, a magnified image of the person can be displayed, and if printed out, not only the image but also additional information such as the name of the person in the image can be printed. Therefore, related information can be collectively referred to by a simple handling.

Although the present embodiment has been described for the purpose of the function of the digital album for the captured image, the present invention is not limited to this embodiment, and the personal authentication for recognizing and specifying the face is specified. It can be widely applied to technical fields such as photo collation.

[0087]

As described above, if the registration means according to the present invention is used, a person can be specified and images can be searched for by a simple operation.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an image processing apparatus according to the present invention.

FIG. 2 is a menu screen showing processing functions of the image processing apparatus.

FIG. 3 is a flowchart showing a schematic operation procedure of an image designation registration unit.

FIG. 4 is a diagram showing a face extraction result by a face extraction unit.

FIG. 5 is a diagram showing the structure of an information input screen.

FIG. 6 is a flowchart showing a schematic operation procedure of a new registration unit.

FIG. 7 is a diagram showing a person image displayed on the display unit of the image pickup apparatus.

FIG. 8 is a flowchart showing a schematic operation procedure of an image storage processing unit.

FIG. 9 is a flowchart showing a schematic operation procedure of an image storage processing unit.

FIG. 10 is a diagram showing a method of deciding positional information of each face in image data.

FIG. 11 is a diagram showing a configuration of a recognition screen.

FIG. 12 is a diagram showing a data configuration of an image storage unit.

FIG. 13 is a flowchart showing a schematic operation procedure of a reference image designation search unit.

FIG. 14 is a flowchart showing a schematic operation procedure of a reference image designation search unit.

FIG. 15 is a diagram showing a list of reference images.

FIG. 16 is a flowchart showing a schematic operation procedure of a reference image designation search unit.

FIG. 17 is a flowchart showing a schematic operation procedure of the image display unit.

[Explanation of symbols]

1 ... Image processing unit 2 ... Display 4 ... Printer 5 ... Imaging device 6 ... Image recording medium 11a ... Image designation registration unit 11b ... New registration section 12 ... Image storage processing unit 13a ... Reference image designation search unit 14 ... Image display section 15a ... Face extraction unit 15b ... Individual recognition unit 16 ... Print processing unit 21 ... Image storage unit 22 ... Reference image storage unit

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5B050 AA09 BA10 BA12 BA15 CA07                       DA01 EA05 EA12 EA18 FA02                       FA09 FA12 FA19 GA08                 5B057 BA02 DA08 DA12 DB02 DC06                       DC33 DC39                 5B075 ND06 NK25 NK37                 5C052 AB04 AC08 DD02 EE02 EE03

Claims (6)

[Claims] 1. An image processing apparatus having an image search means, extracting means for extracting a face image from an image picked up by an image pickup device, and the extracted face image being suitable as a reference image serving as a reference for search. And a registration unit for registering a face image determined to be appropriate as a reference image. The image search unit searches for another image based on the registered reference image. An image processing device characterized by the above. 2. The image processing apparatus according to claim 1, further comprising a warning unit that warns that the extracted face image is not suitable as a reference image when it is not suitable. 3. The image processing apparatus according to claim 2, wherein the warning unit further includes a display unit that displays an inappropriate reason. 4. The image according to claim 1, further comprising template means for outputting a template for obtaining an appropriate human image to the image pickup device. Processing equipment. 5. The adding means for registering at least one of the name of the person and the shooting year of the face image as additional information of the reference image is provided. The image processing apparatus according to claim. 6. An image processing program of an image processing device having an image search means, wherein an extraction procedure for causing a computer to extract a face image from an image captured by the image capturing device, wherein the extracted face image is a search criterion. Determination procedure for determining whether or not it is appropriate as a reference image, a registration procedure for registering a face image determined as appropriate as a reference image, an image search procedure for searching for another image based on the registered reference image, A program to execute.