JP2009049886A - Image retrieval device, photographing device, image retrieval method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval device for retrieving a desired image without requiring a complicated operation, to provide a photographing device, an image retrieval method and a program. <P>SOLUTION: An electronic camera 10 serving as an image retrieval device stores registered image data registered by registration processing, in a storage part 17 in association with at least one keyword. When a shift to a retrieval mode is instructed by the operation of an operating part 18, keywords associated with the registered image data stored in the storage part 17 are read and displayed as a list on a display part 15. Photographed image data including an image of the registered image data corresponding to the keyword selected out of the displayed keywords is retrieved from the photographed image data stored in the storage part 17, and the retrieved result is displayed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image search device, a photographing device, an image search method, and a program.

  Conventionally, a digital photographed image photographed by a photographing device such as an electronic camera is stored in a main body of the photographing device or a storage device of a personal computer, and a user (user) can select a desired image from the stored photographed images. To find and browse and edit.

  In recent years, with the increase in capacity of storage devices, a large amount of images can be stored in the storage devices. Therefore, it is a difficult task for the user to find a desired image.

Therefore, as a technique for supporting the user's image search operation, for example, in Patent Document 1, a user's face area is extracted from an image obtained by photographing the user's face, and the user's face search is performed based on the extracted feature value of the face area. An image search apparatus is described in which an image including a face is searched from a plurality of images.
JP 2007-109130 A

  However, in Patent Document 1, only an image including the same face image as the face image taken from a plurality of images is searched, and the image cannot be searched according to the search condition.

  An object of the present invention is to provide an image search device, a photographing device, an image search method, and a program capable of searching for a desired image without performing complicated work.

In order to solve the above-described problem, an image search apparatus according to the invention described in claim 1 provides:
Photographic image storage means for storing photographic images;
Search information storage means for storing a registered image registered in advance and at least one search information in association with each other;
Search information selection means for the user to select desired search information from the search information stored in the search information storage means;
Image search for searching for a registered image associated with the search information selected by the search information selection unit from the search information storage unit, and searching for a captured image including the searched registered image from the captured image storage unit Means,
It is characterized by providing.

The invention according to claim 2 is the invention according to claim 1,
The search information storage means stores the identification information of the registered image and the at least one search information in association with each other,
Collating the photographed image with each of the registered images stored in the search information storage means to detect a registered image included in the photographed image, and associating the identification information of the detected registered image with the photographed image With means,
The image retrieval unit retrieves a registered image associated with the retrieval information selected by the retrieval information selection unit from the retrieval information storage unit, and captures the identification information associated with the retrieved registered image. An image is retrieved from the photographed image storage means.

The invention according to claim 3 is the invention according to claim 2,
The registered image is a face image;
The collating unit detects a face image area included in the photographed image, and collates the detected face image area with each of the face images stored in the search information storage unit, thereby obtaining the photographed image. It is characterized by detecting the included face image.

The invention according to claim 4 is the invention according to claim 2 or 3,
When storing the photographed image in the photographed image storage means, a collation selection means for selecting whether or not to collate the photographed image with a registered image stored in the search information storage means,
When the collation selecting unit selects collation, the collation unit collates the captured image with each of the registered images stored in the search information storage unit, and obtains a registered image included in the photographed image. It detects, and the identification information of the detected registered image is added to the photographed image.

The invention according to claim 5 is the invention according to claim 4,
The image search means searches for a captured image including identification information of the searched registered image from a captured image stored in the captured image storage means.

The invention according to claim 6 is the invention according to claim 2 or 3,
Collation selection means for selecting whether or not to collate the captured image with a registered image stored in the search information storage means;
Collation information storage means for storing the photographed image and the selection result of the collation selection means in association with each other,
When the collation selecting unit selects collation, the collation unit collates the captured image with each of the registered images stored in the search information storage unit, and obtains a registered image included in the photographed image. The identification information of the detected registered image is further stored in the collation information storage unit in association with the captured image and the selection result.

The invention according to claim 7 is the invention according to claim 6,
The image search means refers to the collation information storage means and searches for a captured image associated with identification information of the searched registered image.

The invention according to claim 8 provides:
In a photographing device that photographs a subject and obtains a photographed image,
An image search apparatus according to any one of claims 1 to 7 is provided.

The image search method of the invention according to claim 9
A captured image storing step for storing the captured image;
A search information storage step of storing a registered image registered in advance and at least one search information in association with each other;
A search information selection step for the user to select desired search information from the search information stored in the search information storage step;
A registered image associated with the search information selected in the search information selection step is searched from the registered image stored in the search information storage step, and a captured image including the searched registered image is stored in the captured image storage. An image search step for searching from the captured image stored in the step;
It is characterized by including.

The program of the invention according to claim 10 is:
On the computer,
Captured image storage function for storing captured images,
A search information storage function for storing a registered image registered in advance and at least one search information in association with each other;
A search information selection function for the user to select desired search information from the search information stored by the search information storage function;
The registered image associated with the search information selected by the search information selection function is searched from the registered image stored by the search information storage function, and the captured image including the searched registered image is stored in the captured image storage. Image search function to search from captured images stored by the function,
It is characterized by realizing.

  According to the present invention, it is possible to search for a desired image without performing complicated operations.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, the case where the image search apparatus of the present invention is applied to the electronic camera 10 will be described as an example. However, the present invention is not limited to this. Further, the case where the registered image is a human face image will be described as an example, but the present invention is not limited to this.

(Embodiment 1)
Hereinafter, Embodiment 1 will be described in detail with reference to the drawings.
First, the configuration will be described.
FIG. 1 shows a functional configuration example of the electronic camera 10 according to the first embodiment.
As shown in FIG. 1, the electronic camera 10 includes an optical lens unit 11, an image sensor 12, an image processing unit 13, a video output unit 14, a display unit 15, a control unit 16, a storage unit 17, an operation unit 18, and an external device interface. Unit 19 and external storage interface unit 20.

  The optical lens unit 11 includes a focusing lens, a zoom lens, a diaphragm, and the like. The optical lens unit 11 captures an optical image of a subject and forms it on the image sensor 12.

  The image sensor 12 includes a photoelectric conversion element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and the subject image formed on the imaging surface of the image sensor 12 via the optical lens unit 11. And an analog imaging signal is output.

  The image processing unit 13 converts the analog imaging signal input from the image sensor 12 into a digital imaging signal, performs signal processing such as color interpolation processing, and acquires image data in a predetermined format.

  The video output unit 14 converts the image data input by the image processing unit 13 into a display signal and outputs the display signal to the display unit 15.

  The display unit 15 is configured by a monitor such as an LCD (Liquid Crystal Display), and displays an image to be photographed, a photographed image, and the like on a display screen in accordance with an instruction of a display signal input from the video output unit 14. .

  The control unit 16 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. The CPU of the control unit 16 reads out the system program stored in the program storage area 171 of the storage unit 17 in accordance with the operation of the operation unit 18 and expands it in the RAM. Centralized control of operation.

  For example, the control unit 16 calculates a lens movement amount and an aperture amount of the optical lens unit 11 based on a detection signal detected by a distance sensor, a light amount sensor, and the like (not shown), and performs lens drive control and aperture amount control. To adjust the focus (focus) and zoom. In addition, the control unit 16 controls the readout timing of the imaging signal from the image sensor 12.

  Further, the control unit 16 reads out various processing programs such as a registration mode program, a shooting mode program, a search mode program, and an album creation mode program stored in the storage unit 17, and performs a registration process, a shooting process, Various processes such as a search process and an album creation process are executed.

  The storage unit 17 is composed of a semiconductor memory or the like that can be electrically erased and rewritten, and stores various programs for operating the control unit 16 and various data written from the control unit 16. Remember. As shown in FIG. 1, the storage unit 17 includes a program storage area 171, a registration information storage area 172, a registered image storage area 173, and a captured image storage area 174, and corresponds to a writing operation from the control unit 16. The program is stored in the program storage area 171, the registration information is stored in the registration information storage area 172, the registered image data is stored in the registration image storage area 173, and the captured image data is stored in the captured image storage area 174.

  The registration information storage area 172 includes registration information, specifically, image data (registered image data) registered in a registration process (see FIG. 6) to be described later, and search information (keyword) input for the image data. Are stored in association with each other. Here, the image data captured in the registration process described later and registered in the registration information storage area 172 and the registration image storage area 173 is referred to as registration image data. As shown in FIG. 2, the registration information storage area 172 is a “registration number” area for storing a registration number uniquely assigned to identify registration image data (ie, an identification number of registration image data). It has a “file name (registered image)” area for storing data indicating the file name of the data, and a “search information (keyword)” area for storing search information (keyword) input for registered image data. The number, the registered image data file name, and the keyword are stored in association with each other. The keyword indicates person information (for example, a person's name) for specifying a person in the registered image data, and a phrase related to the person in the registered image data (for example, a hobby of the person or a group to which the person belongs). Stored in related information.

  The registered image storage area 173 is an area for storing registered image data registered in a registration process (see FIG. 6) described later, as in the example shown in FIG.

  The photographed image storage area 174 is an area for storing photographed image data photographed in a photographing process (see FIG. 8) described later, as in the example shown in FIG. The captured image data is stored in the file format shown in FIG. That is, the captured image data includes a header portion and a data portion. The header portion includes information (file name, compression method, shooting date / time, image size, notes, etc.) related to the shot image data. The data part consists of actual data (image data) of the photographed image data.

  The operation unit 18 includes input keys such as a shutter key, a numeric keypad, a menu key, a cross key, an enter key, and various function keys, and outputs an operation signal corresponding to the key operated by the user to the control unit 16. .

  The external device interface unit 19 includes a wired communication I / F for connecting to an external device by wired communication such as USB (Universal Serial Bus) or IEEE 1394 (Institute of Electrical and Electronic Engineers 1394), infrared data communication, Bluetooth ( R) and the like, and is configured by a wireless communication I / F or the like for wireless connection with an external device, and performs data transmission / reception with an external device such as a personal computer or a printer.

The external storage interface unit 20 can be connected to an external storage device 20a such as a CF (Compact Flash) card, an SD card, or a USB memory, and is an interface for inputting / outputting data to / from the external storage device 20a.
The above is the functional configuration of the electronic camera 10.

  Next, the operation of the electronic camera 10 in this embodiment will be described.

〔registration process〕
First, registration processing executed in the electronic camera 10 will be described with reference to a flowchart shown in FIG. This registration process is started when the user of the electronic camera 10 instructs to shift to the registration mode by operating the operation unit 18.

  When the operation of the instruction to shift to the registration mode is detected, the registration mode is activated by loading the registration mode program stored in the program storage area 171 into the RAM of the control unit 16 (step S11). Next, a registered image is shot in accordance with the operation of the shutter key of the operation unit 18 (step S12). Specifically, when the shutter key of the operation unit 18 is operated with the face of a person to be registered as a subject, the optical lens unit 11 and the image sensor 12 are operated according to the operation of the shutter key of the operation unit 18. The image data is acquired under control.

  Next, a screen (not shown) for inquiring whether or not to register the image data together with the image data acquired by photographing is displayed on the display unit 15, and when an input of an instruction not to be registered is detected by the operation unit 18. (Step S13; NO), this process ends.

On the other hand, when an input of an instruction to register the image data is detected by the operation unit 18 (step S13; YES), a keyword input screen 151 is displayed on the display unit 15 (step S14).
FIG. 7 shows an example of the keyword input screen 151 displayed on the display unit 15. As shown in FIG. 7, the keyword input screen 151 includes a thumbnail display field 151a, a registration number display field 151b, a person information input field 151c, related information input fields 151d and 151e, a “OK” button 151f, and a “Clear” button 151g. It is displayed. The thumbnail display field 151a displays a thumbnail image of the image data captured in step S12. The registration number display field 151b displays the registration number assigned to the captured image data. The person information input field 151c receives an input of a keyword (person information) associated with the captured image data. The related information input fields 151d and 151e accept input of keywords (related information) associated with the captured image data. The “decision” button 151f is a button for instructing registration of photographed image data with input contents. The “clear” button 151g is a button for instructing cancellation of registration of photographed image data.

  When a keyword is input from the keyword input screen 151 by the operation of the operation unit 18 (step S15) and a key operation corresponding to the “OK” button 151f is detected (step S16; YES), the imaged image data is input. The keywords are associated with each other and stored in the storage unit 17 (step S18), and this process ends. Specifically, the file name, registration number, and input keyword of the captured image data are stored in the registration information storage area 172 as one record. The captured image data is stored in the registered image storage area 173 as registered image data.

  On the other hand, when the key operation corresponding to the “clear” button 151g is detected from the keyword input screen 151 by the operation unit 18 (step S17; YES), this process ends.

[Shooting process]
Next, photographing processing executed in the electronic camera 10 will be described with reference to a flowchart shown in FIG. This photographing process is started when the user of the electronic camera 10 instructs to shift to the photographing mode by operating the operation unit 18.

  When the operation of the instruction to shift to the shooting mode is detected, the shooting mode is activated by loading the shooting mode program stored in the program storage area 171 into the RAM of the control unit 16 (step S21). Next, shooting is performed according to the operation of the shutter key of the operation unit 18 (step S22). Specifically, when the shutter key of the operation unit 18 is operated, the optical lens unit 11 and the image sensor 12 are controlled in accordance with the operation of the shutter key of the operation unit 18 to acquire captured image data.

Next, a selection screen (not shown) for the user to select whether or not to allow the photographed image data acquired by photographing to be verified with the registered image data stored in the storage unit 17 is displayed on the display unit 15. When an operation for selecting “Do not allow collation” is detected on the screen by the operation unit 18 (step S23; NO), the process proceeds to step S29.
In the present embodiment in which a person's face image is used as a registered image, for example, even if a captured image obtained by shooting a landscape is compared with a registered image, there is no matching image, which is useless. Therefore, in such a case, unnecessary processing can be omitted by selecting “Do not allow collation” from the selection screen.

On the other hand, when an operation for selecting “permit collation” is detected by the operation unit 18 on the selection screen (step S23; YES), detection of a face image area is performed on the captured image data captured in step S22. (Step S24). The detection of the face image area is performed to detect whether or not the face image area of the person is included in the photographed image data, and when it is included, the position of the face image area is specified.
The detection of the face image area is performed by using, for example, face detection information (feature color information, feature parts (face parts) information corresponding to eyes, nose, rho, etc.) representing the characteristics of a person prepared in advance. A skin color area is extracted from each of the extracted areas, and it is determined whether or not there is a characteristic part (face part) corresponding to eyes, nose, or rho in each extracted area. An area is extracted as a face image area.
Further, for example, as described in Japanese Patent Application Laid-Open No. 2002-189724, a probability density function obtained in advance from a plurality of sample images, a plurality of first partial spaces indicating face image areas, and a first indicating non-face image areas. You may use the method of detecting a face image area | region from the image | photographed image using 2 partial space.

  Next, face recognition processing is executed for each detected face image region (step S25). In step S25, each detected face image area is compared with the face image area of each registered image data stored in the registered image storage area 173, and registered image data of the same person as the detected face image area. Is identified. That is, the registered image included in the captured image is specified. For example, for each of the face image areas extracted in step S24, a correlation coefficient with the face image area of each registered image data stored in the registered image storage area 173 is obtained by the template matching method while enlarging or reducing the image. Then, registered image data having a face image area having the largest correlation coefficient and exceeding a predetermined threshold is detected as a face image of the same person as the face image area extracted in step S24.

When the face recognition process is completed, the collation result screen 152 is displayed on the display unit 15 (step S26).
FIG. 9 shows an example of the matching result screen 152. The collation result screen 152 is a screen showing the collation results, that is, the face image area detection result in step S24 and the face recognition result in step S25. In the captured image display field 152a of the verification result screen 152, the image captured in step S22 is displayed. In the photographed image display field 152a, the face image area detected in step S24 is displayed by a thin line frame 152b so that the user can identify it. The face image area in which the person is specified in step S25 is displayed in a identifiable manner by the thick frame 152c. Further, a “decision” button 152 d and a “list” button 152 e are displayed on the collation result screen 152, and when a key corresponding to the “list” button 152 e is operated by the operation unit 18, the collation result list screen 153 is displayed. Is displayed.

  FIG. 10 shows an example of the matching result list screen 153. The matching result list screen 153 is registered in the thumbnail image 153a and the registration information storage area 172 of the registered image recognized as the face image of the same person as the person in the face image area extracted in step S24 in the face recognition process in step S25. Registration information (registration number, keyword (person information, related information)) 153b associated with the image is displayed in a list. When a plurality of persons are specified from one captured image, thumbnail images and registration information of registered images for the specified persons are displayed. A “result” button 153c and an “image” button 153d are displayed on the collation result list screen 153. When the “image” button 153d is operated by the operation unit 18, the collation result screen 152 shown in FIG. .

  When the operation unit 18 detects a key operation corresponding to the “decision” button 153c on the collation result list screen 153 or the “decision” button 152d on the collation result screen 152, whether or not to register the collation result in the captured image data A screen (not shown) for inquiring is displayed on the display unit 15, and after the user confirms the collation result, it is possible to input an instruction as to whether or not to register the collation result in the captured image data.

  Returning to FIG. 8, when the operation unit 18 detects a key operation corresponding to the “decision” button 152 d or the “decision” button 153 c, and further, an instruction to register a collation result in the captured image data is input by the operation unit 18. (Step S27; YES), a note including collation permission information and collation result (registration number of registered image data specified as including the image in the captured image data) is generated and added to the header portion of the captured image data ( In step S28), the process proceeds to step S30. On the other hand, when an instruction not to register the collation result in the photographed image data is input by the operation unit 18 (step S27; YES), a note including collation prohibition information is generated and added to the header part of the photographed image data (step S29). ), The process proceeds to step S30.

  As described above, the note is information (see FIG. 5) to be written in the header part of the captured image data. The verification permission information is information that is written when an instruction to permit verification is input by the user in step S23 and the user selects to register the verification result in step S27. The captured image data in which the verification permission information is written is a search target in the search process shown in FIG. The collation prohibition information is information written when an instruction not permitting collation is input in step S23 and collation with a registered image is not performed, or when an instruction not registering a collation result is selected in step S27. . The captured image data in which the verification prohibition information is written is not a search target in the search process shown in FIG.

  In step S30, the image data captured in step S22 is compressed in a predetermined file format (eg, JPEG format, etc., see FIG. 5) (step S30), and the compressed captured image data is stored in the captured image storage area 174. Stored (step S31), the process ends.

[Search processing]
Next, a search process executed in the electronic camera 10 will be described with reference to a flowchart shown in FIG. This search process is started when the user of the electronic camera 10 instructs the shift to the search mode by operating the operation unit 18.

  When the operation of the instruction to shift to the search mode is detected, the search mode is activated by loading the search mode program stored in the program storage area 171 into the RAM of the control unit 16 (step S40). Next, search information stored in the registration information storage area 172, that is, keywords are read out, and a search condition selection screen 154 displaying a list of the read keywords is displayed on the display unit 15 (step S41).

  FIG. 12 shows an example of the search condition selection screen 154. As shown in FIG. 12, the search condition selection screen 154 displays a list of keywords stored in the registration information storage area 172. On the search condition selection screen 154, the user can select a keyword by moving the cursor to a desired keyword by operating the cross key of the operation unit 18 and operating the key corresponding to the “select” button 154b. A plurality of keywords can be selected, and a search method such as AND search or OR search of a plurality of displayed keywords may be set. In the search condition selection screen 154, when a key corresponding to the “search” button 154a is operated in a state where a keyword is selected, it is possible to search for photographed image data associated with the selected keyword. Alternatively, the keyword selection may be canceled by operating the key corresponding to the “clear” button 154 c by the operation unit 18 on the search condition selection screen 154.

  In the search condition selection screen 154, a keyword is selected by a key operation corresponding to the “select” button 154b of the operation unit 18 (step S42), and then a key operation corresponding to the “search” button 154a is detected (step S43; YES), registration information including the selected keyword is searched in the registration information storage area 172, and the registration number of the registered image data associated with the selected keyword is specified (step S44). Next, the header portion of each photographed image data stored in the photographed image storage area 174 is referred to, and the photographed image data including the registration number specified in step S44, in which the verification permission information is written as a note, is searched. (Step S45). Then, the thumbnail image of the searched photographic image data is displayed on the search result display screen 155 (step S46), and this process ends.

  FIG. 13 shows an example of the search result display screen 155. On this screen, the retrieval result of the photographed image data corresponding to the keyword selected in step S42 among the photographed image data stored in the photographed image storage area 174 is displayed. As described above, the user can easily display only the photographed image data corresponding to the keyword simply by performing an operation of selecting the keyword with the operation unit 18.

[Album creation process]
Next, album creation processing executed in the electronic camera 10 will be described with reference to a flowchart shown in FIG. This album creation process is a process that uses the search process shown in FIG. 11 and is started when the user of the electronic camera 10 instructs the transition to the album creation mode by operating the operation unit 18. The

  When the operation of the instruction to shift to the album creation mode is detected, the album creation mode is activated by loading the album creation mode program stored in the program storage area 171 into the RAM of the control unit 16 (step S51). . Next, the search process of FIG. 11 is executed, and a search result display screen is displayed on the display unit 15 (step S52). Note that the search result display screen in the album creation process is substantially the same as the search result display screen 155 shown in FIG. 13, and a list of thumbnail images of searched photographed images is displayed. An “edit” button is provided, and album editing such as image selection, arrangement order change, page layout, text insertion, and the like can be performed by operating keys on the operation unit 18.

  When the operation unit 18 is operated on the search result display screen, album editing is executed in accordance with the operation (step S53). Until the “end” button on the search result display screen is operated by the operation unit 18 and the end of editing is instructed, the processes of steps S52 to S53 are repeatedly executed according to the operation of the operation unit 18. When the operation unit 18 operates the “end” button on the search result display screen to instruct the end of editing (step S54; YES), the edited album data is stored in the storage unit 17 (step S55).

  If the operation unit 18 does not input an instruction to transmit the edited album data (step S56; NO), this process ends. When the operation unit 18 inputs an instruction to transmit the edited album data and the data transmission destination (step S56; YES), the data is transmitted to the external device as the transmission destination via the external device interface unit 19. (Step S57), this process ends.

  As described above, according to the electronic camera 10 of the first embodiment, the registered image data registered by the registration process is stored in the storage unit 17 in association with at least one keyword. When switching to the search mode is instructed by the operation of the operation unit 18, all the keywords stored in association with the registered image data in the registration information storage area 172 of the storage unit 17 are read out and displayed in a list on the display unit 15. The photographed image data including the image of the registered image data corresponding to the keyword selected from the displayed keywords is searched from the photographed image data stored in the storage unit 17, and the search result is displayed.

  Therefore, the user can easily select a captured image keyword to be searched from the captured images stored in the storage unit 17 without performing a complicated operation. You can search.

  Further, at the time of shooting, the captured image data is collated with each of the registered image data stored in the storage unit 17 to identify the image of the registered image data included in the captured image data, and the registration number of the registered image is set as the captured image data. The captured image data stored in the storage unit 17 is stored in the captured image storage area 174 of the storage unit 17 in association with the captured image data corresponding to the registration number indicating the registered image data corresponding to the keyword. Therefore, it is not necessary to collate the registered image data with the photographed image data at the time of retrieval, and the retrieval process can be performed in a short time.

  Also, registered image data, which is a person's face image, is stored in association with a keyword, and when collating the captured image data with the registered image data, a face image area included in the captured image data is detected, and this By comparing the detected face image area with each of registered image (face image) data stored in the storage unit 17, the face image included in the photographed image data is specified, and the face image included in the photographed image data is identified. Since the registration number is stored in association with the photographed image data, it is possible to search for photographed image data that includes the face image of the person associated with the keyword without performing complicated operations by simply selecting the keyword. Is possible.

  In addition, when the photographed image data is stored in the storage unit 17, the user can select whether or not to collate the photographed image data with the registered image data. For example, when a face image is used as a registered image. When a landscape is photographed, useless collation processing can be omitted.

  In addition, as a result of the comparison between the captured image data and the registered image data, the registration number of the registered image data, which is assumed to include the image in the captured image data, is added to the notebook of the captured image data and the captured image data. Registered image data including an image can be associated with the data. Further, when searching for an image using a keyword, it is possible to easily search for photographed image data including an image of registered image data corresponding to the selected keyword.

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described. In the first embodiment, in the photographing process, collation permission (prohibition) information and collation result information are written as notes in the header portion of the photographed image data. In the second embodiment, collation permission (prohibition) is performed. The storage unit 17 does not directly write information or verification result information to the captured image data, but stores the captured image data, verification permission (prohibition) information, and verification result information (registration number) in association with each other. It is the structure provided in.

  FIG. 15 shows a configuration example of the storage unit 17 in the second embodiment. As shown in FIG. 15, in the second embodiment, the storage unit 17 has a collation information storage area 175. As shown in FIG. 16, the collation information storage area 175 has a “photographing number” area, a “file name (photographed image)” area, a “collation permission / prohibition information” area, and a “collation result information” area. The collation information about the target image data, that is, the image number for identifying the image data to be collated, the file name of the image data, the collation permission (prohibition) information, and the collation result information are associated with each other. Remember.

  Other configurations in the second embodiment are the same as those described in the first embodiment, and thus description thereof is omitted.

Next, the operation in the second embodiment will be described.
In the second embodiment, the photographing process is different from that in the first embodiment. In the shooting process in the second embodiment, image data acquired by shooting is compressed into a predetermined file format and stored in the shot image storage area 174. That is, the processes of steps S30 and S31 are executed following the process of step S22 in FIG.

  In the second embodiment, the electronic camera 10 has a collation mode, and the captured image data acquired by the imaging process or the captured image data input via the external device interface unit 19 or the like (by the external device). Acquired photographed image data) and the registered image data registered in the registered image storage area 173 with respect to the photographed image data stored in the external storage device 20a connected via the external storage interface unit 20. Verification is possible.

[Verification processing]
Hereinafter, collation processing executed in the electronic camera 10 will be described with reference to a flowchart shown in FIG. This collation process is started when the user of the electronic camera 10 instructs the shift to the collation mode by operating the operation unit 18.

  When an operation for instructing the transition to the verification mode is detected, the verification mode program is activated by loading the verification mode program stored in the program storage area 171 into the RAM of the control unit 16 (step S60). Next, the captured image data to be collated is acquired according to the operation of the operation unit 18 (step S61). For example, a list of photographed image data stored in the photographed image storage area 174 of the storage unit 17 on the display unit 15 and photographed image data stored in the external storage device 20 a connected via the external storage interface unit 20. When (file name or thumbnail image) is displayed and a captured image to be collated is selected by operating the operation unit 18, the selected captured image is read into the RAM of the control unit 16. The captured image data may be acquired via the external device interface unit 19.

  Next, a selection screen (not shown) for the user to select whether or not to allow the acquired captured image data to be collated with the registered image data stored in the storage unit 17 is displayed on the display unit 15. If an operation for selecting “not permitting collation” is detected on the screen by the operation unit 18 (step S62; NO), the process proceeds to step S68.

  On the other hand, when an operation for selecting “permit collation” is detected by the operation unit 18 on the selection screen (step S62; YES), detection of a face image area is executed on the acquired captured image data (step S62). Step S63). The detection of the face image area can be performed by the same method as described in step S24 of FIG.

  Next, face recognition processing is executed for each detected face image region (step S64). The face recognition process can be performed by the same method as described in step S25 of FIG.

  When the face recognition process is completed, a matching result screen 152 is displayed on the display unit 15 (step S65). The verification result screen 152 is the same as that shown in FIG. A “result” button 152d and a “list” button 152e are displayed on the collation result screen 152. When the key corresponding to the “list” button 152e is operated by the operation unit 18, the collation result list screen 153 is displayed. Is done. The matching result list screen 153 is the same as that shown in FIG.

  When the operation unit 18 detects a key operation corresponding to the “decision” button 153c on the collation result list screen 153 or the “decision” button 152d on the collation result screen 152, whether or not to register the collation result in the captured image data A screen (not shown) for inquiring whether or not registration of the verification result is permitted in the verification information storage area 175 after the user confirms the verification result can be input. It becomes.

  A key operation corresponding to the “decision” button 152d on the collation result screen 152 or the “decision” button 153c on the collation result list screen 153 by the operation unit 18 is detected, and further, registration of the collation result to the collation information storage area 175 is permitted. When an instruction to do so is input by the operation unit 18 (step S66; YES), the shooting number assigned to the captured image data in the verification information storage area 175, the file name of the captured image data, verification permission information, and verification result information (imaging result) The registration number of the registered image data in which the image is included in the image data) is additionally registered, and the collation information is updated (step S67). Next, the process proceeds to step S69.

On the other hand, when an instruction not to register the collation result in the photographed image data is input by the operation unit 18 (step S66; NO), the photographing number assigned to the photographed image data and the photographed image data are stored in the collation information storage area 175. The file name and collation prohibition information are additionally registered and the collation information is updated (step S68), and the process proceeds to step S69.
In steps S67 and S68, the collation information storage area 175 is searched with the file name of the photographic image data to be collated, and collation information having the same file name as the photographic image data to be collated is already registered. If it is registered, it is preferable to overwrite the collation information with new information.

  In step S69, it is determined whether or not a collation process end instruction has been input by the operation unit 18, and if it is determined that it has not been input (step S69; NO), the process returns to step S61. If it is determined by the operation unit 18 that a collation process end instruction has been input (step S69; YES), this process ends.

  As described above, according to the electronic camera 10 of the second embodiment, the registration number of the registered image data that is determined to include the image as a result of the comparison between the captured image data and the registered image data is used as the captured image. By storing the data in the collation information storage area 175 in association with the data, the captured image data can be associated with the registered image data including the image in the captured image data. Further, when searching for an image using a keyword, it is possible to easily search for photographed image data including an image of registered image data corresponding to the selected keyword.

  Further, the collation between the captured image data and the registered image data can be started not only at the time of capturing but also in response to a user operation, so that convenience is improved. In addition, captured image data acquired from an external device can also be a target for collation.

  As mentioned above, although Embodiment 1 and Embodiment 2 of this invention were demonstrated, as an electronic camera 10 which concerns on this invention, it is not restricted to embodiment mentioned above, It can apply also to the thing of another structure. The image search apparatus according to the present invention is applicable not only to an electronic camera but also to other electronic devices such as a mobile phone and a PDA (Personal Digital Assistant).

It is a block diagram which shows the functional structure of the electronic camera in embodiment of this invention. It is a figure which shows the example of a data storage of the registration information storage area | region of FIG. It is a figure which shows the example of data storage of the registration image storage area | region of FIG. It is a figure which shows the example of data storage of the picked-up image storage area | region of FIG. It is a figure which shows the file structural example of the picked-up image data of FIG. It is a flowchart for demonstrating the "registration process" in embodiment of this invention. FIG. 7 is a diagram illustrating a display example of a “keyword input screen” in the registration process of FIG. 6. It is a flowchart for demonstrating "the imaging | photography process" in embodiment of this invention. FIG. 9 is a diagram illustrating a display example of a “collation result screen” in the photographing process of FIG. 8. FIG. 9 is a diagram illustrating a display example of a “collation result list screen” in the photographing process of FIG. 8. It is a flowchart for demonstrating the "search process" in embodiment of this invention. FIG. 12 is a diagram showing a display example of a “search condition selection screen” in the search process of FIG. 11. FIG. 12 is a diagram showing a display example of a “search result display screen” in the search process of FIG. 11. It is a flowchart for demonstrating the "album creation process" in embodiment of this invention. 6 is a diagram illustrating a configuration example of a storage unit in Embodiment 2. FIG. It is a figure which shows the example of data storage of the collation information storage area | region of FIG. 10 is a flowchart for explaining “collation processing” in the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Electronic camera 11 Optical lens part 12 Image sensor 13 Image processing part 14 Video output part 15 Display part 16 Control part 17 Storage part 171 Program storage area 172 Registration information storage area 173 Registration image storage area 174 Photographed image storage area 175 Collation information storage Area 18 Operation unit 19 External device interface unit 20 External storage interface unit 20a External storage device

Claims (10)

Photographic image storage means for storing photographic images;
Search information storage means for storing a registered image registered in advance and at least one search information in association with each other;
Search information selection means for the user to select desired search information from the search information stored in the search information storage means;
Image search for searching for a registered image associated with the search information selected by the search information selection unit from the search information storage unit, and searching for a captured image including the searched registered image from the captured image storage unit Means,
An image search apparatus comprising: The search information storage means stores the identification information of the registered image and the at least one search information in association with each other,
Collating the photographed image with each of the registered images stored in the search information storage means to detect a registered image included in the photographed image, and associating the identification information of the detected registered image with the photographed image With means,
The image retrieval unit retrieves a registered image associated with the retrieval information selected by the retrieval information selection unit from the retrieval information storage unit, and captures the identification information associated with the retrieved registered image. Retrieving an image from the captured image storage means;
The image search apparatus according to claim 1. The registered image is a face image;
The collating unit detects a face image area included in the photographed image, and collates the detected face image area with each of the face images stored in the search information storage unit, thereby obtaining the photographed image. Detect included face images,
The image search apparatus according to claim 2. When storing the photographed image in the photographed image storage means, a collation selection means for selecting whether to collate the photographed image with a registered image stored in the search information storage means,
When the collation selecting unit selects collation, the collation unit collates the captured image with each of the registered images stored in the search information storage unit, and obtains a registered image included in the photographed image. Detecting and adding identification information of the detected registered image to the captured image;
The image search device according to claim 2, wherein the image search device is an image search device.   5. The image search apparatus according to claim 4, wherein the image search unit searches a shot image including identification information of the searched registered image from a shot image stored in the shot image storage unit. Collation selection means for selecting whether or not to collate the captured image with a registered image stored in the search information storage means;
Collation information storage means for storing the photographed image and the selection result of the collation selection means in association with each other,
When the collation selecting unit selects collation, the collation unit collates the captured image with each of the registered images stored in the search information storage unit, and obtains a registered image included in the photographed image. Detecting the identification information of the detected registered image, and further storing the identification information in association with the captured image and the selection result in the collation information storage unit.
The image search device according to claim 2, wherein the image search device is an image search device.   The image search device according to claim 6, wherein the image search unit searches the captured image associated with the identification information of the searched registered image with reference to the collation information storage unit. In a photographing device that photographs a subject and obtains a photographed image,
An image capturing apparatus comprising the image search device according to claim 1. A captured image storing step for storing the captured image;
A search information storage step of storing a registered image registered in advance and at least one search information in association with each other;
A search information selection step for the user to select desired search information from the search information stored in the search information storage step;
A registered image associated with the search information selected in the search information selection step is searched from the registered image stored in the search information storage step, and a captured image including the searched registered image is stored in the captured image storage. An image search step for searching from the captured image stored in the step;
An image search method comprising: On the computer,
Captured image storage function for storing captured images,
A search information storage function for storing a registered image registered in advance and at least one search information in association with each other;
A search information selection function for the user to select desired search information from the search information stored by the search information storage function;
The registered image associated with the search information selected by the search information selection function is searched from the registered image stored by the search information storage function, and the captured image including the searched registered image is stored in the captured image storage. Image search function to search from captured images stored by the function,
A program characterized by realizing.

Images