JP5293206B2 - Image search apparatus, image search method and program - Google Patents

Description

The present invention relates to an image search apparatus , an image search method using the image search apparatus , and a program.

  Conventionally, attribute information for identifying a photographer's fingerprint at the time of photographing and identifying the photographer's individual is added to the recording image data and stored, or an object for identifying a subject by face recognition after photographing is identified 2. Description of the Related Art An imaging device that can perform an image search specifying an individual by adding information to recording image data and storing it is known (for example, see Patent Document 1).

JP 2001-333360 A

  However, in the case of Patent Document 1 described above, image data transferred from an external device or the like cannot be used to search for an image using attribute information unless various attribute information for specifying an individual is once stored in association with each other. There's a problem.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image search device that can perform image search more easily, and an image search method and program using the image search device .

In order to solve the above-described problems, an image search apparatus according to an embodiment of the present invention stores recording means for recording a plurality of images and face image data and fingerprint data in association with each other for a plurality of persons. A data storage means; a fingerprint detection means for detecting a fingerprint of the finger to generate fingerprint data; and a fingerprint for specifying the fingerprint detected by the fingerprint detection means as a target designation fingerprint for designating a face to be searched The facial image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detection means in correspondence with the specifying means and the fingerprint for specifying the object specified by the fingerprint specification means is specified. A face image specifying means; a face detecting means for detecting a face related to the face image data specified by the face image specifying means from an image; and a plurality of images recorded in the recording means. For each, a determination unit configured to determine whether a face has been detected by said face detecting means, and output means for outputting as a search result image determined as a face has been detected by the determination unit, the fingerprint detection A plurality of detection judging means for judging whether or not a plurality of fingerprints are detected at the same time or within a predetermined time when the fingerprint is detected by the means, wherein the fingerprint specifying means is simultaneously or When it is determined that a plurality of fingerprints are detected within a predetermined time, the plurality of detected fingerprints are specified as the target designating fingerprints, and the determination unit includes the target designating fingerprints including a plurality of fingerprints. In this case, whether or not at least one of a plurality of faces corresponding to the plurality of fingerprints is detected by the face detection unit for each of the plurality of images recorded in the recording unit. Determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection means, and the output means, when the target designating fingerprint includes a plurality of fingerprints, An image in which at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected by the determining means, or a plurality of faces corresponding to the plurality of fingerprints has been determined to have been detected by the determining means. It is characterized by outputting an image as a search result .

According to the second aspect of the present invention, the determination by the determination unit includes, for each of a plurality of images recorded in the recording unit, a plurality of faces corresponding to the plurality of fingerprints by the face detection unit. The output by the output means includes outputting, as a search result, an image determined by the determination means that all the faces corresponding to the plurality of fingerprints have been detected. It is characterized by including .

According to a third aspect of the present invention, in the determination by the determination unit, a plurality of faces corresponding to the plurality of fingerprints are detected by the face detection unit for each of the plurality of images recorded in the recording unit. comprises at least one determination of whether the detected inner, wherein the output by the output means, at least one of a plurality of faces corresponding to the plurality of fingerprints has been determined to have been detected by the determination unit image Is output as a search result.

The invention according to claim 4 further comprises search condition designating means for designating whether the search is performed using an AND condition or an OR condition, and the output means is configured such that an AND condition is designated by the search condition designating means. If the determination means outputs an image determined as having detected all of the plurality of faces corresponding to the plurality of fingerprints as a search result, and the OR condition is specified by the search condition specifying means Is characterized in that an image determined to have detected at least one of a plurality of faces corresponding to the plurality of fingerprints by the determining means is output as a search result .

According to a fifth aspect of the present invention, the fingerprint detection means further comprises a fingerprint sensor provided at a predetermined position of the apparatus main body, and further includes a finger detection means for detecting that a finger is placed on the fingerprint sensor. It is characterized by providing .

According to the sixth aspect of the present invention, when the finger detection unit detects that a finger is placed on the fingerprint sensor, the face image specifying unit specifies the face image data, and the determination The process sequentially controls execution of a process for determining whether or not a face has been detected by the face detection means, and a process for outputting an image in which the face has been detected by the determination means as a search result. It is further characterized by further comprising a control means .

The image search method of the invention described in claim 7 is an image comprising recording means for recording a plurality of images, and data storage means for storing face image data and fingerprint data in association with each other for a plurality of persons. Fingerprint detection processing for detecting a fingerprint of a finger and generating fingerprint data in the search device, and fingerprint identification for specifying the fingerprint detected by the fingerprint detection processing as a fingerprint for specifying a target to be searched A face for identifying the face image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detection process corresponding to the fingerprint for object specification identified by the fingerprint identification process An image specifying process; a face detecting process for detecting a face related to the face image data specified by the face image specifying process from an image; and a plurality of images recorded in the recording unit For each, a determination process for determining whether or not a face has been detected by the face detection process, an output process for outputting an image determined to have detected a face by the determination process as a search result, and the fingerprint detection process A plurality of detection determination processes for determining whether a plurality of fingerprints are detected simultaneously or within a predetermined time when a fingerprint is detected, and the fingerprint identification process is performed simultaneously or with a predetermined number of the detection processes. When it is determined that a plurality of fingerprints have been detected within a time period, the detected plurality of fingerprints are specified as the target designating fingerprint, and the determination process is performed when the target designating fingerprint includes a plurality of fingerprints. Whether or not at least one of a plurality of faces corresponding to the plurality of fingerprints is detected by the face detection processing for each of a plurality of images recorded in the recording unit Or determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection processing, and the output processing is performed when the target designating fingerprint includes a plurality of fingerprints, It is determined that at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected by the determination process, or a plurality of faces corresponding to the plurality of fingerprints has been detected by the determination process. It is characterized by outputting the retrieved image as a search result.
According to an eighth aspect of the present invention, there is provided an image search apparatus comprising: a recording unit that records a plurality of images; and a data storage unit that stores face image data and fingerprint data in association with each other for each of a plurality of persons. Fingerprint detecting means for generating fingerprint data by detecting a fingerprint of a finger, and fingerprint specifying means for specifying the fingerprint detected by the fingerprint detecting means as a target specifying fingerprint for specifying a face to be searched Face image specification for specifying the face image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detection means corresponding to the target specifying fingerprint specified by the fingerprint specifying means Means for detecting a face related to the face image data specified by the face image specifying means from an image, and a plurality of information recorded in the recording means For each of the images, a determination unit that determines whether or not a face is detected by the face detection unit, an output unit that outputs an image determined to have detected a face by the determination unit as a search result, and a fingerprint detection unit When detecting a fingerprint, it is made to function as a plurality of detection judging means for judging whether or not a plurality of fingerprints are detected at the same time or within a predetermined time, and the fingerprint specifying means is operated simultaneously or for a predetermined time by the plurality of detection judging means. If it is determined that a plurality of fingerprints have been detected, the plurality of detected fingerprints are identified as the target designating fingerprints, and the determination means is configured to include a plurality of fingerprints. Whether or not at least one of a plurality of faces corresponding to the plurality of fingerprints is detected by the face detection unit for each of the plurality of images recorded in the recording unit Or determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection unit, and the output unit includes the case where the target designating fingerprint includes a plurality of fingerprints. An image in which at least one of a plurality of faces corresponding to the plurality of fingerprints is detected by the determination unit, or a determination that all the plurality of faces corresponding to the plurality of fingerprints have been detected by the determination unit It is characterized by outputting the obtained image as a search result.

  According to the present invention, an image search can be more easily performed only by detecting a fingerprint of a finger of a person who performs an image search.

It is a rear view which shows the imaging device of one Embodiment to which this invention is applied. It is a block diagram which shows schematic structure of the imaging device of FIG. FIG. 2 is a diagram illustrating an example of a fingerprint registration table stored in a fingerprint registration DB of the imaging apparatus in FIG. 1. 3 is a flowchart showing an example of an operation related to fingerprint registration processing by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to an image search process performed by the imaging apparatus of FIG. 1. It is a figure for demonstrating the image search process of FIG. It is a figure for demonstrating the image search process of FIG.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.
FIG. 1 is a rear view illustrating an imaging apparatus 100 according to an embodiment to which the present invention is applied, and FIG. 2 is a block diagram illustrating a schematic configuration of the imaging apparatus 100.

The imaging apparatus 100 according to the present embodiment associates face image data and fingerprint data in association with the fingerprint data generated by the fingerprint detection unit 6 in association with face image data and fingerprint data for each of a plurality of persons. Face image data stored in the fingerprint registration DB 8 is specified, and for each of a plurality of images recorded in the recording medium 3, it is determined whether or not a face related to the face image data is detected by the face detection unit 2a Then, an image determined to have detected a face is output as a search result.
Specifically, as illustrated in FIGS. 1 and 2, the imaging apparatus 100 includes an imaging unit 1, an image processing unit 2, a recording medium 3, a display processing unit 4, a display unit 5, and a fingerprint detection unit. 6, a face image specifying unit 7, a fingerprint registration DB 8, an image search unit 9, an operation input unit 10, a memory 11, and a CPU 12.

  The imaging unit 1 captures a subject and generates an image frame as an imaging unit. Specifically, the imaging unit 1 includes a lens unit 1a, an electronic imaging unit 1b, and an imaging processing unit 1c.

  Although not shown, the lens unit 1a includes a plurality of lenses, and includes, for example, a zoom lens and a focus lens. Although not shown, the lens unit 1a includes a zoom drive unit that moves the zoom lens in the optical axis direction and a focus drive unit that moves the focus lens in the optical axis direction when imaging a subject. Yes.

  The electronic imaging unit 1b is composed of, for example, a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and converts a subject image that has passed through various lenses of the lens unit 1a into a two-dimensional image signal.

Although not shown, the imaging processing unit 1c includes a timing generator, a driver, and the like. Then, the imaging processing unit 1c scans and drives the electronic imaging unit 1b with a timing generator and a driver, converts the subject image into a two-dimensional image signal with the electronic imaging unit 1b at a predetermined period, and the electronic imaging unit 1b. The image frames are read out from the imaging area for each screen and output to the image processing unit 2.
Further, the imaging processing unit 1c performs control of AE (automatic exposure processing), AF (automatic focusing processing), AWB (automatic white balance), and the like.

The image processing unit 2 performs image quality adjustment processing, resolution conversion processing, image encoding processing, and the like based on the image frame transferred from the imaging processing unit 1c, and generates display image data and recording image data. Specifically, the image processing unit 2 appropriately adjusts the gain for each color component of RGB with respect to the analog value signal of the image frame transferred from the imaging processing unit 1c, and then performs a sample with a sample hold circuit (not shown). Hold, convert to digital data by an A / D converter (not shown), and perform color process processing including pixel interpolation processing and γ correction processing in a color process circuit (not shown), and then the luminance of the digital value A signal Y and color difference signals Cb and Cr are generated.
The luminance signal Y and the color difference signals Cb and Cr output from the color process circuit are DMA transferred to a memory 11 used as a buffer memory via a DMA controller (not shown).

In addition, the image processing unit 2 includes a face detection unit 2 a that detects a face using a predetermined face detection method among a plurality of images recorded on the recording medium 3.
Specifically, the face detection unit 2 a detects a face related to the face image data specified by the face image specifying unit 7 (described later) from among a plurality of images recorded on the recording medium 3. Since this face detection process is a known technique, detailed description thereof is omitted here.
Here, the face detection unit 2a constitutes a face detection unit that detects a face related to the face image data specified by the face image specifying unit 7 from the image.

  The recording medium 3 is composed of, for example, a nonvolatile memory (flash memory) or the like, and records still image recording image data encoded by a JPEG compression unit (not shown) of the image processing unit 2.

  The display processing unit 4 performs control to read display data temporarily stored in the memory 11 and display it on the display unit 5. Specifically, the display processing unit 4 includes a VRAM, a VRAM controller, a digital video encoder, and the like. The digital video encoder periodically reads the luminance signal Y and the color difference signals Cb and Cr read from the memory 11 and stored in the VRAM (not shown) under the control of the CPU 12 from the VRAM via the VRAM controller. And a video signal is generated based on these data and output to the display unit 5.

  The display unit 5 includes a liquid crystal display device provided to expose the display screen on the back side of the main body of the imaging device 100 (see FIG. 4B). And the display part 5 displays the image etc. which were imaged by the electronic imaging part 1b based on the video signal from the display process part 4 on a display screen. Specifically, the display unit 5 displays a live view image based on a plurality of image frames generated by imaging the subject by the imaging unit 1 in the imaging mode, or a REC view captured as a main captured image. Display an image.

The display unit 5 includes a fingerprint detection unit 6 that detects a fingerprint of a user's finger and generates fingerprint data under the control of the CPU 12.
The fingerprint detection unit 6 includes a fingerprint sensor 6a that generates fingerprint data of a user's finger. The fingerprint sensor 6a is incorporated in the liquid crystal display device of the display unit 5, and is composed of an optical sensor built in each pixel. That is, the fingerprint sensor 6 a is arranged over the front surface of the display screen of the display unit 5. Then, the fingerprint sensor 6a reads the uneven shape of the fingerprint of the fingertip placed on the display screen of the display unit 5, and outputs the signal as fingerprint data.
Here, the fingerprint detection unit 6 and the CPU 12 constitute a fingerprint detection unit that detects a fingerprint of any finger of the user and generates fingerprint data.

  The fingerprint detection unit 6 detects, as a finger detection unit, that the finger is placed on the fingerprint sensor 6a when the light receiving state of the optical sensor changes as a predetermined area of the fingerprint sensor 6a is covered with the finger.

The fingerprint detection unit 6 includes a fingerprint identification unit 6b that identifies target designation fingerprint data for designating a face to be subjected to image search.
The fingerprint identification unit 6b is for specifying a plurality of fingerprint data generated by the fingerprint sensor 6a when a plurality of human fingers are placed simultaneously or within a predetermined time within a predetermined detection area of the fingerprint sensor 6a. Specify as fingerprint data.
Here, the fingerprint detection unit 6 and the CPU 12 constitute a fingerprint specifying means for specifying all fingerprints detected simultaneously or within a predetermined time within the predetermined detection area of the fingerprint sensor 6a as the target specifying fingerprint. Yes.

  The display unit 5 displays the image searched by the image search process. That is, the display unit 5 constitutes an output unit that outputs, as a search result, an image for which a face is determined to be detected by the determination unit 9a of the image search unit 9.

The face image specifying unit 7 specifies face image data registered in the fingerprint registration table T1 of the fingerprint registration DB 8 in association with the fingerprint data generated by the fingerprint detection unit 6 under the control of the CPU 12.
The fingerprint registration DB 8 stores face image data and fingerprint data in association with each other for each of a plurality of persons as data storage means. Specifically, as shown in FIG. 3, the fingerprint registration DB 8 stores a fingerprint registration table T1 in which a person, face image information, and fingerprint information are associated with each other.
The fingerprint registration table T1 stores, for example, “Mr. A” as a person, “face image data A” as face image information, and “fingerprint A” as fingerprint information in association with each other. Similarly, “Mr. B” as a person, “face image data B” as face image information, and “fingerprint B” as fingerprint information are stored in association with each other. Similarly, “Mr. C” as a person, “face image data C” as face image information, and “fingerprint C” as fingerprint information are stored in association with each other.
The face image information is not limited to the face image data, and may be any information as long as it is information related to a face image for specifying a person, and may be face feature data or the like.

Then, the face image specifying unit 7 includes each of the plurality of fingerprint data registered in the fingerprint registration table T1, fingerprint data generated by the fingerprint sensor 6a, and specified as fingerprint data for object specification by the fingerprint specifying unit 6b. Are compared to identify matching fingerprint data. Then, the face image specifying unit 7 specifies face image data associated with the specified fingerprint data.
At this time, the face image specifying unit 7 specifies face image data associated with the fingerprint data of at least one fingerprint among the plurality of fingerprints specified as the target specifying fingerprint.
Here, the face image specifying unit 7 and the CPU 12 constitute face image specifying means for specifying the face image data stored in the fingerprint registration DB 8 in association with the fingerprint data generated by the fingerprint detecting unit 6. .

The image search unit 9 searches for image data desired by the user from among a plurality of image data recorded on the recording medium 3 under the control of the CPU 12.
Specifically, the image search unit 9 includes a determination unit 9a that determines whether or not a face is detected by the face detection unit 2a for each of a plurality of images recorded on the recording medium 3.
The determination unit 9a, as a determination unit, determines whether or not the face detection unit 2a detects the face related to the face image data specified by the face image specification unit 7 for each of the plurality of images recorded on the recording medium 3. Determine. Specifically, the determination unit 9a extracts facial feature data from the face image data specified by the face image specifying unit 7, and also from the face detected by the face detection unit 2a in the plurality of image data. Face feature data is extracted, and it is determined whether or not the difference between these feature data is equal to or less than a predetermined threshold. When the difference between these feature data is equal to or less than a predetermined threshold, it is determined that the face related to the face image data specified by the face image specifying unit 7 has been detected by the face detecting unit 2a10.
Further, when a plurality of face image data is specified by the face image specifying unit 7, the determination unit 9a has at least one or all of the faces related to the plurality of face image data detected by the face detecting unit 2a. It is determined whether or not it has been detected.

The image search unit 9 includes an automatic execution control unit 9b that automatically executes an image search process.
When the finger detection unit 6 detects that the finger is placed on the fingerprint sensor 6a as a control means, the automatic execution control unit 9b controls the face image specifying unit 7 to convert the face image data under the control of the CPU 12. A process for specifying, a process for determining whether or not a face is detected by the face detection unit 2a, and a display unit 5 displaying an image determined to have detected a face by the determination unit 9a as a search result The execution of processing is sequentially controlled.

  The operation input unit 10 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 10 includes a shutter button 10a related to a subject shooting instruction, a menu button 10b related to a menu screen display instruction related to function selection, display setting, and the like (see FIG. 4B, etc.), operation A selection / decision button 10c (see FIG. 4B, etc.) related to selection of a mode or the like, and a zoom button (see FIG. 4B, etc.) related to an instruction to adjust the zoom amount are provided. And the operation input part 10 outputs a predetermined operation signal to CPU12 according to operation of these buttons.

  The CPU 12 controls each unit of the imaging device 100. Specifically, the CPU 12 performs various control operations according to various processing programs for the imaging apparatus 100.

  The memory 11 is composed of, for example, a flash memory, and temporarily stores data processed by the CPU 12.

"
Next, fingerprint registration processing by the imaging apparatus 100 will be described with reference to FIG.
FIG. 4 is a flowchart illustrating an example of an operation related to the fingerprint registration process.

The fingerprint registration process is a process executed when the fingerprint registration mode is selected based on a predetermined operation of the operation input unit 10 by the user.
As shown in FIG. 4, when the fingerprint registration mode is selected and designated among a plurality of operation modes based on a predetermined operation of the operation input unit 10 by the user (step S <b> 1), the CPU 12 performs an operation input by the user. An input of a person name (for example, Mr. A) is accepted based on a predetermined operation of the unit 10.
When a person name is input (step S2), the CPU 12 causes the image capturing unit 1 to capture an image (face image) including the face of the person (step S3), and face image data (for example, face image data A). ) Is generated (step S4).

  Next, the CPU 12 causes the fingerprint detection unit 6 to detect the fingerprint of the person whose face image is captured by the fingerprint sensor 6a (step S5) and generate fingerprint data (for example, fingerprint A) of the fingerprint (step S6). ).

  Thereafter, the CPU 12 associates the person name, the face image data generated by the imaging unit 1 and the fingerprint data generated by the fingerprint detection unit 6 and registers them in the fingerprint registration table T1 of the fingerprint registration DB 8 (step S7). ).

Next, the CPU 12 determines whether or not an instruction to end the fingerprint registration mode is input based on a predetermined operation of the operation input unit 10 by the user (step S8). Here, if it is determined that an instruction to end the fingerprint registration mode has not been input (step S8; NO), the CPU 12 determines the next person name (for example, for example, based on a predetermined operation of the operation input unit 10 by the user). It is determined whether or not (Mr. B) has been input (step S9). The next person name includes a new person name different from the already set person name, as well as a person name that is already set and that the user desires to reset.
When it is determined that the next person name has been input (step S9; YES), the CPU 12 shifts the process to step S2 and repeatedly executes the subsequent processes. In this way, a face image and a fingerprint are registered in association with each other for each of a plurality of persons.

Thereafter, when it is determined in step S9 that the next person name has not been input (step S9; NO), the CPU 12 shifts the processing to step S8.
If it is determined in step S8 that an instruction to end the fingerprint registration mode has been input (step S8; YES), the CPU 12 ends the fingerprint registration process.

Next, image search processing by the imaging apparatus 100 will be described with reference to FIGS.
FIG. 5 is a flowchart illustrating an example of an operation related to the image search process. FIG. 6 is a diagram illustrating an example of the state of the imaging apparatus 100 during the image search process, and FIG. 7 is a diagram illustrating an example of an image retrieved by the image search process.

The image search process is a process described as an example of an image search method using the imaging apparatus 100.
First, as shown in FIG. 5, an image search mode is selected and designated from among a plurality of operation modes based on a predetermined operation of the operation input unit 10 by the user (step S21). At this time, image search conditions such as searching for a plurality of persons to be searched using an AND condition or searching using an OR condition may be set.

Next, the CPU 12 causes the fingerprint sensor 6a of the fingerprint detection unit 6 to detect a fingerprint of a person to be searched (step S22). For example, as shown in FIG. 6, two persons (for example, Mr. A and Mr. B) who are to be searched place a finger whose fingerprint is registered in the fingerprint registration table T1 on the fingerprint sensor 6a. It should be noted that the act of placing the finger on the fingerprint sensor 6a may be performed at the same time, or if there is a certain time lag and the finger is placed at the same time for a predetermined time or more, it is determined that the act has been performed at the same time. Furthermore, even if a plurality of fingers are not placed at the same time on the fingerprint sensor 6a, if a plurality of fingers are placed continuously within a predetermined time, these fingers may be determined as search targets.
Thereby, the fingerprint sensor 6a detects the fingerprints of all the fingers placed on the fingerprint sensor 6a, and generates the respective fingerprint data (step S23).

Subsequently, the CPU 12 causes the fingerprint detection unit 6 to determine whether or not the detection of the fingerprints of all the fingers placed on the fingerprint sensor 6a has been completed (step S24). For example, when the sensor area of the fingerprint sensor 6a is divided into a plurality of areas and the fingerprint is detected for each area, the fingerprint detection unit 6 uses the number of areas where the fingerprint is detected and the number of generated fingerprint data. It is determined whether or not the detection of the fingerprints of all the fingers has been completed depending on whether or not.
Here, when it is determined that the fingerprints of all fingers have not been detected (step S24; NO), for example, the fingerprint of Mr. A's finger is detected, but the fingerprint of Mr. B's finger is still detected. If not, the CPU 12 shifts the process to step S22 and executes the subsequent processes.
When it is determined that the fingerprint detection of all fingers has been completed (step S24; YES), the CPU 12 sends a plurality of fingerprint data generated by the fingerprint sensor 6a to the fingerprint identification unit 6b of the fingerprint detection unit 6. It is specified as the fingerprint data for object designation (step S25).

  Next, the CPU 12 causes the automatic execution control unit 9b of the image search unit 9 to automatically execute image search processing. In other words, the automatic execution control unit 9b first causes the face image specifying unit 7 to match the fingerprint data specified as the target specifying fingerprint data among the plurality of fingerprint data registered in the fingerprint registration table T1. The data is specified, and the face image data associated with the fingerprint data is specified (step S26).

Thereafter, the automatic execution control unit 9b branches the process according to the search condition (step S27).
That is, when searching for a plurality of persons using the AND condition (step S27; all members), the automatic execution control unit 9b uses the determination unit 9a of the image search unit 9 to determine each of the plurality of images recorded on the recording medium 3. By determining whether or not all the faces related to all the face image data specified by the face image specifying unit 7 have been detected by the face detecting unit 2a, an image in which all persons to be searched are shown at the same time is obtained. Search is performed (step S28).
On the other hand, when searching for a plurality of persons under the OR condition (step S27; at least one person), the automatic execution control unit 9b stores a plurality of images recorded on the recording medium 3 in the determination unit 9a of the image search unit 9. For each of the above, at least one of the persons to be searched is determined by determining whether or not the face related to any one of the face image data specified by the face image specifying unit 7 is detected by the face detecting unit 2a. An image in which is shown is searched (step S29).

Then, the automatic execution control unit 9b causes the display unit 5 to display an image determined to have a face detected by the image search unit 9 as a search result (step S30).
Thereby, the image search process is terminated.

As described above, according to the imaging apparatus 100 of the present embodiment, face image data and fingerprint data are associated with each other and registered in the fingerprint registration DB 8 and generated by the fingerprint detection unit 6. Face image data associated with the fingerprint data and stored in the fingerprint registration DB 8 is specified by the face image specifying unit 7, and each of a plurality of images recorded on the recording medium 3 is detected by the face detecting unit 2 a. The determination unit 9a determines whether or not a face related to the image data has been detected, and the image determined to have detected the face is displayed as a search result on the display unit 5, so that the person who is the search target in the fingerprint detection unit 6 The image including the person's face can be searched and displayed by simply detecting the fingerprint of the person.
Thus, it is possible to search for a desired image without adding attribute information for identifying an individual to a captured image in advance, and without performing multiple input operations by key input or menu selection of an operation input unit. In both cases, it is possible to easily specify an individual to be searched.
Accordingly, it is possible to more easily search for an image desired by a person who performs image search.
In particular, since the fingerprint detection unit 6 is configured to be incorporated in the display unit 5, the person who performs the image search does not remove his line of sight from the display unit 5, and the person to be searched for is displayed on the display screen of the display unit 5. By simply placing a finger, it is possible to easily search for an image desired by the person. Furthermore, even if the search target is switched while browsing the image, it is not necessary to switch the screen display to the menu selection screen or to shoot a new face image for specifying the search target. It is possible to quickly and easily browse images without removing the line of sight from the image.

  The fingerprint detection unit 6 includes a fingerprint sensor 6a provided at a predetermined position of the main body of the imaging apparatus 100. When it is detected that a finger is placed on the fingerprint sensor 6a, the face image specifying unit 7 detects the face image. A process for specifying data, a process for determining whether the face is detected by the face detection unit 2a by the determination unit 9a, and an image in which the display unit 5 determines that a face is detected by the determination unit 9a are used as search results. Since the execution of the display process is sequentially controlled, the image search process can be automatically performed simply by placing a finger on the fingerprint sensor 6a, and the convenience of the imaging apparatus 100 can be improved. it can.

In addition, since all fingerprints detected simultaneously or within a predetermined time within a predetermined detection area of the fingerprint sensor 6a are specified as target specifying fingerprints for specifying a face to be searched, a plurality of people are specified. Even when there are a plurality of persons to be searched, such as when people gather together and appreciate a photo, the fingerprints of the fingers of these persons can be detected simultaneously or within a predetermined time by the fingerprint detection unit 6 Thus, it is possible to quickly perform an image search such as a group photo using the plurality of persons as search targets, and the convenience of the imaging apparatus 100 can be improved.
Further, the face image data associated with the fingerprint data of at least one fingerprint among the plurality of fingerprints specified as the target specifying fingerprint is specified, and at least one or all of the faces related to the face image data are specified. Since an image determined to be detected is output as a search result, a plurality of persons can be searched under an AND condition or an OR condition, the range of image search can be further expanded, and the convenience of the imaging apparatus 100 can be increased. Can be improved.
At this time, the designation of whether to search a plurality of persons using the AND condition or the OR condition is performed, for example, when a plurality of persons simultaneously place their fingers on the fingerprint sensor 6a to detect a fingerprint. On the other hand, when a finger is placed in order and a fingerprint is detected, the OR condition may be set.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, the display unit 511 is illustrated as an output unit, but the display unit 511 is not limited thereto, and may be an external output terminal (not shown) that can be connected to a TV apparatus or the like, and the TV apparatus is searched from the external output terminal. The resulting image data may be output and an image related to the image data may be displayed.

Moreover, in the said embodiment, although what was integrated in the display part 5 was illustrated as the fingerprint detection part 6, it is not restricted to this, You may provide in any position of the imaging device 100 main body.
Furthermore, although the surface sensor is illustrated as the fingerprint detection sensor 6a, the present invention is not limited to this, and for example, a line sensor that detects a fingerprint by moving the fingertip so as to be displaced may be used.

  Furthermore, in the above embodiment, the fingerprint of one finger of one person and the face image data are associated with each other. For example, for each of the thumb to little finger of a person, a front face, a right face, a left face, Image data such as a downward face and an upward face are associated and registered in the fingerprint registration table T1, and the search target is narrowed down according to the type of finger detected by the fingerprint detection unit 6 during image search. May be.

In addition, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto.
In addition, in the above-described embodiment, the functions of the fingerprint detection unit, the face image specifying unit, the face detection unit, the determination unit, the output control unit, and the control unit are controlled by the fingerprint detection unit 6 under the control of the CPU 12. The configuration is realized by driving the face image specifying unit 7, the face detection unit 2 a, the image search unit 9, the determination unit 9 a, and the automatic execution control unit 9 b, but the configuration is not limited to this, and the CPU 12 performs predetermined processing. A configuration realized by executing a program or the like may be employed.
That is, a program memory (not shown) that stores a program stores programs including a fingerprint detection control processing routine, a face image specifying processing routine, a face detection processing routine, a determination processing routine, an output control processing routine, and an automatic execution control processing routine. To do. Then, the fingerprint detection control processing routine causes the CPU 12 to function as fingerprint detection means for generating fingerprint data by causing the fingerprint sensor 6a to detect the fingerprint of the person's finger related to the image search. Further, the CPU 12 is caused to function as face image specifying means for specifying the face image data stored in the fingerprint registration DB 8 in association with the fingerprint data generated by the fingerprint detection unit 6 by the face image specifying processing routine. Further, the face detection processing routine functions as face detection means for detecting a face related to the face image data specified by the face image specifying means from the image. In addition, the determination processing routine causes each of a plurality of images recorded on the recording medium 3 to function as a determination unit that determines whether or not a face is detected by the face detection unit 2a. In addition, the CPU 12 is caused to function as output control means for outputting from the display unit 5 an image for which a face has been detected by the determination means by the output control processing routine as a search result. When the automatic execution control processing routine detects that the finger is placed on the fingerprint sensor 6a, the face image specifying means specifies the face image data, and the determination means detects the face by the face detecting means. The CPU 12 is caused to function as a control unit that sequentially controls execution of a process for determining whether or not a face has been detected by the determination unit and a process for outputting an image for which a face is detected by the determination unit as a search result.

DESCRIPTION OF SYMBOLS 100 Imaging device 2 Image processing part 2a Face detection part 5 Display part 6 Fingerprint detection part 6a Fingerprint sensor 6b Fingerprint specific part 7 Face image specific part 8 Fingerprint registration DB
9 Image search unit 9a Determination unit 9b Automatic execution control unit 12 CPU

Claims (8)

Recording means for recording a plurality of images;
Data storage means for storing face image data and fingerprint data in association with each other for each of a plurality of persons;
Fingerprint detection means for detecting a fingerprint of a finger and generating fingerprint data;
Fingerprint identification means for identifying the fingerprint detected by the fingerprint detection means as an object-designating fingerprint for designating a face to be searched;
Face image specifying means for specifying the face image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detection means corresponding to the target specifying fingerprint specified by the fingerprint specifying means When,
Face detecting means for detecting a face related to the face image data specified by the face image specifying means from an image;
For each of a plurality of images recorded in the recording means, determination means for determining whether or not a face has been detected by the face detection means,
An output means for outputting an image determined to have detected a face by the determination means as a search result;
A plurality of detection judging means for judging whether or not a plurality of fingerprints are detected simultaneously or within a predetermined time when a fingerprint is detected by the fingerprint detecting means;
With
The fingerprint specifying means specifies the detected plurality of fingerprints as the target specifying fingerprints when it is determined by the plurality of detection determining means that a plurality of fingerprints are detected simultaneously or within a predetermined time,
In the case where the target designating fingerprint includes a plurality of fingerprints, the determination unit determines, for each of a plurality of images recorded in the recording unit, a plurality of faces corresponding to the plurality of fingerprints by the face detection unit. Determining whether or not at least one of them has been detected, or determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection means;
In the case where the target designating fingerprint includes a plurality of fingerprints, the output means determines whether at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected by the determination means, or the determination An image search apparatus that outputs, as a search result, an image determined by the means that all of the plurality of faces corresponding to the plurality of fingerprints have been detected . The determination by the determination unit includes determination of whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection unit for each of the plurality of images recorded in the recording unit,
2. The output by the output unit includes outputting, as a search result, an image determined by the determination unit that all of a plurality of faces corresponding to the plurality of fingerprints have been detected. Image search device. In the determination by the determination unit, for each of the plurality of images recorded in the recording unit, whether or not at least one of the plurality of faces corresponding to the plurality of fingerprints is detected by the face detection unit is determined. Including judgment,
Claims wherein the output by the output means, characterized in that it comprises outputting the results determined image and at least one has been detected among the plurality of faces corresponding to the plurality of fingerprints by said determining means Item 3. The image search device according to Item 2. It further comprises search condition designating means for designating whether to search using an AND condition or an OR condition,
When the AND condition is specified by the search condition specifying unit, the output unit outputs, as a search result, an image determined by the determination unit that all the faces corresponding to the plurality of fingerprints have been detected. When the OR condition is specified by the search condition specifying means, an image determined by the determination means that at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected is used as a search result. The image search apparatus according to claim 3, wherein the image search apparatus outputs the image. The fingerprint detection means includes
A fingerprint sensor provided at a predetermined position of the apparatus main body,
The image search device according to claim 1 , further comprising finger detection means for detecting that a finger is placed on the fingerprint sensor. When the finger detection unit detects that a finger is placed on the fingerprint sensor, the face image specifying unit specifies the face image data, and the determination unit detects the face by the face detection unit. Control means for sequentially controlling execution of processing for determining whether or not the output means outputs a search result as an image for which the determination means has determined that a face has been detected. The image search apparatus according to claim 5 . An image search apparatus comprising: a recording unit that records a plurality of images; and a data storage unit that stores face image data and fingerprint data in association with each other for each of a plurality of persons.
Fingerprint detection processing for detecting fingerprints of fingers and generating fingerprint data;
A fingerprint identification process for identifying a fingerprint detected by the fingerprint detection process as a target-designating fingerprint for designating a face to be searched;
Face image specifying process for specifying the face image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detecting process corresponding to the target specifying fingerprint specified by the fingerprint specifying process When,
A face detection process for detecting a face related to the face image data identified by the face image identification process from an image;
Determination processing for determining whether or not a face has been detected by the face detection processing for each of a plurality of images recorded in the recording means;
An output process of outputting the search result image determined as a face has been detected by the determination process,
A multiple detection determination process for determining whether a plurality of fingerprints are detected simultaneously or within a predetermined time when a fingerprint is detected by the fingerprint detection process;
And execute
In the case where it is determined that the plurality of fingerprints are detected at the same time or within a predetermined time by the plurality of detection determination processes, the plurality of fingerprints that are detected are specified as the fingerprints for target specification.
In the determination process, when the target designating fingerprint includes a plurality of fingerprints, a plurality of faces corresponding to the plurality of fingerprints are detected by the face detection process for each of a plurality of images recorded in the recording unit. Determining whether or not at least one of them has been detected, or determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection process;
In the output process, when the target designating fingerprint includes a plurality of fingerprints, the determination process determines that at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected, or the determination An image search method, wherein an image determined to have detected all of a plurality of faces corresponding to the plurality of fingerprints by processing is output as a search result . A computer of an image search apparatus comprising: a recording unit that records a plurality of images; and a data storage unit that stores face image data and fingerprint data in association with each other for each of a plurality of persons.
Fingerprint detection means for detecting fingerprints of fingers and generating fingerprint data;
Fingerprint identification means for identifying the fingerprint detected by the fingerprint detection means as a fingerprint for object designation for designating a face to be searched;
Face image specifying means for specifying the face image data stored in the data storage means in association with the fingerprint data generated by the fingerprint detection means corresponding to the target specifying fingerprint specified by the fingerprint specifying means ,
Face detecting means for detecting a face related to the face image data specified by the face image specifying means from an image;
Wherein each of the plurality of images in the recording means are recorded, determining means for determining whether a face has been detected by the face detection means,
Output means for outputting an image determined to have detected a face by the determination means as a search result;
A plurality of detection judging means for judging whether or not a plurality of fingerprints are detected simultaneously or within a predetermined time when a fingerprint is detected by the fingerprint detecting means;
Function as
The fingerprint specifying means specifies the detected plurality of fingerprints as the target specifying fingerprints when it is determined by the plurality of detection determining means that a plurality of fingerprints are detected simultaneously or within a predetermined time,
In the case where the target designating fingerprint includes a plurality of fingerprints, the determination unit determines, for each of a plurality of images recorded in the recording unit, a plurality of faces corresponding to the plurality of fingerprints by the face detection unit. Determining whether or not at least one of them has been detected, or determining whether or not all of the plurality of faces corresponding to the plurality of fingerprints have been detected by the face detection means;
In the case where the target designating fingerprint includes a plurality of fingerprints, the output means determines whether at least one of a plurality of faces corresponding to the plurality of fingerprints has been detected by the determination means, or the determination A program for outputting, as a search result, an image determined by means to determine that a plurality of faces corresponding to the plurality of fingerprints have all been detected .

Images