JP5613304B2 - Image search apparatus and image search method - Google Patents

Description

  The present invention relates to an image search device and an image search method, and more particularly to an image search device and an image search method that can quickly and easily search an image from a large number of photographic images.

  In recent years, photographic images have been digitized, the capacity of recording media has increased, and the frequency of photography has increased compared to when silver film cameras were used. For this reason, the camera user owns a large amount of images, and it takes time to search for images.

  Therefore, various image search apparatuses that facilitate image search have been proposed. For example, Patent Document 1 discloses an electronic camera in which registered person face recognition data is registered in advance, and whether or not a person in a captured image is a registered person. Patent Document 2 discloses a digital still camera that replaces an icon of a silhouette image representing a person with an icon image created from a photographed image.

JP 2007-150604 A JP 2007-159152 A

  Although a technique for classifying similar images in this way is known, an image retrieval apparatus that retrieves classified images with a small number of operations on a display device such as a liquid crystal panel is not known.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image search apparatus and an image search method that can easily search for classified images on the display means with few operations. To do.

In order to achieve the above object, an image search apparatus according to a first invention includes an image recording unit that stores a plurality of image data, a display unit, and a face that detects a face part from image data included in the image recording unit. A detection unit; an angle determination unit that determines an angle at which a face portion detected by the face detection unit is facing; a thumbnail unit that records a plurality of face images; and a plurality of face images recorded by the thumbnail unit. a face image selected from among the similarity determination unit determining similarity between the detected face by the face detection unit, and displays on the display section a plurality of the facial image as in de box face image, a plurality part for the selected index face images from the index face images, by the similarity determination section, the face angle is determined to be similar facing of the above selected index face images and the face An image including, and having a display control unit for displaying on the display unit.

  In the image search device according to a second aspect of the present invention, in the first aspect, when the index face image is oriented at a different angle, the display unit selects a face portion that is determined to be similar in parallel. The image including it is displayed sequentially.

  The image search device according to a third invention is characterized in that, in the second invention, the display control unit further includes a morphing processing unit for performing a morphing process when a plurality of images are sequentially displayed.

According to a fourth aspect of the present invention, there is provided an image search method for detecting a face portion from image data of a plurality of images included in an image recording unit, and determining an angle at which the detected face portion is facing. An angle determination step, a similarity determination step for determining that the face image selected from the plurality of face images and the face detected by the face detection unit are similar, and indexing the plurality of face images A face image is displayed on the display unit, and an index face image selected from a plurality of the index face images is a face whose face is determined to be similar in angle to the selected index face image. And a display control step for displaying an image including the portion on the display unit.

  According to the present invention, it is possible to provide an image search device and an image search method capable of easily searching classified images with few operations on a display unit and further performing a search in consideration of the face direction. Can do.

It is a block diagram which shows the structure concerning the camera concerning one Embodiment of this invention, and its periphery. FIG. 6 is a diagram for describing recording of information related to the face of a subject at the time of shooting in an embodiment of the present invention. In one Embodiment of this invention, it is a figure explaining database imaging | photography. In one embodiment of the present invention, it is a figure showing the relation between the dynamic display of the index face image and the captured image displayed corresponding thereto, (a) (b) (c) is the index face image , (D), (e), and (f) are taken images. FIG. 10 is a diagram illustrating a technique for obtaining an index face image by performing morph processing on a face image in an embodiment of the present invention. It is a flowchart which shows the operation | movement of the camera control in the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the search reproduction | regeneration in the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the morphing moving image preparation in the camera concerning one Embodiment of this invention. In one Embodiment of this invention, it is a figure which shows the display state of the camera display part at the time of search reproduction | regeneration. In the modification of one Embodiment of this invention, it is a figure which shows the display state of the display part at the time of search reproduction | regeneration.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. FIG. 1 is a block diagram showing a configuration of a camera 10 according to an embodiment of the present invention. The camera 10 is a digital camera, and includes a control unit 1, a similarity determination unit 1a, a timing control unit 1b, an imaging unit 2, an optical system control unit 3, a recording unit 4, a feature (angle) data recording unit 4a, and a face database (DB). 4b, thumbnail section 4c, face detection section 5, angle determination section 5a, size determination section 5b, clipping section 6, moving image processing section 7, display section 8, display control section 8a, multi-image display section 8b, moving image display control Unit 8 c, clock unit 9, operation determination unit 11, six-axis sensor 12, and communication unit 14, and can be connected to the external server 20 via the Internet network 15.

  The control unit 1 is connected to each unit in the camera 10 and performs overall control of the camera 10 according to the operation of the operation member determined by the operation determination unit 11 and according to the timing control by the timing control unit 1b. The control unit 1 includes an image processing unit (not shown), and performs image compression processing and the like. Further, the control unit 1 has a similarity determination unit 1a, determines the similarity of a plurality of face images detected by the face detection unit 5 described later, and identifies who is the person shown in the photographed image. To do.

  The imaging unit 2 includes an optical system for forming a subject image, an imaging element that photoelectrically converts the subject image, and the like, and outputs image data. The optical system control unit 3 performs control such as focusing of the optical system in the imaging unit 2. The recording unit 4 records the image data compressed by the image processing unit in the control unit 1 as a captured image. When recording the captured image, the face detection unit 5 described later detects the angle of the face of the person image, and this angle is recorded in the feature (angle) data recording unit 4a in association with the captured image.

  The recording unit 4 includes a face database (DS) 4b and a thumbnail unit 4c. The face data DB 4b is a database that collects the face portions of a person image in a photographed image recorded in the recording unit 4, specifies a person for each photographed image, and includes a combination of an image and a person name. The thumbnail part 4c records the face image cut out by the cut-out part 6, and is used when displaying a list as a face index.

  The face detection unit 5 determines a face pattern and detects a face portion when a captured image includes a human image. Based on the detection result, exposure control, automatic focus adjustment (AF), and the like are performed, and when a person is specified, it is performed based on the face detected by the face detection unit 5. The face detection unit 5 includes an angle determination unit 5a and a size determination unit 5b. The angle determination unit 5 a determines the orientation (angle) of the face of the person detected by the face detection unit 5. The size determination unit 5b determines the size of the detected face portion.

  The cutout unit 6 cuts out image data corresponding to the face portion of the human image detected by the face detection unit 5. The face image data cut out here is recorded in the thumbnail portion 4c, and is also used in similarity determination performed in the similarity determination portion 1a. When the moving image processing unit 7 sequentially displays captured images of a person captured from various angles, the moving image processing unit 7 does not simply display the image as a still image, but reproduces a plurality of images little by little to reproduce the image. Image processing for display on the screen is performed.

  The display unit 8 is configured by a display device such as a liquid crystal monitor disposed on the back surface of the camera 10, and displays and reproduces image data recorded in the recording unit 4. In addition, the subject image is displayed instead of the optical viewfinder. Live view display for observation. In addition, a display when searching for a photographed image recorded in the recording unit 4 is also performed. The display control unit 8a controls the display unit 8 and includes a multi-image display unit 8b and a moving image display control unit 8c. The multi-image display unit 8b divides and displays the display unit of the display unit 8 when searching for an image. The moving image display control unit 8c performs control when displaying an image such as an index face image after moving image processing during image search.

  The clock unit 9 has a timekeeping function and a calendar function, and is used for acquiring date and time information at the time of shooting. The date and time information obtained here is recorded together when the captured image is recorded in the recording unit 4. The operation determination unit 11 determines an operation state of an operation member or the like of the camera 10 by the user. In accordance with this operation state, the control unit 1 controls the camera 10 as described above. The six-axis sensor 12 is a sensor that detects the movement of the camera 10 in and around the three-dimensional axial directions of the x-axis, y-axis, and z-axis. The communication unit 14 performs transmission and reception with devices such as the server 20 via the Internet network 15 and the like. Via this communication part 14, a picked-up image can be transmitted outside and an image etc. can be received from the outside.

  Next, the operation of the camera 10 according to this embodiment configured as described above will be described. In the present embodiment, information about the angle at which the person faces is recorded together with the photographed image, and when the face of the person image is displayed as a thumbnail, it is oriented in the same direction as the angle at which the face is facing. Images are displayed sequentially.

  The still image captured by the imaging unit 2 is subjected to image processing such as compression processing in the image processing unit in the control unit 1 and recorded as a captured image in the recording unit 4. The recorded captured image is reproduced and displayed on the display unit 8. This photographing control is performed by determining the user's camera operation by the operation determination unit 11 and controlling the control unit 1 according to the determination result.

  If necessary, the captured image can be transmitted to the external server 20 or the like via the communication unit 14 and the Internet network 15. Note that, by using a communication function, an external computer may perform a substitute process for part of the image processing control in the present embodiment.

  When a person is included in the photographed image, the face detection unit 5 determines a face pattern and detects a face part. Then, the detected face and the face recorded in the face database 4b in the recording unit 4 are compared by the similarity determining unit 1a to identify who the subject is. In addition, the shadow of the face and nose of the face is analyzed, and the angle determination unit 5a determines whether the face is facing left or right (detects the angle), and the captured image indicates “the angle at which the face is facing”. The information “who is the subject person” is added and recorded in the recording unit 4.

  Recording of information relating to the face of the subject at the time of shooting will be described with reference to FIG. When the person 100 is photographed, the similarity determination unit 1a can identify a personal name (herein referred to as “Mr. B”) by comparing the photographed image 100a with each image recorded in the face DB 4b. The orientation of the face of the captured image is determined by the angle determination unit 5a, and based on the determination result, the relationship between the captured image 100a and the subject and the face angle is recorded in the feature (angle) data recording unit 4a as a classification signal. The

  At this time, the size of the face is also determined by the size determination unit 5b. The determined face size data is used to correct the face size so that it can be easily compared with the face image recorded in the face DB 4b.

  Thus, in this embodiment, even if it is the same person, the angle which the face is facing is recorded on face DB4b, and it is databased. In creating the database, as shown in FIG. 3, a person 100 as a subject is photographed in advance by continuous shooting from various angles, and image data obtained from the continuous shooting is recorded in the face DB 4b. Also good. At the time of this database shooting, the movement of the camera 10 is acquired by the six-axis sensor 12, and based on this information, it is recorded in the face DB 4b together with the shot image as angle information regarding the orientation of the face.

  Further, the face image obtained by continuous shooting as shown in FIG. 3 can also be used as an index image for image search. That is, the face portion of the photographed image is cut out from the image data by the cut-out unit 6, is converted into a face index image, is recorded in the thumbnail portion 4c in the recording unit 4, and is used for image search. In addition to obtaining face images by continuous shooting as shown in FIG. 3, the face images of the same person obtained during normal shooting are arranged for each angle facing the face, and the moving image processing unit 7 Alternatively, the moving image processing may be performed as if the face is moving, and the thumbnail image may be used as a face index.

  The thumbnail image recorded in the thumbnail portion 4c is displayed as a thumbnail image 81 on the index portion 84 of the display portion 8, as shown in FIG. In the example shown in FIG. 1, a thumbnail image 81 in which individual face index images 83 are arranged vertically is displayed. When the user selects a subject (face index image 83) to view from the thumbnail images 81. The image 82 of the selected person facing in the same direction is searched from the feature (angle) data recording unit 4 a in the recording unit 4 and displayed on the search display unit 85.

  In the present embodiment, as described with reference to FIG. 2, each person image in the photographed image corresponds to which face of the index each time it is photographed and compared with the face index image stored in the face DB 4b. Moreover, it is determined which angle is classified, and the determination result is recorded in the feature data recording unit 4a. Therefore, by referring to this, it is possible to search for an image 82 that is the same person as the selected index face image 83 and faces the same direction.

  Further, the index face image 83 displayed as the thumbnail image 81 on the display unit 8 is moved in a moving manner from the right direction to the left direction or in the reverse direction by the moving image display control unit 8c, and according to the movement of the face of the index face image 83, A captured image 82 facing the same direction as the face is displayed. Therefore, as the face direction of the index face image 83 changes, the captured image 82 also switches. Such display control is performed by the display control unit 8a, the multi-image display unit 8b for screen division control in the display control unit 8a, and the moving image display control unit 8c.

  The moving image display of the index face image 83 in the thumbnail image 81 and the captured image 82 displayed corresponding to this will be described with reference to FIG. 4A, 4B, and 4C are images of Mr. A taken by the method described in FIG. 3, FIG. 4A shows a negative face angle, and FIG. 4B shows a face angle. 4 is a case where the face angle is positive.

  Every time the face orientation of the index face image 83 changes, a photographed image 82 of the same person and facing the same direction is searched, and the photographed image 82 as shown in the right column of FIG. 4 is displayed. The That is, when the index face image 83 changes from FIG. 4 (a) → (b) → (c) → (b) → (a), the search condition changes accordingly, and the feature (angle) data recording unit The photographed image 82 is selected from 4a, and the photographed image 82 to be displayed changes in the order of (d) → (e) → (f) → (e) → (d) in FIG.

  The good points of the index face image 83 created in this way are that one person can be photographed from many angles and the characteristics of that person can be grasped, and this image itself is shown in FIGS. 4 (a) → (b). → (c) → (b) → (a) The index display is changed to make it easier to draw the user's attention. In addition, since the corresponding captured image 82 is displayed in accordance with the moving image display of the index face image 83, the display image is narrowed down without displaying many images at once. Can be organized.

  The display speed of the index face image with movement may be selectable by the user or may be changed for each operation. Further, even if shooting is not performed for an index image as shown in FIG. 3, face images of the same person are collected from many images, classified by angle of face orientation, and these face images are morphed. Then, as shown in FIG. 5, an index face image 83 may be obtained. Using the index face image 83 subjected to the morphing process, the image of the photographed image 82 is switched as shown in FIG. 4 in the same manner as described with reference to FIG.

  Here, the display is performed for each face angle of a specific individual. However, if the morphing process as shown in FIG. 5 is assumed, images of an unspecified number of persons are classified according to the angle of the face direction. The index face image may be changed, and images corresponding to the angle may be sequentially displayed. In addition, the index display processing is performed in a movie-like manner by changing the angle of the face orientation. However, the size of the face, the orientation angle of the whole body, or a change in facial expression may be processed in an animated manner to be indexed. . Even in this case, it is the same to retrieve and display a similar image corresponding to each frame of the index image that changes like a moving image.

  Thus, in the present embodiment, a specific person image can be searched at high speed. Also, because the searched image changes depending on the orientation of the face, even if the same person is photographed in large numbers like a family photo, the target image can be obtained by setting conditions such as the orientation of the face for that person. You can search easily. If you find an image you like, you can zoom in on the full screen and enjoy it by selecting that image.

  Next, the operation of the camera 10 capable of performing an image search using the index face image 83 that has been thumbnailed as described above will be described using the flowchart shown in FIG. First, a determination is made as to whether or not the photographer has set the normal shooting mode and has performed a full press operation of the release button, that is, whether or not to perform normal shooting (S1).

  As a result of the determination, when normal shooting is executed, shooting is performed by the imaging unit 2 and image data of a still image is acquired (S2). Thereafter, in steps S3 to S6, as described with reference to FIG. 2, a person is specified and feature (angle) data is recorded. First, based on the acquired image data, the face detection unit 5 performs face detection, and the similarity determination unit 1a determines whether there is a similar index face image (S3). In this determination, the angle determination unit 5a also detects the angle at which the face is facing. Subsequently, it is determined whether there is a similar face as a result of the face determination (S4).

  If the result of determination in step S4 is that there are no similar faces, the photographed image is recorded in the recording unit 4 (S6). On the other hand, when there is a similar face, similar face data, that is, data for linking with the index face image of the same person stored in the face DB 4b, the angle (feature data) that the face is facing, and the image Is recorded in the recording unit 4 (S5). When the recording to the recording unit 4 ends, the process returns and returns to step S1.

  If the result of determination in step S <b> 1 is not normal shooting, it is determined whether or not database shooting is performed (S <b> 11). The database photographing is photographing for acquiring the index face image 83 by continuously photographing a person as a subject from various angles, for example, from right to left, as described with reference to FIG. In this step, based on the output of the operation determination unit 11, it is determined whether or not this shooting mode has been set.

  If database shooting is set as a result of the determination in step S11, image data is acquired by the imaging unit 2 (S12), and face determination is performed by the face detection unit 5 or the similarity determination unit 1a (S13). Subsequently, based on the output of the six-axis sensor 12, it is determined whether or not the angle in the shooting direction of the camera 10 has changed (S14). If the angle changes as a result of the determination, the process returns to step S12 for each angle change, and photographing is performed.

  If there is no angle change as a result of the determination in step S14, it is determined whether the camera 10 has moved by a predetermined angle (S15). As a result of the determination, if there is a movement by a predetermined angle, the face part is cut out from the image data so that it can be displayed as if only the face part is moved, and the face part is organized for each angle and recorded in the recording unit 4. Thereby, an index face image as shown in FIG. 4 is recorded. On the other hand, if there is no movement by a predetermined angle, a warning is displayed (S17). When steps S16 and S17 are completed, the process returns and returns to step S1. Note that the predetermined angle only needs to be such that it can be displayed as an index face image, for example, an angle of about 15 degrees.

  If the result of determination in step S11 is not database shooting, it is determined whether or not search reproduction is in effect (S21). In the present embodiment, there are two playback modes: selective playback and search playback. If the result of determination is search playback, search playback is performed in step S22. If not, selective playback is performed in step S23.

  The selective reproduction is a normal reproduction mode, in which image data recorded in the recording unit 4 is sequentially reproduced or displayed as a thumbnail, and the selected image is enlarged and displayed. Further, the search reproduction is performed in the same direction while changing the face direction of the index face image displayed as a thumbnail on the display unit 8 shown in FIG. 1 (FIGS. 4A, 4B, and 4C). This is a mode for displaying a captured image (FIGS. 4D, 4E, and 4F). When the search reproduction or selective reproduction ends, the process returns and returns to step S1.

  Next, the search reproduction in step S22 will be described using the flowchart shown in FIG. In this flow, first, a sample image, that is, a thumbnail image 81 is displayed on the index portion 84 of the display portion 8 (S31). Subsequently, it is determined whether or not the user has selected the index face image 83 that the user wants to see (S32). If the selected index face image 83 is selected as a result of the determination, the selected index face image 83 changes cyclically as shown in FIGS. 4 (a) → (b) → (c).

  At this time, the index face image 83 which has not been selected remains stationary. By converting the selected index face image 83 from the still image state to the moving image state, the user can grasp that the selected index face image 83 is selected and can also enjoy watching a moving image.

  Subsequently, it is searched and determined whether or not there is a photographed image 82 that has a face similar to the index face image 83 displayed as a moving image and a face facing the same direction (S34). If the photographed image 82 is retrieved as a result of the determination, the photographed image 82 is displayed (S35). As a result, the orientation of the face of the index face image 83 is sequentially changed, and the photographed images 82 in which the faces of the same person facing the same direction are sequentially displayed in conjunction with this orientation.

  Next, it is determined whether or not the photographed image 82 has been selected by the user (S36). The captured image 82 is displayed as a reduced image in step S35. When the user desires an enlarged display in order to view the captured image 82, the captured image 82 is selected. As a result of the determination, if it is not selected, the process jumps to step S38, and if it is selected, the image is enlarged and displayed on the display screen of the display unit 8 (S37). Subsequently, it is determined whether or not the search display is to be ended. If it is to be ended, the process returns to the original routine. If it is not to be ended, the process returns to step S32.

  If the result of determination in step S32 is that the index face image 83 has not been selected from the thumbnail images 81, the sample list, ie, the thumbnail image 81 is switched (S41). If all the recorded thumbnail images 81 cannot be displayed at the same time, a part of them is displayed on the display unit 8. In this step, the next priority thumbnail image 81 is displayed.

  If the result of determination in step S41 is that there is no switching, processing returns to step S32. On the other hand, if switching has been made, the next sample list, ie, the thumbnail image 81 is displayed (S41), and jumps to step S38. To do. When the search reproduction is continued, the process returns from step S38 to step S32, and the search and display are performed in step S33 and subsequent steps.

  In the flowchart of FIG. 6 described above, database shooting was performed from step S12 to step S16. However, creation of an index face image using a morphing video instead will be described with reference to FIG. This creation method can eliminate the photographing method shown in FIG. 3 and the special photographing described in step S12 and subsequent steps in FIG.

  When the morphing moving image creation flowchart is entered and photographing is performed in the same manner as in step S2, it is determined whether or not the person shown in the photographed image is the same person (S51). That is, it is determined whether or not the index face image is already recorded in the recording unit 4. Subsequently, as a result of this determination, it is determined whether it is the same as the index face image already recorded, that is, the same as the sample (S52). As a result of the determination, if there is an identical face image, a morphing moving image is created in step S53 and subsequent steps.

  First, the angle of the face image is determined (S53). In this angle determination, the angle determination unit 5a determines the angle in the direction in which the face is facing from the position of the eyes of the face image, the balance of the position of the nose, and the like. Subsequently, the face image is extracted by the cutout unit 6 (S54), and the relationship between the person and the angle is recorded (S55). Next, a plurality of recorded face images are rearranged for each angle, and an intermediate image between the previous image and the next image is created as shown in FIG. 5, thereby creating a morphing video (S56). ). At this time, in order to make the size of the face uniform, in step S44, the size determination unit 5b makes the faces substantially the same size.

  For example, when creating a morphing video from images as shown in FIGS. 4D, 4E, and 4F, the face is small in FIG. 4D, and the face is large in FIG. If the faces are different in size, use the image enlargement or reduction process to cut out the images so that the faces are the same size, and align them so that the faces are the same size. Yes.

  As a result of the determination in step S52, if there is no same person in the sample, that is, if it is not similar to the index face image 83 already recorded in the recording unit 4, there is no image of the same person, so morphing The index face image 83 based on moving images cannot be created. In this case, steps S61 and the subsequent steps are executed so that a morphing video can be created when the second frame is shot for the same person.

  First, angle determination is performed in the same manner as in step S53 (S61), and the size is adjusted so that the face image is extracted and displayed to a predetermined size (S62). Subsequently, as in step S55, the relationship between the person and the angle is recorded (S64), and a new sample, that is, an index face image is recorded (S64). As described above, in order to perform the morphing moving image, it is only necessary to shoot an image of the same person, and it is not necessary to perform a special shooting operation as in FIG.

  Next, transition of the display screen of the display unit 8 when the user performs an image search operation will be described with reference to FIG. First, when the search is started, the display screen 8-1 is displayed on the display unit 8, and the index face image 83 is displayed for each person as the thumbnail image 81 on the index unit (first search unit) 84.

  When the user operates the operation member of the camera 10 and selects the index face image 83 of Mr. A, a captured image 82 in which Mr. A is reflected is displayed on the search display unit 85. Then, the screen changes to the display screen 8-2, and the face direction of the index face image 83 of Mr. A displayed on the index part (second search part) 84 changes. A photographed image 82 retrieved corresponding to the change in the orientation of Mr. A's face is displayed on the retrieval display unit 85.

  Next, when the user selects Mr. B's index face image 83, the screen is switched to the display screen 8-3, the photographed image 82 of Mr. A displayed on the search display unit 85 disappears, and Mr. B's index face image 83. The face changes like a video. In the search display section 85, a photographed image 82 having the same angle as the face direction of Mr. B's index face image 83 is searched and displayed.

  As described above, when searching for an image, the camera 10 according to the present embodiment not only searches for an individual person's image, but also narrows down the image in consideration of the movement of the individual (specifically, the orientation of the face). It is carried out. Further, since the index face image 83 is displayed as a moving image, it is easy to understand when selecting, and the moving image itself can be enjoyed.

  In the above description, an example in which a search is performed based only on the orientation of a person's face has been shown. However, as a modification of the present embodiment, a search may be performed using a theme such as fish or flowers as shown in FIG. In addition, instead of performing everything in the camera 10, an image search may be performed in a personal computer (PC) 21 connected to the Internet in cooperation with the external server 20 via the communication unit 14.

  With the face detection technique, a computer incorporated in the camera 10 can sufficiently perform the determination process. However, depending on the subject, a configuration in which processing is performed by the external server 20 or the PC 21 may be adopted.

  In other words, depending on the subject, not only partial extraction with a simple pattern but also a more advanced technique for extracting the main subject is necessary, and it is better to use a technique for separating the background and the main subject by the movement of the subject. For example, there is a background separation method that processes a moving object as a main subject and a non-moving object as a background. Of those extracted by this process, the one in the center of the screen may be converted into a theme image (index image) as a representative example.

  In the modification shown in FIG. 10, themes such as “mas” and “bara” are displayed as an index image 93 in the index portion 84 of the display screen 8-1 so that they can be selected. If these theme objects move, background separation is possible. When one of the index images 93 is selected, the main subject image (in the example of FIG. 10) moves around as shown in the display screen 8-1 → display screen 8-2. Then, captured images 82 similar to the main subject image are sequentially displayed on the search display unit 85.

  Next, when an index image 93 of another theme is selected (in the example of FIG. 10, a wild rose), the theme object is reproduced and displayed as a moving image (a state in which a flower sways in the wind on the display screen 8-3). Such). Then, a photographed image 82 (the same direction and the like) similar to the theme object is displayed on the search display unit 85.

  In this way, even if the theme is the same, the image you want to see may change depending on its movement and composition. Especially when you select many images and create one album, depending on which and which are combined. The whole atmosphere changes. Therefore, there is a request to select the best-looking image from among many images. In this embodiment, the index image is moved in a moving image format, and the moment of movement indicated by the moving image is displayed. The captured still image is selected. Therefore, it is possible to select a still image having a similar aspect while enjoying a moving image and at an effective moment. In this way, to search various images, by adopting video display as an index, one theme can be handled on the same screen from various perspectives. Narrowing becomes easy.

  As described above, in the present embodiment, the index images are sequentially displayed in a moving image, and the photographed image corresponding to the moving image change of the index image is retrieved and displayed. For this reason, the classified images such as the orientation (angle) of the face can be easily searched on the display unit 8 with few operations.

  In the present embodiment, the angle of the face is recorded in the recording unit 4 together with the image data based on the face portion detected from the image data at the time of shooting. For this reason, when searching for an image, it is possible to search and display using the angle of the face, which is very convenient. Further, an image similar to the face portion of the photographed image is searched from the images recorded in the recording unit 4, and image data including this information is recorded. For this reason, it is possible to speed up the image search.

  In the present embodiment, the index unit 84 that displays the thumbnail image 81 and the search display unit 85 that displays the captured image 82 are displayed on the same screen. However, the present invention is not limited to this. Of course, it doesn't matter. In addition, the face portion of the photographed image 82 is detected and angle determination, size determination, and similarity determination are performed. However, the present invention is not limited to the face. That's fine.

  In the embodiment of the present invention, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be a mobile phone or a portable information terminal ( Of course, a camera built in PDA (Personal Digital Assist) or the like may be used. Dynamic display means displaying multiple still images in sequence, displaying them as if they were moving images, generating intermediate images from multiple still images by image processing, and displaying these images sequentially. By doing so, it is included to display like a moving image.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 ... Control part, 1a ... Similarity determination part, 1b ... Timing control part, 2 ... Imaging part, 3 ... Optical system control part, 4 ... Recording part, 4a ... Feature (angle) data recording unit, 4b ... face database (DB), 4c ... thumbnail part, 5 ... face detection unit, 5a ... angle determination unit, 5b ... size determination unit, 6... Cutout section, 7... Video processing section, 8... Display section, 8 a... Display control section, 8 b. ... Clock part, 11 ... Operation determination part, 12 ... Six-axis sensor, 14 ... Communication part, 15 ... Internet network, 20 ... External server, 81 ... Thumbnail image, 82 ... taken image, 83 ... index face image, 84 ... index part, 85 ... search display part, 93. Index image, 8-1 ... display screen, 8-2 ... display screen, 8-3 ... display screen

Claims (4)

An image recording unit for storing a plurality of image data;
A display unit;
A face detection unit for detecting a face portion from the image data included in the image recording unit;
An angle determination unit that determines an angle at which the face portion detected by the face detection unit is facing;
A thumbnail section that records multiple face images;
A similarity determination unit that determines similarity between a face image selected from the plurality of face images recorded in the thumbnail portion and the face detected by the face detection unit;
A plurality of said face images as in de box face image and displays on the display unit, the selected index face images from the plurality of the index face images, by the similarity determination section, the selected index face images A display control unit that causes the display unit to display an image including a part of the face that is determined to have a similar face angle.
An image search apparatus comprising:   The image according to claim 1, wherein the display unit sequentially displays images including face portions determined to be similar in parallel when the index face images face different angles. Search device.   The image search apparatus according to claim 2, wherein the display control unit further includes a morphing processing unit that performs morphing processing when sequentially displaying a plurality of images. A face detection step of detecting a face portion from each image data of a plurality of images included in the image recording unit;
An angle determination step for determining an angle at which the detected face portion is facing;
A similarity determination step for determining that the face image selected from the plurality of face images is similar to the face detected by the face detection unit;
The plurality of face images are displayed as index face images on the display unit, and the index face image selected from the plurality of index face images has an angle that the face is facing to the selected index face image. A display control step of causing the display unit to display an image including a face portion determined to be similar; and
An image search method characterized by comprising: