JP2012075020A - Image processing device, imaging device, face image recognition method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an imaging device, a face image recognition method and program, capable of continuing face recognition even in such a case in which no person of a face recognition object is imaged at all in an in-frame image.SOLUTION: The image processing device includes: in-frame image acquisition means for acquiring an in-frame image of the imaging device; face image recognition means for recognizing the face image of the person of the face recognition object and a related person in regard to the in-frame image, based on the face image of the object person of the face recognition object registered in advance, and the face image of the related person linked to the face image of the object person; and when the face image recognition of the object person fails and the face image recognition of the related person is successful, control means for controlling the face image recognition means to continue the face image recognition of the object person for a prescribed time duration.

Description

  The present invention relates to an image processing apparatus, an imaging apparatus, a face image recognition method, and a program for recognizing a face image of a specific person in an image in a frame.

  For example, in an imaging apparatus such as a digital still camera, when recognizing a face image of a specific person A (person who is a target of face recognition) in an image in a frame, a feature point (for example, the face image of the person A facing front) The face of the person A is recognized by extracting the eyes, nose, mouth, etc., but if the person A is not facing the front, the feature points cannot be extracted and the face cannot be recognized. .

  As a technique for solving this problem, for example, Patent Documents 1 and 2 disclose a face recognition method when the person A is not facing the front. Patent Document 1 discloses a method of determining that the face of the person A is sideways or backwards from the color ratio of the hair area and the skin area, and continuing the recognition as the face of the person A as it is. Patent Document 2 discloses a method of estimating the location of the current face area based on the movement of the face area and the body area and continuing the recognition as the face of the person A as it is.

JP 2008-263424 A JP 2009-81714 A

  However, in both of the techniques of Patent Documents 1 and 2, it is necessary that a part of the body of the person A (for example, a laterally or backwardly facing face, hair, torso, etc.) is reflected in the image in the frame. Therefore, in these techniques, when the body of the person A is not shown at all in the image in the frame, or when a part of the body of the person A is not able to perform face recognition, the face recognition is performed. Is not continued and face recognition is stopped.

  The present invention has been made in view of the above problems, and in the image in the frame, when a person who is a target of face recognition is not shown at all, or a part of the body of the person who is shown can perform face recognition. An object of the present invention is to provide an image processing apparatus, an imaging apparatus, a face image recognition method, and a program capable of continuously performing face recognition even if they are not present.

  In order to achieve such an object, an image processing apparatus of the present invention includes an in-frame image acquisition unit that acquires an in-frame image of an imaging apparatus, a face image of a target person that is a target of face recognition registered in advance, Face image recognition means for recognizing the face image of the target person and the face image of the related person with respect to the in-frame image based on the face image of the related person linked to the face image of the target person; Control means for controlling the face image recognition means so that recognition of the face image of the target person is continued for a predetermined time when the face image cannot be recognized and the face image of the related person can be recognized. It is characterized by.

  An imaging apparatus according to the present invention includes the image processing apparatus according to the present invention.

  According to the face image recognition method of the present invention, a step of acquiring an intra-frame image of an imaging device, a face image of a target person that is a target of face recognition registered in advance, and a relationship associated with the face image of the target person The step of recognizing the face image of the target person and the face image of the related person with respect to the in-frame image based on the face image of the person, and the face image of the target person cannot be recognized. A step of continuing the recognition of the face image of the target person for a predetermined time when the recognition is successful.

  The program according to the present invention includes a process for acquiring an in-frame image of an imaging device, a face image of a target person that is registered in advance, and a face of a related person linked to the face image of the target person. Based on the image, the process of recognizing the face image of the target person and the face image of the related person with respect to the image in the frame, and the face image of the target person cannot be recognized, and the face image of the related person In this case, the computer is caused to execute processing for continuing recognition of the face image of the target person for a predetermined time.

  According to the present invention, face recognition can be performed even when a person to be face-recognized is not shown at all in the image in the frame, or when a part of the body of the person being shown cannot perform face recognition. Can be continued.

It is a block diagram showing an example of composition of an imaging device concerning one embodiment of the present invention. It is a flowchart which shows the operation example of the preparation operation | movement 1 of the imaging device which concerns on one Embodiment of this invention. It is a flowchart which shows the operation example of the preparation operation 2 of the imaging device which concerns on one Embodiment of this invention. It is a flowchart which shows the operation example of the face recognition operation | movement of the imaging device which concerns on one Embodiment of this invention. It is a block diagram which shows the structural example of the image processing apparatus which concerns on one Embodiment of this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments (embodiments) for carrying out the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration example of a digital still camera (hereinafter referred to as a camera) as an embodiment of an imaging apparatus of the present invention. As shown in FIG. 1, the camera of this embodiment includes an optical system 1, an image sensor 2, an AFE (Analog Front End) 3, a memory 4, a liquid crystal panel 5, an image processing unit 6, and an input unit 7. The image processing unit 6 is an embodiment of the image processing apparatus of the present invention.

  The optical system 1 includes a photographic lens and the like, and forms an image of an incident subject on the image sensor 2. The imaging device 2 is composed of a CCD (Charge Coupled Device) or a CMOS (Complementary MOC), and changes an image formed by the optical system 1 into an analog signal by photoelectric conversion. The AFE 3 converts the analog signal converted by the image sensor 2 into a digital signal by A / D conversion, and outputs the digital signal to the image processing unit 6 as image data. The image processing unit 6 outputs the image data input from the AFE 3 to the liquid crystal panel 5 (also serving as a finder) as an in-frame image. Thereby, the photographer can visually recognize the in-frame image. Further, the image processing unit 6 has a function of performing face recognition on an intra-frame image displayed on the liquid crystal panel 5 based on face image data registered in advance. Since this face recognition function can apply a conventionally known technique, the description thereof is omitted here. The memory 4 is a storage medium built in the camera or detachable from the camera. The memory 4 is photograph data taken in the past (still image data obtained from an in-frame image) or a face image for use in face recognition. Store data etc. The liquid crystal panel 5 displays the intra-frame image output by the image processing unit 6 and the photo data and face image data read from the memory 4 by the image processing unit 6 and output. Thereby, the photographer can visually recognize the in-frame image, the photo data, and the face image data. The input unit 6 is a button or key for receiving input of various operations (instructions) by the photographer. The input unit 7 may be provided independently of the liquid crystal panel 5 or may be provided together with the liquid crystal panel 5 like a touch panel. Although not shown in FIG. 1, the camera also includes a timer unit (clock unit) that measures time.

  Next, the operation of the camera according to the present embodiment will be described with reference to FIGS. In the camera of the present embodiment, prior preparation operations (FIGS. 2 and 3) are required before the face recognition is performed (FIG. 4).

  First, the preparatory operation 1 will be described with reference to FIG. The preparatory operation 1 is an operation when a user (a person who uses a camera) registers a face image of a person who is a target of face recognition (hereinafter referred to as a target person) in the camera.

  First, the input unit 7 accepts designation of a face image of the target person A in predetermined photo data (S1). The photograph data here may be, for example, one selected by the user from the photograph data photographed in the past stored in the memory 4, or the user actually photographed the target person A. It may be the photo data immediately after. In any case, this photographic data is displayed on the liquid crystal panel 5, and the user designates an image of the face portion (face image area) from the image of the target person A shown in the displayed photographic data. As a designation method, for example, there is a method of marking a face image by using a marking function (known technique) mounted on the camera.

  Next, when executing the face recognition in FIG. 4 described later, the input unit 7 continues (continues) the recognition of the face image of the target person A specified in S1 in a predetermined intra-frame image ( Hereinafter, the designation of the recognition duration is accepted. This recognition duration time is preferably, for example, several seconds. The recognition duration is set to a default value (for example, 3 seconds) in the memory 4, and the user uses the default value in S2 or sets it to a desired value (for example, 5 seconds). Specify whether to change.

  Note that when the face image is designated in S1 and the recognition duration time is designated in S2, a screen and a message for guiding acceptance of the designation are displayed on the liquid crystal panel 5. The user inputs a desired designation by operating the input unit 7 while viewing the screen or message.

  Next, the image processing unit 6 associates the face image of the target person A specified in S <b> 1 with the recognition duration time specified in S <b> 2 and assigns a predetermined ID (identification information) to the memory 4. Store (S3). When the storage in the memory 4 is completed, a message indicating that the registration of the face image of the target person has been completed may be displayed on the liquid crystal panel 5 or may be output as audio from a speaker (not shown). Both can be output).

  As described above, the registration of the target person's face image as the preparation operation 1 is completed. Note that the user can register face images of a plurality of different target persons by repeating S1 to S3.

  Next, the preparatory operation 2 will be described with reference to FIG. The preparation operation 2 is a person related to the target person registered in the preparation operation 1 (a person who has been shown in a photo together with the target person and is a person different from the target person, hereinafter referred to as a related person). This is the operation when registering the face image of the camera to the camera.

  First, the input unit 7 accepts designation of a face image of a target person desired by the user from the face images of the target person registered in advance (S11). For example, when the input unit 7 receives an instruction to register a related person from the user, the image processing unit 6 reads a list of face images of the target person registered in advance from the memory 4 and displays the list on the liquid crystal panel 5. The user looks at the list and designates (selects) a face image of a desired target person. That is, the designation of the face image here is not “designating the area of the face image from the image (image of the target person's body) shown in the photographic data” in S1 of the preparation operation 1. This means “designating a desired face image itself from a list of face images registered in advance”. Here, it is assumed that the face image of the target person A is designated by the user.

  Next, the image processing unit 6 searches the predetermined photograph data for an image including the face image of the target person A designated in S11 and the face image of at least one related person (S12). The predetermined photograph data here is, for example, photograph data taken in the past stored in the memory 4. Therefore, for example, when 100 pieces of photographic data are stored in the memory 4, the image processing unit 6 selects the face image of the target person A and the face image of at least one related person from the 100 pieces of photographic data. Search all photo data including

  Note that the photo data that includes the face image of the related person at the time of the search in S2, but the face image does not exist around the face image of the target person A (within a preset range). May be excluded. In other words, only the photograph data in which the face image of the related person is located in the vicinity of the face image of the target person A is used as the final search result. The reason for this is that a person who is photographed in the vicinity of the target person A is assumed to be highly related to the target person A.

  If there are a plurality of pieces of photo data stored in the memory 4 and a plurality of pieces of photo data are searched at the time of the search in S2, one photo data having the latest shooting date and time is obtained as a final search result. It is good. The reason for this is that a person who has recently been photographed together with the target person A is assumed to be highly related to the target person A.

  Next, the image processing unit 6 recognizes face images of all related persons included in the retrieved photographic data (S13). Here, as an example, it is assumed that face images of three related persons B, C, and D are recognized.

  Next, the image processing unit 6 associates an ID assigned to the face image of the target person A with each of the recognized face images of the related persons B, C, and D, and a predetermined ID (identification information). Is stored in the memory 4 (S14). When the storage in the memory 4 is completed, a message indicating that the registration of the face image of the related person is completed may be displayed on the liquid crystal panel 5 or may be output as sound from a speaker (not shown). Both can be output).

  As described above, the registration of the related person's face image as the preparation operation 2 is completed. In the description of FIG. 3, the camera (image processing unit 6) dynamically searches for photo data including a face image of a related person and recognizes the face image of the related person. Image registration is not limited to this. For example, when the user can grasp or guess the related person of the target person, the user can manually register the face image of the related person. In this case, in FIG. 3, after S11, instead of S12 and S13, the face image of the related person is designated by the user in the same manner as S1 in FIG. 2, and finally S14 is performed.

  In the description of FIG. 3, the ID assigned to the face image of the target person A is associated with the face images of the related persons B, C, and D in S14. The face image of D may be directly linked to the face image of the target person A. That is, in addition to the ID registered in the preparatory operation 1, the face image of the target person A, and the recognition duration time, the face images of the related persons B, C, and D may be associated and registered.

  Next, the face recognition execution operation will be described with reference to FIG. Here, an example in which a user (photographer) holds a camera for a plurality of persons and recognizes a face image of the target person A when shooting a still image. The face image of the target person A is registered in advance in the preparation operation 1, and the face images of the related persons B, C, and D related to the target person A are also registered in advance in the preparation operation 2. Suppose that

  The user selects (specifies) a face image to be recognized by the input unit 7 in the same manner as S11 in FIG. Here, it is assumed that the face image of the target person A is selected. By this selection, the face images of the related persons B, C, and D associated with the face image of the target person A are also recognized.

  When the user holds the camera at a place where a plurality of persons are present, the image processing unit 6 causes the liquid crystal panel 5 to display the image in the frame (S21). As a result, the user can visually recognize an image to be taken (an image ahead of the camera). At this time, the user does not move the camera.

  The image processing unit 6 starts recognizing the face image of the target person A and the face images of the related persons B, C, and D in the in-frame image displayed on the liquid crystal panel 5 (S22). This face recognition is performed using a known face recognition technique based on the face images of the target person A and related persons B, C, and D registered in the memory 4.

  If the face image of the target person A can be recognized after S22 (S23 / YES), the image processing unit 6 causes at least one of the liquid crystal panel 5 and a speaker (not shown) to notify the user to that effect. (S32). That is, at least one of screen display and sound output indicating that the face image of the target person A has been recognized is performed. At this time, the image processing unit 6 displays a display that allows the user to visually recognize the position of the recognized face image of the target person A (for example, a display that puts a mark on the face image of the target person A). 5 may be performed. The case where the image processing unit 6 can recognize the face image of the target person A is a case where the face image (or an image of a part of the body) of the target person A is reflected in the in-frame image.

  After S22, when neither the face image of the target person A nor the face images of the related persons B, C, D can be recognized (S23 / NO, S24 / NO), the image processing unit 6 notifies the user to that effect. At least one of the liquid crystal panel 5 and a speaker (not shown) is notified (S29). That is, at least one of a screen display and an audio output indicating that the face image of the target person A and the face images of the related persons B, C, and D cannot be recognized is performed. S29 and subsequent steps will be described later. Note that when the image processing unit 6 cannot recognize both the face image of the target person A and the face images of the related persons B, C, and D, all of the target person A and the related persons B, C, and D are outside the frame. Or is hidden behind an object or person and is not reflected in the frame image at all, or a part of the body is reflected in the frame image. This is a case where the face of the target person A or the related person B, C, or D is not recognized even if known techniques such as Documents 1 and 2 are used.

  After S22, the face image of the target person A cannot be recognized, but if any of the face images of the related persons B, C, D can be recognized (S23 / NO, S24 / YES), the image processing unit 6 A notification to that effect is sent to at least one of the liquid crystal panel 5 and a speaker (not shown) (S25). That is, at least one of the screen display and the sound output indicating which face image of the related person B, C, and D can be recognized and indicating that the recognition of the face image of the target person A is continued is performed. It will be. Note that the image processing unit 6 cannot recognize the face image of the target person A, but if any of the face images of the related person B, C, or D is recognized, it means any of the related person B, C, or D. Although the face image (or the image of a part of the body) is reflected in the image in the frame and any of the related persons B, C, D can be recognized, the target person A exists outside the frame. Or, it is hidden behind an object or a person and is not reflected in the image in the frame at all, or a part of the body of the target person A is reflected in the image in the frame. This is a case where it is not recognized as the face of the target person A even if a known technique such as 1 or 2 is used.

  After S25, the image processing unit 6 continues to recognize the face image of the target person A in the in-frame image displayed on the liquid crystal panel 5 (S26). That is, conventionally, when the target person A is not shown in the image within the frame or when the face image cannot be recognized (S23 / NO), the recognition of the face image of the target person A has been stopped. However, in this embodiment, if any of the related persons B, C, D is recognized (S23 / NO, S24 / YES), the recognition of the face image of the target person A is continued without being stopped. Will be.

  If the face image of the target person A can be recognized after S26 (S27 / YES), the image processing unit 6 performs the above-described operation of S32 (description is omitted). Note that when the image processing unit 6 can recognize the face image of the target person A, the target person A that exists outside the frame or is hidden behind an object or person moves. For example, the face image of the target person A (or an image of a part of the body) appears in the in-frame image and is recognized as the face of the target person A.

  If the face image of the target person A cannot be recognized after S26 (S27 / NO), the image processing unit 6 continues to recognize the face image of the target person A until a predetermined recognition duration time elapses ( S28 / NO). The image processing unit 6 determines whether or not the recognition continuation time has elapsed, based on a time measurement result of a timer unit (not shown).

  When the recognition continuation time has elapsed (S28 / YES), the image processing unit 6 performs the above-described operation of S29 (description is omitted).

  Next, the image processing unit 6 determines whether or not all related persons B, C, and D associated with the face image of the target person A have been recognized (S30). For example, in the case of S24 / YES, if the face of related person B can be recognized, face recognition cannot be performed for related persons C and D (S30 / NO), and the process proceeds to S31.

  Next, the image processing unit 6 determines whether or not the in-frame image currently displayed on the liquid crystal panel 5 has been changed (S31). The in-frame image is changed by moving the camera that has been held by the user and holding the camera at another location. That is, since the user has failed to recognize the face image of the target person A in the currently displayed frame image, the user moves the camera from the current position, holds the camera at another location, Attempts to display images.

  When detecting the change of the intra-frame image (S31 / YES), the image processing unit 6 starts the operation again from S21. In this case, however, the face image of the related person to be recognized in S22 is excluded from the face image already recognized in the intra-frame image before the change. For example, when the face recognition of the related person B is performed in the pre-change intra-frame image (S24 / YES), in the post-change intra-frame image (the newly displayed intra-frame image), the related persons C and D The face image is a recognition target. In the above description, the trigger for the image processing unit 6 to start the face recognition again after the face recognition of the target person A has failed is as follows: “The user moves the camera to another location, holds the camera, Although “when the image processing unit 6 detects a change in the intra-frame image when attempting to display the intra-frame image” is taken as an example, the present invention is not limited to this. For example, face recognition may be performed when the camera is moving, when the in-frame image is zoomed in or when the in-frame image is zoomed out.

  If no change in the image within the frame is detected for a certain time in S31 (S31 / NO), the series of operations shown in FIG. 4 is terminated. In S30, all the face images of the related persons B, C, and D can be recognized along with the change of the in-frame image, but the face image of the target person A cannot be recognized in any of the in-frame images. If this is the case (S30 / YES), the series of operations shown in FIG. Although not shown, when the input unit 7 receives an instruction to end face recognition from the user at any stage of S21 to S31, the series of operations shown in FIG.

  In the example of FIG. 4, when face recognition of all the related persons cannot be performed (S30 / NO), after the intra-frame image is changed (S31 / YES), the face images of the target person A and the remaining related persons are changed. Although the recognition is started, the recognition of the face images of the target person A and the remaining related persons may be started even if the intra-frame image is not changed.

  As described above, according to the imaging apparatus of the present embodiment, the target person's face image and the related person's face image are linked in advance, and the target person is not captured at all in the predetermined frame image. Even if a part of the body of the target person is shown but the body part cannot recognize the face, the face of the related person can be detected even if the face image of the target person cannot be recognized. If the image can be recognized, it can be continued without stopping the recognition of the face image of the target person.

  In the description of FIG. 4, still image shooting by the camera of the present embodiment has been described as an example. However, the present invention can be similarly applied to moving image shooting.

  As mentioned above, although embodiment of this invention was described, it is not limited to the said embodiment, A various deformation | transformation is possible in the range which does not deviate from the summary. Hereinafter, modified examples will be described.

[Modification 1]
In the above-described embodiment, the example in which the image processing unit 6 that is one embodiment of the image processing apparatus of the present invention is mounted on the camera (digital still camera) that is one embodiment of the imaging apparatus of the present invention has been described. The image processing unit 6 may be mounted on an information processing apparatus other than the camera, or may be used alone without being mounted on the information processing apparatus. Here, an example in which the image processing unit 6 is mounted on a PC (Personal Computer) will be described with reference to FIG.

  As shown in FIG. 5, the imaging device 10 and the PC 20 are connected by wire or wirelessly and can transmit and receive data. The imaging device 10 may be, for example, a digital still camera or a video camera disposed at a remote place (a general imaging device capable of capturing an image in a frame). In addition to the storage device 21, the display device 22, and the input device 23, the PC 20 includes the image processing unit 6 of this embodiment. The storage device 21 is a nonvolatile memory such as an HDD (Hard Disk Drive), and corresponds to the memory 4 shown in FIG. The display device 22 is a display and corresponds to the liquid crystal panel 5 shown in FIG. The input device 23 is a device such as a keyboard or a mouse, and corresponds to the input unit 7 shown in FIG. Although not shown in FIG. 5, the image processing unit 6, the storage device 21, the display device 22, and the input device 23 are electrically connected to each other by a bus or the like, and data can be transmitted and received between them. It has become. 5, the storage device 21 is built in the PC 20, but may be provided outside the PC 20 (for example, a server on a network).

  The image processing unit 6 illustrated in FIG. 5 includes an intra-frame image acquisition unit 61, a face image recognition unit 62, and a control unit 63. The above-described face recognition in FIG. 4 is executed by these means. An operation example at this time will be described below.

  First, the intra-frame image acquisition unit 61 captures the intra-frame image output from the imaging device 10 to the PC 20 via a predetermined interface into the image processing unit 6. This in-frame image is displayed on the screen by the display device 22 (S21).

  Next, the face image recognition unit 62 applies the target person to the in-frame image based on the face image of the target person registered in advance in the storage device 21 and the face image of the related person associated therewith. And the recognition of the related person's face image is started (S22). The registration of each face image in the storage device 21 can be performed on the PC 20 by performing each of the preparation operations described with reference to FIGS. In particular, the controller 63 may perform the preparation operation 2 shown in FIG. That is, the control means 63 is based on the face image of the target person designated by the user, and from at least one person face image of the target person from predetermined photo data (for example, those stored in advance in the storage device 21). The photograph data including the related person's face image is searched (S11, S12 in FIG. 3). Next, the control means 63 controls the face image recognition means so as to recognize face images of all related persons included in the retrieved photographic data (S13 in FIG. 3). Next, the control means 63 associates the recognized related person's face image with the face image of the target person designated by the user and registers it in a predetermined storage means (for example, the storage device 21) (S14 in FIG. 3). ).

  When the face image of the target person cannot be recognized after the start of the face image recognition in S22 (S23 / NO), but the face image of the related person can be recognized (S24 / YES), the control means 63 recognizes the related person who has been recognized. A predetermined notification means (for example, the display device 22) is controlled so as to notify the user of the face image (S25). Next, the control means 63 controls the face image recognition means 62 so as to continue recognition of the face image of the target person for a predetermined time (for example, the above-described recognition continuation time) (S26, S27 / NO, S28 / NO). ). If the face image of the target person cannot be recognized even after the face image of the target person has been recognized for a predetermined time (S28 / YES), the control means 63 indicates that the face image of the target person could not be recognized. A predetermined notification means (for example, the display device 22) is controlled so as to notify the user (S29).

  Next, the control means 63 determines whether or not all related person face images have been recognized (S30). If there is a related person's face image that has not yet been searched (S30 / NO), and if the intra-frame image has been changed, that is, if a new intra-frame image has been sent from the imaging device 10 (S31). / YES), the face image recognition means 62 is controlled so as to recognize the face image of the target person and the face image of the related person with respect to the new in-frame image (the changed in-frame image). However, at this time, the control means 63 performs control so that the face image of the related person that can be recognized in the frame image before the change is excluded from the recognition target with respect to the new frame image. . As a result, a new in-frame image is displayed on the display device 22 (S21), and the face image recognition means 62 performs the face image of the target person and the unrecognized related person face image with respect to the new in-frame image. Is started (S22).

[Modification 2]
For example, the operation in the above-described embodiment can be executed by hardware, software, or a combined configuration of both. When executing processing by software, a program in which a processing sequence is recorded may be installed and executed in a memory in a computer incorporated in dedicated hardware. Or you may install and run a program in the general purpose computer which can perform various processes. Such a program can be recorded in advance on, for example, a hard disk or a ROM (Read Only Memory) as a recording medium. Alternatively, the program is stored on a removable recording medium such as a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, a magnetic disc, and a semiconductor memory. It is possible to store (record) temporarily or permanently. Such a removable recording medium can be provided as so-called package software.

  The program may be wirelessly transferred to a computer from a download site, in addition to being installed on the computer from a removable recording medium as described above. Or you may wire-transfer to a computer via networks, such as LAN (Local Area Network) and the internet. The computer can receive the transferred program and install it on a recording medium such as a built-in hard disk.

  In addition, the operations in the above-described embodiments are not only executed in time series as described above, but are also executed in parallel or individually as required by the processing capability of the apparatus that executes the processing. It is also possible to construct.

DESCRIPTION OF SYMBOLS 1 Optical system 2 Image sensor 3 AFE
4 Memory 5 Liquid Crystal Panel 6 Image Processing Unit 7 Input Unit 10 Imaging Device 20 PC
DESCRIPTION OF SYMBOLS 21 Memory | storage device 22 Display apparatus 23 Input device 61 In-frame image acquisition means 62 Face image recognition means 63 Control means

Claims (9)

An intra-frame image acquisition means for acquiring an intra-frame image of the imaging apparatus;
Based on the face image of the target person to be face-recognized in advance and the face image of the related person linked to the face image of the target person, the target person's face image is compared with the in-frame image. Face image recognition means for recognizing a face image and a face image of the related person;
When the face image of the target person cannot be recognized and the face image of the related person can be recognized, the face image recognition means is controlled so as to continue recognition of the face image of the target person for a predetermined time. Control means;
An image processing apparatus comprising: The control means includes
When the face image of the target person cannot be recognized and the face image of the related person can be recognized, a predetermined notification unit is controlled so as to notify the user of the recognized face image of the related person. The image processing apparatus according to claim 1. The control means includes
If the face image of the target person cannot be recognized even if the recognition of the face image of the target person is continued for a predetermined time, the user is notified that the face image of the target person cannot be recognized. 3. The image processing apparatus according to claim 2, wherein a predetermined notification means is controlled. The control means includes
If the in-frame image is changed after the face image of the target person cannot be recognized even if the recognition of the face image of the target person is continued for a predetermined time, the in-frame image after the change 4. The apparatus according to claim 1, wherein the face image recognition means is controlled to recognize the face image of the target person and the face image of the related person. Image processing device. The control means includes
When the face image of the target person and the face image of the related person are recognized with respect to the image in the frame after the change, the face image of the related person that can be recognized in the image in the frame before the change is recognized from the recognition target. The image processing apparatus according to claim 4, wherein the face image recognition unit is controlled so as to exclude the face image and exclude the face image. The control means includes
Based on the face image of the target person specified by the user, search for photo data including the face image of the target person and the face image of at least one related person from predetermined photo data,
Controlling the face image recognition means to recognize face images of all the related persons included in the searched photographic data;
6. The recognized face image of the related person is registered in a predetermined storage unit in association with the face image of the target person specified by the user. The image processing apparatus described.   An imaging apparatus comprising the image processing apparatus according to claim 1. Obtaining an in-frame image of the imaging device;
Based on the face image of the target person to be face-recognized in advance and the face image of the related person linked to the face image of the target person, the target person's face image is compared with the in-frame image. Recognizing a face image and a face image of the related person;
Continuing recognition of the face image of the target person for a predetermined time when the face image of the target person cannot be recognized and the face image of the related person can be recognized;
A face image recognition method characterized by comprising: Processing to acquire an image within the frame of the imaging device;
Based on the face image of the target person to be face-recognized in advance and the face image of the related person linked to the face image of the target person, the target person's face image is compared with the in-frame image. Processing for recognizing a face image and a face image of the related person;
A process of continuing recognition of the face image of the target person for a predetermined time when the face image of the target person cannot be recognized and the face image of the related person can be recognized;
A program that causes a computer to execute.

Images