JP2020087305A - Information processing apparatus, information processing method and program - Google Patents

Abstract

To be able to suppress repetition of false recognition of a person.SOLUTION: An information processing apparatus 100 recognizes a person included in a captured image by using a list in which a face image of the person to be recognized is registered. The information processing apparatus 100 comprises: recognition means for recognizing a face image included in an image of interest as a specific person on the basis of the face image included in the image of interest and the list; and input means for inputting a recognition result for the face image by means of the recognition means is false. The recognition means controls, in recognition after receiving the input, such that the face image determined to be false is not recognized as the specific person.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique of recognizing a face of a specific person from an image.

In recent years, many techniques have been proposed in which an image of an object included in a captured image is highly processed to extract useful information. In particular, it recognizes human face images and compares the input face images (called input face images) with the face images of a plurality of people registered in advance (called registered face images). , Has been actively researched and developed for face recognition for determining who the input face image is.

Face recognition in stores or the like includes whitelist recognition for recognizing pre-registered important customers or blacklist recognition for recognizing sensitive customers. In these recognitions, the face image of the person to be recognized is registered in advance, and the face image of the person reflected by the camera is compared with the face images of the registered persons. When the face image of the person reflected on the camera matches any of the face images of the registered persons, the clerk or the security guard is notified.

JP 2012-141708 A

In the face recognition, erroneous recognition (i.e., erroneously recognizing the face image as another person) may occur depending on the state in which the face image is captured. For example, this is a state in which a person with a face similar to the registered person appears. In addition, the captured face image is small or shaken. Other than that, erroneous recognition is likely to occur depending on the orientation of the imaged face and the state of illumination.

With respect to the above problem, in Patent Document 1, the recognition result is corrected by using the feedback that the user has determined to be erroneous recognition. However, in the method disclosed above, when a person who has been erroneously recognized appears in the image continuously in time series, erroneous recognition may repeatedly occur. For example, when face recognition is performed on a walking person, face recognition may be performed on each frame image of a moving image. In this case, if face recognition in a certain frame is erroneous recognition, there is a possibility that erroneous recognition may be repeatedly performed in subsequent frames in which appearance does not change much from that erroneously recognized frame. The present invention has been made in view of the above problems, and suppresses repeated occurrence of erroneous recognition of a person shown in a time-series image.

An information processing apparatus according to the present invention that solves the above problem is an information processing apparatus that recognizes a person included in a captured image using a list in which face images of a person who is a recognition target are registered, and is included in a target image. Recognition means for recognizing a face image included in the attention image as a specific person based on the face image and the list, and input means for inputting that the recognition result of the recognition means by the recognition means is incorrect. In the recognition after receiving the input, the recognizing unit controls the face image input that the recognition result is erroneous so as not to be recognized as the specific person. ..

It is possible to prevent repeated erroneous recognition of a person in a time-series image.

Block diagram showing a functional configuration example of an information processing apparatus Figure explaining the registered person list Flowchart explaining processing executed by the information processing apparatus Block diagram showing a functional configuration example of an information processing apparatus Figure explaining the registered person list Block diagram showing a functional configuration example of an information processing apparatus Diagram showing an example of the hardware configuration of an information processing device Diagram showing an example of GUI Diagram showing an example of GUI Flowchart explaining processing executed by the information processing apparatus

<Embodiment 1>
In service facilities such as stores, face recognition technology using surveillance cameras installed inside and outside the facility is used in order to quickly and surely know the visit of important customers or customers requiring attention. The service facility manages a list in which face images of specific persons such as important customers and IDs indicating personal information are assigned, and the person registered using such a list and the face images of customers in the store To recognize. However, since the customer freely moves around in the store, it is not always possible to acquire the face image of the customer in a stationary state. Since such a situation is possible, it is necessary to specify that the customer who moves around from the images acquired in time series is the same person, and then recognize whether the specified person is the registered customer. Here, if the recognition of the specific person specified from the time-series image fails once, there is a possibility that the same erroneous recognition may continue in the images acquired thereafter. One of the objects in the following embodiments is to prevent the same erroneous recognition from being repeated in the subsequent recognition process for the same person, once the erroneous recognition is performed in the face recognition technology. .. Therefore, if you make a mistake in recognizing once, in the face recognition for the same person after that, you should exclude the registered person who was erroneously recognized earlier from the original registered person set and then perform face recognition. To do.

In order to realize this, it is specified by face tracking that the persons appearing in the captured image are the same person over a plurality of frames. Then, a tracking identifier is assigned to each identified person. At the same time, the registered person ID that should be excluded from the recognition process is managed for each tracking identifier. Prior to describing the embodiments according to the present invention, the definition of terms will be described.

The registered person list is information in which at least the facial image characteristics (or facial image) and the ID are associated with each other for a person to be identified from an unspecified number of persons. For example, the list is as shown in FIG. It is used to identify customers (or dangerous persons) who are important to users such as facility managers in service facilities where an unspecified number of people come and go. The registered person list prepares in advance a face image of a person the user wants to recognize, predetermined features extracted from the face image, and an ID for distinguishing each face image. The registered person list may hold category information (sex, age, height, etc.) about the registered persons.

The tracking identifier list is information in which an ID assigned to a face image detected from a surveillance camera image at a certain time, a face image feature (face image), and a registered person exclusion ID are associated with each other. For example, the list is as shown in FIG. 2B or 2C. The registered person exclusion ID is an ID corresponding to a person who skips the recognition process for a certain tracking identifier in the registered person list. Note that the tracking identifier list has no particular information at the time when the processing is started. When a face is detected from the image and a new face image is detected by face tracking, a new tracking identifier and exclusion ID (initial value is None) are updated.

The information processing apparatus that executes face recognition according to this embodiment will be described in detail below. FIG. 1 is a block diagram showing a functional configuration example of an information processing apparatus showing an embodiment to which the present invention is applicable. The information processing apparatus 100 recognizes a person from an image using the registered person list in which the face of the person to be recognized is registered. The information processing device 100 is connected to the imaging device 200 and the display device 300. The information processing apparatus 100 includes an image input unit 101, a detection unit 102, a tracking unit 103, a personal characteristic acquisition unit 104, a recognition unit 105, a registered person list storage unit 106, a tracking identifier list storage unit 107, a display control unit 108, and a feedback input. It has a part 109. It should be noted that the information processing apparatus 100 may not have all the functional configurations listed here. For example, the display control unit 108 and the feedback input unit 109 may be included in the display device 300 connected to the outside.

The image input unit 101 inputs a time-series captured image captured by the image capturing apparatus 200, that is, an image that is a part of a moving image. In the present embodiment, an image (captured image) obtained by capturing a person in a predetermined space is input. The predetermined space is a space where an unspecified large number of people such as shops and public facilities enter and leave. In the present embodiment, it is assumed that the image being captured by the image capturing apparatus 200 in real time is input from the image input unit 101. The imaging device 200 may be placed in a place physically separated from other components, or may be configured to transfer an image to other components via a network. Further, not limited to the image being picked up, the reproduced image of the picked-up recorded image may be input from the image input unit 101. The captured image is held by the storage device.

The detection unit 102 detects the face image included in the attention image by extracting the image features of the eyes and the nose, which are the constituent elements of the face, from the attention image input from the image input unit 101. If the target image does not include a person, no face image is detected. When the attention image includes a plurality of persons, the face image of each person is detected. An existing known method may be used as the face detection method. For example, the image feature indicating the shape corresponding to the constituent elements of the face image such as the nose, mouth and eyes is extracted. The size of the face is estimated from the extracted sizes of both eyes and their distances, and the region surrounded by the region of the extracted size is used as a face image with the position corresponding to the center of the nose as a reference. The detected face image is input to the tracking unit 103 and the individual feature acquisition unit 104 together with the position within the angle of view and the size of the region in the image for the tracking process and the individual feature extraction process described later.

In the tracking unit 103, based on the target image and the first image captured in the previous frame, the face image included in the target image and the face image included in the first image are detected within a predetermined period of time. When is in the movable range, the face is tracked by giving the same identifier to the face image. Specifically, a face image similar to each of the N face images detected from the current captured image is specified from the past frame images, and the tracking identifiers of the N face images are specified. When the detection unit 102 detects N face images, identifiers i=0 to N−1 (where i is a natural number) are assigned to each face image. The matching process is performed for each face image in order from i=0 with each face image in the previous frame. When the i-th face image is similar to the face image of the previous frame, the tracking identifier is specified from the tracking face image of the previous frame, and the same tracking identifier is given to the i-th face image. That is, the face image of the current frame, which is close in position to the face image of the previous frame and has similar brightness data, is specified as the face image of the same person. Furthermore, when there is no corresponding face image in the current frame, an area having a high correlation with the brightness data of the face image in the previous frame is searched in a certain range in the current frame. If there is a corresponding area, that area is specified as the face image of the same person. Alternatively, if there is no face image similar to the jth face image in the previous frame image, a new tracking identifier is added to the jth image. This is effective when the detection unit does not detect the face because the person faces the sideways or up and down for a moment, and it is possible to detect a face having a detection capability higher than that of the detection unit.

In this way, the tracking unit 103 tracks the face of the same person over a plurality of frames. Further, the tracking unit 103 specifies a common tracking identifier for a series of time-series images in order to specify faces that are the same person. The same tracking identifier is set for a series of face image groups in which the face of the same person is made by the tracking unit 103. That is, by using the image feature of the face image to which the tracking identifier is assigned at a certain time as a template, a matching area is searched for in each time series image, and the same ID is assigned to the matched area. The tracking identifier is input to the personal recognition unit 105 together with the personal characteristic extracted by the personal characteristic acquisition unit 104.

The personal characteristic acquisition unit 104 acquires, from the face image, a personal characteristic indicating the positional relationship of face parts that can identify an individual. The feature here is assumed to be an image feature that can be obtained from an image by edge detection or the like. The acquisition method may be arbitrary, but in the present embodiment, the individual characteristics are extracted as follows. First, based on the positions of representative organs such as eyes, nose, and mouth specified by the face detection unit 102, the width of both eyes is set to a predetermined distance, and the line segment connecting both eyes is made horizontal on the image. Rotate/scale images. Then, a rectangular area for extracting features is set in the face image. The size of the area is arbitrary, but one side is about 1.5 times the width of the eye so that all the organs such as eyes and mouth that show the characteristics of an individual can be completely filled in, but not the background. Should be placed in the center of the face. Subsequently, the pixel values in the rectangular area are taken out in order from the upper left to the lower right, and are connected in a line to form a vector. This is an individual feature. However, the extraction of the personal feature used in this embodiment is not limited to the method described above. For example, a deep neural network may be used to extract individual features. Specifically, the image feature and the like that can be acquired from the convolutional layer in CNN (Convolutional Neural Network) are acquired. In this case, the personal characteristic acquisition unit 104 acquires the personal characteristic necessary for individual identification by inputting the face image into the neural network (learned model) in which the identification of the person has been learned. The learned model is a network structure based on a neural network that outputs a recognition result corresponding to a captured image from the captured image and its parameters. The recognition may be to identify an individual from an image, for example. By using the learned model, the individual can be recognized with higher accuracy.

The personal recognition unit 105 compares the facial feature of the person registered in the registered person list with the characteristic of the first face image to determine the first facial image among the persons registered in the registered person list. Recognize that you are the first person. More specifically, first, the personal feature of the captured image sent from the personal feature acquisition unit 104 and the personal feature of the registered face image sent from the registered person list storage unit 106 are compared to acquire the similarity. .. The personal feature extracted from each face image is called an input personal feature. In the registered person list (registered person list), the personal feature held in the face image corresponding to the registered person is called a registered personal feature. Here, the reciprocal of the L2 distance (Euclidean distance) between the input individual feature and the registered individual feature is set as the similarity. The personal recognition unit 105 compares the acquired degree of similarity with a preset threshold value and outputs the recognition result. Alternatively, all registered persons whose degree of similarity is larger than a predetermined threshold value are output as the recognition result.

When a plurality of registered individual characteristics are stored in the registered person list storage unit 106, the degree of similarity between the input individual characteristic acquired from the captured image and each of the plurality of registered individual characteristics is acquired. In this case, the same degree of similarity as the number of registered personal characteristics is acquired. The personal recognition unit 105 performs threshold processing on the maximum similarity among the plurality of similarities. Then, the registered person ID associated with the registered individual characteristic that has obtained the maximum similarity is output as a recognition result. If there is no similarity exceeding the threshold value, the result that there is no corresponding person in the registered persons is output.

In the registered person list storage unit 106, the facial features (face image) of the registered persons are acquired and stored in advance. The facial features of the registered person are called registered personal features. The registered individual characteristic is represented by, for example, a characteristic vector or a numerical value. The method of acquiring the personal characteristics of the registered image may be the same as the procedure shown so far. That is, the registered image may be input as a captured image, face detection may be performed, and the personal feature may be acquired from the detected face. Here, an individual feature that can identify an individual is acquired from a face image based on a learned model that identifies a person included in the image. In the registered person list storage unit 106, as shown in FIG. 2A, the registered person's person ID and the registered individual characteristic of the person are stored in association with each other.

In the identifier information storage unit 107, based on the tracking identifier list reflecting the feedback information up to now, the person ID of the registered person who does not perform the recognition process performed by the individual recognition unit 105 for each tracking identifier (this is referred to as an exclusion ID). Call) is held. FIG. 2B shows an initial tracking identifier list generated based on the face image detected from the captured image. Fig. 2(c) shows the tracking identifier list updated by receiving several feedbacks from the state of Fig. 2(b). In FIG. 2C, there is no exclusion ID for the tracking IDs TrackID_0 and TrackID_3. Further, for the tracking identifier TrackID_1, it is indicated that RegID_3 is an exclusion ID. Similarly, for the tracking identifier TrackID_2, it is indicated that RegID_0 and RegID_2 are registered person exclusion IDs. In the registered person list management unit 107, the registered person exclusion ID is managed in association with the tracking identifier as described above.

Here, the exchange of information between the individual recognition unit 105, the registered person list storage unit 106, and the tracking identifier list storage unit 107 when an image is input from the image input unit 101 will be described in a little more detail.

Face detection, face tracking, and individual feature extraction are performed on the image input from the image input unit 101 in the detection unit 102, the tracking unit 103, and the individual feature acquisition unit 104, respectively. As a result, the tracking identifier corresponding to the face image to be subjected to the recognition process this time is specified, and at the same time, the input personal feature corresponding to the face image is acquired. The input personal feature thus obtained and the tracking identifier are input to the personal recognition unit 105.

The personal recognition unit 105 refers to the input tracking identifier and the tracking identifier list storage unit 107, and confirms whether or not there is a registered person ID to be excluded when authenticating the face image corresponding to the tracking identifier i. That is, the tracking identifier list storage unit 107 notifies the registered person list storage unit 106 of the registered person ID if there is an exclusion ID corresponding to the indicated tracking identifier. The registered person list storage unit 106 outputs the registered individual characteristics used in the recognition process this time to the individual recognition unit 105. At that time, the registered individual characteristics corresponding to the registered person ID transmitted from the registered person list management unit 107 are excluded and output. That is, when it is determined that the first face image is not the first person based on the determination result, the personal recognition unit determines that the first face image is not the first person among the persons registered in the registered person list. Certify that there is.

The above procedure will be specifically described with reference to FIG. For example, as shown in FIG. 2A, it is assumed that six registered persons (whose respective registered person section IDs are RegID_0 to RegID_5) are registered in the registered person list storage section 106. The registered personal characteristics of all registered personal IDs registered in advance are referred to as a default registered personal characteristic set.

FIG. 2C is an example of a tracking identifier list generated by the tracking unit 103 assigning a common tracking identifier to similar face images detected by the detection unit 102. Prior to feedback, this list holds only the specified tracking identifiers and all the exclusion IDs are held as None, as shown in FIG. 2B. This list is updated by feedback from the user (FIG. 2(c)). For example, if the tracking identifier input to the personal recognition unit 105 is TrackID_0 (or TrackID_3), the personal recognition unit 105 calculates the similarity from all registered personal features (default registered personal feature set) and input personal features. To get. Alternatively, if the tracking identifier input to the personal recognition unit 105 is TrackID_1, the personal recognition unit 105 acquires the degree of similarity from the default registered personal characteristic set excluding the registered personal characteristic of RegID_3 and the input personal characteristic. Alternatively, if the tracking identifier input to the personal recognition unit 105 is TrackID_2, the personal recognition unit 105 acquires the similarity from the default registered personal feature set excluding the registered personal features of RegID_0 and RegID_2 and the input personal feature. To do. By doing so, it is possible to limit the registered person ID for which the degree of similarity should be acquired in the personal recognition unit 105, according to the tracking identifier.

The display control unit 108 controls the display device to perform a display for notifying the user that the i-th face image is a registered person. For example, the captured image that is the target of the face recognition processing, the registered person image of the determination result determined by the individual recognition unit 105, and the registered person ID (name of the registered person) are displayed on the GUI owned by the user. The display device 300 is assumed to be a monitor that can be viewed by a supervisor of the information processing device 100, a mobile terminal, or the like. Here, the supervisor refers to a person who uses the information processing apparatus 100 to execute a predetermined service. For example, when the information processing apparatus 100 is used to realize white list (or black list) detection for detecting an important customer (or caution customer) in a store, it refers to a store clerk or a security guard.

The display control unit 108 notifies the supervisor (store clerk or security guard) of the personal recognition result determined by the personal recognition unit 105. The store clerk or the guard can compare the captured image and the registered person image displayed on the display control unit 108 to provide a predetermined service. For example, when the display control unit 108 is notified of the person ID of an important customer, it is possible to provide customer service or the like according to the taste. Further, when the supervisor compares the captured image and the registered person image displayed on the display control unit 108 and determines that the face recognition result is erroneous (erroneous recognition), the supervisor inputs the feedback input unit 109. And notify that effect.

The feedback input unit (reception unit) 109 receives from the user that the recognition result for the face image is incorrect as a determination result. For example, the supervisor (user) receives feedback as to whether or not the i-th face image is the registered person ID. As described above, the feedback from the supervisor means that when the supervisor determines that the face recognition result this time is incorrect, the fact is notified (misrecognized).

The feedback from the supervisor input to the feedback input unit 109 is notified to the registered person list management unit 107. When the registered person list management unit 107 receives feedback from the supervisor that the recognition result this time is an erroneous recognition, the registered person list management unit 107 updates the exclusion ID of the tracking identifier list managed internally. That is, the registered person ID that is erroneously recognized this time is added to the exclusion ID of the tracking identifier to which the face image that has been recognized this time is assigned. By doing so, it becomes possible to avoid repeating the same erroneous recognition in the subsequent recognition processing for the captured image having the same tracking identifier as this time.

FIG. 7 is a diagram showing a hardware configuration of the information processing device 100. H11 is a CPU, which controls various devices connected to the system bus H18. H12 is a ROM that stores a BIOS program and a boot program. H13 is a RAM and is used as a main storage device of the CPU H11. H14 is an external memory that stores programs processed by the information processing apparatus 10. The input unit H15 is a keyboard, a mouse, and a robot controller, and performs processing related to input of information and the like. The display unit H16 outputs the calculation result of the information processing device 100 to the display device according to the instruction from H11. The display device may be of any type such as a liquid crystal display device, a projector, an LED indicator, or the like. H17 is a communication interface for performing information communication via a network. The communication interface may be Ethernet (registered trademark), and any type such as USB or serial communication may be used.

The above is the description of the information processing apparatus 100. Next, a processing procedure in this embodiment will be described. FIG. 3 is a flowchart showing an example of the processing procedure of the information processing apparatus 100 according to this embodiment. In the following description, each process (step) will be described by adding S to the beginning, and the description of the process (step) will be omitted. However, the information processing system 1 does not necessarily have to perform all the steps described in this flowchart. Hereinafter, it is assumed that the flowchart is realized by the CPU (H11) of FIG. 7 which is a computer executing the computer program stored in the external memory (H14).

In S1, the detection unit 102 detects the face image included in the captured image by extracting the image features of the eyes and the nose, which are the constituent elements of the face, from the image input from the image input unit 101. .. Here, it is assumed that N persons (N is an integer of N≧0) are included in the input image. The detected face image includes face image characteristics, area size, and position information within the angle of view.

In S2, the tracking unit 103 identifies, from the captured image, a first face image similar to the face image detected from the past frame image captured before the captured image. From the image input in the previous frame, a similar face image is specified for each of the N face images detected in the current frame, and the tracking identifier of each face image is specified.

In S3, the tracking unit 103 determines whether or not the tracking identifier i is the face image existing in the frame image immediately before the captured image. It should be noted that the processes from S3 onward are sequentially processed from i=0 to i=N-1. If the tracking identifier i is also detected in the previous frame image, the process proceeds to S40. If the tracking identifier i is not detected in the previous frame image, that is, if it is a newly set tracking identifier, the process proceeds to S41.

In S40, the tracking identifier list storage unit 107 refers to the recognition unit 105 of the tracking identifier list generated in the previous frame image. When the tracking identifier i is detected in the previous frame, the tracking identifier list storage unit 107 holds the tracking identifier list generated in the previous frame. The recognition unit 105 uses this tracking identifier list to perform face recognition on the tracking identifier i. In S41, the tracking identifier list storage unit 107 updates the list for the tracking identifier i. With respect to the newly added tracking identifier, no feedback has been given yet, so a list is generated as None for the exclusion ID.

In step S5, the tracking identifier list storage unit 107 determines the person IDs of registered persons who do not perform recognition processing for each tracking identifier (referred to as exclusion IDs), based on the tracking identifier list in which the feedback information up to now is reflected. Judge whether it holds. If there is no exclusion ID for the i-th tracking identifier, the process proceeds to S60. If there is an exclusion ID, the process proceeds to S61.

In S60, the personal recognition unit 105 compares the facial features of the persons registered in the list with the features of the first facial image, so that the first facial image is identified among the persons registered in the list. Recognize that you are a person. The personal feature acquired from the i-th face image is compared with the personal feature of each registered person in the registered person list to obtain the degree of similarity with each registered person. The registered person having the highest similarity to the i-th face image is acquired. Further, when the degree of similarity is larger than a predetermined threshold value, the i-th face image is recognized as the corresponding registered person. First, the personal characteristic acquisition unit 104 acquires a personal characteristic necessary for identifying an individual from the i-th face image. Here, an individual feature that can identify an individual is acquired from a face image based on a learned model that identifies a person included in the image. In the registered person list storage unit 106, individual characteristics corresponding to each registered person are acquired in advance from the face image of the registered person based on the learned model and stored. The recognition unit 105 obtains the degree of similarity between the i-th face image and each registered person by collating the individual characteristic acquired from the i-th face image with the individual characteristic corresponding to each registered person. The recognition result of the i-th face image is the registered person corresponding to the personal feature having the maximum similarity and larger than the predetermined threshold. If there is no similarity exceeding the threshold value, the result that there is no corresponding person in the registered persons is output.

In S61, the personal recognition unit 105 compares the features of the faces other than the excluded persons among the persons registered in the list with the features of the first face image, so that the first face image is included in the list. Recognize that the person is a specific person other than the exclusion ID. The personal feature acquired from the i-th face image is compared with the personal feature of each registered person in the registered person list other than the exclusion ID, and the degree of similarity with each registered person is acquired. The registered person having the highest similarity to the i-th face image is acquired. Further, when the degree of similarity is larger than a predetermined threshold value, the i-th face image is recognized as the corresponding registered person. If there is no similarity exceeding the threshold value, the result that there is no corresponding person in the registered persons is output. By using the registered person information excluding the exclusion ID, the same misrecognition will not be repeated.

In S7, the personal recognition unit 105 determines whether or not the i-th face image corresponds to the registered person. If it is recognized from the recognition result of S60 or S61 that the i-th face image is one of the registered persons, the process proceeds to S8. When it is recognized from the recognition result of S60 or S61 that the i-th face image does not correspond to any of the registered persons, the process proceeds to S12.

In S8, the display control unit 108 controls the display device to perform a display for notifying the user that the i-th face image is the specific person among the registered persons. For example, a notification is displayed on the display screen of the display device 300 to notify that the specific person who is the recognition target has arrived. An example of GUI display will be described later. In addition to the display, control may be performed such that the user is notified by voice or light.

In S9, the feedback input unit 109 inputs feedback regarding whether or not the i-th face image is recognized by the supervisor (user).

In S10, the feedback input unit 109 determines whether the recognition result is incorrect based on the feedback. If the supervisor determines that the face recognition result this time is incorrect, the process proceeds to S10. When the supervisor determines that the face recognition result this time is correct, the process proceeds to S12.

In S11, the tracking identifier list storage unit 107 updates the tracking identifier list corresponding to the i-th face image with the registered person ID corresponding to the recognized registered person as the exclusion ID, based on the recognition result or the feedback. After the tracking identifier list storage unit 107 updates the tracking identifier list, the process proceeds to S61, and the i-th face image is recognized again. By doing so, in the subsequent processing, it is possible to limit the registered person ID for which the personal recognition unit 105 should acquire the degree of similarity in accordance with the tracking identifier. As a result, face recognition can be performed quickly and accurately for a face image whose tracking identifier list has been updated. After updating the tracking identifier list, the process returns to S61, and the recognition processing is performed with reference to the updated tracking identification list. When the process returns to S61 and the recognition process is performed on the same face image, only the threshold process may be performed based on the similarity acquired in the previous process. Alternatively, instead of returning to S61, the process may proceed to S12 to proceed with the recognition process for the next face image.

In S12, the personal recognition unit 105 determines whether or not there is a next face image. Judgment is made by incrementing i and comparing with N. If there is a next face image, the process returns to S5. When there is no next face image (when recognition is completed for all face images in the current frame), the process ends.

The above procedure will be described in some detail. For example, when the registration exclusion information is managed as shown in FIG. 2C, an image is input, and as a result of face detection and face tracking, it is determined that the tracking identifier of the person in the image is TrackID_3. Think As a result of personal recognition of this person, it is assumed that the person ID is recognized as RegID_0. In this case, the recognition result display control unit 108 displays the captured image, the registered image of the person ID, and the person ID of RegID_0.

If the supervisor visually judges that the recognition result is an erroneous recognition, the information is notified to the registered person list management unit 107 through the feedback input unit 109. The registered person list management unit 107 recognizes that the recognition result of RegID_0 is erroneous recognition for the person of the tracking identifier TrackID_3, and thus adds RegID_0 to the registration exclusion person ID of the tracking identifier TrackID_3. In this case (FIG. 2(c)), since the registration exclusion person ID of the tracking identifier TrackID_3 is None (there is no registration exclusion person), the None is updated to the RegID_0.

The GUI will be described with reference to FIG. FIG. 8 is an example of a GUI relating to the display control unit 108 or the feedback input unit 109. FIG. 8A shows a display device G10 such as a tablet held by the user. The display device G10 may be any device having a screen such as a personal computer or a smartphone. In G200, the image for which the face recognition processing has been completed is displayed. G201 is a bounding box indicating a face image detected by the face detection processing. A tracking identifier (TrackID_XX) is displayed near the bounding box. Here, it is assumed that the face images of four persons are detected from G200. G300 is a face image of one person who is a feedback target of the user. Displayed when the tracking identifier is one of the registered persons. By this display, the user who is the supervisor is notified that the registered person has appeared in the monitored space. G301 is a face image corresponding to the recognition result of the recognition unit 105 regarding the tracking identifier. The registered person's ID (name) and face image associated in advance in the registered person list are displayed. Other related category information may be displayed at the same time. G400 and G401 are input units through which the user gives feedback on the recognition result. The user compares G300 and G301 and selects G400 if the recognition results match, and selects G401 if the recognition results are incorrect. In this GUI, only the correctness of the recognition result can be input. When there are many objects to be recognized, such as 100 or more registered persons, such a UI can efficiently detect misrecognition.

On the other hand, when the number of registered people is small, such as about 10, a GUI as shown in FIG. 8B can be considered. G302 shows the facial image of the registered person, which is the recognition result, and the degree of similarity. G501 displays a list of registered persons. The user selects the face image of the registered person included in the list and compares G300 and G302. If the number of people in the registered person list is about 10, the user is likely to be able to memorize the person to be recognized, and the correct answer may be fed back. For example, when the person in question is in the list, the person who gives the correct recognition result is fed back. As a result, the personal recognition unit 105 is the second face image detected from the second frame image captured after the captured image, and the second face image having the same identifier as the first face image is added. It is recognized that the face image is a second person different from the first person. When the user feeds back the correct recognition result, the recognition result for a certain tracking identifier is fixed. As a result, the registered person can be efficiently recognized.

As described in detail above, by configuring the face recognition system as in the present embodiment, even if an erroneous recognition is made once, the same erroneous recognition is not repeated in the subsequent recognition process for the same person. Can be realized.

<Embodiment 2>
In this embodiment, an example in which the feedback from the supervisor shown in the first embodiment is expanded is shown. The feedback from the supervisor shown in the first embodiment is only whether or not the recognition result is a false recognition. That is, the person in the captured image and the person indicated by the recognition result are visually judged to determine whether the person is the same person, and the determination result is fed back. In the present embodiment, in addition to the above, information regarding whether or not the attributes of the person in the captured image and the person indicated by the recognition result are the same is also fed back. The term "attribute" as used herein refers to a characteristic such as gender, age, race, etc. that a person has and that can be judged by appearance. If the attributes do not match, all IDs having the attribute can be registered as exclusion IDs.

In this embodiment, attribute information obtained by feedback is also used in order to reduce erroneous recognition. Therefore, the registered person list storage unit 106 and the tracking identifier list management unit 107 described in the first embodiment are also changed to manage the attribute information. The face recognition system of this embodiment will be described in detail below.

FIG. 4 is a block diagram showing the configuration of the face recognition system 400 in this embodiment. 4, parts having the same meanings as in FIG. 1 are given the same numbers as in FIG. 1, and description thereof will be omitted.

In the registered person list storage unit 406, as in the case of the first embodiment, the person ID of the registered person and the individual characteristic of the person are held in association with each other. Further, in the present embodiment, the attributes of registered persons are also managed in association with each other. In this embodiment, a case where sex is used as the registered person attribute will be described. The attribute may be the result of a person's determination in advance. Alternatively, the result obtained by using a neural network or the like for estimating the attribute from the face image may be held as the attribute.

FIG. 5A is a table showing the management state of registered person IDs, registered individual characteristics, and attributes in the registered person list storage unit 406. In FIG. 5A, the registered individual characteristic of the person whose registered person ID is RegID_0 is RegFeature_0, and the sex is male. The same applies to other registered person IDs.

In the tracking identifier list storage unit 407, exclusion IDs are managed for each tracking identifier. Further, in the present embodiment, a person attribute (which is referred to as a registration exclusion attribute) that is not used in the recognition processing performed by the individual recognition unit 105 is managed for each tracking identifier. That is, in the present embodiment, the exclusion ID and the registration exclusion attribute are managed as the registration exclusion information.

FIG. 5B shows a table of the management states of the tracking identifier, the exclusion ID, and the exclusion attribute in the tracking identifier list storage unit 407. In FIG. 5B, it is shown that there is no registration exclusion attribute for the tracking IDs TrackID_0 and TrackID_2. Further, for the tracking identifier TrackID_1, it is indicated that the attribute of female is a registration exclusion attribute. Similarly, for the tracking identifier TrackID_3, it indicates that the attribute of male is a registration exclusion attribute.

In the tracking identifier list storage unit 407, the registered person exclusion ID and the registration exclusion attribute are managed in association with the tracking identifier as described above.

The display control unit 408 displays the captured image subjected to the face recognition processing, the registered person image of the determination result determined by the individual recognition unit 105, and the registered person ID (name of the registered person) on the display device. Control. Further, in the present embodiment, control is performed so that the registered person attribute corresponding to the registered person of the determination result determined by the individual recognition unit 105 is also displayed. For example, a GUI as shown in FIG. 9 is displayed.

As in the case of the first embodiment, when the supervisor determines that the face recognition result of this time is incorrect, the feedback input unit 409 inputs feedback to that effect (misrecognizing). .. Further, in the present embodiment, the supervisor compares the registered person attribute displayed on the display control unit 408 with the captured image to determine whether or not the person attributes match, and when it determines that they do not match. Notifies that (attribute mismatch) via the feedback input unit 409.

Because of the nature of the person attributes, if there is an attribute disagreement, there is always an erroneous recognition. Therefore, the feedback in this embodiment is that "attribute disagreement and erroneous recognition" and "attribute matching but erroneous recognition". There are two cases. Feedback from the supervisor input to the feedback input unit 409 is notified to the tracking identifier list storage unit 407.

When the tracking identifier list storage unit 407 receives feedback from the supervisor that the current recognition result is “attribute mismatch and erroneous recognition”, the registration exclusion information managed internally is updated. In this case, the registered person ID that has been erroneously recognized this time is added to the registration excluded person ID of the tracking identifier to which the face image that has been recognized this time is assigned. Further, the personal attribute of the registered person who is erroneously recognized this time is added to the registration exclusion attribute of the same tracking identifier. When the feedback that "the attributes are the same but the recognition is incorrect" is received, only the registration exclusion person ID is updated. Alternatively, as in the present embodiment, when there is no overlap and no omission in the classification of person attributes, attributes other than the matching attributes may be added to the registration exclusion attributes of the same tracking identifier. ..

For example, when the person in the captured image is a woman, but the recognition result is that the person attribute is erroneously recognized as a registered person having a male attribute, the feedback from the supervisor is “attribute mismatch and erroneous recognition”. In that case, the registration person ID is added to the registration exclusion person ID, and at the same time, the person attribute (male) of the registration person who is erroneously recognized this time is added to the registration exclusion attribute.

Further, for example, when the person in the captured image is a woman and the recognition result is erroneously recognized as a registered person having a woman attribute, the feedback from the supervisor is “mismatched attribute, but misrecognized”. In that case, at the same time as adding the registered person ID to the registered exclusion person ID, a person attribute (that is, male) other than the person attribute (female) of the registered person who is erroneously recognized this time is added to the registered exclusion attribute.

Here, the exchange of information between the individual recognition unit 105, the registered person list storage unit 406, and the tracking identifier list storage unit 407 when an image is input from the image input unit 101 will be described in a little more detail.

Face detection, face tracking, and personal characteristic acquisition unit 104 perform face detection, face tracking, and personal characteristic extraction on the image input from image input unit 101, respectively. As a result, the tracking identifier corresponding to the face image to be subjected to the recognition process this time is specified, and at the same time, the input personal feature corresponding to the face image is acquired. The input personal feature thus obtained and the tracking identifier are input to the personal recognition unit 105.

The personal recognition unit 105 confirms the presence/absence of the registered person ID to be excluded from the recognition process this time and the presence/absence of the registration exclusion attribute by displaying the input tracking identifier in the tracking identifier list storage unit 407. That is, if there is a registration exclusion person ID corresponding to the indicated tracking identifier, the tracking identifier list storage unit 407 notifies the registration person list storage unit 106 of the registration person ID. Similarly, if there is a registration exclusion attribute corresponding to the indicated tracking identifier, that person attribute is transmitted to the registered person list storage unit 406.

The registered person list storage unit 406 outputs the registered individual characteristics used in the recognition process this time to the individual recognition unit 105. In that case, the registered individual characteristic corresponding to the registered person ID transmitted from the tracking identifier list storage unit 407 and the registered individual characteristic corresponding to the registered person attribute similarly transmitted from the tracking identifier list storage unit 407 are excluded. Output.

The above procedure will be specifically described with reference to FIGS. 5 and 6. For example, as shown in FIG. 5A, it is assumed that six registered persons (whose respective registered person section IDs are RegID_0 to RegID_5) are registered in the registered person list storage section 406. The registered personal characteristics of all registered personal IDs registered in advance are referred to as a default registered personal characteristic set.

At this time, if the tracking identifier input to the individual recognition unit 105 is TrackID_0, both the registered person exclusion ID and the registration exclusion attribute are None from FIG. 5B, so there is no registered individual characteristic to be excluded. Therefore, the individual recognizing unit 105 acquires the degree of similarity from all the registered individual characteristics (default registered individual characteristic set) and the input individual characteristics.

If the tracking identifier input to the personal identification unit 105 is TrackID_1, it can be seen from FIG. 5B that the registered person exclusion ID is RegID_3 and the registration exclusion attribute is female. In that case, it can be seen from FIG. 5B that registered personal characteristics corresponding to RegID_3, RegID_1, RegID_2, and RegID_4 are excluded. Therefore, the individual recognizing unit 105 acquires the degree of similarity from the default registered personal feature set excluding the registered personal features of RegID_3, RegID_1, RegID_2, and RegID_4, and the input personal feature.

If the tracking identifier input to the personal identification unit 105 is TrackID_2, it can be seen from FIG. 5B that the registered person exclusion IDs are RegID_0 and RegID_2, and there is no registration exclusion attribute. Therefore, the individual recognizing unit 105 obtains the degree of similarity from the input personal feature and the default registered personal feature set excluding the registered personal features of RegID_0 and RegID_2.

If the tracking identifier input to the personal identification unit 105 is TrackID_3, it can be seen from FIG. 5B that there is no registered person exclusion ID and the registration exclusion attribute is male. In that case, it can be seen from FIG. 5B that registered personal characteristics corresponding to RegID_0, RegID_3, and RegID_5 are excluded. Therefore, the individual recognizing unit 105 acquires the degree of similarity from the default registered individual characteristic set excluding the registered individual characteristics of RegID_0, RegID_3, and RegID_5 and the input individual characteristic. By doing so, it is possible to limit the registered person ID for which the degree of similarity should be acquired in the personal recognition unit 105, according to the tracking identifier.

The flow of processing in this embodiment will be described with reference to FIG. FIG. 10 is a flowchart illustrating the processing executed by the information processing device. The description of the same processing as that of the first embodiment will be omitted, and the description will focus on the differences. The personal identification unit 105 further recognizes the attribute information of the person in the face image obtained from the captured image. The feedback input unit 109 further receives a determination result as to whether or not the attribute information matches. When it is determined from the determination result that the attribute information of the face image is incorrect, the personal recognition unit 105 authenticates the face image by comparing it with a registered person having attribute information different from the attribute information of the face image.

In S1, the detection unit 102 detects the face image by extracting the features of the face from the target image. In S2, the tracking unit 103 tracks a face similar to the face image detected from the past frame image captured before the target image. In S3, the tracking unit 103 determines whether or not the tracking identifier i is a face image existing in the previous frame. It should be noted that the processes from S3 onward are sequentially processed from i=0 to i=N-1. If the tracking identifier i is also detected in the previous frame, the process proceeds to S40. If the tracking identifier i is not detected in the previous frame, that is, if it is a newly set tracking identifier, the process proceeds to S41. In S40, the tracking identifier list storage unit 107 refers to the individual recognition unit 105 to the tracking identifier list generated in the previous frame image. When the tracking identifier i is detected in the previous frame, the tracking identifier list storage unit 107 holds the tracking identifier list generated in the previous frame. In this embodiment, the exclusion attribute information is also held. Therefore, when the attribute information of the face image i is limited by the feedback in the previous frame image, the registered person ID to be recognized is a limited list. The personal recognition unit 105 uses this tracking identifier list to perform face recognition on the tracking identifier i. In S41, the tracking identifier list storage unit 107 updates the list for the tracking identifier i. With respect to the newly added tracking identifier, no feedback has been given yet, so a list is generated as None for the exclusion ID.

In step S5, the tracking identifier list storage unit 107 determines the person IDs of registered persons who do not perform recognition processing for each tracking identifier (referred to as exclusion IDs), based on the tracking identifier list in which the feedback information up to now is reflected. Judge whether it holds. If there is no exclusion ID for the i-th tracking identifier, the process proceeds to S60. If there is an exclusion ID, the process proceeds to S61. In S60, the personal recognition unit 105 compares the facial feature of the person registered in the list with the characteristic of the first facial image to determine whether the first facial image is the first person among the persons registered in the list. Recognize that the person is In S61, the personal recognition unit 105 compares the features of the faces other than the excluded persons among the persons registered in the list with the features of the first face image, so that the first face image is included in the list. Recognize that the person is a person other than the exclusion ID. Here, a list excluding the registered individual characteristic corresponding to the registered person ID transmitted from the tracking identifier list storage unit 407 and the registered individual characteristic corresponding to the registered person attribute also transmitted from the tracking identifier list storage unit 407 is set. refer.

In S8, the display control unit 108 controls to perform a display for notifying the user that the i-th face image is a registered person. In S9, the feedback input unit 109 receives feedback from the supervisor (user) as to whether or not the i-th face image is the registered person recognized. In S10, the feedback input unit 109 determines whether the recognition result is incorrect based on the feedback. If the supervisor determines that the face recognition result this time is incorrect, the process proceeds to S20. When the supervisor determines that the face recognition result this time is correct, the process proceeds to S12.

In S20, the feedback input unit 109 receives feedback from the supervisor (user) whether the recognition result regarding the attribute information of the i-th face image is consistent. If the attribute information matches, the process proceeds to S21. If the attribute information does not match and is incorrect, the process proceeds to S22. In S21, the tracking identifier list storage unit 407 updates the tracking identifier list corresponding to the i-th face image with the registered person ID corresponding to the recognized registered person as the exclusion ID, based on the recognition result or the feedback. In S22, the tracking identifier list storage unit 407 adds the registered person ID that is erroneously recognized this time to the registration excluded person ID of the face image i, and further adds the registered person ID that is erroneously recognized this time to the registration exclusion attribute of the same tracking identifier. Add attributes. In S12, the personal recognition unit 105 determines whether or not there is a next face image. Judgment is made by incrementing i and comparing with N. If there is a next face image, the process returns to S5. When there is no next face image (when recognition is completed for all face images in the current frame), the process ends.

The GUI in this embodiment will be described with reference to FIG. The description of the components having the same reference numerals as those in FIG. G303 is a face image corresponding to the tracking identifier authenticated as the registered person among the tracking identifiers displayed in G200. Here, in addition to the face image and ID specified from G200, the gender as attribute information is also displayed. G304 is an image showing the authentication result for the tracking identifier displayed in G303. Here, in addition to the face image and the ID from the registered person list, the gender which is the attribute information is also displayed. G501 is a tracking identifier list. Here, TrackID_0 is selected, and the attribute information which is the recognition result of the face image corresponding to TrackID_0 is displayed. Here, it is estimated that TrackID_0 is a female. G400, G404 and G405 are UIs for inputting feedback determined by the user. The G400 receives feedback that the recognition result is correct when the user determines that the recognition result is correct. When the attribute information (here, gender) is incorrect and the person is not a registered person, G404 receives feedback that the attribute is “mismatched and misrecognized”. When the attribute information (here, gender) is present but the person is not a registered person, G405 receives feedback that "attributes match but is erroneously recognized".

In the first embodiment, when the misrecognized person is once erroneously recognized, the registered person who is erroneously recognized earlier is excluded from the original registered person set, and thereafter, the recognition process for the same person is performed. This prevents the same misrecognition from being repeated.

In the second embodiment, when the attributes do not match in addition to the erroneous recognition, the attribute information is also used to exclude them from the original registered person set. In addition to the above effects, it is possible to reduce the possibility that another erroneous recognition will occur in the subsequent recognition process for the same person.

<Embodiment 3>
In the first and second embodiments, an example in which the face tracking is performed by a single camera has been described, but the present invention is also applicable when the face tracking is performed over a plurality of cameras. In the present embodiment, an example in which face tracking is performed across a plurality of cameras is shown.

FIG. 6 is a diagram showing a configuration of a system including an information processing system 700 (including the information processing system 100 and the information processing system 704) and an inter-camera person integration unit 701. 6, parts having the same meanings as in FIG. 1 are given the same numbers as in FIG. 1 and their description is omitted.

FIG. 6 is a block diagram showing the configuration of the information processing system 700. Reference numeral 704 indicates another information processing system having the same functional configuration.

Reference numeral 701 is an inter-camera person integration unit. The inter-camera person integration unit 701 investigates the correspondence between face images captured by a plurality of information processing systems 700 and determines the same person. That is, it is determined whether the face imaged by a certain information processing system 700 and the face imaged by another information processing system 700 belong to the same person. Further, the judgment result is managed in association with the tracking identifier sent from the information processing system 700. The registration exclusion information sent from the information processing system 700 is also managed. Details of the inter-camera person integration unit 701 will be described below.

The inter-camera tracking unit 702 has a function of using face detection images sent from a plurality of information processing systems 700 to establish a correspondence relationship with face images between a plurality of cameras. The method of associating face images between a plurality of cameras is not particularly limited. As described in the first embodiment, the method of taking correspondence using the brightness data and color information of the face image may be used. If the correspondence cannot be sufficiently determined only by the face image, the information processing system 700 may send a full-body image, and the correspondence may be determined based on the color of the clothes and the presence or absence of belongings. In the information processing system 700, the whole body area may be estimated based on the face detection result.

When it is determined that a plurality of faces sent from a plurality of information processing systems 700 belong to the same person, in order to identify the person, a tracking identifier (a tracking identifier set in the tracking between cameras is used as a tracking identifier between cameras). (Called an identifier).

The inter-camera tracking unit 702 sends the inter-camera storage unit 703 the inter-camera tracking identifier and the tracking identifier (both tracking identifiers determined to be the same) sent from the information processing system 700.

The inter-camera storage unit 703 manages the information on the tracking identifier and the registration-excluded person ID sent from the information processing system 700 in association with the inter-camera tracking identifier.

FIG. 6 is a table showing a management state of inter-camera tracking identifiers, tracking identifiers (of the information processing systems 700 and 704), and registered person exclusion IDs in the inter-camera storage unit 703. In FIG. 6, the inter-camera tracking identifier integTrackID_0 indicates that the person is managed as the tracking identifier TrackID_0 in the information processing system 700, and there is no registered person exclusion ID. Further, the inter-camera tracking identifier integTrackID_1 indicates that the information processing system 700 is a person managed as the tracking identifier TrackID_1, and the registered person exclusion ID is RegID_3. Further, the inter-camera tracking identifier integTrackID_2 is a person who is managed as the tracking identifier TrackID_2 in the information processing system 700 and as the tracking identifier TrackID_1 in the face recognition system 704. This indicates that the registered person exclusion ID is RegID_0. In addition, the inter-camera tracking identifier intTrackID_3 indicates that the person is managed as the tracking identifier TrackID_2 in the face recognition system 704 and that there is no registered person exclusion ID.

In the inter-camera storage unit 703, the registered person exclusion ID is managed in association with the inter-camera tracking identifier and the tracking identifier as described above. In the inter-camera storage unit 703, the registration exclusion ID is updated in association with the update of the registration exclusion information performed in the tracking identifier list storage unit 707 of the information processing system 700 (or 704). Further, the updated information is notified to the registration exclusion information management unit 707 of the face recognition system 704 (or 700). Upon receipt of the notification, the registration exclusion information management unit 707 updates the registration exclusion information based on the information.

In addition to the operation described with reference numeral 107, the tracking identifier list storage unit 707 exchanges the registration exclusion information with the inter-camera storage unit 703 as described above, and updates the registration exclusion information managed by itself.

With the above configuration, the present invention can be applied even when face tracking is performed across a plurality of cameras.

<Other embodiments>
In the above embodiments, no reference was made to the case where face recognition was correct recognition. In the case of correct recognition, the supervisor uses the information to start a service (for example, customer service according to the taste of the customer), but depending on the form of the service, the same tracking identifier may be used. Subsequent face recognition may not be necessary for the same person represented. Therefore, if the correct recognition is performed on a certain tracking identifier, it is possible to skip the subsequent face recognition.

In the embodiments described so far, the feedback from the supervisor is limited to the case of erroneous recognition. However, in the case of correct recognition as well, if feedback is given, it is possible to determine correct recognition. Alternatively, if there is no feedback as false recognition, it may be determined as correct recognition.

The present invention is also realized by executing the following processing. That is, the software (program) that realizes the functions of the above-described embodiments is supplied to the system or device via the network for data communication or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program. Alternatively, the program may be recorded in a computer-readable recording medium and provided.

100 Information processing device 101 Image input unit 102 Detection unit 103 Tracking unit 104 Personal feature acquisition unit 105 Personal recognition unit 106 Registered person list storage unit 107 Tracking identifier list storage unit 108 Display control unit 109 Feedback input unit

Claims (17)

An information processing apparatus for recognizing a person included in a captured image using a list in which face images of a person to be recognized are registered,
A face image included in the image of interest, and a recognition unit that recognizes the face image as a specific person based on the list,
An input unit for inputting that the recognition result of the face image by the recognition unit is incorrect,
The information processing apparatus, wherein the recognition means controls, in the recognition after receiving the input, not to recognize the input face image as an error in the recognition result as the specific person. The captured image is a part of a moving image,
When the image captured after the target image includes a face image corresponding to the face image included in the target image, the recognizing unit captures the target image after the target image based on the input. The information processing apparatus according to claim 1, wherein a face image included in an image is not recognized as the specific person. The recognition means recognizes the face image as the specific person based on the similarity between the characteristics of the face image of the person who is the recognition target and the characteristics of the face image included in the attention image. The information processing device according to claim 1 or 2. The information processing apparatus according to claim 3, wherein the recognition unit recognizes the face image included in the image of interest as the specific person when the similarity is included in a predetermined range. The information processing apparatus according to claim 3, wherein the features include a positional relationship between parts of a face and a size of the face image. The information processing apparatus according to claim 1, wherein the input unit is input by a user that the recognition result of the face image by the recognition unit is incorrect. Based on at least the attention image and the first image captured in a frame prior to the attention image, the face image included in the attention image and the face image included in the first image have a predetermined value. 7. The tracking device according to claim 2, further comprising a tracking unit for tracking the face by assigning the same identifier to the face image when the person is within a movable range within a time period. The information processing device described. 8. The tracking unit tracks a face by assigning the identifier to each of the face images when there are a plurality of face images included in the target image, according to any one of claims 2 to 7. The information processing device described. The recognizing unit is a person registered in the list acquired from the learned model and individual features that can identify an individual for the face included in the image of interest based on a learned model that recognizes a face included in the image. 9. The personal feature of each of the images is compared to recognize that the face image included in the image of interest is a person registered in the list, according to any one of claims 1 to 8. The information processing device described. 10. The display control means for controlling the display means to display a recognition result indicating that the face is the specific person by the recognition means, further comprising: Information processing equipment. The display control means controls to display a face to be recognized included in the target image and a face image of a specific person recognized by the recognition means side by side on the display means. 10. The information processing device according to 10. The display control unit controls the display unit that further displays the degree of similarity between the face image of the recognition target included in the target image and the face image of the specific person recognized by the recognition unit. The information processing device according to claim 10 or 11. The input means accepts that the face image is registered as a determination result by the user and is a second person different from the specific person.
The recognizing unit is the second face image included in the second image captured after the target image, and the second face image similar to the face image included in the target image is The information processing apparatus according to any one of claims 2 to 12, characterized in that it is recognized as a second person. The identification means further recognizes attribute information of a person corresponding to a face image included in the attention image,
The input means further receives a determination result as to whether or not the attribute information matches,
When the determination result determines that the attribute information of the face image included in the attention image is incorrect, the recognition unit does not recognize the face image as a person having the error attribute information. The information processing apparatus according to any one of claims 1 to 13, wherein the information processing apparatus is configured as described above. The method according to any one of claims 1 to 14, further comprising update means for updating the list so as to exclude the specific person from the recognition target for the face image included in the attention image based on the input. Information processing equipment. A program for causing a computer to function as each unit included in the information processing apparatus according to claim 1. An information processing method for recognizing a person included in a captured image using a list in which face images of a person to be recognized are registered,
A face image included in the image of interest, and a recognition step of recognizing the face image included in the image of interest as a specific person based on the list;
An input step of inputting that the recognition result of the face image by the recognition means is incorrect,
In the recognition step, in the recognition after receiving the input, the information processing method is controlled so as not to recognize the input face image as an error in the recognition result as the specific person.

Images