JP2022018808A - Information processing device, information processing method, and program - Google Patents

Abstract

To make it possible to suppress the recognition of a person detected in an image as a registered person by mistake.SOLUTION: An information processing device includes detection means for detecting a predetermined object from an image, acquisition means for acquiring a second object that is different from a first object registered in advance on the basis of spatial information representing the space in which the object was detected and the time at which the object was present in the space, and determination means for determining whether the object detected by the detection means is the first object or the second object.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing technique for processing an image.

In the monitoring system, another person having characteristics similar to the registered person may be erroneously detected as a registered person. Hereinafter, this is referred to as misrecognition. It is desired that detection and recognition be performed with high accuracy while effectively reducing false recognition. For example, in Patent Document 1, information on a similar person who caused the misrecognition when a misrecognition occurs is stored in association with the registered person, and the similar person is stored at the time of collation. A method of collating the information of the above is disclosed.

Japanese Unexamined Patent Publication No. 2008-257329

In cases such as city surveillance, monitoring is often performed by multiple surveillance cameras (multi-cameras), and in such cases, as the number of surveillance cameras increases, the number of false recognitions by others similar to the registered person increases. It is likely to increase. For example, when all similar persons causing erroneous recognition are registered and operated as in the method described in Patent Document 1, there is a possibility that they are erroneously recognized as similar persons.

Therefore, an object of the present invention is to prevent a person detected from an image from being mistakenly recognized as a registered person.

The information processing apparatus of the present invention is registered in advance based on a detection means for detecting a predetermined object from an image, spatial information representing the space in which the object is detected, and a time when the object exists in the space. An acquisition means for acquiring a second object different from the first object, and a determination means for determining whether the object detected by the detection means is the first object or the second object. It is characterized by having.

According to the present invention, it is possible to prevent a person detected from an image from being mistakenly recognized as a registered person.

Diagram showing an example of hardware configuration of an information processing system Flow chart showing the processing executed by the information processing device Flow chart showing the processing executed by the information processing device Diagram showing an example of a surveillance camera installation map The figure which shows the screen example when a person is detected Explanatory drawing of the window displayed when adding similar person information A diagram showing the distribution image of data in the feature space The figure which showed the screen display example which concerns on 2nd Embodiment Flow chart showing the processing executed by the information processing device Block diagram showing a functional configuration example of an information processing device

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown. The same configuration or processing will be described with the same reference numerals.
<First Embodiment>
In the first embodiment, an information processing system that monitors a wide area such as an urban area with a plurality of surveillance cameras will be described. In the information processing system of the present embodiment, when a predetermined object is registered in advance as a monitoring target and the registered object is photographed and detected by any of a plurality of surveillance cameras, alert information is generated. It shall be operated in a usage form in which it is issued and notified to the user. In the present embodiment, for example, a person is assumed as a predetermined object to be monitored, but the object is not limited to the person. In the information processing system of the present embodiment, information on a person's face image and gait characteristics is acquired as information derived from an image in a person, and personal information including such information and other information related to the person is registered. To. In the present embodiment, as the person registered in advance in the information processing system, for example, a wanted criminal, a suspicious person, an important person, or the like is assumed. Hereinafter, in the description of the present embodiment, a person who is a registered object registered in advance will be referred to as a registered detection target person.

FIG. 1 is a diagram showing a hardware configuration example of an information processing system to which the information processing apparatus of the present embodiment is applied.
The information processing system according to the present embodiment includes a plurality of surveillance cameras 101, 102, 103, 104, a recording server 11, an information processing device 12, an output device 13, and an input device 14. There is. In the example of FIG. 1, four surveillance cameras are connected to the information processing device 12 and the recording server 11, but the number of surveillance cameras does not have to be four, and may be several thousand or tens of thousands. However, it may be one.

The image data (hereinafter referred to as an image) of the moving image taken by each of the plurality of surveillance cameras is stored in the recording server 11 in a moving image format, and is also transmitted to the information processing apparatus 12 as an input image. In addition to the case where the moving image from the surveillance camera is recorded as it is, the recording server 11 also records the data processed by the information processing device 12 described below, the information handled by the information processing device 12, and the like. Various information such as information on registered objects, time information, spatial information, person information, labels, scores, and lists, which will be described later, are recorded on the recording server 11.

Next, the hardware configuration in which the information processing apparatus 12 is mounted will be described with reference to FIG. The information processing apparatus 12 includes a CPU 121, a RAM 122, a ROM 123, an output I / F 124, an input I / F 125, a bus 120, and the like. Further, the output device 13 and the input device 14 are connected to the information processing device 12. In the case of FIG. 1, the information processing device 12 assumes a personal computer to which the output device 13 and the input device 14 are connected. The present invention is not limited to this, and a device provided with an input / output device such as a tablet terminal or a smartphone may be used.

The CPU (Central Processing Unit) 121 comprehensively controls each device connected to the bus 120. The ROM (read-only memory) 123 stores an information processing program (application program), an operating system (OS), a device driver, and the like according to the present embodiment. The RAM (random access memory) 122 primarily stores the above-mentioned program, data in the process of processing, and the like at the time of processing by the CPU 121. The CPU 121 reads out the information processing step and other programs stored in the ROM 123, first stores them in the RAM 122, and executes the process according to each flowchart described later. As an alternative to the software processing using the CPU 121, the processing by the hardware composed of the arithmetic unit and the circuit corresponding to the processing of each functional unit described later may be performed.

The input I / F (interface) 125 converts the information input from an external operating device or the like into a format that can be processed by the information processing device 12. The output I / F 124 converts the output information into a format that can be processed by an external output device or the like.
Although only the minimum configuration is shown in FIG. 1, a GPU (graphics processing unit) may be provided as a device connected to the bus 120 in an environment where higher speed processing is required.

The output device 13 is a display device that displays the result processed by the information processing device 12. For example, when alert information is issued to notify a user such as a monitor when a registered detection target person is detected, the output device 13 is from a surveillance camera showing a person detected as a registered detection target person. A video or a registered image of a registered person to be detected is displayed. As a result, the user can confirm whether or not the person detected by the information processing apparatus 12 is a registered detection target person by observing the image displayed by the output device 13. Then, for example, when the registered detection target person is a wanted criminal, the user can take measures such as reporting to the police.

However, the information processing apparatus 12 may detect a person who is different from the registered detection target person, and erroneously recognize the other person as the registered detection target person. When such an erroneous recognition occurs, the user informs the information processing apparatus 12 that the recognition result is erroneous. In this case, the user gives a command to correct the recognition result displayed on the output device 13 via the input device 14 composed of a keyboard, a mouse, or the like (a command to teach that the recognition is erroneous). And commands for adjustment etc. can be input.

However, when monitoring by a plurality of surveillance cameras (monitoring by a multi-camera) as in the present embodiment, there is a high possibility that the number of erroneously recognizing another person similar to the registered detection target person will increase. Become. Further, for example, when all similar persons causing misrecognition are registered and operated as in the method described in Patent Document 1, the processing time becomes long as described above, and oversight may occur. become.

Here, for example, for a person who does not need to be compared with a registered person to be detected, if there is a mechanism that does not need to be compared as much as possible, not only the processing time can be reduced, but also the occurrence of oversight can be reduced, and the recognition accuracy can be improved. Can also be obtained.
For example, it is generally known that there is a tendency for a person's behavior pattern, and that the time zone and usage route of each person's daily activities outdoors do not change so much. Therefore, it is considered that it is possible to avoid conducting comparisons with unnecessary persons by reflecting this general mechanism.

As an example, consider an information processing system for moving images input from a large number of surveillance cameras spread all over the world. Further, for example, it is assumed that the registered detection target person A and the person P who is another person having characteristics similar to the person A are registered as similar persons associated with the registered detection target person A. In the case of the method described in Patent Document 1, the videos of all the surveillance cameras arranged all over the world are collated with the registered detection target person A and the person P registered as a similar person. , A huge amount of processing will be executed. Further, it is considered that there are many persons having characteristics similar to the registered detection target person A not only in the person P but also in various countries around the world, and the collation process is performed for these persons as well. Assuming that N similar persons are registered by collecting the characteristics of similar persons from all over the world, the characteristics of those N similar persons will also be collated, and the registered detection will be performed. Since the possibility of identifying the target person A is reduced, the number of oversights increases.

Here, it is assumed that the person P is observed by a specific surveillance camera installed in the district D every night, for example. When the area or time zone in which the person P is observed is limited to some extent in this way, processing using the video of each surveillance camera in a different area or time zone different from those areas or time zones is almost unnecessary processing. Is considered to be. Also, if the person P is detected from the video of each surveillance camera in a completely different area or time zone from the area or time zone where the person P is observed, it is very likely that the detection is erroneous. Conceivable.

Therefore, the information processing apparatus 12 of the present embodiment performs collation processing in consideration of the appearance likelihood based on the area and time zone in which another person P having a characteristic similar to the registered detection target person A appears. As a result, the wasteful processing amount and processing time for the moving image in the area or time zone in which the person P hardly appears can be significantly reduced. Similarly, it is possible to significantly reduce the fact that the person P is erroneously recognized as the registered detection target person A and that the registered detection target person A is overlooked.

An example of the functional configuration of the information processing apparatus 12 will be described with reference to FIG. The information processing device 12 has an image acquisition unit 1201 and a detection unit 1202. The detection unit 1202 further includes an object detection unit 12020, a spatial information acquisition unit 12021, a similar object information acquisition unit 12022, a collation unit 12023, a generation unit 12024, a reception unit 12025, and an update unit 12026. Further, the information processing device 12 is connected to a plurality of external devices.

Here, an outline of the processing of each functional configuration will be described. Details will be described together with the flowchart.
The image acquisition unit 1201 acquires an image captured by an image pickup device (surveillance camera 101 (to 104)). The detection unit 1202 detects a pre-registered registered object (first object) from the object (person) detected from the acquired image. The registered object refers to the person to be detected in this task. Specifically, it is assumed that a person is wanted or is being searched for by a lost child.

The detection unit 1202 further executes the following processing. The object detection unit 12020 detects a person from the acquired image. The spatial information acquisition unit 12021 acquires information indicating the space and time when the currently detected person is detected. Based on the acquired space and time, the similar object information acquisition unit 12022 misrecognizes in the past by the image acquired from the surveillance camera corresponding to the spatial information s (location) in the time zone close to the time t. Acquires similar object information about the person who was made. The collation unit 12023 collates the information about the detected person with the information about the registered detection target person A (first object) and the information about the person B (n) which is a similar object. The generation unit 12024 generates output information indicating a determination result as to whether or not the detected person is collated with the first object or the second object. The reception unit 12025 receives a correction instruction by the user regarding the determination result. That is, the reception unit 12025 receives information as to whether or not the recognition result for the detected person is erroneous recognition. If the recognition result is erroneous, the update unit 12026 updates the threshold value in the collation process.

FIG. 2A is a flowchart showing a process executed by the information processing apparatus 12 of the present embodiment. It shows the process repeated from the image input from the surveillance camera to the end of surveillance. Further, FIG. 2B is a detailed flowchart of the detection process of the registered detection target person performed in step S21 of the flowchart of FIG. 2A. FIG. 3 is a detailed flowchart of the recognition result display process performed in step 215 of the flowchart of FIG. 2 (B). The details of the process will be described later.

Here, as an example in which the effect of the present embodiment is more clearly exhibited, a case where a plurality of surveillance cameras take pictures of different roads for city surveillance, and a case where a plurality of surveillance cameras are used for in-hospital surveillance, respectively. The explanation is based on the assumption that a different floor will be photographed. In addition, it is assumed that the people photographed by each of these surveillance cameras have different tendencies such as facial images and gait characteristics. The information processing system according to the present embodiment is not particularly limited in the situations where it is used as long as it is a detection targeting a person. For example, it can be applied to cars and animals. In the following, for the sake of better understanding and explanation of the effect, city monitoring will be explained as an example.

FIG. 4 is a diagram showing an installation example of a plurality of surveillance cameras in city surveillance.
Here, it is assumed that moving images from 14 surveillance cameras C301 to C314 are processed at the same time. The surveillance cameras C301 to C314 correspond to the surveillance cameras 101 to 104 in FIG. Further, the number of registered detection target persons may be one, but may be several thousand or tens of thousands.

First, in step S20 of the flowchart of FIG. 2A, the image acquisition unit 1201 acquires images (surveillance images) of moving images taken from the surveillance cameras C301 to C314, respectively.
Next, in step S21, the detection unit 1202 detects a pre-registered registered object (first object) from the object (person) detected from the acquired image.
Then, the information processing apparatus 12 continues the processing of steps S20 and S21 until the user inputs an instruction to end monitoring in step S22.

In step S210 of the flowchart of FIG. 2B, the detection unit 1202 executes a detection process of detecting a person as a predetermined object from input images sent from a plurality of surveillance cameras C301 to C314, respectively. For example, the detection unit 1202 detects the position of a person in an image using a trained model that has learned the characteristics of the person. Other methods may be used.

Next, in step S211 the object detection unit 12020 determines whether or not a person has been detected by the person detection process in step S210. If the person is not detected, the information processing apparatus 12 does not perform the processing after step S212, and proceeds to step S22 of FIG. 2 (A). On the other hand, when it is determined that a person has been detected, the information processing apparatus 12 performs a collation process to see if a person corresponding to the registered detection target person has been detected by the processing in step S212 and subsequent steps.

In the collation process after step S212, the detection unit 1202 not only compares and collates whether the person detected in step S210 is a registered detection target person, but also erroneously recognizes the person as a registered detection target person in the past. It also collates with the person who was made. Further, whether or not the information processing apparatus 12 of the present embodiment collates a person who has been erroneously recognized in the past is currently detected with information indicating the space and time when the person who was erroneously recognized in the past was detected. It is determined based on the information indicating the space and time when the person is detected. In the present embodiment, the space in which a person is detected is a space representing a shooting area of a surveillance camera that has shot the person. The time when the person is detected is the time when the person exists in the space where the person is detected, and is the time when the person is photographed by the surveillance camera. Therefore, the spatial information acquisition unit 12021 obtains a shooting area (location) specified by the information at the time t when the currently detected person was shot and the identification information (camera ID) of the surveillance camera that shot the person. The spatial information s to be represented is acquired. The spatial information s includes not only the shooting area specified by the camera ID but also an identifier representing a plurality of shooting areas shot by a plurality of surveillance cameras installed in the vicinity of the surveillance camera of the camera ID. You may.

Here, in the present embodiment, the meaning of retaining the person information of another person (hereinafter, similar object or second object) that has caused misrecognition in the past will be described. Hereinafter, another person similar to the registered person will be referred to as another person B. If the other person B is photographed again by the surveillance camera of this information processing system on another occasion, it is very likely that the person B will be detected again as the registered detection target person A and unnecessary alert information will be output. Therefore, in order to simply reduce the misrecognition caused by another person B, the person information misrecognized in the past like another person B for the registered detection target person A is retained in association with the registered detection target person A. I will deal with it by keeping it.

When associating, a similar person P who is newly detected by the information processing system and is considered to be a registered detection target person A is compared with another person B at the same time, and the feature quantities are closer to each other. It can be determined that it is one person. As a result, it is possible to improve the accuracy of whether or not the person is truly a registered detection target person A, and at least reduce erroneous recognition derived from another person B.

However, as a practical problem when monitoring and operating a wide area with many surveillance cameras, a person like another person B who has characteristics similar to the registered detection target person A is proportional to the number of surveillance cameras. Increases the frequency of detection. When these different persons are set for all the registered detection target persons, comparison and collation with all of the registered detection target persons is performed every time a person similar to the registered detection target person is detected. In this case, the following problems are likely to occur. For example, if the variation of similar persons is larger than the original variation in the feature space of the registered detection target person, the possibility that the detection accuracy is lowered increases. Further, in this case, the processing time increases due to the increase in the number of comparisons and collations.

Therefore, in the information processing apparatus 12 of the present embodiment, in step S212, the spatial information acquisition unit 12021 acquires information indicating the space and time in which the currently detected person is detected. By this processing, the occurrence of the erroneous recognition is suppressed according to the appearance likelihood of another person B. In the present embodiment, the time t and the spatial information s (location) at which the other person B is detected are used as the appearance likelihood of the other person B. The spatial information acquisition unit 12021 limits the persons to be collated based on the time and spatial information when the person is detected and the time and spatial information when another person B appears. Therefore, when the person is detected, the spatial information acquisition unit 12021 acquires the time t at which the person was photographed and the spatial information s corresponding to the surveillance camera that photographed the person.

Further, in step S213, the similar object information acquisition unit 12022 describes an object erroneously recognized in the past by an image acquired from a surveillance camera corresponding to spatial information s (location) in a time zone close to the time t. Acquire information (information about a second object). An object misrecognized in the past is a similar object having characteristics similar to a registered object, and refers to an individual different from the registered object.
Then, in step S214, the collation unit 12023 uses information about the detected person, information about the registered detection target person A (first object), and information about the similar object B (n). Match with. That is, the collation unit 12023 collates the person who was erroneously recognized in the past, the person detected in step S210, and the registered detection target person. The method of collation processing will be described below, but as an example, the distance between the feature vector of the detected object and the feature vector indicating the first object or the feature vector indicating the second object is calculated. The smaller the distance, the more similar it can be determined.

The reason why such a process is effective is that many people tend to repeat similar movements when focusing on the movement pattern of a person. For example, the route used by a specific person for the movement from the specific point R to the point Q is often the same route even though there are a plurality of possibilities. Similarly, a person observed in one time zone tends to be more likely to be observed in the same time zone on another day. These mean that there are repeated patterns in human behavior, and it is not necessary to search from infinite possibilities when predicting human behavior, and it is effective to make predictions based on repeated patterns. is doing. Therefore, for example, even in a registered detection target person, there is a high possibility that the same routine movement is repeated every day, and it is considered easy to monitor the person. However, it is possible that there is a motive for accurately detecting the registered person to be detected even if a random movement different from that of the routine is performed. On the other hand, many of the other non-detection target persons other than the registered detection target persons repeat similar movements every day, so it is considered that the above-mentioned routine-based detection works to some extent. Therefore, in the present embodiment, as the information regarding the space in which the person appears as described above, the space information s representing the shooting area of the surveillance camera in which the person is photographed is acquired. That is, spatial information about a person is paraphrased as information indicating routines and periodic habits about that person. Using information on the periodicity of each person (spatial information), a sample group of people who collate with the detected person is extracted from the feature space whose population is the registered person and the person detected in the past.

Specific examples of the temporal and spatial constraints (spatiotemporal constraints) as described above will be described.
As an example of the time constraint condition, when the time when the person P is detected is set to time t (P), the time t (P) is set as the time threshold for determining the person P as the same person. It is assumed that the sandwiched time zone Tth is used. The time zone Tth is 30 minutes or the like. Further, the time t (P) is set to the central time within the time zone Tth, but is not limited to the central time. The time zone Tth represents a time zone close to the time t (P), and the user can define in advance what time zone is to be the close time zone in terms of time, but the system can be defined in advance. It may be defined in.

Further, for example, as an example of spatial constraints, only the spatial position coordinates defined by the surveillance camera that captured the person P are used as the spatial information s. On the map shown in FIG. 4, the spatial position coordinates defined in all the surveillance cameras within a specific distance range from the surveillance camera in which the person P is detected may be used as the spatial information s. The spatial information s is the spatial position coordinates defined by the surveillance camera that captured the person P, and the range of the spatial information can be defined in advance by the user, for example. It may be defined in the system in advance.

Further, the above-mentioned movement pattern of a person does not include the validity of the movement route, and as the simplest example, a method of determining whether or not the positions of the surveillance cameras are close to each other (for example, determining 0 or 1). Can be used. In addition, there is a probabilistic continuity in the time and space in which a person appears. Therefore, for example, when calculating the score of another person B's uniqueness, not only the feature amount extracted from the person but also the temporal distance from the time when the other person B was observed in the past and the spatial distance from the observed space are reflected. You may find the probability of making it. When determining the likelihood considering the temporal distance and the spatial distance at the same time, it is desirable to consider the camera topology and the spatiotemporal transition probability.

For example, it is conceivable that the camera topology and the transition probability are sequentially learned and used while performing person collation by the method described in Reference 1 below. In general, the spatiotemporal transition probability is stochastically determined whether or not a person is the same person in person tracking by considering the average travel time, distance, and route of a passerby for a person who seems to be the same person. In this case, two people who are paired in a space-time close to each other are compared and associated. However, the two to be compared in the example of this embodiment are the currently observed person P and the previously observed person B. Here, if there are a plurality of similar persons B (n = 1 ... N) observed in the past to be compared with the person P, they are distinguished by n and expressed as the person B (n).

Reference 1: A. Gilbert and R. Bowden: "Incremental, scalable tracking of objects inter camera", Computer Vision and Image Understanding, 111, 1, pp. 43-58 (2008).

The value of the spatial distance may be reflected in the probability, such that the farther the distance is, the lower the probability of being the same person. However, the treatment of temporal distances is practically different. Human behavior on the time axis tends to repeat a specific pattern at a specific time interval (often a 24-hour cycle). Specifically, the behavior of a specific individual at time t is closer to the behavior after 24 hours or 72 hours than the behavior after 3 hours. Therefore, in the present embodiment, the temporal distance is defined by the difference of only the time regardless of the date when the two parties are observed.

The collation unit 12023 of the present embodiment calculates the product of the likelihood obtained as described above with the similarity defined by the feature quantity extracted from the two comparison targets, thereby temporally and spatially. The similarity S (n) is derived in consideration of the appropriate validity.

Then, in step S215, the generation unit 12024 generates output information indicating a determination result as to whether or not the detected person is collated with the first object or the second object. In step S2151 of the flowchart of FIG. 3 showing the details of step S215, the generation unit 12024 searches for whether or not there is a person who exceeds the threshold value among the person B (n) who is finally a candidate for the person P. do. Details of the processing of the flowchart of FIG. 3 will be described later.

Further, the generation unit 12024 uses the person having the highest similarity S (n) among the candidates searched in step S2151 as the determination result of the detected object. Thereby, it is possible to compare two scores of the similarity calculated from the highest similarity S (n) and the registered detection target person A. That is, in step S2152, the generation unit 12024 can determine whether the person P is the registered detection target person A or the person B (n).

Here, as the collation processing method, any method may be used as long as it is a collation method using the feature amount of the detected person, but as an example, the face recognition method described below can be used. When the face recognition method is used, the collation unit 12023 further identifies the face region from the person region detected in step S210. When the face area is detected, the collation unit 12023 determines the person in the face area image based on the face area image and the face image of the registered detection target person. The generation unit 12024 generates information indicating the collation result of the object detected by the collation process.

Any method may be used for comparison and collation processing between facial images. For example, feature extraction processing (extraction of the person's personality) is performed on both the face area image of the detected person and the face image of the registered detection target person. Then, comparison and collation are performed by calculating the distance between the feature vectors obtained as a result of the feature extraction process. As the feature extraction process, a method using DNN (Deep Neural Networks) by deep learning may be used, and the distance between feature vectors may be calculated by using the Euclidean distance or the cosine distance.

Therefore, the generation unit 12024 of the present embodiment has a face image of a person detected according to the distance obtained as described above, a face image of a registered detection target person, or a face of a person erroneously recognized in the past. The similarity score with the image can be calculated.
Here, the similarity score is an evaluation value indicating that the similarity between faces increases as the score increases. Therefore, in the case of the cosine distance, the similarity score may be used as it is, or in the case of the Euclidean distance. May use its reciprocal as a similarity score. In the comparison collation process using only the face image, the similarity by face recognition cannot be measured unless the face image area is detected. In that case, the face of the registered detection target person or the person who was erroneously recognized in the past as the collation result. It does not match the face of. Therefore, in addition to the face recognition method, the collation process may be collated using a similarity score defined by a feature quantity obtained from a human body region, for example, as in Reference 2 below. Alternatively, a collation may be performed using a score defined by a feature amount obtained from a gait obtained from a series of moving images by a method as in Reference 3 below. Further, various scores may be combined and operated, such as taking the sum of the similarity scores described above.

Reference 2: M.I. Farenza et al. , Person Re-Identification by Symmetry-Driven Accumulation of Local Features, CVPR2010.
Reference 3: Murase, Moving object recognition in eigenspace representation: gait analysis and lip reading, Pattern Recognition Letters, 1996.

Returning to the description of the flowchart of FIG. 2 (B).
As a result of the collation process in step S214, the likelihood that the person detected in step S210 is either a registered detection target person or a person misrecognized in the past is defined based on the similarity score described above. To. Usually, the likelihood may be the similarity score itself described above.

Then, in step S215, the generation unit 12024 generates a display image showing the determination result of the collation unit 12023. A display image showing the result determined by the above-mentioned similarity is displayed on the output device 13. Hereinafter, the flowchart of FIG. 3 showing the details of step S215 will be described.

First, in step S2151, the generation unit 12024 determines whether or not any of the above-mentioned similarity scores exceeds the threshold value in the collation result. If none of the generation units 12024 exceeds the threshold value, the information processing apparatus 12 ends the process of FIG. 3 and proceeds to step S216 of FIG. On the other hand, when there is one or more similarity scores exceeding the threshold value, the generation unit 12024 proceeds to the process in step S2152.

Proceeding to step S2152, the generation unit 12024 refers to the person to be collated having the highest similarity score. Then, the generation unit 12024 determines whether or not the person with the highest similarity score is the registered detection target person. At this time, if the person with the maximum similarity score is not the registered detection target person, the generation unit 12024 determines that the similarity score with the person erroneously recognized in the past is high. In this case, the detection unit 1202 outputs information to the output device 13 that there is no registered person. Then, the generation unit 12024 ends without updating the screen display state. On the other hand, if the person with the highest similarity score is the registered detection target person, the generation unit 12024 proceeds with the process in step S2153.

Proceeding to step S2153, the generation unit 12024 causes the output device 13 to display the image of the registered detection target person having the maximum similarity score in the previous step on the screen, and ends the processing of the flowchart of FIG. For example, when the detected object is similar to the registered person (similarity is greater than a predetermined value), the generation unit 12024 generates information indicating that the detected object is the registered person (first object). do. Further, when the detected object is similar to a similar person (second object), the generation unit 12024 generates information indicating that the detected object is a similar person (second object). When the detected object is not similar to either the registered person (first object) or the similar person (second object) (similarity is smaller than a predetermined value), the generation unit 12024 may perform the registered person (first object). Generates information indicating that it is neither an object) nor a similar person (second object).

FIG. 5 is a diagram showing an example of a screen displayed on the output device 13 in step S2153. FIG. 5 is an example of a screen for notifying the user that the registered detection target person has been detected.
In FIG. 5, for example, an image of a registered detection target person A having the highest degree of similarity to a currently detected person (hereinafter referred to as person P) is displayed in the area 41. In the area 42, a map for grasping the positional relationship between the location of the person P and the surveillance camera is displayed, and this map is similar to the one shown in FIG. 4 described above. Looking at the map displayed in the area 42, the position of the person being tracked, such as when a registered detection target person is detected, is displayed in real time on the map. In the area 43, the image of the person P having a high degree of similarity to the registered detection target person A is displayed.

By displaying the screen as shown in FIG. 5, the user compares the image of the registered detection target person displayed in the area 41 with the image of the person P displayed in the area 43, and is it likely to be the same person? I can judge. When it is difficult to immediately determine whether or not the person is the same person, when the user selects the tracking button 44, the information processing apparatus 12 tracks and displays the person P displayed in the area 43 for a while.
When the user selects one of the surveillance cameras (surveillance cameras C301 to C314 in FIG. 4) on the map of the area 42, for example, with a mouse pointer or the like, the information processing apparatus 12 displays the image to be displayed in the area 43. It shall be switched to the image by the selected surveillance camera.

As the tracking method, the method described in Reference 2 or the like described above can be used, and the same person can be tracked by the tracking method. For example, monitoring in which the person P is photographed when the person P moves. Continue tracking even if the camera changes. Also, for example, when the user wants to see the past tracking information, the user moves the slider bar 49. In this case, the information processing apparatus 12 acquires the past moving image corresponding to the position of the slider bar 49 from the recording server 11. Then, when the user presses, for example, the play button 47, the information processing apparatus reproduces the moving image from the specific time point to the present time point according to the position of the slider bar 49 and displays it on the output device 13. When the user presses the stop button 48, the information processing apparatus 12 displays the image displayed when the stop button 48 is pressed in a paused state. As described above, according to the present embodiment, the user can view the video from a specific time point in the past to the present time point, for example, the video before the same person is registered and the score of the detection target person-likeness exceeds the threshold value and the alert information is output. You can browse. Further, according to the present embodiment, it is possible to examine and confirm in detail whether or not the person is a registered detection target person by the video paused and displayed at a desired time point. Further, when the image control instruction is given by the play button 47, the stop button 48, or the slider bar 49, the information processing apparatus 12 of the present embodiment is tracked on the map of the area 42 in response to the image control instruction. The position of the person also changes.

Further, in the present embodiment, since processing in real time is assumed, the image displayed in the area 43 is a real-time image taken by the surveillance camera at the present time, but when the image is confirmed retroactively. However, the same display may be made.
Further, in the processing of the above-mentioned flowchart, an example is given in which a person having the highest similarity is regarded as a detected candidate and displayed. For example, it is assumed that all registered detection target persons whose similarity exceeds the threshold value are detected. It may be regarded as full display, and the same processing may be performed thereafter.

The explanation is returned to the flowchart of FIG. 2 (B).
After step S215 described above, when the process proceeds to step S216, the reception unit 12025 accepts corrections regarding the determination result of the object detected by the user. That is, the reception unit 12025 receives an instruction from the user to determine whether the recognition result in step S215 in the previous stage is correct or incorrect. Specifically, the information processing apparatus 12 includes a person displayed in the area 41 of FIG. 5 and a person P (a person being tracked) displayed in the area 43 due to the confirmation work performed by the user on the screen display. It is assumed that the same person is confirmed and, for example, the report button 46 is pressed. In FIG. 5, it is assumed that the city is monitored by the information processing system and the registered detection target person A is a wanted criminal. When the report button 46 is pressed by the user, the information processing device 12 is set. For example, call a nearby police station. This is an example, and in the case where the store or the like is monitored by the information processing system, the information processing apparatus 12 notifies the store staff when the report button 46 is pressed. As described above, the report button 46 is used for reporting to different report destinations depending on the purpose of monitoring, and can be arbitrarily customized and used.

Next, when the process proceeds to step S217, the reception unit 12025 receives information as to whether or not the recognition result for the detected person is erroneous recognition. Here, when the report button 46 is pressed by the user, it is considered that the user has confirmed that the detected person is the same person as the registered detection target person, and the recognition result for that person can be regarded as correct. .. In this case, since it is not necessary to make any changes to the subsequent processing, the information processing apparatus 12 ends the processing of the flowchart of FIG. 2 (B) and performs the processing in step S22 of FIG. 2 (A). Proceed. On the other hand, when the erroneous detection button 45 is pressed, it is considered that the user determines that the detected person is different from the registered detection target person. Therefore, when the erroneous detection button 45 is pressed, the information processing apparatus 12 proceeds with the process in step S218, assuming that the user has determined that the registered detection target person has been erroneously recognized.

In step S218, when the user accepts the correction of the determination result, the update unit 12026 updates the determination criterion in the determination based on the correction. That is, the information processing apparatus 12 receives information on whether the recognition result is correct or incorrect from the user, and when the user corrects the information on the detected person, the information processing apparatus 12 updates the determination criteria. In the case of determination based on similarity, the determination criterion update process may be aimed at suppressing the occurrence of erroneous recognition by adjusting the threshold value. The process of updating the determination criteria may be performed by re-learning the discriminator described above. If there is no erroneously recognized person associated with the registered detection target person A, it means that the person is erroneously recognized for the first time. Therefore, the update unit 12026 records the person information (feature amount, image, time, spatial information) as it is in the recording server 11 for the person who is erroneously recognized for the first time.

Further, when the determination result by the user is not the registered detection target person but another person, it is not necessary to issue the alert information that the user is the registered detection target person. However, based on the fact that the person has the highest similarity S (n) among all the different people B (n) observed in the past, the person P is one of the different people B (n) observed in the past. It is very possible that the person recognized as k) is not another person B (k). This is the case where the person P is another person C who has not been observed in the past. If it is a new person C, it is necessary to urge the user to newly register the person information of this person C.

As a method of registering new person information of another person C, the feature amount extracted from the person P is separated from the feature amount extracted from all the registered other person B (n) in the feature space by a preset threshold value or more. A method is used in which the case is used as a criterion. Specifically, for example, using the screen of the window 50 as shown in FIG. 6A, additional registration of a new person C is performed. In the window 50, a person similar to the registered detection target person is a different person by comparing the person image of another person registered in the past, or the person is a new person because the corresponding registered person does not exist. Whether or not it can be accepted from the user. Then, the update unit 12026 additionally registers the new person information of another person C as the person information of the person similar to the registered detection target person in response to the instruction from the user via the window 50.

In the window 50 shown in FIG. 6A, candidates of different persons are displayed in order of score. In the example of FIG. 6A, images of two people are displayed as candidates, but more person images may be displayed as long as they are in order of score. Further, in this display example, not only the face image of a person who has been erroneously recognized in the past but also the time when the person was photographed and the camera ID of the surveillance camera are displayed to facilitate the user's judgment. The face image displayed at this time is an image obtained from an image taken by a surveillance camera in the past.

Then, if the person P is the same person as any one of the plurality of persons detected in the past displayed in the window 50, the user can use the plurality of persons displayed in the window 50. Select the person who is judged to be correct from the list. On the other hand, when the corresponding person does not exist, the user selects [Not applicable] prepared in the window 50. When [Not applicable] is selected by the user, the update unit 12026 registers the person P as a newly recognized person and the n of another person B (n) as N + 1. As a result, in the flowchart of FIG. 2B, the collation process according to step S214 is updated from the next loop.

Depending on the rules of the operation destination of the information processing system, it may be desired to operate without saving the images of ordinary people other than the registered detection target person. In this case, among the information displayed in the window 50, the image related to the person may not be displayed. In the case of this example, the information processing apparatus 12 displays the window 51 shown in FIG. 6 (B). With this window 51, the user can appropriately determine whether the person P is a person who has been repeatedly seen in the past or a person who is seen for the first time, even if the person image is not displayed. ..

In the present embodiment, a method of correctly labeling and operating the same individual and others among the registered misrecognized different person B (n) is described mainly by the screen of FIG. 6 (A). As described above, even if it is another person who is not the detection target, there are a plurality of effects obtained by teaching that a similar person is correctly the same person or a different person.

For example, a case where two persons B (r) and B (s) similar to the registered detection target person A exist will be described as an example. It is assumed that the user inputs these correctly as different people. In this case, the information processing apparatus 12 separately collates the highly likely time zone and spatial information in which another person B (r) appears with the highly likely time zone and spatial information in which another person B (s) appears. By doing so, the correct recognition result can be obtained.

Further, in this case, it is possible to separately hold the variation in the feature amount derived from another person B (r) and the variation in the feature amount derived from another person B (s) in the feature space. Therefore, the information processing apparatus 12 can improve the detection accuracy of the registered detection target person A.
FIG. 7 is an image diagram showing a feature space. In the example of FIG. 7, both another person B (r) and another person B (s) have been observed a plurality of times in the past, and the feature positions extracted for each of them are represented by black circles and white circles in the figure. Further, in the example of FIG. 7, it is assumed that the variation of the feature amount derived from another person B (r) is large in the feature space, and the variation of the feature amount derived from another person B (s) is relatively small in the feature space. .. Further, it is assumed that the distribution of the feature amount derived from another person B (s) is included in the distribution of the feature amount derived from another person B (r).

In the case of the feature amount distribution as shown in FIG. 7, it is difficult to determine that the two are different persons only by the feature amount without the judgment by the user. For example, if there is no judgment by the user, the same label (referred to as v) will be automatically given to the two parties. Here, for example, it is assumed that the time when another person B (r) is observed is 9:00 am and the time when another person B (s) is observed is 7:00 pm. At this time, if the same label v is given to the two persons, the two persons will be treated as the person B (v) without distinction, and the person B (v) will be treated as the person B (v) on the system. Retained as appearing at 9 and 7 pm. Then, during operation, for example, a person P detected as having a feature close to the registered detection target person A is detected at 7:00 pm, and the extracted feature amount is the position of the white circle P in FIG. 7 in the feature space. If so, it is determined that the person P is likely to be another person B (r). On the other hand, assuming that the judgment is made by the user and the different person B (r) and the different person B (s) are judged as different persons, in the feature space of only the person B (r) at 7:00 pm when the person P is detected. It is not necessary to consider the distance to the points in the distribution area. In this case, it suffices to collate with respect to the person P whether it is another person B (s) or the registered detection target person A.

However, depending on the operation method, there is a case where it is desired to reduce the user cost by reducing the labor for visual confirmation of the user rather than the strict detection accuracy. In that case, as described above, the operation may be performed without assigning a label that the other person B (r) and the other person B (s) are strictly different persons. In this case, the display as shown in FIG. 6A is not performed, or even if the display is displayed, the user skips the input work by turning off the window 50, and the system automatically automatically selects all the misrecognized persons. Correspond by the process of adding to another person B (n). That is, in this case, even if another person B (r) and another person B (s) are held as different person information, the data that they are actually the same person will increase. As a result, the time zone / spatial information with high likelihood of appearance of another person B (r) and another person B (s) is specified, and the distribution state of each feature amount in the feature space is not distinguished, and all are the same. Correspond by performing comparison and collation with the registered detection target A as personal information of another person.

As described above, the detection process of the registered detection target person described above is continued until the user inputs the end of monitoring in step S22 of FIG.
As described above, according to the information processing apparatus 12 of the first embodiment, when a person pre-registered by a surveillance camera is detected and recognized, the false recognition rate is reduced and further processing is performed while suppressing the occurrence of oversight as much as possible. It is possible to suppress the increase in cost.

<Second embodiment>
In the first embodiment, in order to enable the detection of a specific registered detection target person efficiently and with high accuracy, the spatiotemporal information of the appearance pattern of the person having characteristics similar to the registered detection target person is used. The second embodiment also utilizes the property that a person usually repeats the same routine every day as in the first embodiment, and further defines that the routine repeated every day is a behavior in a normal state.

In the second embodiment, not only the detection of the registered specific object but also the unregistered object can be detected with low cost and high accuracy, and the detection result can be presented to the user. Also in the second embodiment, a person is mentioned as an object. In the following description, a specific registered person is referred to as a registered person (first object), and a normal routine behavior of a registered person is referred to as a normal behavior. In addition, behavior in an abnormal state that deviates from the normal routine by a registered person is called abnormal behavior. Further, in the present embodiment, the abnormal state includes a state in which an unregistered person appears, and the abnormal state caused by the appearance of the unregistered person is also referred to as an abnormal behavior. Further, in the second embodiment, when a person with abnormal behavior is detected, that person is set as a person requiring monitoring. The configuration of the information processing system according to the second embodiment is the same as that of FIG. 1 described above. Further, the functional configuration of the information processing apparatus is the same as that of FIG. 10 described above.

In an information processing system for abnormal behavior in city monitoring, for example, among the feature quantities obtained from images of a surveillance camera, the feature quantity related to the abnormal behavior is learned in advance, and the feature quantity related to the abnormal behavior is detected during operation. If abnormal behavior occurs, alert information is issued. Alternatively, the information processing system learns a normal model using the features related to normal behavior in the registered person, and when the features deviating from the normal model are acquired, the information processing system alerts that the abnormal behavior is detected. Give out information. However, since it is too late to issue alert information after abnormal behavior is detected, there is also a need to detect signs of abnormal behavior. However, the detection of signs of abnormal behavior is more difficult than the detection of abnormal behavior, and since there are large individual differences, it is often the case that the expected effect cannot be obtained in practical use.

Therefore, in the second embodiment, the registered person and the behavior thereof, which tend to be repeatedly observed in a specific time zone and space, are regarded as normal behavior. On the other hand, when a person who tends not to be observed in a specific time zone and space is detected, the person is a person with abnormal behavior and alert information is issued assuming that the person is a person requiring monitoring. This facilitates the detection of signs of abnormal behavior in the second embodiment.

FIG. 8 is a diagram showing an image of a display screen of the output device 13 when detecting a person requiring monitoring and issuing alert information in the information processing device 12 of the second embodiment.
In FIG. 8, the area 61 is an area for displaying an image taken by a surveillance camera. The detection unit 1202 considers that only a registered person who repeats the same routine every day during this time period is in a state of normal behavior when it is shown in the image from the surveillance camera. On the other hand, when a person different from a person who repeats the same routine every day appears in the image from the surveillance camera in that time zone, the detection unit 1202 detects the person as a person requiring surveillance. Further, for example, when the behavior of the registered person is different from the daily routine in that time zone, the detection unit 1202 also detects the registered person as a person requiring monitoring.

Then, the detection unit 1202 encloses the image of the person to be monitored by the frame 62 and generates a display so as to call attention to the user. Further, the detection unit 1202 displays an enlarged image of the person to be monitored in the area 63, and further outputs a warning sound to notify the user. As a result, the user can determine by looking at the screen whether the person is the person who should continue the monitoring and whether the behavior of the person is the behavior which the monitoring should be continued. Here, when the user determines that the person is a registered person who repeats the same routine every day and presses the erroneous detection button 65, the information processing apparatus 12 assumes that the person is erroneously recognized. Is processed in the same manner as in the first embodiment described above. On the other hand, when the user determines that the person needs to be continuously tracked and presses the tracking button 64, the detection unit 1202 resembles the example of the first embodiment described above. Track by method. Further, when the user determines that the person is likely to cause an abnormal behavior and presses the report button 66 while continuing to monitor the person to be monitored, the detection unit 1202 uses the above-mentioned first method. Report in the same manner as in the embodiment of.

FIG. 9A is a flowchart showing the process executed by the information processing apparatus 12 of the second embodiment, and shows the process repeated from the image input from the surveillance camera to the end of the surveillance. Further, FIG. 9B is a detailed flowchart of the detection process of the person to be monitored, which is executed by the detection unit 1202 in step S71 of the flowchart of FIG. 9A. As with the first embodiment, the information processing system of the second embodiment is not particularly limited in the situations in which it is used, but for the sake of understanding and explaining the effects, monitoring of the city area is an example. Listed as. Further, also in the second embodiment, it is an example that 14 cameras of the surveillance cameras C301 to C314 shown in FIG. 4 are processed at the same time.

First, in step S70, the image acquisition unit 1201 acquires images captured by the surveillance cameras from the surveillance cameras C301 to C314, respectively.
Next, in step S71, the detection unit 1202 starts the detection process of the person requiring monitoring. The detection unit 1202 detects a pre-registered person to be monitored (first object) from the object (person) detected from the acquired image. The first object includes a registered predetermined object (similar to the first embodiment) and an arbitrary object in an abnormal state. That is, the monitoring target in the present embodiment is an arbitrary object that performs abnormal behavior, that is, an object that is not in a normal state. Since it is not possible to determine whether the behavior of an object that has not been registered is normal or abnormal, it is detected as a person to be monitored.
Then, the information processing apparatus 12 continues the process of step S71 until the user inputs an instruction to end monitoring in step S72.

Steps S710 to S712 of FIG. 9A are substantially the same as the processes of steps S210 to S212 of FIG. 2B described above, and the description of these similar processes will be omitted.
In step S711, if the person is not detected, the object detection unit 12020 proceeds to the process of step S716. In step S716, the generation unit 12024 generates a normal screen on the output device 13, that is, a normal screen in which the person requiring monitoring is not detected, and causes the output device to display the screen. On the other hand, when a person is detected, the detection unit 1202 performs a collation process for checking whether the person detected in step S710 is a person requiring monitoring by the processing after step S712. Hereinafter, the person detected in step 710 is referred to as a person U.

In the collation process after step S712, the spatial information acquisition unit 12021 represents a shooting area specified by the time t at which the person U detected in step S710 was shot and the camera ID of the surveillance camera that shot the person U. Acquires spatial information s. In the second embodiment as in the case of the first embodiment, the spatial information s is not only the shooting area specified by the camera ID, but also a plurality of surveillance cameras in the vicinity of the surveillance camera of the camera ID. It may include an identifier representing a plurality of shooting areas to be shot.

Next, in step S713, the similar object information acquisition unit 12022 is a registered person of normal behavior that has been detected in the past by the time zone defined by the time t and the surveillance camera group defined by the spatial information s. Get a list (let's call it a normal person list). The normal behavior referred to here is a person who has been observed more than once over a specific period, for example, a certain period in the past, does not perform any abnormal behavior, and excludes a person who has been notified by pressing the report button 66 in FIG. Is the action of. The normal person list is a person who has been observed at least once in a specific time and space over a certain period of time in the past, does not perform any abnormal behavior, and is notified by the report button 66 to indicate that the person is a person with abnormal behavior. This is a list of registered people who have not been registered. That is, the person included in the normal person list acquired in step S713 is considered to be a registered person who may be observed by a surveillance camera in the vicinity in that time zone, and therefore is not a person requiring surveillance.

Next, in step S714, the collation unit 12023 collates the person U detected in step S710 with each registered person included in the normal person list. That is, in order to detect a person who is not normally observed or a person whose behavior deviates from the normal behavior defined by the time t and the spatial information s as a person to be monitored, in step S714, the person detected in step S710 is normal. Match with the person list.

At this time, the collation unit 12023 sets the feature amount group of each registered person included in the normal person list defined by the time t and the spatial information s in the past and the feature amount extracted from the person U in the feature space. Compare. Then, the collation unit 12023 determines whether or not the feature amount of the person U is a registered person detected in the past in which the feature amount is close within a distance equal to or less than a specific threshold value. Further, when the collation unit 12023 determines that the person is a registered person, the collating unit 12023 determines whether or not the behavior of the person deviates from the normal behavior defined by the time t and the spatial information s. That is, when it is determined that the detected object is in the normal state, the collation unit 12023 further determines whether or not the detected object is a registered person (first object).

Next, in step S715, the generation unit 12024 is a registered person detected in the past in the collation process of step S714, and when a collation result is obtained that the behavior is normal, the person to be monitored is detected. It is determined that there was no such thing, and the process proceeds to step S716. Further, when the generation unit 12024 determines that the detected object is not the normal behavior of the registered person detected in the past, the generation unit 12024 determines that the object needs to be monitored. Further, when the generation unit 12024 obtains a collation result that the detected object is an unregistered person (third object) that is not a registered person, the generation unit 12024 determines that the person needs to be monitored. Alternatively, when the generation unit 12024 obtains a collation result that the behavior of the detected object is not the normal behavior of the registered person, the generation unit 12024 determines that the detected object is a person to be monitored and proceeds to step S717. When the person U is a person who has never been detected by the surveillance camera defined by the spatial information s in the time zone defined by the time t in the past, or the person U is a registered person who does not behave normally, the generation unit 12024 is the person. The person U generates information indicating the result of determining that the person needs to be monitored.

In step S717, the generation unit 12024 generates an image as shown in FIG. 8 described above on the output device 13 as a collation result by step S714 and step 715, and the output device displays the screen.
Then, in the next step S718, the reception unit 12025 receives the input from the user. When the user is instructed by the user that the detected object that is determined to be in an abnormal state is not a registered person, the reception unit 12025 is an unregistered person (third object) that has not been detected in the past. A label indicating that is attached. Further, when the user is instructed that the detected object determined to be not in the normal state is the first object, the detected object is associated with the spatial information in the detected object and the time. Holds information indicating that the first object is in a normal state.

Here, in the screen display of FIG. 8, when the user determines that the person to be monitored, which is displayed with a frame, is a registered person of normal behavior and presses the false detection button 65, the information processing apparatus 12 will perform the information processing device 12. , It is determined that the person U is not the person to be monitored. In this case, the detection unit 1202 updates the normal person list with the time t when the person U was photographed and the spatial information s as new information indicating the normal behavior in the person U.

Further, if the user cannot immediately determine whether or not the person U is a person to be monitored and presses the tracking button 64, the detection unit 1202 continues to track the person U.
On the other hand, when the user determines that the person U is a person who performs an abnormal behavior and presses the report button 66, the detection unit 1202 performs the report process.
In this way, the detection unit 1202 reflects the user input received in step S718, generates a normal person list defined by the time t and the spatial information s that reflects the user input in step S719, and generates the next suspicious person list. Used to detect behavior.

As described above, the information processing according to the second embodiment described above is continued until the end of monitoring is input by the user in step S72. As a result, the user does not have to monitor all the behaviors of the person observed on the screen, and it is possible to monitor only when a person who has not been observed in the corresponding time zone or space in the past is observed. Therefore, the cost of monitoring can be significantly reduced.

The information processing apparatus 12 of the first embodiment and the second embodiment described above can be realized with the same hardware configuration as shown in FIG. Therefore, it is also possible to simultaneously detect a specific registered detection target person registered in advance in the first embodiment and detect a monitoring target person in the second embodiment. Further, even if it is determined that the person is not detected as the person to be monitored as defined in the second embodiment, it is determined in two stages that the person is not the registered detection target person or the registered detection target person. It becomes possible to carry out recognition that is more in line with the user's intention.

Further, when comparing with the registered detection target person, it is the same as in the first embodiment, and in the present embodiment, the monitoring is required by comparing with the non-detection target person registered in advance, for example. It is possible to suppress the number of alert information issued by the person detected as. The non-detection target person referred to here may be, for example, the staff of a store that visits the store at random timing in the monitoring of the store, and the information of the staff is registered as the non-detection target person. ..

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
The above-mentioned embodiments are merely examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

11: Storage device, 12: Information processing device, 13: Output device, 14: Input device, 101: Image pickup device, 1201: Image acquisition unit, 1202: Detection unit, 12020: Object detection unit, 12021: Spatial information acquisition unit, 12022: Similar object information acquisition unit, 12023: Collation unit, 12024: Generation unit, 12025: Reception unit, 12026: Update unit

Claims (15)

A detection means that detects a predetermined object from an image,
An acquisition means for acquiring a second object different from the pre-registered first object based on the spatial information representing the space in which the object is detected and the time when the object existed in the space.
A determination means for determining whether or not the detected object by the detection means is the first object or the second object.
An information processing device characterized by having. The information according to claim 1, further comprising an output means for outputting alert information when the determination means determines that the detected object is the first object. Processing equipment. The information processing apparatus according to claim 2, further comprising an updating means for updating the determination criteria in the determination based on the modification when the user accepts the modification of the determination result by the determination means. When the determination result that the detected object is determined to be the first object by the determination means is incorrect, the spatial information and time of the detected object and the detected object are extracted. The information processing apparatus according to any one of claims 1 to 3, wherein the information processing apparatus has a holding means for holding the obtained feature amount as information of the second object. The determination means determines the detected object as a candidate for the first object, and further obtains information on the second object held by the holding means with respect to the candidate for the first object. The information processing apparatus according to claim 4, wherein the detection means determines whether or not the detected object is the first object or the second object. The information processing apparatus according to claim 4 or 5, further comprising a presentation means for presenting information on the feature amount of the second object held by the holding means to the user. When the determination result that the detected object is determined to be the first object by the determination means is incorrect, the determination means is held by the holding means with respect to the detected object. The information processing apparatus according to any one of claims 4 to 6, further comprising a label that distinguishes whether or not the object is the same as the first object. A detection means that detects a predetermined object from an image,
Specifying the pre-registered first object detected corresponding to the spatial information representing the space in which the object is detected and the time when the object existed in the space, and the normal state of the first object. Means and
Determination of determining whether or not the detected object by the detection means is in the normal state based on the first object specified by the specific means and the normal state of the first object. Means and
An information processing device characterized by having. The determination means is characterized in that when it is determined that the detected object is in the normal state, it is further determined whether or not the detected object is the first object. The information processing apparatus according to 8. The determination means has not been detected in the past when the user teaches that the detected object determined to be not in the normal state of the first object is not the first object. The information processing apparatus according to claim 8 or 9, wherein a label indicating that the object is a third object is attached. When the user is instructed by the user that the detected object that is determined not to be in the normal state is the first object, the determination means includes the spatial information and the time in the detected object. 10. The information processing apparatus according to claim 10, further comprising a holding means for holding the first object in a normal state in association with the above. The determination means excludes the first object in a normal state detected in a specific time and space in the past from the object of the determination, which is not detected for a specific period. The information processing apparatus according to claim 11. It is an information processing method executed by an information processing device.
A detection process that detects a predetermined object from an image,
An acquisition step of acquiring a second object different from the pre-registered first object based on the spatial information representing the space in which the object is detected and the time when the object existed in the space.
A determination step of determining whether the detected object is the first object or the second object, and
An information processing method characterized by having. It is an information processing method executed by an information processing device.
A detection process that detects a predetermined object from an image,
Specifying the pre-registered first object detected corresponding to the spatial information representing the space in which the object is detected and the time when the object existed in the space, and the normal state of the first object. Process and
A determination step of determining whether or not the detected object is in the normal state based on the specified first object and the normal state of the first object.
An information processing method characterized by having. A program for making a computer function as each means of the information processing apparatus according to any one of claims 1 to 12.

Images