JP2017194798A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To search for a person continuously from the past, without reducing processing speed.SOLUTION: An information processing apparatus includes: first searching means of executing first search processing which searches for a target person, from a video stored in storage means, on the basis of characteristics of a person extracted from a video input by imaging means before the time when an instruction to search for a person is received, and stored in the storage means, and characteristics extracted from a target person relating to the search instruction; and second searching means of executing second search processing which searches for the target person from a video input after the receiving time, on the basis of the characteristics of the person extracted from the video input by the imaging means after the receiving time, and the characteristics of the target person extracted from a query image relating to the search instruction. The first searching means executes first search processing on a video which has been input during a period from the receiving time to the time when the second search processing is ready and started.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  2. Description of the Related Art Conventionally, a technique for detecting a lost child or a passenger who misses a train by searching a person's face from each frame of a monitoring video is known. In Patent Document 1, a human face is detected from each frame of a monitoring video, an image feature is calculated from the face, and is stored in a database (DB) in association with a video frame. As a query, a technique for retrieving a video of a lost child from a DB is disclosed. Patent Document 2 discloses a technique for registering a person's face to be detected in advance and detecting the person in real time from an input video.

JP 2013-153304 A JP 2014-215747 A

  However, in the technique of Patent Document 1, although the video stored in the DB at the time when the query is issued is the search target, the target person cannot be searched from the video obtained thereafter. For this reason, it is impossible to inform and secure the current location of lost children or late passengers. On the other hand, in the technique of Patent Document 2, when performing a person search retroactively to a video before the time when a query is issued, the processing speed is slowed down. For this reason, the technique of Patent Document 2 is not sufficient for the purpose of securing lost children and late passengers.

  The present invention has been made in view of such problems, and an object thereof is to search for a person continuously from the past without causing a reduction in processing speed.

  Therefore, the present invention is an information processing apparatus, which is a receiving unit that receives a search instruction for a person, and is extracted from an image that is input from an imaging unit and stored in a storage unit before the reception point when the search instruction is received. A first search process for searching for the target person from the video stored in the storage unit based on the characteristics of the selected person and the characteristics extracted from the target person related to the search instruction. Based on the search means, the characteristics of the person extracted from the video input from the imaging means after the reception time point, and the characteristics of the target person extracted from the query image related to the search instruction, Second search means for performing a second search process for searching for the target person from a video input after the reception time point, wherein the first search means starts the second search means from the reception time point. by Serial are ready for the second search processing, the relative input image until starting the second search processing, and performs the first search process.

  According to the present invention, it is possible to search for a person continuously from the past without causing a decrease in processing speed.

1 is a diagram illustrating a monitoring system according to a first embodiment. It is a figure which shows the software structure of information processing apparatus. It is a flowchart which shows a video recording process. It is explanatory drawing of a 1st search process and a 2nd search process. It is a flowchart which shows a search process. It is a flowchart which shows the search process which concerns on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram illustrating a monitoring system according to the first embodiment. The monitoring system includes the information processing apparatus 100 and a camera 112 as an imaging unit, and searches for a person specified as a query. Although FIG. 1 illustrates a case where the person detection system includes three cameras 112, the number of cameras 112 is not limited to the embodiment. The information processing apparatus 100 and the camera 112 are connected via a network 111.

  The CPU 101 is a central processing unit that controls the entire information processing apparatus 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not need to be changed. The RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device. The external storage device 104 is a storage device such as a hard disk or a memory card that is fixedly installed in the information processing apparatus 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be detached from the information processing apparatus 100, a magnetic or optical card, an IC card, a memory card, and the like. Note that the functions and processing of the information processing apparatus 100 described later are realized by the CPU 101 reading a program stored in the ROM 102 or the external storage device 104 and executing the program.

  The input I / F 105 is an interface with an input unit 109 such as a pointing device or a keyboard that receives data from a user and inputs data. The output device I / F 106 is an interface with the monitor 110 for displaying data held by the information processing apparatus 100 and supplied data. The communication I / F 107 is connected to a network 111 such as the Internet. The camera 112 is a video imaging device such as a surveillance camera, and is connected to the information processing apparatus 100 via the network 111. The system bus 108 is a transmission path that connects the units 101 to 107 so that they can communicate with each other.

  FIG. 2 is a diagram illustrating a software configuration of the information processing apparatus 100. The information processing apparatus 100 uses a face image feature obtained from a face as an image feature, and performs an image feature search using this. The video reception unit 201 receives video captured by the camera 112 and input from the camera 112 to the information processing apparatus 100. The video storage unit 202 stores the video received by the video reception unit 201.

The tracking unit 203 tracks a person in the video received by the video receiving unit 201. Specifically, the tracking unit 203 detects an object from a motion vector, estimates a search position in the next frame, and performs person tracking by template matching. Regarding the person tracking process, the following documents can be referred to.

JP 2002-373332 A

The tracking unit 203 issues the same tracking track ID to tracking tracks that track the same person, and issues different tracking track IDs to tracking tracks of different persons. Thereby, uniqueness is ensured and the same person can be identified from the tracking track ID. Note that the tracking unit 203 issues a different tracking track ID when tracking is interrupted, even for the same person.

The face detection unit 204 performs face detection from a frame image or the like in the video. For example, the face detection unit 204 performs face detection from each of the person's frame images tracked by the tracking unit 203. Further, face detection is performed from the video input from the video reception unit 201 and a query image described later. The face detection unit 204 detects a one-eye candidate area for the processing target image, performs pairing from the plurality of one-eye candidate areas, and determines a face area based on the paired eye positions. Note that the following documents can be referred to for a method of detecting a human face from an image.

JP 2010-165156 A

  The representative determining unit 205 selects a representative face image from the frame image group of the tracked person. Hereinafter, a representative face image is referred to as a representative image. For example, the representative determining unit 205 selects an image having a large face size from among the face images detected by the face detecting unit 204. The reason for using the face size is the problem of the accuracy of image features. This is because the larger the face image, the more accurate image features can be obtained. That is, when calculating the image feature from the face image, it is necessary to perform a face size normalization process for scaling the size of the face image to a certain size. At that time, if the face image is larger than the certain size, the reduction processing is performed and the information loss is relatively small. If the face image is smaller than the certain size, it is necessary to perform pixel complementation such as super-resolution processing. There is severe degradation of information.

  As another example, the representative determining unit 205 may select a plurality of images from the frame image group as representative images. For example, there is a method of selecting images with a plurality of face orientations. This is because even if the images of the same person are different in face orientation, the image characteristics obtained from the images are different.

The representative determination unit 205 extracts a gradient direction histogram (HOG) as a feature, and estimates the face direction using SVR. The gradient direction histogram is a feature obtained by making the luminance gradient information of an image into a histogram for each part of the image, and is known as a feature that is robust to local noise and image contrast. By selecting features that are robust to fluctuations that are not related to face orientation, such as noise and lighting fluctuations, stable face orientation estimation is realized even in real environments. The following documents can be referred to for a method of detecting the face direction of a person from an image.

Erik Muphy-Chutorian, "Head posture estimation for driver assistance systems: A robust algorithm and experimental evaluation," in Proc. IEEE Conf. Intelligent Transportation Systems, 2007, pp. 709-714.

Further, the representative determining unit 205 may detect the orientation of the person's human body instead of the face orientation of the person. For example, the method described in the following document can be used to detect the orientation of the human body.

JP 2011-186576 A

  As another example, the representative determining unit 205 may further select an image with less image blur as the representative image. A camera that shoots a moving image may change the shutter speed according to the brightness of the location, as in the case of a still image camera. Accordingly, the face image may be blurred depending on the dark place or the moving speed of the subject, which directly causes deterioration of the image characteristics and attribute information. As for blur estimation, the representative determining unit 205 obtains a frequency component of the face image region, obtains a ratio between the low frequency component and the high frequency component, and produces a shake when the ratio of the low frequency component exceeds a predetermined value. You may judge that it is. In addition, the representative determination unit 205 may not select a face image such as a blinking eye or a mouth opening as a representative image. This is because there is a possibility that the image characteristics of the organ may be altered if there is a blinking eye or a mouth opening.

  The feature calculation unit 206 calculates the image feature of the face image to be processed. Specifically, the feature calculation unit 206 obtains organ points such as eyes and mouths in a person's face image, and calculates SIFT (Scale Invariant Feature Transform) features of each organ point. This feature is an example, and the type of feature to be calculated is not limited to the embodiment. The first feature storage unit 207 stores the image feature calculated by the feature calculation unit 206 for the video received by the video reception unit 201. The first feature storage unit 207 further stores, as image feature metadata, the ID of the person, the tracking track ID when the person is tracked, the shooting time, and the camera ID of the camera 112 that has been shot. The first feature storage unit 207 is, for example, the external storage device 104.

  The query receiving unit 208 receives a query image specification. Here, the query image is a face image of a person serving as a search key. A person serving as a search key is referred to as a target person. For example, the CPU 101 displays the face image stored in the external storage device 104 on the monitor 110. Then, while viewing the monitor 110, the user selects a face image of a person to be searched as a query image via the input unit 109. In response to this, the query receiving unit 208 receives a query image selected by the user. The query image may be one or two or more.

  The second feature storage unit 209 stores the image feature of the query image (face image) received by the query reception unit 208. Here, the image feature of the query image is an example of a reference feature predetermined for the target person. Note that the image feature of the query image is calculated by the feature calculation unit 206. The second feature storage unit 209 is, for example, the external storage device 104. The CPU 101 records the image feature of the query image in the second feature storage unit 209. The process of recording the image feature of the query image in the second feature storage unit 209 is a preparation process for starting an input image search process executed by the second search unit 212 described later. When the CPU 101 receives an instruction to exclude a target, which will be described later, the CPU 101 deletes the image feature from the second feature storage unit 209.

  The first search unit 211 executes a first search process. Here, the first search process is a process of searching for the target person from the video in the search range set for the first search process using the image feature calculated from the query image as a query. The first search unit 211 according to the present embodiment uses an image feature stored in the first feature storage unit 207 as a search range, and the first feature storage unit 207 has an image feature whose similarity is higher than a predetermined threshold. Is identified. In calculating the similarity, the sum of the distances of SIFT features of each organ point is obtained, the sum of the distances is normalized, and the similarity is obtained.

  The second search unit 212 executes a second search process. Here, the second search process uses the image feature (reference feature) stored in the second feature storage unit 209 as a query, and selects the target person from the video in the search range set for the second search process. This is a search process. The second search unit 212 according to the present embodiment uses the image feature calculated from the video received by the video receiving unit 201 as the search range after the time when the search instruction specifying the query image is received. Here, the search range includes the video stored in the temporary storage unit 210 in addition to the video received by the video reception unit 201. The second search unit 212 specifies an image feature having a similarity higher than a predetermined threshold in the search range. The method for calculating the similarity is the same as that described in the first search unit 211.

  The display processing unit 213 integrates the search result of the first search unit 211 and the identification result of the second search unit 212 and displays the result on the monitor 110. For example, the display processing unit 213 classifies the search results for each camera 112 and displays them on the monitor 110 in order of photographing time. As another example, the display processing unit 213 displays the installation position of the camera 112 on a map, and displays the movement history on the monitor 110 based on the shooting time. The display content is not limited to the embodiment.

  The exclusion receiving unit 214 receives information on a person who no longer needs to be searched, for example, by securing a lost child or a late passenger, and deletes the image feature of the person from the second feature storage unit 209. On the user interface of the display processing unit 213, it is possible to specify that a lost child or a late passenger has been secured. When searching for passengers who missed the aircraft, it is also possible to detect that the passenger has passed the boarding gate of the aircraft, and to delete the image feature of the person from the second feature storage unit 209 using that as a trigger.

  FIG. 3 is a flowchart showing video recording processing by the information processing apparatus 100. The video recording process is a process of accumulating the video received by the video receiving unit 201 so as to be searchable. In S301, the video reception unit 201 receives a video. Next, in S <b> 302, the CPU 101 records the video received in S <b> 301 in the video storage unit 202. At this time, the CPU 101 records the shooting time and the camera ID of the shooting camera 112 in association with each other as video metadata. This process is an example of a recording process for recording the video input from the camera 112 in the storage unit.

  Next, in S303, the tracking unit 203 detects a person from each frame image and performs tracking. Here, different person IDs are assigned to the detected persons for each frame image, and are temporarily stored together with the coordinates of the person in the frame image. The same tracking track ID is assigned to the person who is tracking, and is temporarily stored together with the ID of the frame image that is tracking. Next, in S304, when there is a person whose tracking is interrupted (YES in S304), the tracking unit 203 advances the process to S305. If there is no person whose tracking is interrupted (NO in S304), the tracking unit 203 advances the process to S301.

  In step S <b> 305, the face detection unit 204 performs face detection from each of the person frame images tracked by the tracking unit 203. In step S <b> 306, the representative determination unit 205 selects one or more representative face images from the group of tracked person frame images. Next, in S307, the feature calculation unit 206 calculates face image features from one or more face images selected in S306.

  Next, in S <b> 308, the CPU 101 records the image feature calculated in S <b> 307 in the first feature storage unit 207. Further, the CPU 101 records, as image feature metadata, a person ID, a tracking track ID when the person is tracked, a shooting time, and a shooting camera in association with each other. Next, in S309, the CPU 101 confirms whether or not an instruction to end the accumulation process has been received. When the CPU 101 receives an end instruction (YES in S309), the CPU 101 ends the process. If the CPU 101 has not received an end instruction (NO in S309), the process proceeds to S301. Through the above processing, the image features of the person's face image shown in the video input from the camera 112 are accumulated in the first feature storage unit 207 and become searchable.

  Next, person search processing by the information processing apparatus 100 will be described. Prior to the description of the person search process, the first search process by the first search unit 211 and the second search process by the second search unit 212 will be described with reference to FIG. FIG. 4A is an explanatory diagram of the first search process and the second search process. In the first search process, matching is performed using the extracted image features as a query. For this reason, a search using an arbitrary person as a query is possible. On the other hand, in the second search process, it is necessary to register in advance the image characteristics of the query person.

  Regarding the accuracy, the second search process has an advantage in accuracy by machine learning or the like. Regarding the speed, the first search process is fast with respect to the search of the video for which registration has been completed, but it takes a little time to be reflected in the first feature storage unit 207, so it is slightly inferior in terms of throughput (registration and search). On the other hand, in the second search process, if the number of persons registered in advance is limited to a certain number, authentication can be performed in real time, so that the throughput is considered high.

  Therefore, as a preferable application, the first search process is a search from an accumulated video of an arbitrary person from a large number of past images, and the second search process is to identify a limited number of specific persons in real time. It is to be.

  For this reason, the information processing apparatus 100 according to the present embodiment receives an instruction to specify a query person, performs a first search process on a video before (in the past) the time when the person search is started, and after that time. The second search process is performed for the (future) video. Thereby, for example, a lost child search can be performed appropriately. The reason for searching for past images is that information on past locations is also important because it is possible to be in locations that are not reflected in all the cameras 112, and it is also useful for estimating the destination.

  However, simply combining the first search process and the second search process causes a problem. FIG. 4B is a diagram illustrating a search range (time range) searched in the first search process and the second search process. In the first search process, the video recorded in the first feature storage unit 207 is set as a search range. For this reason, a short preparation period (delay) 401 occurs before the video at the reception time point when the query specification is accepted can be searched. Here, the preparation period is a period from the time when the video reception unit 201 receives the search instruction to the time when the preparation for the search process by the second search unit 212 described later is completed.

  Also in the second search processing, initialization processing such as setting of image features to be searched from the video of each camera 112 and learning takes time, and a preparation period (delay) 402 until detection processing is possible occurs. For this reason, a non-searchable time zone 403 that is a time zone obtained by adding the preparation period 401 and the preparation period 402 is generated. In security applications such as a monitoring system, it is desirable to reduce the non-searchable time zone 403.

  FIG. 5 is a flowchart showing search processing by the information processing apparatus 100. In step S <b> 501, the query reception unit 208 receives a search instruction that specifies a query image of a person serving as a search query. The CPU 101 executes processing in three threads after the processing in S501. The first thread is a thread that performs the first search process, and includes the processes of S502 to S505. The second thread is a thread that performs the second search process, and includes the processes of S511 to S513. The third thread is a process for integrating and displaying the search results, and includes the processes of S521 and S522.

  In S502 of the first thread, the first search unit 211 performs a first search process to search for a target person. The search range in this case is input to the information processing apparatus 100 before the reception time when the video stored in the first feature storage unit 207, that is, the search instruction is received, and stored in the video storage unit 202. It is an image feature of the recorded video. The search query is an image feature of the face image detected from the query image related to the search instruction. The first search unit 211 sends information such as the image obtained by the search, the shooting location of the image, and the time to the display processing unit 213.

  Next, in S <b> 503, the first search unit 211 confirms whether or not the second search process by the second search unit 212 has started. When the second search process is started (Yes in S503), the first search unit 211 advances the process to S505. If the second search process has not started (No in S503), the first search unit 211 advances the process to S504.

  In S504, the first search unit 211 performs the first search process again. Note that the search range of the first search process in S504 is an image feature that is not set to the search range of the first search process that has already been executed among the image features stored in the first feature storage unit 207. It is. The first search unit 211 then advances the process to S503. In this manner, the first search unit 211 repeatedly executes the first search process using the newly input video feature as a search range until the second search process is started. The first search unit 211 further sends information such as the image obtained by the search, the shooting location of the image, and the time to the display processing unit 213. In step S505, the first search unit 211 stops the first search process. This completes the processing of the first thread. As described above, for the video input during the period until the second search process is started, the search range of the first search process is set, and the first search process is performed. Therefore, the non-searchable time zone 403 can be shortened.

  In S511 of the second thread, the second search unit 212 performs a preparation process for starting the second search process. The preparation process is a process of registering (recording) the image feature of the query image of the target person in the second feature storage unit 209 as a reference feature. In S511, the display processing unit 213 may display the query image on the monitor 110. Next, in S512, the second search unit 212 starts a second search process for searching for a target person from the video input to the information processing apparatus 100 after the reception time point based on the reference feature. Then, the second search unit 212 sends information such as the image obtained by the search, the shooting location of the image, and the time to the display processing unit 213. When the second search process is started in S512, the first search unit 211 determines that the second search process has started in S503, and the first search process is stopped.

  Next, in S513, the second search unit 212 confirms whether or not a CPU 101 stop instruction has been accepted. If the CPU 101 has not received a stop instruction (No in S513), the CPU 101 continues the second search process. When the CPU 101 receives a stop instruction (Yes in S513), the CPU 101 ends the process. For example, the user inputs a stop instruction when searching for a person designated as a query is no longer necessary, for example, by securing a lost child or a late passenger.

  In S521 of the third thread, the display processing unit 213 integrates the detection results obtained in S502, S504, and S512, and displays the detection results after integration on the monitor 110. Next, in S522, the CPU 101 confirms whether or not a stop instruction has been accepted. If the CPU 101 has not received a stop instruction (No in S522), the CPU 101 advances the process to S521 and continues the display process. When the CPU 101 receives a stop instruction (Yes in S522), the CPU 101 ends the process.

  As described above, the information processing apparatus 100 searches for the target person by the first search process for the non-searchable time zone 403, and then stops the first search process. Until the information processing apparatus 100 receives a stop instruction. The target person is searched by the second search process. In addition, a search impossible time zone corresponding to a combination of a time zone in which the feature amount is not yet indexed in the face image feature search and a time zone until the start of facial image feature identification at the time of executing the search by specifying the query person Can be reduced by performing the first search process recursively. As described above, the information processing apparatus 100 according to the present embodiment can search for a person continuously from the past without causing a decrease in processing speed.

(Second Embodiment)
Next, a monitoring system according to the second embodiment will be described. The information processing apparatus 100 of the monitoring system according to the second embodiment performs the first search process before the second search process, and detects the image detected in the first search process in the second search process. Use. Thereby, search accuracy can be improved.

  Hereinafter, the difference between the monitoring system according to the second embodiment and the monitoring system according to the first embodiment will be described. FIG. 6 is a flowchart showing search processing by the information processing apparatus 100 according to the second embodiment. In step S <b> 601, the query receiving unit 208 receives a search instruction that specifies a query image of a person who is a search query. The CPU 101 executes processing with two threads after the processing of 601. The first thread is a thread that performs a search, and includes the processes of S602 to S617. The second thread is a process for integrating and displaying the search results, and includes the processes of S621 and S622.

  In step S602 of the first thread, the first search unit 211 performs a first search process to search for a target person. The search range in this case is input to the information processing apparatus 100 before the reception time when the video stored in the first feature storage unit 207, that is, the search instruction is received, and stored in the video storage unit 202. It is an image feature of the recorded video. The search query is an image feature of the face image detected from the query image related to the search instruction. The first search unit 211 sends information such as the image obtained by the search, the shooting location of the image, and the time to the display processing unit 213.

  The CPU 101 further executes processing with two threads after the processing of S602. The eleventh thread is a thread that continues the first search process, and includes the processes of S603 to S605. The twelfth thread is a thread that performs the second search process, and includes the processes of S611 to S617. The processes of S603 to S605 of the eleventh thread are the same as the processes of S503 to S505 of the search process of the first embodiment described with reference to FIG.

  In S611 of the twelfth thread, the second search unit 212 confirms whether or not the target person has been detected in the first search process in S602. If the second search unit 212 detects the target person (Yes in S611), the process proceeds to S612. If the second search unit 212 does not detect the target person (No in S611), the process proceeds to S615. In S612, the second search unit 212 prepares to perform the second search process by using not only the query image but also the detection image in which the target person is detected in the first search process in S602. Do.

  Next, in S613, the second search unit 212 starts a second search process using the query image and the detected image. Next, in S614, the second search unit 212 confirms whether or not a stop instruction has been accepted. When receiving a stop instruction (Yes in S614), the second search unit 212 ends the twenty-first thread process. If the second search unit 212 does not receive a stop instruction (No in S614), the second search unit 212 continues the second search process. In S615, the second search unit 212 performs a preparation process to perform a second search process using only the query image. Note that the processing of S615 to S617 is the same as the processing of S511 to 513 of the search processing of the first embodiment described with reference to FIG.

  In S621 of the second thread, the display processing unit 213 integrates the detection results obtained in S602, S604, S613, and S616, and displays the integrated training result on the monitor 110. Next, in S622, the CPU 101 confirms whether or not a stop instruction has been accepted. If the CPU 101 has not received a stop instruction (No in S622), the CPU 101 advances the process to S621 and continues the display process. When the CPU 101 receives a stop instruction (Yes in S622), the CPU 101 ends the process. The remaining configuration and processing of the monitoring system according to the second embodiment are the same as the configuration and processing of the monitoring system according to the first embodiment.

  As described above, the information processing apparatus 100 according to the second embodiment uses the video features input to the information processing apparatus 100 and stored in the video storage unit 202 before the search instruction reception time as a search range. The first search process is executed before the second search process. Then, after the first search process, the information processing apparatus 100 performs a second search process using the target person detection image obtained in the first search process. Thereby, search accuracy can be improved. Furthermore, as in the first embodiment, the information processing apparatus 100 continues the first search process in a different search range until the second search process starts. Therefore, the information processing apparatus 100 can search for a person continuously from the past without causing a decrease in processing speed.

  Hereinafter, modified examples of the first embodiment and the second embodiment will be described. As a first modification, the image feature is not limited to the image feature of the face image. As another example, the image feature may be a human body image feature obtained from the entire human body. Examples of human body image characteristics include numerical values such as a ratio of height to body width and a ratio of foot length to head and body and height. As another example, the information processing apparatus 100 may use attribute information of a person obtained from the entire human body including the face, instead of the image feature. Examples of attribute information include race, gender, age, presence / absence of glasses, heels, clothing colors, and the like.

  As another example, the information processing apparatus 100 may use both image features and attribute information as image features. For example, the information processing apparatus 100 individually calculates the scores of the first search process and the second search process for image features and the scores of the first search process and the second search process for attribute information. To do. Then, the information processing apparatus 100 may calculate the weighted sum of both scores as a total score.

  As a second modification, the information processing apparatus 100 may display the query person and the image as the search result so as to be distinguishable. Thereby, the user can confirm whether it is an erroneous search.

  As a third modification, when at least one of the first search unit 211 and the second search unit 212 detects a plurality of images of the target person, it preferentially detects images having different face orientations. It is good to do.

  As a fourth modification, at least one of the first search unit 211 and the second search unit 212 may preferentially detect an image taken at a time closer to the processing time. This is useful for securing a person such as a lost child.

  As a fifth modification example, if the first search unit 211 of the information processing apparatus 100 performs the second search process on the video input to the information processing apparatus 100 before the second search process starts. Well, the specific processing for that is not limited to the embodiment. As another example, when the second search process is started, the first search unit 211 is input to the information processing apparatus 100 and stored in the video storage unit 202 until the second search process starts. The first search process with the feature of the video being searched as the search range may be performed once.

  As described above, according to each embodiment described above, it is possible to search for a person continuously from the past without causing a decrease in processing speed.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

  In the above description, in the first search process, the detected image of the target person can be acquired by specifying an image feature whose similarity is higher than a predetermined threshold. However, for example, when a user (monitor) who uses the information processing apparatus 100 specifies a target person and the information is input by the input unit of the information processing apparatus 100, a detection image of the target person can be acquired. You may do it.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 201 Image | video reception part 202 Image | video storage part 211 1st search part 212 2nd search part

Claims (12)

A receiving means for receiving a person search instruction;
Based on the characteristics of the person extracted from the video input from the imaging means and stored in the storage means before the reception time when the search instruction is received, and the characteristics extracted from the target person related to the search instruction First search means for performing a first search process for searching for the target person from the video stored in the storage means;
Based on the characteristics of the person extracted from the video input from the imaging means after the reception time point and the characteristics of the target person extracted from the query image related to the search instruction, input after the reception time point Second search means for performing a second search process for searching for the target person from the recorded video,
The first search means performs an operation on the input video from the reception time point until the second search process is completed by the second search means until the second search process is started. An information processing apparatus that performs the first search process. Integration means for integrating the search results of the first search means and the search results of the second search means;
The information processing apparatus according to claim 1, further comprising display means for displaying the search result after integration by the integration means.   The first search means has not started the second search process after the first search process is completed with the characteristics of the person extracted from the video stored in the storage means as the search range. In this case, the characteristics of the person extracted from the video that is not set in the search range in the first search process that has already been executed in the first search process that has been extracted from the video stored in the storage means. The information processing apparatus according to claim 1, wherein the first search process is performed again as a search range.   4. The information processing apparatus according to claim 1, wherein the first search unit ends the first search process when the second search process is started. 5.   The second search unit searches for the target person based on the image in which the target person is detected and the query image when the target person is detected by the first search unit. The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus.   The second search unit performs the first search process on the feature of the video that is input from the imaging unit and stored in the storage unit before the reception time point. 6. The information processing apparatus according to claim 5, wherein the second search process is started after the end.   The information processing apparatus according to claim 1, wherein the second search unit terminates the second search process when a stop instruction is received.   The second search means is a predetermined time when the person extracted from the video input from the image pickup means is out of the search range or after the person is extracted from the video input from the image pickup means. The information processing apparatus according to any one of claims 1 to 6, wherein the second search process is terminated when the value exceeds the threshold.   The information processing apparatus according to claim 2, wherein the display unit displays the query image and the search result in a distinguishable manner.   The at least one of the first search unit and the second search unit preferentially detects images having different face orientations of the target person or human bodies of the target person. The information processing apparatus according to any one of 1 to 9. An information processing method executed by an information processing apparatus,
A reception step for receiving a person search instruction;
Based on the characteristics of the person extracted from the video input from the imaging means and stored in the storage means before the reception time when the search instruction is received, and the characteristics extracted from the target person related to the search instruction A first search step of performing a first search process for searching for the target person from the video stored in the storage means;
Based on the characteristics of the person extracted from the video input from the imaging means after the reception time point and the characteristics of the target person extracted from the query image related to the search instruction, input after the reception time point A second search step of performing a second search process for searching for the target person from the recorded video,
In the first search step, for the input video from the reception time point until the second search process in the second search step is completed and the second search process is started. And performing the first search process. Computer
A receiving means for receiving a person search instruction;
Based on the characteristics of the person extracted from the video input from the imaging means and stored in the storage means before the reception time when the search instruction is received, and the characteristics extracted from the target person related to the search instruction First search means for performing a first search process for searching for the target person from the video stored in the storage means;
Based on the characteristics of the person extracted from the video input from the imaging means after the reception time point and the characteristics of the target person extracted from the query image related to the search instruction, input after the reception time point Functioning as a second search means for performing a second search process for searching for the target person from the recorded video,
The first search unit is configured to process an image input from the reception time point until the second search process is completed by the second search unit and the second search process is started. A program for performing the first search process.

Images