JP2021018518A - Information processing device, information processing method, and program - Google Patents

Abstract

To make it possible to specify and provide information supporting the determination of identity between an image of a comparison source and an image of a comparison destination.SOLUTION: An information processing device retrieves a person whose image feature is similar to that of a person of a comparison source from a stored image and outputs a retrieval result, separately acquires a first image including the person of the comparison source and a second image including the person of the retrieval result, extracts an object of a processing target different from the person of the comparison source and the person of the retrieval result from the first image and the second image, performs collation processing of the object of the processing target extracted from the first image and the second image, and specifies information for supporting the determination of identify of the person of the comparison source and the person of the retrieval result on the basis of a result of the collation processing and provides the information.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, an information processing method, and a program.

With the spread of surveillance camera systems, the number of cases of shooting a large number of people is increasing in many facilities and streets. There is a technique for searching for a desired person from these captured images. A face recognition technology is typical, but since a surveillance camera does not always capture a face, there is also a collation technology that uses a feature amount obtained from a full-body image of a person (see, for example, Patent Document 1). ..

International Publication No. 2011/046128

Collation using the feature amount of the whole body image often does not obtain collation accuracy as high as face recognition, and Patent Document 1 uses it together with face collation. In the collation using only the features of the whole body image, it is generally necessary for a person to confirm the collation result. However, simply displaying the comparison source image and the comparison destination (result) image is an image of a person whose image characteristics are similar, so it is possible to determine which result is the same person. It is difficult for a person to judge.

The present invention has been made in view of such circumstances, and an object of the present invention is to be able to identify and provide information that supports determination of identity from an image of a comparison source and an image of a comparison destination.

The information processing apparatus according to the present invention searches for a person whose image characteristics are similar to the person to be compared from the stored image, and outputs a search result, and a first image including the person to be compared. And the image acquisition means for acquiring the second image including the person of the search result, and the person of the comparison source and the person of the search result are different from the first image and the second image. The extraction means for extracting the object to be processed and the object to be processed extracted from the first image and the second image are collated, and based on the result of the collation process, the person to be compared and the person to be compared It is characterized by having a specific means for identifying and providing information that supports determination of the identity of a person in the search result.

According to the present invention, it is possible to identify and provide information that supports the determination of identity based on the image of the comparison source and the image of the comparison destination.

It is a figure which shows the hardware configuration example of the information processing apparatus in 1st Embodiment. It is a figure which shows the structural example of the information processing apparatus in 1st Embodiment. It is a figure explaining the person search result display in 1st Embodiment. It is a figure explaining the hint display in 1st Embodiment. It is a flowchart which shows the processing example in the information processing apparatus in 1st Embodiment. It is a figure which shows the structural example of the information processing apparatus in 3rd Embodiment. It is a figure which shows the structural example of the information processing apparatus in 4th Embodiment. It is a figure explaining the hint display in 8th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment)
The first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a hardware configuration example of the information processing device according to the first embodiment. The information processing device in the present embodiment may be realized by, for example, a single computer device, or may be realized by distributing each function to a plurality of computer devices as required. When realized by a plurality of computer devices, each computer device shall be connected so as to be able to communicate with each other via a network such as Local Area Network (LAN).

As shown in FIG. 1, the information processing device 100 in the present embodiment includes a CPU 101, a ROM 102, a RAM 103, a storage device 104, an input device interface 105, and an output device interface 106. The CPU 101, ROM 102, RAM 103, storage device 104, input device interface 105, and output device interface 106 are communicably connected to each other via the system bus 107.

The CPU (Central Processing Unit) 101 controls the entire information processing device 100. The ROM (Read Only Memory) 102 stores programs, parameters, and the like that do not require changes. The RAM (Random Access Memory) 103 temporarily stores programs, data, and the like supplied from the storage device 104 and the like. The CPU 101 controls the entire information processing apparatus 100 by using computer programs and data stored in the ROM 102, the RAM 103, and the like.

The storage device 104 is a hard disk drive (HDD) or flash memory drive (SSD) fixedly installed in the information processing device 100, or an optical disk, magnetic or optical card, IC card, memory card that can be attached to and detached from the information processing device 100. And so on. The input device interface 105 is an interface with an input device 108 such as a pointing device or a keyboard that inputs data in response to a user's operation. The output device interface 106 is an interface with an output device 109 such as a monitor for outputting the data held by the information processing apparatus 100, the supplied data, and the execution result of the program.

FIG. 2 is a block diagram showing a functional configuration example of the information processing apparatus according to the first embodiment. The information processing apparatus according to the first embodiment includes a video storage unit 201, a person search unit 202, a processing target determination unit 203, a peripheral image acquisition unit 204, a hint object candidate extraction unit 205, a hint object identification unit 206, and a hint display unit. Has 207.

The video storage unit 201 records the captured image captured by the monitoring system 208 in the storage device 104, and stores and manages the captured image together with the captured camera information and the capture time information. It is assumed that the surveillance system 208 in the present embodiment manages at least a surveillance camera installed at a position where a person can be photographed, and can acquire an image to be captured by the camera together with information on the installation location and shooting time of the camera. .. The camera installation location information may be any information that allows the approximate position between the cameras to be known, and may hold coordinate information of a general geographic coordinate system such as WGS84. It should be noted that the monitoring system 208 may be able to acquire sound by using a microphone or the like together with the image captured by the camera, and the captured image and the acquired sound may be recorded in the storage device 104 together. ..

Further, the surveillance system 208 in the present embodiment shall have a user interface that provides the user with a function such as viewing a surveillance camera image, which is possessed by a general surveillance system. As for the provided function in the present embodiment, the user interface of the monitoring system 208 also receives an instruction input from the user and displays the processing result to the user via the input device 108 and the output device 109. The functions and configurations of the monitoring system 208 that are not directly related to the processing in the present embodiment will not be described.

The person search unit 202 inputs a person image as a comparison source, and searches for a person whose image characteristics are similar to those of the person image from the images stored and managed by the video storage unit 201. In the present embodiment, it is assumed that the user using the surveillance system 208 selects and searches the comparison source person to be searched from the images taken by the surveillance camera. The person image input by the person search unit 202 is an image of an area defined by a rectangular bounding box that includes a person as a comparison source. The person search unit 202 searches for another person similar to the image feature amount obtained from the image in the rectangle from the image stored and managed by the video storage unit 201.

The search result by the person search unit 202 is output and displayed on the output device 109, for example. In the present embodiment, a rectangular area of a person is specified in advance from the recorded images stored and managed by the video storage unit 201. The method of specifying the rectangular area of the person is not particularly limited. For example, the area of the face or head of the person is detected, and the position and size of the area of the face or head are used to determine the approximate person. The entire rectangular area may be obtained. Similarly, the rectangular area of the comparison source person may be obtained by any method. For example, the user may specify the rectangular area by using a device such as a mouse at the time of selection, or the approximate rectangular area is automatically set based on the area of the face or head as described above. It may be determined by.

Further, the image feature amount extracted from the person image obtained in this way is not particularly limited. Any image feature amount that captures features that people think are visually similar may be used. For example, a general image feature amount such as a HOG (Histograms of Oriented Gradients) feature amount using the similarity of the appearance frequency of the luminance gradient in the local region may be used. Alternatively, the output of the intermediate layer may be extracted as a feature amount from the network in which the task of determining similar persons has been learned by a deep learning method and used.

Further, the method of comparing the feature quantities is not particularly limited. In the present embodiment, when the rectangular area of the person is specified in advance, the feature amount obtained from the specified rectangular area is held in the RAM 103 or the like. At this time, the information processing apparatus in the present embodiment obtains the information of the camera that has taken a picture, the information of the taking time, and the information of the rectangular position so that the person can be identified from which image frame the person is obtained from when collated. , Manage it in association with the feature amount. As a result, the feature amount obtained in advance is compared with the feature amount obtained from the comparison source person image, and a person having similar feature amounts is specified as a search result. The similarity of the feature amount may be determined by, for example, a general method in which the feature amount is regarded as a vector and is determined by the closeness of the vector distance.

The processing target determination unit 203 compares the information on the shooting camera position and shooting time of the comparison source person image with the information on the shooting camera position and shooting time of a certain search result, and determines the approximate moving speed from the difference in distance and time. By asking for, the object to be used as a hint is determined. In the processing target determination unit 203, the moving speed of an approximate person walking and the moving speed of the vehicle are set in advance. If the movement speed is about the walking speed of a person (for example, 4 km / h), the processing target determination unit 203 determines that the hint object that can be used is a "person", and the movement speed is close to that of the vehicle (for example, 30 km / h). ), The hint object that can be used is determined to be "vehicle". If the speed is between the set values and it is difficult to specify one of them, each process may be performed using the "person" and the "vehicle" as hint objects.

The classification of moving speed is not limited to the above two classifications. You may make finer classifications and use the corresponding objects in association with each other. For example, in the case of a vehicle, a bicycle and an automobile may be distinguished and used as hint objects. In addition, in order to realize higher judgment accuracy, the movement of the actual person is tracked by the camera, the site is verified in the field, and the numerical value of the movement time of the actual person between the cameras is acquired and processed. It may be used for the determination by the target determination unit 203. When the movement time is much shorter (there is a high possibility that the vehicle has moved at a high speed), the processing target determination unit 203 may determine that the vehicle is adopted as the hint object.

Peripheral image acquisition unit 204 acquires an image including each of the comparison source person and the search result person. In the present embodiment, the entire captured image including each of the comparison source person and the search result person is acquired from the video storage unit 201. As described above, the video storage unit 201 manages the image together with the shooting camera information and the shooting time information, so that the shooting camera information and the shooting time information linked to the search result are used as keys. , Get the whole image including the person in the search result.

The hint object candidate extraction unit 205 extracts the object determined by the processing target determination unit 203 from the images acquired by the peripheral image acquisition unit 204. In the present embodiment, the hint object candidate extraction unit 205 extracts a "person" or a "vehicle (vehicle)" as a hint object. The method of identifying each object is not limited. For example, the hint object candidate extraction unit 205 may extract the “person” from the image based on the above-mentioned face and head regions.

Further, when determining the "automobile", the hint object candidate extraction unit 205 may determine whether or not the vehicle is an automobile based on the discovery of a certain characteristic region (characteristic of shape or color). Alternatively, a class discrimination method based on deep learning may be used. It should be noted that the hint object identification unit 206, which will be described later, only needs to be able to determine the identity of the objects, so that the region of the automobile in the image is not strictly required, and it is conceivable to collate by comparing license plates, for example. In that case, it suffices if the approximate vehicle position that makes the position of the license plate recognizable can be specified.

The hint object identification unit 206 compares the hint object candidates around the person of the comparison source and the person around the search result extracted by the hint object candidate extraction unit 205, and performs a predetermined collation process to obtain the hint object. Identify the pair. In order to compare object candidates at different locations and times in the comparison source and the search result, the hint object identification unit 206 identifies a pair of objects having a one-to-one correspondence with each other.

In this embodiment, it may be a "person" or a "car". If it is a "person", for example, among the hint object candidates, those with a face can be authenticated using the face feature, so that may be used. Further, when using the feature amount obtained from the whole body image, the collation result may be used by setting the threshold value of the distance of the similarity determination in the person search unit 202 to be closer. Also, in the case of "automobiles", for example, by recognizing the license plate as characters, it is possible to collate the same vehicle with high accuracy, so if the image of the plate can be acquired with a resolution that can identify the characters, You may use this. In addition, many features may be used in combination to increase the accuracy. For example, in the case of a car, text information is often attached to the exterior and windows with stickers or the like. In addition to the identity of these characters, the similarity of the image features of the entire vehicle may be judged in a complex manner to increase the accuracy.

If incorrect information is presented to the user as a hint object, it becomes difficult to support the person identification. Therefore, the hint object identification unit 206 identifies the object as a hint by a determination process capable of collating with high accuracy. If an object that can be collated with high accuracy is not found, the hint object identification unit 206 processes it as if the object cannot be found. On the other hand, there are cases where multiple matching object pairs are found by highly accurate collation processing. In that case, the hint object specifying unit 206 may present an object having a higher degree of similarity / matching obtained as a collation processing result up to a predetermined number of pairs.

The hint display unit 207 displays the pair of hint objects specified by the hint object identification unit 206 on the output device 109 in addition to displaying the comparison source and the search result. An example of hint display by the hint display unit 207 will be described with reference to FIGS. 3 and 4. FIG. 3 is a diagram showing a display example of the person search function provided by the monitoring system 208 to the user. In FIG. 3, the map information 301 indicating the camera installation position displays the shooting position and time information 302 of the person who is the comparison source, and displays the shooting position and time information 303 of the search result selected by the user. ing. FIG. 3 shows a state in which a hint for supporting the determination of the identity between the comparison source person and the search result person is not displayed.

At this time, it is assumed that each camera installation position information is acquired from the monitoring system 208 and it is found that there is a distance of about 5 km. Further, assuming that the shooting times are 16:00 and 16:10, respectively, it means that the vehicle travels in a straight line at a speed of 30 km / h. In this case, the processing target determination unit 203 determines the "automobile" as a hint object based on the moving speed, and the hint object candidate extraction unit 205 determines the person of the comparison source and the person of the search result from the entire image including each of them. Extract cars. When automobiles can be extracted in the vicinity of the person, the hint object identification unit 206 executes the collation processing of the extracted automobiles. As described above, the hint object identification unit 206 performs a highly accurate collation process according to the object. When the vehicle is a hint object candidate, for example, the hint object identification unit 206 performs character recognition of the license plate and determines whether or not the vehicle is the same vehicle.

FIG. 4 is an example of displaying the specified hint object and the peripheral image of the range including the person of the comparison source and the person of the search result. It may be displayed instead of the comparison source and the search result shown in FIG. 3, or may be displayed in a place different from that of FIG. 3 as hint information.

In FIG. 4, as the overall display 401, hint objects in the peripheral image including the comparison source person and the search result person are displayed side by side in a comparable manner. The image 402 displayed on the left side is a peripheral image including the person to be compared, and the image 403 displayed on the right side is a peripheral image including the person in the search result. In the image 402, 404 is a person image of the comparison source, and in the image 403, 405 is a person image of the search result. The same icon is displayed so that they can be distinguished from each other. For example, here, an example surrounded by a double frame is shown. However, if it is known that the person is the compared person, the display method does not matter.

Further, in the images 402 and 403, 406 and 407 are a pair of the specified hint objects, and similarly, they are displayed to distinguish that they are a pair. In the figure, a frame is attached with a dotted line, but in reality, a frame of the same color may be attached, or information 408 that is the basis used for collation may be presented. In the present embodiment, since the hint object identification unit 206 collates the same object based on the number of the automobile, the number information 408 is also displayed.

Next, the processing in the information processing apparatus in the first embodiment will be described. FIG. 5 is a flowchart showing a processing example in the information processing apparatus according to the first embodiment.

In step S501, the user of the monitoring system 208 specifies a person to be searched for in the displayed image, and the person search unit 202 searches based on the image feature amount by inputting the designated person as the comparison source. Execute and search for a person whose image features are similar to those of the comparison source person. Then, the person search unit 202 displays a list of people in the search result on the output device 109 via the monitoring system 208, and provides the search result to the user.

In step S502, the user specifies a person who seems to be the same person or wants to confirm from the list of people in the search result displayed in step S501 via the user interface of the monitoring system 208. Then, the processing target determination unit 203 receives the search result selection instruction by the user via the user interface of the monitoring system 208.

In step S503, the processing target determination unit 203 acquires the information of the shooting camera that shot the image including each of the designated search result person and the comparison source person, and the shooting time information. The information on the shooting camera and the shooting time related to the person to be compared is specified by the monitoring system 208 as described above, and the information on the shooting camera and the shooting time related to the person in the search result is stored in the video described above. The information managed by the section 201 is used. Further, since the installation position information of each camera is managed by the monitoring system 208, it is acquired from there. The processing target determination unit 203 determines the object to be processed from the distance and time difference information (spatiotemporal difference information) obtained based on these information.

In step S504, it is determined whether or not there is a processing target object determined by the processing target determination unit 203 in step S503. If there is an object to be processed (YES), the process proceeds to step S505, and if there is no object to be processed (NO), the process shown in FIG. 5 is terminated. By determining the presence or absence of the object to be processed, if there is no object to be processed, it is not necessary to perform the processing after step S505, and the processing cost can be suppressed.

In step S505, the peripheral image acquisition unit 204 acquires an image including the comparison source person and the search result person from the image storage unit 201 using the information of the photographing camera and the information of the photographing time as keys.

In step S506, the hint object candidate extraction unit 205 extracts the object to be processed in step S503 from each of the entire images acquired in step S505.

In step S507, the hint object identification unit 206 performs collation processing of hint object candidates around the person of the comparison source and the person around the search result extracted from each overall image in step S506 to specify the pair of hint objects. To do.

In step S508, it is determined in step S507 whether or not the hint object pair can be specified, and if it can be specified (YES), the process proceeds to step S509, and if it cannot be specified (NO), the figure. The process shown in 5 is completed.

In step S509, the hint display unit 207 displays the pair of hint objects identified in step S507 on the output device 109 via the monitoring system 208, and provides the user with the pair. Then, the process shown in FIG. 5 is terminated.

As described above, according to the first embodiment, it is possible to use it as a hint for judgment in the vicinity of the person between the comparison source person image and the comparison destination person image, which makes it difficult to identify and judge the same person. Identify pairs that are considered to be the same person or the same object, and display them together with the results. As a result, if a person or object that supports the determination of identity is found in the comparison source image and the comparison destination image, it can be displayed and provided to the user to support the identification determination. It is possible to reduce the time required to identify a person in the past.

Next, other embodiments of the present invention will be described. In each of the embodiments described below, the description overlapping with the above-described embodiment will be omitted, and only the differences will be described.
(Second Embodiment)
A second embodiment of the present invention will be described. In the first embodiment described above, the hint object identification unit 206 performs highly accurate collation processing for each object to specify a pair of hint objects. However, it is possible to add a process of further narrowing down to objects that are more valuable as hints to assist the user in determining identity.

For example, when the comparison source person and the search result person are moving, an object having information having a high correlation with the movement is specified as a hint object. For example, if there is a person who has moved in the same direction in the same direction, the person who has moved in the same direction is preferentially displayed as a hint. In this case, the hint object specifying unit 206 acquires the frames of the front and rear monitoring images from the image storage unit 201, estimates the moving direction of the person from the moving image, and uses the face orientation. As a result of the processing, the hint object identification unit 206 preferentially identifies an object that is likely to move in the same direction as a hint object. By performing such processing, it is possible to select a high-value hint that can be used for the identification judgment of the user.

(Third Embodiment)
A third embodiment of the present invention will be described. In the first embodiment described above, an object that can be used as a hint object is determined from the position and time of the person to be compared and the person in the search result, and if there is such an object, processing is performed. Not exclusively. When the user selects the search result in step S502 shown in FIG. 5, the user is further inquired as to whether or not to display the hint. When the user instructs the hint display as a response to this inquiry, the information processing apparatus may perform the processing after step S503.

In this case, as shown in FIG. 6, the information processing apparatus may further include the hint display instruction receiving unit 601. Then, the hint display instruction receiving unit 601 inquires the user whether or not to display the hint via the user interface of the monitoring system 208, receives the instruction for the inquiry, and displays the hint according to the received instruction. It is sufficient to switch whether or not to perform. As a result, for example, it is possible to suppress wasteful resource consumption due to processing even when the user can identify by just looking at the search result.

(Fourth Embodiment)
A fourth embodiment of the present invention will be described. In the third embodiment described above, the process is performed when the user instructs to display the hint, but the present invention is not limited to this. When the peripheral image of the search result selected in step S502 shown in FIG. 5 is acquired and the difference between the visual feature of the comparison source person image or the person image of the search result and the visual feature of the acquired peripheral image is small, A process for specifying a hint object may be performed.

In this case, as shown in FIG. 7, the information processing apparatus may further include the peripheral object similarity determination unit 701. As a result, the peripheral object similarity determination unit 701 determines whether or not, for example, the feature amount of the person to be compared or the person in the search result and the feature amount of a certain number or more of the peripheral objects are similar by a certain value or more. judge. As a result of the determination, when the peripheral object similarity determination unit 701 determines that there are many similar objects in the periphery, the process related to the hint display is performed.

For example, if the person to be compared or the person in the search result wears a suit and there are many people wearing suits in the vicinity, and there is little visual difference between the people, the users are the same. It takes more time to judge the sex. In such a situation, if a person who can be a hint can be identified and presented, it is possible to further support the work of determining the identity of the user.

(Fifth Embodiment)
A fifth embodiment of the present invention will be described. In the first embodiment described above, the comparison source person image is selected from the surveillance images of the surveillance system 208, but the present invention is not limited to this. If there is an entire image in which the shooting time and shooting position comparable to the information possessed by the monitoring system 208 can be known and the peripheral image of the comparison source person can be obtained, that image may be input.

(Sixth Embodiment)
A sixth embodiment of the present invention will be described. In the first embodiment described above, the image acquired by the peripheral image acquisition unit 204 is the entire captured image including each of the comparison source person and the search result person, but the present invention is not limited to this. As the objects move away from the comparison source person and the search result person, it becomes difficult to use as a hint to support the judgment of identity, and the number of people and objects reflected increases when the shooting range is wide, so it is used for comparison. The processing cost is high and the accuracy is low. Therefore, the effect of judgment support may decrease as a whole. In such a case, the number of pixels in a predetermined range in the x-axis and y-axis directions including the person may be set in advance around the person, and the image in that range may be used as the peripheral image. .. As a result, the increase in cost as described above can be suppressed.

(7th Embodiment)
A seventh embodiment of the present invention will be described. In the first embodiment described above, the search result by the person search unit 202 is selected and instructed in step S502 to display a hint, but the present invention is not limited to this. When there are a plurality of people in the search result obtained by the person search unit 202, the process shown in FIG. 5 may be performed on the whole, and hints may be displayed in association with each result. By doing so, it becomes possible to assume a plausible result depending on the presence or absence of the hint display, so that the user identification work can be further supported.

(8th Embodiment)
An eighth embodiment of the present invention will be described. In the first embodiment described above, the case where the comparison source person is the person input to the person search unit 202 has been described. Therefore, the process of comparing the object around the person input to the person search unit 202 with the object around the person as the search result is performed. However, when the movements of the comparison source person and its peripheral objects are various, the peripheral objects that existed at the time of input to the person search unit 202 do not always appear around the person in the search result. Therefore, the comparison source image may be replaced with a person who can be identified as a search result from the person input to the person search unit 202, and the processing may be continuously performed.

For example, assume the case shown in FIG. FIG. 8 is a place such as a shopping mall, and it is assumed that a person is moving in various directions. In FIG. 8, 801 to 803 and A to D indicate a person, respectively, and are photographed by the first camera 804, the second camera 805, and the third camera 806, respectively. FIG. 8 conceptually shows the situation.

At this time, it is assumed that the person 801 is the person who is the first comparison source, and the people 802 and 803 are the people obtained as the search result of the person search unit 202. That is, the person 801 is moving to the positions of the people 802 and 803 as shown by the arrows in FIG. There are persons A and B around the person 801 and each of them advances in the direction of the dotted arrow. Since the persons A and B also exist around the person 802, the persons A and B are displayed as hints.

On the other hand, since the persons A and B do not exist around the person 803, the hint is not displayed. However, if it is already identified that the person 801 and the person 802 are the same person, the persons C and D exist around the person 802 and proceed in the direction of the dotted arrows, respectively. Since the persons C and D also exist around the person 803, the persons C and D can be displayed as hints between the person 802 and the person 803.

In this way, by sandwiching the identified person in between, the comparison source is changed one after another, and the hint is displayed even with the search result for which the hint cannot be displayed with the first comparison source. You will be able to do it. The display method is not limited, but in the example shown in FIG. 8, hint objects are displayed on the peripheral images of the persons 801, 802, and 803 so that the difference between the pairs can be understood. It should be displayed. It is expected that the above-mentioned processing will increase the chances of displaying hints when the user identifies a person, and the effect of support will be improved.

It should be noted that an embodiment in which the above-mentioned first to eighth embodiments are appropriately combined is also possible, and those embodiments are also included in the embodiment of the present invention.

(Other Embodiments of the present invention)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device read and execute the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

It should be noted that the above-described embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

100: Information processing device 101: CPU 102: ROM 103: RAM 104: Storage device 105: Input device interface 106: Output device interface 107: System bus 108: Input device 109: Output device 201: Video storage unit 202: Person search unit 203: Processing target judgment unit 204: Peripheral image acquisition unit 205: Hint object candidate extraction unit 206: Hint object identification unit 207: Hint display unit 208: Monitoring system 601: Hint display instruction receiving unit 701: Peripheral object similarity judgment unit

Claims (13)

A search method that searches for people with similar image characteristics to the person to be compared from the stored images and outputs the search results.
An image acquisition means for acquiring a first image including the person to be compared and a second image including the person from the search result, respectively.
An extraction means for extracting an object to be processed different from the comparison source person and the search result person from the first image and the second image.
The collation process of the object to be processed extracted from the first image and the second image is performed, and the identity of the person of the comparison source and the person of the search result is determined based on the result of the collation process. An information processing device characterized by having a specific means for specifying and providing information to be supported. The first aspect of claim 1, wherein the information that supports the determination of the identity specified by the specific means is displayed together with an image including the person of the comparison source and the person of the search result. Information processing equipment. The information processing apparatus according to claim 1 or 2, further comprising a determination means for determining an object to be processed based on the first image and the second image. The information processing apparatus according to claim 3, wherein the determination means determines an object to be processed based on a difference in position and time at which the first image and the second image are captured, respectively. .. The information processing device according to any one of claims 1 to 4, wherein the search means performs a search based on an image feature amount of the entire person. Any of claims 1 to 5, wherein the first image is a peripheral image including the person of the comparison source, and the second image is a peripheral image including the person of the search result. The information processing apparatus according to item 1. The information according to any one of claims 1 to 6, further comprising a receiving means for receiving an instruction as to whether or not to specify and provide information that supports the determination of the identity by the specific means. Processing equipment. It has a similarity determination means for determining whether or not the feature amount of the person of the comparison source or the person of the search result and the feature amount of a certain number or more of the objects around the person are similar by a certain value or more. ,
Any one of claims 1 to 7, wherein when it is determined by the similarity determination means that the information is similar by a certain value or more, a process relating to identification of information supporting the determination of identity is performed. The information processing device described in. The display means displays information that supports the determination of the identity specified by the specific means and information related to the collation process together with an image including the person of the comparison source and the person of the search result, respectively. 2. The information processing apparatus according to claim 2. When the determination means determines that the movement speed based on the difference in the position and time at which the first image and the second image are taken is the movement speed of a person, the object to be processed is a person. The information processing apparatus according to claim 3, wherein the information processing device is determined to be present. When the determination means determines that the movement speed based on the difference in the position and time at which the first image and the second image are taken is the movement speed of the vehicle, the object to be processed is processed by the vehicle. The information processing apparatus according to claim 3 or 10, wherein the information processing apparatus is determined to be present. A search process that searches for people with similar image features to the person to be compared from the stored images and outputs the search results.
An image acquisition step of acquiring a first image including the person of the comparison source and a second image including the person of the search result, respectively.
An extraction step of extracting an object to be processed different from the comparison source person and the search result person from the first image and the second image.
The collation process of the object to be processed extracted from the first image and the second image is performed, and the identity of the comparison source person and the search result person is determined based on the result of the collation process. An information processing method characterized by having a specific process for specifying and providing information to be supported. A search step that searches for a person whose image characteristics are similar to the person to be compared from the stored image and outputs the search result,
An image acquisition step of acquiring a first image including the person of the comparison source and a second image including the person of the search result, respectively.
An extraction step of extracting an object to be processed different from the comparison source person and the search result person from the first image and the second image.
The collation process of the object to be processed extracted from the first image and the second image is performed, and the identity of the comparison source person and the search result person is determined based on the result of the collation process. A program that lets a computer perform specific steps that identify and provide information to assist.

Images