JP6079447B2 - SEARCH DEVICE, SEARCH METHOD, AND SEARCH PROGRAM - Google Patents

Description

  The present invention relates to a search device, a search method, and a search program.

  In recent years, surveillance cameras have been installed in various places such as offices, shopping centers, and shopping streets. In general, in these facilities, since a large number of surveillance cameras are installed at a plurality of locations, it is impossible for the surveillance staff to constantly monitor the images of all the cameras on a 24-hour basis.

  Therefore, the usage of these surveillance cameras is to search for images of the suspicious person using the suspicious person's information when information about the suspicious person is provided by the witness. It is often used to check behavior and obtain more detailed information. Therefore, in such a security system, in order to reduce the burden on the work of the observer, there is a demand for a method in which the observer efficiently searches for a specific person from the witness testimony.

  The following method is known as a method for extracting an image that a specific person may be captured from images taken by a group of cameras installed at a plurality of locations. First, each person shown in the image is classified for each person based on the characteristics of each person. Then, when the characteristics of the search target person are given, the person having the relevant characteristics is identified from the characteristics of each person, and an image showing the identified person is extracted. In this method, a person photographed with a surveillance camera is detected, and feature amounts are extracted and stored in a database. indicate.

JP 2010-257449 A

  However, in general, witnesses who have found suspicious individuals, etc., have completely remembered the characteristics of suspicious individuals (appearance, dress, appearance, time, etc.), and depending on the memory of the witness Suspicious person information may be ambiguous. In such a case, it is necessary to set the features that serve as the search conditions so that they cover a wider range, and the search results that match the search conditions may become enormous if they are only grouped for each person. is there. Therefore, there is a problem that a method for reducing the search results (target person candidates) is necessary even when the search is performed using the information of the witnesses that are easily ambiguous.

  Therefore, as one aspect, the present invention reduces the number of objects hitting a search (candidates for a search object) even when there are many objects with similar external information obtained by sighting information, and the images An object of the present invention is to provide a search device, a search method, and a search program that reduce the load on the person in charge of visually confirming the search.

  A search device is provided. The search device moves the position at the plurality of times with respect to each of the plurality of moving bodies included in the plurality of images obtained by the imaging device that acquires a series of images at a plurality of times at a predetermined location. A tracking section that is acquired as a history of the above, a feature extraction section that extracts a physical feature for each of the plurality of moving bodies, and a witness that has witnessed a search target from the plurality of moving bodies The physical characteristics of the moving object and the physical characteristics of a plurality of moving objects, and a witness route acquisition unit that acquires the movement history of the witness, a predetermined time period determined by the movement history of the witness, and A search condition creation unit that sets a search condition to be present in a region, a candidate acquisition unit that acquires a part of the plurality of moving objects that satisfy a search condition as search target candidates, and outputs search target candidates An output unit that, characterized in that it comprises a.

  Even when there are many objects with similar external information, it is possible to reduce the number of objects that are hit in the search (candidates for the search object), and to reduce the burden on the person in charge who visually confirms these images.

It is a figure explaining the condition where the search device of an embodiment is used. It is an example of a functional block diagram of a search device of an embodiment. It is a figure which shows the example of the person information table stored in a person information database. It is a figure which shows the example of the route information table stored in a route information database. It is a figure which shows the example of the flow of a process in a data registration process. It is a figure which shows the example of the flow of a process in a search process. It is a figure explaining the example of a person's movement. It is a figure which shows the example of the person information table of the person stored in a person information database in Example 1. FIG. It is a figure which shows the example of the route information table of the person S stored in a route information database in Example 1. FIG. It is a figure which shows the example of the path | route information table of all the persons stored in a path | route information database in Example 1. FIG. It is a figure for demonstrating the example of the search conditions at the time of considering route information of witnesses. It is a figure for demonstrating the example of the search conditions at the time of considering route information of witnesses. It is a figure for demonstrating the example of the search conditions at the time of considering route information of witnesses. It is a figure for demonstrating the example of the search conditions when not considering route information of witnesses. It is a figure for demonstrating the example of the search conditions when the direction which the witness is facing is considered. It is a figure for demonstrating the example of the search conditions when the direction which the witness is facing is considered. It is a figure for demonstrating the example of the search conditions when the direction which the witness is facing is considered. It is a figure which shows the example of the path | route information table stored in a path | route information database in Example 2. FIG. It is a figure for demonstrating the example of search conditions when the moving direction of witnesses is considered (when moving to the same direction as a suspicious person). It is a figure for demonstrating the example of search conditions when the moving direction of witnesses is considered (when moving to the same direction as a suspicious person). It is a figure for demonstrating the example of search conditions when the moving direction of witnesses is considered (when moving to the same direction as a suspicious person). It is a figure for demonstrating the example of search conditions when the moving direction of an witness is considered (when moving in the direction opposite to a suspicious person). It is a figure for demonstrating the example of search conditions when the moving direction of an witness is considered (when moving in the direction opposite to a suspicious person). It is a figure for demonstrating the example of search conditions when the moving direction of an witness is considered (when moving in the direction opposite to a suspicious person). It is a figure which shows the example of the path | route information table stored in a path | route information database in Example 3. FIG. It is a figure for demonstrating the example of the search conditions when the movement speed of an witness is considered (when catching up with a suspicious person). It is a figure for demonstrating the example of the search conditions when the movement speed of an witness is considered (when catching up with a suspicious person). It is a figure for demonstrating the example of the search conditions when the movement speed of an witness is considered (when catching up with a suspicious person). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when overtaken by a suspicious person). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when overtaken by a suspicious person). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when overtaken by a suspicious person). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when a suspicious person has stopped). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when a suspicious person has stopped). It is a figure for demonstrating the example of search conditions when the movement speed of an eyewitness is considered (when a suspicious person has stopped). It is a figure which shows the example of a structure of the search device of embodiment.

Hereinafter, a search device and a search method according to an embodiment will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a situation in which a search device according to an embodiment is used. The example shown in FIG. 1 is for the following description, and the search device of one embodiment may be used in a more general situation. In the following description, the search target is a person, but the search target is not limited to a person. For example, the vehicle may be a moving body in which the witness's testimony is useful for searching, such as a vehicle, a non-human animal, a flying object, or a ship.

  As shown in FIG. 1, it is assumed that a path shaped like a letter H rotated 90 degrees clockwise is monitored by six imaging devices. The imaging device can be a camera. The passage includes passages R1 and R3 extending in the lateral direction and a passage R2 connecting the passages R1 and R3.

  It is assumed that the region including the passage shown in FIG. 1 is, for example, a range of 8 m in length and 10 m in width. Assume that each of the six imaging devices can capture the range of the areas A1 to A6. At this time, the imaging range of each imaging device is assumed to be 2 m square. One imaging device is arranged in each of the areas A1 to A6, and is arranged in the upper left corner of each area shown in FIG. As for the coordinates of the space, the upper left corner of the area A1 in FIG. 1 is the origin, the horizontal direction is the x axis, and the lower direction is the y axis.

  The search device and the search method described below can reduce the number of search object candidates by referring to the witness testimony of the witness who has moved through such a passage from among those who have moved through such a passage. .

<Search device>
FIG. 2 is an example of a functional block diagram of the search device 100 according to the embodiment. As will be described later, the apparatus 100 can be configured by a general-purpose computer such as a personal computer. However, the apparatus 100 may be configured by a dedicated circuit. In the present apparatus 100, when searching for a specific person from a large amount of video using information on the characteristics of a suspicious person obtained from a witness, the number of suspicious persons is reduced and the burden on the work of the observer is reduced. Can do. That is, when a search target is extracted from a plurality of moving objects shown in an image obtained by the imaging device, a moving object that satisfies a predetermined relationship between the witness and the moving object is extracted and set as a search target candidate. Thus, the number of candidates can be reduced.

  The search apparatus 100 includes a video acquisition unit 102, a person detection / search unit 104, a feature extraction unit 106, a person information database (DB) 108, a person association unit 110, a person feature acquisition unit 112, a witness route acquisition unit 114, a route An information database (DB) 116, a search condition creation unit 118, a suspicious candidate acquisition unit 120, and a result output unit 122 are included.

  Two or more imaging devices 200a, 200b,..., 200n for capturing a monitoring video are connected to the search device 100. The imaging devices 200a, 200b,..., 200n may be cameras such as CCD cameras.

  The search device 100 and the imaging devices 200a, 200b,..., 200n are combined to form the search system 10.

  Surveillance images captured by the imaging devices 200a, 200b,..., 200n are sent to the video acquisition unit 102 of the search device 100. In the example of FIG. 1, one image pickup device is arranged in each of the areas A1 to A6 in the upper left corner of each area shown in FIG. 1.

  The video acquisition unit 102 captures video from the imaging devices 200a, 200b,. If the acquired video is a moving image, it is converted into an image for each frame for subsequent processing.

  The person detection / tracking unit 104 detects a person from the image created by the video acquisition unit 102 and tracks the same person between temporally continuous images. The person detection / tracking unit 104 tracks a person by associating with a person detected in a frame corresponding to the previous time. The person detection / tracking unit 104 may be simply referred to as the tracking unit 104. As described above, the person detection / tracking unit 104 performs each of a plurality of moving bodies included in the plurality of images obtained by the imaging devices 200a to 200n that acquire a series of a plurality of images at a plurality of times at a predetermined place. Thus, the positions at a plurality of times are acquired as movement histories.

  The feature extraction unit 106 extracts the feature of the person detected by the person detection / tracking unit 104. The feature amount includes, for example, a person's body shape such as a color of a person's clothes and a height and a body width. If it is the color of a person's clothes, it can obtain by averaging each RGB component of the detected upper part of the person. Furthermore, the feature extraction unit 106 stores the extracted person features in the person information database (DB) 108 as a person information table.

  The person information database (DB) 108 stores information related to a person information table including the characteristics of each person detected from the image created by the video acquisition unit 102.

  The person information table includes information on the RGB components, height, and body width of the upper body of each person.

FIG. 3 is a diagram illustrating an example of a person information table stored in the person information DB 108.
The person information DB 108 stores a person information table including movement route information and person characteristic information. The movement path information describes how it moved within the shooting range of the camera in terms of time and position coordinates x and y, and the person feature information is obtained by integrating the person features of each frame acquired during tracking as one piece of information. Is described. As for the movement route information, the position information is stored in order of time. In this example, position data is stored every second, but it may be stored in finer units. As for how to summarize the person feature information, for example, a method of using the value when the largest person was photographed during tracking of the person may be used. For example, a method may be used in which all values are averaged when completed (when a person disappears from the image). This person information DB is a collection of person information using images in each camera, and is not considered for movement between cameras.

  In the example of the person information table shown in FIG. 3, for example, the color of the person specified by the data number 0001 is a color represented by (R, G, B) = (100, 50, 60). . The height is 170 cm and the width of the body is 45 cm.

  The example of the person information table shown in FIG. 3 includes the movement path of the person specified by the data number 0001. In the example shown in FIG. 3, for example, the person specified by the data number 0001 has an xy coordinate of (x, y) = (110, 210) at 10: 00.00 on January 10, 2013. At the position specified by. Even one second later, at 10: 00.01 on January 10, 2013, the xy coordinates are at the position specified by (x, y) = (110, 210). Then, at 10: 02.00 on January 10, 2013, the xy coordinates have moved to the position specified by (x, y) = (200, 400). Such route information can be entered in the person information table by the person association unit 110 described later. In the person information table, the average speed for each person is entered. The average speed can be calculated by the person association unit 110 described later and entered in the person information table.

  The person association unit 110 associates the same person among images obtained from the monitoring images obtained by the plurality of imaging devices 200a, 200b,.

  When the information data on the new person is created, the person association unit 110 compares the person information data with the data stored in the path information database (DB) 116, and is the same person as the person of any path information. It is determined whether it is. When it is determined that the person is the same person as one of the persons included in the route information, the route information table is updated using the person information data. If the corresponding route information data does not exist, information about the person is added and registered in the route information table of the route information DB 116 as new route information data. In addition, the person association unit 110 enters the route information in a person information table stored in the person information DB 108.

  FIG. 4 is a diagram illustrating an example of a route information table stored in the route information database. The structure of the route information table is the same as that of the person information DB 108, and includes a movement route table that includes movement route information that is information about the movement route and person feature information that is information about the characteristics of the person.

  In the example shown in FIG. 4, for example, the person specified by the data number 0001 has an xy coordinate of (x, y) = (110, 210) at 10: 00.00 on January 10, 2013. At the position specified by. Even one second later, at 10: 00.01 on January 10, 2013, the xy coordinates are at the position specified by (x, y) = (110, 210). Then, at 10: 10.30 on January 10, 2013, the xy coordinates have moved to the position specified by (x, y) = (5000, 1000). Data such as the color (R, G, B), height, and width of the person specified by the data number 0001 is entered with reference to the person information DB 108. Further, the average speed for each person is entered in the route information table. The average speed may be calculated based on the route information by the person association unit 110 and entered in the route information table.

  Further, the movement direction of the witness may be added to the movement route information in the route information database (DB). This movement direction may be obtained by using the positions of previous and subsequent times when the movement route is recorded in the route information table. Thus, by considering the movement direction of the witness, it is possible to reduce the number of suspicious candidates.

The determination of the same person is regarded as the same person when the feature amounts of the person information data and the path information data are similar. For example, if the similarity is calculated using color information, a correlation value between two values (R1, G1, B1) and (R2, G2, B2) may be obtained as shown in the following equation. The distance normalized correlation cor in the three-dimensional space of RGB is, for example,

Can be defined as

Alternatively, the distance between two points in the RGB three-dimensional space may be obtained. The distance dis between two points on the three-dimensional space of RGB is
Can be defined as If these values are within the threshold, it is determined that the colors are similar.

  The person association unit 110 also merges person information data and route information data. Specifically, for the route information, the route information of the person information data is added to the end of the associated route information data. However, in the person information data, only path information in a situation where a person is photographed by the imaging devices 200a, 200b,..., 200n and detection processing is being performed is recorded. That is, there is no information at a place where the imaging devices 200a, 200b,. Therefore, at the time of the update, a movement route at a place not photographed by the imaging devices 200a, 200b,. For example, there is a method of interpolation using the final position of the route information data and the first position of the person information data to be added, assuming that the vehicle travels straight at a constant speed. Alternatively, by using map information of the shooting environment, a person's movement route in a section not photographed by the camera may be registered in advance, and the position information may be obtained from the person's movement speed and the registered route.

  The information of the person feature amount is updated by using an average value of both data or by adding a weight to the feature amount of each data.

  The person feature acquisition unit 112 determines whether there is a search request from the user. The person feature acquisition unit 112 acquires the characteristics of the witness and the suspicious person input from the user, and uses the data stored in the route information database (DB) 116 or the person information database (DB) 108. Identify the witness.

  Specifically, the input feature quantity is compared with the feature quantity registered in the route information database (DB) 116 or the person information database (DB) 108, and the person with the highest similarity is found. At this time, if there are a plurality of persons with high similarity, the information of all persons is displayed, and the witnesses are surely selected by the route requesting database (DB) 116 by having the person who made the search request select. Or it searches from person information database (DB) 108.

  In addition, since the information of the witness becomes the information of itself, it can be searched with more detailed information, so that the number of candidates can be very small.

  The witness route acquisition unit 114 refers to the route information database (DB) 116 and acquires the movement route of the witness extracted by the person feature acquisition unit 112.

The search condition creation unit 118 creates a search condition for a suspicious person. Search criteria for suspicious people include
(S1) Where we were at a certain time
(S2) What kind of clothes were you wearing?
(S3) Which direction did you go?
(S4) How fast were you moving?
Etc. are included. Since the sighting information includes ambiguity, regarding time and place, a time zone and a region may be determined, and a person existing in that time zone and that region may be a candidate.

  A relationship when a suspicious person is witnessed may be added to the search condition. The relationship when the suspicious person is witnessed may include, for example, “passed the suspicious person”, “walked in the same direction as the suspicious person”, and “passed by the suspicious person”.

  Further, the search condition creating unit 118 may correct the position and time information of the search condition of the suspicious person. Specifically, the location of the witness at a certain distance from the location at a certain distance, the time of the feature information of the suspicious person input, the time obtained by ANDing the location information, and the location of the suspicious person The position of the condition and time information may be used. This change in condition always adds a condition that there was a suspicious person in the vicinity of the witness from the fact that the witness is looking at the suspicious person. Further, by using the movement route, it is possible to determine whether or not the image is present in the vicinity of the witness even in a range not photographed by the imaging devices 200a, 200b,.

  When the search condition creation unit 118 corrects the position of the suspicious person's search condition using the movement history of the witness, the range is within a certain distance from the position of the witness. Only the visible range may be used as the search position range. The estimation method of the range that can be seen by the witness is, for example, that the person is looking forward when moving, so the movement direction of the witness is obtained from the position information of the time before and after, and a certain range is set only in the movement direction Alternatively, the face direction may be obtained, and a certain range may be set only in the face direction. By setting the range in which the witness can visually recognize, the range where the suspicious person is present can be narrowed, and the number of suspicious candidates can be reduced.

  Further, when creating the search condition from the movement route, the search condition creating unit 118 may add the movement direction of the partner at the search position to the search condition. For example, when the information “I passed the suspicious person” is input and the witness has moved in the south direction, the suspicious person can be limited to those who have moved in the north direction.

  Further, when creating the search condition from the movement route, the search condition creating unit 118 may add the movement speed of the other party at the search position to the search condition. For example, when the information “I was overtaken by a suspicious person” is input, it can be understood that the suspicious person moves faster than the witness. Therefore, by comparing the moving speeds of the suspicious person and the witness at each time, it becomes possible to further limit the suspicious person candidates. Further, if the information “I saw a suspicious person who has stopped” is input, the candidate can be limited by searching for a person whose moving speed is slower than a predetermined threshold. As described above, the search condition creating unit 118 sets the search condition to be present in a predetermined time zone and region determined by the movement history of the witness. The search condition creating unit 118 may acquire a part of the plurality of moving objects as search target candidates using at least one of the color, height, and width of the moving object as a search condition. The search condition creating unit 118 also sets, as the search condition, that the correlation value or distance in the RGB three-dimensional space is equal to or less than a predetermined threshold as the predetermined condition when the distance is in the RGB three-dimensional space. Also good. The search condition creating unit 118 may also set at least one of the location, orientation, and speed of the moving body as viewed from the witness as the search condition.

  The suspicious candidate acquisition unit 120 refers to the route information database (DB) 116 and searches for a person matching the condition. That is, the suspicious person candidate acquisition unit 120 acquires a part of the plurality of moving objects that satisfy the search condition as search target candidates.

The result output unit 122 outputs search target candidates. For example, the result output unit 122 displays data matching the search condition as a suspicious candidate in a list. The result output unit 122 is merely an example of the processing that may be referred to simply as the output unit 122, and other methods can be used as long as the route information DB indicating the movement route of each person can be created. . For example, although color feature amounts are used when associating route information data and person information data, it is not simple color information, and as described in Non-Patent Document 2, a person is referred to as an arm or body. For example, there is a method in which each part is decomposed and color information is obtained and compared for each part, or a method using the gradient information of the luminance of the image called SIFT feature amount may be used.

  In addition, this time, the same person is determined using the feature amount (feature amount used for search) described in the route information table stored in the route information database (DB) 116, but the same person is determined. Different feature quantities may be used and the feature quantities used as search conditions may be different.

  According to the search device 100 configured in this way, even when there are many objects with similar external information, the number of objects hitting the search (search object candidates) is reduced and those images are visually confirmed. Can reduce the burden on the person in charge.

<Search method>
The processing of the search method will be described with reference to FIGS.
The search method process includes a data registration process and a search process.
FIG. 5 is a diagram illustrating an example of a process flow in the data registration process.

  When the search device 100 is configured by a general-purpose computer 300 as shown in FIG. 20 described later, the following description defines a control program that performs such processing. That is, hereinafter, it is also a description of a control program that causes a general-purpose computer to perform the processing described below.

  When the data registration process is started, in S102, the video acquisition unit 102 acquires videos from the imaging devices 200a, 200b,..., 200n and converts them into images for each frame. When the process of this step ends, the process proceeds to S104.

  In step S <b> 104, the person detection / search unit 104 detects a person from the image created by the video acquisition unit 102 and tracks the same person between temporally continuous images. When the process of this step ends, the process proceeds to S106.

  In S106, the feature extraction unit 106 extracts the features of the person detected in S104. When the process of this step ends, the process proceeds to S108.

  In step S108, the person association unit 110 determines whether the person being tracked has moved out of the shooting range. If the result of this determination is “Yes”, that is, if the person being tracked has moved out of the shooting range, the process proceeds to S110. If the result of this determination is “No”, that is, if the person being tracked has not moved out of the shooting range, the process returns to S102.

  In S <b> 110, the person association unit 110 collectively registers the characteristics of the person in the person information database (DB) 108. When the process of this step ends, the process proceeds to S112.

  In S112, the person association unit 110 compares the data stored in the person information database (DB) 108 with the data stored in the path information database (DB) 116, and determines the person being tracked as the path information database. (DB) Corresponds to a person in the route information table of 116. When the process of this step ends, the process proceeds to S114.

In step S114, the person association unit 110 updates the route information database using the person information data. When there is route information corresponding to the person information data in S112, the person association unit 110 is stored in the person information data stored in the person information database (DB) 108 and the route information database (DB) 116. Merge the route information data. If the corresponding route information data does not exist, information about the person may be added to the route information table of the route information database (DB) 116 and registered as new route information data.
With the above processing, the data registration processing is completed.

FIG. 6 is a diagram illustrating an example of a flow of processing in the search processing.
When the search process is started, in step S202, the person feature acquisition unit 112 determines whether there is a search request from the user. If the result of this determination is “Yes”, that is, if there is a search request from the user, the process proceeds to S204. If the result of this determination is “No”, that is, if there is no search request from the user, the process of S202 is repeated.

  In S204, the person feature acquisition unit 112 acquires the characteristics of the witness and the suspicious person input from the user, and uses the data stored in the route information database (DB) 116 or the person information database (DB) 108. Identify the witness. When the process of this step ends, the process proceeds to S206.

  In S <b> 206, the witness route acquisition unit 114 refers to the route information database (DB) 116 and acquires the movement route of the witness extracted by the person feature acquisition unit 112. When the process of this step ends, the process proceeds to S208.

  In S208, the witness route acquisition unit 114 calculates the time / place where the witness may have seen the suspicious person from the movement route. For example, the witness route acquisition unit 114 refers to the route information database (DB) 116 and calculates the movement route of the witness extracted by the person feature acquisition unit 112. When the process of this step ends, the process proceeds to S210.

  In S210, the personal information DB 108 is searched using the characteristic amount of the suspicious person and the time / place where there is a possibility of sighting. When the process of this step ends, the process proceeds to S212.

  In S212, the result output unit 122 displays the search result. For example, the result output unit 122 may display a list of suspicious candidate candidates on the display for data that matches the search condition.

  By using such a method, even when there are many objects with similar external information, the number of objects that hit the search (search object candidates) is reduced, and the person in charge who visually confirms these images The load can be reduced.

<Example 1>
Hereinafter, an example of the operation of the search device 100 will be described with reference to FIGS. In this example, as a search condition for a suspicious person, reference is made to (S1) where the person was at a certain time and (S2) what color clothes were worn.

  As shown in FIG. 1, an environment is assumed in which six imaging devices 200a, 200b, 200c, 200d, 200e, and 200f are shooting a range of 8 m in length and 10 m in width. Assume that the areas A1 to A6, which are the imaging ranges of the respective imaging devices 200a, 200b,..., 200f, are 2 m square, and four persons move as shown in FIG. The coordinate origin is the upper left of the area A1, and the x coordinate is taken in the horizontal direction and the y coordinate is taken in the vertical direction. In order to simplify the explanation, it is assumed that the person information DB 108 has only the color information of the person as the person feature amount.

FIG. 7 is a diagram for explaining an example of the movement of a person in the place as shown in FIG.
In FIG. 7, the person X follows a path P2 that passes through the areas A1, A2, and A3. The person Y follows a path P3 that passes through the areas A3, A2, A5, and A6. The person Z follows a path P4 that passes through the areas A6, A5, A2, and A3. Then, the person S follows a route P1 that passes through the areas A1, A2, A5, and A6. It is assumed that the person S is a witness.

  Since the path of the person S who is the witness is at least partially coincident with any of the persons X, Y, and Z, the person S may witness any of the persons X, Y, and Z as long as the timing is met. is there.

First, a person is detected from images captured by each imaging device.
FIG. 8 is a diagram illustrating an example of a person information table of persons stored in the person information database in the first embodiment. For example, the color (R, G, B) of the person specified by the data number 0001 is (230, 50, 60). The color (R, G, B) of the person specified by the data number 0002 is (240, 55, 55).

  When the person information is gathered, the person information is associated between the images obtained by the six imaging devices 200a, 200b, 200c, 200d, 200e, and 200f, and the route information DB 116 is created. The movement paths in the areas not photographed by the six imaging devices 200a, 200b, 200c, 200d, 200e, and 200f are estimated by linear interpolation, and position information at all times is stored.

  FIG. 9 is a diagram illustrating an example of the route information table of the person S stored in the route information database in the first embodiment. In the example shown in FIG. 9, the person is identified by the data number 01, and the colors (R, G, B) are (230, 50, 54). When a person closest to this characteristic is searched from the persons described in the person information DB 108, the person specified by the data number 0001 is specified, and this person is specified as the person S.

  FIG. 10 is a diagram illustrating an example of the route information table for all persons stored in the route information database in the first embodiment.

  Now, it is assumed that the person S who is a witness testifies that he saw a suspicious person wearing white clothes around 9:30 around the imaging device 200b installed in the area A2. Since there is ambiguity in this information in terms of “around the imaging device 200b” and “about 9:30”, it is necessary to specify a range with a margin in order to reliably search for suspicious persons. For example, the search condition is “a person in white clothes at a position in the range (300, 0) to (1000, 800)” from 9:00 to 10:00 so that the imaging devices 200b, 200c, 200d, and 200f are included.

  FIG. 12 is a diagram for explaining an example of the search condition when the route information of the witness is not considered. As shown in FIG. 12, if the movement path of the witness is not used, searcher candidates are searched from a wide range such as “inside the frame B”.

  In this case, all of the three persons, that is, the person X with the data number 02, the person Y with the data number 03, and the person Z with the data number 04 shown in FIG.

  However, in this example, the condition of the witness is further input as a condition. For example, the condition is “a person in red clothes at the position of xy coordinates (0, 0) to (1000, 800)” from 9:00 to 10:00. Then, the data of the data number 01 is acquired from the route information table stored in the route information DB 116, and the movement route of the witness can be specified. Next, the search condition is changed using the movement path of the witness. In this example, if the person is in a 1.5 m square centered on the position where the witness is located as a condition, from the route information, as indicated by frame B in FIGS. Conditions such as coordinates (−150, −50) to (150, 250) and xy coordinates (−40, −50) to (240, 250) are created at time 9:20:01. When a search is performed as a logical product condition between this condition and the initially set condition, only the person Z with the data number 04 is acquired as a search result.

  In the above, the target was three people, but in an environment where more people move, the use of this method can surely reduce the number of suspicious candidates and reduce the burden on the user. Is possible.

  13A to 13C are diagrams for explaining examples of search conditions when the direction in which the witness is facing is considered.

  By estimating the range that can be visually recognized by the witness, it is possible to further identify the range where the suspicious person is present and to reduce the number of suspicious candidates. Here, in FIGS. 13A to 13C, the arrow V2 is the estimated direction of the witness, the region C is a region where a suspicious person may be present, and the region B surrounded by a dotted line is the direction of the witness. Represents an area where it is recognized that there is a possibility of a suspicious person not using. In this case, region C can be considered as the field of view of the witness. That is, if the witness witnesses the suspicious person, the suspicious person is in the area C where the witness is possible. Therefore, since it is not necessary to search for the region B surrounded by the dotted line, even when there are many similar objects, the number of objects hitting the search (search object candidates) is reduced, and those images are visually confirmed. The burden on the person in charge can be reduced.

<Example 2>
Another example of the operation of the search device 100 will be described with reference to FIGS. In this example, as a search condition for a suspicious person, (S1) not only where he was at a certain time and (S2) what color clothes he was wearing, but also (S3) which direction he was heading , Is referred to.

  In this example, the movement direction is added to the movement route information in the route information table of the route information DB 116. This movement direction is stored in addition to the route information by using the positions of previous and subsequent times when the movement route is registered in the route information DB. The moving direction is set from 0 degrees to 360 degrees so that the positive direction of the x axis is 0 degrees and the angle increases toward the positive direction of the y axis (clockwise in FIG. 1).

  An example of the route information DB in this example is shown in FIG. For example, the person identified by data number 0001 is designated by the (x, y) = (110, 210) xy coordinates at 10: 00.00 on January 10, 2013. You are in a position. And it turns out that it is moving toward the direction of 50 degrees. By using this, it becomes possible to reduce the number of suspicious candidates.

Only the parts different from the first embodiment will be described below.
In this example, when inputting the search condition, the relationship when the suspicious person is witnessed is input in addition to the case of the first embodiment. For example, information such as “passed suspicious person”, “walked in the same direction as suspicious person”, “passed by suspicious person”, or the like is input.

  Further, in addition to the case of the first embodiment, when the search condition is created from the movement route, the movement direction of the partner at the search position is added to the search condition.

  15A to 15C are diagrams for explaining examples of search conditions when the witness moves in the same direction as the suspicious person. In the figure, an arrow Vm indicates the moving direction of the suspicious person.

  For example, if the witness has moved in the 0 degree direction and “walked in the same direction as the suspicious person”, it exists in the area indicated by the range as shown in FIGS. It becomes a condition of a person with direction.

  16A to 16C are diagrams for explaining examples of search conditions when the witness moves in the opposite direction to the suspicious person. In the figure, an arrow Vm indicates the moving direction of the suspicious person. For example, if the witness is moving in the 0 degree direction and the information “I passed the suspicious person” is input, it exists in the area indicated by the range as shown in FIGS. Furthermore, it is a condition that the person has the moving direction of the arrow.

<Example 3>
With reference to FIGS. 17A to 19C, another example of the operation of the search device 100 will be described. In this example, as a search condition for a suspicious person, (S1) where the person was at a certain time, (S2) what kind of clothes he was wearing, (S3) which direction he was heading, and ( S4) It is referred to how fast the vehicle is moving.

  In this example, the moving speed is added to the moving route information in the route information table of the route information DB 116 of the second embodiment. This movement speed is obtained by using the positions of the previous and next times (positions around one time or longer times may be used) when registering the movement route in the route information DB 116, In addition, store. An example of the route information table stored in the route information DB 116 is shown in FIG. For example, the person specified by the data number 0001 is at a position specified by (x, y) = (110, 210) at 10: 00.00 on January 10, 2013. And it is moving toward the direction of 50 degrees, and it turns out that the speed is 10 cm / sec. In this way, it is possible to reduce the number of suspicious candidates by using the moving speed of the person.

Only the parts different from the second embodiment will be described below.
When inputting the search condition, in addition to the case of the second embodiment, the relationship between the movement speed of the witness and the suspicious person when the suspicious person is witnessed is inputted. For example, information such as “I saw a suspicious person who stopped”, “I was overtaken by a suspicious person”, and “I caught up with a suspicious person”.

  In addition to the case of the second embodiment, when the search condition is created from the movement route, the movement speed of the opponent at the search position is added to the search condition. For example, if the information “I was overtaken by a suspicious person” has been input, it can be seen that the suspicious person has a faster moving speed than the witness. Therefore, by comparing the moving speeds of the suspicious person and the witness at each time, it becomes possible to further limit the suspicious person candidates. In addition, if the information “I saw a suspicious person who has stopped” is input, the candidate can be limited by searching for a person whose moving speed is slower than a predetermined threshold.

  For example, in the case of “catch up with a suspicious person”, a person who has a moving direction of an arrow that exists in a region indicated by a darkly painted area at a specific time as shown in FIGS. The condition is that it is slow. As in the case of the first embodiment, when the search position is displayed at each time using the route of the witness, there is a suspicious person in the darkly painted range B shown in FIGS. I understand. Further, the movement speed of the witness is 100 cm / sec in the direction of V1 at time T = 09: 20.00, 120 cm / sec in the direction of V1 at time T = 09: 20.01, and time T = 09: It is assumed that the robot moves at a speed of 70 cm / sec in the direction of V1 at 20.02. Then, because the witness “has caught up with the suspicious person”, the suspicious person is less than 100 cm / sec in the direction of V1 at time T = 09: 20.00, and the speed in the direction of V1 at time T = 09: 20.01. The condition of a person who moved at a speed of less than 70 cm / sec in the direction of V1 at time T = 09: 20.02 is added, and the range, the moving direction, and the moving speed shown in FIGS. 18A to 18C are search conditions. It becomes. This makes it possible to limit suspicious candidates.

  On the other hand, the case of “passed by a suspicious person” is as shown in FIGS. 19A to 19C. In this case as well, in the same way as in the first embodiment, when the search position is displayed at each time using the route of the witness, the suspicious person is within the darkly painted range B shown in FIGS. 19A to 19C. It can be seen that exists. In the same manner as described above, the movement speed of the witness is 100 cm / sec in the direction of V1 at time T = 09: 20.00, 120 cm / sec in the direction of V1 at time T = 09: 20.01, It is assumed that the robot moves at a speed of 70 cm / sec in the direction of V1 at time T = 09: 20.02. Then, because the witness was “passed by the suspicious person”, the suspicious person moved at a speed of 100 cm / sec or more in the direction of V1 at time T = 09: 20.00, and in the direction of V1 at time T = 09: 20.01. A condition of a person moving at a speed of at least 70 cm / sec in the direction of V1 at time T = 09: 20.02 at time T = 09: 20.02 is added, and the range, moving direction, and moving speed shown in FIGS. 19A to 19C are searched. It becomes a condition. This makes it possible to limit suspicious candidates. In the case of “I have seen a suspicious person who has stopped,” the suspicious person's condition is not the moving direction but only the moving speed, as shown in FIGS. 20A to 20C. As shown in FIGS. 20A to 20C, each time T = 09: 20.00 to 09: 20.02 is in a region B that overlaps the region A1, and moves at a moving speed V3 = 5 cm / sec or less. It can be limited to a person. The moving speed is set to 5 cm / sec in consideration of a slight movement even when the movement is stopped, but only a person with 0 cm / sec may be searched out.

  In this example, the moving speed is calculated at each time. However, the moving speed of each person is calculated by averaging the moving speed or divided by a predetermined period (for example, every minute) and used. May be.

  FIG. 21 is a diagram illustrating an example of the configuration of the search device 100 according to the embodiment. The search device 100 can be realized as a general-purpose computer 300.

  The computer 300 includes an MPU 302, a ROM 304, a RAM 306, a hard disk device 308, an input device 310, a display device 312, an interface device 314, and a recording medium driving device 316. Note that these components are connected via a bus line 320, and various data can be exchanged under the management of the MPU 302.

  An MPU (Micro Processing Unit) 302 is an arithmetic processing unit that controls the operation of the entire computer 300, and functions as a control processing unit of the computer 300.

  A ROM (Read Only Memory) 304 is a read-only semiconductor memory in which a predetermined basic control program is recorded in advance. The MPU 302 can control the operation of each component of the computer 300 by reading out and executing the basic control program when the computer 300 is activated.

  A RAM (Random Access Memory) 306 is a semiconductor memory that can be written and read at any time and used as a working storage area as necessary when the MPU 302 executes various control programs.

  The hard disk device 308 is a storage device that stores various control programs executed by the MPU 302 and various data. The MPU 302 can perform various control processes described later by reading and executing a predetermined control program stored in the hard disk device 308.

  The input device 310 is, for example, a mouse device or a keyboard device. When operated by a user of the search device 100, the input device 310 acquires input of various information associated with the operation content, and sends the acquired input information to the MPU 302. To do.

  The display device 312 is a liquid crystal display, for example, and displays various texts and images according to display data sent from the MPU 302.

  The interface device 314 manages the exchange of various information with various devices connected to the computer 300.

  The recording medium driving device 316 is a device that reads various control programs and data recorded on the portable recording medium 318. The MPU 302 can read out and execute a predetermined control program recorded in the portable recording medium 318 via the recording medium driving device 316 to perform various control processes described later. As the portable recording medium 318, for example, a flash memory provided with a USB (Universal Serial Bus) standard connector, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Only Only). and so on.

  In order to configure the search device 100 using such a computer 300, for example, a control program for causing the MPU 302 to perform the processing in each processing unit described above is created. The created control program is stored in advance in the hard disk device 308 or the portable recording medium 318. Then, a predetermined instruction is given to the MPU 302 to read and execute the control program. By doing so, the MPU 302 provides the functions of the search device 100.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
For each of a plurality of moving objects included in the plurality of images obtained by an imaging device that acquires a series of images at a plurality of times at a predetermined location, the positions at the plurality of times are used as movement histories. A tracking unit to obtain,
A feature extraction unit for extracting a physical feature for each of the plurality of moving bodies;
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, An eyewitness route acquisition unit for acquiring the movement history;
A search condition creation unit that sets a search condition to exist in a predetermined time zone and region determined by the movement history of the witness;
A candidate acquisition unit that acquires, as search target candidates, a part of the plurality of the moving objects that satisfy the search conditions;
An output unit for outputting the search target candidates;
Search device including
(Appendix 2)
The search condition creating unit further acquires a part of the plurality of moving objects as search target candidates using at least one of the color, height, and width of the moving object as the search condition. apparatus.
(Appendix 3)
The search condition creating unit further sets, as the search condition, that the correlation value or the distance in the RGB three-dimensional space is equal to or less than a predetermined threshold as the predetermined condition when the distance is in the RGB three-dimensional space. Search device according to appendix 2 (appendix 4)
The search device according to any one of appendices 1 to 3, wherein the search condition creating unit further sets at least one of the location, orientation, and speed of the moving body as viewed from the witness as the search condition. .
(Appendix 5)
A search method processed by a computer,
Acquiring a series of images at a plurality of times at a given location;
For each of a plurality of moving bodies included in the plurality of images, the position at the plurality of times is acquired as a movement history.
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, Obtaining a history of the movement;
Setting as a search condition to exist in a predetermined time zone and region determined by the movement history of the witness;
Acquiring a part of the plurality of moving objects that satisfy the search condition as search target candidates;
Outputting the search candidate,
Search method including
(Appendix 6)
The search method according to appendix 5, including acquiring a part of the plurality of moving objects as search target candidates using at least one of the color, height, and width of the moving object as the search condition.
(Appendix 7)
When the distance is in the RGB three-dimensional space, the predetermined condition includes setting as a search condition that a correlation value or a distance in the RGB three-dimensional space is a predetermined threshold value or less. Search method (Appendix 8)
The search method according to any one of appendices 5 to 7, further comprising setting at least one of a location, an orientation, and a speed of the moving body viewed from the witness as the search condition.
(Appendix 9)
Get a series of images at multiple times in a given location,
For each of a plurality of moving bodies included in the plurality of images, the position at the plurality of times is acquired as a movement history,
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, Get the movement history,
It is set as a search condition that it exists in a predetermined time zone and region determined by the movement history of the witness,
Obtaining a part of the plurality of moving objects satisfying the search condition as search target candidates;
Outputting the search target candidates;
A program that causes a computer to execute processing.
(Appendix 10)
The program according to appendix 9, wherein a part of the plurality of moving objects is acquired as a search target candidate using at least one of the color, height and width of the moving object as the search condition.
(Appendix 11)
The program according to appendix 10, wherein when the distance is in the RGB three-dimensional space, the search condition that the correlation value or the distance in the RGB three-dimensional space is equal to or less than a predetermined threshold is set as the predetermined condition. 12)
The program according to any one of appendices 9 to 11, wherein at least one of the location, orientation, and speed of the moving body viewed from the witness is set as the search condition.

DESCRIPTION OF SYMBOLS 10 Search system 100 Search apparatus 102 Image | video acquisition part 104 Person detection and tracking part (tracking part)
106 Feature Extraction Unit 108 Person Information Database (DB)
110 Person association unit (association unit)
112 person feature acquisition unit 114 witness route acquisition unit 116 route information database (DB)
118 Search condition creation unit 120 Suspicious candidate acquisition unit 122 Result output unit (output unit)
200a, ..., 200n Imaging device

Claims (6)

For each of a plurality of moving objects included in the plurality of images obtained by an imaging device that acquires a series of images at a plurality of times at a predetermined location, the positions at the plurality of times are used as movement histories. A tracking unit to obtain,
A feature extraction unit for extracting a physical feature for each of the plurality of moving bodies;
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, An eyewitness route acquisition unit for acquiring the movement history;
A search condition creation unit that sets a search condition to exist in a predetermined time zone and region determined by the movement history of the witness;
A candidate acquisition unit that acquires, as search target candidates, a part of the plurality of the moving objects that satisfy the search conditions;
An output unit for outputting the search target candidates;
Search device including   2. The search condition creating unit according to claim 1, further comprising acquiring a part of the plurality of moving objects as search target candidates using at least one of color, height, and width of the moving object as the search condition. Search device.   The search condition creating unit further sets, as the search condition, that the correlation value or the distance in the RGB three-dimensional space is equal to or less than a predetermined threshold as the predetermined condition when the distance is in the RGB three-dimensional space. The search device according to claim 2.   The search according to any one of claims 1 to 3, wherein the search condition creating unit further sets at least one of the location, orientation, and speed of the moving body as viewed from the witness as the search condition. apparatus. A search method processed by a computer,
Acquiring a series of images at a plurality of times at a given location;
For each of a plurality of moving bodies included in the plurality of images, the position at the plurality of times is acquired as a movement history.
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, Obtaining a history of the movement;
Setting as a search condition to exist in a predetermined time zone and region determined by the movement history of the witness;
Acquiring a part of the plurality of moving objects that satisfy the search condition as search target candidates;
Outputting the search candidate,
Search method including Get a series of images at multiple times in a given location,
For each of a plurality of moving bodies included in the plurality of images, the position at the plurality of times is acquired as a movement history,
The witness who witnessed the search object is identified from the plurality of the moving objects by comparing the physical characteristics of the witness with the physical characteristics of the plurality of the moving objects, Get the movement history,
It is set as a search condition that it exists in a predetermined time zone and region determined by the movement history of the witness,
Obtaining a part of the plurality of moving objects satisfying the search condition as search target candidates;
Outputting the search target candidates;
A program that causes a computer to execute processing.