JP2020047259A - Person search system and person search method - Google Patents

Abstract

To provide a person search system and a person search method in which, when an incident or the like occurs at a point where many persons come and go, a captured image of a person who is visually similar to a suspect is extracted with high accuracy, a prompt grasp of visual characteristics or the like of the suspect is efficiently supported, and the convenience of a police investigation is certainly improved.SOLUTION: When different captured videos transmitted respectively from n cameras are received, a server analyzes position information and characteristic information on a person appearing in each of the captured videos and accumulates as analysis results. A client terminal displays, on a screen, the captured videos of k cameras (k≤n). According to a selection of a search target person appearing in any of them, the client terminal transmits, to the server, a search request for the search target person in the remaining k-1 captured videos. The server extracts the analysis results regarding the search target person in the k-1 captured videos, and transmits them to the client terminal. The client terminal displays, side by side, cut-out images of multiple persons identical or similar to the search target person on the basis of the analysis results regarding the search target person.SELECTED DRAWING: Figure 15

Description

  The present disclosure relates to a person search system and a person search method for searching for a person appearing in the field of view of a camera using an image captured by the camera.

  Patent Literature 1 extracts a captured image of a suspect from a captured image recorded in a recorder when a predetermined event is detected in a monitoring area, and displays a transition captured image based on the behavior of the suspect. In addition, a security system that displays map information indicating the imaging position of a captured image is disclosed. The security system displays, on the map information, the position information of the imaging point of the corresponding captured image when the selection input of the captured image of the position display target is received for the transition captured image. This makes it possible to visually obtain tracking information on the behavior of a monitoring target such as a person in the monitoring area.

JP 2017-40983 A

  However, in Patent Literature 1, a camera that captures a person visually resembling a person (that is, a criminal) that caused an incident or accident (hereinafter, “incident or the like”) that occurred at a point (for example, an intersection) where many people come and go is described. No consideration is given to specifying a captured image and retrieving a captured image similar to a clue from the captured video of another camera with the specified captured image.

  When an incident or the like occurs, it is important to grasp the visual characteristics and the whereabouts of the criminal at an early stage in an initial police investigation. However, in the conventional technology up to now, clues such as images taken by cameras installed at intersections and sighting information are collected, and police officers rely on these images and sighting information, etc. I knew the characteristics and the direction of escape. For this reason, it takes time for police officers to grasp the characteristics of the culprit and the direction of the escape, and there is a problem that the initial investigation may be delayed, which is inefficient.

  The present disclosure is devised in view of the conventional circumstances described above, and when an incident or the like occurs at a point where many people come and go, a captured image of a person visually similar to the criminal is extracted with high accuracy, and the criminal It is an object of the present invention to provide a person search system and a person search method that efficiently support early grasp of visual features and the like, and appropriately improve the convenience of police investigation.

  The present disclosure includes a server communicably connected to each of n (n: an integer of 3 or more) cameras, and a client terminal communicably connected to the server, wherein the server includes the n Receiving different captured images sent from each of the cameras, analyzes the position information and characteristic information of the person shown in each of the captured images and accumulates the information as an analysis result; and the client terminal includes the n cameras Of the k cameras (k: an integer of 2 or more that satisfies k ≦ n) selected by the user operation are displayed on the screen, and are displayed on any of the k cameras. According to the selection of the search target person, the search target person is searched for in the video images captured by (k-1) cameras other than the camera that captured the video image of the search target person. Request to the server, and in response to the search request, the server uses the analysis results and uses the analysis results to obtain the search target person and the (k) of the search target person in the video images captured by the (k-1) cameras. -1) The moving direction at the intersection where the cameras are arranged is extracted and sent to the client terminal. The client terminal moves the search target person at the intersection where the (k-1) cameras are arranged. Provided is a person search system that displays a direction in a manner superimposed on map data indicating an intersection where the k cameras are installed.

  Further, the present disclosure provides a person search method using a server communicably connected to each of n (n: an integer of 3 or more) cameras, and a client terminal communicably connected to the server. When the server receives different captured images sent from each of the n cameras, the server analyzes the position information and feature information of the person reflected in each of the captured images, accumulates the information as an analysis result, and The client terminal displays, on a screen, each of the captured images of k (k: an integer of 2 or more that satisfies k ≦ n) cameras selected by a user operation among the n cameras, and displays the k cameras According to the selection of the search target person appearing in any one of the captured images, the captured image captured by (k-1) cameras other than the camera that captured the image captured by the search target person The server sends a search request for the search target person to the server, and the server uses the analysis result in response to the search request and uses the analysis result to obtain the search target person and the image in (k-1) cameras. The direction of movement of the search target person at the intersection where the (k-1) cameras are arranged is extracted and sent to the client terminal, and the client terminal is connected to the intersection where the (k-1) cameras are arranged. In which the moving direction of the search target person is superimposed and displayed on map data indicating an intersection where the k cameras are installed.

  According to the present disclosure, when an incident or the like occurs at a point where many people come and go, a captured image of a person visually similar to the criminal can be extracted with high accuracy, and the visual characteristics and the escape direction of the criminal can be quickly grasped. Can be efficiently supported, and the convenience of police investigations can be accurately improved.

Block diagram showing a system configuration example of a person search system Block diagram showing an example of the internal configuration of the camera Block diagram showing an example of the internal configuration of each of the person search server and the client terminal Block diagram showing an example of the internal configuration of a video recorder The figure which shows an example of the person search screen which displays the map data containing the intersection where the camera was installed The figure which shows an example of the person search screen which displays the picked-up image (live image) of the selected camera Diagram showing an example of a setting screen of feature information of a search target person to be searched The figure which shows an example of the person search screen which displays the person search result by the metasearch using the set characteristic information as a search condition in time series The figure which shows the example of a display of the reproduction screen of the picked-up image | video corresponding to the selected cut-out image. 4 is a flowchart illustrating an example of an operation procedure of the person search system according to the first embodiment. A diagram showing an example of setting a selected cutout image to a search condition of an image search. The figure which shows the example of a setting of the search condition of the image search using the person frame displayed on the reproduction screen of the captured image of FIG. FIG. 11 is a diagram illustrating an example of a person search screen that displays a person search result by image search using the selected cutout image of FIG. 11 or the person frame of FIG. 12 in chronological order; Diagram showing an example of deleting unnecessary person search results by a predetermined user operation 9 is a flowchart for explaining an example of an operation procedure of the person search system according to the second embodiment. The figure which shows an example of the person search screen which displays the person search result at the time series when the person frame of another person displayed on the reproduction screen of the captured image of FIG. 9 is used as a search condition of image search. Diagram showing an example of deleting unnecessary person search results by user operation Flowchart for explaining an example of an operation procedure of the person search system according to the third embodiment

(Process leading to the configuration of Embodiment 1)
In the above-mentioned Patent Literature 1, images captured by a plurality of cameras are targeted based on characteristic information relating to the appearance of a person (that is, a criminal) who caused an incident or the like at a place where many people come and go (for example, an intersection). , And visually presenting a cutout image of a person who satisfies the feature information is not considered.

  In the event of an incident, it is important in the initial police investigation to grasp the characteristics of the criminal's appearance and whereabouts at an early stage. However, in the conventional technology up to now, clues such as images taken by cameras installed at intersections and sighting information are collected, and police officers rely on these images and sighting information, etc. I knew the appearance characteristics and the direction of the escape. For this reason, it takes time for police officers to grasp the appearance characteristics and escape direction of the criminal, and there has been a problem that the initial investigation may be delayed and it is not efficient.

  In the following first embodiment, when an incident or the like occurs at a point where many people come and go, an image of a visually similar person is captured from the captured images of a plurality of cameras based on characteristic information on the appearance of the criminal's clothes and the like. An example of a person search system and a person search method that extracts an image, efficiently supports the visual characteristics of the criminal and the early direction of the escape, and appropriately improves the convenience of the police investigation will be described.

(Embodiment 1)
Hereinafter, an embodiment that specifically discloses a person search system and a person search method according to the present disclosure will be described in detail with reference to the accompanying drawings as appropriate. However, an unnecessary detailed description may be omitted. For example, a detailed description of a well-known item or a redundant description of substantially the same configuration may be omitted. This is to prevent the following description from being unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter.

  Hereinafter, an example will be described in which a person search system supports a police search for searching for a person such as a criminal who caused an incident or the like (for example, an incident or an accident) at a point where many people come and go (for example, at or near an intersection).

  FIG. 1 is a block diagram illustrating a system configuration example of the person search system 100. The person search system 100 includes n (n: an integer of 3 or more) cameras 10, 10a,..., A person search server 50, a video recorder 70, and a client terminal 90. Each of the cameras 10, 10a,... Is installed outdoors (for example, at a point such as an intersection where many people come and go) as shown in FIG. However, each of the cameras 10, 10a,... Is not limited to being installed outdoors. The person search server 50, the video recorder 70, and the client terminal 90 are installed, for example, in a predetermined room in a police station. However, the person search server 50, the video recorder 70, and the client terminal 90 are not limited to being installed in a police station. In the following description, the video recorder 70 may be provided as online storage connected to the person search server 50 via a communication line such as the Internet, instead of on-premises management in a police station.

  In the person search system 100, one camera (for example, camera 10) is installed at one intersection. Note that a plurality of cameras (for example, the camera 10 or an existing camera having an internal configuration different from that of the camera 10) may be installed at one intersection. Therefore, the camera 10 is installed at a certain intersection, and the camera 10a is installed at another intersection. It is assumed that the cameras 10, 10a,... Have the same internal configuration. The cameras 10, 10a,... Are communicably connected to the person search server 50 and the video recorder 70 via a network NW1 such as an intranet or Internet communication line. The network NW1 is configured by a wired communication line (for example, an optical communication network using an optical fiber), but may be configured by a wireless communication network.

  Each of the cameras 10, 10a,... Is a surveillance camera capable of capturing an object (for example, an image indicating the state of an intersection) at an angle of view set when installed at an intersection, and includes a person search server 50 and an image recorder. The data of the picked-up video is repeatedly sent to each of 70. The data of the captured video is not limited to the data of the captured video, but includes identification information of the camera that captured the captured video (in other words, position information of the intersection where the corresponding camera is installed) and information of the capturing date and time. It is.

  The person search server 50 (an embodiment of a server) is installed, for example, in a police station, and is sent from each of the cameras 10, 10a,... Installed in all or some of the intersections in the jurisdiction of the police station. The data of the captured video is received and temporarily stored in the memory 52 or the storage unit 56 (see FIG. 3) for various processes by the processor PRC1 (see FIG. 3).

  Each time the stored captured image data is sent from each of the cameras 10, 10 a,... And received by the person search server 50, the person search server 50 causes the person position information and feature information of the person to appear in the captured image. Is analyzed. The analysis result is stored in the storage unit 56 of the person search server 50 so as to be used for searching for the culprit who caused the case or the like. Specifically, the person search server 50 obtains, as the analysis result described above, tag information relating to the content of the captured video (for example, characteristic information such as the face, gender, age, height, body type, and clothes of a person reflected in the captured video). May be acquired, the tag information may be added in association with the data of the captured video, and stored in the storage unit 56. For example, when an event such as an incident occurs, the client terminal 90 receives a user operation of inputting characteristic information (see FIG. 7) relating to the visual appearance of the culprit (one mode of the search target person), and A search request for a criminal person image satisfying the characteristic information is generated and sent to the person search server 50. In response to the search request, the person search server 50 searches for a person who satisfies the characteristic information included in the search request, using the analysis result of the captured image of each camera described above.

  The person search server 50 retrieves a part of the captured video (for example, a captured video selected by an operation of a terminal (not shown) used by an administrator in a police station (for example, a captured video of an important incident or a serious incident)). It may be sent to the video recorder 70 and accumulated.

  The client terminal 90 is installed, for example, in a police station and is used by a staff member in the police station (that is, a police officer who is a user in the police station), and is, for example, a laptop or notebook PC (Personal Computer). is there. For example, when a case or the like occurs, the user operates the client terminal 90 by using various information related to the case or the like as sighting information by telephone from a reporter who notified the occurrence of the case or the like to the police station. Enter and record. The client terminal 90 is not limited to a PC of the type described above, and may be a computer having a communication function such as a smartphone, a tablet terminal, or a PDA (Personal Digital Assistant). The client terminal 90 sends a search request to the person search server 50 to cause the person search server 50 to search for a search target person (that is, a culprit who caused an incident or the like) that matches the above-mentioned sighting information, and sends the search result to the person search server 50. Is received and displayed on the display 94.

  The video recorder 70 is installed, for example, in a police station, and receives data of the captured video transmitted from each of the cameras 10, 10a,... Installed in all or some of the intersections in the jurisdiction of the police station. , Save for backup etc. The video recorder 70 may transmit the stored captured image data of the camera to the client terminal 90 in response to a request from the client terminal 90 in response to a user operation. The person search server 50, the video recorder 70, and the client terminal 90 are communicably connected to each other via a network NW2 such as an intranet in a police station or the Internet.

  Although FIG. 1 shows only one person search server 50, one video recorder 70, and one client terminal 90 installed in the police station, a plurality of each may be provided. In the person search system 100, a police station may include a plurality of police stations. In the person search system 100 shown in FIG. 1, each of the configuration of the person search server 50 and the client terminal 90 may be configured by the same computer device (for example, a desktop or laptop personal computer). That is, instead of the person search server 50 and the client terminal 90, a computer device having the functions of the person search server 50 and the client terminal 90 may be provided.

  FIG. 2 is a block diagram showing an example of the internal configuration of the cameras 10, 10a,. As described above, each of the cameras 10, 10a,... Has the same configuration, and the camera 10 will be described below as an example. As shown in FIG. 2, the camera 10 has a configuration including an imaging unit 11, a processor 12, a memory 13, a communication unit 14, and a recording unit 15. The camera 10 may have a plurality of photographing units, and may be, for example, a multi-sensor camera having angles of view in two directions. This is because the first imaging unit (for example, the imaging unit 11) captures an image of the entire intersection in a wide area so that the entire intersection can be captured, and the second imaging unit (for example, the camera 10) has a blind angle of view of the first imaging unit. (The area where the pedestrian on the lower side in the vertical direction walks) from the installation location of (1). Even when a plurality of photographing units are provided, the internal configuration of each photographing unit is the same.

  The imaging unit 11 is configured to include a condensing lens and a solid-state imaging device such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. While the power of the camera 10 is on, the imaging unit 11 constantly outputs to the processor 12 the data of the captured video of the subject obtained based on the imaging by the solid-state imaging device. Further, the photographing unit 11 may include a mechanism for changing the zoom magnification at the time of imaging.

  The processor 12 is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array). The processor 12 functions as a control unit of the camera 10, performs control processing for totally controlling the operation of each unit of the camera 10, input / output processing of data with each unit of the camera 10, and calculation (calculation) of data Performs processing and data storage processing. Processor 12 operates according to programs and data stored in memory 13. The processor 12 uses the memory 13 at the time of operation, acquires current time information, performs various known image processing on the data of the captured video imaged by the imaging unit 11, and then stores the information in the recording unit 15. Or to record. Although not shown in FIG. 2, when the camera 10 has a GPS (Global Positioning System) receiving unit, the current position information is acquired from the GPS receiving unit, and the position information is further added to the data of the captured video. They may be recorded in association with each other.

  Here, the GPS receiver will be briefly described. The GPS receiving unit receives satellite signals including their signal transmission times and position coordinates transmitted from a plurality of GPS transmitters (for example, four navigation satellites). The GPS receiver calculates the current position coordinates of the camera and the reception time of the satellite signal using the plurality of satellite signals. This calculation may be performed by the processor 12 to which the output from the GPS receiving unit is input, instead of the GPS receiving unit. Note that the information on the reception time may be used for correcting the system time of the camera. The system time is used, for example, for recording an image capturing time of a captured image forming a captured image.

  Further, the processor 12 may variably control a photographing condition (for example, a zoom magnification) of the photographing unit 11 according to an external control command received by the communication unit 14. For example, when an external control command instructs to change the zoom magnification, the processor 12 changes the zoom magnification at the time of imaging of the imaging unit selected by the control command according to the control command.

  Further, the processor 12 repeatedly sends the data of the captured video recorded in the recording unit 15 to each of the person search server 50 and the video recorder 70 via the communication unit 14. Here, the term “repeated transmission” is not limited to transmission every time a predetermined period of time elapses, but also includes transmission every time a predetermined irregular time interval elapses instead of a predetermined period. And may include transmitting multiple times.

  The memory 13 is configured using, for example, a random access memory (RAM) and a read only memory (ROM), and programs and data necessary for executing the operation of the camera 10, and information or data generated during the operation. To save temporarily. The RAM is a work memory used when the processor 12 operates, for example. The ROM stores, for example, a program and data for controlling the processor 12 in advance. The memory 13 stores, for example, identification information (for example, a serial number) for identifying the camera 10 and various setting information.

  The communication unit 14 sends the captured video data recorded in the recording unit 15 to the person search server 50 and the video recorder 70 via the network NW1 based on the instruction of the processor 12. In addition, the communication unit 14 receives a control command of the camera 10 transmitted from the outside (for example, the person search server 50) and transmits state information of the camera 10 to the outside (for example, the person search server 50). .

  The recording unit 15 is configured using an external storage medium such as a semiconductor memory (for example, a flash memory) built in the camera 10 or a memory card (for example, an SD card) not built in the camera 10. The recording unit 15 records the data of the captured video generated by the processor 12 in association with the above-described identification information of the camera 10 and the information of the imaging date and time. The recording unit 15 always pre-buffers and holds captured video data for a predetermined time (for example, 30 seconds), and stores the data while overwriting data of the captured video for a predetermined time (for example, 30 seconds) before the current time. Keep doing. When the recording unit 15 is configured by a memory card, the recording unit 15 is detachably attached to the housing of the camera 10.

  FIG. 3 is a block diagram showing an example of the internal configuration of each of the person search server 50 and the client terminal 90. The person search server 50, the client terminal 90, and the video recorder 70 are all connected using an intranet such as a wired LAN (Local Area Network) provided in a police station, but are connected via a wireless network such as a wireless LAN. It may be connected.

  The person search server 50 includes a communication unit 51, a memory 52, a person search unit 53, a person analysis unit 54, a tag assignment unit 55, and a storage unit 56. The person search unit 53, the person analysis unit 54, and the tag assignment unit 55 are configured by a processor PRC1 such as a CPU, a DSP, and an FPGA.

  The communication unit 51 communicates with the cameras 10, 10a,... Connected via a network NW1 such as an intranet, and captures images (that is, people who cross the intersection) transmitted from the cameras 10, 10a,. (A video showing the situation). The communication unit 51 communicates with the client terminal 90 via a network NW2 such as an intranet provided in a police station, and receives a search request for a search target person sent from the client terminal 90. Or send a response to the search request. In addition, the communication unit 51 sends the captured video data stored in the memory 52 or the storage unit 56 to the video recorder 70.

  The memory 52 is configured using, for example, a RAM and a ROM, and temporarily stores programs and data necessary for executing the operation of the person search server 50, and information or data generated during the operation. The RAM is a work memory used when the processor PRC1 operates, for example. The ROM previously stores, for example, a program and data for controlling the processor PRC1. Further, the memory 52 stores, for example, identification information (for example, a serial number) for identifying the person search server 50 and various setting information.

  Based on the search request for the search target person sent from the client terminal 90, the person search unit 53 selects, from data recorded in the storage unit 56, characteristic information included in the search request (one mode of search conditions). Is searched for a captured image in which a person who satisfies is satisfied. The person search unit 53 extracts the search result, and sends the data of the extracted search result to the client terminal 90 via the communication unit 51.

  Each time the data of the captured video from each of the cameras 10, 10a,... Is stored in the storage unit 56, the person analysis unit 54 analyzes the stored data of the captured video. The person analyzing unit 54 analyzes and acquires information on a person (in other words, people existing at or near the intersection where the camera is installed) shown in the captured video. The person analyzing unit 54 generates a cut-out image in which the face of the person is reflected as an example of information regarding the person, and further, for example, characteristic information such as gender, age, height, body type, clothes, and position information indicating the location of the person, The moving direction (specifically, the inflow direction to the intersection and the outflow direction from the intersection) of the person when passing through the intersection is acquired as information on the person and sent to the tag attaching unit 55. The person analyzing unit 54 can determine the traveling direction of the person when passing through the intersection, for example, based on a temporal difference between frames of a plurality of captured images. The moving direction indicates, for example, in which direction the person has moved and passed the pedestrian crossing provided at the intersection.

  The tag assigning unit 55 associates the information on the person obtained by the person analyzing unit 54 with the shooting date and time and the point (that is, the position of the intersection) of the captured video used for analysis by the person analyzing unit 54 (one of the tagging operations). Aspect) and records it in the analysis information DB (Database) 56a of the storage unit 56. Accordingly, the person search server 50 can clearly determine what kind of person information is added to the captured video captured at which intersection and at what time. Note that the processing of the tag attaching unit 55 may be executed by the person analyzing unit 54, and in this case, the configuration of the tag attaching unit 55 can be made unnecessary.

  The storage unit 56 is configured using, for example, a hard disk (HDD: Hard Disk Drive) or a solid state drive (SSD: Solid State Drive). The storage unit 56 stores the data of the captured image transmitted from the cameras 10, 10a,..., The identification information of the camera that captured the captured image (in other words, the position information of the intersection where the corresponding camera is installed) and the imaging date and time. Is recorded in association with the information. The storage unit 56 also records road map information indicating the positions of intersections where the cameras 10, 10a,... Are installed. For example, information on the road map is updated by new road construction or maintenance work. Each time it is performed, the updated road map information is recorded. In addition, the storage unit 56 records intersection camera installation data indicating the correspondence between one camera installed at each intersection and the intersection. In the intersection camera installation data, for example, intersection identification information and camera identification information are associated with each other. Therefore, the storage unit 56 records the data of the image captured by the camera in association with the information of the image capturing date and time, the camera information, and the intersection information. The information of the road map is recorded in the memory 95 of the client terminal 90.

  The storage unit 56 has an analysis information DB 56a and a case DB 56b.

  The analysis information DB 56a stores the output of the tag assigning unit 55 (that is, the person information obtained as a result of the analysis of the image captured by the camera by the person analyzing unit 54, and the date and time and point information of the image captured used in the analysis). Set). The analysis information DB 56a is referred to, for example, when the person search unit 53 extracts person information that satisfies the characteristic information included in the search target person search request.

  The case DB 56b stores, for each case such as a case, the date and time when the case occurred, sighting information such as a point, and the result of the search performed by the person search unit 53 based on the sighting information and the search request of the search target person from the client terminal 90. Case detailed information such as data (for example, map data MP1, search conditions, and cutout image data as search results) is registered and stored. The case detailed information includes, for example, case information such as the date and time and location of the case, a thumbnail image of the searched person (the above-described cut-out image), peripheral map information including the position where the case occurred, and inflow and outflow of intersections of people. Includes direction, person's intersection passing time, and user's memo. Note that the case detailed information is not limited to the contents described above.

  The client terminal 90 is configured to include an operation unit 91, a processor 92, a communication unit 93, a display 94, a memory 95, and a recording unit 96. The client terminal 90 is used by a staff member in a police station (that is, a police officer who is a user). When there is a phone call for notifying the occurrence of the case or the like by a witness or the like of the case or the like, the user wears the headset HDS and answers the call. The headset HDS is used by being connected to the client terminal 90, and picks up a voice uttered by the user and outputs a voice uttered by the other party (that is, a reporter).

  The operation unit 91 is a user interface (UI: User Interface) that detects a user operation, and is configured using a mouse, a keyboard, or the like. The operation unit 91 outputs a signal based on a user operation to the processor 92. For example, when the user wants to confirm a captured image of an intersection of a point and an occurrence date and time of a case such as a case to be examined by a user, the operation unit 91 receives an input of a search condition including the feature of the date and time, the point, and a person.

  The processor 92 is configured using, for example, a CPU, a DSP, or an FPGA, functions as a control unit of the client terminal 90, and performs control processing for totally controlling the operation of each unit of the client terminal 90. During this period, data input / output processing, data arithmetic processing, and data storage processing are performed. Processor 92 operates according to programs and data stored in memory 95. The processor 92 uses the memory 95 during operation to acquire the current time information, or to display the search result of the search target person sent from the person search server 50 or the captured image sent from the image recorder 70 to the display 94. Display. Further, the processor 92 generates a search request for a search target person including the search condition input by the operation unit 91, and transmits the search request to the person search server 50 via the communication unit 93.

  The communication unit 93 performs communication with the person search server 50 or the video recorder 70 connected via a network NW2 such as an intranet. For example, the communication unit 93 transmits a search request generated by the processor 92 to the person search server 50 or receives a search result of a search target person sent from the person search server 50. Further, the communication unit 93 transmits an acquisition request of the captured video generated by the processor 92 to the video recorder 70, and receives the captured video transmitted from the video recorder 70 corresponding to the acquisition request.

  The display 94 is configured using a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and displays various data sent from the processor 92.

  The memory 95 is configured using, for example, a RAM and a ROM, and temporarily stores programs and data necessary for executing the operation of the client terminal 90, as well as information or data generated during the operation. The RAM is a work memory used when the processor 92 operates, for example. The ROM stores, for example, a program and data for controlling the processor 92 in advance. Further, the memory 95 stores, for example, identification information (for example, a serial number) for identifying the client terminal 90 and various setting information.

  The recording unit 96 is configured using, for example, a hard disk drive or a solid state drive. The recording unit 96 also records road map information indicating the positions of intersections where the cameras 10, 10a,... Are installed. For example, information on the road map is updated by new road construction or maintenance work. Each time, the updated road map information is recorded. The recording unit 96 records intersection camera installation data indicating the correspondence between one camera installed at each intersection and the intersection. In the intersection camera installation data, for example, intersection identification information and camera identification information are associated with each other. Therefore, the recording unit 96 records the data of the image captured by the camera in association with the information of the image capturing date and time, the camera information, and the intersection information.

  FIG. 4 is a block diagram showing an example of the internal configuration of the video recorder 70. The video recorder 70 is communicably connected to each of the cameras 10, 10a,... Via a network NW1 such as an intranet, and communicates with the person search server 50 and the client terminal 90 via a network NW2 such as an intranet. Are communicably connected between

  The video recorder 70 is configured to include a communication unit 71, a memory 72, a video search unit 73, a video recording processing unit 74, and a video storage unit 75. The video search unit 73 and the video recording processing unit 74 are configured by a processor PRC2 such as a CPU, a DSP, and an FPGA.

  The communication unit 71 communicates with each of the cameras 10, 10a,... Connected via a network NW1 such as an intranet, and captures images (that is, intersections) transmitted from each of the cameras 10, 10a,. Video) showing the situation of people passing by. The communication unit 71 communicates with the client terminal 90 via a network NW2 such as an intranet provided in a police station, receives a video request sent from the client terminal 90, and Or send a response to a request.

  The memory 72 includes, for example, a RAM and a ROM, and temporarily stores programs and data necessary for executing the operation of the video recorder 70, and information or data generated during the operation. The RAM is a work memory used when the processor PRC2 operates, for example. The ROM previously stores, for example, a program and data for controlling the processor PRC2. The memory 72 stores, for example, identification information (for example, a serial number) for identifying the video recorder 70 and various setting information.

  Based on the video request sent from the client terminal 90, the video search unit 73 extracts a captured image of a camera that matches the video request by searching the video storage unit 75. The video search unit 73 sends the data of the extracted captured video to the client terminal 90 via the communication unit 71.

  The video recording processing unit 74 records the received captured video data in the video storage unit 75 each time the captured video data from each of the cameras 10, 10a,.

  The video storage unit 75 is configured using, for example, a hard disk or a solid state drive. The video storage unit 75 stores the data of the captured video transmitted from each of the cameras 10, 10a,... As identification information of the camera that captured the captured video (in other words, the position information of the intersection where the corresponding camera is installed). The information is recorded in association with the information on the imaging date and time.

  Next, various screen examples displayed on the display 94 of the client terminal 90 during a police investigation will be described with reference to FIGS. In the description of FIGS. 5 to 9, the same elements as those shown in the drawings are referred to by the same reference numerals, and description thereof will be simplified or omitted.

  In the police investigation, the client terminal 90 activates and executes a previously installed person search application (hereinafter, “person search application”) by an operation of a police officer (one mode of a user). The program and data of the person search application are stored in the ROM of the memory 95 of the client terminal 90. The person search application is activated and executed by the processor 92 by an operation of a police officer. Note that various data or information created by the processor 92 while the person search application is running is temporarily stored in the RAM of the memory 95.

  FIG. 5 is a diagram illustrating an example of a person search screen WD1 that displays map data MP1 including an intersection where a camera is installed. The processor 92 displays the person search screen WD1 (one mode of the screen) on the display 94 after the activation of the person search application. The person search screen WD1 has a configuration in which both the map data MP1 corresponding to the road map information recorded in the recording unit 96 of the client terminal 90 and a plurality of search condition input fields are displayed side by side. In the following description, the person search application executed by the processor 92 communicates with the person search server 50 or the video recorder 70 during the execution.

  On the map data MP1, for example, an icon ITM1 indicating the position of the first camera arranged at the intersection of "AA Street" and "CC Street" and the intersection of "BB Street" and "CC Street" are arranged. The icon ITM2 indicating the position of the second camera, the icon ITM3 indicating the position of the third camera arranged at the intersection of "AA Street" and "DD Street", and the "BB Street" and "DD Street". An icon ITM4 indicating the position of the fourth camera arranged at the intersection is shown to be identifiable. Therefore, in the following description, the parameter n (n: an integer of 3 or more) indicating the number of installed cameras is set to “4”.

  In the following description, for convenience, the place name of the intersection of “AA Street” and “CC Street” is “Point 1”, and the place name of the intersection of “BB Street” and “CC Street” is “Point 2”. , And the place name of the intersection of “AA Street” and “DD Street” is “Point 3”, and the place name of the intersection of “BB Street” and “DD Street” is “Point 4”. The first camera, the second camera, the third camera, and the fourth camera have the same configuration as the cameras 10, 10a,... Shown in FIG.

  Further, the person search application displays a camera selection frame ASWK1 on the map data MP1 by a user operation. The camera selection frame ASWK1 is used to select a captured image (for example, a live image) of a camera displayed on the person search screen WD1. For example, in FIG. 5, the first camera and the second camera are selected in the camera selection frame ASWK1 by a user operation. In response to the selection in the camera selection frame ASWK1, the person search application sets the point (that is, “Point 1”, “Point 1”, “Point 1”) where the camera (specifically, the first camera and the second camera) selected by the user operation is installed. A point selection frame SEL0 indicating that “Point 2”) is a display target of a captured image on the person search screen WD1 is displayed on the person search screen WD1. The camera selection frame ASWK1 shown in FIG. 5 is rectangular, but is not limited to this shape, and may be any of various well-known shapes such as a circle, an ellipse, and a triangle.

  FIG. 6 is a diagram illustrating an example of a person search screen WD1 that displays a captured video (live video) of a selected camera. As shown in FIG. 6, k (k: an integer of 2 or more that satisfies k ≦ n) cameras (specifically, k = 3, corresponding to k = 1) in the map data MP1 by a user operation. The first camera, the second camera corresponding to k = 2, and the third camera corresponding to k = 3) have already been selected in the camera selection frame ASWK1. The person search application changes the display mode of the icons ITM1, ITM2, and ITM3 so as to be different from the other icons ITM4 in order to indicate that the icons have been selected by the camera selection frame ASWK1. Specifically, the person search application changes and displays the icons ITM, ITM2, and ITM3 to the selected icons SEL1, SEL2, and SEL3, which are repainted with a specific color (for example, red). In this way, the person search application sets the above-mentioned point selection frame SEL0 to three points of “Point1,” “Point2,” and “Point3” corresponding to the first camera, the second camera, and the third camera, respectively. A point selection frame SEL0 indicating that the captured image is to be displayed on WD1 is displayed on the person search screen WD1.

  The person search application displays the display screen of the captured video MV1 (live video) of the first camera (that is, the location of “Point 1”) corresponding to the location selection frame SEL0 or the selected icon SEL1 together with the capturing time and the capturing location name. It is displayed on the person search screen WD1. Similarly, the person search application changes the display screen of the captured video MV2 (live video) of the second camera (that is, the location of “Point 2”) corresponding to the location selection frame SEL0 or the selected icon SEL2 to the capture time and the capture time. It is displayed on the person search screen WD1 together with the point name. Similarly, the person search application changes the display screen of the captured video MV3 (live video) of the third camera (that is, the location of “Point 3”) corresponding to the location selection frame SEL0 or the selected icon SEL3 to the capture time and the capture time. It is displayed on the person search screen WD1 together with the point name. The person search application displays a live marker MK1 indicating a live image on each of the captured images MV1, MV2, and MV3.

  When the person search server 50 cannot receive the captured video (live video) from at least one of the first camera, the second camera, and the third camera (for example, the first camera), the client terminal 90 uses the camera. The video recorder 70 may request the video recorder 70 for data of a captured video that has been captured in the past and stored in the video recorder 70 at “Point 1” where the first camera (for example, the first camera) is installed. In this case, the client terminal 90 may receive the data of the captured video in the past “Point 1” sent from the video recorder 70 in response to the request from the client terminal 90 and display the data on the person search screen WD1. Also, the client terminal 90 is used when the person search server 50 cannot receive a captured video (live video) from at least one of the first camera, the second camera, and the third camera (for example, the first camera). Data of the captured video input via an external input terminal 90 (not shown) or data of the captured video already stored in the recording unit 96 is obtained, and the captured video data is displayed on the person search screen WD1. May be.

  Further, the person search application displays various buttons on the person search screen WD1 to assist in searching for a search target person in a police search. For example, a period start button DB1 for selecting the start date and time of the search target period and a period end button DE1 for selecting the end date and time of the search target period are arranged. In addition, a search target person image display frame PL1 indicating the appearance image or atmosphere of the search target person to be searched is arranged.

  Further, the person search application can execute a search request of two types of search methods to the person search server 50 in searching for a search target person. The first search method is a meta search. For example, based on meta information set on a setting screen DTL1 shown in FIG. 7, feature points in which the similarity of the feature points in the meta information is the same or within a predetermined value are searched. This is a method of searching for a search target person having the search target. Embodiment 1 describes this meta search. The person search application displays a meta search button MS1 for requesting the meta search to the person search server 50 on the person search screen WD1.

  The second search method is an image search. For example, based on image data selected in the cut-out image Xpc1 shown in FIG. 11 or the person frame WK1 shown in FIG. This is a method of searching for a search target person having a feature point within a predetermined value. Embodiment 2 describes this image search. The person search application displays an image search button IS1 for requesting the image search to the person search server 50 on the person search screen WD1.

  In addition, the person search application displays a report button RP1 on the person search screen WD1 for saving the search result data of the search target person on the person search screen WD1 as a case report. When the report button RP1 is selected by a user operation, the person search application displays the search result data of the search target person on the person search screen WD1 (for example, map data MP1, search conditions, and cutout image data as a search result). It is stored in the recording unit 96 as a case report. Further, the client terminal 90 may send the data of the case report to the person search server 50 in association with the identification information (for example, the file name) of the case report. The person search server 50 stores the data of the case report associated with the identification information (for example, the file name) of the case report in the case DB 56b.

  Further, the person search application displays a reset button RS1 on the person search screen WD1 for returning the displayed person search screen WD1 to the initial state (in other words, before starting the search for the search target person) to the person search screen WD1. This allows the user to easily restart the search for the search target person from the beginning, for example, when the user performs an incorrect operation when searching for the search target person on the person search screen WD1.

  FIG. 7 is a diagram illustrating an example of a setting screen DTL1 of feature information of a search target person to be searched. The person search application can mainly set a date (Time & Date), a characteristic (Characteristics), and a dress (Decoration) as three main elements of search conditions for searching for a search target person by a user operation. The setting screen DTL1 is superimposed and displayed on the person search screen WD1. In addition, the person search application displays a search target person image IPS1 specified by setting of characteristics (Characteristics) and clothes (Decoration) by a user operation on the setting screen DTL1. Thereby, the user can visually confirm the search target person image IPS1 corresponding to the result of the characteristics (Characteristics) and the clothes (Decoration) set by the user, and can easily determine whether or not the user's setting is appropriate.

  For the date (Time & Date), a calendar icon for enabling setting of the start date and time and the end date and time of the search period of the search target person is displayed. For example, when a calendar icon of a date (Time & Date) is selected by a user operation, the person search application superimposes and displays a calendar box CAL1 on the map data MP1. Thereby, the start date and time and the end date and time of the search period of the search target person can be selected by the user operation. The start date and time and the end date and time of the search period of the search target person may be directly input by a user operation without being limited to the selection of the calendar icon. For example, in FIG. 7, “12:00 pm on August 24, 2018” to “1:00 pm on August 24, 2018” are selected as the date (Time & Date).

  Further, in the feature (Characteristics), information options regarding a part of the appearance of the search target person are displayed so as to be selectable. The displayed options include a gender option (specifically, “Male” or “Female”), an age option (specifically, “Younger”, “Middle”, or “Older”), and a height option ( Specifically, there are “Small”, “Normal” or “Large”), and options related to the body type (specifically, “Slim”, “Normal” or “Thick”). For example, in FIG. 7, "Male", "Older", and "Slim" are selected as the characteristics (Characteristics) in FIG. .

Further, in the clothing (Decoration), information options regarding a part of the appearance of the search target person are displayed so as to be selectable. The displayed options include an option for belongings (specifically, “with bag” or “no bag”), an option for the attached article (specifically, “with hat” or “no hat”), and an upper body. Clothing options (specifically "Long sleeved" or "Short sleeved") and lower body clothing options (specifically "Long Pants", "Short")
“Pants” or “Skirt”) and options related to the hairstyle (specifically, “Long Hair”, “Short Hair”, or “No Hair”). In addition, colors can be selected for clothes, personal belongings, and attachments. For example, red (Red), blue (Blue), green (Green), yellow (Yellow), orange (Orange), brown (Brown), purple (Purple), gray (Gray), black (Black), white (White) ) Can be selected. As a result, the user can finely select the appearance characteristics related to the clothes and the like of the search target person. For example, in FIG. 7, “with bag”, “with hat”, “long sleeved”, and “long pants” are selected as the clothes (Decoration). For the upper body clothes, “Blue” is selected, and for the lower body clothes, “Gray” is selected.

  FIG. 8 is a diagram illustrating an example of a person search screen WD1 that displays, in chronological order, person search results obtained by metasearch using the set feature information as search conditions. The person search application displays the search target person image IPS1 and some search conditions CON1 selected on the setting screen DTL1 shown in FIG. 7 in the search target person image display frame PL1.

  The person search application detects that the meta search button MS1 has been pressed by a user operation while the search target person image IPS1 is displayed in the search target person image display frame PL1. In this case, the person search application (specifically, the processor 92) performs various search conditions (for example, a date indicating a search period) for specifying the search target person image IPS1 displayed in the search target person image display frame PL1. A search request for the search target person including the search target person characteristic information and the identification information of the search target camera is generated and transmitted to the person search server 50 via the communication unit 93. When receiving the search request sent from the client terminal 90, the person search server 50 performs various search conditions included in the search request by meta search using the analysis results stored in the analysis information DB 56a of the storage unit 56. A person who is the same as or similar to the search target person that satisfies the condition (for example, a person having a similarity threshold equal to or greater than a predetermined similarity threshold) is searched. The person search server 50 sends the search result to the client terminal 90.

  Upon acquiring the search result received via the communication unit 93, the person search application (specifically, the processor 92) cuts out an image of the same or similar person as the search target person (that is, the search target person image IPS1). Xpc1, Xpc2, Xpc3, Xpc4, Xpc5, Xpc6, Xpc7, Xpc8, Xpc9, Xpc10, Xpc11, Xpc12, Xpc13, Xpc14, Xpc15,... Are arranged and displayed in the list display frame RST1. Each cut-out image is, for example, a thumbnail image. Note that, when not all cutout images can be displayed in the list display frame RST1, the person search application can scroll and display the remaining cutout images by a user operation of the scroll bar SCR1, and the same applies to the following embodiments. is there.

  For example, the person search application outputs the cut-out images Xpc1, Xpc2, Xpc3, Xpc4, Xpc5, Xpc6, Xpc7, Xpc8, Xpc9, Xpc10, Xpc11, and the same or similar to the search target person (that is, the search target person image IPS1). Xpc12, Xpc13, Xpc14, Xpc15,... May be displayed in the same or similar order as the search target person image IPS1. Thus, the user can find the search target person he or she wants to grasp by checking the displayed cut-out image preferentially without performing the scroll process using the scroll bar SCR1.

  For example, the person search application outputs the cut-out images Xpc1, Xpc2, Xpc3, Xpc4, Xpc5, Xpc6, Xpc7, Xpc8, Xpc9, Xpc10, Xpc11, and the same or similar to the search target person (that is, the search target person image IPS1). , Xpc12, Xpc13, Xpc14, Xpc15,... May be displayed in chronological order in the order of the imaging times at which the respective cut-out images were captured (for example, in the order of oldest or newest imaging times). This allows the user to find the search target person he or she wants to grasp early by preferentially checking the clipped images displayed in the order of the oldest or newest imaging time.

  For example, the person search application outputs the cut-out images Xpc1, Xpc2, Xpc3, Xpc4, Xpc5, Xpc6, Xpc7, Xpc8, Xpc9, Xpc10, Xpc11, and the same or similar to the search target person (that is, the search target person image IPS1). Xpc12, Xpc13, Xpc14, Xpc15,... May be displayed collectively for each intersection where a camera corresponding to each cut-out image is installed. This allows the user to preferentially check one or more cutout images collectively displayed corresponding to the relevant intersection, for example, when grasping an intersection where the search target person may be located. By doing so, it is possible to quickly find the search target person that he / she wants to grasp.

  FIG. 9 is a diagram illustrating a display example of a playback screen of the captured video MV1 corresponding to the selected cut-out image Xpc1. The person search application detects that one of the cutout images Xpc1 to Xpc15,... Displayed in the list display frame RST1 (for example, the cutout image Xpc1) is selected by a predetermined user operation. . In this case, the person search application displays a playback screen on which the captured video MV1 corresponding to the selected cutout image Xpc1 (that is, the video captured by the camera at the time when the person shown in the cutout image Xpc1 is captured) is displayed. The person search application has an icon for accepting various user operations such as “play”, “pause”, “fast forward”, “fast rewind”, “return to first”, and “next” on the replay screen. And display it.

  In addition, in association with the display of the playback screen of the captured video MV1, the person search application changes the display screen of the captured video MV2 or MV3 displayed before selecting any of the cutout images (for example, the cutout image Xpc1) to a map. It is rearranged and displayed around the corresponding intersection of the data MP1. Thus, the user can intuitively and visually understand the correlation between the display screens of the captured images MV2 and MV3 and the corresponding intersections in an intuitive and easy-to-understand manner, and can efficiently support the police investigation.

  Next, an operation procedure of the person search system 100 according to Embodiment 1 will be described with reference to FIG. FIG. 10 is a flowchart illustrating an example of an operation procedure of person search system 100 according to Embodiment 1. The process illustrated in FIG. 10 is executed by the client terminal 90 and the person search server 50.

  In FIG. 10, each time the person search server 50 receives the data of the captured video transmitted from each of the cameras 10, 10a,... Which configure the person search system 100, the personal search server 50 analyzes the data of the captured video. The person search server 50 generates information on the person by analysis (for example, a cut-out image of the person, the characteristics of the person, the position of the person, and the moving direction of the person when passing through the intersection) as an analysis result, and analyzes the accumulation unit 56. The information is stored in the information DB 56a (St1). The process of step St1 is performed before the person search server 50 acquires a search request from the client terminal 90.

  The client terminal 90 accepts, as a search condition of the search target person (for example, the search target person image IPS1), a selection (see FIG. 7) of various characteristic information for configuring the search target person image IPS1 by a user operation (see FIG. 7) (St2). . The client terminal 90 accepts, by a user operation, selection of a captured image of a camera to be searched for a search target person (St3). Note that the processing of steps St2 and St3 may be performed in any order. After detecting Steps St2 and St3, for example, when the meta search button MS1 is pressed by a user operation, the client terminal 90 specifies the search target person image IPS1 displayed in the search target person image display frame PL1. A search request for a search target person including various search conditions (for example, a date indicating a search period, characteristic information of a search target person) and identification information of a camera to be searched is generated and transmitted to the person search server 50.

  When receiving the search request sent from the client terminal 90, the person search server 50 analyzes the captured image corresponding to the captured image corresponding to the identification information of the camera to be searched on the date indicating the search period included in the search request ( (Step St1) is extracted from the analysis information DB 56a (St4). The person search server 50 reads out information on the previously selected similarity threshold from, for example, the memory 52 and sets the information (St5). After step St5, the person search server 50 compares the analysis result (information about the person) extracted in step St4 and the characteristic information of the search target person included in the search request sent from the client terminal 90 (see FIG. 7). The degree of similarity is calculated by comparison using a meta search (St6).

  The person search server 50 determines whether or not the similarity calculated in Step St6 is equal to or larger than a threshold (see Step St5) (St7). If the person search server 50 determines that the calculated similarity is equal to or larger than the threshold (St7, YES), the cutout image (for example, a thumbnail image) in which a person exceeding the threshold is shown as an example of the search result, and the cutout is performed. The image data and an instruction to display the cut-out image on the client terminal 90 are sent to the client terminal 90 (St8). The client terminal 90 displays the corresponding cut-out image in the list display frame RST1 of the person search screen WD1 based on the instruction sent from the person search server 50 (St8). As described above, when the client terminal 90 receives a plurality of cutout images to be displayed in the list display frame RST1, the client terminal 90 arranges the cutout images in the order of similarity, in chronological order, or in the order of intersection. May be displayed.

  On the other hand, when determining that the calculated similarity is smaller than the threshold (St7, NO), the person search server 50 omits the process of Step St8. After this or after step St8, the person search server 50 determines whether or not there is a person to be compared whether or not the similarity calculated in step St6 is equal to or greater than a threshold (St9). . If it is determined that there is no person to be compared whether the calculated similarity is equal to or greater than the threshold value in Step St6 (St9, NO), the process of the person search server 50 (that is, the client terminal 90) The processing corresponding to the search request from is terminated. The client terminal 90 displays the cutout images of a plurality of persons corresponding to the search result by the person search server 50 side by side (see FIG. 8 or FIG. 9).

  If the person search server 50 determines that there is a person to be compared whether or not the similarity calculated in step St6 is equal to or larger than the threshold (St9, YES), the person corresponding to the next person is determined as follows. It is determined that the processing of step St6 is to be executed (St10). Thereafter, the person search server 50 performs the process of step S6 for the next corresponding person. The person search server 50 repeatedly executes the processing from Step St6 to Step St10 until there is no more person to be compared whether or not the similarity calculated in Step St6 is equal to or larger than the threshold.

  As described above, the person search system 100 according to the first embodiment enables the person search server 50 communicably connected to each of n (n: an integer of 3 or more) cameras and the communicable with the person search server 50. And a connected client terminal 90. When receiving the different captured images sent from each of the n cameras, the person search server 50 analyzes the position information and the characteristic information of the person appearing in each of the captured images, and accumulates the analysis results. The client terminal 90 displays, on the person search screen WD1, captured images of k cameras (k: an integer of 2 or more that satisfies k ≦ n) selected by a user operation among the n cameras. The client terminal 90 sends a search request for a search target person that satisfies the characteristic information in the k captured images to the person search server 50 in accordance with the selection of the characteristic information related to the appearance of the search target person. In response to the search request, the person search server 50 extracts an analysis result regarding the search target person that satisfies the characteristic information in the k captured images using the analysis result and sends the result to the client terminal 90. The client terminal 90 displays side-by-side cut-out images of a plurality of persons that are the same as or similar to the search target person based on the analysis result regarding the search target person.

  Accordingly, when an incident or the like occurs at a point (for example, an intersection) where many people come and go, the person search system 100 visually recognizes the captured image of the plurality of cameras based on the characteristic information on the appearance of the criminal's clothes and the like. A captured image of a similar person can be extracted. Therefore, since the person search system 100 can efficiently support the visual characteristics of the criminal and the early grasp of the escape direction, it is possible to appropriately improve the convenience of the police investigation.

  Further, the client terminal 90 displays map data MP1 indicating a point where k cameras are installed on the person search screen WD1. Thereby, when the user requests the search of the search target person from the client terminal 90 to the person search server 50, the user may browse the map data MP1 to determine a geographical positional relationship while viewing the map data MP1. After grasping, it is possible to visually and easily select a camera to be searched.

  In addition, the client terminal 90 superimposes and displays the movement routes of a plurality of persons the same as or similar to the search target person on the map data MP1 based on the analysis result regarding the search target person (see FIG. 13). This allows the user to grasp all the escape route candidates of the culprit such as an incident in the map data MP1, so that it is easy to narrow down precisely.

  Further, the client terminal 90 updates and displays the moving route of the search target person displayed on the map data in accordance with a user operation for deleting at least one cutout image of the cutout images of the plurality of persons ( See FIG. 14). Thus, the user only needs to perform an operation of deleting a cutout image of an unnecessary person different from the search target person from among the plurality of cutout images displayed in the list display frame RST1, and the destination of the target search target person is deleted. Can be accurately narrowed down.

  In addition, the client terminal 90 accepts selection of appearance information such as the sex, age, height, body type, and clothes of the person as characteristic information on the appearance of the search target person. This allows the user to finely select the appearance characteristics such as the sex and size of the search target person and the appearance characteristics related to the clothes and the like of the search target person. Therefore, the client terminal 90 can display a cut-out image of the same or similar person as the search target person carefully selected by the person search server 50, and can appropriately improve the search efficiency of the user.

  In addition, the client terminal 90 displays cutout images of a plurality of persons that are the same as or similar to the search target person in chronological order along the order of the imaging times when the persons were imaged. This allows the user to find the search target person he or she wants to grasp early by preferentially checking the clipped images displayed in the order of the oldest or newest imaging time.

  In addition, the client terminal 90 displays cutout images of a plurality of persons that are the same as or similar to the search target person, arranged in descending order of similarity with the search target person. This allows the user to check the displayed cut-out image preferentially without performing the scroll processing using the scroll bar SCR1 shown in FIG. 8 or FIG. Can be found.

(Process leading to the configuration of Embodiment 2)
In Patent Document 1 described above, a camera that captures a person visually resembling a person (that is, a criminal) that caused an incident or an accident (hereinafter, “incident or the like”) that occurred at a point (for example, an intersection) where many people come and go is described. No consideration is given to specifying a captured image and retrieving a captured image similar to a clue from the captured video of another camera with the specified captured image.

  When an incident or the like occurs, it is important to grasp the visual characteristics and the whereabouts of the criminal at an early stage in an initial police investigation. However, in the conventional technology up to now, clues such as images taken by cameras installed at intersections and sighting information are collected, and police officers rely on these images and sighting information, etc. I knew the characteristics and the direction of escape. For this reason, it takes time for police officers to grasp the characteristics of the culprit and the direction of the escape, and there is a problem that the initial investigation may be delayed, which is inefficient.

  In the following embodiment 2, when an incident or the like occurs at a point where many people come and go, a captured image of a person visually similar to the criminal is extracted with high accuracy, and the visual characteristics of the criminal are grasped at an early stage. The following describes an example of a person search system and a person search method that efficiently supports the search and accurately improves the convenience of a police investigation.

(Embodiment 2)
The internal configuration of the person search system according to the second embodiment is the same as the internal configuration of person search system 100 according to the first embodiment. Therefore, in the description of the second embodiment, the same configuration as the internal configuration of the person search system 100 according to the first embodiment will be denoted by the same reference numeral, and the description will be simplified or omitted, and different contents will be described.

  FIG. 11 is a diagram illustrating an example of setting the selected cut-out image Xpc1 as a search condition in an image search. The person search application detects that one of the cut-out images (for example, cut-out image Xpc1) has been selected by a predetermined user operation from the plurality of cut-out images displayed in list display frame RSR1. In this case, the person search application displays the cut-out image Xpc1 by copying and pasting it into the search target person image display frame PL1. That is, the person search application displays the person appearing in the cut-out image Xpc1 in the search target person image display frame PL1 as the search target person image IPS2 of the image search. In addition, the person search application uses the analysis result of the person search server 50 corresponding to the person in the cut-out image Xpc1 and sets the main feature information of the person (search target person image IPS2) as a part of the search condition CON2. It is displayed in the person image display frame PL1.

  FIG. 12 is a diagram illustrating a setting example of search conditions for image search using the person frames WK1 and WK2 displayed on the playback screen of the captured video MV1 in FIG. As shown in FIG. 12, when displaying the playback screen of the captured video MV1 corresponding to the cut-out image Xpc1 selected by a predetermined user operation on the person search screen WD1, the person search application searches for the person corresponding to the cut-out image Xpc1. Using the analysis result by the server 50, the person frames WK1 and WK2 and the gender and age group of the person reflected on the reproduction screen may be displayed. For example, the person search application displays “M40” (in other words, a man in his 40s) as the gender and age group of the person corresponding to the person frame WK1. Similarly, the person search application displays “M30” (that is, a man who is in his thirties) as the gender and age group of the person corresponding to the person frame WK2.

  The person search application detects that one of the person frames WK1 and WK2 displayed on the playback screen of the captured video MV1 has been selected by a predetermined user operation (for example, the person frame WK1). In this case, the person search application cuts out the image in the selected person frame WK1, and copies and pastes the cut image into the search target person image display frame PL1. That is, the person search application displays the person appearing in the pasted cut-out image as the search target person image IPS2 of the image search in the search target person image display frame PL1. In addition, the person search application uses the analysis result of the person search server 50 corresponding to the person in the cut-out image that has been pasted, and sets the main feature information of the person (the search target person image IPS2) as a part of the search condition CON2. It is displayed in the search target person image display frame PL1.

  FIG. 13 is a diagram illustrating an example of a person search screen WD1 that displays a person search result by image search using the selected cutout image Xpc1 of FIG. 11 or the person frame WK1 of FIG. 12 in a time series. The person search application displays the search target person image IPS2 in the search target person image display frame PL1 when the selected cut-out image Xpc1 in FIG. 11 or the person frame WK1 in FIG. 12 is selected by a predetermined user operation.

  Thereafter, the person search application detects that the image search button IS1 has been pressed by a user operation. In this case, the person search application (specifically, the processor 92) performs various search conditions (for example, a date indicating a search period) for specifying the image data of the search target person image IPS2 and the search target person image IPS2. A search request for a search target person including the search request and the identification information of the camera to be searched is generated, and transmitted to the person search server 50 via the communication unit 93. However, in the second embodiment, in generating the search request, the person search application uses a camera other than the camera corresponding to the captured video MV1 or the cut-out image Xpc1 in which the person serving as the source of the search target person image IPS2 is shown. Is selected as a search target to generate a search request. When receiving the search request sent from the client terminal 90, the person search server 50 uses the analysis results stored in the analysis information DB 56a of the storage unit 56 to perform an image search to perform various search conditions included in the search request. Search for a person who is the same or similar to the search target person who satisfies. The person search server 50 sends the search result to the client terminal 90.

  Upon acquiring the search result received via the communication unit 93, the person search application (specifically, the processor 92) cuts out an image of the same or similar person as the search target person (that is, the search target person image IPS2). Xpc1, Xpc21, Xpc22, Xpc5, Xpc23, Xpc24, and Xpc25 are arranged and displayed in the list display frame RST1. Each cut-out image is, for example, a thumbnail image.

  In addition, the person search application is used when a plurality of cut-out images Xpc1, Xpc21, Xpc22, Xpc5, Xpc23, Xpc24, and Xpc25 displayed in the list display frame RST1 are passed through an intersection corresponding to a camera at which a person is imaged. Each of the moving directions DRC1, DRC2, and DRC3 is displayed so as to be superimposed on the map data MP1. For example, the movement direction DRC1 is a direction along the “AA Street” from the intersection where the third camera is arranged toward the intersection where the first camera is arranged. The movement direction DRC2 is a direction along the “CC Street” that moves from the intersection where the first camera is arranged toward the intersection where the second camera is arranged. The movement direction DRC3 is a direction along the “BB Street” that moves from the intersection where the second camera is arranged toward the intersection where the fourth camera is arranged.

  In the first embodiment, the person search application maps each of the moving directions at the time of passing through the intersection corresponding to the camera in which the person shown in all the cut-out images displayed in the list display frame RST1 has been captured. All the data may be superimposed and displayed on the data MP1. This allows the user to grasp all the escape route candidates of the culprit such as an incident in the map data MP1, so that it is easy to narrow down precisely.

  FIG. 14 is a diagram illustrating an example of deleting an unnecessary person search result by a predetermined user operation. The person search application selects an unnecessary cutout image (for example, cutout image Xpc5 or Xpc5) from cutout images Xpc1, Xpc21, Xpc22, Xpc5, Xpc23, Xpc24, and Xpc25 displayed in list display frame RST1 shown in FIG. Xpc24) is accepted. In this case, the person search application superimposes and displays the deletion icons DLT1 and DLT2 indicating that the selected cutout images Xpc5 and Xpc24 are to be deleted on the cutout images Xpc5 and Xpc24. Note that the person search application may delete the selected cutout images Xpc5 and Xpc24 from the list display frame RST1 instead of displaying the delete icons DLT1 and DLT2 in a superimposed manner.

  In addition, in conjunction with displaying the delete icons DLT1 and DLT2 superimposed on the cutout images Xpc5 and Xpc24, the person search application uses the analysis result by the person search server 50 corresponding to the person shown in the cutout images Xpc5 and Xpc24. Then, the moving direction displayed on the map data MP1 is updated and displayed. More specifically, the person search application is configured to perform a search for all the moving directions DRC1, DRC2, and DRC3 displayed on the map data MP1 before displaying the delete icons DLT1 and DLT2, at the time of the intersection of a person reflected in the cut-out images Xpc5 and Xpc24. Is updated so as to delete the moving direction, and only the moving directions DRC1 and DRC2 at the time of passing through the intersection of the person shown in the cut-out images Xpc1, Xpc21, Xpc22, Xpc23, and Xpc25 are displayed.

  Next, an operation procedure of the person search system 100 according to Embodiment 2 will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of an operation procedure of person search system 100 according to Embodiment 2. The processing illustrated in FIG. 15 is executed by the client terminal 90 and the person search server 50. In the description of the processing illustrated in FIG. 15, the same processing as the processing illustrated in FIG. 10 will be assigned the same step numbers, and the description will be simplified or omitted, and different contents will be described.

  In FIG. 15, after step St1, the client terminal 90 sets a cut-out image Xpc1 (for example, a thumbnail) showing the person who is the source of the search target person image IPS2 as a search condition of the search target person (for example, search target person image IPS2). The selection by the user operation (see FIG. 11) of the image) is received (St11). The client terminal 90 selects the person frame WK1 in the captured video MV1 in which the person serving as the source of the search target person image IPS2 is selected by a user operation as a search condition of the search target person (for example, the search target person image IPS2). (See FIG. 11) may be accepted (St11). The client terminal 90 displays the cut-out image Xpc1 by copying and pasting it into the search target person image display frame PL1.

  When the client terminal 90 detects, for example, that the image search button IS1 has been pressed by a user operation after Step St11, the client terminal 90 detects the image data of the search target person image IPS2 displayed in the search target person image display frame PL1 and the search target thereof. A search request for a search target person including various search conditions (for example, a date indicating a search period) for identifying the person image IPS2 and identification information of a camera to be searched is generated. Send it to the search server 50. However, as described above, in the second embodiment, in the generation of the search request, the person search application sets the camera corresponding to the captured image MV1 or the cut-out image Xpc1 in which the person who is the source of the search target person image IPS2 is shown. A search request is generated by selecting the excluded cameras as search targets.

  Upon receiving the search request sent from the client terminal 90, the person search server 50 includes the identification information of the camera to be searched for on the date indicating the search period included in the search request, and captures an image corresponding to the identification information. Is transmitted to the video recorder 70. Based on the transmission request from the person search server 50, the video recorder 70 acquires captured image data corresponding to the camera identification information included in the transmission request, and transmits the data to the person search server 50. The person search server 50 acquires the data of the captured image of the camera transmitted from the video recorder 70 (St12). Note that the person search server 50 may store the data of the captured video transmitted from each camera in the storage unit 56. In this case, instead of receiving and acquiring the data from the video recorder 70, the storage unit 56 May be directly read out from and acquired.

  The person search server 50 analyzes each of the captured images of the camera acquired in Step St12, extracts and identifies the person appearing in the captured image, and temporarily stores the same in the memory 52 (St13). After step St5, the person search server 50 compares the analysis result of the person analyzed in step St13 and the characteristic information of the search target person included in the search request sent from the client terminal 90 (see FIG. 11 or FIG. 12). The degree of similarity is calculated by comparison through image search (St14). In addition, in step St13, the person search server 50 uses the analysis result of the captured image of each camera generated in step St1, and identifies the identification information of the camera to be searched on the date indicating the search period included in the search request. May be obtained by reading out an analysis result (for example, a cut-out image) of a captured video corresponding to. In this case, the person search server 50 determines in step St14 the analysis result (for example, a clipped image) of the read captured video and the characteristic information of the search target person included in the search request sent from the client terminal 90 (FIG. 12 (see FIG. 12) by image search to calculate the similarity (St14).

  The person search server 50 determines whether or not the similarity calculated in Step St14 is equal to or larger than a threshold (see Step St5) (St7). If the person search server 50 determines that the calculated similarity is equal to or larger than the threshold (St7, YES), the cutout image (for example, a thumbnail image) in which a person exceeding the threshold is shown as an example of the search result, and the cutout is performed. The image data and an instruction to display the cut-out image on the client terminal 90 are sent to the client terminal 90 (St8). The client terminal 90 displays the corresponding cut-out image in the list display frame RST1 of the person search screen WD1 based on the instruction sent from the person search server 50 (St8). As described above, when the client terminal 90 receives a plurality of cutout images to be displayed in the list display frame RST1, the client terminal 90 arranges the cutout images in the order of similarity, in chronological order, or in the order of intersection. May be displayed.

  On the other hand, when determining that the calculated similarity is smaller than the threshold (St7, NO), the person search server 50 omits the process of Step St8. After this or after step St8, the person search server 50 determines whether or not there is a person to be compared whether or not the similarity calculated in step St14 is equal to or greater than a threshold (St9). . If it is determined that there is no person to be compared whether or not the similarity calculated in step St14 is equal to or greater than the threshold (St9, NO), the processing of the person search server 50 (that is, the client terminal 90) The processing corresponding to the search request from is terminated. The client terminal 90 displays the cutout images of a plurality of persons corresponding to the search result by the person search server 50 side by side (see FIG. 11 or FIG. 12).

  If the person search server 50 determines that there is a person to be compared whether or not the similarity calculated in step St14 is equal to or more than the threshold (St9, YES), the person search server 50 It is determined that the processing of step St14 is to be executed (St15). Thereafter, the person search server 50 performs the process of step S14 for the next person. The person search server 50 performs the processing of Step St14, Step St7 to Step St9, and Step St15 until there is no more person to be compared whether or not the similarity calculated in Step St14 is equal to or larger than the threshold. Execute repeatedly.

  As described above, the person search system 100 according to the second embodiment enables the person search server 50 communicably connected to each of n (n: an integer of 3 or more) cameras and the communicable with the person search server 50. And a connected client terminal 90. When receiving the different captured images sent from each of the n cameras, the person search server 50 analyzes the position information and the characteristic information of the person appearing in each of the captured images, and accumulates the analysis results. The client terminal 90 displays, on the person search screen WD1, captured images of k cameras (k: an integer of 2 or more that satisfies k ≦ n) selected by a user operation among the n cameras. In response to selection of a search target person appearing in any of the k captured images, the client terminal 90 requests the search target person to be searched in (k-1) captured images other than the captured image in which the search target person appears. To the person search server 50. In response to the search request, the person search server 50 extracts an analysis result regarding the search target person in the (k−1) captured images using the analysis result and sends the result to the client terminal 90. The client terminal 90 displays side-by-side cut-out images of a plurality of persons that are the same as or similar to the search target person based on the analysis result regarding the search target person.

  Thus, when an incident or the like occurs at a point (for example, an intersection) where many people come and go, the person search system 100 extracts a captured image of a person to be searched for a person visually similar to the criminal by image search with high accuracy. it can. Therefore, since the person search system 100 can efficiently support the visual characteristics of the criminal and the early grasp of the escape direction, it is possible to appropriately improve the convenience of the police investigation.

  Further, the client terminal 90 displays map data MP1 indicating a point where k cameras are installed on the person search screen WD1. Thereby, when the user requests the search of the search target person from the client terminal 90 to the person search server 50, the user may browse the map data MP1 to determine a geographical positional relationship while viewing the map data MP1. After grasping, it is possible to visually and easily select a camera to be searched.

  Further, the client terminal 90 displays, based on the analysis result regarding the search target person, the moving paths of a plurality of persons that are the same as or similar to the search target person, superimposed on the map data MP1. This allows the user to grasp all the escape route candidates of the culprit such as an incident in the map data MP1, so that it is easy to narrow down precisely.

  In addition, the client terminal 90 updates and displays the movement route of the search target person displayed on the map data MP1 in accordance with a user operation for deleting at least one cut image among the cut images of the plurality of persons. . Thus, the user only needs to perform an operation of deleting a cutout image of an unnecessary person different from the search target person from among the plurality of cutout images displayed in the list display frame RST1, and the destination of the target search target person is deleted. Can be accurately narrowed down.

  In addition, the client terminal 90 superimposes and displays the outline of the person (for example, the person frames WK1 and WK2) shown in the captured video MV1 of any one of the k cameras. The client terminal 90 sends a search request for a search target person to the person search server 50 in which a person corresponding to the selected outline frame (for example, a person frame) is selected by a user operation. . Accordingly, the user simply selects a person (for example, a person in the person frame WK1) of interest on the reproduction screen of the captured video MV1 displayed in a magnified and reproducible manner, and the person becomes a search target person. This allows the client terminal 90 to execute a highly accurate image search, thereby improving the efficiency of police investigation.

  In addition, in response to selection of one of the cut-out images of a plurality of persons, the client terminal 90 searches for a search target person whose search target person is a person corresponding to the selected cut-out image (for example, cut-out image Xpc1). The request is sent to the person search server 50. Thus, the user simply selects a person to be worried about (for example, the person in the cut-out image Xpc1) from among the people appearing in the cut-out image displayed in the list display frame RST1, and sets that person as a search target person. It is possible to cause the client terminal 90 to execute a highly accurate image search, thereby improving the efficiency of police investigation.

  In addition, the client terminal 90 displays cutout images of a plurality of persons that are the same as or similar to the search target person in chronological order along the order of the imaging times when the persons were imaged. This allows the user to find the search target person he or she wants to grasp early by preferentially checking the clipped images displayed in the order of the oldest or newest imaging time.

  In addition, the client terminal 90 displays cutout images of a plurality of persons that are the same as or similar to the search target person, arranged in descending order of similarity with the search target person. This allows the user to check the displayed cut-out image preferentially without performing the scrolling process using the scroll bar SCR1 shown in FIG. 11 or FIG. Can be found.

(Process leading to the configuration of Embodiment 3)
However, in the above-mentioned Patent Document 1, when there are a plurality of persons who have caused an incident or the like at a place where many people come and go (for example, at an intersection), for example, another person (for example, Searching for a captured image similar to the accompanying person from the captured video of another camera based on the captured image of (1) is not considered.

  When an incident or the like occurs, it is important in a police initial investigation to grasp the whereabouts of a criminal group consisting of a plurality of persons at an early stage. The offender group may act collectively or may act separately as a pattern. However, in the conventional technology up to now, clues such as captured images and sighting information of cameras installed at intersections are collected, and police officers rely on these captured images and sighting information and the like, and a plurality of criminal groups likely to correspond to a criminal group are collected. He knew the escape direction of the person. For this reason, when the criminal group is separated and acts separately, it takes time for police officers to grasp the escape direction of the criminal group, and there is a problem that the initial investigation may be delayed and it is not efficient. Was.

  In the following third embodiment, when an incident or the like occurs at a point where many people come and go, a captured image of a person visually similar to a selected person from a group of criminals is extracted with high accuracy. An example of a person search system and a person search method that efficiently assists in visually grasping the visual characteristics and the escape direction of the criminal and improves the convenience of police investigations accurately will be described.

(Embodiment 3)
The internal configuration of the person search system according to the second embodiment is the same as the internal configuration of person search system 100 according to the first embodiment. Therefore, in the description of the second embodiment, the same configuration as the internal configuration of the person search system 100 according to the first embodiment will be denoted by the same reference numeral, and the description will be simplified or omitted, and different contents will be described.

  FIG. 16 is an example of a person search screen WD1 that displays a person search result in a time series when the person frame WK2 of another person displayed on the playback screen of the captured video MV1 of FIG. 9 is used as a search condition of the image search. FIG. As shown in FIG. 16, the person search application displays the cutout image when displaying the playback screen of the captured video MV1 corresponding to the cutout image Xpc1 selected by a predetermined user operation on the person search screen WD1, as shown in FIG. Using the analysis result by the person search server 50 corresponding to Xpc1, the person frames WK1, WK2, gender, and age group of the person shown on the playback screen may be displayed. For example, the person search application displays “M40” (in other words, a man in his 40s) as the gender and age group of the person corresponding to the person frame WK1. Similarly, the person search application displays “M30” (that is, a man who is in his thirties) as the gender and age group of the person corresponding to the person frame WK2.

  The person search application performs a predetermined user operation on the person frames WK1 and WK2 displayed on the playback screen of the captured video MV1, and performs a user operation on another person different from the one that is the source of the display of the playback screen of the captured video MV1. It detects that a person frame (for example, person frame WK2) is selected. In this case, the person search application cuts out the image in the selected person frame WK2, and copies and pastes the cut-out image into the search target person image display frame PL1. That is, the person search application displays the person appearing in the pasted cut-out image in the search target person image display frame PL1 as the search target person image IPS3 of the image search. Further, the person search application uses the analysis result by the person search server 50 corresponding to the person in the pasted cut-out image and sets the main characteristic information of the person (the search target person image IPS2) as a part of the search condition CON3. It is displayed in the search target person image display frame PL1.

  The person search application detects that the image search button IS1 has been pressed by a user operation. In this case, the person search application (specifically, the processor 92) performs various search conditions (for example, a date indicating a search period) for specifying the image data of the search target person image IPS3 and the search target person image IPS3. A search request for a search target person including the search request and the identification information of the camera to be searched is generated, and transmitted to the person search server 50 via the communication unit 93. However, in the third embodiment, in generating the search request, the person search application sets another camera except for the camera corresponding to the captured video MV1 in which the person serving as the source of the search target person image IPS3 is shown as a search target. Select to generate a search request. When receiving the search request sent from the client terminal 90, the person search server 50 uses the analysis results stored in the analysis information DB 56a of the storage unit 56 to perform an image search to perform various search conditions included in the search request. Search for a person who is the same or similar to the search target person who satisfies. The person search server 50 sends the search result to the client terminal 90.

  Upon acquiring the search result received via the communication unit 93, the person search application (specifically, the processor 92) cuts out an image of a person the same as or similar to the search target person (that is, the search target person image IPS3). Xpc31, Xpc32, Xpc33, Xpc34, Xpc35, Xpc36, and Xpc37 are displayed side by side in the list display frame RST1. Each cut-out image is, for example, a thumbnail image.

  In addition, the person search application is used when the plurality of cut-out images Xpc31, Xpc32, Xpc33, Xpc34, Xpc35, Xpc36, and Xpc37 displayed in the list display frame RST1 pass through the intersection corresponding to the camera at which the person was imaged. Each of the moving directions DRC1, DRC2, and DRC3 is displayed so as to be superimposed on the map data MP1.

  FIG. 17 is a diagram illustrating an example of deleting an unnecessary person search result by a user operation. The person search application outputs unnecessary cutout images (for example, cutout images Xpc36, Xpc36, Xpc37, Xpc34, Xpc35, Xpc36, and Xpc37) of the cutout images Xpc31, Xpc32, Xpc33, Xpc34, Xpc35, and Xpc37 displayed in the list display frame RST1 illustrated in FIG. Xpc37) is accepted. In this case, the person search application displays deletion icons DLT1 and DLT2 indicating that the selected cutout images Xpc36 and Xpc37 are to be deleted, superimposed on the cutout images Xpc5 and Xpc24. Note that the person search application may delete the selected cutout images Xpc5 and Xpc24 from the list display frame RST1 instead of displaying the delete icons DLT1 and DLT2 in a superimposed manner.

  In addition, in conjunction with displaying the delete icons DLT1 and DLT2 superimposed on the cutout images Xpc5 and Xpc24, the person search application uses the analysis result by the person search server 50 corresponding to the person shown in the cutout images Xpc5 and Xpc24. Then, the moving direction displayed on the map data MP1 is updated and displayed. More specifically, the person search application is configured to perform a search for all the moving directions DRC1, DRC2, and DRC3 displayed on the map data MP1 before displaying the delete icons DLT3 and DLT4, at the time of passing an intersection of a person shown in the cut-out images Xpc36 and Xpc37. Is updated so as to delete the moving direction, and only the moving directions DRC2 and DRC3 at the time of passing through the intersection of the person shown in the cut-out images Xpc31, Xpc32, Xpc33, Xpc34, and Xpc35 are displayed. Thereby, the user refers to FIG. 14 and FIG. 17, and the two persons (for example, the criminal group) of the person frames WK1 and WK2 shown in the captured video MV1 captured by the first camera are installed with the first camera. Since it is possible to recognize that they have joined at an intersection, the efficiency of police investigations can be improved.

  Next, an operation procedure of person search system 100 according to Embodiment 3 will be described with reference to FIG. FIG. 18 is a flowchart illustrating an example of an operation procedure of person search system 100 according to Embodiment 3. The process illustrated in FIG. 18 is executed by the client terminal 90 and the person search server 50. In the description of the processing illustrated in FIG. 18, the same processing as the processing illustrated in FIG. 10 will be assigned the same step numbers, and the description will be simplified or omitted, and different contents will be described. As a premise of the processing illustrated in FIG. 18, as described with reference to FIG. 10 or FIG. 15, the client terminal 90 includes a person search server corresponding to each of the plurality of cut-out images displayed in the list display frame RST1. The analysis result (search result) obtained by 50 is obtained.

  In FIG. 18, after detecting that one of the plurality of cutout images displayed in the list display frame RST1 is selected by a user operation after step St1, the client terminal 90 selects the selected cutout image (for example, The captured video (for example, the captured video MV1) obtained by capturing the person shown in the cut-out image Xpc1) is displayed on the person search screen WD1 and repeatedly played back (St21). The client terminal 90 has obtained the analysis result generated by the person search server 50 in step St1, and uses this analysis result to place the person frames WK1, WK2 at the positions of the respective persons on the playback screen of the captured video MV1. Is displayed (St22).

  The client terminal 90 operates the user terminal to select one of the person frames WK1 and WK2 displayed in step St22 (for example, a person different from the person in the person frame WK1 that is the source of the display of the playback screen of the captured video MV1). It is detected that the person frame WK2) has been selected (St23). In this case, the client terminal 90 displays the cut-out image of the person in the selected person frame WK2 by copying and pasting it into the search target person image display frame PL1.

  When the client terminal 90 detects, for example, that the image search button IS1 has been pressed by a user operation after step St23, the client terminal 90 detects the image data of the search target person image IPS3 displayed in the search target person image display frame PL1 and the search target thereof. A search request for a search target person including various search conditions (for example, a date indicating a search period) for identifying the person image IPS3 and identification information of a camera to be searched is generated. Send it to the search server 50. However, as described above, in the third embodiment, in generating the search request, the person search application excludes the camera corresponding to the captured image MV1 in which the person who is the source of the search target person image IPS3 is shown. A search request is generated by selecting a camera as a search target.

  Upon receiving the search request sent from the client terminal 90, the person search server 50 includes the identification information of the camera to be searched for on the date indicating the search period included in the search request, and captures an image corresponding to the identification information. Is transmitted to the video recorder 70. Based on the transmission request from the person search server 50, the video recorder 70 acquires captured image data corresponding to the camera identification information included in the transmission request, and transmits the data to the person search server 50. The person search server 50 acquires the data of the captured image of the camera transmitted from the video recorder 70. Note that the person search server 50 may store the data of the captured video transmitted from each camera in the storage unit 56. In this case, instead of receiving and acquiring the data from the video recorder 70, the storage unit 56 May be directly read out from and acquired.

  The person search server 50 analyzes each of the acquired images captured by the camera, extracts and identifies the person appearing in the captured image, and temporarily stores the extracted person in the memory 52. The person search server 50 images the analysis result of the person in the person frame WK2 selected in Step St23 and the feature information (see FIG. 11 or FIG. 12) of the search target person included in the search request sent from the client terminal 90. The degree of similarity is calculated by comparison through a search (St24). Note that the person search server 50 uses the analysis result of the captured images of the respective cameras generated in step St <b> 1 to obtain an image corresponding to the identification information of the search target camera on the date indicating the search period included in the search request. A video analysis result (for example, a clipped image) may be read and acquired. In this case, the person search server 50 determines in step St24 the analysis result (for example, a cut-out image) of the read captured video and the characteristic information of the search target person included in the search request sent from the client terminal 90 (FIG. 12 may be compared by image search to calculate the similarity (St24).

  The person search server 50 determines whether or not the similarity calculated in Step St24 is equal to or larger than a threshold (see Step St5) (St7). If the person search server 50 determines that the calculated similarity is equal to or larger than the threshold (St7, YES), the cutout image (for example, a thumbnail image) in which a person exceeding the threshold is shown as an example of the search result, and the cutout is performed. The image data and an instruction to display the cut-out image on the client terminal 90 are sent to the client terminal 90 (St8). The client terminal 90 displays the corresponding cut-out image in the list display frame RST1 of the person search screen WD1 based on the instruction sent from the person search server 50 (St8). As described above, when the client terminal 90 receives a plurality of cutout images to be displayed in the list display frame RST1, the client terminal 90 arranges the cutout images in the order of similarity, in chronological order, or in the order of intersection. May be displayed.

  On the other hand, when determining that the calculated similarity is smaller than the threshold (St7, NO), the person search server 50 omits the process of Step St8. After this or after step St8, the person search server 50 determines whether or not there is a person to be compared whether or not the similarity calculated in step St24 is equal to or greater than a threshold (St9). . When it is determined that there is no person to be compared with whether the similarity calculated in step St24 is equal to or greater than the threshold (St9, NO), the processing of the person search server 50 (that is, the client terminal 90) The processing corresponding to the search request from is terminated. The client terminal 90 displays a plurality of cutout images corresponding to the search result by the person search server 50 side by side (see FIG. 16 or FIG. 17).

  If the person search server 50 determines that there is a person to be compared whether or not the similarity calculated in step St24 is equal to or larger than the threshold (St9, YES), the person corresponding to the next person is It is determined that the processing of step St24 is to be executed (St25). Thereafter, the person search server 50 performs the process of step St24 for the next person. The person search server 50 performs the processing of Step St24, Step St7 to Step St9, and Step St25 until there is no more person to be compared whether the similarity calculated in Step St24 is equal to or larger than the threshold. Execute repeatedly.

  As described above, the person search system 100 according to the third embodiment is configured such that the person search server 50 communicably connected to each of n (n: an integer of 3 or more) cameras and the person search server 50 can communicate with each other. And a connected client terminal 90. When receiving the different captured images sent from each of the n cameras, the person search server 50 analyzes the position information and the characteristic information of the person appearing in each of the captured images, and accumulates the analysis results. The client terminal 90 displays, on the person search screen WD1, captured images of k cameras (k: an integer of 2 or more that satisfies k ≦ n) selected by a user operation among the n cameras. The client terminal 90 selects one of the captured images in which the accompanying person appears in accordance with the selection of the accompanying person (for example, the person in the person frame WK2) acting together with the search target person in any of the k captured images. A search request for a companion person in the (k-1) captured images other than the above is sent to the person search server 50. In response to the search request, the person search server 50 extracts an analysis result regarding the accompanying person in the (k−1) captured images using the analysis result, and sends the result to the client terminal 90. The client terminal 90 displays side-by-side cut-out images of a plurality of persons who are the same as or similar to the accompanying person based on the analysis result of the accompanying person.

  Thus, when an incident or the like occurs at a point (for example, an intersection) where many people come and go, the person search system 100 can select a plurality of persons (for example, persons corresponding to each of the person frames WK1 and WK2) from a criminal group. A picked-up image of a person visually similar to the selected person (for example, an accomplice in the person frame WK2) can be extracted with high accuracy by an image search. Accordingly, the person search system 100 can efficiently support the visual characteristics of the criminal and the early grasp of the escape direction, and can appropriately improve the convenience of the police investigation.

  Further, the client terminal 90 displays map data MP1 indicating a point where k cameras are installed on the person search screen WD1. Thereby, when the user requests the search of the search target person from the client terminal 90 to the person search server 50, the user may browse the map data MP1 to determine a geographical positional relationship while viewing the map data MP1. After grasping, it is possible to visually and easily select a camera to be searched.

  In addition, the client terminal 90 superimposes and displays the moving routes of a plurality of persons who are the same as or similar to the accompanying person on the map data MP1 based on the analysis result regarding the accompanying person. This allows the user to grasp all the escape route candidates of the accompanying person (for example, accomplice person) of the criminal group of the incident or the like in the map data MP1, so that it is easy to narrow down accurately.

  In addition, the client terminal 90 updates and displays the traveling route of the accompanying person displayed in the map data MP1 in accordance with a user operation for deleting at least one of the cut-out images of a plurality of persons. Accordingly, the user only needs to perform an operation of deleting a cut-out image of an unnecessary person different from a companion person (for example, an accomplice person) from among a plurality of cut-out images displayed in the list display frame RST1, thereby obtaining a target companion image. It is possible to accurately narrow down whereabouts of a person (for example, an accomplice).

  Further, the client terminal 90 superimposes and displays the outline of the person (for example, the person frames WK1 and WK2) shown in the captured image of any one of the k cameras. In response to the selection of the outer frame (for example, the person frame WK2) by the user operation, the client terminal 90 sends to the person search server 50 a search request for a companion person having the person corresponding to the selected outer frame as an accompanying person. . Thereby, the user is anxious among the persons appearing in the reproduction screen of the captured video MV1 being reproduced (for example, the main criminal is often accompanied by the person in the person frame WK1 assumed by the user. By simply selecting a person in the user frame WK2), it is possible to cause the client terminal 90 to execute a high-precision image search with that person as a companion (for example, an accomplice), thereby improving the efficiency of police investigation. Is improved.

  In addition, the client terminal 90 displays cut-out images of a plurality of persons who are the same as or similar to the accompanying persons in a time-series manner in the order of the imaging time when each person was imaged. Thereby, the user can find the companion person (for example, accomplice person) he / she wants to grasp earlier by preferentially checking the cut-out images displayed in the order of the oldest or newest imaging time.

  In addition, the client terminal 90 displays cut-out images of a plurality of persons that are the same as or similar to the accompanying person, arranged in descending order of similarity with the accompanying person. This allows the user to preferentially check the displayed cut-out image without performing the scrolling process using the scroll bar SCR1 shown in FIG. 11 or FIG. ) Can be found early.

  Although various embodiments have been described with reference to the drawings, it is needless to say that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents can be made within the scope of the claims. Naturally, it is understood that they belong to the technical scope of the present disclosure. Further, the components in the above-described various embodiments may be arbitrarily combined without departing from the spirit of the invention.

  In the above-described first, second, and third embodiments, the installation locations of the cameras 10, 10a,... Have been described as being outdoors (for example, at intersections), but the installation locations are not limited to outdoors. For example, it may be installed in an indoor facility such as a shopping mall. In this case, the map data MP1 may not be a road map, but a map of the relevant facility (eg, a map of a shopping mall) may be displayed in the person search screen WD1.

  In the first, second, and third embodiments, a criminal who caused a case or the like or an accomplice who acts together with the criminal is described as an example of a person to be searched in a police investigation. It is not limited to these examples. For example, a child who has been lost outdoors or indoors may be set as the search target person. In this case, the person retrieving system 100 can search for a lost child who is a search target person with the patrol officer or security guard on patrol as a user.

  The present disclosure, when an incident occurs at a point where many people come and go, extracts a captured image of a person visually similar to the criminal with high accuracy, and efficiently grasps the visual characteristics of the criminal and the early direction of the escape. The present invention is useful as a person search system and a person search method for providing accurate support and appropriately improving the convenience of police investigations.

10, 10a camera 11 imaging unit 12, PRC1, PRC2, 92 processor 13, 52, 72, 95 memory 14, 51, 71, 93 communication unit 15, 96 recording unit 50 person search server 53 person search unit 54 person analysis unit 55 Tag assigning unit 56 Storage unit 56a Detection information DB
56b Incident DB
70 video recorder 73 video search unit 74 video recording processing unit 75 video storage unit 90 client terminal 91 operation unit 94 display 100 person search system HDS headset

Claims (4)

a server communicably connected to each of n (n: an integer of 3 or more) cameras, and a client terminal communicably connected to the server;
The server is
Upon receiving different captured images sent from each of the n cameras, analyze the position information and feature information of the person shown in each of the captured images and accumulate as analysis results,
The client terminal,
Displaying, on a screen, captured images of k (k: an integer of 2 or more satisfying k ≦ n) cameras selected by a user operation among the n cameras,
According to selection of a search target person appearing in any of the images picked up by the k cameras, the image picked up by (k-1) cameras other than the camera picked up the image picked up image showing the search target person Sending a search request for the search target person in the video to the server,
The server is
In response to the search request, using the analysis result, the search target person in the image captured by the (k-1) cameras and the intersection where the (k-1) cameras of the search target person are arranged Extract the movement direction in and send it to the client terminal,
The client terminal,
The moving direction of the search target person at the intersection where the (k-1) cameras are arranged is displayed so as to be superimposed on the map data indicating the intersection where the k cameras are installed.
People search system. The client terminal,
An outer frame of a person shown in a captured image of one of the k cameras is superimposed on the captured image and displayed.
In response to the selection of the outer shape frame by the user operation, send a search request of the search target person to the server with the person corresponding to the selected outer shape frame as the search target person,
The person search system according to claim 1. The client terminal,
In response to selection of one of the cutout images of the plurality of search target persons, a search request for the search target person to be a person corresponding to the selected cutout image as the search target person is sent to the server.
The person search system according to claim 1. A person search method using a server communicably connected to each of n (n: an integer of 3 or more) cameras, and a client terminal communicably connected to the server,
The server is
Upon receiving different captured images sent from each of the n cameras, analyze the position information and feature information of the person reflected in each of the captured images and accumulate as analysis results,
The client terminal,
Displaying, on a screen, captured images of k (k: an integer of 2 or more satisfying k ≦ n) cameras selected by a user operation among the n cameras,
According to selection of a search target person appearing in any of the images picked up by the k cameras, the image picked up by (k-1) cameras other than the camera picked up the image picked up image showing the search target person Sending a search request for the search target person in the video to the server,
The server is
In response to the search request, using the analysis result, the search target person in the image captured by the (k-1) cameras and the intersection where the (k-1) cameras of the search target person are arranged Extract the movement direction in and send it to the client terminal,
The client terminal,
The moving direction of the search target person at the intersection where the (k-1) cameras are arranged is displayed so as to be superimposed on map data indicating the intersection where the k cameras are installed.
Person search method.

Images