JP2021099709A - Investigation support system and investigation support method - Google Patents

Abstract

To provide an investigation support system and an investigation support method, which quickly and efficiently support identification of an escape vehicle used for escape by a suspect who caused an affair or the like, and improve a convenience of investigation by police and other investigative agencies.SOLUTION: An investigation support system 1 comprises a video management server 40, a face recognition server 50, a person search server 60, an action detection server 70, a vehicle search server 80, an LPR server 90, and an AI integrated server 10 as an analysis server. The AI integrated server 10 issues an alarm notification regarding an affair or the like when it is determined that license number information of a first analysis result and license number information of the second analysis result are coincident with each other, and facial images of both occupants are different.SELECTED DRAWING: Figure 1

Description

This disclosure relates to an investigation support system and an investigation support method that support an investigation of an incident by an investigative agency.

A plurality of camera devices are arranged at predetermined locations on the vehicle's travel path, and camera image information captured by each camera device is displayed on a display device in a terminal device mounted on the vehicle via a network and a wireless information exchange device. A technique of displaying is known (see, for example, Patent Document 1). According to Patent Document 1, a user can obtain a real-time camera image having a large amount of information from camera image information captured by a plurality of cameras arranged on a traveling path of a vehicle.

JP-A-2007-174016

In Patent Document 1, since the camera image information captured by each of the plurality of camera devices can be displayed on the display device in the terminal device mounted on the vehicle, the user (for example, the driver) can arrange the respective camera devices. You can check the real-time camera image information in. However, in Patent Document 1, in view of the recent fact that the forms of incidents or accidents (hereinafter referred to as "incidents, etc.") are diversifying, the escape vehicle used for escape by the suspect who caused the incident, etc. Is not considered to detect and notify the user. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the escape vehicle used by the suspect for escape in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage.

This disclosure was devised in view of the above-mentioned conventional circumstances, and assists the suspect who caused the incident, etc., to quickly and efficiently identify the escape vehicle used for escape, and is provided by an investigative agency such as the police. The purpose is to provide an investigation support system and an investigation support method that improve the convenience of an investigation.

The present disclosure is an investigation support system in which an analysis server and an integrated server are communicably connected, and when the analysis server receives a first captured image of a vehicle captured at the first imaging timing, the analysis server is described. The face of the occupant of the vehicle and the license plate are analyzed, the first analysis result including the occupant's face image and the license number information is sent to the integrated server, and the integrated server sends the first analysis result to the integrated server. Registered in the database in association with the first imaging timing, the analysis server analyzes the occupant's face and license plate of the vehicle when it receives the second captured image showing the vehicle captured at the second imaging timing. Then, the second analysis result including the occupant's face image and the license number information is sent to the integrated server, and the integrated server uses the license number information included in the first analysis result and the second analysis. When it is determined that the license number information included in the result matches and the face image of the occupant included in the first analysis result is different from the face image of the occupant included in the second analysis result. , Provide an investigation support system that notifies alarms about incidents, etc.

Further, the present disclosure is an investigation support method executed by an investigation support system in which an analysis server and an integrated server are communicably connected, and a vehicle imaged by the analysis server at the first imaging timing is displayed. Upon receiving the first captured image, the face of the occupant of the vehicle and the license plate are analyzed, and the first analysis result including the occupant's face image and the license number information is sent to the integrated server by the integrated server. When the first analysis result is registered in the database in association with the first imaging timing and the analysis server receives a second captured image of the vehicle captured at the second imaging timing, the vehicle is subjected to the analysis. The analysis of the occupant's face and the license plate is performed, the second analysis result including the occupant's face image and the license number information is sent to the integrated server, and the integrated server includes the first analysis result included in the first analysis result. The license number information and the license number information included in the second analysis result match, and the face image of the occupant included in the first analysis result and the face of the occupant included in the second analysis result. Provide an investigation support method that notifies an alarm regarding an incident, etc. when it is determined that the image is different.

Further, the present disclosure is an investigation support system including a user terminal and an integrated server that is communicably connected between a plurality of analysis servers, and the user terminal is a plurality of different object feature elements related to an incident or the like. An analysis server selection screen showing a list of the plurality of analysis servers provided corresponding to each is displayed, a first analysis server is selected from the analysis server selection screen, and the selected first analysis server is used. Based on the input of the search condition including the object feature element to be searched, a search request for the corresponding object satisfying the search condition is sent to the integrated server, and the integrated server performs the first analysis based on the search request. Provided is an investigation support system that sends a search instruction for the corresponding object to a server, receives a search result from the first analysis server, and sends the search result to the user terminal.

Further, the present disclosure is an investigation support method executed by an investigation support system including an integrated server that is communicably connected between a user terminal and a plurality of analysis servers. A analysis server selection screen showing a list of the plurality of analysis servers provided corresponding to each of a plurality of different object feature elements is displayed, and the user terminal selects the first analysis server from the analysis server selection screen. Then, based on the input of the search condition including the object feature element searched by the selected first analysis server, a search request for the corresponding object satisfying the search condition is sent to the integrated server, and the integrated server sends the search request to the integrated server. Based on the search request, a search support method is provided in which a search instruction for the corresponding object is sent to the first analysis server, a search result from the first analysis server is received, and the search result is sent to the user terminal.

Further, the present disclosure is an investigation support system including an integrated server that is communicably connected between a user terminal and n (n: 2 or more integers) analysis servers. When the communication status with each of the n analysis servers is periodically monitored and a search condition including k (an integer from k: 1 to n) of different object feature elements is received from the user terminal, the above Of the n analysis servers, k analysis servers provided corresponding to each of the k object feature elements included in the search condition are determined, and the corresponding object is assigned to each of the k analysis servers. When a search request is sent and it is detected that a communication interruption has occurred with a specific analysis server among the k analysis servers, the search results received from the (k-1) analysis servers and the specific analysis are detected. A communication disconnection notification that a communication interruption with the server has occurred is sent to the user terminal, and the user terminal sends the search results received from the (k-1) analysis servers and the communication interruption notification. Provide an investigation support system that displays the search result screen including.

Further, the present disclosure is an investigation support method executed by an investigation support system including an integrated server that is communicably connected between a user terminal and n (n: 2 or more integers) analysis servers. The integrated server periodically monitors the communication status with each of the n analysis servers, and the integrated server periodically monitors k (k: integers from 1 to n) different object feature elements. When a search condition including is received from the user terminal, k analysis servers provided corresponding to each of the k object feature elements included in the search condition are determined among the n analysis servers. The integrated server sends a search request for the corresponding object to each of the k analysis servers, and the integrated server causes a communication interruption with a specific analysis server among the k analysis servers. When it is detected, the search results received from the (k-1) analysis servers and the communication interruption notification that the communication interruption between the specific analysis servers has occurred are sent to the user terminal, and the user terminal sends the communication interruption notification. Provided is an investigation support method for displaying a search result screen including search results received from the (k-1) analysis servers and the communication interruption notification.

Further, the present disclosure is an investigation support system including a user terminal and an integrated server that is communicably connected between a first analysis server and a second analysis server, and the integrated server is an incident. When a search condition including a first object feature element related to the above and the like and a second object feature element related to the case or the like is received from the user terminal, a search request for the corresponding object satisfying the first object feature element is made. The search request for the corresponding object that is sent to the analysis server and satisfies the second object feature element is sent to the second analysis server, and the first search result including the corresponding object that satisfies the first object feature element is sent to the second analysis server. The second search result including the corresponding object received from the analysis server 1 and satisfying the second object feature element is received from the second analysis server, and the second search result is excluded from the first search result. The third search result is sent to the user terminal, and the user terminal provides an investigation support system that displays a search result screen including the third search result sent from the integrated server.

Further, the present disclosure is an investigation support method executed by an investigation support system including a user terminal and an integrated server that is communicably connected to at least the first analysis server and the second analysis server. When the integrated server receives a search condition including a first object feature element related to an incident or the like and a second object feature element related to the incident or the like from the user terminal, the corresponding object satisfying the first object feature element. The search request is sent to the first analysis server and the search request for the corresponding object satisfying the second object feature element is sent to the second analysis server, and the integrated server satisfies the first object feature element. The first search result including the corresponding object is received from the first analysis server, and the second search result including the corresponding object satisfying the second object feature element is received from the second analysis server, and the integrated server is received. The third search result obtained by excluding the second search result from the first search result is sent to the user terminal, and includes the third search result sent from the integrated server by the user terminal. Provide an investigation support method that displays the search result screen.

Further, the present disclosure is an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server, and each of the plurality of analysis servers is connected to each of a plurality of cameras. It holds a collation list of objects that are different from the objects that can be searched by other analysis servers for incidents, etc., and each of the plurality of analysis servers holds the collation list that it holds by analyzing the images captured by the camera. When an object satisfying the above conditions is detected, a detection notification including the event name indicating the detection of the object, the detection date and time, and the information on the arrangement position of the camera is sent to the user terminal via the integrated server, and the user terminal sends the detection notification Provided is an investigation support system that displays an event detection screen including information on the event name, detection date and time, and the placement position of the camera included in the detection notification sent from the analysis server.

Further, the present disclosure is an investigation support method executed by an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server, and a plurality of analysis support methods are provided by each of the plurality of analysis servers. By connecting to each of the cameras of the above, each of the plurality of analysis servers holds a collation list of objects different from the objects that can be searched by other analysis servers regarding an incident, etc., and each of the plurality of analysis servers When an object satisfying the collation list held by the server is detected by analysis of the captured image of the camera, a detection notification including information on the event name indicating the detection of the object, the detection date and time, and the arrangement position of the camera is transmitted. An event detection screen that is sent to the user terminal via the integrated server and includes information on the event name, detection date and time, and the camera placement position included in the detection notification sent from the analysis server by the user terminal. Provide an investigation support method to display.

Further, the present disclosure is an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server, and the first analysis server among the plurality of analysis servers captures images of a camera. When a specific action related to an incident or the like is detected by video analysis, an alarm notification including at least an image captured by the camera in which the action is detected is sent to the user terminal via the integrated server, and the user terminal sends the user terminal to the user terminal. An event notification screen including the captured image included in the alarm notification sent from the first analysis server and the person analysis button of the actor of the action reflected in the captured image is displayed, and the person analysis button can be specified. Based on this, the person analysis of the actor reflected in the captured image is requested via the integrated server, and the second analysis server among the plurality of analysis servers is said based on the request from the user terminal. The person analysis result of the actor reflected in the captured image is sent to the user terminal via the integrated server, and the user terminal is the actor included in the person analysis result sent from the second analysis server. An analysis result screen including the cutout image and the tracking button of the person appearing in the cutout image is displayed, and based on the designation of the tracking button, the blacklist registration of the person corresponding to the tracking button is performed via the integrated server. The second analysis server registers the cutout image of the person corresponding to the tracking button in the blacklist data of the second analysis server based on the request from the user terminal. Provide a support system.

Further, the present disclosure is an investigation support method executed by an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server, and is the first of the plurality of analysis servers. When the analysis server detects a specific action related to an incident or the like by analyzing the image captured by the camera, an alarm notification including at least the captured image of the camera in which the action is detected is sent to the user terminal via the integrated server. The user terminal displays an event notification screen including the captured image included in the alarm notification sent from the first analysis server and the person analysis button of the actor of the action reflected in the captured image. Based on the designation of the person analysis button, the user terminal requests the person analysis of the actor reflected in the captured image via the integrated server, and the second analysis server among the plurality of analysis servers requests the person analysis. Based on the request from the user terminal, the person analysis result of the actor reflected in the captured image is sent to the user terminal via the integrated server, and is sent from the second analysis server by the user terminal. An analysis result screen including a cutout image of the actor included in the person analysis result and a tracking button of the person reflected in the cutout image is displayed, and the tracking is performed by the user terminal based on the designation of the tracking button. The blacklist registration of the person corresponding to the button is requested via the integrated server, and the second analysis server cuts out the person corresponding to the tracking button based on the request from the user terminal. Provided is an investigation support method for registering an image in the blacklist data of the second analysis server.

According to the present disclosure, it is possible to quickly and efficiently support the identification of the suspect who caused the incident or the escape vehicle used by the suspect for escape, and it is possible to improve the convenience of the investigation by the police or other investigative agencies. ..

Block diagram showing a system configuration example of the investigation support system A block diagram showing an example of the hardware configuration of the cameras that make up the investigation support system Block diagram showing hardware configuration examples of various analysis servers that make up the investigation support system Block diagram showing hardware configuration examples of client terminals and mobile terminals that make up an investigation support system Diagram showing an example of the alarm notification screen Sequence diagram showing an example of the operation procedure of the investigation support system according to the first embodiment Diagram showing a part of the display example of the search condition input screen A sequence diagram showing an example of another operation procedure of the investigation support system according to the first embodiment. A diagram showing an example of a search result screen that notifies the occurrence of a connection error with the vehicle search server. A diagram showing an example of a search result screen that notifies the return of the connection with the vehicle search server. Sequence diagram showing an example of the operation procedure of the investigation support system according to the second embodiment The figure which shows the example of the search condition detail screen which concerns on Embodiment 3. The figure which shows the display example of a part of the search condition input screen which concerns on Embodiment 3. Sequence diagram showing an example of the operation procedure of the investigation support system according to the third embodiment Figure showing an example of event detection operation screen Sequence diagram showing an example of the operation procedure of the investigation support system according to the fourth embodiment Diagram showing the display transition from the event detection screen to the analysis result screen Diagram showing an example of a person tracking operation screen A sequence diagram showing an example of an operation procedure of the investigation support system according to the fifth embodiment.

Hereinafter, embodiments in which the investigation support system and the investigation support method according to the present disclosure are specifically disclosed will be described in detail with reference to the attached drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

Hereinafter, the suspect or the suspect who caused the incident in the city or in the building will use the captured image captured by cameras installed in various places in the city or in the building (for example, a surveillance camera such as a security camera). An example of supporting the investigation of a police officer who identifies or tracks an escape vehicle used for escape by an investigation support system will be described. The police are an example of an investigative agency. Further, in the following description, it is assumed that the captured image data conceptually includes the captured image data.

(Embodiment 1)
FIG. 1 is a block diagram showing a system configuration example of the investigation support system 1. The investigation support system 1 includes an AI (Artificial Intelligent) integrated server 10, a video management server 40, a face authentication server 50, a person search server 60, an action detection server 70, a vehicle search server 80, and an LPR (License Plate). Recognition) It is a configuration including at least a server 90. Further, the investigation support system 1 may be configured to further include a client terminal VW1 as a viewer and a mobile terminal VW2. The video management server 40, the face recognition server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90 are each of a plurality of (for example, 20) cameras C1 to C20 via the network NW1. Connected to communicate with each. Each of the video management server 40, the face authentication server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90 may be on-premises servers in the police station, or may be networks such as the Internet. It may be provided as a cloud server connected to the network.

Although only one client terminal VW1 and one mobile terminal VW2 are shown in FIG. 1, a plurality of each may be provided. Further, the investigation support system 1 is not limited to being used only within a single police station, and may be applied to an example in which a plurality of police stations carry out a joint investigation.

Each of the cameras C1 to C20 is installed in various places in the city or in a building for surveillance purposes, generates image data of an imaging area (in other words, a subject), and generates image data of an imaging area (in other words, a subject), and each server (specifically, specifically,) via network NW1. The data is sent to the face recognition server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90). In the following description, the captured image data includes not only the captured image data itself, but also the identification information of the camera that captured the captured image and the information of the imaging date and time. Further, the camera identification information may include camera placement position information in addition to the camera identification information. Each of the cameras C1 to C20 may be fixedly installed on the road side of a main highway such as a national road or a prefectural road, or may be fixedly installed near an intersection. Each of the cameras C1 to C20 is a server (specifically, a video management server 40, a face authentication server 50, a person search server 60, an action detection server 70, and a vehicle search) via a network NW1 such as an intranet communication line. It is connected to the server 80 and the LPR server 90) so as to be able to communicate with each other. The network NW1 is composed of a wired communication line (for example, an optical communication network using an optical fiber), but may be composed of a wireless communication network. The cameras C1 to C20 may all be made by the same manufacturer, or some cameras may be made by another company. Further, in the configuration example of FIG. 1, the captured video data of the cameras C1 to C20 are common to each server (specifically, the video management server 40, the face authentication server 50, the person search server 60, and the behavior detection server 70). , Vehicle search server 80, LPR server 90), but the common captured video data received by each server may be all captured video data of cameras C1 to C20, or of some cameras. Only the captured image data may be used. Further, although the details will be described with reference to FIG. 2, the cameras C1 to C20 may be capable of performing video analysis of captured video data (for example, detection of the face of an occupant in a vehicle).

The video management server 40 as an analysis server is installed in, for example, a police station, and has a configuration including at least a processor 41 and a database 42. Hereinafter, the database may be abbreviated as "DB". The image management server 40 stores the data of the processing results of the face recognition server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90, and captures images of the cameras C1 to C20. Save data. The image management server 40 may receive and save the captured image data of the cameras C1 to C20 via the network NW1, or the face authentication server 50, the person search server 60, the action detection server 70, and the vehicle search. The captured image data of the cameras C1 to C20 may be received and stored from either the server 80 or the LPR server 90. Further, the video management server 40 stores captured video data satisfying the request in response to a request sent from the client terminal VW1 in response to the operation of the operator in the police station or the mobile terminal VW2 in response to the operation of the police officer in the field. It may be read from and sent to the client terminal VW1 or the mobile terminal VW2.

The face recognition server 50 as an analysis server is installed in, for example, a police station, and has a configuration including at least a processor 51 and a database 52. Each time the processor 51 of the face recognition server 50 receives the captured image data of the cameras C1 to C20, the processor 51 performs image analysis such as detecting the face of a person (an example of an object to be searched) reflected in the captured image data. , The video analysis result is saved in the database 52. When the processor 51 of the face recognition server 50 detects a face that is the same as or similar to the face image registered in the blacklist data (see below) during video analysis, the person whose face image is registered in the blacklist data. An alarm notification for notifying the detection of is may be generated as a video analysis result. The face image to be generated for the alarm notification is registered in advance in the face authentication server 50, and this registration may be performed by an instruction of registration from the client terminal VW1 or the mobile terminal VW2, for example, by an operation of an operator or the like. This alarm notification is sent from the face recognition server 50 to the AI integration server 10 each time it is generated. The image analysis results include, for example, a person's face image reflected in the captured image data (that is, a face image generated by cutting out the person's face in the captured image frame constituting the captured image data) and used for image analysis. The imaging date and time of the captured image data and the identification information of the camera are included. Further, when the processor 51 of the face recognition server 50 receives a processing instruction (for example, a face image collation instruction) from the AI integrated server 10, the face image to be collated included in the processing instruction is the blacklist data of the database 52 (for example, the face image collation instruction). It is collated whether or not it is registered in (see below), and the collation result is saved in the database 52. Here, the blacklist data (an example of a face database) is data in which personal information including a face image of a criminal record who has caused a past incident or the like is classified and registered for each incident or the like, and is registered in the database 52. Has been done. The blacklist data may be registered in the database 52 or may be registered in another external database (not shown).

The person search server 60 as an analysis server is installed in, for example, a police station, and has a configuration including at least a processor 61 and a database 62. Each time the processor 61 of the person search server 60 receives the captured image data of the cameras C1 to C20, the processor 61 extracts information about the person (for example, a suspect, an example of the object to be searched) reflected in the captured image data. And save the video analysis result in the database 62. When the processor 61 of the person search server 60 detects a person who satisfies the person attribute information (for example, information indicating the appearance characteristics of a suspicious person, an example of an object characteristic element) during video analysis, the processor 61 of the person who satisfies the person attribute information An alarm notification for notifying the detection may be generated as a video analysis result. The person attribute information for which the alarm notification is to be generated is registered in advance in the person search server 60, and this registration may be performed by an instruction of registration from the client terminal VW1 or the mobile terminal VW2, for example, by an operation of an operator or the like. This alarm notification is sent from the person search server 60 to the AI integration server 10 each time it is generated. The video analysis results include, for example, the person information shown in the captured video data (for example, the face, gender, age, hairstyle, height, body shape, belongings, equipment) of the person shown in the captured video and the imaging used for the video analysis. It includes the imaging date and time of the video data and the identification information of the camera. The processor 61 of the person search server 60 assigns this person information in association with the captured image data and stores it in the database 62. This video analysis result is referred to at the time of searching for the presence or absence of the corresponding person information, which is performed based on the processing instruction (for example, the search instruction of the person information) sent from the AI integrated server 10 when an incident or the like occurs. ..

The behavior detection server 70 as an analysis server is installed in, for example, a police station, and has a configuration including at least a processor 71 and a database 72. Each time the processor 71 of the action detection server 70 receives the captured image data of the cameras C1 to C20, the processor 71 detects the presence or absence of a predetermined action (see below) caused by at least one person reflected in the captured image data. And save the video analysis result in the database 72. The image analysis result includes, for example, the content (type) of a predetermined action, the imaging date and time of the captured image data used for the image analysis, and the identification information of the camera. Here, the predetermined action is at least one of actions that may trigger an incident such as a person's wobbling, fighting, possession of a pistol, shoplifting, etc., but is not limited to these actions. When the processor 71 of the action detection server 70 detects a predetermined action, the processor 71 generates an alarm notification including the imaging date and time and camera identification information corresponding to the captured image data in which the predetermined action is detected, and causes the AI integrated server 10 to generate an alarm notification. send.

The vehicle search server 80 as an analysis server is installed in, for example, a police station, and has a configuration including at least a processor 81 and a database 82. Each time the processor 81 of the vehicle search server 80 receives the captured image data of the cameras C1 to C20, the processor 81 extracts information about the vehicle (for example, an escape vehicle, an example of the object to be searched) reflected in the captured image data. And save the video analysis result in the database 82. When the processor 81 of the vehicle search server 80 detects a vehicle that satisfies vehicle attribute information (for example, information indicating external characteristics such as the vehicle type or color of an escape vehicle, an example of an object characteristic element) during video analysis, the vehicle An alarm notification for notifying the detection of a vehicle satisfying the attribute information may be generated as a video analysis result. The vehicle attribute information for which the alarm notification is to be generated is registered in advance in the vehicle search server 80, and this registration may be performed by an instruction of registration from the client terminal VW1 or the mobile terminal VW2, for example, by an operation of an operator or the like. This alarm notification is sent from the vehicle search server 80 to the AI integrated server 10 each time it is generated. The video analysis results include, for example, vehicle information reflected in the captured video data (for example, information on the vehicle type, vehicle type, vehicle color, license number, etc. of the vehicle shown in the captured video), the imaging date and time of the captured video data used for the video analysis, and Includes camera identification information. The processor 81 of the vehicle search server 80 assigns this vehicle information in association with the captured image data and stores it in the database 82. This video analysis result is referred to when searching for the presence or absence of the corresponding vehicle information, which is performed based on a processing instruction (for example, a vehicle information search instruction) sent from the AI integrated server 10 when an incident or the like occurs. ..

The LPR server 90 as an analysis server is installed in a police station, for example, and includes at least a processor 91 and a database 92. Each time the processor 91 of the LPR server 90 receives the captured image data of the cameras C1 to C20, the processor 91 obtains the license number (an example of the object to be searched) printed on the license plate of the vehicle reflected in the captured image data. The video analysis to be extracted is performed, and the video analysis result is stored in the database 92. When the processor 91 of the LPR server 90 detects a license plate that satisfies the license number information of the suspicious vehicle (for example, the license number information of the escape vehicle) during video analysis, the processor 91 notifies the detection of the license plate that satisfies the license number information of the suspicious vehicle. An alarm notification for this may be generated as a video analysis result. The license number information of the suspicious vehicle for which the alarm notification is generated is registered in advance in the LPR server 90, and this registration is performed by, for example, an operation of an operator or the like according to a registration instruction from the client terminal VW1 or the mobile terminal VW2. Good. This alarm notification is sent from the LPR server 90 to the AI integrated server 10 each time it is generated. Based on the processing instruction (for example, license number verification instruction) sent from the AI integrated server 10, the processor 91 of the LPR server 90 has the license number information of the verification target included in the processing instruction as the license number list data of the database 92 (for example, the license number list data of the database 92). It is collated whether or not it is registered in (see below), and the collation result is saved in the database 92. Here, the license number list data is data in which the license number information and the information (for example, face image and personal information) about the purchaser (in other words, the owner) of the corresponding vehicle are registered in advance in association with each other. , Is registered in the database 92. The license number list data may be registered in the database 92 or may be registered in another external database (not shown).

The client terminal VW1 as an example of the user terminal is installed in a police station, for example, and is used by an operator (police officer) in the police station, and is configured by using a laptop-type or desktop-type PC (Personal Computer), for example. .. For example, when an incident occurs, the operator listens to various information (witness information) related to the incident by telephone from a person who has notified the police station of the incident (for example, a witness), and the client terminal VW1 Data is input and recorded by operating. The client terminal VW1 sends a search processing request including, for example, a person or vehicle object feature element that matches or is similar to the sighting information as a search condition to the AI integrated server 10, and the AI integrated server 10 sends each server (for example, the face authentication server 50). , The person search server 60, the vehicle search server 80, and the LPR server 90) receive the search results obtained through the search from the AI integrated server 10, and display the search results (see below). Further, when the client terminal VW1 is connected to the video management server 40 via a network in a police station such as a wireless LAN, the client terminal VW1 accesses the video management server 40 to acquire desired captured video data and reproduces and displays it. You can.

The mobile terminal VW2 as an example of the user terminal is installed in a police station, for example, and is used by a police officer who is out on the spot or the like, and is configured by using a computer such as a smartphone or a tablet terminal. The mobile terminal VW2 sends a search processing request including, for example, an object feature element of a person or a vehicle that matches or is similar to the sighting information heard near the site as a search condition to the AI integrated server 10, and the AI integrated server 10 sends each server ( For example, the search results obtained through the search by the face authentication server 50, the person search server 60, the vehicle search server 80, and the LPR server 90) are received from the AI integrated server 10 and the search results (see below) are displayed. Further, when the mobile terminal VW2 is connected to the video management server 40 via a network (not shown) such as a wireless LAN or a mobile phone network, the mobile terminal VW2 accesses the video management server 40 and acquires desired captured video data. And play it back.

The AI integrated server 10 as an integrated server is installed in a police station, for example, and when it receives a processing request for searching for a person or vehicle from a client terminal VW1 or a mobile terminal VW2 (see above), it is necessary for searching the processing request. Identify a good analysis server. The AI integrated server 10 generates and sends a processing instruction corresponding to the specified analysis server (for example, face recognition server 50, person search server 60, vehicle search server 80, LPR server 90). Here, in the investigation support system 1 according to the first embodiment, the manufacturer of each analysis server (specifically, face authentication server 50, person search server 60, behavior detection server 70, vehicle search server 80, LPR server 90). (Manufacturers) may be the same or different.

For example, when the manufacturers of each analysis server (specifically, face authentication server 50, person search server 60, vehicle search server 80, LPR server 90) are all the same, the client terminal VW1 or mobile terminal VW2 can be transferred to each server. It is conceivable that the application screen for requesting the search (for example, the input screen for the search condition or the collation condition) is generated by the common layout unique to the manufacturer. Therefore, the operator or the like can perform a cross-sectional search (AND search) in which a plurality of objects (for example, a person, a vehicle, a face, a license plate) are mixed on one search condition input screen.

However, if the manufacturers of the analysis servers (specifically, the face authentication server 50, the person search server 60, the vehicle search server 80, and the LPR server 90) are not all the same, they are different from the client terminal VW1 or the mobile terminal VW2. An application screen (for example, a search condition input screen) for requesting a search to a manufacturer's analysis server will be generated by a search algorithm, layout, etc. that differ for each manufacturer. That is, when viewed from the client terminal VW1 or the mobile terminal VW2, the matching condition input screen (application) to the face authentication server 50, the search condition input screen (application) to the person search server 60, and the vehicle search server 80 are displayed. The input screen (application) of the search condition of is different from each other. It will drop.

Therefore, in the first embodiment, the AI integrated server 10 is a client even if the manufacturers of the analysis servers (specifically, the face authentication server 50, the person search server 60, the vehicle search server 80, and the LPR server 90) are different. When a search processing request from the terminal VW1 or the mobile terminal VW2 is received, a common interface (IF: Interface) is used for communication (access) to each analysis server that is the destination of the processing request. The interface referred to here is, for example, a common search algorithm in which the object search algorithm used in each server is generalized, and the AI integrated server 10 holds this common search algorithm in advance. The AI integrated server 10 sends a search or collation processing instruction to the corresponding analysis server by using a common search algorithm (interface) for each analysis server. Further, the interface may be, for example, an interface having a common protocol or agreement regarding communication specifications with each analysis server (specifically, face authentication server 50, person search server 60, vehicle search server 80, LPR server 90). It may be a separate interface suitable for communication to each analysis server. The AI integrated server 10 may send and receive data or information (for example, receive an alarm notification) using an interface suitable for communication with each analysis server.

The AI integrated server 10 is composed of, for example, a high-performance server computer, and specifically includes a memory MM1, a processor 11, a database 12, a server IF controller 13, and a client IF controller 14.

The memory MM1 is configured by using, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory), a program necessary for executing the operation of the AI integrated server 10, and data or information generated during the operation. Temporarily save. The RAM is, for example, a work memory used when the processor 11 operates. The ROM stores, for example, a program for controlling the processor 11 in advance. The memory MM1 records road map information indicating the positions where each of the cameras C1 to C20 are installed. For example, every time the road map information is updated due to new construction or maintenance work of the road, etc., the updated road map information is updated. Record road map information.

The processor 11 is configured by using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), functions as a control unit of the AI integrated server 10, and is a control unit of the AI integrated server 10. It performs control processing for overall control of operations, data input / output processing with and from each part of the AI integrated server 10, data arithmetic processing, and data storage processing. The processor 11 operates according to the computer program stored in the memory MM1. This computer program uses, for example, a step of communicating with a terminal (for example, client terminal VW1 or mobile terminal VW2) on the AI integrated server 10 which is a computer device, and captured video data of each of the plurality of cameras C1 to C20. A step of communicating with a plurality of analysis servers (for example, a face authentication server 50, a person search server 60, an action detection server 70, a vehicle search server 80, and an LPR server 90) that analyze images of different objects in relation to an incident, and a plurality of steps. Based on the input from the terminal of different object feature elements, the step of sending the search request of the corresponding object to the analysis server corresponding to the object feature element and the terminal that receives the search result of the corresponding object from each analysis server and integrates them. To realize the steps to be displayed on the server.

For example, when the processor 11 receives a processing request for searching for a person or a vehicle from the client terminal VW1 or the mobile terminal VW2, the processor 11 identifies at least one server necessary for searching the processing request. The processor 11 generates and sends a processing instruction (for example, a search instruction) corresponding to the specified analysis server (for example, face recognition server 50, person search server 60, vehicle search server 80, LPR server 90). Thereby, the processor 11 can search across a person and a vehicle by using, for example, each of a plurality of analysis servers (for example, a person search server 60 and a vehicle search server 80) (cross-search processing). For example, the processor 11 performs a narrow-down search using two objects of a person's feature and a face, or three objects of a vehicle, a person, and a face as a cross-search process on the corresponding analysis server.

For example, when the processor 11 receives the search results from each analysis server, the processor 11 sorts the search results for each object (for example, a person or a vehicle) (search result sorting process). For example, the processor 11 uses an image (for example, a probability value indicating the likelihood of the search result obtained based on the processing of the AI engine) included in the search result from each analysis server. For example, a ranking indicating the degree of conformity of a person thumbnail image, a face thumbnail image, and a vehicle thumbnail image) is determined, and each image is rearranged according to the ranking.

For example, the processor 11 is an analysis server (specifically, a video management server 40, a face authentication server 50, a person search server 60, an action detection server 70, a vehicle search server 80, and an LPR) connected to the AI integrated server 10. A predetermined command is sent to each of the servers 90). The processor 11 monitors the life-and-death state of the analysis server (that is, whether or not a process in the server computer is operating) according to whether or not a command response is received from each analysis server (life-and-death monitoring process).

The database 12 is configured by using, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores data or information acquired or generated by the processor 11.

The server IF controller 13 is located between the AI integrated server 10 and each analysis server (specifically, the face authentication server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90). It is composed of a communication interface circuit that controls communication (transmission and reception). Further, even if the manufacturers of the face recognition server 50, the person search server 60, and the vehicle search server 80 are different, the server IF controller 13 selects and uses an interface suitable for communication (access) to each server. For example, the server IF controller 13 has an input / output interface for the face authentication server 50, an input / output interface for the person search server 60, and an input / output interface for the vehicle search server 80, respectively, from the client terminal VW1 or the mobile terminal VW2. Select and use an interface that meets the search processing requirements of.

The client IF controller 14 is composed of a communication interface circuit that controls communication (transmission / reception) between the client terminal VW1, the mobile terminal VW2, and the video management server 40, respectively. The client IF controller 14 sends the search results sorted by the processor 11 to the client terminal VW1 or the mobile terminal VW2. The client IF controller 14 instructs the video management server 40 to distribute the captured video data to the client terminal VW1 or the mobile terminal VW2, or to record (record) the captured video data of the cameras C1 to C20. Further, the client IF controller 14 transfers the alarm notification from the action detection server 70 to each terminal (specifically, the client terminal VW1 and the mobile terminal VW2). The client IF controller 14 sends alarm notifications from other analysis servers (for example, face authentication server 50, person search server 60, vehicle search server 80, LPR server 90) other than the behavior detection server 70 to each terminal (specifically). May be transferred to each of the client terminal VW1 and the mobile terminal VW2).

FIG. 2 is a block diagram showing a hardware configuration example of the cameras C1 to C20 constituting the investigation support system 1. Since each of the cameras C1 to C20 has the same configuration, the camera C1 will be described below as an example.

The camera C1 includes an imaging unit CK1, a memory CK2, a processor CK3, a communication unit CK4, and a recording unit CK5. The camera C1 is installed (arranged) in various places in the city or a building, for example, and generates captured image data by imaging an imaging area within the angle of view. In addition, the camera C1 can detect the entire frame of various objects (for example, a person, a vehicle) related to an incident or the like by analyzing the captured image data.

The image pickup unit CK1 includes a lens for condensing light and a solid-state image sensor such as a CCD (Charge Coupled Device) type image sensor or a CMOS (Complementary Metal Oxide Semiconductor) type image sensor. While the power of the camera C1 is on, the image pickup unit CK1 constantly outputs the captured image data of the subject obtained based on the image pickup by the solid-state image sensor to the processor CK3. Further, the imaging unit CK1 may include a mechanism for changing the zoom magnification at the time of imaging.

The memory CK2 is configured by using, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory), and includes programs and data necessary for executing the operation of the camera C1, information or data generated during the operation, and the like. Temporarily save. The RAM is, for example, a work memory used during the operation of the processor CK3. The ROM stores, for example, a program and data for controlling the processor CK3 in advance. Further, the memory CK2 stores, for example, identification information (for example, a serial number) of the camera C1 and various setting information.

The processor CK3 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array). The processor CK3 functions as a control unit of the camera C1, and controls processing for overall control of the operation of each part of the camera C1, data input / output processing with each part of the camera C1, and data calculation (calculation). Performs processing and data storage processing. The processor CK3 operates according to the programs and data stored in the memory CK2. The processor CK3 uses the memory CK2 during operation to acquire the current time information, and after performing various known image processing on the captured image data captured by the imaging unit CK1, records it in the recording unit CK5. To do. Although not shown in FIG. 2, when the camera C1 has a GNSS (Global Navigation Satellite System) receiving unit, the current position information is acquired from the GNSS receiving unit, and the position information is further added to the captured image data. It may be recorded in association with each other.

Here, the GNSS receiver will be briefly described. The GNSS receiver receives satellite signals including its own signal transmission time and position coordinates transmitted from a plurality of GNSS transmitters (for example, four navigation satellites). The GNSS receiving unit uses a plurality of satellite signals to calculate the current position coordinates of the camera and the reception time of the satellite signals. Note that this calculation may be executed not by the GNSS receiving unit but by the processor CK3 to which the output from the GNSS receiving unit is input. The reception time information may also be used to correct the system time of the camera. The system time is used, for example, for recording the imaging time of the captured image constituting the captured image.

Further, the processor CK3 may variably control the shooting conditions (for example, zoom magnification) by the imaging unit CK1 according to an external control command received by the communication unit CK4. For example, when an external control command instructs to change the zoom magnification, the processor CK3 changes the zoom magnification at the time of shooting of the imaging unit CK1 specified by the control command according to the control command.

Further, the processor CK3 may detect an object feature element (for example, the face of an occupant) of an object (for example, a vehicle) reflected in the captured image data by analyzing the captured image data. As the analysis result of the captured image data, the processor CK3 generates face area information composed of coordinates indicating the positions of the object feature elements, and transmits the face area information to the analysis server (for example, the LPR server 90) via the communication unit CK4.

Based on the instruction of the processor 12P, the communication unit CK4 sends the captured image data recorded in the recording unit 15 to the vehicle search server 80 and the image management server 40, respectively, via the network NW1 described above. Further, the communication unit CK4 receives the control command of the camera C1 sent from the outside (for example, the vehicle search server 80), and transmits the state information of the camera C1 to the outside (for example, the vehicle search server 80). ..

The recording unit CK5 is configured by using an external storage medium such as a semiconductor memory (for example, a flash memory) built in the camera C1 or a memory card (for example, an SD card) not built in the camera C1. The recording unit 15 records the data of the captured image generated by the processor 12P in association with the identification information of the camera C1 (an example of the camera information) and the information of the imaging date and time. The recording unit 15 constantly prebuffers and holds the captured image data for a predetermined time (for example, 30 seconds), and accumulates the captured image data for a predetermined time (for example, 30 seconds) before the current time while writing the data. Continue to do. When the recording unit 15 is composed of a memory card, it is freely inserted and removed from the housing of the camera C1.

FIG. 3 is a block diagram showing a hardware configuration example of various analysis servers constituting the investigation support system 1. Specifically, the various analysis servers are a video management server 40, a face recognition server 50, a person search server 60, a behavior detection server 70, a vehicle search server 80, and an LPR server 90. Therefore, the analysis server in the description of FIG. 3 is used as a generic term for the video management server 40, the face recognition server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90. In FIG. 3, a detailed description of the hardware configuration common to each analysis server will be given, and details of the characteristic operations of each part will be described with reference to FIG. 1 or later, and thus the description will be simplified here.

The analysis server of FIG. 3 is configured by, for example, a server computer machine, and specifically includes a processor PRC1, a memory MM2, a database 52, a camera IF controller 53, and a server IF controller 54. In particular, the face authentication server 50, the person search server 60, the behavior detection server 70, the vehicle search server 80, and the LPR server 90 are high-performance server computer machines equipped with artificial intelligence (AI), and are AI trained models. Can be used to detect objects that are different from other analysis servers.

The processor PRC1 is configured by using, for example, a GPU (Graphical Processing Unit) or an FPGA, functions as a control unit of an analysis server, controls processing for overall control of the operation of each part of the analysis server, and each part of the analysis server. Performs data input / output processing, data arithmetic processing, and data storage processing. The processor PRC1 operates according to the program stored in the memory MM2. The processor PRC1 of each analysis server (specifically, face authentication server 50, person search server 60, behavior detection server 70, vehicle search server 80, LPR server 90) is suitable for processing in, for example, the corresponding analysis server. The trained model (AI) generated by machine learning can be executed. Each analysis server outputs a processing result and a score (see below) indicating the likelihood (reliability probability) of the processing result by executing the processing using the trained model.

For example, the face recognition server 50 uses the trained model for the face recognition server 50 to detect the face of a person reflected in the captured image data of the cameras C1 to C20, or to give a collation instruction from the AI integrated server 10. A collation process is executed between the included face image and the blacklist data of the database 52 (in other words, a collation list having one or more face images or face thumbnail images for which discovery is requested). The face recognition server 50 outputs a face image or face thumbnail image registered in the blacklist data and a score indicating the likelihood of the face image or face thumbnail image as a processing result of detecting the face of a person.

For example, the person search server 60 uses the trained model for the person search server 60 to show the person reflected in the captured image data of the cameras C1 to C20 and the characteristic elements such as the appearance of the person or the presence or absence of belongings. Database 62 (in other words, a request for discovery is made) is a search process for a person who detects and extracts person information (person attribute information) or satisfies the person search condition included in the collation instruction from the AI integrated server 10. It is executed by referring to a whole body image of one or more people, a thumbnail image of the whole body of a person, or person attribute information). The person search server 60 outputs a whole body image of a person, a thumbnail image of the whole body of the person, person information (person attribute information), and a score indicating the likelihood of the thumbnail image as a processing result of detecting the person and the person attribute information. ..

For example, the behavior detection server 70 detects the presence or absence of a predetermined behavior caused by a person reflected in the captured image data of the cameras C1 to C20 by using the trained model for the behavior detection server 70. The action detection server 70 has a thumbnail image showing the content (result) of a predetermined action determined to have the highest likelihood as the processing result of action detection and the captured image data in which the action is detected, the date and time of imaging, and the camera. Outputs identification information.

For example, the vehicle search server 80 uses the trained model for the vehicle search server 80 to show the vehicle reflected in the captured image data of the cameras C1 to C20, and the vehicle information indicating characteristic elements such as the vehicle type and vehicle color of the vehicle. Database 82 (in other words, one vehicle for which discovery is requested) is a search process for vehicles that detect and extract (vehicle attribute information) or satisfy the vehicle search conditions included in the collation instruction from the AI integrated server 10. It is executed by referring to the above-mentioned whole image of the vehicle, thumbnail image of the whole vehicle, or vehicle attribute information). The vehicle search server 80 outputs the entire image of the vehicle, the thumbnail image of the entire vehicle, the vehicle information (vehicle attribute information), and the score indicating the likelihood of the thumbnail image as the processing result of detecting the vehicle and the vehicle attribute information. To do.

For example, the LPR server 90 uses the trained model for the LPR server 90 to detect and extract the license number information of the license plate displayed in the captured video data of the cameras C1 to C20, or from the AI integrated server 10. The collation process of collating the license number information included in the collation instruction of the database 92 with the license number list data of the database 92 is executed. The LPR server 90 outputs the license number information registered in the license number list data, the face image of the vehicle purchaser (owner) corresponding to the license number information, and personal information as the processing result of detecting the license number information. ..

The memory MM2 is configured by using, for example, a RAM and a ROM, and temporarily stores a program necessary for executing the operation of the analysis server, and data or information generated during the operation. The RAM is, for example, a work memory used during the operation of the processor PRC1. The ROM stores, for example, a program for controlling the processor PRC1 in advance.

The database 52 is configured by using, for example, an HDD or an SSD, and stores data or information acquired or generated by the processor PRC1 of the analysis server. The data generated by the processor PRC1 was obtained, for example, as a result of face image matching processing when the analysis server was the face authentication server 50, and as a result of the search processing when the analysis server was the person search server 60. Person attribute information (see above), vehicle attribute information obtained as a result of the search process when the analysis server is the vehicle search server 80 (see above), search processing when the analysis server is the LPR server 90. This is the license number information obtained as a result.

The camera IF controller 53 is composed of a communication interface circuit that controls communication (transmission / reception) between the analysis server and each of the cameras C1 to C20. The camera IF controller 53 receives the captured video data captured by each of the cameras C1 to C20 and outputs the captured video data to the processor PRC1.

The server IF controller 54 is composed of a communication interface circuit that controls communication (transmission / reception) between the analysis server and the AI integrated server 10. The server IF controller 54 receives the processing instruction from the AI integrated server 10 and responds (returns) the processing result of the processor PRC1 based on the processing instruction to the AI integrated server 10. Further, the server IF controller 54 transmits an alarm notification (see above) corresponding to the object detected by the processor PRC1 of each analysis server to the AI integrated server 10.

FIG. 4 is a block diagram showing a hardware configuration example of the client terminal VW1 and the mobile terminal VW2 constituting the investigation support system 1. In FIG. 4, while exemplifying the client terminal VW1, only the elements common to the hardware configurations of the client terminal VW1 and the mobile terminal VW2 are shown, and the illustration of the configuration peculiar to the client terminal VW1 or the mobile terminal VW2 is omitted. There is. In FIG. 4, a detailed description of the hardware configuration will be given by exemplifying the client terminal VW1.

The client terminal VW1 of FIG. 4 is configured by, for example, a computer machine, and specifically includes a processor PRC2, a memory MM3, a recording device 112, a server IF controller 114, an input device 115, a display device 116, and a speaker 117.

The processor PRC2 is configured by using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA, functions as a control unit of the client terminal VW1, and controls the operation of each part of the client terminal VW1 as a whole. Control processing, data input / output processing with each part of the client terminal VW1, data arithmetic processing, and data storage processing. The processor PRC2 operates according to the program stored in the memory MM3.

The memory MM3 is configured by using, for example, a RAM and a ROM, and temporarily stores a program necessary for executing the operation of the client terminal VW1 and data or information generated during the operation. The RAM is, for example, a work memory used during the operation of the processor PRC2. The ROM stores, for example, a program for controlling the processor PRC2 in advance. The memory MM3 records road map information indicating the position where each of the cameras C1 to C20 is installed. For example, every time the road map information is updated due to new construction or maintenance work of the road, etc., the updated road map information is updated. Record road map information.

The recording device 112 is configured by using, for example, an HDD or an SSD, and stores data or information acquired or generated by the processor PRC2 of the client terminal VW1. The recording device 112 stores data of various search results sent from the AI integrated server 10.

The server IF controller 114 is composed of a communication interface circuit that controls communication (transmission / reception) between the client terminal VW1 and the AI integrated server 10. The server IF controller 114 sends a search processing request generated by the processor PRC2 to the AI integrated server 10. Further, the server IF controller 114 receives various search results (processing results) or alarm notifications (see above) sent from the AI integrated server 10.

The input device 115 accepts the operation of an operator in the police station (for example, a user of the client terminal VW1) or a police officer who is out on the scene (for example, a user of the mobile terminal VW2). The input device 115 is composed of, for example, a mouse, a keyboard, and a touch panel.

The display device 116 is composed of, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and displays various data sent from the processor PRC2.

The speaker 117 acoustically outputs the sound when the data (for example, the video data included in the search result from the AI integrated server 10) is reproduced by the processor PRC2.

Next, in a police investigation using the investigation support system 1 according to the first embodiment, an example of a screen displayed on the display device 116 of the client terminal VW1 will be described with reference to FIG. FIG. 5 is a diagram showing an example of an alarm notification screen. The alarm notification screen displayed in FIG. 5 may be displayed on the mobile terminal VW2, but the client terminal VW1 will be described as an example in order to make the description uniform.

In a police investigation, the client terminal VW1 or the mobile terminal VW2 activates and executes a pre-installed investigation support application (hereinafter referred to as "investigation support application") by the operation of an operator or a police officer. The investigation support application is stored in the ROM of the memory MM3 of the client terminal VW1 or the mobile terminal VW2, for example, and is executed by the processor PRC2 when activated by the operation of an operator or a police officer. In other words, the investigation support application as the operating subject in the following description can be read as the processor PRC2. The data or information generated by the processor PRC2 during the activation of the investigation support application is temporarily held in the RAM of the memory MM3.

In the first embodiment, when the investigation support application receives the notification of the event related to the incident or the like from the AI integrated server 10 on the client terminal VW1, the alarm notification screen WD1 shown in FIG. 5 is displayed on the display device 116.

The alarm notification screen WD1 shown in FIG. 5 is a display area of the input field SCON1 for search conditions (for example, object feature elements (for example, person attribute information, vehicle attribute information)) of various objects (for example, a person, a vehicle) related to an incident or the like. , The display area of the list window AMLT1 of the event related to the incident or the like sent from the AI integrated server 10. The search condition input field SCON1 is used as standard by the investigation support application.

In the search condition input field SCON1, each of the date and time (Time & Date), camera (Camera), event (Event), reset icon RSE1, and search execution icon SCR1 where the object to be searched should be detected can be input or selected. , Each display area is provided.

In the date / time (Time & Date) display area, the date / time start input field FR1 and the date / time end input field TO1 are arranged.

The date and time start input field FR1 is input by a user operation as a start date and time when the captured image data to be searched for an object (for example, a person such as a suspect or a vehicle such as an escape vehicle) is captured. In the date and time start input field FR1, for example, the date and time when the incident or the like occurred or a date and time slightly earlier than that date and time is input. When input by the user operation, the investigation support application sets the date and time input in the date and time start input field FR1 as the search condition (for example, the start date and time) as the search condition.

The date and time end input field TO1 is input by the operator's operation as the end date and time when the captured image data to be searched for the object (for example, a person such as a suspect or a vehicle such as an escape vehicle) is captured. In the date and time end input field TO1, for example, a predetermined period (for example, a date and time slightly after the occurrence date and time of an incident or the like) is input from the date and time entered in the date and time start input field. When input by the operator's operation, the investigation support application sets the date and time entered in the date and time end input field TO1 as the search condition (for example, the end date and time) as the search condition.

In the display area of the camera (Camera), a camera selection icon SCL1 for prompting the selection of the camera to be searched is arranged.

The camera selection icon SCL1 displays a list of cameras that have captured the captured image data to be searched for an object (for example, a person such as a suspect, a vehicle such as an escape vehicle), and prompts the operator to select. When at least one camera is selected by the operation of the operator, the investigation support application sets the identification information of the selected camera as a search condition.

In the event display area, a check box EVE1 indicating the type of event to be displayed is arranged in the list window AMLT1 of the alarm notification screen WD1.

In the example of FIG. 5, three of "Face matching", "People fighting", and "LPR matching" are selected by the operation of the operator. “Face matching” indicates, for example, an event in which a face that is the same as or similar to a face registered in blacklist data (an example of a collation list) is detected based on video analysis of camera captured video data (live video). .. “People fighting” indicates, for example, an event in which two people are in a fight, which is detected based on the image analysis of the image data (live image) captured by the camera. "LPR matching" is based on video analysis of camera image data (live video) of a vehicle having the same license number as the license number of the license number information registered in blacklist data (an example of a collation list), for example. Indicates the detected event.

The reset icon RSE1 instructs the reset of the search condition input by the operator's operation. When the investigation support application detects that the reset icon RSE1 is pressed after the search condition is input by the operator, for example, the investigation support application resets the input search condition (that is, deletes all and returns to the initial state).

The search execution icon SCR1 requests execution of a search satisfying the search conditions input by the operator's operation. When the search support application detects that the search execution icon SCR1 is pressed after the search condition is input by the operation of the operator, for example, the investigation support application generates a search request satisfying the input search condition and sends it to the AI integrated server 10.

The event list window AMLT1 has content data of the date and time when the event was detected (Date / Time), the camera (Camera), the event (Event), the detected image (Detected), the registered image (Registered), and the details (Data). Is displayed for each event. In the example of FIG. 5, the content data of each of the above-mentioned items for each of the three events of "Face matching", "Unknown driver", and "People fighting" is illustrated. The content data of "Face matching" and "People fighting" are shown in the default simplified style, and the content data of "Unknown driver" is shown in the detailed style DTL1 by the operation of the operator.

For example, the event "Face matching" was detected at 10:12:33 am on June 13, 2019, based on the video analysis of the captured video data of a camera (for example, camera C1) placed in the "North Entrance". .. A "Detected" face image that is the same as or similar to the "Registered" face image was detected. In the details column, for example, information such as the similarity of the face image, the name, age, gender, and memo of the person who is the target of the event "Face matching" is displayed.

For example, the event "Unknown driver" was detected based on the video analysis of the captured video data of the camera (see FIG. 2) placed in the "Parking lot ABC" at 10:00:13 am on June 13, 2019. It was. Further, at the time of detecting this event, the license number information of the license plate LPT1 and the face image FCEN1 of the occupant (for example, the driver) are registered when the vehicle VCL1 is stored in the "Parking lot ABC". This event "Unknown driver" was detected when the face image FCGO1 of a different occupant (for example, driver) was detected in the vehicle VCL1 having the same license number information as at the time of warehousing at the time of leaving "Parking lot ABC". become. The event "Unknown driver" may be an event belonging to a lower group of the event "LPR matching", and if "LPR matching" is specified by the check box EVE1, the notification of the AI integrated server 10 may be received.

The investigation support application displays the content data of the "Unknown driver" in the list window AMLT1 in the detailed style DTL1. Specifically, the investigation support application puts the face image FCEN1 of the occupant of the vehicle VCL1 having the license plate LPT1 stored in the "Parking lot ABC" at the time of warehousing (9:06 pm on June 12, 2019). It is displayed in the list window AMLT1 that it has been registered in (minutes and 0 seconds). In addition, the investigation support application includes a plurality of playback windows PLY1 capable of reproducing captured image data of the state of the vehicle VCL1 at the time of warehousing in the "Parking lot ABC", and a camera (see FIG. 2) that images the warehousing of the vehicle VCL1. The map data MP1 on which the camera arrangement location and the thumbnail image THM1 of the vehicle VCL1 are superimposed is displayed in the list window AMLT1. Indicators IC1, IC2, and IC3 indicating locations where a plurality of cameras are arranged are identifiablely displayed on the map data MP1. In addition, the investigation support application displays in the list window AMLT1 that it has detected different face images FCGO1 of the occupants of the vehicle VCL1 having the same license plate LPT1 (Detected) issued from "Parking lot ABC".

That is, the client terminal VW1 receives the notification of the event "Unknown driver" and displays it on the list window AMLT1 so that the operator or the police officer can have the same license number information at the time of entering and leaving the parking lot. It is possible to notice that the occupant's face image is different (in other words, the occupant was replaced while parking). Therefore, the operator or the police officer can grasp from the event notification from the AI integrated server 10 that the suspect in the case or the like has disappeared or suddenly appeared in the parking lot, and the investigation efficiency can be improved.

Next, an example of the operation procedure of the investigation support system 1 according to the first embodiment will be described with reference to FIG. FIG. 6 is a sequence diagram showing an example of an operation procedure of the investigation support system 1 according to the first embodiment. In FIG. 6, the flow of processing from the detection of the event “Unknown driver” to the notification to the client terminal VW1 is explained in chronological order, the camera that detects the arrival of the vehicle is the “camera α” (see FIG. 2), and the vehicle leaves the vehicle. The camera that detects the above is defined as "camera β" (see FIG. 2). The camera α and the camera β may be the same.

In FIG. 6, the camera α detects and discriminates the face area of a vehicle occupant who intends to enter a parking lot (for example, “Parking lot ABC”) by analyzing the captured image data obtained by his / her own imaging. (St1). The camera α sends a pair of the captured image used for the detection in step St1 and the face area information of the occupant to the LPR server 90 (St2). The LPR server 90 analyzes the captured image sent from the camera α in step St2 (St3), cuts out the occupant's face portion reflected in the captured image to generate a face image, and generates the face image J2 and the vehicle license number J1. The pair with the information is sent to the AI integrated server 10 (St4). The AI integrated server 10 registers the pair of the face image J2 sent from the LPR server 90 and the vehicle license number J1 information in the database 12 in association with the time of the detection timing of the vehicle warehousing by the camera α (St5).

The camera β detects and discriminates the face area of the occupant of the vehicle trying to leave the parking lot (for example, “Parking lot ABC”) by analyzing the captured image data obtained by its own imaging (St6). .. The camera α and the camera β may be different or the same, and if they are the same, the time in step St1 and the time in step St6 are different. The camera β sends a pair of the captured image used for the detection in step St6 and the face area information of the occupant to the LPR server 90 (St7). The LPR server 90 analyzes the captured image sent from the camera β in step St7 (St8), cuts out the face portion of the occupant reflected in the captured image to generate a face image, and generates the face image J4 and the vehicle license number J3. The pair with the information is sent to the AI integrated server 10 (St9). To simplify the explanation of FIG. 6, the license number J1 and the license number J3 are the same.

When the AI integrated server 10 receives the pair of the face image J4 sent from the LPR server 90 and the vehicle license number J3 (= J1) information, the license number J3 (= J1) is already registered in the database 12. Whether or not it is determined (St10). When the AI integrated server 10 determines that the license number J3 (= J1) is not registered in the database 12 (St10, NO), the process of the investigation support system 1 according to the first embodiment ends.

On the other hand, when the AI integrated server 10 determines that the license number J3 (= J1) has been registered in the database 12 (St10, YES), the AI integrated server 10 reads the face image J2 corresponding to the license number J3 information from the database 12 and reads the face image J2. A collation request including the face image J2 and the face image J4 is generated and sent to the face recognition server 50 (St11).

The face recognition server 50 determines whether or not the face image J2 and the face image J4 are the same or similar according to the collation request from the AI integrated server 10 (St12). When the face authentication server 50 determines that the face image J2 and the face image J4 are the same or similar (St12, YES), the face image J2 and the face image J4 match (in other words, the person and the face of the face image J2). A report to the effect that it is the same as the person in image J4) is generated and sent to the AI integrated server 10 (St13). Upon receiving the report from the face recognition server 50, the AI integrated server 10 determines that there is no need to notify the event "Unknown driver" because no incidental event has occurred according to the report, and determines that the event "Unknown driver" does not need to be notified. The notification to the client terminal VW1 of "driver" is omitted (St14).

On the other hand, when the face authentication server 50 determines that the face image J2 and the face image J4 are dissimilar (St12, NO), the face image J2 and the face image J4 do not match (in other words, the face image J4). A report to the effect that the person in J2 and the person in the face image J4 are different persons) is generated and sent to the AI integrated server 10 (St15). When the AI integrated server 10 receives a report from the face authentication server 50, it is necessary to notify the event "Unknown driver" that the occupants (for example, the driver) are replaced during parking and an incidental event occurs according to the report. It is determined that there is, and the event "Unknown driver" including the license number J1 (= J3) information, the face image J2 and the face image J4 is notified to the client terminal VW1 (St16). The client terminal VW1 displays the alarm notification screen WD1 shown in FIG. 5 based on the notification from the AI integrated server 10 (St17).

In addition, as a background to the first embodiment, in Patent Document 1, in view of the recent fact that the forms of cases and the like are diversified, the suspect or the suspect who caused the case or the like uses it for escape. Searching for escape vehicles is not considered. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the suspect or the escape vehicle in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage. In addition, it is required to improve convenience by simplifying the user operation when inputting search conditions.

Therefore, in the investigation support system 1 according to the first embodiment, an example of a UI (User Interface) for further simplifying the user operation when searching for a suspect or an escape vehicle will be described with reference to FIG. 7. FIG. 7 is a diagram showing a display example of a part of the search condition input screen WD2. The investigation support application displays the search condition input screen WD2 on the display device 116 by the operation of a user (for example, a police station operator or a police officer at the scene). Note that FIG. 7 shows an excerpt of the characteristic contents of the search condition input screen WD2 for realizing a simpler user interface.

The search condition input screen WD2 can be displayed by (T1) when the person search server 60 is selected, (T3) when the person search server 60 and the face recognition server 50 are selected, or (T4) by user operation starting from the initial state. The display content transitions to either the vehicle search server 80 or the LPR server 90 when they are selected.

(T1) Initial state search condition input screen WD2 displays input fields for search conditions (for example, object feature elements (for example, person attribute information, vehicle attribute information)) of various objects (for example, person, vehicle) related to an incident or the like. Includes area. In the display area of the input field of the search condition, each of the date and time (Time & Date), the search filter (Search filter), the save (Save), the reset (Reset), and the search (Search) where the object to be searched should be detected. Icons are arranged so that they can be entered or selected. In the description of FIG. 7, the same reference numerals are given to the components overlapping with the description of FIG. 5, and the description is simplified or omitted, and different contents will be described. (T1) In the initial state, since the search condition has not been input yet, the save, reset, and search icons of the search condition input screen WD2 are all set to be non-pressable by the investigation support application.

The search filter SFL1 shows a list of various objects related to an incident or the like that can be searched from the client terminal VW1 in the sub-window FLT1 and prompts the user to perform a selection operation. The list of various objects includes, for example, "People", "Face", "Vehicle", "License Plates". In other words, the search filter SFL1 of FIG. 7 allows the user to know that he / she can specify a search for a person, face, vehicle, or license plate using a cursor that can be moved with a finger FG, a mouse, or the like.

When the investigation support application detects that "People (person)" in the sub-window FLT1 of the search filter SFL1 is selected by a user operation (for example, finger FG), for example, in the search condition input screen WD2, the person search condition field PSC1 Is displayed (see T2).

(T2) The search condition input screen WD2 when the person search server 60 is selected is (T1) investigation support that "People (person)" is selected by a cursor (not shown) such as a finger FG or a mouse from the initial state. If detected by the app, it will be displayed by the investigation support app. When a person (People) is selected, for example, a person search condition field PSC1 indicating the search condition PCH1 used in the previous person search is displayed. In addition, instead of the search condition used in the previous person search, the person search condition field PSC1 indicating the search condition newly input by the user operation may be displayed. The search condition PCH1 indicates, for example, all four object feature elements "male", "yellow clothes", "black trousers", and "no belongings". In addition, the adjustment bar SBR1 that can adjust the similarity of the search and the display icon CHDTL1 of the detailed search condition detail screen (see FIG. 12) of the object feature element (for example, the appearance feature element of the person) are also the person. It may be displayed in the search condition field PSC1.

When the investigation support application detects that "Face" in the sub-window FLT1 of the search filter SFL1 is selected by a user operation (for example, finger FG) after inputting the search condition for the person search, for example, the search condition input screen. The face search condition field PSC2 is displayed below the person search condition field PSC1 in WD2 (see T3).

(T3) The search condition input screen WD2 when the person search server 60 and the face recognition server 50 are selected is displayed on the (T2) person search server 60 with a cursor (not shown) such as a finger FG or a mouse. When the investigation support application detects that ")" is selected, it is displayed by the investigation support application. When the face is selected, for example, the face search condition field PSC2 indicating the upload icon UPL1 of the face image to be searched is displayed. In addition, the adjustment bar SBR2 capable of adjusting the similarity of the search may also be displayed in the face search condition column PSC2.

When the investigation support application detects that "Vehicle (vehicle)" in the sub-window FLT1 of the search filter SFL1 is selected by a user operation (for example, finger FG) in the initial state (T1), for example, the vehicle search condition column VSC1 is displayed. indicate. Further, the investigation support application uses "License Plates" in the sub-window FLT1 of the search filter SFL1 by user operation (for example, finger FG) after the vehicle search condition is input in the vehicle search condition field VSC1. When it is detected that is selected, for example, the license plate search condition field VSC2 is displayed.

(T4) The search condition input screen WD2 when the vehicle search server 80 and the LPR server 90 are selected is displayed as “License Plates” with a cursor (not shown) such as a finger FG or a mouse from the time when the vehicle search server 80 is selected. When the investigation support application detects that is selected, the investigation support application displays it. When a vehicle is selected, for example, a vehicle search condition column VSC1 indicating the search conditions used in the previous vehicle search is displayed. In addition, instead of the search condition used in the previous vehicle search, the vehicle search condition field VSC1 indicating the search condition newly input by the user operation may be displayed. The search condition indicates, for example, all four object feature elements "vehicle type = sedan", "vehicle type = van", "vehicle type = SUV", and "vehicle color = white". In addition, an adjustment bar in which the degree of similarity of the search can be adjusted may also be displayed in the vehicle search condition column VSC1.

When the license plate (License Plates) is selected, one of the four types of license number search condition fields VSC21, VSC22, VSC23, and VSC24 shown in FIG. 7 is displayed. Each of the license number search condition fields VSC21, VSC22, VSC23, and VSC24 has four check boxes (specifically, "All", "Plates", "Groups", and "Search").

In the license number search condition field VSC21, the check box "All" is selected by user operation when all license numbers are to be searched. The investigation support application includes all license numbers as search conditions and generates them.

In the license number search condition field VSC22, the check box "Plates" is selected by the user operation when a specific license number specified by the user operation is to be searched. In addition to a specific license number, it is also possible to specify the country or state of the license number. The investigation support application generates, for example, including the license number of "BFP8526" as a search condition.

In the license number search condition field VSC23, the check box "Groups" is selected by the user operation when the license number of a specific group specified by the user operation is to be searched. The investigation support application generates, for example, the license number of the vehicle shown in the "Video recorder" (that is, all the captured image data recorded in the image management server 40) as a search condition.

In the license number search condition field VSC24, the check box "Search" is selected by the user operation when an arbitrary license number specified by the user operation is to be searched. The investigation support application generates, for example, a license number of "B%" using the wildcard symbol "%" (that is, a license number including "B") as a search condition.

Next, another operation procedure example of the investigation support system 1 according to the first embodiment will be described with reference to FIG. FIG. 8 is a sequence diagram showing another operation procedure example of the investigation support system 1 according to the first embodiment. In FIG. 8, the flow of processing leading to the search for the vehicle and license number and the notification to the client terminal VW1 will be described in chronological order. In FIG. 8, for simplification of explanation, an OR search for individually searching each of a plurality of objects is illustrated, but it can also be applied to an AND search in which each of a plurality of objects is satisfied at the same time.

In FIG. 8, the client terminal VW1 generates an OR search (see FIG. 7) search request including a vehicle search condition that characterizes the vehicle and a license number search condition that characterizes the license number by the operation of the operator, and the AI integrated server 10 Send to (St101). Here, the vehicle search condition is, for example, that the vehicle type of the vehicle is an SUV (sport utility vehicle). The license number search condition includes, for example, the character "B".

When the AI integrated server 10 receives the OR search search request from the client terminal VW1, the AI integrated server 10 first extracts the vehicle search condition from the search request sent in step St101 and sends the vehicle search request for the vehicle satisfying the vehicle search condition to the vehicle search server 80. Send to (St102).

The vehicle search server 80 refers to the database 82 and executes a vehicle search process satisfying the vehicle search condition based on the search request from the AI integrated server 10 (St103). When the person search server 60 extracts a vehicle that matches the characteristics of the vehicle search condition (St103, match), the person search server 60 returns the processing result including the thumbnail image of the corresponding vehicle to the AI integrated server 10 (St104). On the other hand, when the vehicle search server 80 cannot extract a vehicle that matches the characteristics of the vehicle search condition (St103, mismatch), the vehicle search server 80 returns a processing result indicating that there is no corresponding vehicle information to the AI integrated server 10 (St105).

Following step St104 or step St105, the AI integrated server 10 extracts the license number search condition from the search request sent in step St101 and sends the search request for the license number satisfying the license number search condition to the LPR server 90 ( St106).

Based on the search request from the AI integrated server 10, the LPR server 90 refers to the database 92 and executes a license number search process that satisfies the license number search condition (St107). When the LPR server 90 extracts a license number that matches the characteristics of the license number search condition (St107, match), the LPR server 90 displays a thumbnail image of the license number and information about the purchaser of the vehicle corresponding to the license number (see above). The including processing result is returned to the AI integrated server 10 (St108). On the other hand, when the LPR server 90 cannot extract the license number that matches the characteristics of the license number search condition (St107, mismatch), the LPR server 90 returns the processing result indicating that the corresponding license number information does not exist to the AI integrated server 10 (St109). ).

The AI integrated server 10 integrates (aggregates) the vehicle search processing result by the vehicle search server 80 and the license number search processing result by the LPR server 90, including the result that the corresponding vehicle or the corresponding license number does not exist, and the client. It is returned to the terminal VW1 (St110). The client terminal VW1 generates a search result screen (not shown) showing the processing result of the search returned in step St110 and displays it on the display device 116.

Although FIG. 8 has described an example of an OR search for a vehicle search and a license number search using the vehicle search condition and the license number search condition, the AI integrated server 10 performs an AND search for a person search and a vehicle search. , Do as follows. For example, the AI integrated server 10 sets the vehicle search processing result sent in step St104 with the subsequent license number search population, and the processing result satisfying the license number search condition in the vehicle search processing result. Is searched by the LPR server 90.

As described above, in the investigation support system 1 according to the first embodiment, the analysis server and the AI integrated server 10 are communicably connected to each other. When the analysis server receives the first captured image of the vehicle captured at the first imaging timing, it analyzes the face of the occupant of the vehicle and the license plate, and includes the occupant's face image J2 and the license number J1 information. The first analysis result is sent to the AI integrated server. The AI integrated server 10 registers the first analysis result in the database 12 in association with the first imaging timing. When the analysis server receives the second captured image of the vehicle captured at the second imaging timing, it analyzes the vehicle occupant's face and license plate, and includes the occupant's face image J4 and license number J3 information. The second analysis result is sent to the AI integrated server 10. The AI integrated server 10 matches the license number J1 information included in the first analysis result and the license number J3 information included in the second analysis result with the occupant's face image J2 included in the first analysis result. When it is determined that the face image J4 of the occupant included in the second analysis result is different, an alarm notification regarding the incident or the like is given.

As a result, the investigation support system 1 can quickly and efficiently support the identification of the suspect who caused the incident or the escape vehicle used by the suspect for escape, and the convenience of the investigation by the police or other investigative agency. Can be improved. Therefore, the user of the client terminal VW1 has the same license number information, but the occupant's face image is different at different imaging timings (for example, at the time of entering and leaving the parking lot) (in other words, the occupant during parking). Can be reminded that (has been replaced). Therefore, the operator or the police officer can grasp from the event notification from the AI integrated server 10 that the suspect in the case or the like has disappeared or suddenly appeared in the parking lot, and the investigation efficiency can be improved.

Further, the camera that captured the first captured image at the first imaging timing and the camera that captured the second captured image at the second imaging timing are the same. As a result, the first captured image and the second captured image captured at different imaging timings are the same, so that the situation is monitored by the same camera at the time of warehousing and at the time of warehousing, for example. Even in a large-scale parking lot, the operator or the police officer can easily grasp that the occupants such as the driver have been replaced (for example, the suspect in the case has disappeared or suddenly appeared in the parking lot).

Further, the camera that captured the first captured image at the first imaging timing and the camera that captured the second captured image at the second imaging timing are different. As a result, the first captured image and the camera that captured the second captured image at different imaging timings are different. Therefore, for example, a medium-scale or medium-sized image in which the state is monitored by different cameras at the time of warehousing and at the time of warehousing. Even in a large-scale parking lot, the operator or the police officer can easily grasp that the occupants such as the driver have been replaced (for example, the suspect in the case has disappeared or suddenly appeared in the parking lot).

In addition, the investigation support system 1 further includes a user terminal that is communicably connected to the AI integrated server 10. Based on the alarm notification, the user terminal has an alarm notification screen WD1 that includes at least the license number J1 information, the occupant's face image J2 included in the first analysis result, and the occupant's face image J4 included in the second analysis result. Is displayed. As a result, the user of the client terminal VW or the mobile terminal VW2 can easily grasp the existence of an event related to an incident such as "Unknown driver" without noticing it, so that, for example, the appearance of a suspect who is the target of tracking appears. You can grasp the location etc. and go to the site in a hurry, improving the efficiency of the investigation.

Further, the user terminal includes the map data MP1 on which the arrangement position of the camera that captured the first captured image is superimposed on the alarm notification screen WD1 and displays it. As a result, the user of the client terminal VW or the mobile terminal VW2 can quickly check the map data MP1 around the appearance location of the suspect that he / she is tracking on the alarm notification screen WD1 and can rush to the site. It is possible to easily predict the destination of the vehicle on which the suspect is riding.

Further, the investigation support system 1 according to the first embodiment includes an AI integrated server 10 that is communicably connected between a user terminal and a plurality of analysis servers. The user terminal displays an analysis server selection screen (see the sub-window FLT1 of the search filter SFL1) showing a list of a plurality of analysis servers provided for each of a plurality of different object feature elements related to an incident or the like. The user terminal selects a first analysis server (for example, a person search server 60) from the analysis server selection screen, and an object feature element (for example, of a person) of an object (for example, a person) searched by the selected first analysis server. Based on the input of the search condition including the external feature element), the search request of the corresponding object satisfying the search condition is sent to the AI integrated server 10. Based on the search request, the AI integrated server 10 sends a search instruction for the corresponding object to the first analysis server, receives the search result from the first analysis server, and sends it to the user terminal.

As a result, the investigation support system 1 has a plurality of analysis servers (for example, face recognition server 50, person search server 60, vehicle search server 80, LPR server 90) that can independently search for each of the object feature elements of a plurality of different objects. ) Can be presented on the user terminal. Therefore, it is possible to further simplify the user operation when searching for a suspect or an escape vehicle by a user of the client terminal VW1 or the mobile terminal VW2 (for example, an operator of a police station or a police officer who is out of the field).

Further, the user terminal selects the second analysis server from the analysis server selection screen after selecting the first analysis server, and is an object that can be searched by each of the selected first analysis server and the second analysis server. Based on the input of the search condition including the object feature element of, the search request of the corresponding object satisfying the search condition is sent to the AI integrated server 10. The AI integrated server 10 sends a search instruction for the corresponding object to each of the first analysis server and the second analysis server based on the search request, and searches from each of the first analysis server and the second analysis server. Receive the result and send it to the user terminal. As a result, the investigation support system 1 can easily input the search condition including the object feature element of the object corresponding to each of the plurality of analysis servers, and the convenience of the user can be further improved.

(Background to Embodiment 2)
In Patent Document 1, in view of the recent fact that the forms of cases and the like are diversifying, it is considered to search for the suspect who caused the case or the escape vehicle used by the suspect for escape. Not. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the suspect or the escape vehicle in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage. Also, when searching for multiple different objects related to an incident, etc., the analysis server that can search for each object is not always in a communicable state, so if there is an analysis server that is in a non-communication state, Visualization of its existence may also be required.

(Embodiment 2)
Therefore, in the second embodiment, a user interface for user-friendly search conditions at the time of search is provided, and the escape vehicle used for escape by the suspect who caused the incident or the like is quickly and efficiently identified. Then, an example of an investigation support system that improves the convenience of investigation by an investigative agency such as the police will be described. Since the configuration of the investigation support system according to the second embodiment is the same as that of the investigation support system 1 according to the first embodiment, the description of the same contents is omitted, and the same reference numerals are given to the contents different from the first embodiment. It will be explained with reference to it.

FIG. 9 is a diagram showing an example of a search result screen for notifying the occurrence of a connection error with the vehicle search server 80. In the search result screen WD3 of FIG. 9, for example, the date and time from 15:00:00 on June 13, 2019 to 15:00:00 on June 14, 2019 are input as the search condition. In addition, since four object feature elements "male", "yellow clothes", "black trousers", and "no belongings" are entered as search conditions, a person who includes these four object feature elements by the investigation support application. The search condition field PSC1 is displayed. In addition, four object feature elements "vehicle type = sedan", "vehicle type = van", "vehicle type = SUV", and "vehicle color = white" were entered as search conditions, and these four objects were entered by the investigation support application. The vehicle search condition column VSC3 including the feature element is displayed.

However, although the details will be described with reference to FIG. 11, after the search conditions are sent from the client terminal VW1 (hereinafter, mobile terminal VW2 is also possible) to the AI integrated server 10, the AI integrated server 10 and the vehicle search server 80 It is assumed that a communication interruption occurs between the two. In this case, the investigation support application displays the communication error message ERR1 indicating the connection error with the vehicle search server 80 on the search result screen WD3, and the vehicle that satisfies the search condition of the vehicle search condition column VSC3 cannot be searched. The communication disconnection icon ERR0 indicating that is displayed in the vehicle search condition field VSC3.

On the other hand, since the communication is not interrupted between the AI integrated server 10 and the person search server 60, the number of applicable objects (for example, 517) and the search target of the objects satisfying the search condition of the person search condition field PSC1 are obtained. The search result PSRST1 including the date and time is sent from the person search server 60 to the AI integrated server 10. Therefore, the search result PSRST1 is also displayed on the search result screen WD3 of FIG. 9 by the investigation support application.

FIG. 10 is a diagram showing an example of a search result screen for notifying the return of the connection with the vehicle search server 80. In the description of FIG. 10, the description of the elements that overlap with the contents of FIG. 9 will be omitted, and different contents will be described.

Details will be described with reference to FIG. 11, but after the communication error message ERR1 shown in FIG. 9 is displayed on the search result screen WD3 of the client terminal VW1, communication is performed between the AI integrated server 10 and the vehicle search server 80. It is assumed that the state has returned to the possible state. In this case, the investigation support application displays the communication return message STB1 indicating that the connection with the vehicle search server 80 has been restored on the search result screen WD3, and hides the communication disconnection icon ERR0 from the vehicle search condition column VSC3. (In other words, the communication disconnection icon ERR0 disappears).

Next, an example of the operation procedure of the investigation support system 1 according to the second embodiment will be described with reference to FIG. FIG. 11 is a sequence diagram showing an example of an operation procedure of the investigation support system 1 according to the second embodiment. In FIG. 11, when the search conditions for AND search that simultaneously satisfy both the search conditions of the person search condition field PSC1 and the vehicle search condition field VSC3 of FIG. 9 are input, the person search server 60 and the vehicle search are performed from the AI integrated server 10. The flow of processing for instructing a search request to each of the servers 80 will be described in chronological order.

In FIG. 11, the AI integrated server 10 periodically performs life-and-death monitoring with various analysis servers (for example, face recognition server 50, person search server 60, vehicle search server 80, and LPR server 90) (for example, face recognition server 50, person search server 60, vehicle search server 80, and LPR server 90). St21). Various analysis servers (for example, face recognition server 50, person search server 60, vehicle search server 80, LPR server 90) send a response from the AI integrated server 10 (in other words, a message indicating that the communication status is good). Transmit (St22). For example, after step St22, some kind of communication failure has occurred between the AI integrated server 10 and the vehicle search server 80, and a communication disconnection (connection disconnection) between the AI integrated server 10 and the vehicle search server 80 has occurred. (St23). That is, at this point, the communication disabled period between the AI integrated server 10 and the vehicle search server 80 starts.

The client terminal VW1 generates an AND search (see FIG. 9) search request including a person search condition that characterizes a person and a vehicle search condition that characterizes a vehicle by an operator's operation, and sends the search request to the AI integrated server 10 (St24). Here, the person search condition is, for example, a person whose upper body is dressed in blue. As for the vehicle search condition, for example, the vehicle type is an SUV.

When the AI integrated server 10 receives the AND search search request from the client terminal VW1, the AI integrated server 10 first extracts the person search condition from the search request sent in step St24 and makes a search request for a person satisfying the person search condition in the person search server 60. Send to (St25).

Based on the search request from the AI integrated server 10, the person search server 60 refers to the database 62 and executes a search process for a person who satisfies the person search condition (St26). When the person search server 60 extracts a person who matches the characteristics of the person search condition (St26, match), the person search server 60 returns the processing result including the thumbnail image of the person to the AI integrated server 10 (St27). On the other hand, when the person search server 60 cannot extract a person who matches the characteristics of the person search condition (St26, mismatch), the person search server 60 returns a processing result indicating that there is no corresponding person information to the AI integrated server 10 (St28).

Following step St27 or step 28, the AI integrated server 10 extracts the vehicle search condition from the search request sent in step St24 and sends a search request for a vehicle satisfying the vehicle search condition to the vehicle search server 80 (St29). .. However, at the time of step St29, the communication between the AI integrated server 10 and the vehicle search server 80 is inactive, and the AI integrated server 10 cannot receive any response from the vehicle search server 80 (St30). ). The AI integrated server 10 is a process result of a person search by the person search server 60 (for example, a thumbnail image of the person and an imaging time) or a result indicating that the person was not found, and various analysis servers (for example, the person search server 60 and the vehicle search server). It is integrated (aggregated) including the connection status data indicating the connection status with each of the 80) and returned to the client terminal VW1 (St31).

The client terminal VW1 uses the communication error message ERR1 indicating the communication disconnection with the vehicle search server 80 and the communication disconnection icon using the person search processing result and the connection status data transmitted from the AI integrated server 10 in step St31. The search result screen WD3 including the processing results of ERROR0 and the person search is displayed on the display device 116 (St32).

Here, it is assumed that after step St32, the communication between the AI integrated server 10 and the vehicle search server 80 is restored (St33). That is, at this point, the communication disabled period between the AI integrated server 10 and the vehicle search server 80 ends. Therefore, the AI integrated server 10 periodically performs alive monitoring with various analysis servers (for example, person search server 60 and vehicle search server 80) (St34), but various analysis servers (for example, for example). The face authentication server 50, the person search server 60, the vehicle search server 80, and the LPR server 90) can transmit a response from the AI integrated server 10 (in other words, a message indicating that the communication state is good) (St35).

Upon receiving the processing result of step St35, the AI integrated server 10 sends a notification to the client terminal VW1 that the communication (connection) with the vehicle search server 80 has been restored (St36). Based on the notification sent from the AI integrated server 10 in step St36, the client terminal VW1 displays the communication return message STB1 on the search result screen WD3 in place of the communication error message ERR1 displayed at the time of step St32. , And stop the display of the communication disconnection icon ERR0 (St37).

As described above, the investigation support system 1 according to the second embodiment includes a user terminal and n (n: 2 or more integers) analysis servers (for example, face recognition server 50, person search server 60, vehicle search server 80, and so on. It includes an AI integrated server 10 that is communicably connected to the LPR server 90). In the examples of FIGS. 9 to 11, n = 4. The AI integrated server 10 periodically monitors the communication status with each of the n analysis servers. When the AI integrated server 10 receives a search condition containing k (an integer from k: 1 to n) of different object feature elements from the user terminal, k of the n analysis servers are included in the search condition. Determine the k analysis servers provided corresponding to each of the object feature elements of. In the examples of FIGS. 9 to 11, k = 2. When the AI integrated server 10 sends a search request for the corresponding object to each of the k analysis servers and detects that a communication interruption with a specific analysis server among the k analysis servers has occurred, (k-). 1) A communication interruption notification (for example, communication error message ERR1) indicating that a communication interruption has occurred between the search results received from the analysis servers and a specific analysis server is sent to the user terminal. The user terminal displays the search result screen WD3 including the search results received from the (k-1) analysis servers and the communication interruption notification.

As a result, the investigation support system 1 can quickly and efficiently support the identification of the escape vehicle used by the suspect who caused the incident, etc., and improve the convenience of the investigation by the police or other investigative agency. .. Further, in the investigation support system 1, there is an analysis server in which communication with the AI integrated server 10 is not possible when searching for a plurality of different objects related to an incident or the like by the operation of a user using the client terminal VW1. In the case, the existence of the analysis server can be visualized in the client terminal VW1 in an easy-to-understand manner by the user.

Further, the user terminal includes the communication disconnection icon ERR0 indicating that the communication interruption with the specific analysis server has occurred in the search result screen WD3 and displays it. As a result, even if the user deletes (hides) the communication error message ERR1 from the search result screen WD3 due to, for example, an erroneous operation, the communication disconnection icon ERR0 displayed on the search result screen WD3 causes a communication interruption. You can easily confirm that you are doing it. It is more preferable that the communication interruption icon ERR0 is an icon that differs depending on the type of analysis server so that it is possible to visualize with which analysis server the communication interruption is occurring.

Further, when the AI integrated server 10 detects that the communication with the specific analysis server has been restored, the user receives a restoration notification (for example, the communication restoration message STB1) indicating that the communication with the specific analysis server has been restored. Send to the terminal. The user terminal includes the return notification sent from the AI integrated server 10 in the search result screen WD3 and displays it. As a result, the investigation support system 1 cannot communicate between the AI integrated server 10 and the specific analysis server when searching for a plurality of different objects related to an incident or the like by the operation of the user using the client terminal VW1. It is possible for the client terminal VW1 to visualize in an easy-to-understand manner that the state has returned to the communicable state.

In addition, the user terminal displays the communication disconnection icon ERR0 indicating that the communication interruption with the specific analysis server has occurred before receiving the return notification from the AI integrated server 10 on the search result screen in response to the reception of the return notification. Hide from WD3. As a result, even if the user deletes (hides) the communication return message STB1 from the search result screen WD3 due to, for example, an erroneous operation, the communication disconnection icon ERR0 displayed on the search result screen WD3 is not displayed. You can easily confirm that there is no communication interruption with a specific analysis server.

(Background to Embodiment 3)
In Patent Document 1, in view of the recent fact that the forms of cases and the like are diversifying, it is considered to search for the suspect who caused the case or the escape vehicle used by the suspect for escape. Not. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the suspect or the escape vehicle in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage. In particular, in order to avoid obtaining unnecessary search results (so-called search noise) during a search, it is expected that the efficiency of the search will increase if the search can be performed after inputting unnecessary object feature elements in advance. Will be done.

(Embodiment 3)
Therefore, in the third embodiment, the execution of the exclusion search that can eliminate unnecessary search results is provided, and the escape vehicle used by the suspect who caused the incident or the like for escape can be identified quickly and efficiently. An example of an investigation support system that supports and improves the convenience of investigations by police and other investigative agencies will be described. Since the configuration of the investigation support system according to the third embodiment is the same as that of the investigation support system 1 according to the first embodiment, the description of the same contents is omitted, and the same reference numerals are given to the contents different from the first embodiment. It will be explained with reference to it.

FIG. 12 is a diagram showing an example of a detailed search condition screen according to the third embodiment. Whether or not the search condition detail screen WD4 of FIG. 12 includes a plurality of different object feature elements (for example, appearance feature elements of a person) related to the object (for example, a person) as search conditions when searching for an object (for example, a person). This is a detailed screen where you can select. The search condition detail screen WD4 is displayed on the display device 116 by the investigation support application based on the user operation (for example, pressing the display icon CHDTL1 in FIG. 7). The search condition detail screen WD4 has an input field display area for the date and time (Time & Date) at which the object to be searched should be detected, a selection field display area for the camera (Camera), and a specification field display area for the object to be searched. , Includes a designated field display area for object feature elements for the specified object. In the example of FIG. 12, a person is specified by a user operation as an object to be searched.

The date and time start input field FR2 is input by user operation as the start date and time when the captured image data to be searched for the object (for example, a person such as a suspect, a vehicle such as an escape vehicle) is captured. In the date and time start input field FR2, for example, the date and time when the incident or the like occurred or a date and time slightly earlier than that date and time is input. When input by user operation, the investigation support application searches for the date and time entered in the date and time start input field FR2 (for example, 7:13:03 pm on September 5, 2019) as a search condition (for example, start date and time). Set as a condition.

The date and time end input field TO2 is input by user operation as the end date and time when the captured image data to be searched for the object (for example, a person such as a suspect or a vehicle such as an escape vehicle) is captured. In the date and time end input field TO2, for example, a predetermined period (for example, a date and time slightly after the occurrence date and time of an incident or the like) is input from the date and time entered in the date and time start input field. When input by user operation, the investigation support application searches for the date and time (for example, 7:13:03 pm on September 6, 2019) entered in the date and time end input field TO2 as a search condition (for example, end date and time). Set as a condition.

In the camera (Camera) selection field, a camera selection icon is arranged to prompt the selection of the camera to be searched.

In the object designation field OBDTL1, for example, four types of objects such as a person (People), a face (Face), a vehicle (Vehicle), and a license plate (License Plates) are displayed so that they can be designated. In the example of FIG. 12, a person (People) is designated as an object by a user operation. The investigation support application executes both a search that satisfies one or more appearance feature elements (Fature) and a search that is the same as or similar to a specific person image (Image) as a search for a person (People). It is possible. Here, a search that satisfies the appearance feature element (Fature) is specified. Further, in the object designation field OBDTL1, the object feature element designation result CON1 by the user operation in the object feature element designation field PSDTL1 is displayed.

In the object feature element designation field PSDTL1, the gender designation field GN1, the hair type designation field HTY1, the hair color designation field HCL1, and the upper color designation field UCC1. , A designated column LCC1 for lower color, a designated column BG1 for belongings, and a designated column PSOCH1 for other features are provided.

In the gender (Gender) designation field GN1, it is specified by the user operation whether the person to be searched is male or female.

In the hairstyle (Hair type) designation field HTY1, it is specified by the user operation whether the hairstyle of the person to be searched is long, semi-long, or the like.

In the hair color (Hair color) designation field HCL1, at least one of five types of hair colors of the person to be searched is designated by the user operation. In addition, a hair color to be excluded from the search can be specified, and a hair color exclusion column EXL1 is also provided. For example, when both yellow and the hair color exclusion column EXL1 are specified, the investigation support application sets a color other than yellow hair color as a search condition for a person.

In the designation field UCC1 of the clothing color of the tops (Upper cross color), at least one color of the tops (that is, the clothing that covers the upper body) of the person to be searched is specified from, for example, 11 kinds of colors by the user operation. To. In addition, the color of the tops to be excluded from the search can be specified, and the clothing color exclusion column EXL2 is also provided. For example, when both white and the clothing color exclusion column EXL2 are specified, the investigation support application sets a color other than white as the clothing color of the tops as a search condition for the person.

In the specification field LCC1 of the clothing color of the bottoms (Lower cross color), at least one of the five colors of the bottoms (that is, the clothing that covers the lower body) of the person to be searched is specified by the user operation. Will be done. In addition, the color of the bottoms to be excluded from the search can be specified, and the clothing color exclusion column EXL3 is also provided. For example, when both blue and the clothing color exclusion column EXL3 are specified, the investigation support application sets a color other than blue as the clothing color of the bottoms as a search condition for the person.

In the personal belongings designation field BG1, it is specified by the user operation whether the person to be searched is a rucksack, a carry case (a suitcase is also possible), or no personal belongings.

In the designation field PSOCH1 of other features, the corresponding items are designated for each of the age, skin, glasses, hat, and beard of the person to be searched by the user operation. The age can be specified from 0 to 12 years old, 13 to 19 years old, 20 to 60 years old, and 61 years old or older. The skin can be designated as bright, medium, or dark. The spectacles can be either worn or not worn. The hat can be either worn or not worn. The beard can be specified with or without.

FIG. 13 is a diagram showing a display example of a part of the search condition input screen according to the third embodiment. The search condition input screen WD5 of FIG. 13 is displayed on the display device 116 by the investigation support application by the operation of the user (for example, a police station operator or a police officer at the scene) during an AND search that simultaneously satisfies a plurality of objects (for example, a face or a person). Is displayed. Note that FIG. 13 is an excerpt of a part of the search condition input screen WD5 relating to the input of the search condition. In the description of FIG. 13, the same elements as those of FIG. 7 are given the same reference numerals to simplify or omit the description, and different contents will be described.

In the search condition input screen WD5 of FIG. 13, the person search condition field PSC1 is designated by the user operation according to the search condition detail screen (see, for example, FIG. 12) displayed by pressing the search exclusion icon EXC1 and the display icon CHDTL1. The search condition specification result CON2 is displayed. This designation result CON2 indicates, for example, four object feature elements "male", "white clothes", "black trousers", and "no belongings". However, since the search exclusion icon EXC1 is selected by user operation, the investigation support application has "male", "non-white clothes", "non-black pants", and "possession" as object feature elements for person search. Set "No item" as a search condition. By selecting the search exclusion icon EXC1, the three object feature elements are the hair color provided with the hair color exclusion column EXL1, the tops provided with the clothing color exclusion column EXL2, and the bottoms provided with the clothing color exclusion column EXL3. It is possible to specify a color other than the specified color. Further, a face image BOY1 of a specific person is uploaded (pasted) in the face search condition field PSC2. Therefore, the investigation support application has the same or similar face as the face image BOY1 of a specific person by the search condition input screen WD5 of FIG. 13, and has "male", "clothes other than white", and "other than black". A search request for searching for a person who satisfies "trousers" and "no belongings" (in other words, a man who does not wear a uniform) is generated and sent to the AI integrated server 10. The search request also includes data on the face image BOY1 of a specific person.

Next, an example of the operation procedure of the investigation support system 1 according to the third embodiment will be described with reference to FIG. FIG. 14 is a sequence diagram showing an example of an operation procedure of the investigation support system 1 according to the third embodiment. In FIG. 14, the flow of the search process using the search condition specified on the search condition input screen WD5 of FIG. 13 will be described in chronological order.

In FIG. 14, the client terminal VW1 has a face that is the same as or similar to the face image of a specific person, and has "male", "non-white clothes", "non-black trousers", and "no belongings". A search request for searching for a person to be satisfied is generated and sent to the AI integrated server 10 (St41).

The AI integrated server 10 transfers the search request sent in step St41 with the face image of a specific person attached to the face recognition server 50 (St42). The face recognition server 50 determines whether or not there is a face image of a person who is the same as or similar to the face image of a specific person by referring to the database 52 according to the search request sent in step St42 (St43). When the face authentication server 50 determines that there is a face image of a person who is the same as or similar to the face image of a specific person (St43, YES), the corresponding face image and the identification information of the camera that captured the face image (St43, YES). For example, the corresponding information (search result) including the camera ID), the imaging time at which the face image was captured, and the face coordinates that specify the position of the face in the face image is sent to the AI integrated server 10 (St44). If it is determined that there is no face image of a person who is the same as or similar to the face image of a specific person (St43, NO), the face recognition server 50 does not have the above-mentioned corresponding information (search result). Is sent to the AI integrated server 10 (St45).

The AI integrated server 10 searches for the camera ID and the imaging time of the corresponding information (search results) sent in step St44 in order to perform an AND search that simultaneously fills a plurality of objects (for example, a face and a person). It is decided to use it as a search condition of (St46).

Next, the AI integrated server 10 searches for the camera ID and imaging time determined in step St46 and the four object feature elements "male", "white clothes", "black trousers", and "no belongings". A request is generated and sent to the person search server 60 (St47). According to the search request sent in step St47, the person search server 60 refers to the database 62 and refers to the above-mentioned four object feature elements “male”, “white clothes”, “black trousers”, and “no belongings”. It is determined whether or not there is a person who satisfies the condition (St48). When the person search server 60 determines that there is a corresponding person (St48, YES), the whole body image of the corresponding person, the identification information (for example, camera ID) of the camera that captured the whole body image, and the whole body image are displayed. The corresponding information (search result) including the imaged imaging time is sent to the AI integrated server 10 (St49).

The AI integrated server 10 has a search result sent in step St49 (that is, a face image search result) among the search results sent in step St44 (that is, a face image search result) from the search result sent in step St44 (that is, a face image search result). The face image search result in which the camera ID and the imaging time are the same as the whole body image of the person included in the person search result) is excluded (St50). As a result, the AI integrated server 10 has the same or similar face as the face image of a specific person, while "male", "non-white clothes", "non-black trousers", and "no belongings". The search results of the satisfying person (that is, the exclusion search results described with reference to FIG. 13) can be efficiently extracted.

On the other hand, when it is determined that there is no corresponding person (St48, NO), the person search server 60 sends a report to the AI integrated server 10 (St51) that the above-mentioned corresponding information (search result) is not found. The AI integrated server 10 sends the search result (including the corresponding face image, camera ID, and imaging time) extracted in step St50 to the client terminal VW1 (St52). The client terminal VW1 displays an exclusion search result screen (not shown) including the search result (see above) sent in step St52 on the display device 116 (St53).

As described above, the investigation support system 1 according to the third embodiment communicates between the user terminal and at least the first analysis server (for example, face recognition server 50) and the second analysis server (for example, person search server 60). Includes an AI integrated server 10 that can be connected. The AI integrated server 10 includes a first object feature element related to an incident or the like (for example, a face image of a specific person) and a second object feature element related to an incident or the like to be searched excluded (for example, an external feature element of a person). When a search condition including is received from the user terminal, a search request for the corresponding object satisfying the first object feature element is sent to the face authentication server 50, and a search request for the corresponding object satisfying the second object feature element is sent to the person search server 60. send. The AI integrated server 10 receives the first search result including the corresponding object satisfying the first object feature element from the face authentication server 50 and receives the second search result including the corresponding object satisfying the second object feature element as a person. Received from the search server 60. The AI integrated server 10 sends a third search result, which is obtained by excluding the second search result from the first search result, to the user terminal. The user terminal displays a search result screen including the third search result sent from the AI integrated server 10.

As a result, the investigation support system 1 can quickly and efficiently support the identification of the escape vehicle used by the suspect who caused the incident, etc., and improve the convenience of the investigation by the police or other investigative agency. .. Further, in the investigation support system 1, it is possible to specify an object feature element that does not need to be searched for each object (for example, a person or a vehicle) by a user operation, so that an unnecessary search result (so-called so-called) when searching for an object desired by the user can be specified. Search noise) is less likely to be extracted, and the accuracy of object search is improved.

In addition, the first search result includes identification information and imaging time information of a camera that has imaged the corresponding object satisfying the first object feature element. The second search result includes identification information and imaging time information of a camera that has imaged the corresponding object satisfying the second object feature element. The AI integrated server 10 extracts a third search result based on the second search result in which the camera identification information and the imaging time information included in each of the first search result and the second search result are common. To do. As a result, the second search result having the same camera ID and imaging time corresponds to the object specified as the exclusion target by the user operation. Therefore, by excluding (that is, removing) from the first search result, the AI integrated server No. 10 can efficiently and easily extract the exclusion search result.

Further, the user terminal sends a search condition with exclusion to the AI integrated server 10 in which the first object feature element or a part of the object feature elements constituting the second object feature element is designated as the search exclusion target. The AI integrated server 10 sends a search request for the corresponding object satisfying the search condition with exclusion to the first analysis server (for example, face recognition server 50) or the second analysis server (for example, person search server 60). The AI integrated server 10 receives the search result from the first analysis server or the second analysis server and sends it to the user terminal. As a result, the user can specify some object feature elements as search exclusion targets on the search condition detail screen WD4 that can specify the object feature elements of one object in detail, and can specify complicated search conditions, and search variations. Can be planned.

(Background to Embodiment 4)
In Patent Document 1, in view of the recent fact that the forms of cases and the like are diversifying, the existence of the suspect who caused the case or the escape vehicle used by the suspect for escape is transmitted to an external terminal or the like. Notification is not considered. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the suspect or the escape vehicle in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage. In particular, when the investigation range covers a wide area, if the suspect or the object feature element of the escape vehicle related to the incident is registered as a collation list in advance, if a person or vehicle matching the collation list is detected, the suspect Alternatively, it is more likely that it is an escape vehicle, and it is expected that the progress of the investigation will be expected.

(Embodiment 4)
Therefore, in the fourth embodiment, even if the investigation area is wide, the detection of the event indicating the existence of the suspect or the escape vehicle related to the incident or the like is notified, and the escape used by the suspect who caused the incident or the like for the escape. An example of an investigation support system that supports the identification of a vehicle quickly and efficiently and improves the convenience of an investigation by an investigative agency such as the police will be described. Since the configuration of the investigation support system according to the fourth embodiment is the same as that of the investigation support system 1 according to the first embodiment, the description of the same contents is omitted, and the same reference numerals are given to the contents different from the first embodiment. It will be explained with reference to it.

FIG. 15 is a diagram showing an example of an event detection operation screen. The event detection operation screen WD6 of FIG. 15 is, for example, when the investigation area for an incident or the like is a floor on a certain floor in a large building, each of the plurality of cameras (see FIG. 2) arranged in the floor. At least the map data FLMP1 on which the arrangement positions are superimposed is displayed. The event detection operation screen WD6 is displayed by the investigation support application based on the user operation or the event notification from the AI integrated server 10. The event detection operation screen WD6 displays the content data of the items of the date and time (Date / Time) of various events detected on the floor, the camera (Camera), and the name (event name) of the event (Event) indicating the event type. The event list EVLST1 composed of the event records to be held is displayed. In the event list EVLST1, the new event record with the date and time of the detected event is displayed at the upper side, and the old event record with the date and time of the detected event is displayed at the lower side.

Further, in the map data FLMP1 of the event detection operation screen WD6, the event abstract window NUM1 indicating the event name and the number of events detected based on the captured image data of the cameras for each of the arrangement positions of the cameras C1, C2, and C3. , NUM2 is displayed by the investigation support application. For example, it is shown that one event "People Matching" was detected at the arrangement position of the camera C1. Similarly, it is shown that one event "Face Matching" and one event "People Matching" were detected at the arrangement position of the camera C2. In addition, the event icons EV1, EV2, EV3, and EV4 indicating the date and time (Date / Time) of the detected event are displayed by the investigation support application together with the event abstract windows NUM1 and NUM2 for each position of the camera. As a result, the user can find out the date and time of one event indicated by the event abstract window NUM1 by the event icon EV1.

In addition, when the investigation support application detects that, for example, the camera placement position, the event icon, or the event name in the event abstract window is pressed by a user operation, the investigation support application highlights the corresponding event record in the event list EVLST1 and highlights the event record. The event details window EVDTL1 of the event corresponding to the pressing is displayed on the event detection operation screen WD6. The event details window EVDTL1 includes at least map data FLMP1v on which the arrangement position DTARE1 of the camera that has captured the captured image data in which the corresponding event is detected is superimposed, the live image button LV1, and the play button PY1.

When it is detected that the live video button LV1 is pressed by the user operation, the investigation support application outputs the live video EVD1 of the camera at the arrangement position DTARE1 to the live video screen MOV1. A snapshot icon SNP1 that enables saving of a snapshot of the live video is displayed on the live video screen MOV1.

Further, when it is detected that the play button PY1 is pressed by the user operation, the investigation support application reproduces the past captured image (not shown) at the event detection date and time of the camera at the arrangement position DTARE1. This captured image may be reproduced and displayed on, for example, the live image screen MOV1 by the investigation support application.

Next, an example of the operation procedure of the investigation support system 1 according to the fourth embodiment will be described with reference to FIG. FIG. 16 is a sequence diagram showing an example of an operation procedure of the investigation support system 1 according to the fourth embodiment. In FIG. 16, the flow of processing leading to the display of the event detection operation screen WD6 of FIG. 15 will be described in chronological order. Further, as a premise of the explanation of FIG. 16, a person search server 60, a vehicle search server 80, and a face recognition server 50 are connected as an example of an analysis server capable of searching for an object.

In FIG. 16, collation lists in which the object feature elements of the person search server 60, the vehicle search server 80, the face authentication server 50, and the object that is the executor of the event to be detected are registered are stored in the databases 62, 82, 52, respectively. It holds (St61, St62, St63). For example, the person search server 60 registers a collation list having a thumbnail image of a person and a person attribute (for example, an external feature element) and holds the collation list in the database 62. Similarly, the vehicle search server 80 registers and holds a collation list having a thumbnail image of a vehicle and vehicle attributes (for example, vehicle type, vehicle color) in the database 82. The face recognition server 50 registers and holds a collation list having a thumbnail image of a person's face in the database 52.

For example, the person search server 60 detects a person who satisfies the collation list (that is, a person who is the actor of the event) based on the image analysis of the captured image data of any camera (for example, camera C1) arranged in the investigation area. (St64). In this case, the person search server 60 generates a real-time notification including the date and time when the event is detected, the camera (including the information on the arrangement position), and the content data of the event name, and sends it to the AI integrated server 10 (St65). The AI integrated server 10 transfers the real-time notification sent in step St65 to the client terminal VW (St66). The client terminal VW1 generates an event record (see FIG. 15) based on the content data of each item included in the real-time notification sent in step St66, and detects an event including the event list EVLST1 in which the event record is additionally updated. The operation screen WD6 is generated and displayed on the display device 116 (St661).

For example, the face recognition server 50 fills the collation list based on the image analysis of the captured image data of any camera (for example, camera C3) arranged in the investigation area (that is, the face of the person who is the actor of the event). Is detected (St67). In this case, the face recognition server 50 generates a real-time notification including the date and time when the event is detected, the camera (including the information on the arrangement position), and the content data of the event name, and sends it to the AI integrated server 10 (St68). The AI integrated server 10 transfers the real-time notification sent in step St68 to the client terminal VW (St69). The client terminal VW1 generates an event record (see FIG. 15) based on the content data of each item included in the real-time notification sent in step St69, and detects an event including the event list EVLST1 in which the event record is additionally updated. The operation screen WD6 is generated and displayed on the display device 116 (St691).

For example, the person search server 60 detects a person who satisfies the collation list (that is, a person who is the actor of the event) based on the image analysis of the captured image data of any camera (for example, camera C3) arranged in the investigation area. (St70). In this case, the person search server 60 generates a real-time notification including the date and time when the event is detected, the camera (including the information on the arrangement position), and the content data of the event name, and sends it to the AI integrated server 10 (St71). The AI integrated server 10 transfers the real-time notification sent in step St71 to the client terminal VW (St72). The client terminal VW1 generates an event record (see FIG. 15) based on the content data of each item included in the real-time notification sent in step St72, and detects an event including the event list EVLST1 in which the event record is additionally updated. The operation screen WD6 is generated and displayed on the display device 116 (St721).

For example, the vehicle search server 80 detects a vehicle that satisfies the collation list (that is, a vehicle that is the actor of the event) based on the image analysis of the captured image data of any camera (for example, camera C10) arranged in the investigation area. (St73). Although the camera C10 is not shown in FIG. 15, for example, it is possible to take an image of a parking lot outside the building in the investigation area, and the suspect can take an image of the escape vehicle at the time of escape. In this case, the vehicle search server 80 generates a real-time notification including the date and time when the event is detected, the camera (including the information on the arrangement position), and the content data of the event name, and sends it to the AI integrated server 10 (St74). The AI integrated server 10 transfers the real-time notification sent in step St74 to the client terminal VW (St75). The client terminal VW1 generates an event record (see FIG. 15) based on the content data of each item included in the real-time notification sent in step St75, and detects an event including the event list EVLST1 in which the event record is additionally updated. The operation screen WD6 is generated and displayed on the display device 116 (St751).

For example, the person search server 60 detects a person who satisfies the collation list (that is, a person who is the actor of the event) based on the image analysis of the captured image data of any camera (for example, camera C3) arranged in the investigation area. (St76). In this case, the person search server 60 generates a real-time notification including the date and time when the event is detected, the camera (including the information on the arrangement position), and the content data of the event name, and sends it to the AI integrated server 10 (St77). The AI integrated server 10 transfers the real-time notification sent in step St77 to the client terminal VW (St78). The client terminal VW1 generates an event record (see FIG. 15) based on the content data of each item included in the real-time notification sent in step St78, and detects an event including the event list EVLST1 in which the event record is additionally updated. The operation screen WD6 is generated and displayed on the display device 116 (St781). In FIG. 16, for example, the processing from the time when five events are detected to the real-time notification to the client terminal VW1 is described in chronological order, but it goes without saying that the number of detected events is not limited to five.

As described above, in the investigation support system 1 according to the fourth embodiment, the user terminal and the plurality of analysis servers are connected to each other so as to be able to communicate with each other via the AI integrated server 10. Each of the plurality of analysis servers is connected to each of the plurality of cameras C1 to C20, and holds a collation list of objects different from the objects that can be searched by other analysis servers regarding an incident or the like. When each of the plurality of analysis servers detects an object that satisfies the collation list held by itself by analyzing the image captured by the camera, a detection notification (detection notification) including information on the event name indicating the detection of the object, the detection date and time, and the camera placement position ( For example, real-time notification) is sent to the user terminal via the AI integrated server 10. The user terminal displays an event detection screen (for example, event detection operation screen WD6) including information on the event name, detection date and time, and camera placement position included in the detection notification sent from any of the analysis servers.

As a result, the investigation support system 1 can quickly and efficiently support the identification of the escape vehicle used by the suspect who caused the incident, etc., and improve the convenience of the investigation by the police or other investigative agency. .. Further, the investigation support system 1 can notify the client terminal VW1 of the detection of an event indicating the existence of a suspect or an escape vehicle related to an incident even if the investigation area is wide, so that the progress of the investigation of the suspect or the escape vehicle can be accelerated. Can be expected.

Further, the user terminal includes the event list including the event name, the detection date and time, and the information items of the camera arrangement position on the event detection operation screen WD6 and displays the event list. As a result, by visually observing the event list, the user can efficiently grasp the list of events detected around the position where the camera is arranged even if the investigation area is wide.

In addition, each time the user terminal receives a detection notification (for example, real-time notification) from any analysis server, the user terminal adds the event name, detection date and time, and camera placement position information items included in the detection notification to the event list. And update. As a result, the user can check the progress of the list of events detected around the position where the camera is arranged even if the investigation area is wide by visually observing the event list that is updated from time to time.

In addition, the user terminal has map data FLMP1 of the area where each of the plurality of cameras is arranged, and corresponds to the event name, the number of events, and the detection date and time detected based on the image captured by the cameras for each camera arrangement position. It is attached and superimposed on the map data MP1 and displayed on the event detection operation screen WD6. As a result, the user can visually and intuitively grasp the details of the event detected by the camera arranged in the map data FLMP1 of the investigation area in an easy-to-understand manner.

Further, the user terminal includes the live image output window (for example, live image screen MOV1) of the camera at the arrangement position corresponding to the specified detection date and time in the event detection operation screen WD6 and displays it based on the specified detection date and time. .. As a result, the user can easily visually check the current live image of the camera at the position where the event of concern is detected, so that it is possible to efficiently check the current situation without going to the position of the camera, for example. Can be done.

Further, the user terminal has map data FLMP1 of the area where each of the plurality of cameras is arranged, and based on the designation of any item of the event list EVLST1, the arrangement position of the camera corresponding to the specified item is set. The event detail window EVDTL1 having at least the superimposed map data FLMP1 and the playback button PY1 of the image captured by the camera on the detection date and time is included in the event detection operation screen WD6 and displayed. As a result, the user can easily visually check the captured image of the camera at the time of event detection (past) at the position where the event of concern is detected, so that the user does not have to go to the position of the camera at the time of the event, for example. You can efficiently check the status of.

Further, the user terminal includes the live image output button (for example, the live image button LV1) of the camera at the arrangement position corresponding to the specified item in the event detail window EVDTL1 and displays it. As a result, the user can selectively switch between the current live image of the camera at the position where the event of concern is detected or the past captured image at the time of event detection and easily visually check the position, for example, the position of the camera. You can efficiently check the current status or the situation when an event is detected without having to go to.

(Background to Embodiment 5)
In Patent Document 1, in view of the recent fact that the forms of cases, etc. are diversifying, the suspect who caused the case, etc., or a predetermined action that makes the suspect suspect the existence of an escape vehicle used for escape. It is not considered to notify an external terminal or the like when (for example, a fight, a pistol firing, a sudden acceleration escape) is detected. In particular, in an investigation by an investigative agency such as the police (especially an initial investigation), it is often required to quickly identify the suspect or the escape vehicle in order to detect the suspect at an early stage. However, even if the technology of Patent Document 1 is used, if an investigator (for example, a police officer) manually watches and confirms the images of each camera device, it takes time to identify the escape vehicle and it is inefficient. There was a problem that it was difficult to detect a person or an escape vehicle at an early stage. In particular, if the investigation covers a wide area, if the above-mentioned behavior is detected in a place not known by the police officer, it can be blacklisted so that the actor or vehicle of the behavior can be tracked. It is expected that the possibility of being a vehicle will increase and that the progress of the investigation will be expected.

(Embodiment 5)
Therefore, in the fifth embodiment, even if the investigation area is wide, the detection of a predetermined action that makes the suspect or the escape vehicle suspect the existence of the incident or the like is notified, and the suspect who caused the incident or the like uses it for escape. An example of an investigation support system that assists in the early and efficient identification of a fugitive vehicle and improves the convenience of an investigation by an investigative agency such as the police will be described. Since the configuration of the investigation support system according to the fifth embodiment is the same as that of the investigation support system 1 according to the first embodiment, the description of the same contents is omitted, and the same reference numerals are given to the contents different from the first embodiment. It will be explained with reference to it.

FIG. 17 is a diagram showing a display transition from the event detection screen WD7 to the analysis result screen WD8. The event detection screen WD7 of FIG. 17 is a behavior detection server 70 and an AI integrated server when, for example, a behavior detection server 70 detects a predetermined behavior (for example, a fight) that makes the suspect or an escape vehicle suspect the existence of an incident or the like. It is displayed by the investigation support application based on the alarm notification sent after 10. On the event detection screen WD7, for example, the playback window PLY2 of the captured image on which the quarrel thumbnail image THM2 of the camera when the quarrel is detected is superimposed, the arrangement position IC4 of the camera where the quarrel is detected, and the quarrel thumbnail image THM2 are superimposed. The map data MP2 and the still image analysis button ALY1 for the fight thumbnail image THM2 are displayed. A time indicator DAT1 indicating the date and time when the quarrel was detected (for example, Tuesday, June 17, 15:55:22) is superimposed on the playback window PLY2.

When the investigation support application detects that the still image analysis button ALY1 is pressed by the user operation, it generates an analysis request (see below) based on the pressing of the still image analysis button ALY1 and sends it to the AI integrated server 10. Although the details will be described later, when the investigation support application receives the analysis result from the AI integrated server 10, the analysis result screen WD8 is displayed on the display device 116.

The analysis result screen WD8 of FIG. 17 is displayed by the investigation support application using the analysis result sent from the AI integrated server 10 based on the pressing of the still image analysis button ALY1 for the fight thumbnail image THM2. Specifically, the analysis result screen WD8 includes a captured image (captured image) of the camera when the quarrel is detected, on which the quarrel thumbnail image THM2 is superimposed, and three images as the analysis result (Detected). Various buttons provided corresponding to the image are displayed.

Specifically, the three images obtained as the analysis result are the face image FCIMG1 and the whole body image TLIMG1 of one person PERS1 of the two people (that is, the actors of the fight) reflected in the fight thumbnail image THM2, and the other person. It is a whole body image TLIMG2 of PERS2. The search button SC1 and the blacklist registration button BLKRG1 are displayed corresponding to the face image FCIMG1. The search button SC2 and the blacklist registration button BLKRG2 are displayed corresponding to the whole body image TLIMG1. The search button SC3 and the blacklist registration button BLKRG3 are displayed corresponding to the whole body image TLIMG2.

The search buttons SC1, SC2, and SC3 are buttons for requesting the AI integrated server 10 to search for a person who is the same as or similar to the corresponding face image FCIMG1, whole body image TLIMG1, and TLIMG2, respectively. The blacklist registration buttons BLKRG1, BLKRG2, and BLKRG3 are applied to the corresponding analysis servers (specifically, face recognition server 50, person search server 60, person search server 60) for the corresponding face image FCIMG1, whole body image TLIMG1, and TLIMG2. This is a button for requesting registration as blacklist data from the AI integrated server 10.

FIG. 18 is a diagram showing an example of a person tracking operation screen. The person tracking operation screen WD9 of FIG. 18 is, for example, when the investigation area for an incident or the like is a floor on a certain floor in a large building, an image of the person to be tracked (for example, full-body images TLIMG0, TLIMG1) and the inside of the floor. At least the map data FLMP1 on which the arrangement positions of the plurality of arranged cameras (see FIG. 2) are superimposed is displayed. The person tracking operation screen WD9 is displayed by the investigation support application based on the user operation or the alarm notification from the AI integrated server 10. Here, the whole body image TLIMG1 corresponding to the blacklist registration button BLKRG2 pressed by the user operation on the analysis result screen WD8 of FIG. 18 is additionally displayed on the watch list (Watchlist) of the person tracking operation screen WD9 of FIG. There is.

In addition, the person tracking operation screen WD9 is an event that has the content data of the items named as the date and time (Date / Time) of a number of events detected on the floor, the camera (Camera), and the event (Event) indicating the event type. The event list EVLST2 composed of records is displayed. In the event list EVLST2, the new event record with the date and time of the detected event is displayed at the upper side, and the old event record with the date and time of the detected event is displayed at the lower side.

Further, on the map data FLMP1 of the person tracking operation screen WD9, the whole body image TLIMG0 when the person displayed in the watch list is first detected is superimposed and displayed on the corresponding camera arrangement position, and further, the watch is displayed. Thumbnail image DTPS1, quarrel thumbnail image THM2, thumbnail image DTPS3 when a predetermined action (see above) by a person who is the same as or similar to the person displayed in the list is detected is superimposed on the position of the corresponding camera. It is displayed.

In addition, event icons EV5, EV6, EV7, and EV8 indicating the date and time (Date / Time) of the detected event are displayed by the investigation support application. As a result, the user can find out the date and time when the predetermined action (see above) performed by the whole body image TLIMG0 or the thumbnail image DTPS1, the fight thumbnail image THM2, and the thumbnail image DTPS3 is detected by the event icons EV5, EV6, EV7, and EV8. Is.

Further, when the investigation support application detects that, for example, a full-body image, a thumbnail image, or an event icon is pressed by a user operation, the investigation support application highlights the corresponding event record in the event list EVLST1 and displays the event corresponding to the pressing. The event details window EVDTL2 is displayed on the person tracking operation screen WD9. The event details window EVDTL2 shows the full-body image TLIMG1 when the person who is the actor of the corresponding predetermined action (see above) is first added to the watch list, and when the predetermined action (see above) is detected. At least the thumbnail image DTPS1, the map data FLMP1v on which the arrangement position DTARE2 of the camera that captured the captured image data when a predetermined action (see above) is detected, the live image button LV2, and the play button PY2 are superimposed. Including.

When it is detected that the live image button LV2 is pressed by the user operation, the investigation support application outputs the live image of the camera at the arrangement position DTARE2 to the live image screen (not shown).

Further, when it is detected that the play button PY2 is pressed by the user operation, the investigation support application reproduces the past captured image (not shown) at the event detection date and time of the camera at the arrangement position DTARE2. This captured image may be reproduced and displayed on a live image screen, for example, by an investigation support application.

Next, an example of the operation procedure of the investigation support system 1 according to the fifth embodiment will be described with reference to FIG. FIG. 19 is a sequence diagram showing an example of an operation procedure of the investigation support system 1 according to the fifth embodiment. In FIG. 19, the flow of processing leading to the display of the person tracking operation screen WD9 of FIG. 18 will be described in chronological order. Further, as a premise of the explanation of FIG. 19, a person search server 60 is connected as an example of an analysis server capable of searching an object.

In FIG. 19, the behavior detection server 70 detects a predetermined behavior (for example, a fight) based on the video analysis of the captured video data of any of the cameras C1 to C20 (St81). The action detection server 70 sends an alarm notification including the fact that a predetermined action (for example, a fight) has been detected, the captured image (or the fight thumbnail image THM2) at the time of detection, the camera identification information, and the imaging time to the AI integrated server 10. (St82). The AI integrated server 10 transfers the alarm notification sent from the behavior detection server 70 to the client terminal VW1 (St83). When the client terminal VW1 receives the alarm notification transferred from the AI integrated server 10, the client terminal VW1 displays the event detection screen WD7 of FIG. 17 on the display device 116 based on the alarm notification (St84).

It is assumed that the user finds a person who is interested in the actor of a predetermined action (for example, a fight) displayed on the event detection screen WD7. When the investigation support application (in other words, the client terminal VW1) detects that the still image analysis button ALY1 is pressed by the user operation (St85), the analysis request of the person based on the pressing of the still image analysis button ALY1 (fight is detected). (Including the image captured by the camera or the image of the fight thumbnail image THM2) is generated and sent to the AI integrated server 10 (St86). The AI integrated server 10 transfers the analysis request of the person sent from the client terminal VW1 in step St86 to the person search server 60 (St87).

When the person search server 60 receives the analysis request of the person transferred from the AI integrated server 10 in step St87, the person search server 60 is reflected in the image captured by the camera or the thumbnail image THM2 of the quarrel when the quarrel is detected based on the analysis request. A person analysis process (for example, detection of a person and a number of people reflected in a captured image or a fight thumbnail image THM2, and extraction of coordinates of a face frame and a person frame) is executed (St88). The person search server 60 sends the analysis result including the person, the detection of the number of people, the face frame, and the coordinates of the person frame to the AI integrated server 10 (St89). The AI integrated server 10 transfers the analysis result sent from the person search server 60 in step St89 to the client terminal VW1 (St90). When the client terminal VW1 receives the analysis result transferred from the AI integrated server 10, the client terminal VW1 displays the analysis result screen WD8 of FIG. 17 on the display device 116 based on the analysis result (St91).

It is assumed that the user finds a person (for example, a suspect in an incident) to be added as a tracking target among the images of at least one person (for example, a face image or a whole body image) displayed on the analysis result screen WD8. In the investigation support application (in other words, the client terminal VW1), the blacklist registration button BLKRG2 of the image of the person to be tracked (for example, the whole body image TLIMG1 as the analysis result displayed on the analysis result screen WD8) was pressed by the user operation. When this is detected (St92), a blacklist registration request (including the whole body image TLIMG1 to be registered) of a person's image based on pressing the blacklist registration button BLKRG2 is generated and sent to the AI integrated server 10 (St93). ..

Further, the investigation support application switches from the analysis result screen WD8 to the person tracking operation screen WD9 of FIG. 18 and displays it on the display device 116 based on the pressing of the blacklist registration button BLKRG2 (St92). At this time, the investigation support application adds the whole body image TLIMG1 selected by the user operation as the target of blacklist registration to the watch list of the person tracking operation screen WD9 and displays it. Alternatively, the investigation support application may add the person tracking operation screen WD9 of FIG. 18 together with the analysis result screen WD8 based on the pressing of the blacklist registration button BLKRG2 and display it on the display device 116 (for example, in another window). Good (St92). The AI integrated server 10 transfers the blacklist registration request for the image of the person sent from the client terminal VW1 in step St93 to the person search server 60 (St94). When the person search server 60 receives the blacklist registration request transferred from the AI integrated server 10 in step St94, the person search server 60 registers the image of the person (for example, whole body image TLIMG1) included in the blacklist registration request in the database 62 (St95). ).

After a certain period of time has elapsed after step St95, the person search server 60 and the whole body image TLIMG1 of the person registered in the database 62 in step St95 based on the image analysis of the captured image data of the cameras C1 to C20. It is assumed that the same or similar person is detected (St96). The person search server 60 sends an alarm notification including the fact that a person has been detected, the captured image (or thumbnail image DTPS1) at the time of detection, the camera identification information, and the imaging time to the AI integrated server 10 (St97). The AI integrated server 10 transfers the alarm notification sent from the person search server 60 to the client terminal VW1 (St98). When the client terminal VW1 receives the alarm notification transferred from the AI integrated server 10, the image of the corresponding person (for example, thumbnail image DTPS1) and the blacklist are registered on the person tracking operation screen WD9 of FIG. 18 based on the alarm notification. The event detail window EVDTL2 including the whole body image TLIMG1 is displayed (St99).

As described above, in the investigation support system 1 according to the fifth embodiment, the user terminal and the plurality of analysis servers are connected to each other so as to be able to communicate with each other via the AI integrated server 10. When the first analysis server (for example, the action detection server 70) among the plurality of analysis servers detects a specific action (for example, a fight) related to an incident or the like by analyzing the captured image of any one of the cameras C1 to C20, the action is taken. An alarm notification including at least the captured image of the detected camera is sent to the user terminal via the AI integrated server 10. The user terminal has an event notification screen (for example, an event) including an captured image included in the alarm notification sent from the first analysis server and a person analysis button (for example, still image analysis button ALY1) of the actor of the action reflected in the captured image. The detection screen WD7) is displayed, and based on the designation of the person analysis button, the person analysis of the actor reflected in the captured image is requested via the AI integrated server 10. The second analysis server (for example, the person search server 60) among the plurality of analysis servers obtains the person analysis result of the actor reflected in the captured image on the user terminal via the AI integrated server 10 based on the request from the user terminal. Send to. The user terminal includes a cut-out image of the actor included in the person analysis result sent from the second analysis server (for example, full-body image TLIMG1) and a tracking button for the person shown in the cut-out image (for example, blacklist registration button BLKRG2). The analysis result screen WD8 is displayed. Based on the designation of the tracking button, the user terminal requests the blacklist registration of the person corresponding to the tracking button via the AI integrated server 10. The second analysis server registers the cutout image of the person corresponding to the tracking button in the blacklist data of the second analysis server based on the request from the user terminal.

As a result, the investigation support system 1 can quickly and efficiently support the identification of the escape vehicle used by the suspect who caused the incident, etc., and improve the convenience of the investigation by the police or other investigative agency. .. Further, the investigation support system 1 can notify the client terminal VW1 of the detection of a predetermined action that makes the suspect or the escape vehicle suspect the existence of the suspect or the escape vehicle even if the investigation area is wide, so that the investigation of the suspect or the escape vehicle progresses. Can be expected to accelerate.

Further, when the second analysis server (for example, the person search server 60) detects a person who is the same as or similar to the person registered in the blacklist data by analyzing the image captured by the camera, the captured image showing the detected person, A detection notification including information on the detection date and time and the position of the camera is sent to the user terminal via the AI integrated server 10. The user terminal displays the person tracking operation screen WD9 including the captured image, the detection date and time, and the information on the camera placement position included in the detection notification sent from the second analysis server. As a result, the user blacklists the person he / she is interested in (for example, a suspect in an incident or a similar person) as a tracking target, and as a result, the person is detected by the person tracking operation screen WD9. Intuitive and visually easy to understand.

Further, the user terminal includes a tracking target list (for example, a watch list) showing a cut-out image of a person corresponding to the tracking button in the person tracking operation screen WD9 and displays it. This allows the user to quickly and intuitively grasp the person currently being tracked.

Further, the user terminal includes and displays the event list EVLST2 having the items of the action name, the action detection date and time, and the camera placement position information on the person tracking operation screen WD9. As a result, by visually observing the event list, the user can efficiently grasp the list of events detected around the position where the camera is arranged even if the investigation area is wide.

In addition, each time the user terminal receives a detection notification from the second analysis server, the user terminal adds the action name, action detection date and time, and camera placement position information items included in the detection notification to the event list WVLST2. And update. As a result, the user can check the progress of the list of events detected around the position where the camera is arranged even if the investigation area is wide by visually observing the event list that is updated from time to time.

Further, the user terminal has map data FLMP1 of the area where the camera is arranged, and the captured image in which the action is detected and the detection date and time of the action are associated with the arrangement position of the camera and superimposed on the map data FLMP1 to obtain a person. It is included in the tracking operation screen WD9 and displayed. As a result, the user can visually and intuitively grasp the details of the event detected by the camera arranged in the map data FLMP1 of the investigation area in an easy-to-understand manner.

Further, when the user terminal is registered in the blacklist data corresponding to the action, the cut-out image (for example, thumbnail image DTPS1) of the actor of the action reflected in the designated captured image based on the designation of the captured image. The event detail window EVDTL2 in which the cut-out image of the person (for example, the whole body image TLIMG1) is arranged is included in the person tracking operation screen WD9 and displayed. As a result, the user can visually check the image of the person at the time of event detection of the camera at the position where the event of concern is detected and the image of the person at the time of blacklist registration, so that the camera can be visually confirmed, for example. You can efficiently check the situation at the time of the event without having to go to the location of.

Further, the third analysis server (for example, the face recognition server 50) among the plurality of analysis servers uses the AI integrated server 10 to obtain the face analysis result of the actor's person reflected in the captured image based on the request from the user terminal. It may be sent to the user terminal via. In this case, the user terminal displays the analysis result screen WD8 including the cut-out image of the actor's face included in the face analysis result sent from the third analysis server and the face tracking button reflected in the cut-out image. Based on the designation of the tracking button, the user terminal requests the blacklist registration of the face corresponding to the tracking button via the AI integrated server 10. The third analysis server registers the cutout image of the face corresponding to the tracking button in the blacklist data of the third analysis server based on the request from the user terminal via the AI integrated server 10. As a result, the face of a person who is an actor of a predetermined action (for example, a fight) can be easily registered in the blacklist data of the face recognition server 50, so that the suspect can be investigated even if the investigation area is wide. Can be expected to accelerate the progress of.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications, modifications, substitutions, additions, deletions, and equality within the scope of the claims. It is understood that it naturally belongs to the technical scope of the present disclosure. Further, each component in the various embodiments described above may be arbitrarily combined as long as the gist of the invention is not deviated.

In the first embodiment described above, as the search handled by the AI integrated server 10, a person search by the person search server 60 and a vehicle search by the vehicle search server 80 have been described as examples, but the search is not limited thereto. In other words, the search criteria are not limited to people or vehicles. For example, the search handled by the AI integrated server 10 may be applied to face detection (face search) by the face authentication server 50, license plate search by the LPR server 90, and further, person search and vehicle search by the person search server 60. A vehicle search by the server 80 may also be used in combination. In this case, a face image (image search) or a license plate (text search or image search) may be used as the search condition. As the search result using the face image, for example, the face image may be displayed. As the search result using the license plate, for example, the face image of the purchaser of the corresponding vehicle may be displayed.

As an example of the AND search (cross-sectional search) handled by the AI integrated server 10 in the above-described first embodiment, an AND search of a person search by the person search server 60 and a vehicle search by the vehicle search server 80, and a face authentication server. Although the AND search of the face matching by 50, the person search by the person search server 60, and the vehicle search by the vehicle search server 80 has been described as an example, the example of the AND search is not limited to these. For example, the AI integrated server 10 can perform an AND search (cross-sectional search) by the following combinations.

For example, in addition to face matching by the face authentication server 50, person search by the person search server 60, action detection by the action detection server 70 (for example, matching of a face image similar to the face image of a person who has a fight or a fist gun), a vehicle Vehicle search by the search server 80 (for example, matching the face image of a man in his thirties in a white car), number plate matching by the LPR server 90 (for example, a face image in which the upper two digits are in a vehicle with a specific number) At least one of (matching the face image of a person similar to) and the like can be combined.

Further, for example, in addition to the person search by the person search server 60, the action detection by the action detection server 70 (for example, the search for a person wearing black clothes who is destroying an object or possessing a fist gun), and the search by the vehicle search server 80 (for example, For example, at least one of (searching for a person in red clothes approaching a white car), license plate verification by the LPR server 90 (for example, searching for a person wearing black clothes approaching a vehicle with a specific number), etc. Can be combined.

Further, for example, in addition to the vehicle search by the vehicle search server 80, the behavior detection by the behavior detection server 70 (for example, a search for a sedan type vehicle running in reverse or a taxi vehicle threatened by a fist gun from the outside), an LPR server At least one of 90's license plate verification (for example, searching for a vehicle in which the upper two digits are running in reverse with a specific number) and the like can be combined.

Further, for example, in addition to the license plate verification by the LPR server 90, the action detection by the action detection server 70 (for example, the detection of the reverse running action of a specific number in the upper two digits, and the taxi vehicle in which the upper two digits are a specific number) At least one of (detection of actions threatening the driver with a fist gun from the outside) and the like can be combined.

In the first embodiment described above, it is illustrated that the object in each of the captured images of the cameras C1 to C20 is a person or a vehicle, but the object is not limited to the person or the vehicle, and other objects (for example, It may be a mobile body). The moving object may be, for example, a flying object such as a drone operated by a person such as a suspect who caused an incident or the like.

This disclosure provides early and efficient support for identifying the suspect who caused the incident or the escape vehicle used by the suspect for escape, and improves the convenience of investigation by police and other investigative agencies. It is useful as a system and an investigation support method.

1 Investigation support system 10 AI integrated server 11, 41, 51, 61, 71, 81, 91, PRC1, PRC2 Processor 12, 42, 52, 62, 72, 82, 92 Database 13, 54, 114 Server IF controller 14 Client IF controller 40 Video management server 50 Face authentication server 53 Camera IF controller 60 Person search server 70 Behavior detection server 80 Vehicle search server 90 LPR server 112 Recording device 115 Input device 116 Display device 117 Speakers C1, C2, C3, C20 Camera MM1, MM2, MM3 Memory NW1 Network VW1 Client terminal VW2 Mobile terminal

Claims (35)

It is an investigation support system in which the analysis server and the integrated server are connected so that they can communicate with each other.
The analysis server
Upon receiving the first captured image of the vehicle captured at the first imaging timing, the face of the occupant of the vehicle and the license plate are analyzed, and the first analysis including the occupant's face image and the license number information is included. Send the result to the integrated server
The integrated server
The first analysis result is registered in the database in association with the first imaging timing, and the result is registered in the database.
The analysis server
Upon receiving the second captured image of the vehicle captured at the second imaging timing, the face of the occupant of the vehicle and the license plate are analyzed, and the second analysis including the occupant's face image and the license number information is included. Send the result to the integrated server
The integrated server
The license number information included in the first analysis result and the license number information included in the second analysis result match, and the face image of the occupant and the second analysis result included in the first analysis result. If it is determined that the face image of the occupant included in the analysis result of the above is different, an alarm notification regarding the incident etc. is given.
Investigation support system. The camera that captured the first captured image at the first imaging timing and the camera that captured the second captured image at the second imaging timing are the same.
The investigation support system according to claim 1. The camera that captured the first captured image at the first imaging timing and the camera that captured the second captured image at the second imaging timing are different.
The investigation support system according to claim 1. A user terminal, which is connected to the integrated server so as to be communicable, is further provided.
The user terminal is
Based on the alarm notification, an alarm notification screen including at least the license number information, the occupant's face image included in the first analysis result, and the occupant's face image included in the second analysis result is displayed. To do,
The investigation support system according to claim 1. The user terminal is
The map data in which the arrangement position of the camera that captured the first captured image is superimposed is included in the alarm notification screen and displayed.
The investigation support system according to claim 4. It is an investigation support method executed by an investigation support system in which the analysis server and the integrated server are communicably connected.
When the analysis server receives the first captured image of the vehicle captured at the first imaging timing, it analyzes the occupant's face and license plate of the vehicle, and obtains the occupant's face image and license number information. The first analysis result including is sent to the integrated server,
The integrated server registers the first analysis result in the database in association with the first imaging timing.
When the analysis server receives the second captured image of the vehicle captured at the second imaging timing, it analyzes the occupant's face and license plate of the vehicle, and obtains the occupant's face image and license number information. The second analysis result including the result is sent to the integrated server, and the result is sent to the integrated server.
By the integrated server, the license number information included in the first analysis result and the license number information included in the second analysis result match, and the face of the occupant included in the first analysis result. When it is determined that the image and the face image of the occupant included in the second analysis result are different, an alarm notification regarding the incident or the like is given.
Investigation support method. An investigation support system that includes an integrated server that is communicably connected between a user terminal and a plurality of analysis servers.
The user terminal is
Display the analysis server selection screen showing a list of the plurality of analysis servers provided corresponding to each of a plurality of different object feature elements related to an incident or the like.
The first analysis server is selected from the analysis server selection screen, and the corresponding object satisfying the search condition is based on the input of the search condition including the object feature element of the object searched by the selected first analysis server. Send the search request to the integrated server
The integrated server
Based on the search request, a search instruction for the corresponding object is sent to the first analysis server, a search result from the first analysis server is received, and the search result is sent to the user terminal.
Investigation support system. The user terminal is
After selecting the first analysis server, select the second analysis server from the analysis server selection screen, and the object of the object that can be searched by each of the selected first analysis server and the second analysis server. Based on the input of the search condition including the feature element, a search request for the corresponding object satisfying the search condition is sent to the integrated server.
The integrated server
Based on the search request, a search instruction for the corresponding object is sent to each of the first analysis server and the second analysis server, and a search from each of the first analysis server and the second analysis server is performed. Receive the result and send it to the user terminal,
The investigation support system according to claim 7. An investigation support method executed by an investigation support system including an integrated server that is communicably connected between a user terminal and a plurality of analysis servers.
The user terminal displays an analysis server selection screen showing a list of the plurality of analysis servers provided corresponding to each of a plurality of different object feature elements related to an incident or the like.
The user terminal selects the first analysis server from the analysis server selection screen, and the search is based on the input of the search condition including the object feature element of the object searched by the selected first analysis server. Send a search request for the corresponding object that meets the conditions to the integrated server,
Based on the search request, the integrated server sends a search instruction for the object to the first analysis server, receives a search result from the first analysis server, and sends it to the user terminal.
Investigation support method. An investigation support system including an integrated server that is communicably connected between a user terminal and n (n: integers of 2 or more) analysis servers.
The integrated server
The communication status with each of the n analysis servers is periodically monitored, and the communication status is monitored.
When a search condition containing k (integer from k: 1 to n) different object feature elements is received from the user terminal, the k object feature elements included in the search condition among the n analysis servers The k analysis servers provided corresponding to each are determined, and a search request for the corresponding object is sent to each of the k analysis servers.
When it is detected that communication with a specific analysis server among the k analysis servers has occurred, the search result received from (k-1) analysis servers and the specific analysis server A communication disconnection notification that a communication interruption has occurred is sent to the user terminal,
The user terminal is
A search result screen including the search results received from the (k-1) analysis servers and the communication interruption notification is displayed.
Investigation support system. The user terminal is
A communication disconnection icon indicating that a communication interruption with the specific analysis server has occurred is included in the search result screen and displayed.
The investigation support system according to claim 10. The integrated server
When it is detected that the communication with the specific analysis server has been restored, a restoration notification indicating that the communication with the specific analysis server has been restored is sent to the user terminal.
The user terminal is
The return notification sent from the integrated server is included in the search result screen and displayed.
The investigation support system according to claim 10. The user terminal is
The communication disconnection icon indicating that the communication interruption with the specific analysis server occurred before the return notification is received from the integrated server is hidden from the search result screen in response to the reception of the return notification. ,
The investigation support system according to claim 12. An investigation support method executed by an investigation support system including an integrated server that is communicably connected between a user terminal and n (n: integers of 2 or more) analysis servers.
The integrated server periodically monitors the communication status with each of the n analysis servers.
When the integrated server receives a search condition containing k (integer from k: 1 to n) of different object feature elements from the user terminal, the k of the n analysis servers included in the search condition. Determine the k analysis servers provided for each of the object feature elements of
The integrated server sends a search request for the corresponding object to each of the k analysis servers.
When the integrated server detects that communication with a specific analysis server among the k analysis servers has occurred, the search results received from the (k-1) analysis servers and the specific analysis are performed. A communication disconnection notification that a communication interruption with the server has occurred is sent to the user terminal, and the communication is interrupted.
The user terminal displays a search result screen including the search results received from the (k-1) analysis servers and the communication interruption notification.
Investigation support method. An investigation support system that includes a user terminal and an integrated server that is communicably connected to at least a first analysis server and a second analysis server.
The integrated server
When a search condition including a first object feature element related to an incident or the like and a second object feature element related to the case or the like is received from the user terminal, a search request for the corresponding object satisfying the first object feature element is made by the first object. A search request for the corresponding object that satisfies the second object feature element is sent to the second analysis server.
The second search result including the corresponding object satisfying the first object feature element is received from the first analysis server and the second search result including the corresponding object satisfying the second object feature element is received from the first analysis server. Received from the analysis server of
A third search result obtained by excluding the second search result from the first search result is sent to the user terminal.
The user terminal is
Display the search result screen including the third search result sent from the integrated server.
Investigation support system. The first search result includes identification information and imaging time information of a camera that has imaged the corresponding object satisfying the first object feature element.
The second search result includes identification information and imaging time information of a camera that has imaged the corresponding object satisfying the second object feature element.
The integrated server
The third search result is extracted based on the second search result in which the identification information of the camera and the imaging time information included in each of the first search result and the second search result are common. ,
The investigation support system according to claim 15. The user terminal is
A search condition with exclusions in which the first object feature element or a part of the object feature elements constituting the second object feature element is designated as a search exclusion target is sent to the integrated server.
The integrated server
The search request for the corresponding object satisfying the search condition with exclusion is sent to the first analysis server or the second analysis server, and the search result from the first analysis server or the second analysis server is received and described. Send to user terminal,
The investigation support system according to claim 15. An investigation support method performed by an investigation support system that includes a user terminal and an integrated server that is communicably connected to at least the first analysis server and the second analysis server.
When the integrated server receives a search condition including a first object feature element related to an incident or the like and a second object feature element related to the incident or the like from the user terminal, a search for the corresponding object satisfying the first object feature element is performed. A request is sent to the first analysis server, and a search request for the corresponding object satisfying the second object feature element is sent to the second analysis server.
The integrated server receives a first search result including the corresponding object satisfying the first object feature element from the first analysis server and a second search including the corresponding object satisfying the second object feature element. Receive the result from the second analysis server
The integrated server sends a third search result obtained by excluding the second search result from the first search result to the user terminal.
The user terminal displays a search result screen including the third search result sent from the integrated server.
Investigation support method. It is an investigation support system in which a user terminal and a plurality of analysis servers are connected to each other so as to be able to communicate with each other via an integrated server.
Each of the plurality of analysis servers
It is connected to each of multiple cameras and holds a collation list of objects that are different from the objects that can be searched by other analysis servers regarding incidents, etc.
Each of the plurality of analysis servers
When an object that satisfies the collation list held by itself is detected by analyzing the image captured by the camera, the integrated server sends a detection notification including information on the event name indicating the detection of the object, the detection date and time, and the position of the camera. To the user terminal via
The user terminal is
The event detection screen including the event name, the detection date and time, and the information on the placement position of the camera included in the detection notification sent from the analysis server is displayed.
Investigation support system. The user terminal is
An event list having items of the event name, the detection date and time, and the information of the camera placement position is included in the event detection screen and displayed.
The investigation support system according to claim 19. The user terminal is
Each time the detection notification is received from the analysis server, the event name, the detection date and time, and the information items of the camera placement position included in the detection notification are added to the event list and updated.
The investigation support system according to claim 20. The user terminal is
The map has map data of an area in which each of the plurality of cameras is arranged, and associates the event name, the number of events, and the detection date and time detected based on the captured image of the cameras with each arrangement position of the cameras. Superimpose on the data and display it on the event detection screen.
The investigation support system according to claim 19. The user terminal is
Based on the designation of the detection date and time, the output window of the live image of the camera at the arrangement position corresponding to the specified detection date and time is included in the event detection screen and displayed.
The investigation support system according to claim 22. The user terminal is
The map which has map data of the area where each of the plurality of cameras is arranged and superimposes the arrangement position of the camera corresponding to the designated item based on the designation of any item of the event list. The event details window having at least the data and the play button of the image captured by the camera on the detection date and time is included in the event detection screen and displayed.
The investigation support system according to claim 21. The user terminal is
The output button of the live image of the camera at the arrangement position corresponding to the specified item is included in the event details window and displayed.
The investigation support system according to claim 24. This is an investigation support method executed by an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server.
Each of the plurality of analysis servers connects to each of the plurality of cameras.
Each of the plurality of analysis servers holds a collation list of objects that are different from the objects that can be searched by other analysis servers regarding incidents and the like.
When each of the plurality of analysis servers detects an object that satisfies the collation list held by itself by analyzing the captured image of the camera, the event name indicating the detection of the object, the detection date and time, and the arrangement position of the camera. A detection notification including the above information is sent to the user terminal via the integrated server.
The user terminal displays an event detection screen including information on the event name, detection date and time, and camera placement position included in the detection notification sent from the analysis server.
Investigation support method. It is an investigation support system in which a user terminal and a plurality of analysis servers are connected to each other so as to be able to communicate with each other via an integrated server.
The first analysis server among the plurality of analysis servers is
When a specific action related to an incident or the like is detected by analyzing the image captured by the camera, an alarm notification including at least the captured image of the camera in which the action is detected is sent to the user terminal via the integrated server.
The user terminal is
An event notification screen including the captured image included in the alarm notification sent from the first analysis server and the person analysis button of the actor of the action reflected in the captured image is displayed.
Based on the designation of the person analysis button, the person analysis of the actor reflected in the captured image is requested via the integrated server.
The second analysis server among the plurality of analysis servers is
Based on the request from the user terminal, the person analysis result of the actor reflected in the captured image is sent to the user terminal via the integrated server.
The user terminal is
An analysis result screen including a cutout image of the actor included in the person analysis result sent from the second analysis server and a tracking button of the person reflected in the cutout image is displayed.
Based on the designation of the tracking button, the blacklist registration of the person corresponding to the tracking button is requested via the integrated server.
The second analysis server is
Based on the request from the user terminal, the cutout image of the person corresponding to the tracking button is registered in the blacklist data of the second analysis server.
Investigation support system. The second analysis server is
When a person who is the same as or similar to the person registered in the blacklist data is detected by analyzing the image captured by the camera, the detection including the captured image showing the detected person, the detection date and time, and the information on the position of the camera is detected. The notification is sent to the user terminal via the integrated server,
The user terminal is
A person tracking operation screen including the captured image, the detection date and time, and the information on the placement position of the camera included in the detection notification sent from the second analysis server is displayed.
The investigation support system according to claim 27. The user terminal is
A tracking target list showing a cut-out image of the person corresponding to the tracking button is included in the person tracking operation screen and displayed.
The investigation support system according to claim 28. The user terminal is
An event list having items of the action name, the detection date and time of the action, and the information of the camera placement position is included in the person tracking operation screen and displayed.
The investigation support system according to claim 28. The user terminal is
Each time the detection notification is received from the second analysis server, items of the action name, the detection date and time of the action, and the information of the camera placement position included in the detection notification are added to the event list. Update,
The investigation support system according to claim 30. The user terminal is
It has map data of the area where the camera is arranged, and the captured image in which the action is detected and the detection date and time of the action are associated with the arrangement position of the camera and superimposed on the map data to track the person. Display by including in the operation screen,
The investigation support system according to claim 28. The user terminal is
Based on the designation of the captured image, the cut-out image of the actor of the action reflected in the designated captured image and the cut-out image of the person registered in the blacklist data corresponding to the action are arranged side by side. Display the event details window by including it in the person tracking operation screen.
The investigation support system according to claim 32. The third analysis server among the plurality of analysis servers is
Based on the request from the user terminal, the face analysis result of the person of the actor reflected in the captured image is sent to the user terminal via the integrated server.
The user terminal is
The analysis result screen including the cut-out image of the actor's face included in the face analysis result sent from the third analysis server and the face tracking button reflected in the cut-out image is displayed.
Based on the designation of the tracking button, blacklisting of the face corresponding to the tracking button is requested via the integrated server.
The third analysis server is
Based on the request from the user terminal, the cut-out image of the face corresponding to the tracking button is registered in the blacklist data of the third analysis server.
The investigation support system according to claim 27. This is an investigation support method executed by an investigation support system in which a user terminal and a plurality of analysis servers are communicably connected via an integrated server.
When the first analysis server among the plurality of analysis servers detects a specific action related to an incident or the like by analyzing the image captured by the camera, the alarm notification including at least the captured image of the camera in which the action is detected is sent. Send to the user terminal via the integrated server
The user terminal displays an event notification screen including the captured image included in the alarm notification sent from the first analysis server and the person analysis button of the actor of the action reflected in the captured image.
Based on the designation of the person analysis button, the user terminal requests the person analysis of the actor reflected in the captured image via the integrated server.
Based on the request from the user terminal, the second analysis server among the plurality of analysis servers sends the person analysis result of the actor reflected in the captured image to the user terminal via the integrated server.
The user terminal displays an analysis result screen including a cutout image of the actor included in the person analysis result sent from the second analysis server and a tracking button of the person reflected in the cutout image.
Based on the designation of the tracking button, the user terminal requests the blacklist registration of the person corresponding to the tracking button via the integrated server.
Based on the request from the user terminal, the second analysis server registers a cutout image of the person corresponding to the tracking button in the blacklist data of the second analysis server.
Investigation support method.

Images