JP2023039504A - Parameter browsing system, parameter adjusting system, server, and server control program - Google Patents

Abstract

To provide a parameter browsing system, a parameter adjustment system, a server, and a server control program that allow a user such as a system operation manager to easily input an appropriate condition setting parameter.SOLUTION: A parameter browsing system allows a user to detect past visualization statistical information similar to visualization statistical information based on an imaged image of a camera to be currently set (S15), and output to a PC display a screen including a condition setting parameter for imaged image analysis that is applied to a past imaged image that is a basis of the detected past visualization statistical information (S16). The system allows the user to refer to the condition setting parameter applied to the imaged image that is a basis of the past visualization statistical information, input the condition setting parameter for analyzing the current imaged image of the camera, and then easily input an appropriate condition setting parameter (e.g., human detection frame).SELECTED DRAWING: Figure 12

Description

The present invention relates to a parameter browsing system, a parameter adjusting system, a server, and a server control program.

Conventionally, there are known devices and systems that perform analysis processing such as object detection on frame images captured by a camera such as an AI camera using a trained neural network model (for example, Patent Document 1 reference).

Japanese Patent No. 6573297

It is difficult to set conditions such as an object detection frame for a camera to be set in an apparatus or system that performs analysis processing such as object detection on a frame image captured by a conventional camera as described above. For example, when setting the detection frame for people entering the store within the angle of view of the camera to be set, if you try to input it in a form that the computer can understand, the detection frame will be inside the angle of view of the camera to be set. It is necessary to enter the numerical coordinate information of the four vertices of , but the angle of view, the way people enter (into the angle of view), and the way people and objects are captured vary for each camera. It is difficult for a person (user) such as an administrator of the system to appropriately input condition setting parameters such as the numerical coordinate information of the detection frame.

Also, even if it is possible to appropriately input the condition setting parameters such as the numerical coordinate information of the detection frame, how correct (appropriate) is the condition setting by inputting the above parameters? In order to confirm this, it is necessary for a user such as a system administrator to compare and inspect the analysis result of the captured image (frame image) using the above condition setting parameters and the actual image. There was a problem that it took a lot of man-hours to confirm the accuracy of the condition setting.

The present invention is intended to solve the above problems, and a parameter viewing system, a parameter adjustment system, a server, and a server that enable a user such as a system administrator to easily input appropriate condition setting parameters. The purpose is to provide a control program. Another object of the present invention is to provide a parameter adjustment system that allows a user to easily check the accuracy of condition setting by inputting condition setting parameters.

In order to solve the above-mentioned problems, the parameter browsing system according to the first aspect of the present invention includes display means, an image captured by an edge-side camera, and an edge-side image analysis device for a frame image included in the captured image. Visualization statistical information generation means for statistically processing a superimposed video superimposed with a result of recognition processing to generate visualization statistical information; and statistical information storage means for storing the visualization statistical information generated by the visualization statistical information generation means. , a detecting means for detecting past visualization statistical information similar to the visualization statistical information based on the image captured by the camera to be set this time, from the past visualization statistical information accumulated in the statistical information accumulation means; parameter output means for outputting, to the display means, a screen including condition setting parameters for analysis of the captured image applied to the past captured image based on the past visualization statistical information detected by the detection means; .

In this parameter viewing system, the detection means includes a vector corresponding to the movement direction of the person obtained from the visualization statistical information based on the captured image of the camera to be set this time, and the past accumulated in the statistical information accumulation means. By comparing the vector corresponding to the moving direction of the person obtained from the visualization statistical information of , and detecting the past visualization statistical information similar to the visualization statistical information based on the image captured by the camera targeted for this setting. may

In this parameter browsing system, the detection means displays the visualization statistical information based on the captured image of the camera to be set this time and the plurality of past visualization statistical information accumulated in the statistical information accumulation means to the display means. The parameter viewing system includes a display control means for controlling to display the above, and the parameter viewing system selects the image captured by the camera to be set this time from among the plurality of past visualization statistical information displayed on the display means. A selection input means for selecting the past visualization statistical information most similar to the based visualization statistical information may be further provided.

A parameter adjustment system according to a second aspect of the present invention comprises display means, reception means for receiving an image captured by a camera to be set this time, and conditions for analyzing the current captured image of the camera to be set this time. The setting parameters are superimposed on the image including the image captured by the camera to be set this time, or the visualization statistical information based on the image captured by the camera to be set this time, and displayed on the display means. and adjustment input means for adjusting and inputting the condition setting parameters displayed on the display means.

In this parameter adjustment system, re-analyzing means for re-analyzing the photographed image of the camera to be set this time, received by the receiving means, using the condition setting parameters after adjustment by the adjustment input means; It is desirable to further include re-analysis result display control means for controlling to display the analysis result by the re-analysis means on the display means.

A server according to a third aspect of the present invention statistically processes a superimposed video obtained by superimposing an image captured by an edge-side camera and a recognition process performed by an edge-side image analysis device on a frame image included in the captured image. Visualization statistical information generation means for generating visualization statistical information, statistical information accumulation means for accumulating visualization statistical information generated by the visualization statistical information generation means, and past visualization statistics accumulated in the statistical information accumulation means From information, a detection means for detecting past visualization statistical information similar to the visualization statistical information based on the image taken by the camera to be set this time, and based on the past visualization statistical information detected by the detection means and parameter output means for outputting to the display means a screen including condition setting parameters for photographed image analysis applied to the past photographed image.

In this server, the detection means includes a vector corresponding to the moving direction of the person obtained from the visualization statistical information based on the image captured by the camera to be set this time, and the past visualization accumulated in the statistical information accumulation means. By comparing the vector corresponding to the moving direction of the person obtained from the statistical information, it is possible to detect past visualization statistical information similar to the visualization statistical information based on the image captured by the camera to be set this time. good.

A server control program according to a fourth aspect of the present invention controls the server so as to perform processing for finding condition setting parameters for photographed image analysis suitable for the photographed image of the camera to be set this time. A program that causes the server to statistically process superimposed video obtained by superimposing an image captured by an edge-side camera and a result of recognition processing of a frame image included in the captured image by an image analysis device on the edge side. , visualization statistical information generation means for generating visualization statistical information, statistical information accumulation means for accumulating visualization statistical information generated by the visualization statistical information generation means, and past visualization statistical information accumulated in the statistical information accumulation means Among them, a detection means for detecting past visualization statistical information similar to the visualization statistical information based on the image captured by the camera to be set this time, and the past visualization statistical information detected by the detection means. It functions as parameter output means for outputting to the display means a screen including condition setting parameters for photographed image analysis applied to past photographed images.

In this server control program, the detection means obtains a vector corresponding to the moving direction of a person from the visualization statistical information based on the captured image of the camera to be set this time, and the past accumulated in the statistical information accumulation means. By comparing the vector corresponding to the moving direction of the person obtained from the visualization statistical information of , and detecting the past visualization statistical information similar to the visualization statistical information based on the image captured by the camera targeted for this setting. may

According to the parameter browsing system according to the first aspect, the server according to the third aspect, and the server control program according to the fourth aspect of the present invention, the camera to be set this time (hereinafter abbreviated as "current camera") Detected past visualization statistics similar to the image-based visualization statistics of and applied to the past image captures from which the detected past visualization statistics were based. is output to the display means so that a user such as a system administrator can view it. Here, the past visualization statistical information similar to the visualization statistical information based on the image captured by the current camera is obtained by: There is a high possibility that this is the visualization statistical information using the result of recognition processing similar to the recognition processing performed on the image captured by the camera this time. For this reason, the user refers to the condition setting parameters applied to the captured image that was the basis of the past visualization statistical information (similar to the visualization statistical information based on the captured image of the camera this time) viewed above. By inputting the condition setting parameters for analyzing the captured image of the camera, the user such as the system administrator can easily input the appropriate condition setting parameters (for example, the human detection frame). can.

Further, according to the parameter adjustment system according to the second aspect of the present invention, the condition setting parameters for analysis of the current captured image of the current camera are set to an image including the current captured image of the camera, or the current captured image of the camera. is superimposed on the visualization statistical information and displayed on the display means, and a user such as a system administrator can perform adjustment input of the displayed condition setting parameters. This allows the user to adjust and input the condition setting parameters while viewing the current condition setting parameters superimposed on the image including the image captured by the current camera or the visualization statistical information based on the image captured by the current camera. Therefore, it is possible to easily input appropriate condition setting parameters for the current camera.

1 is a block configuration diagram showing a schematic configuration of an image analysis system according to one embodiment of the present invention; FIG. FIG. 2 is a block diagram showing a schematic hardware configuration of the signage in FIG. 1; FIG. 2 is a block diagram showing a schematic hardware configuration of an analysis box in FIG. 1; FIG. 2 is a block diagram showing a schematic hardware configuration of a management server in FIG. 1; FIG. 2 is a functional block configuration diagram of the same management server and a personal computer; FIG. 4 is a block diagram showing the flow of generation and display of analysis result superimposed video in the image analysis system; FIG. 4 is a block diagram showing the flow of statistical processing of image analysis results in the image analysis system and viewing/editing of detection conditions superimposed on video. The figure which shows the present count status confirmation screen in the said image analysis system. 4 is a flowchart of simulation processing for setting a detection area (detection frame) in the image analysis system; FIG. 10 is a diagram showing a visitor count area setting simulation screen used in the simulation process; The figure which shows the visitor count accuracy ranking display screen in the said image analysis system. 10 is a flowchart of processing for searching past visualization statistical information for visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time; The figure which shows the similar visualization statistical information search screen in the said image analysis system. The figure which shows the similar visualization statistical information search result display screen in the said image analysis system. The figure which shows the similar pattern image setting display screen in the said image analysis system. The figure which shows the similar visualization statistical information selection screen in the said image analysis system.

Hereinafter, a parameter viewing system, a parameter adjusting system, a server, and a server control program according to embodiments embodying the present invention will be described with reference to the drawings. FIG. 1 is a block configuration diagram showing a schematic configuration of an image analysis system 10 (corresponding to "parameter viewing system" and "parameter adjustment system" in the claims) according to this embodiment. In this embodiment, a plurality of fixed cameras (surveillance cameras) 2, which are network cameras for monitoring that photograph a predetermined photographing area, an analysis box 3 that analyzes images from the fixed cameras 2, a built-in camera 4 (or An example in which a plurality of signages 5, which are digital signage tablet terminals equipped with web cameras, are arranged in a store S such as a chain store will be described. As shown in FIG. 1, the image analysis system 10 includes the plurality of fixed cameras 2, the analysis box 3, the plurality of signages 5, the WiFi AP (WiFi Access Point) 6 and , a hub 7, a router 8, a management server 1 (corresponding to a "server" in the claims) arranged on the cloud C, and a personal computer used by a user such as a system administrator. ) 9. Note that the camera on the signage 5 side may be the built-in camera 4 disposed within the housing of the signage 5 or may be a web camera attached to the signage 5 . An example in which the camera is the built-in camera 4 will be mainly described.

The above fixed camera 2 has an IP address and can be directly connected to the network. As shown in FIG. 1, the analysis box 3 is connected to a plurality of fixed cameras 2 via a LAN (Local Area Network) and a hub 7, and analyzes images input from each of these fixed cameras 2. . Specifically, the analysis box 3 performs object detection processing (customer and customer face detection processing) for images input from each of the fixed cameras 2, and object detection processing for the image of the object detected by this object detection processing. Recognition processing (recognition processing such as customer attribute (gender and age (age)) estimation) is performed.

The signage 5 is mainly installed on the product shelf in the store S, and displays contents such as advertisements for customers visiting the store S on the touch panel display 14 (see FIG. 2), and also displays the content from the built-in camera 4. object detection processing (customer or customer's face detection processing) for the frame image of the above, object recognition processing for the image of the object detected by this object detection processing, and the like.

The above-described management server 1 is installed in a management department (headquarters, etc.) of stores including the store S, and manages a large number of fixed cameras 2, analysis boxes 3, and signage 5 arranged in each store. Specifically, the management server 1 performs control such as installation of application packages to the analysis box 3 and signage 5 of each store, starting and stopping of the fixed camera 2 connected to the analysis box 3, and the like. Also, the management server 1 receives an image captured by the fixed camera 2 or the built-in camera 4 and analysis results including recognition processing results for the captured image from the image analysis device (analysis box 3 or signage 5) on the edge side. Then, using the received captured image and the above analysis result, to find condition setting parameters for captured image analysis suitable for the captured image of the camera (fixed camera 2 or built-in camera 4) to be set this time. process. As shown in FIG. 1, a personal computer 9 used by a user such as an administrator of the image analysis system 10 is connected to the management server 1. The user uses the personal computer 9 to access the management server 1 (portal ) to find the condition setting parameters for captured image analysis suitable for the camera (fixed camera 2 or built-in camera 4) to be set this time (browsing past condition setting parameters, etc.). be able to.

Next, referring to FIG. 2, the hardware configuration of the tablet-type signage 5 will be described. In addition to the built-in camera 4, the signage 5 includes a SoC (System-on-a-Chip) 11, a touch panel display 14, a speaker 15, various programs including application packages, and various programs including condition setting parameters. , a communication unit 17, a secondary battery 18, a charging terminal 19, an image captured by the built-in camera 4, and an analysis (result) superimposed image (see FIG. 6, etc.), which will be described later. and a storage 20 such as an SD memory card. The SoC 11 includes a CPU 12 that controls the entire device and performs various calculations, and a GPU 13 that is used for inference processing of various learned DNN (Deep Neural Networks) models. The GPU 13 corresponds to the DNN calculation chip 69 shown in FIGS. 6 and 7. FIG.

The programs stored in the memory 16 include learned DNN models 70 (various learned inference models) shown in FIGS. The communication unit 17 has a communication IC and an antenna. The signage 5 is connected to the management server 1 on the cloud C via the communication unit 17 and the network. The secondary battery 18 is a battery such as a lithium-ion battery that can be used repeatedly by charging. Supplied to each part of the signage 5 .

Next, the hardware configuration of the analysis box 3 will be described with reference to FIG. The analysis box 3 includes a CPU 21 that controls the entire apparatus and performs various calculations, a hard disk 22 that stores various data and programs, a RAM (Random Access Memory) 23, and a DNN (Deep Neural Networks) inference processor. It has chips (hereinafter abbreviated as “chips”) 24 a to 24 h and a communication control IC 25 . The CPU 21 is a general general-purpose CPU or a CPU designed to enhance parallel processing performance in order to simultaneously process multiple video streams. The data stored in the hard disk 22 includes video data obtained by decoding (data of) video streams input from each of the fixed cameras 2, analysis result superimposed video data to be described later with reference to FIG. and analysis data 58 (same as video metadata 74 in FIG. 7). In addition, the programs stored in the hard disk 22 include trained DNN models for inference processing such as human detection processing, face detection processing, and attribute recognition processing (learned DNN models for various inference processing: the DNN shown in FIGS. 6 and 7). model 57), an analysis box OS program, a VMS (Video Management System) program, a video analysis program, and an analysis superimposed video generation program.

Next, referring to FIG. 4, the hardware configuration of the management server 1 will be described. The management server 1 includes a CPU 31 that controls the entire apparatus and performs various calculations, a hard disk 32 that stores various data and programs (corresponding to "statistical information storage means" in the claims), and a RAM (Random Access Memory) 33. , a display 34, an operation unit 35, and a communication unit 36 (corresponding to "receiving means" in the claims).

The programs stored in the hard disk 32 include a server control program 37, a portal (program for) 38, and a dashboard 39, and the data stored in the hard disk 32 include statistical data 40. be The server control program 37 is a program for controlling the management server 1 . This server control program 37 performs processing (accumulated parameters) for finding condition setting parameters for photographed image analysis suitable for the photographed image of the camera to be set this time (hereinafter sometimes abbreviated as "current camera"). viewing of the condition setting parameters in the past) and adjustment (editing) of the condition setting parameters of the current camera. The portal 38 is a so-called corporate portal (software for collectively displaying various information, applications, etc. scattered within a company on a computer screen in order to efficiently search for and use them). ) is a kind of The processing that can be performed using this portal 38 is described in the discussion of FIGS. 6 and 7. FIG. The dashboard 39 is software for aggregating and visualizing statistical information such as customer attributes, length of stay, tracking results of customer behavior, and the like. The dashboard 39 extracts the counts of visitors, passers-by, stopers, etc. from the statistical data 40 based on requests from the user using the personal computer 9, and displays them on the display 50b of the personal computer 9 (see FIG. 5). Used for display processing.

FIG. 5 shows functional blocks of the management server 1 and the personal computer 9 described above. The CPU 31 of the management server 1 includes, as functional blocks, a visualization statistical information generation unit 41, a detection unit 42, a parameter output unit 43, a parameter display control unit 45, a reanalysis unit 46, and a reanalysis result display control unit 47. and Further, the detection unit 42 described above includes a display control unit 44 . The above visualization statistical information generation unit 41, the detection unit 42, the parameter output unit 43, the display control unit 44, the parameter display control unit 45, the reanalysis unit 46, and the reanalysis result display control unit 47, They respectively correspond to visualized statistical information generation means, detection means, parameter output means, display control means, parameter display control means, re-analysis means, and re-analysis result display control means in the claims. The personal computer 9 also includes an operation unit 50a (corresponding to "adjustment input means" and "selection input means" in the claims) and a display 50b (corresponding to "display means" in the claims).

The above-mentioned visualization statistical information generation unit 41 includes the captured image of the edge-side camera (fixed camera 2 or built-in camera 4) and the edge-side image analysis device (analysis box 3 or signage 5) in the captured image. Statistical processing is performed on the superimposed video obtained by superimposing the result of recognition processing on the frame image, and visualization statistical information is generated. This visualization statistical information refers to a superimposed video composed of each superimposed image obtained by superimposing each frame image included in the captured images for a predetermined time and the result of recognition processing for each of these frame images. It is the image of the result of statistical processing. For example, the face detection summary image 89 (a type of visualization statistical information) shown in FIG. 13 is obtained by superimposing each frame image included in the images captured by a certain camera for one hour and the face detection result for each frame image. It is a heat map created based on each superimposed image. In this example, a portion with a high face detection frequency in the images captured by the corresponding camera for one hour is shown in a dark color (dark red, etc. in the actual image).

The statistical data 40 including the visualized statistical information 48 generated by the visualized statistical information generator 41 is stored in the hard disk 32 . The above statistical data 40 includes condition setting parameters 49 (detection frames, etc.) for each edge-side camera (fixed camera 2 or built-in camera 4).

The detection unit 42 in FIG. 5 selects from the past visualization statistical information accumulated in the hard disk 32, similar to the visualization statistical information based on the captured image of the camera (fixed camera 2 or built-in camera 4) to be set this time. Detect past visualization statistics that have In addition, the parameter output unit 43 displays a screen including condition setting parameters (detection frame, etc.) for captured image analysis applied to the past captured image that is the basis of the past visualization statistical information detected by the detection unit 42. , to the display 50b of the personal computer 9. Based on the instruction operation of the user of the personal computer 9, the display control unit 44, as shown in FIG. face detection total image) and a plurality of past visualized statistical information accumulated in the hard disk 32 are controlled to be displayed on the display 50b of the personal computer 9. FIG.

The parameter display control unit 45 reads from the hard disk 32 the condition setting parameters 49 (detection frame, etc.) for analyzing the current photographed image of the camera to be set this time, and performs the shooting of the camera to be set this time. It is superimposed on the visualization statistical information based on the image and controlled to be displayed on the display 50b of the personal computer 9. FIG. In the example shown in FIG. 10, the parameter display control unit 45 sets the condition setting parameter (detection frame) for analysis of the current photographed image of the camera to be set this time based on the photographed image of the camera to be set this time. It is displayed on the display 50b of the personal computer 9 superimposed on the visualized statistical information (movement trajectory).

When the user performs adjustment input of the condition setting parameters displayed on the display 50b using the operation unit 50a of the personal computer 9, the reanalysis unit 46 uses the condition setting parameters after adjustment (input). , the image captured by the camera to be set this time (received by the communication unit 36) is analyzed again. Further, the reanalysis result display control unit 47 controls to display the analysis result by the reanalysis unit 46 on the display 50b of the personal computer 9 used by the user.

Next, with reference to FIG. 6, the flow of generation and display of analysis result superimposed video performed in this image analysis system 10 will be described. First, the analysis result superimposed video generation processing on the analysis box 3 side will be described. The CPU 21 of the analysis box 3 decodes the video stream input from the fixed camera 2 using the VMS 51 of FIG. store in Next, the CPU 21 of the analysis box 3 uses the video analysis program 53 to divide the collected data 52 (including personal information) into frame images 55 (including personal information) and store them in the main memory 54 (in FIG. 3). (equivalent to RAM 23). Then, the CPU 21 of the analysis box 3 uses the video analysis program 53 to input each of the frame images 55 to the (learned) DNN model 57 for various image recognition, and the DNN model 57 converts each frame image into 55 is subjected to image recognition processing (human detection, attribute analysis processing, etc.), and analysis data 58 including the result of the above recognition processing is output. A DNN calculation chip 56 (corresponding to the (inference) chips 24a to 24h in FIG. 3) is used for image recognition processing by the DNN model 57 described above. Further, the frame image 55 is erased from the main memory 54 immediately after being input to the DNN model 57 . Note that the video analysis program 53 includes the DNN model 57 described above, as indicated by the dashed line in FIG.

The above analysis data 58 is text data indicating at what position and with what attributes a person is present in a frame image of which camera and at what time, and does not contain personal information. The analysis data 58 includes the result of counting visitors and passers-by, which is performed by the video analysis program 53 using the human detection result by the DNN model 57 and the condition setting parameters 59 .

After outputting the analysis data 58 using the video analysis program 53, the CPU 21 of the analysis box 3 uses the analysis superimposed video generation program 60 to generate an analysis result superimposed video. Specifically, the CPU 21 of the analysis box 3 reads the collected data 52 (including personal information) from the hard disk 22 using the VMS 51 in accordance with the analysis superimposed video generation program 60, and stores the collected data 52 as follows: An analysis result superimposed image is generated by superimposing the recognition results (human detection result and attribute analysis result) by the DNN model 57 . Then, the CPU 21 of the analysis box 3 uses the VMS 51 according to the analysis superimposed video generation program 60 to store the above analysis result superimposed video in the hard disk 22 as part of the collected data 52 (including personal information). Therefore, the collected data 52 includes the video (data) obtained by decoding the video stream input from the fixed camera 2 and the analysis result superimposed video. An example of the analysis result superimposed video is an analysis result superimposed video 63 shown in FIG. This analysis result superimposed video 63 is a video obtained by superimposing the human detection result by the DNN model 57 on the collected data 52 (video data of the entrance camera).

The analysis result superimposed video generation process on the signage 5 side is basically the same as the analysis result superimposed video generation process on the analysis box 3 side. The collected data 65 on the signage 5 side, the video analysis program 66, the main memory 67, the frame image 68, the DNN calculation chip 69, the DNN model 70, the analysis data 71, the condition setting parameters 73, and the analysis superimposed video generation program 72 are, respectively, Corresponds to collected data 52 on analysis box 3 side, video analysis program 53, main memory 54, frame image 55, DNN calculation chip 56, DNN model 57, analysis data 58, condition setting parameters 59, and analysis superimposed video generation program 60. . Also, the storage 20 on the signage 5 side plays the same role as the hard disk 22 on the analysis box 3 side. Although not shown in FIGS. 6 and 7, the signage 5 also has a program having the same functions as the VMS 51 on the analysis box 3 side.

Next, processing on the management server 1 side and the personal computer 9 side in FIG. 6 will be described. The management server 1 has an analysis result superimposed video display program 61 and a parameter adjustment program 62 . These programs are included in the server control program 37 in FIG. A user such as an operation manager on the personal computer 9 side (a user of the portal 38 shown in FIG. 4) uses the operation unit 50a of the personal computer 9 to instruct the management server 1 to display the analysis result superimposed video. Then, the CPU 31 of the management server 1 uses the analysis result superimposed image display program 61 to access the analysis box 3, and uses the VMS 51 to read the analysis data contained in the collected data 52 from the hard disk 22 of the analysis box 3. The result superimposed video is read, or the signage 5 is accessed to read the analysis result superimposed video included in the collected data 65 from the storage 20 of the signage 5 . Then, a screen including this analysis result superimposed image (for example, the current count situation confirmation screen 81 shown in FIG. 8) is output (displayed) on the display 50b of the personal computer 9. FIG. As a result, a user such as an operation manager can view a video (analysis result superimposed video) superimposed on video data (collected data) input from a camera (fixed camera 2 or built-in camera 4), and see the current Analysis status can be checked. In addition, although the details will be described later, when a user such as an operation manager uses the operation unit 50a of the personal computer 9 to perform a parameter adjustment input operation on the management server 1, the CPU 31 of the management server 1 , the parameter adjustment program 62 is used to rewrite the condition setting parameters 59 on the analysis box 3 side or the condition setting parameters 73 on the signage 5 side. As a result, users such as operation managers (including system development contractors and system operation contractor workers) can use the condition setting parameters of the camera to be set this time (condition setting parameters 59, or conditioning parameters 73) can be readily made.

Next, referring to FIG. 7, statistical processing of image analysis results (analysis data) performed in this image analysis system 10 and detection conditions (condition settings) superimposed on video (visualization statistical information or analysis result superimposed video) parameters) will be explained. The processing program 76 in FIG. 7, the video search/display program 78, the detection condition data extraction program 79, and the viewing/editing program 80 for the detection conditions superimposed on the video (hereinafter referred to as "detection condition viewing/editing program" ) are programs included in the portal 38 shown in FIG.

First, statistical processing of the above image analysis results will be described. Using the processing program 76, the CPU 31 of the management server 1 uses the analysis data 58 or the analysis data 71 (in FIG. 7, , "video metadata 74" or "video metadata 75") are processed (counted and statistically processed), and the processing results are stored in the database 77 in the hard disk 32 (see FIG. 4) as statistical data 40. Store. Note that the video metadata 74 on the analysis box 3 side and the video metadata 75 on the signage 5 side are not the analysis (recognition) results per frame image per se, but are processed and aggregated (summarized) for each person. ) is sent to the management server 1 and processed by the processing program 76 . The above-mentioned statistical data 40 is, for example, data such as the number of people visiting a store between 9:00 and gender-age distribution. be. As shown in FIG. 5, the statistical data 40 includes visualization statistical information 48 and condition setting parameters 49 (detection frames, etc.) for each edge-side camera (fixed camera 2 or built-in camera 4). .

In this image analysis system 10, as described above, personal information (collected data 52 and collected data 65) is stored only on the edge side (analysis box 3 and signage 5), and on the cloud side (management server 1) , only statistical data 40 (including condition setting parameters 49), which is non-personal information, is stored. As a result, the operation of the cloud side (management server 1) can be entrusted to a company other than the store operating company (system development contractor, system operation contractor, etc.).

Next, the flow of viewing/editing processing of detection conditions (condition setting parameters) superimposed on video in the image analysis system 10 will be described. When a user such as an operation manager uses the operation unit 50a of the personal computer 9 to instruct the management server 1 to view and edit detection conditions (condition setting parameters such as detection frames), the management server 1 First, the CPU 31 uses the image search/display program 78 to retrieve images of past collected data 52 and 65 stored in the analysis box 3 and the signage 5 (including analysis result superimposed images) and images stored in the database 77 . The visualization statistical information included in the statistical data 40 is searched, and the video (including the visualization statistical information) of the search result is extracted. Next, the CPU 31 of the management server 1 uses the detection condition data extraction program 79 to extract the current detection conditions (condition setting parameters 49 such as the detection frame) applied to the extracted video of the search result, Extract from data 40 . Then, the CPU 31 of the management server 1 uses the detection condition viewing/editing program 80 to create a video or image in which the extracted current detection conditions are superimposed on the search result video (including the visualization statistical information). , is displayed on the display 50b of the personal computer 9, and the user's editing operation of the detection conditions (detection frame, etc.) using the operation unit 50a of the personal computer 9 is accepted.

Next, the user interface employed in this image analysis system 10 will be described with reference to FIGS. 8 to 16. FIG. FIG. 8 shows a current count status confirmation screen 81 . This screen is a screen including an analysis result superimposed image 63 which is a kind of analysis result superimposed image described in the explanation of FIG. This analysis result superimposed image 63 is obtained by superimposing the human detection result by the DNN model 57 of FIG. It is a video. A bounding box 82 corresponding to each person detected by the DNN model 57 is displayed on the analysis result superimposed image 63 . In addition, in the count result display field 81a in the current count status confirmation screen 81, the visitor count result based on the current visitor count condition of the entrance camera is displayed. By using the current count status confirmation screen 81 as described above, it is possible to easily visualize the detection status of visitors and the count results based on the current count conditions.

Further, the user interface employed in this image analysis system 10 includes a simulation screen for setting condition setting parameters (detection frame, etc.). FIG. 9 is a flow chart of a simulation process for setting condition setting parameters (detection frame, etc.) using this simulation screen. FIG. 10 shows a visitor count area setting simulation screen 83, which is an example of the above simulation screen. In the visitor count area setting simulation screen 83, the detection frame (setting detection frame 85) of the visitor count area corresponds to the condition setting parameter in FIG. On this visitor count area setting simulation screen 83, the visualization statistical information of the camera to be set this time for a predetermined time period (movement trajectory total image 84 shown in FIG. 10, face detection total image 89 (see FIG. 13), or gender-age recognition tally image), a setting detection frame 85 (a kind of condition setting parameter) of the current store visitor count area of the camera to be set this time is superimposed and displayed. Here, in the example of FIG. 10, the movement locus aggregated image 84 is an image showing the movement locus created based on the human detection results for the images captured by the entrance camera from 9:00 to 10:00. Also, in the example of FIG. 13, the above-mentioned face detection total image 89 is the face detection result for the images taken from 9:00 to 10:00 by the entrance camera of the 〇〇 store (more precisely, each area in the taken image). This is a heat map created based on the frequency of face detection in Japan), and the gender and age recognition aggregate image above is the gender and age recognition result (more precisely , sex age recognition frequency in each area in the captured image). In addition, in the face detection aggregated image 89 (see FIG. 13) and the sex/age recognition aggregated image, the background image of the original captured image (the image obtained by removing the moving body (person) from the captured image) is displayed on the back side of the heat map. Is displayed.

Using the visitor count area setting simulation screen 83 of FIG. 10 as an example, the simulation process of setting the condition setting parameters (detection frame, etc.) using the simulation screen will be described along the flowchart of FIG. . When the user uses the operation unit 50a of the personal computer 9 to instruct the management server 1 to display the visitor count area setting simulation screen 83, the CPU 31 of the management server 1 displays the visitor count area setting simulation screen. 83 is displayed. Then, the user operates the camera selection button 83a, the time period selection button 83b, and the statistical information selection button 83c in FIG. When the camera, the acquisition time period of the inference result (analysis result), and the type of visualization statistical information to be displayed are selected (S1), the CPU 31 of the management server 1 selects the selected camera, the acquisition time period, and the visualization statistical information The current condition setting parameters (setting detection frame 85) of the selected camera are superimposed on the visualization statistical information corresponding to the type (in the example of FIG. 10, the moving trajectory total image 84), and displayed on the display 50b of the personal computer 9. At the same time, the analysis result (visitor count result) by the current setting detection frame 85 is displayed on the display 50b (count result display column 83h) (S2). In the process of S2, the CPU 31 of the management server 1 obtains the selected camera, the acquisition time period, the visualization statistical information corresponding to the type of visualization statistical information, and the current condition setting parameters of the selected camera (setting detection frame 85 ) is extracted from the statistical data 40 (see FIG. 7, etc.).

When the process of S2 is completed, the user such as the operation manager displays the current condition setting parameters (setting detection frame 85) displayed on the display 50b and the analysis result (current setting detection) using the condition setting parameters. (S3). As a result of this confirmation, when the user adjusts and inputs the condition setting parameters (setting detection frame 85) using the operation unit 50a (mouse, etc.) of the personal computer 9 (YES in S4), the CPU 31 of the management server 1 opens the analysis box. 3 or signage 5, the past captured image of the camera to be set this time (the captured image corresponding to the camera selected in S1 above and the acquisition time period) is acquired (S5), and the adjustment input of S4 is performed. Using the subsequent condition setting parameters (setting detection frame 85), the captured image of the current setting target camera obtained in S5 is re-analyzed (re-analyzed) (S6). Then, the re-analyzed visitor count result is displayed on the display 50b (count result display column 83h) of the personal computer 9 (S7).

The method of adjusting and inputting the condition setting parameters (setting detection frame 85) in S4 will be described more specifically. First, the user drags the vertex 85a of the setting detection frame 85 on the visitor count area setting simulation screen 83 using the mouse or the like of the personal computer 9, so that the setting detection frame 85 (setting detection area (visitor count area) frame) can be adjusted. The user can also use the keyboard of the personal computer 9 or the like to directly enter coordinates in the detection area adjustment input field 83f on the visitor count area setting simulation screen 83. 85 adjustments can be made. Also, on the moving locus tally image 84 of the visitor count area setting simulation screen 83, In addition to the current setting detection frame 85 of the camera to be set this time, a recommended detection frame 86 is displayed. When the user clicks the recommended detection area setting button 83g on the visitor count area setting simulation screen 83 using the mouse or the like of the personal computer 9, recommended The x-coordinate and y-coordinate values of each vertex 86a of the detection frame 86 are input. After adjusting and inputting the setting detection frame 85 by various methods as described above, when the user clicks the calculation button 83i using the mouse or the like of the personal computer 9, the camera to be set this time is displayed in the count result display field 81h. is displayed based on the visitor count conditions (including the setting detection frame 85) after adjustment input.

On the visitor count area setting simulation screen 83, the user operates the count type selection button 83d and movement direction selection button 83e using the mouse or the like of the personal computer 9 to select the count type among the visitor count conditions. It is also possible to adjust and input the direction of movement. After the user uses the mouse or the like of the personal computer 9 to adjust and input the visitor count conditions (set detection frame 85 (set detection area), count type, and movement direction) for the camera to be set this time. , the calculation button 83i is clicked, and if the count result (accuracy represented by) displayed in the count result display column 81h is satisfied, the apply button 83j is clicked to Person count conditions may apply. By clicking the apply button 83j, the visitor count conditions (setting detection frame (setting detection area), count type , and direction of movement) are rewritten.

The user interface employed in this image analysis system 10 includes a screen for displaying rankings of the accuracy of various counts (for example, counts of visitors, passers-by, and people who stop by). FIG. 11 shows a visitor count accuracy ranking display screen 87, which is an example of the count accuracy ranking display screen. This store visitor count accuracy ranking display screen 87 is a screen displaying the ranking of setting conditions (especially, setting detection frame) for store visitor counting that have been set in the past, with high counting accuracy under the setting conditions. On this visitor count accuracy ranking display screen 87, setting detection frames 87a and 87b are displayed in addition to accuracy, detection conditions, and uses (of detection) for each condition. In the image analysis system 10, by using a count accuracy ranking display screen such as the store visitor count accuracy ranking display screen 87, the same detection conditions among the setting conditions for counting (persons) that have been set in the past. You can refer to (browse) the setting conditions for which the count accuracy was high.

Next, referring to the flowchart of FIG. 12, using the visualization statistical information based on the captured image of the camera to be set in the past, the setting detection frame etc. to be applied to the captured image of the camera to be set this time. A method for setting the condition setting parameters of is explained. Using the communication unit 36, the CPU 31 of the management server 1 superimposes the captured image of the camera and the result of recognition processing on the frame image included in this captured image from the edge side (analysis box 3 or signage 5). When the video is received (S11), the visualization statistical information generation unit 41 (see FIG. 5) is used to statistically process the superimposed video for a predetermined time period to generate visualization statistical information (S12). Then, as shown in FIG. 5, the CPU 31 of the management server 1 accumulates the generated visualization statistical information 48 in the hard disk 32 as part of the statistical data 40 (S13).

When the user searches past visualization statistical information accumulated in the hard disk 32 of the management server 1 for past visualization statistical information similar to the visualization statistical information based on the image captured by the camera to be set this time. For example, a similar visualization statistical information search screen 88 shown in FIG. 13 is used. While the similar visualization statistical information search screen 88 is displayed on the display 50b of the personal computer 9, the user uses the operation unit 50a of the personal computer 9 to select the store selection button 88a, the camera selection button 88b, the time selection button 88c, and the visualization statistical information selection button 88d to select the type of desired visualization statistical information based on the captured image of the camera to be set this time in the desired time period. display 50b displays the selected shop, camera, time zone, and visualization statistical information (in the example of FIG. 13, face detection total image 89) corresponding to the type of visualization statistical information. In the example of FIG. 13, this face detection tally image 89 is the face detection result for the frame images included in the images taken from 9:00 to 10:00 by the camera at the entrance of the ○○ store (more precisely, in the above-mentioned photographed images). This is a heat map created based on the frequency of face detection in each area of .

In a state where the visualization statistical information such as the face detection aggregate image 89 is displayed on the similar visualization statistical information search screen 88, the user can select the To search for past visualization statistical information, click the similar visualization statistical information search button 88e on the similar visualization statistical information search screen 88 using the operation unit 50a (mouse, etc.) of the personal computer 9. FIG. By this click operation, when the user instructs to search for similar past visualization statistical information (YES in S14), the CPU 31 (the detection unit 42 (see FIG. 5)) of the management server 1 detects the data stored in the hard disk 32. Past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time is detected (searched) from the past visualization statistical information (S15).

Methods for retrieving similar visualization statistical information are roughly divided into two methods. In the first method, the CPU 31 (the detection unit 42 thereof) of the management server 1 automatically detects the past visualization statistical information that is most similar to the visualization statistical information based on the captured image of the camera to be set this time. This is a method of selecting and presenting to the user. In the second method, the CPU 31 of the management server 1 stores the visualization statistical information based on the captured image of the camera to be set this time and a plurality of past visualization statistical information accumulated in the hard disk 32 in the personal computer 9. Displayed side by side on the display 50b, the user uses the operation unit 50a (mouse, etc.) of the personal computer 9 to select the camera to be set this time from among the plurality of past visualization statistical information displayed on the display 50b. This is a method of selecting past visualization statistical information that is most similar to the visualization statistical information based on the captured image.

As an example of the first method described above (a method in which the CPU 31 of the management server 1 automatically detects visualization statistical information similar to the visualization statistical information of the camera to be set this time), face detection in FIG. In the case of the total image 89, the CPU 31 of the management server 1 selects the face detection total image 89 currently displayed on the similar visualization statistical information search screen 88 from among the past face detection total images accumulated in the hard disk 32. There is a method of automatically detecting a face detection tally image of a heat map with a pattern similar to the pattern of the heat map in , and presenting it to the user. Specifically, for example, when the user clicks the similar visualization statistical information search button 88e on the similar visualization statistical information search screen 88 shown in FIG. As shown in FIG. 14, the CPU 31 (the detection unit 42 of the CPU 31) of 1, from the face detection total image 89 (based on the captured image) of the camera to be set this time (same as the face detection total image 89 in FIG. 13) A vector 92 corresponding to the moving direction of the person (upper left to lower right) obtained is compared with a vector corresponding to the moving direction of the person obtained from past face detection total images accumulated in the hard disk 32, A past face detection total image 91 having a vector 93 similar to the vector 92 obtained from the face detection total image 89 of the current setting target camera is detected. Then, the CPU 31 of the management server 1 displays the detected (similar) past face detection results on the similar visualization statistical information search result display screen 90 side by side with the face detection summary image 89 of the camera to be set this time. A total image 91 is displayed.

As a method for detecting the past face detection total image 91 having a vector 93 similar to the vector 92 of the face detection total image 89 of the camera to be set this time, the face detection of the camera to be set this time The inner product of the vector 92 of the total image 89 and the vector 93 of the past face detection total image 91 is obtained, and the past face detection total image 91 corresponding to the vector 93 when the inner product value (scalar quantity) is large is obtained. A method of selection is conceivable.

In the above-mentioned similar visualization statistical information search result display screen 90, the user uses the operation unit 50a (mouse, etc.) of the personal computer 9 to operate the above (vector similar to the face detection total image 89 of the camera to be set this time 93), when a past face detection total image 91 is clicked (selected), the CPU 31 of the management server 1 applies to the past photographed image from which the selected past face detection total image 91 is based. Also, the condition setting parameters (mainly the coordinates of each vertex of the detection frame) for photographed image analysis are read from the statistical data 40 of the hard disk 32 . Then, as shown in FIG. 15, the CPU 31 (the parameter output unit 43 thereof) of the management server 1 outputs the past face detection total image 91 (same as the face detection total image 91 in FIG. 14) and the past face detection total image 91. A screen (similar pattern image setting display screen) including condition setting parameters (coordinates of each vertex of the detection frame displayed in the detection area display column 94f) applied to the past captured image that is the basis of the detection total image 91 94) is output (displayed) on the display 50b of the personal computer 9 (S16 in FIG. 12).

As described above, the user can view the past face detection total image 91 (having a vector 93 similar to the face detection total image 89 of the camera to be set this time) displayed on the display 50b of the personal computer 9. By referencing (browsing) the condition setting parameters (coordinates of each vertex of the detection frame) applied to the original captured image, the values of the condition setting parameters suitable for the image captured by the camera to be set this time can be determined. Know and be able to enter.

In addition to the detection area display field 94f, the similar pattern image setting display screen 94 shown in FIG. A direction display field 94e and a count result display field 94g are provided. In the store display field 94a and the camera display field 94b, information is displayed as to which store and which camera was used to capture the captured image that is the basis of the past face detection tally image 91, and the time is displayed. In the band display field 94c, the time zone in which the photographed images that are the basis of the past face detection tally image 91 were photographed is displayed. Also, in the count type display column 94d and the movement direction display column 94e, the count type (what state of the person is counted) applied to the photographed image that is the basis of the past face detection tally image 91 is displayed. type of movement) and direction of movement (information indicating which direction of movement people are to be counted) are displayed. Furthermore, in the count result display field 94g, the count type display field 94d, the movement direction display field 94e, and the detection area display field 94f are used to display the store visitor count conditions, and the past face detection tally image 91 is displayed. The number of people after performing visitor count processing on the original photographed image is displayed.

Next, the second method described above (visualization statistical information of the camera to be set this time and a plurality of past visualization statistical information are displayed side by side on the display 50b of the personal computer 9, and the user can A specific example of the method of selecting past visualization statistical information that is most similar to the visualization statistical information of the camera to be set will be described. When adopting this method, the user also uses the operation unit 50a (mouse, etc.) of the personal computer 9 to operate the store selection button 88a, the camera selection button 88b, the time When the selection button 88c and the visualization statistical information selection button 88d are selected to select the type of desired visualization statistical information based on the captured image of the camera to be set this time in a predetermined time period, the CPU 31 of the management server 1 displays on the display 50b of the personal computer 9 the visualization statistical information (in the example of FIG. 13, the face detection total image 89) corresponding to the selected store, camera, time zone, and type of visualization statistical information. In this state, when the user clicks the similar visualization statistical information search button 88e on the similar visualization statistical information search screen 88 using the mouse or the like of the personal computer 9, the CPU 31 (the display control unit 44 thereof) of the management server 1 As shown in the similarity visualization statistical information selection screen 97 of FIG. 96i are displayed side by side on the display 50b (candidate image display area 95) of the personal computer 9. FIG. It should be noted that the face detection tally images 96a to 96i displayed in the candidate image display area 95 in FIG. 16 show only the heat map portion, omitting the background image portion of these images.

The plurality of past face detection tally images 96a to 96i displayed in the candidate image display area 95 are the cameras and applications to be set this time among the past plural face detection tally images accumulated in the hard disk 32. and detection conditions are preferably the same. In addition, the plurality of past face detection aggregated images 96a to 96i displayed in the above candidate image display area 95 are selected from among the plurality of past face detection aggregated images accumulated in the hard disk 32 by the cameras to be set this time. It is desirable that the face detection total image of the heat map has a pattern similar to the face detection total image 89 based on the photographed image.

The user uses the operation unit 50a (mouse, etc.) of the personal computer 9 to select from among the plurality of past face detection tally images 96a to 96i displayed in the above-described candidate image display area 95, which are to be set this time. When the past face detection tally image that is most similar to the face detection tally image 89 based on the image taken by the camera is clicked (selected), the CPU 31 of the management server 1 displays the similar visualization statistical information search result display screen 90 in FIG. , the same processing as when the past face detection summary image 91 is clicked. That is, the CPU 31 of the management server 1 sets the condition setting parameters for photographed image analysis applied to the past photographed image that is the basis of the selected past face detection total image (any of 96a to 96i). (the coordinates of each vertex of the detection frame) are read from the statistical data 40 of the hard disk 32, and as shown in FIG. A screen (similar pattern image setting display screen 94) including condition setting parameters (coordinates of each vertex of the detection frame displayed in the detection area display column 94f) applied to the past captured image that is the basis of 91. , is output (displayed) on the display 50 b of the personal computer 9 .

As described above, according to the image analysis system 10, the management server 1, and the server control program 37 of the present embodiment, the camera to be set this time (hereinafter abbreviated as "current camera") (fixed camera 2 or Detect past visualization statistical information (face detection aggregate image 91, etc.) similar to the visualization statistical information (face detection aggregate image 89, etc.) based on the image captured by the built-in camera 4), and the source of the detected past visualization statistical information A screen (similar pattern image setting display The screen 94) is output (displayed) on the display 50b of the personal computer 9 so that a user such as a system administrator can view it. Here, the past visualization statistical information similar to the visualization statistical information based on the image captured by the current camera is obtained by: There is a high possibility that this is the visualization statistical information using the result of recognition processing similar to the recognition processing performed on the image captured by the camera this time. For this reason, the captured image that is the basis of the past visualized statistical information (for example, the past face detection total image 91) viewed by the user (similar to the visualized statistical information based on the current captured image of the camera) By referring to the condition setting parameters applied to (for example, the coordinates of each vertex of the detection frame (detection area) shown in FIG. 15) and entering the condition setting parameters for analyzing the captured image of the camera this time, A user such as an administrator of the system can easily input appropriate condition setting parameters (for example, the coordinates of each vertex of a human detection frame (detection area), etc.).

Further, according to the image analysis system 10 of the present embodiment, the condition setting parameters (for example, the (human) setting detection frame 85 in FIG. 10) for analysis of the current photographed image of the current camera (to be set) are set to , is displayed on the display 50b of the personal computer 9 by superimposing it on the visualization statistical information (for example, the moving trajectory total image 84 in FIG. 10) based on the image captured by the camera this time, and the user such as the system operation manager is displayed. It is now possible to input the adjustment of the condition setting parameters. As a result, the user can set the condition setting parameters while looking at the current condition setting parameters (setting detection frame 85, etc.) superimposed on the visualization statistical information (moving trajectory total image 84, etc.) based on the image captured by the current camera. Since adjustment input can be performed, it is possible to easily input appropriate condition setting parameters for the current camera.

Further, according to the image analysis system 10, the management server 1, and the server control program 37 of the present embodiment, as shown in FIG. A vector 92 corresponding to the movement direction of a person obtained from visualization statistical information (for example, face detection total image 89) based on images captured by the camera, and past visualization statistical information accumulated in the hard disk 32 (for example, past face Compare the vector 93 corresponding to the moving direction of the person obtained from the detected total image 91) and detect the past visualization statistical information similar to the visualization statistical information based on the image captured by the camera to be set this time. I made it As a result, it is possible to easily detect past visualization statistical information (face detection aggregated image 91, etc.) similar to the visualization statistical information (face detection aggregated image 89, etc.) based on the captured image of the camera to be set this time. can.

Further, according to the image analysis system 10 of the present embodiment, the CPU 31 (the display control unit 44 thereof) of the management server 1 outputs the visualization statistical information (for example, the face in FIG. Detected total image 89) and a plurality of past visualization statistical information accumulated in the hard disk 32 (for example, face detection total images 96a to 96i in FIG. 16) are displayed on the display 50b of the personal computer 9, and the user can , using the mouse or the like of the personal computer 9, from among a plurality of past visualized statistical information (for example, aggregated face detection images 96a to 96i) displayed on the display 50b, the image captured by the camera to be set this time. It is made possible to select the past visualization statistical information that is most similar to the visualization statistical information (for example, the face detection total image 89) based on. As a result, with a simple configuration, it is possible to search for the past visualization statistical information that is most similar to the visualization statistical information based on the captured image of the camera to be set this time.

Further, according to the image analysis system 10 of the present embodiment, the user uses the condition setting parameters (for example, the (human) setting detection frame 85 in FIG. 10) after adjustment using the operation unit 50a of the personal computer 9. Then, the image captured by the camera to be set this time is analyzed again (re-analyzed), and the analysis result after re-analysis (for example, the visitor count result in the count result display column 83h in FIG. 10) is displayed. The information is displayed on the display 50b of the personal computer 9. As a result, it is possible to simulate the difference in the analysis result (for the captured image of this camera) in accordance with the adjustment details of the condition setting parameters for the camera that is the current setting target. Therefore, the user can easily check the accuracy of the condition setting (for example, the setting detection frame 85 in FIG. 10) set by adjusting and inputting the condition setting parameters.

Variant:
The present invention is not limited to the configuration of each of the above embodiments, and various modifications are possible without changing the gist of the invention. Next, modified examples of the present invention will be described.

Variant 1:
In the above embodiment, the current condition setting parameters of the camera to be set this time (for example, the setting detection The frame 85) is superimposed on the visualized statistical information (for example, the moving trajectory total image 84 in FIG. 10) based on the captured image of the camera to be set this time, and displayed on the display 50b of the personal computer 9. However, on the condition setting parameter simulation screen, (1) the current condition setting parameters (setting detection frame, etc.) of the camera to be set this time are displayed in the image taken for a predetermined time of the camera to be set this time ( (2) The current condition setting parameters (setting detection frame, etc.) of the camera to be set this time may be superimposed on the image) and displayed on the display of the personal computer. It may be superimposed on the analysis result superimposed video for a predetermined time of the camera (for example, the analysis result superimposed video 63 in FIG. 8) and displayed on the display of the personal computer.

In the above embodiment, the current condition setting parameters of the camera to be set this time (for example, the The setting detection frame 85) is displayed on the display 50b of the personal computer 9 by superimposing it on the moving trajectory total image 84 based on the captured image of the camera to be set this time. However, in the condition setting parameter simulation screen, the visualization statistical information on which the current condition setting parameters of the camera to be set this time are superimposed is not limited to the above-described moving trajectory total image, and face detection as shown in FIG. It may be the total image 89, or the above-mentioned gender and age recognition total image (each frame image included in the image captured by the camera to be set this time in a predetermined time period, and the gender and age recognition for each frame image) It may be a heat map generated by statistically processing each superimposed image obtained by superimposing the results.

Modification 2:
In the above embodiment, as shown in FIGS. 13 to 16, the face detection total image 89 based on the image captured by the camera to be set this time is selected from the past face detection total images accumulated in the hard disk 32. An example of detecting (retrieving) similar past visualization statistics is shown. However, among the past movement trajectory aggregated images accumulated in the hard disk 32, past movements of a pattern similar to (the movement trajectory pattern in) the movement trajectory aggregated image based on the image captured by the camera to be set this time The trajectory total image may be detected (searched), or the past sex and age recognition total image of a pattern similar to (the heat map pattern in) the past sex and age recognition total image accumulated in the hard disk 32 is searched. It may be detected (searched).

Variant 3:
In the above embodiment, an example is shown in which the "condition setting parameters for photographed image analysis" in the claims are the (human) detection frame (or the coordinates of each vertex of the detection frame). The condition setting parameter (for photographed image analysis) in is not limited to this, and may be, for example, the reliability of object detection or object recognition (for example, human detection or human face detection) for the photographed image. For example, by setting the condition setting parameter to the reliability of human detection, by adjusting this reliability, the amount of analysis data (corresponding to the video metadata 74 in FIG. 7) sent to the management server 1 on the cloud C side (range of analysis data) can be adjusted.

Variant 4:
In the above embodiment, an example in which the image analysis system 10 includes only the management server 1 on the cloud C is shown, but the configuration of the image analysis system is not limited to this. Additionally, an AI analysis server may be provided in addition to the management server. The AI analysis server, for example, analyzes the behavior of people in each store based on the object recognition results from analysis boxes and signage, and the information of the analysis results can be easily used by applications for various purposes such as marketing and crime prevention. Plays the role of converting to data and outputting it. Also, the image analysis system may include a management server for managing fixed cameras and analysis boxes, and a signage management server for managing signage on the cloud.

Variant 5:
In the above embodiment, an example in which the image analysis system 10 includes the fixed camera 2, the analysis box 3, and the signage 5 in the store S is shown. And the analysis box may be provided without the signage, or the store may be provided with the signage only without the fixed camera and the analysis box.

Modification 6:
In the above embodiment, an example in which the CPU 31 of the management server 1 includes the visualization statistical information generation unit 41 is shown, but the analysis box and the signage include the visualization statistical information generation unit to generate visualization statistical information. You can do it.

In the above embodiment, the display means in the claims is the display 50b of the personal computer 9, and the selection input means and the adjustment input means in the claims are mainly the operation unit 50a of the personal computer 9. However, the display means in the claims may be the display of the management server, and the selection input means and the adjustment input means in the claims may be the operation unit 35 of the management server.

1 Management server (server)
2 fixed camera (part of the camera on the edge side)
3 analysis box (part of the image analysis device on the edge side)
4 Built-in camera (part of the camera on the edge side)
5 Signage (part of image analysis equipment on the edge side)
10 Image analysis system (parameter viewing system, parameter adjustment system)
32 hard disk (statistical information storage means)
36 communication unit (receiving means)
37 Server control program 41 Visualization statistical information generation unit (visualization statistical information generation means)
42 detection unit (detection means)
43 Parameter output unit (parameter output means)
44 display control unit (display control means)
45 Parameter display control unit (parameter display control means)
46 Re-analysis department (re-analysis means)
47 reanalysis result display control unit (reanalysis result display control means)
48 visualization statistical information 49 condition setting parameters 50a operation unit (adjustment input means, selection input means)
50b display (display means)
92 vector (vector corresponding to the direction of movement of a person, obtained from the visualization statistical information based on the image captured by the camera to be set this time)
93 vector (vector corresponding to the direction of movement of people, obtained from past visualization statistics)

Claims (9)

display means;
Visualization statistics for generating visualization statistical information by statistically processing a superimposed video obtained by superimposing an image captured by an edge-side camera and a result of recognition processing performed on a frame image included in the captured image by an image analysis device on the edge side. an information generating means;
statistical information accumulation means for accumulating visualization statistical information generated by the visualization statistical information generating means;
a detection means for detecting past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time, from the past visualization statistical information accumulated in the statistical information accumulation means;
parameter output means for outputting, to the display means, a screen including condition setting parameters for analysis of the captured image applied to the past captured image based on the past visualization statistical information detected by the detection means; Parameter viewing system. The detection means obtains from the vector corresponding to the moving direction of the person obtained from the visualization statistical information based on the image captured by the camera to be set this time, and the past visualization statistical information accumulated in the statistical information accumulation means. Further, past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time is detected by comparing with the vector corresponding to the moving direction of the person. The parameter viewing system described in . The detection means displays, on the display means, the visualization statistical information based on the captured image of the camera to be set this time and a plurality of past visualization statistical information accumulated in the statistical information accumulation means. including display control means for controlling,
The parameter browsing system selects the past visualization statistical information most similar to the visualization statistical information based on the image captured by the camera to be set this time from among the plurality of past visualization statistical information displayed on the display means. 2. The parameter viewing system according to claim 1, further comprising selection input means for selection. display means;
Receiving means for receiving an image captured by a camera to be set this time;
Visualization of the condition setting parameters for analysis of the current shot image of the camera to be set this time, based on the image including the image shot by the camera to be set this time, or the image shot by the camera to be set this time parameter display control means for superimposing statistical information and displaying it on the display means;
and an adjustment input means for performing adjustment input of the condition setting parameters displayed on the display means. re-analyzing means for re-analyzing the captured image of the current setting target camera received by the receiving means, using the condition setting parameters adjusted by the adjustment input means;
5. The parameter adjustment system according to claim 4, further comprising re-analysis result display control means for controlling display of the analysis result by said re-analysis means on said display means. Visualization statistics for generating visualization statistical information by statistically processing a superimposed video obtained by superimposing an image captured by an edge-side camera and a result of recognition processing performed on a frame image included in the captured image by an image analysis device on the edge side. an information generating means;
statistical information accumulation means for accumulating visualization statistical information generated by the visualization statistical information generating means;
a detection means for detecting past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time, from the past visualization statistical information accumulated in the statistical information accumulation means;
A server comprising parameter output means for outputting, to a display means, a screen including condition setting parameters for analysis of a captured image applied to a past captured image based on past visualization statistical information detected by the detection means. . The detection means obtains from the vector corresponding to the moving direction of the person obtained from the visualization statistical information based on the image captured by the camera to be set this time, and the past visualization statistical information accumulated in the statistical information accumulation means. Further, past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time is detected by comparing with the vector corresponding to the moving direction of the person. server described in . A server control program for controlling a server so as to perform processing for finding condition setting parameters for photographed image analysis suitable for the photographed image of a camera to be set this time,
the server,
Visualization statistics for generating visualization statistical information by statistically processing a superimposed video obtained by superimposing an image captured by an edge-side camera and a result of recognition processing performed on a frame image included in the captured image by an image analysis device on the edge side. an information generating means;
statistical information accumulation means for accumulating visualization statistical information generated by the visualization statistical information generating means;
a detection means for detecting past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time, from the past visualization statistical information accumulated in the statistical information accumulation means;
To function as a parameter output means for outputting to a display means a screen including condition setting parameters for analyzing a captured image applied to a past captured image based on past visualization statistical information detected by the detection means. server control program. The detection means obtains from the vector corresponding to the moving direction of the person obtained from the visualization statistical information based on the image captured by the camera to be set this time, and the past visualization statistical information accumulated in the statistical information accumulation means. Further, past visualization statistical information similar to the visualization statistical information based on the captured image of the camera to be set this time is detected by comparing with the vector corresponding to the moving direction of the person. server control program described in .

Images