JP2018207222A - Camera and parameter registration method - Google Patents

Abstract

To improve learning accuracy in a camera, by appropriately controlling a learning amount of parameters for use in detection.SOLUTION: A camera 10 is installed in a monitoring area SA. The camera 10 includes an image sensor 12 for imaging subject light from the monitoring area SA, an object estimation function for detecting at least one object appearing in a captured image, by using the captured image based on imaging of the subject light, a teacher data set memory holding teacher data set prepared for each type of the object, a score derivation function for deriving a score indicating detection accuracy of a detected object by using the teacher data set, and a parameter learning function for learning model parameters for use in detection of the object, according to the derived score. The parameter learning function registers and accumulates learning results of the model parameters in the parameter memory .SELECTED DRAWING: Figure 15

Description

  The present disclosure relates to a camera that is installed in a monitoring area and images a subject in the monitoring area to detect an object, and a parameter registration method that registers a parameter used when detecting the object.

  At present, it is said that the arithmetic processing capability of a camera is 0.3 T (tera) ops (ops). T (terra) is a value indicating 10 12. Ops (ops) is known as a unit indicating arithmetic processing capability. In the future, it is considered that high-performance GPUs (Graphics Processing Units) and FPGAs (Field Programmable Gate Arrays) mounted on game machines and the like will be used for camera processing units. In that case, for example, after one year, it is expected that the arithmetic processing capability of the camera will be dramatically improved to about 2.6 Tops, which is 10 times or more.

  Further, it is pointed out that when the camera performs image recognition processing using deep learning as an example of machine learning, 1.3 Tops is required for the arithmetic processing capability of the camera. In the past, it was thought that it was difficult for the camera to perform image recognition processing using deep learning because of its high processing power. However, in the processing power of the camera one year later, deep learning was used. It is considered possible to perform image recognition processing sufficiently.

  On the other hand, when high-quality (for example, 4K) captured image data captured by the camera is transferred to the server and the server performs image recognition processing, the amount of communication transmitted over the network as the size of the captured image data increases. (Traffic) inevitably increases, resulting in a decrease in communication efficiency and a delay. For this reason, the camera is expected to perform image recognition processing using deep learning in its own device without transferring high-quality (for example, 4K) captured image data.

  In general, when performing image recognition processing using deep learning, a device such as a camera learns an object (that is, an image recognition target object) included in captured image data and uses model parameters ( For example, the learning model is updated by changing a weighting factor or a threshold value. A device such as a camera improves the accuracy of detecting an object (that is, an image recognition object) included in the captured image data based on the updated learning model.

  As a prior art for recognizing an object using captured image data captured by a camera and acquiring object motion information, for example, an object tracking device of Patent Document 1 has been proposed. This object tracking device uses a time-series image group acquired from a camera capable of photographing an object, and includes an object including image information related to the acquired image and position information related to the position of the object in real space. Learning is performed using a teacher data set including motion information and correct information. Further, the object tracking device uses a tracking discriminator that outputs position information that is at least correct in the real space of the object by inputting image information related to the image for each object tracking target image. The position information of the object in real space is acquired.

JP, 2006-207995, A

  In the related art such as Patent Document 1 described above, it is disclosed that the score of the evaluation function related to an object is used in order to obtain object motion information as a correct answer of the object to be tracked in the captured image. However, no particular consideration has been given to controlling the learning amount of a parameter necessary for object detection according to the score indicating the object detection accuracy. For this reason, there is a concern that learning of parameters used for detection, which originally does not require learning, causes variations in parameter learning accuracy and affects object detection accuracy.

  The present disclosure is devised in view of the above-described conventional circumstances, and a score indicating the detection accuracy of an object obtained by detecting at least one object in a captured image captured by a camera installed in a monitoring area. Accordingly, an object of the present invention is to provide a camera and a parameter registration method that appropriately control a learning amount of a parameter used for detection and improve learning accuracy in the camera.

  The present disclosure is a camera used in a monitoring system that is installed in a monitoring area and is connected to a server so as to be communicable with each other, an imaging unit that images subject light from the monitoring area, and the subject light in the imaging unit A detection unit that detects at least one object appearing in the captured image using a captured image based on the captured image, a memory that holds a teacher data set prepared for each type of the object, and the teacher data set. A derivation unit for deriving a score indicating the detection accuracy of the object detected by the detection unit, and parameter learning for learning a parameter used for detection of the object according to the score derived by the derivation unit And the parameter learning unit registers the learning result of the parameter in the memory. To accumulate, to provide a camera.

  Further, the present disclosure is a parameter registration method using a camera installed in a monitoring area, the step of imaging subject light from the monitoring area, and a captured image based on imaging of the subject light, Detecting at least one object appearing in the captured image, deriving a score indicating detection accuracy of the detected object using a teacher data set prepared for each type of the object, and According to another aspect of the present invention, there is provided a parameter registration method comprising: learning a parameter used for detecting the object according to the score; and registering and storing a learning result of the parameter in a memory.

  According to the present disclosure, learning of parameters used for detection is performed according to a score indicating detection accuracy of an object obtained by detecting at least one object in a captured image captured by a camera installed in a monitoring area. It is possible to appropriately control the amount and improve the learning accuracy in the camera.

1 is a block diagram illustrating an example of a system configuration of a monitoring system according to a first embodiment. Explanatory drawing of an outline example of learning and detection FIG. 2 is a block diagram showing in detail an example of the internal configuration of the camera according to the first embodiment. FIG. 2 is a block diagram showing in detail an example of the internal configuration of the server according to the first embodiment. Explanatory diagram of an outline example of learning on a device Explanatory diagram of an example of camera detection overview Explanatory drawing of an example of processing outline when performing dispersion during learning using multiple cameras in a surveillance system Explanatory drawing of an outline example of resource management in a monitoring system FIG. 3 is a sequence diagram illustrating in detail an example of an operation procedure in which the server instructs the camera to execute a process in the first embodiment. The sequence diagram which shows in detail an example of the operation | movement procedure in which the server controls the feedback amount of a model parameter in Embodiment 1. Explanatory drawing of an outline example of sharing of learning results in a monitoring system The figure which shows an example of UI screen displayed at the time of local learning The figure which shows an example of UI screen displayed on the display part of a server at the time of integrated learning FIG. 3 is a block diagram showing in detail an example of an internal configuration of a processing execution unit of a camera according to a second embodiment. The flowchart which shows an example of the operation | movement procedure of the local learning of a camera in detail Explanatory drawing of an outline example of sharing of learning results in a monitoring system The figure which shows an example of UI screen displayed at the time of local learning The figure which shows an example of UI screen displayed on the display part of a server at the time of integrated learning

(Background to the first embodiment)
In the future, the captured image data handled by the camera is expected to increase in data size, for example, with high definition and large capacity such as 4K and 8K. As the size of the captured image data increases, when the parameters used for detection of the captured image data are learned not by the camera but by the server, the processing load is concentrated on the server. By transmitting to the server one by one, the traffic on the network increases, causing a problem that a corresponding delay occurs during data communication. With regard to the technical countermeasures for such problems, the conventional technology such as Patent Document 1 has not been particularly considered.

  Therefore, in the first embodiment, when detecting at least one object in each captured image captured by a plurality of cameras installed in a monitoring area, processing such as learning of parameters used for the detection is performed between the plurality of cameras. An example of a monitoring system and a monitoring method for distributing and suppressing an increase in traffic on the network and supporting a reduction in processing load on servers connected to a plurality of cameras will be described.

(Embodiment 1)
FIG. 1 is a block diagram illustrating an example of a system configuration of the monitoring system 5 according to the first embodiment.

  The monitoring system 5 is, for example, a crime prevention monitoring system, and is installed indoors such as a bank, a store, a company, or a facility, or outdoors such as a parking lot or a park. Indoors such as banks, stores, companies, and facilities, or outdoors such as parking lots and parks are monitoring areas of the monitoring system 5. The monitoring system 5 according to the present embodiment uses at least one camera 10 that recognizes at least one object (in other words, an object) appearing in a captured image by using artificial intelligence (AI) technology, and a server. 30 and a recorder 50. At least one camera 10, server 30, and recorder 50 are connected to each other via a network NW so as to communicate with each other.

  Hereinafter, when it is necessary to distinguish the plurality of cameras 10 from each other, they are denoted as cameras 10A, 10B, 10C,. The plurality of cameras 10 may be installed as the monitoring area, for example, in the same place in the building, or some of the cameras 10 may be installed in a place different from the other cameras 10. Here, it is assumed that the installation conditions (for example, the installation angle and the angle of view of the camera) of the cameras 10A, 10B, and 10C installed in different places as the monitoring area are the same. For example, the cameras 10A, 10B, and 10C are all attached to the wall surface so as to be positioned above the entrance / exit where the automatic door is installed, and take an image so that the person entering / exiting the entrance / exit is looked down slightly from above. The installation status of the cameras 10A, 10B, and 10C is not limited to the case where the cameras 10A, 10B, and 10C are located in the information on the entrance / exit where the automatic door is installed.

  First of all, learning for generating a neural network (in other words, a learning model) used for deep learning as an example of machine learning of artificial intelligence (AI) technology, and a learned learning model (hereinafter referred to as “learned”). The outline of detection (that is, inference) for inputting data to a model and outputting the result will be described.

  FIG. 2 is an explanatory diagram of an outline example of learning and detection.

  The learning process (hereinafter simply referred to as “learning”) is a process performed by deep learning as an example of machine learning of artificial intelligence (AI) technology, for example. In other words, as one of machine learning, learning is performed using deep learning (that is, deep learning) in a neural network (hereinafter abbreviated as “NN”) that has been attracting attention in recent years. In machine learning by deep learning, “supervised learning” using teacher data and “unsupervised learning” not using teacher data are performed. As a result of machine learning, a learned model is generated. On the other hand, the detection is a process of obtaining data by inputting data into the generated learned model.

  Although learning may be performed in real time, since it requires a lot of arithmetic processing, it is usually performed offline (that is, asynchronously). On the other hand, the detection process (hereinafter simply referred to as “detection”) is normally performed in real time. In addition, the device on which learning is performed may be any of the camera 10, the server 30, and the recorder 50, and here, a case where learning is performed in the camera 10 is shown. On the other hand, the detection is performed in the camera 10. In addition, even if the captured image data captured by the camera 10 is transferred to the server 30 or the recorder 50, the server 30 or the recorder 50 may perform detection if no traffic on the network NW occurs.

  At the time of learning, the device 150 inputs a lot of learning data (for example, image data captured by the camera 10). The device 150 performs machine learning (for example, deep learning processing) based on the input learning data, and updates the model parameter P of the neural network (NN 140) that is a learning model. The model parameter P is a weighting coefficient (that is, bias) or a threshold value set in each of a plurality of neurons constituting the NN 140. When performing machine learning (for example, deep learning processing), the device 150 uses teacher data and acquires correctness for each learning data or calculates an evaluation value (that is, a score). The device 150 changes the learning degree of the model parameter P according to the correctness of the learning data or the level of the score. After learning, the NN 140 is used for detection in the device 150 as a learned model.

  At the time of detection (that is, at the time of inference), the device 150 inputs input data (for example, captured image data imaged in real time by the camera 10), executes an inference at the NN 140, and an inference result obtained by the execution (that is, an inference) , The determination result of the detected object) is output. The determination result includes, for example, information on correct reports and false reports according to the presence or absence of an object included in the captured image data, and information on a score indicating an evaluation value of the object. The correct report is a report indicating that the object is correctly detected with high accuracy when the object is detected. The false alarm is a report indicating that the object is erroneously detected with high accuracy when the object is detected.

  FIG. 3 is a block diagram illustrating in detail an example of the internal configuration of the camera 10 according to the first embodiment.

  For example, the camera 10 captures a subject image in a monitoring area and acquires captured image data. Specifically, the camera 10 includes a lens 11, an image sensor 12, a signal processing unit 13, a processing execution unit 14, a resource monitoring unit 15, a crop encoding unit 17, and a network I / F 16. It is.

  The camera 10 forms an image of the subject that has entered from the monitoring area SA on the image sensor 12 via the lens 11 that can receive the subject image from the monitoring area SA, and the subject image ( That is, an optical image) is converted into an electrical signal and captured. An imaging unit of the camera 10 is configured by at least the lens 11 and the image sensor 12. The camera 10 uses the electrical signal obtained by the image sensor 12 to generate an RGB signal in the signal processing unit 13 or perform predetermined various image processing such as white balance and contrast adjustment to obtain captured image data. Is generated and output.

  The process execution part 14 is comprised, for example using GPU (Graphics Processing Unit) or FPGA (Field Programmable Gate Array). If GPUs or FPGAs with high performance and high processing power are adopted as the processor of the camera 10 in the future, the processing power of the camera 10 will be greatly improved, and deep learning processing can be sufficiently performed in the camera 10. Expected to be The process execution unit 14 includes a learning model or a learned model as the NN 140 generated or updated by execution of a process in the GPU or FPGA, and at least one target (that is, an object that appears in the captured image) (that is, the input captured image data) , Object) determination result is output.

  The resource monitoring unit 15 monitors information on the processing capability of the camera 10 (for example, the amount of free resources) based on the usage status of the GPU, FPGA, memory, or the like in the processing execution unit 14.

  At the time of detection, the crop encoding unit 17 cuts out a part of an object (that is, an object) that appears in the captured image data and outputs it as captured image data or thumbnail data to be processed.

  The network I / F 16 controls connection with the network NW. The camera 10 receives the determination result of the object (that is, the object) output from the process execution unit 14 to the server 30 and the recorder 50 via the network I / F 16 and the free resource monitored by the resource monitoring unit 15. Send volume, thumbnail data, etc. Further, the camera 10 receives the model parameter P as a learning result from the server 30, the recorder 50, and another camera 10 via the network I / F 16.

  FIG. 4 is a block diagram showing in detail an example of the internal configuration of the server 30 according to the first embodiment.

  The server 30 includes a processor (for example, CPU (Central Processing Unit), MPU (Micro Processing Unit) or DSP (Digital Signal Processor)) 31, a memory 32, a communication unit 33, an operation unit 36, a display unit 37, The configuration includes a learning DB (database) 34 and a table memory 35. The processor 31 cooperates with the memory 32 to centrally execute processing and control of each unit of the server 30. The memory 32 includes a nonvolatile memory and a volatile memory. In the nonvolatile memory, for example, information related to unit cost of each camera 10A, 10B, 10C notified from a plurality of cameras 10A, 10B, 10C (for example, information related to power cost of the cameras 10A, 10B, 10C) is stored. . The information related to the power cost is an index value indicating how much power (ie, cost) is consumed as a result, for example, when the cameras 10A, 10B, and 10C are used.

  When the server 30 performs machine learning (for example, deep learning processing), the processor 31 executes a program stored in the nonvolatile memory and generates a learning model (neural network: NN). In addition, the server 30 receives the model parameter P as a learning result from the plurality of cameras 10 and installs each camera 10 installed in the monitoring area SA (that is, an installation environment such as an installation angle and an angle of view). Integrate model parameters P that are identical.

  The learning DB (database) 34 stores model parameters P (for example, weighting coefficients and threshold values), which are learning results transmitted from the plurality of cameras 10 and received by the server 30.

  The table memory 35 stores a table in which information (for example, the amount of free resources) regarding the processing capabilities of the plurality of cameras 10 is registered.

  The operation unit 36 has various buttons such as a learning button bt5 (see, for example, FIG. 13) that can be operated by the user, and receives an input operation of the user.

  The display unit 37 displays a UI (user interface) screen 310 (for example, see FIG. 12 or FIG. 13) that presents the processing results of the integrated learning in the server 30.

  FIG. 5 is an explanatory diagram of an outline example of learning in the device 150.

  Here, the case where the device 150 learns the “car” appearing in the captured image as the object obj will be described as an example. As described above, learning is a process that is normally performed offline (asynchronously), and may be performed by any of the camera 10, the server 30, and the recorder 50. In the present embodiment, the camera 10 performs learning as an example of the device 150. The device 150 includes a process execution unit 164, a resource monitoring unit 165, a network I / F 166, and a parameter gradient calculation unit 168.

  The network I / F 166 controls connection with the network NW and receives learning data via the network NW. Here, the learning data is captured image data gz1 and gz2 in which the vehicle is the object obj. The captured image data gz1 and gz2 are teacher data to which a score (evaluation value) and a correct report or a false report are added, respectively. For example, the captured image data gz1 is a captured image including a “car” as an object, and is teacher data having a high score or correct report. On the other hand, the captured image data gz2 is an image of a “tree” that is not a target vehicle, and is teacher data having a low score or false alarm.

  The process execution unit 164 updates the model parameters P (for example, weighting factors and threshold values) of the learning model by executing inference based on these teacher data input via the network I / F 166. . In addition, the process execution unit 164 transmits the updated model parameter P to other devices such as the camera 10, the server 30, and the recorder 50 via the network I / F 166. Thus, by performing “supervised learning”, the learning ability is enhanced, and the processing execution unit 164 can generate a high-quality learning model.

  The parameter gradient calculation unit 168 calculates the gradient of the object that appears in the captured image of the teacher data. For example, a captured image captured by the camera from the side and a captured image captured by the camera from the front are different from each other even for the same object. That is, the model parameter P of the learning model used when detecting the same object varies depending on the camera installation status (for example, installation angle or angle of view). For this reason, the parameter gradient calculation unit 168 calculates a gradient representing the imaging direction (hereinafter referred to as “parameter gradient”), and the parameter gradient Pt is converted to the camera 10, the server 30, the recorder 50, etc. via the network I / F 166. Send to other devices. The parameter gradient Pt may be transmitted together with the model parameter or separately. In any case, since the camera installation status is not changed frequently, the parameter gradient Pt only needs to be transmitted at least once. By using the parameter gradient Pt, a different learning model can be used for each camera installation situation.

  The resource monitoring unit 165 monitors the amount of free resources based on the usage status of the GPU and memory in the processing execution unit 164. When the device 150 is the camera 10, the processing execution unit 164 and the parameter gradient calculation unit 168 shown in FIG. 5 correspond to the processing execution unit 14 in FIG. 3, and the resource monitoring unit 165 shown in FIG. The network I / F 166 shown in FIG. 5 corresponds to the network I / F 16 shown in FIG.

  FIG. 6 is an explanatory diagram of an outline example of detection by the camera 10.

  Here, the case where the camera 10 detects a “car” appearing in a captured image as an object will be described as an example. The processing execution unit 14 of the camera 10 has a learning model (that is, a learned model) after machine learning (for example, deep learning processing) is performed. The process execution unit 14 receives the captured image og of the subject imaged through the lens 11, performs detection using the learned model (that is, inference of an object appearing in the captured image og), and the detection result (that is, Output inference results). The crop encoding unit 17 cuts out an image to be a target included in the captured image og of the subject, and outputs a cut-out image as a detection result.

  Here, the “car” clipped image tg2 and the “tree” clipped image tg1 cut out by the crop encoding unit 17 are output. The cut-out image tg2 of “car” includes a captured image of the car that is the object, and thus has a high score and correct report. On the other hand, the cut-out image tg1 of “tree” does not include the captured image of the vehicle that is the object, and thus has a low score and false alarm.

  Next, specific operations of the monitoring system 5 of the present embodiment will be described with reference to the drawings.

  FIG. 7 is an explanatory diagram of an example of a processing outline when performing dispersion during learning using a plurality of cameras in the monitoring system 5.

  As described above, in the learning, a process of updating the model parameter P of a learning model (that is, a neural network) generated using a deep learning process as an example of machine learning of artificial intelligence (AI) technology is performed. . As an example, a case where three cameras 10A, 10B, and 10C perform learning as a device that performs learning will be described. The learning device is not limited to the camera, but may be a server or a recorder. Each of the cameras 10A, 10B, and 10C performs, for example, “unsupervised learning” on the input captured image data. In unsupervised learning, the camera 10 generates an alarm if the model parameters of the learning model do not converge. At this time, the user cancels the alarm and performs “supervised learning”. In supervised learning, the user inputs the correct or incorrect report of image data. In inputting teacher data, a score (evaluation value) may be input together with the correct or incorrect report instead of inputting the correct or incorrect report of the image data. The score is a value for evaluating that the captured image data is captured image data including an object, and is expressed by a score of 80 points, 10 points, or a probability of 50%, 20%, for example.

  The three cameras 10 </ b> A, 10 </ b> B, and 10 </ b> C each transmit model parameters P that are learning results to the server 30. Further, the parameter gradient Pt described above is added to the transmitted model parameter P.

  The server 30 updates the model parameter P of the learning model based on the model parameter P transmitted from the three cameras 10A, 10B, and 10C. At this time, model parameters having the same parameter gradient Pt, that is, model parameters having the same camera installation state are integrated. Accordingly, the model parameters of the learning model having the same parameter gradient are updated. Here, the installation status of the cameras 10A, 10B, and 10C is the same, and the server 30 integrates the updated model parameters of the cameras 10A, 10B, and 10C.

  The server 30 sends the integrated model parameters as feedback to the three cameras 10A, 10B, and 10C. As a result, the model parameters stored in the three cameras 10A, 10B, and 10C are the same. Note that the transmission of model parameters from the three cameras 10A, 10B, and 10C to the server 30 is performed asynchronously.

  FIG. 8 is an explanatory diagram of an outline example of resource management in the monitoring system 5.

  In the three cameras 10A, 10B, and 10C, the resource monitoring unit 15 determines the amount of free resources (in other words, the remaining power indicating the remaining degree of the processing capacity) for the processing capacity such as GPU or FPGA that generates the learning model. Monitoring. The three cameras 10A, 10B, and 10C notify the server 30 of the amount of free resources monitored by the resource monitoring unit 15 asynchronously or periodically. The amount of free resources is expressed as a percentage (%) of processing capacity. As an example, when the amount of free resources of the camera 10A is 90%, the amount of free resources of the camera 10B is 20%, and the amount of free resources of the camera 10C is 10%, the server 30 A learning instruction is output to the camera 10A so as to preferentially learn the camera 10A having a large amount, that is, to increase the learning amount.

  In addition, when the network NW has a wide bandwidth or the network NW is free, the server 30 captures captured image data (correct information or incorrect information with information) captured by the camera 10C having a small amount of free resources of 10%. ), The captured image data may be transmitted to the camera 10A having a large amount of free resources of 90% to instruct learning. As a result, it is possible to realize appropriate learning without imposing a processing load biased between the cameras.

  In addition, the server 30 may instruct to perform learning by directly transferring captured image data captured by a camera with a small amount of free resources to a camera having a large amount of free resources. Thereby, learning can be distributed in the monitoring system, and efficient learning is possible without imposing a large load on a specific camera.

  Further, the server 30 may instruct the detection to be performed by directly transferring the captured image data captured by the camera having a small amount of free resources to the camera having a large amount of free resources. Thereby, detection can be distributed within the monitoring system, and efficient detection is possible without imposing a large load on a specific camera.

  Further, the server 30 may instruct to directly transfer the captured image data captured by the camera having a small amount of free resources to the camera having a large amount of free resources, and perform an analysis. Here, the analysis is processing such as tracking an object (that is, an object) obj appearing in the captured image or recognizing whether the object corresponds to a suspicious person. Is not particularly limited in the present embodiment. As a result, the analysis can be distributed within the monitoring system, and an efficient analysis can be performed without imposing a large load on a specific camera.

  In addition, the server 30 monitors the overall processing capability of the monitoring system 5 and instructs each camera 10 to increase the learning amount when the amount of free resources in the entire system is large. When the amount of resources is small, each camera 10 may be instructed to reduce the learning amount. As a result, appropriate learning can be performed without imposing a large load on the entire monitoring system.

  Further, the server 30 may instruct the detection result of each camera 10 to be shared by all the cameras 10. As a result, the detection results can be distributed to the cameras 10, and the detection accuracy can be improved by using the detection results for the next and subsequent detections.

  In addition, when the amount of free resources of the camera 10 is large, the server 30 instructs to increase the feedback amount (for example, the number of feedbacks) of the integrated learning result transmitted to the camera 10. When the amount is small, it may be instructed to reduce the feedback amount (for example, the number of feedbacks) of the result of integrated learning transmitted to the camera 10. Thereby, an appropriate amount of learning results can be fed back (returned) to the camera without imposing a heavy load on the camera.

  The three cameras 10A, 10B, and 10C each notify the server 30 of information about unit cost (for example, information about power cost). The power cost is a value unique to the camera and is expressed in units of watts / frame (W / frame), for example. As an example, 1/20 for the camera 10A, 1/200 for the camera 10B, and 1/400 for the camera 10C. Note that the power cost does not vary greatly depending on the usage status of the camera, so a single notification is sufficient. The unit of power cost may be expressed in frame / watt (frame / W).

  When the power costs of the camera 10A and the camera 10B are equally high, the server 30 preferentially assigns learning to the camera 10C having a low power cost.

  In the server 30, when the amount of free resources of the cameras 10A, 10B, and 10C is the same or similar, for example, the amount of free resources of the camera 10A is 10%, and the amount of free resources of the cameras 10B and 10C are both If it is 45%, a learning instruction is output to the camera 10C so as to preferentially learn with the camera 10C that does not incur power costs.

  Note that the server 30 may instruct the camera 30 to perform learning with a camera with priority on cost and low power cost, regardless of the number of available resources of the camera. Further, although the server 30 manages the free resources and power costs of each device, each camera or recorder may manage them. In this case, all the devices 150 in the monitoring system 5 manage the free resources and power costs. Can be shared. Therefore, each device can also execute processing instructions in consideration of available resources and power costs, and various operations are possible.

  FIG. 9 is a sequence diagram illustrating in detail an example of an operation procedure in which the server 30 instructs the camera 10 to execute a process in the first embodiment.

  In the operation procedure of FIG. 9, the server 30 determines a camera 10 to be distributed from a plurality of cameras 10 based on the information on the free resources of the camera 10, and executes the process on the corresponding camera 10. Instruct. The number N of cameras may be an arbitrary number. Here, two cameras (cameras 10A and 10B) are illustrated for the sake of simplicity. Instead of the server 30, the recorder 50 may determine the camera 10 that instructs the execution of the process.

  The camera 10A repeatedly (for example, constantly or periodically) notifies the server 30 of the information on the free resources monitored by the resource monitoring unit 15 (T1). Similarly, the camera 10B repeatedly notifies the server 30 of the information on the free resources monitored by the resource monitoring unit 15 (for example, constantly or periodically) (T2).

  The server 30 registers and manages information on the free resources of the cameras 10A and 10B in the table memory 35 (T3). The server 30 determines whether or not there is at least one camera having a free resource equal to or greater than a predetermined value (for example, 70%) (T4). Here, it is assumed that only the camera 10B has free resources equal to or greater than a predetermined value.

  When there is a camera having a free resource equal to or greater than a predetermined value (T4, YES), the server 30 generates an execution instruction for both detection and learning for the corresponding camera (T5). The server 30 transmits execution instructions for both detection and learning to the camera 10B via the network NW (T6). The camera 10B performs a corresponding process (T7).

  On the other hand, if there is no camera having free resources equal to or greater than the predetermined value in step T4 (T4, NO), the server 30 generates detection execution instructions for all cameras (here, cameras 10A and 10B) (T8). . Since the cameras 10A and 10B do not have enough free resources to perform learning, only the detection is performed. The server 30 transmits a detection execution instruction to all the cameras (here, the cameras 10A and 10B) (T9). The cameras 10A and 10B execute corresponding processes (T10 and T11).

  When the camera 10B executes the corresponding process in the procedure T7, the camera 10B generates a learning result (T12) and transmits the generated learning result to the server 30 (T13).

  FIG. 10 is a sequence diagram illustrating in detail an example of an operation procedure in which the server 30 controls the feedback amount of the model parameter in the first embodiment.

  In the operation procedure of FIG. 10, the server 30 controls the feedback amount of the model parameter based on the information on the free resources of the camera 10. The number N of cameras may be an arbitrary number, and here is two (cameras 10A and 10B) for ease of explanation. Instead of the server 30, the recorder 50 may determine the camera 10 that feeds back the model parameters.

  The server 30 receives model parameters as learning results from the cameras 10A and 10B, respectively, and accumulates them in the learning DB 34 (T21). The server 30 calculates the feedback amount of the model parameter of the learning model used at the time of the inference (detection) processing among the many model parameters that are the learning results according to the amount of free resources of each camera 10A, 10B. (T22).

  The server 30 transmits model parameter data corresponding to the calculated feedback amount to the camera 10B (T23). Similarly, the server 30 transmits model parameter data corresponding to the calculated feedback amount to the camera 10A (T24). The camera 10B additionally registers and stores the model parameters received from the server 30 in the memory of the process execution unit 14 (T25). Similarly, the camera 10A additionally registers and stores the model parameters received from the server 30 in the memory of the process execution unit 14 (T26).

  Here, the feedback amount is determined by the server 30 on the basis of information on the free resources of each camera. However, the number of correct reports detected based on teacher data and the number of false reports detected based on teacher data is not limited to free resources. May be determined according to

  FIG. 11 is an explanatory diagram of an example of an outline of sharing of learning results in the monitoring system 5.

  Each camera 10 (10A, 10B, 10C) performs local learning using captured image data obtained by imaging, and updates model parameters. Further, each camera 10 can perform learning using only the captured image data for which the correct report is obtained, and can improve the accuracy of the model parameter that is the learning result. In addition, the camera 10 can display a UI screen 320 (see FIG. 12) for evaluating captured image data in local learning on the display 19 connected as an option. The camera 10 can also display the UI screen 320 at the time of local learning on the display unit 37 of the server 30.

  FIG. 12 is a diagram showing a UI screen 320 displayed during local learning.

  The UI screen 320 is displayed on, for example, the display unit 37 of the server 30 or the display unit of a PC (not shown) that is connected to be communicable with the camera 10 during local learning of the camera 10, and specifically, from the captured image data. For each piece of learning data that has been cut out, a correct / incorrect determination, a camera ID, and a reject button bx are displayed. Note that thumbnails of captured image data are not displayed here because the camera 10 stores the original captured image data, but may be displayed. The detection object (object) obj is “person”.

  In the correctness / incorrectness determination process, the server 30 determines a correct report when the object obj can be detected in the captured image data, and determines a false report when the target object obj cannot be detected in the captured image data. It should be noted that the server 30 may determine whether the report is correct or incorrect by inputting to the UI screen 320 displayed on the display unit 37 of the server 30.

  The camera ID is identification information of a camera imaged to obtain learning data.

  The reject button bx is selected by the user and a check mark is displayed. The learning data in which the check mark is added to the reject button bx is not used for learning when the user presses the learning button bt5.

  The camera 10 is not automatically used for learning to adopt false image data, but used to learn to use correct image data. Instead of the camera 10, the user uses the reject button bx. You may instruct captured image data. For example, the user may instruct to use the captured image data of the correct report instead of the learning to adopt the captured image data of the incorrect report. Thereby, it is possible to learn using the erroneously captured image data. In addition, the camera 10 may use a combination of correct captured image data and erroneous captured image data for learning. Thereby, the captured image data used for learning can be selected in light of the quality of the captured image data.

  The server 30 receives the model parameter P transmitted from each camera 10 (10A, 10B, 10C), performs integrated learning by adding the received model parameters P, and adds the combined model parameter P to the learning DB 34. To do. Here, the model parameters to be integrated are model parameters obtained based on image data captured by cameras having the same installation situation. On the other hand, model parameters obtained based on image data captured by cameras with different installation situations are not added together but individually registered as model parameters for different learning models.

  FIG. 13 is a diagram illustrating a UI screen 310 displayed on the display unit 37 of the server 30 during integrated learning.

  The server 30 can display the UI screen 310 (see FIG. 13) at the time of integrated learning on the display unit 37. The UI screen 310 displays a correct / incorrect determination, a thumbnail, a camera ID, and a reject button bx for each learning data cut out from the captured image data. Here, a case where the detection target (object) is “person” is shown.

  In the correct / incorrect determination process, the server 30 determines that the target object obj is detected in the captured image data as a correct report, and determines that the target object obj is not detected in the captured image data as an incorrect report. It should be noted that the server 30 may determine whether the report is correct or incorrect by inputting to the UI screen 310 displayed on the display unit 37 of the server 30.

  The thumbnail is a reduced image of learning data. Since it is a thumbnail, when it is transmitted from the camera 10 to the server 30, the data transfer amount can be suppressed. The camera ID is identification information of a camera imaged to obtain learning data.

  The reject button bx is selected by the user and a check mark is displayed. The learning data in which the check mark is added to the reject button bx is not used for learning when the user presses the learning button bt5.

  The server 30 automatically learns to adopt the correct captured image data (that is, learns to accumulate the model parameters used to detect the correct captured image data), and erroneously captures the captured image data. (I.e., learning so as not to accumulate model parameters used for detection of mis-reported captured image data). However, regarding selection of captured image data used for learning, the user may instruct the captured image data used for learning by using the reject button bx. In addition, the user may instruct to perform the learning using the correct captured image data without performing the learning to exclude the erroneously captured image data.

  As described above, the server 30 performs integrated learning of model parameters, thereby improving the accuracy of model parameter learning. The server 30 transmits the updated model parameter, which is the result of the integrated learning, to the corresponding camera 10 as a feedback. Thereby, the detection accuracy of a camera becomes high, so that the correct report of the captured image data obtained with each camera 10 increases.

  Further, the server 30 controls the amount of feedback according to the number of correct reports of each camera 10 when the updated model parameter P, which is the result of the integrated learning, is transmitted as feedback to each camera 10. That is, the server 30 transmits the updated model parameter to the camera 10 with a large number of false alarms so that the feedback amount (for example, the number of feedbacks) increases. As a result, the number of correct reports increases and the detection accuracy of the camera improves.

  On the other hand, the server 30 transmits the updated model parameters to the camera 10 having a large number of correct reports so that the feedback amount (for example, the number of feedbacks) is reduced. Thereby, the processing load of the camera can be reduced. As described above, the server 30 transmits and shares the same updated model parameter to cameras having the same installation environment.

  Further, when the server 30 instructs each camera 10 to execute learning, the server 30 instructs the amount of learning according to the number of correct reports of each camera 10. A learning execution instruction is given to the camera 10 having a large number of false reports so that the learning amount increases. As a result, the number of correct reports increases and the detection accuracy of the camera improves. On the other hand, the server 30 instructs the camera 10 having a large number of correct reports to execute learning so that the learning amount decreases. Thereby, the processing load of the camera can be reduced.

  In addition, the server 30 may integrate and manage detection results for detecting an object appearing in a captured image captured by each camera 10. When integrating the detection results, the motion of the object may be represented by a vector, and the detection result may be managed by the vector.

  As described above, in the monitoring system 5 of the first embodiment, the server 30 and the plurality of cameras 10 installed in the monitoring area SA are connected to be communicable with each other. The server 30 includes a table memory 35 that holds free resources (that is, information regarding processing capability) of each camera 10 and captured image data obtained by capturing the monitoring area SA by each camera 10. The server 30 performs, for each camera 10, a process executed by the camera 10 for detecting at least one object (object) obj appearing in a captured image obtained by each camera 10 based on information on the processing capability of the camera 10. An instruction to execute the determined process is transmitted for each camera 10. Each camera 10 executes a process corresponding to the execution instruction based on the execution instruction of the process transmitted from the server 30.

  Thereby, when detecting at least one object in each captured image captured by the plurality of cameras 10 installed in the monitoring area SA, the monitoring system 5 performs a plurality of processes such as learning of parameters used for the detection. It is possible to distribute among the cameras 10, suppress an increase in traffic on the network, and assist in reducing the processing load of the server 30 connected to the plurality of cameras 10.

  Further, the above-described processing is learning for learning a model parameter P used for detection of at least one object (object) obj appearing in the captured image. Thereby, the monitoring system 5 can distribute learning with a large load to the plurality of cameras 10.

  Further, the server 30 transmits learning execution instructions to the plurality of cameras 10. Each of the plurality of cameras 10 performs learning in accordance with a learning execution instruction. The server 30 receives the results of learning performed by the plurality of cameras 10. Thereby, the server 30 can obtain the learning result from the plurality of cameras 10 without learning by the own device, for example.

  In addition, the server 30 performs learning by itself and transmits a learning execution instruction to each of the plurality of cameras 10. Each of the plurality of cameras 10 performs learning in accordance with a learning execution instruction. The server 30 receives the results of learning performed by the plurality of cameras 10. Thereby, the server 30 can add the learning result of the own apparatus to the learning results obtained from the plurality of cameras 10, and can improve the efficiency of learning from the next time onward.

  In addition, the server 30 transmits the learning result to the plurality of cameras. The plurality of cameras 10 share the learning result. Thereby, the plurality of cameras can use the same learning result.

  In addition, some of the plurality of cameras 10 are installed in the same installation situation. The server 30 transmits the learning result to each of a plurality of cameras 10 having the same installation status. Some of the plurality of cameras 10 having the same installation status share the learning result transmitted from the server 30. Thereby, the monitoring system 5 can improve the detection accuracy of the object by the some camera 10 with the same installation condition.

  Further, the server 30 controls the processing amount of learning in accordance with the number of objects (objects) obj detected by the camera 10. As a result, the server 30 can reduce the amount of learning that increases the load on a camera with a large number of detected objects and a large processing load. On the other hand, the server 30 can increase the amount of learning for a camera with a large number of detected objects and a small processing load. Therefore, the processing load of the camera is made uniform.

  Further, the server 30 controls the processing amount of learning in the camera 10 according to the number of correct reports of detection of the object (object) obj detected by the camera 10. Thereby, the server 30 can improve the accuracy of the learning result (in other words, the accuracy of detection after the next time) by using many results of correct learning.

  Further, the server 30 controls the amount of learning in the camera 10 according to the number of false reports of detection of the object (object) obj detected by the camera 10. Accordingly, the server 30 can improve the accuracy of the learning result (in other words, the accuracy of detection after the next time) as a result by not using the learning result of the false alarm.

  Further, the server 30 controls the learning processing amount in the camera 10 according to the amount of processing capability of the camera 10. Thereby, the server 30 can distribute learning to a plurality of cameras without imposing a large load on a specific camera, and can realize efficient learning.

  In addition, the server 30 holds information regarding the processing capabilities of the server 30 and the plurality of cameras 10 constituting the monitoring system 5 in the table memory 35. The server 30 controls the processing amount of learning according to the amount of processing capacity of each of the server 30 and the plurality of cameras 10. Thus, the server 30 can distribute learning to a plurality of devices without imposing a large load on a specific device of the monitoring system, and can realize efficient learning.

  Further, the above processing is performed by learning to learn a model parameter P used for detection of at least one object (object) obj appearing in the captured image og, and at least one object (object) obj appearing in the captured image og. Detection to detect, and analysis to analyze at least one object obj detected by the detection. Thereby, the server 30 can distribute the processing to the plurality of cameras 10 not only for learning but also for detection and analysis.

  Further, the server 30 transmits the captured image data held in the table memory 35 to at least one of the plurality of cameras 10 having a relatively high processing capability as compared with the other cameras, and learning. The execution instruction is performed. As a result, the server 30 can also transmit the captured image data to another camera with high processing capability when the network bandwidth is wide or when the network is free. Can be improved.

  In addition, the server 30 transmits and detects the captured image data held in the table memory 35 to at least one of the plurality of cameras 10 having a relatively high processing capability compared to the other cameras. The execution instruction is performed. As a result, the server 30 can also transmit captured image data to another camera with high processing capability when the network bandwidth is wide or when the network is free. Can be improved.

  In addition, the server 30 transmits captured image data held in the table memory 35 to at least one camera 10 having a relatively high processing capability compared to other cameras among the plurality of cameras 10 for analysis. The execution instruction is performed. As a result, the server 30 can also transmit the captured image data to another camera with high processing capability when the network bandwidth is wide or when the network is free. Can be improved.

  Moreover, the said process is a detection which detects the at least 1 target object (object) obj which appears in the captured image og. The server 30 transmits the detection result to each of the plurality of cameras 10. The plurality of cameras 10 share the detection result. Thereby, the server 30 can distribute detection to a plurality of cameras without imposing a large load on a specific camera, and can increase the efficiency of detection.

  The server 30 also integrates the results of learning performed by the plurality of cameras 10. Thereby, the monitoring system 5 can improve the accuracy of the learning result aggregated by the integration in the server 30.

  In addition, some of the plurality of cameras 10 are installed in the same installation situation. The server 30 integrates the results of learning performed by a plurality of cameras 10 with the same installation status of the cameras 10. Thereby, the server 30 can raise the detection accuracy of the object by the camera of the same installation condition.

  The server 30 receives notification of information regarding the installation status of each camera from a plurality of cameras 10 having the same installation status, and integrates the learning results executed by the some cameras 10. . This makes it easy for the server 30 to integrate the learning results from cameras with the same installation status.

  Further, the server 30 and the plurality of cameras 10 share information on the processing capabilities of the plurality of cameras 10 and information on unit cost of the plurality of cameras 10 (for example, information on the power cost of each camera 10). Thus, the server 30 can execute processing instructions for each device such as the server and the plurality of cameras in consideration of available resources and power costs, and various operations can be performed.

(Background to the second embodiment)
In the related art such as Patent Document 1 described above, it is disclosed that the score of the evaluation function related to an object is used in order to obtain object motion information as a correct answer of the object to be tracked in the captured image. However, no particular consideration has been given to controlling the learning amount of a parameter necessary for object detection according to the score indicating the object detection accuracy. For this reason, there is a concern that learning of parameters used for detection, which originally does not require learning, causes variations in parameter learning accuracy and affects object detection accuracy.

  Therefore, in the second embodiment, the parameters used for detection are determined according to the score indicating the detection accuracy of the object obtained by detecting at least one object in the captured image captured by the camera installed in the monitoring area. An example of a monitoring system and monitoring method that appropriately controls the learning amount of the camera and improves the learning accuracy of the camera, and a camera and parameter registration method will be described.

(Embodiment 2)
Since the system configuration of the monitoring system 5 according to the second embodiment is the same as the system configuration of the monitoring system 5 according to the first embodiment described above, the description thereof is simplified or omitted by using the same reference numerals. The contents will be described.

  FIG. 14 is a block diagram illustrating in detail an example of the internal configuration of the processing execution unit 14 of the camera 10 according to the second embodiment.

  The processing execution unit 14 which is a main configuration of the camera 10 includes a teacher data set memory 151 and a parameter memory 152 in addition to a neural network (that is, the NN 140).

  The NN 140 has functions of an object inference function 141, a score derivation function 142, a correctness determination function 143, and a parameter learning function 144.

  In the object inference function 141 as an example of the detection unit, the NN 140 infers (that is, detects) what the target appears in the captured image according to the model parameter.

  In the score derivation function 142 as an example of the derivation unit, the NN 140 derives a score (evaluation value) indicating the detection accuracy of the object at the time of inference using the teacher data registered in the teacher data set memory 151, and the score Is output.

  In the correctness determination function 143, the NN 140 derives the correctness / incorrectness determination of the object using the teacher data registered in the teacher data set memory 151 at the time of inference, and outputs the determination result.

  In the parameter learning function 144 as an example of the parameter learning unit, the NN 140 learns to adopt model parameters used for inferring an object having a high score. In the parameter learning function 144, the NN 140 learns to exclude model parameters used for inference of an object with a low score. The NN 140 registers and stores the learned model parameters in the parameter memory 152. A model parameter used for inferring an object having a higher score than a first predetermined value (for example, 80 points) registered in the parameter memory 152 is transmitted to the server 30 in sharing of learning results described later, and used in integrated learning. Is done.

  FIG. 15 is a flowchart illustrating in detail an example of an operation procedure of local learning of the camera 10.

  In FIG. 15, the camera 10 captures an object from a subject image in the image sensor 12 (S1), and generates captured image data (S2).

  The process execution unit 14 receives the captured image data, and infers (detects) at least one object (that is, an object) appearing in the captured image (S3). The process execution unit 14 performs scoring processing of at least one object at the time of inference (detection) (S4). In this scoring process, the process execution unit 14 outputs an object score (evaluation value) using the teacher data registered in the teacher data set memory 151.

  As a result of the scoring process, the process execution unit 14 infers the model parameter used for inferring an object having a higher score than the first predetermined value (for example, 80 points) and the object having a second predetermined value (for example, 10 points) lower score. The model parameters of NN 140 are learned using the model parameters used in (S5). In step S <b> 5, the process execution unit 14 registers and accumulates model parameters having higher scores than the first predetermined value in the parameter memory 152. Thereafter, the camera 10 ends the process shown in FIG.

  The higher score is, for example, 80 to 100 points. The lower score is, for example, 0 to 10 points. The process execution unit 14 adopts, for example, a model parameter with a higher score and eliminates a model parameter with a lower score.

  An object with a higher score is an object with a high possibility of correct reporting, and an object with a lower score is an object with a high possibility of erroneous reporting. Therefore, by learning to adopt the model parameters used for the inference (detection) of the object with the higher score, the model parameters applied to the estimation of the object with the high possibility of correct reporting can be used, and the model The parameter learning accuracy can be improved.

  In addition, by learning to eliminate the model parameters used for inferring objects with lower scores, the model parameters applied to the estimation of objects with high possibility of false alarms are not used, improving the accuracy of model parameter learning. Can be made.

  Further, the process execution unit 14 may learn by combining the model parameter used for inferring an object with a higher score and the model parameter used for inferring an object with a lower score. In this way, the learning accuracy of the model parameters can be further improved by combining the model parameters used for inferring the object with the higher score and the model parameters used for inferring the object with the lower score. .

  FIG. 16 is an explanatory diagram of an outline example of sharing of learning results in the monitoring system 5.

  Each camera 10 (10A, 10B, 10C) performs local learning using captured image data obtained by imaging. In local learning, each camera 10 executes scoring processing related to the detection accuracy of the object detected in the captured image data, and learns the model parameters of the NN 140 according to the obtained score. Each camera 10 learns by adopting only model parameters used for inference (detection) of an object having a higher score. Thereby, the learning accuracy of model parameters can be improved.

  In local learning, a UI screen 340 (see FIG. 17) for evaluating captured image data can be displayed. For example, the camera 10 may display the UI screen 340 on a display (not shown) connected to the camera 10 as an option, or may display the UI screen 340 by transferring it to the display unit 37 of the server 30. Is possible.

  FIG. 17 is a diagram illustrating an example of a UI screen 340 displayed during local learning.

  The UI 340 displays a score, a camera ID, and a reject button bx for each learning data cut out from the captured image data. The thumbnail of the image data is not displayed here because the camera 10 stores the original captured image data, but it may be displayed. The detection target (object) is “person”.

  The score is quantified in the range of 0 to 100 points. The score is calculated by the scoring process performed by each camera 10, but may be acquired by the user inputting from the UI 320. The camera ID is identification information of a camera imaged to obtain learning data. When the reject button bx is selected by the user, a check mark is displayed. The learning data in which the check mark is added to the reject button bx is not used for learning when the user presses the learning button bt5.

  The camera 10 did not automatically learn to adopt captured image data with a lower score, but performed learning to adopt captured image data with a higher score. However, for example, instead of the camera 10, the user may instruct captured image data used for learning using the reject button bx. For example, the user may instruct to perform learning that adopts the captured image data of the higher score without performing the learning to exclude the captured image data of the lower score.

  Further, each camera 10 has learned to use only the captured image data of the higher score, but may instruct the learning to exclude only the captured image data of the lower score, for example. Thereby, learning can be performed using the captured image data from which the captured image data of the lower score is excluded. Further, it may be set so that the captured image data of the higher score and the captured image data of the lower score are used in combination. Thereby, in view of the quality of the captured image data, the image data used for learning can be individually selected by the camera or the user.

  The server 30 receives the model parameters transmitted from each camera 10 (10A, 10B, 10C), performs integrated learning that adds the received model parameters, and adds the combined model parameters to the learning DB 34. Here, the model parameters to be integrated are model parameters obtained based on image data captured by cameras having the same installation situation. On the other hand, model parameters obtained based on image data captured by cameras with different installation situations are not added together but individually registered as model parameters for different learning models.

  FIG. 18 is a diagram illustrating an example of a UI screen 350 displayed on the display unit 37 of the server 30 during integrated learning.

  The server 30 can display the UI 310 (see FIG. 18) at the time of integrated learning on the display unit 37. The UI 350 displays a score, a thumbnail, a camera ID, and a reject button bx for each learning data cut out from the captured image data. Here, a case where the detection target (object) is “person” is shown.

  The score is quantified in the range of 0 to 100 points. For example, when the target is “person”, the score of the image data in which the person is shown is as high as 80 to 100 points. On the other hand, the score of image data in which “tree” is reflected instead of a person is as low as 10. The thumbnail is a reduced image of learning data. Since it is a thumbnail, when it is transmitted from the camera 10 to the server 30, the data transfer amount can be suppressed. The camera ID is identification information of a camera imaged to obtain learning data. The reject button bx is selected by the user and a check mark is displayed. The learning data in which the check mark is added to the reject button bx is not used for learning when the user presses the learning button bt5.

  The server 30 does not automatically learn captured image data having a higher score, but performs learning that eliminates captured image data having a lower score. However, for example, the user may instruct captured image data used for learning using the reject button bx. For example, the user may instruct to perform learning that adopts the captured image data of the higher score without performing the learning to exclude the captured image data of the lower score.

  As described above, the server 30 performs integrated learning of model parameters, thereby improving the accuracy of model parameter learning. The server 30 transmits the updated model parameter, which is the result of the integrated learning, to the corresponding camera 10 as a feedback. Thereby, the detection accuracy of a camera becomes high, so that the correct report of the image data obtained with each camera 10 increases.

  The server 30 also controls the amount of feedback according to the number of correct reports of each camera 10 when the updated model parameter, which is the result of the integrated learning, is transmitted to each camera 10 in a feedback manner. That is, the server 30 transmits the updated model parameter to the camera 10 with a large number of false alarms so that the feedback amount (for example, the number of feedbacks) increases. As a result, the number of correct reports increases and the detection accuracy of the camera improves.

  On the other hand, the server 30 transmits the updated model parameters to the camera 10 having a large number of correct reports so that the feedback amount (for example, the number of feedbacks) is reduced. Thereby, the processing load of the camera can be reduced. As described above, the server 30 transmits and shares the same updated model parameter to cameras having the same installation environment.

  In addition, when the server 30 instructs each camera 10 to execute learning, the server 30 instructs the processing amount of learning according to the number of correct reports of each camera 10. A learning execution instruction is given to the camera 10 having a large number of false reports so that the learning amount increases. As a result, the number of correct reports increases and the detection accuracy of the camera improves. On the other hand, the server 30 instructs the camera 10 having a large number of correct reports to execute learning so that the learning amount decreases. Thereby, the processing load of the camera can be reduced.

  In addition, the server 30 may integrate and manage detection results for detecting a target appearing in an image captured by each camera 10. When integrating the detection results, the motion of the target may be represented by a vector, and the detection result may be managed by the vector.

  As described above, the camera 10 according to the second embodiment is a camera used in the monitoring system 5 installed in the monitoring area SA and connected to the server 30 so as to be able to communicate with each other. The camera 10 images the subject light from the monitoring area SA in the image sensor 12. In the object inference function 141 as an example of a detection unit, the camera 10 detects at least one object appearing in the captured image using the captured image og based on the imaging of the subject light. The camera 10 holds a teacher data set prepared for each object type in the teacher data set memory 151. In the score derivation function 142 as an example of a derivation unit, the camera 10 derives a score indicating the detection accuracy of the detected object using the teacher data set. In the parameter learning function 144 as an example of the parameter learning unit, the camera 10 learns model parameters used for object detection according to the derived score. In the parameter learning function 144, the camera 10 registers and accumulates model parameter learning results in the parameter memory 152.

  As a result, the camera 10 determines the parameters used for detection according to the score indicating the detection accuracy of the object obtained by detecting at least one object in the captured image captured by the camera installed in the monitoring area. It is possible to appropriately control the learning amount and improve the learning accuracy in the camera.

  In addition, the parameter learning function 144 learns to adopt a model parameter from which a score higher than the first predetermined value is derived. As described above, the camera 10 learns to adopt the model parameter used for the inference of the object having the higher score, so that the model parameter applied to the estimation of the object having a high possibility of correct report is used. Thus, the learning accuracy of the model parameters can be improved.

  Further, the parameter learning function 144 performs learning so as to exclude model parameters for which a score lower than the second predetermined value is derived. As described above, the camera 10 learns to exclude the model parameter used for the inference of the object having the lower score, so that the model parameter applied to the estimation of the object having a high possibility of misreporting is not used. The parameter learning accuracy can be improved.

  Further, the parameter learning function 144 learns to adopt a model parameter from which a score higher than the first predetermined value is derived, and excludes a model parameter from which a score lower than the second predetermined value is derived. To learn. In this way, the camera 10 learns by combining the model parameter used for inferring the object with the higher score and the model parameter used for inferring the object with the lower score, thereby further improving the learning accuracy of the model parameter. Can be made.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood. In addition, the constituent elements in the above-described embodiment may be arbitrarily combined within the scope not departing from the spirit of the invention.

  For example, in the above-described embodiment, the monitoring system is applied to a security monitoring system that detects and tracks a suspicious person such as a thief. The present invention may be applied to a monitoring system for product inspection.

  In the present disclosure, the learning amount of a parameter used for detection is determined according to a score indicating detection accuracy of an object obtained by detecting at least one object in a captured image captured by a camera installed in a monitoring area. It is useful as a camera and parameter registration method for appropriately controlling and improving learning accuracy in the camera.

5 Monitoring System 10, 10A, 10B, 10C Camera 11 Lens 12 Image Sensor 13 Signal Processing Unit 14 Processing Execution Unit 15 Resource Monitoring Unit 16 Network I / F
17 Crop Encoding Unit 30 Server 31 Processor 32 Memory 33 Communication Unit 34 Learning DB
35 Table memory (memory)
36 Operation unit 37 Display unit 50 Recorder 150 Device NW Network P Model parameter

Claims (5)

A camera used in a surveillance system installed in a surveillance area and connected to a server so as to communicate with each other.
An imaging unit for imaging subject light from the monitoring area;
A detection unit that detects at least one object appearing in the captured image using a captured image based on the imaging of the subject light in the imaging unit;
A memory for holding a teacher data set prepared for each type of the object;
A derivation unit for deriving a score indicating the detection accuracy of the object detected by the detection unit using the teacher data set;
A parameter learning unit that learns parameters used for detection of the object according to the score derived by the deriving unit;
The parameter learning unit registers and accumulates the learning result of the parameter in the memory;
camera. The parameter learning unit learns to adopt the parameter from which the score higher than the first predetermined value is derived.
The camera according to claim 1. The parameter learning unit learns to exclude the parameter from which the score lower than a second predetermined value is derived;
The camera according to claim 1. The parameter learning unit learns to employ the parameter from which the score higher than the first predetermined value is derived, and excludes the parameter from which the score lower than the second predetermined value is derived. To learn,
The camera according to claim 1. A parameter registration method using a camera installed in a monitoring area,
Imaging subject light from the monitoring area;
Detecting at least one object appearing in the captured image using a captured image based on imaging of the subject light;
Deriving a score indicating the detection accuracy of the detected object using a teacher data set prepared for each type of the object;
Learning parameters used for detecting the object according to the derived score;
Registering and storing the learning result of the parameter in a memory, and
Parameter registration method.

Images