JP2018173913A - Image processing system, information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of improving recognition rate without using a high-performance identification apparatus.SOLUTION: An image processing system 100 which identifies a predetermined alarm cause from an image taken by an imaging apparatus 10 comprises: an image acquisition means 31 for acquiring the image from the imaging apparatus; a first identification means 34a which identifies the alarm cause captured in the image and calculates degree of certainty of the alarm caused identified; a display process means 32 for displaying the image on a display device; a reception means 33 for receiving input of the alarm cause captured in the image displayed on the display device; an image extraction means 35 which stores the image into an image storage part when the degree of certainty, calculated by the first identification means and for identifying the alarm cause received by the reception means, is equal to or higher than a first threshold value; and an identification means construction means 36 for constructing a second identification means 34b by utilizing the image stored in the image storage part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing system, an information processing apparatus, and a program.

  Image processing is known in which an identification device recognizes an image captured by a camera and detects the cause of an alarm reflected in the image. Since a task similar to that of a human being can be realized by a computer, the function of the identification device may be called artificial intelligence (AI). Artificial intelligence refers to software or a computer that mimics the intellectual work performed by the human brain.

  However, since there are images that are difficult to identify, the identification device may misrecognize or unrecognize the cause of the alarm, and a technique for improving the recognition rate has been devised (see, for example, Patent Document 1). In Patent Literature 1, a person confirms an image that has been misrecognized or unrecognized by a vehicle-mounted identification device, and a moving image of several seconds is transmitted to a higher-performance identification device, which learns from the moving image. A learning method for extracting a power image is disclosed. A vehicle identification device learns an image extracted by a high-performance identification device, evaluates the effect of learning by simulation, and updates the learning result when it is improved. According to such a process, the recognition rate of the vehicle identification device can be gradually improved.

JP 2011-59810 A

  However, the method disclosed in Patent Document 1 has a problem that it is necessary to prepare a high-performance identification device as compared with the on-vehicle identification device. In general, since a high-performance identification device requires a large amount of learning images, a large amount of learning images must be collected manually or by appropriate information processing. In addition, the learning takes time as the amount of images to be learned increases.

  In view of the above problems, an object of the present invention is to provide an image processing system capable of improving the recognition rate without using a high-performance identification device.

  The present invention is an image processing system for identifying a predetermined alarm cause from an image captured by an imaging device, the image acquisition means for acquiring the image from the imaging device, and the alarm cause reflected in the image And a first identification means for calculating the probability that the cause of the alarm is identified, a display processing means for displaying the image on a display device, and the image displayed on the display device An accepting means for accepting an input of an alarm cause, and an image storage unit storing the image when the accuracy of identifying the cause of the alarm calculated by the first identifying means and accepted by the accepting means is greater than or equal to a first threshold value Image extracting means for performing the identification, and identifying means constructing means for constructing a second identifying means using the image stored in the image storage unit.

  According to the present invention, it is possible to provide an image processing system capable of improving the recognition rate without using a high-performance identification device.

It is a figure explaining the outline of the operation | movement which an image processing system extracts the warning image of learning object. 1 is an example of a system configuration diagram of an image processing system. It is an example of the schematic hardware block diagram of a camera apparatus. It is an example of the block diagram which showed the schematic hardware constitutions of information processing apparatus. It is an example of the functional block diagram which illustrated each function with which an image processing system is provided. It is a figure which shows an example of the circumscribed rectangle which the camera apparatus drawn on the alarm image. It is a figure which shows the model called the multilayer perceptron in the neural network which is an example of the model of learning. It is an example of the alarm cause input screen displayed on the LCD. It is an example of the figure explaining the extraction method of the alarm image used for learning. It is a figure which shows the accuracy of the alarm cause which the monitoring person input among the accuracy of each alarm cause calculated from the some alarm image in time series. It is an example of the flowchart figure explaining the process which information processing apparatus extracts a warning image and stores in learning DB. It is an example of the figure explaining the outline of evaluation of the identification device by information processing apparatus. (Example 2) which is an example of the functional block diagram which illustrated each function with which an image processing system is provided. It is an example of the figure which shows the identification result at the time of identifying the image for evaluation in each of a warning cause estimation part (present) and a warning cause estimation part (new). It is an example of the figure which shows the accuracy at the time of identifying the image for evaluation in each of a warning cause estimation part (present) and a warning cause estimation part (new). It is an example of the flowchart figure which shows the procedure which judges whether information processing apparatus evaluates and updates the performance of an alarm cause estimation part.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<Outline of operation of image processing system>
FIG. 1 is a diagram for explaining an outline of an operation in which the image processing system 100 extracts an alarm image 6 to be learned. The image processing system 100 according to the present embodiment relates to a technique for detecting an object shown in an image by image processing. This object is called an alarm cause because it causes an alarm.

  When the camera device 10 at the monitoring location 8 detects an abnormality, it transmits an alarm image 6 to the monitoring center 9. Conventionally, the monitoring staff 7 of the monitoring center 9 has visually checked the alarm images 6 one by one to determine, for example, whether an intruder has been detected. Since the alarm image 6 includes images of at least several seconds before and after the time when the abnormality is detected, it is a burden for the supervisor 7 to visually check each image.

  Therefore, it has been studied to reduce the burden on the supervisor 7 by adopting a mechanical identification device at the monitoring center 9 to identify the cause of the alarm.

  For this purpose, since the recognition rate of the identification device is desirably high, a technique for improving the recognition rate will be described in this embodiment. When the alarm image 6 with the accuracy equal to or greater than the threshold a (second threshold) is not included in the alarm image 6 in which the identification device has identified the alarm cause, the monitor 7 looks at the alarm image 6 and inputs the alarm cause. Then, the alarm image 6 having the alarm cause accuracy equal to or higher than the threshold value b (first threshold value) is learned as new learning data. Therefore, even if there is no alarm image with sufficiently high accuracy in the alarm image 6 and the identification device cannot estimate or determine the alarm cause, the accuracy of the alarm cause estimated or determined by the monitor 7 is more than a certain level. In this case, since the alarm image 6 is learned, the recognition rate can be improved.

An outline of the operation of the image processing system 100 will be described with reference to FIG.
(1) First, the camera apparatus 10 detects a moving object and / or an animal that emits far infrared rays. Animals and the like are hereinafter referred to as detection targets.
(2) The camera device 10 transmits an alarm image 6 of several seconds before and after the time when the detection target is detected to the monitoring center 9.
(3) The alarm cause estimation unit 34 is an identification device that has learned in advance the relationship between the alarm cause and the image. The alarm cause estimation unit 34 identifies the alarm image 6 and calculates each alarm cause and its accuracy for each alarm image 6.
(4) If the accuracy is sufficiently high (accuracy is greater than or equal to threshold a), it may be determined that there is a high possibility that the cause of the alarm with accuracy greater than or equal to the threshold a is reflected in the alarm image 6; Focusing on the alarm cause, the alarm image 6 having the accuracy of the threshold value b or higher is stored in the learning DB 392 (Data Base) together with the recognition result (alarm cause).
(5) On the other hand, when there is no alarm image 6 with the accuracy less than the threshold value a, the supervisor 7 visually checks the alarm image 6 with the accuracy less than the threshold value a and inputs the cause of the alarm. Even if there are a plurality of alarm images 6, the same alarm cause is reflected in one alarm, so the supervisor 7 only needs to confirm the plurality of alarm images 6 and input one alarm cause.
(6) The learning image automatic extraction unit 35 stores only the alarm image 6 with the accuracy of the alarm cause input by the monitoring person 7 with the threshold value b or more together with the input alarm cause in the learning DB 392.

  As a result, the learning DB 392 stores the alarm image 6 with high alarm cause accuracy (threshold a or b or more) together with the alarm cause. Since the learning unit described later learns the alarm image 6 using the cause of the alarm as a teacher signal, an identification device with an improved recognition rate can be created. Moreover, the input burden of the alarm cause by the monitoring person 7 can be reduced, so that a recognition rate improves.

<Terminology>
An alarm cause refers to an object that causes an alarm. For example, it is a detection target such as a moving object shown in an alarm image or an animal in which far infrared rays are detected.

  Moreover, although identification is identifying the cause of the alarm shown in the image, it may be expressed as specifying, judging, or recognizing.

  The accuracy is the degree or method of certainty. In the present embodiment, when an alarm cause is identified, how much it is sure is shown. It may be referred to as likelihood or probability.

<System configuration example>
FIG. 2 shows an example of a system configuration diagram of the image processing system 100. The image processing system 100 mainly includes a camera device 10 and an information processing device 30 connected via a network N. The camera device 10 is an imaging device that is installed in the monitoring place 8 and captures several images per second. The monitoring place 8 is a place where the entrance and exit of a third party such as an office, a private residence, a hotel, a store, a laboratory, a warehouse, and a factory can be monitored. Not limited to indoors, it may be installed outdoors such as parking lots, competition facilities, forests, coastal areas, and restricted areas.

  The camera device 10 has not only a function of capturing an image but also an image processing function of detecting a moving object and a function of detecting far infrared rays emitted from the target. When the moving object is reflected in the image and / or when far-infrared rays are detected, the camera device 10 transmits the images before and after that to the monitoring center 9 as the alarm image 6. The detection target that can be imaged by the camera device 10 being monitored is various and varies depending on the monitoring location 8, and is, for example, an insect, a bird, a small animal (a mouse, a cat, a dog, a squirrel, a lizard, etc.) and a person. Although it is a person that the camera apparatus 10 should detect at many monitoring locations 8, if only the person is detected, the image processing may be complicated, or even if there is a person, it may not be detected. Therefore, a detection target such as a moving object and / or an animal that emits far infrared rays is detected.

  The network N is constructed by a LAN constructed in the monitoring place 8 where the camera device 10 is installed, a provider network of a provider that connects the LAN to the Internet, a line provided by a circuit provider, and the like. When the network N has a plurality of LANs, the network N is called WAN and may include the Internet. The network N may be constructed by either wired or wireless, and wired and wireless may be combined. When the camera device 10 is connected to a wireless public network such as 3G, 4G, 5G, or LTE (Long Term Revolution), the camera device 10 can be connected to the provider network without going through the LAN.

  The monitoring center 9 is a place where the supervisor 7 confirms the alarm image 6 and determines whether or not the detection target is an alarm target. The information processing apparatus 30 has an identification device that improves the recognition rate as described in the present embodiment. The information processing apparatus 30 identifies the detection target, and if it is a person, it determines that it is the target of the alarm, or the supervisor 7 performs final confirmation, and dispatches the guard to the monitoring place 8. .

  The information processing apparatus 30 is an apparatus called a general computer, a PC (Personal Computer), a server, or the like. In FIG. 2, for convenience of explanation, one information processing apparatus 30 is shown. However, the information processing apparatus 30 is not limited to one, and a plurality of information processing apparatuses 30 share the functions described in FIG. 1. It may be.

<Hardware configuration example>
FIG. 3 is an example of a hardware configuration diagram of the camera apparatus 10. The camera device 10 includes an imaging unit 101, an image processing IC 103, a far infrared sensor 102, a signal processing IC 104, a ROM 105, a CPU 106, a RAM 107, and a communication device 108.

  The imaging unit 101 is a camera having a lens, a diaphragm, a shutter (mechanical / electronic), a solid-state imaging device such as a CMOS or a CCD, and a memory. The image may be black and white or color. The imaging unit 101 periodically captures a predetermined range under the control of the CPU 106 and sends the image to the image processing IC 103. The image processing IC 103 is an integrated circuit that performs image processing for detecting a moving object on an image.

  The far-infrared sensor 102 is a sensor that detects a far-infrared ray emitted from an object to detect a temperature change. Specifically, a pyroelectric element or a thermopile sensor is known. It has a lens, and guides far-infrared rays that enter the angle of view of the lens to a pyroelectric element or a thermopile sensor, and converts far-infrared rays radiated mainly by thermostat animals into electrical signals. The signal processing IC 104 is an integrated circuit that monitors an electrical signal and determines whether there is a temperature change.

  The CPU 106 controls the entire camera device 10 by executing a program stored in the ROM 105 using the RAM 107 as a working memory. That is, the imaging by the imaging unit 101 is controlled, or the detection of far infrared rays by the far infrared sensor 102 is controlled. In addition, the alarm device 6 is transmitted to the monitoring center 9 by controlling the communication device 108.

  The communication device 108 is called a network interface or an Ethernet (registered trademark) card, and provides a function of connecting to a network. You may connect to the access point of a wireless LAN, or the base station of a mobile telephone network.

  FIG. 4 is a block diagram illustrating a schematic hardware configuration of the information processing apparatus 30. The information processing apparatus 30 can be generally implemented as a personal computer, a workstation, or an appliance server. The information processing apparatus 30 includes a CPU 201 and a memory 202 that enables high-speed access to data used by the CPU 201. The CPU 201 and the memory 202 are connected to other devices or drivers of the information processing apparatus 30 such as a graphics driver 204 and a network driver (NIC) 205 via the system bus 203.

  An LCD (display device) 206 is connected to the graphics driver 204 and executes a display instruction from the CPU 201 to display a screen. A touch panel may be integrally disposed on the LCD 206. In this case, the monitor 7 can operate the information processing apparatus 30 using a finger as an operation means.

  The network driver 205 connects the information processing apparatus 30 to the network N at the transport layer level and the physical layer level to establish a session with the camera apparatus 10 and the like.

  An I / O bus bridge 207 is further connected to the system bus 203. On the downstream side of the I / O bus bridge 207, a storage device such as an HDD 209 is connected by an IDE, ATA, ATAPI, serial ATA, SCSI, USB, etc. via an I / O bus 208 such as PCI. . Instead of the HDD 209 or together with the HDD 209, an SSD (Solid State Drive) may be included.

  The HDD 209 stores a program 209p for controlling the entire information processing apparatus 30. The information processing apparatus 30 executes the program 209p to accept or learn the operation of the supervisor. The program 209p may be distributed in a state of being stored in a portable storage medium such as a USB memory or an optical storage medium in addition to being distributed from a server that distributes the program.

  Also, an input device 210 such as a keyboard and a mouse (referred to as a pointing device) is connected to the I / O bus 208 via a USB bus or the like, and accepts input and commands from an operator such as the supervisor 7. Yes.

  The information processing apparatus 30 may support cloud computing. Cloud computing refers to a usage pattern in which resources on a network are used without being aware of specific hardware resources. In this case, the hardware configuration shown in FIG. 4 does not need to be housed in a single housing or provided as a single device, and is preferably provided in the information processing device 30. Indicates an element.

<Functional configuration example of image processing system>
FIG. 5 is an example of a functional block diagram illustrating each function provided in the image processing system 100. In the present embodiment, functions related to the improvement of the recognition rate will be mainly described.

<< Functional configuration of camera device >>
The camera device 10 includes an image acquisition unit 11, an image processing unit 12, a far infrared detection unit 13, a temperature change determination unit 14, and a communication unit 15. Each of these functions is a function or means realized by the CPU 106 shown in FIG. 3 executing a program and cooperating with the hardware of the camera apparatus 10. It is not necessary to clearly distinguish functions implemented in hardware or software, and some or all of these functions may be implemented by a hardware circuit such as an IC.

  In addition, the camera apparatus 10 includes a storage unit 19 constructed by the ROM 105 or the RAM 107 illustrated in FIG. An image DB 191 is constructed in the storage unit 19. The image DB 191 does not have to be directly included in the camera apparatus 10 and may be in any place on the network that the camera apparatus 10 can access.

  The image acquisition unit 11 periodically acquires (captures images) regardless of whether or not there is a detection target. The imaging interval may be lengthened until the detection target is detected, and the imaging interval may be shortened when the detection target is detected. The image acquisition unit 11 is realized by the CPU 106 in FIG.

  The image processing unit 12 detects a moving object from a time-series image. For example, the image at time t is stored in the storage unit 19, and the presence / absence of change is determined by comparing the image at time t + 1 with each pixel or pixel block. It is determined that a moving object is reflected in the pixel whose pixel value changes, and a circumscribed rectangle is drawn on the changed pixel. The image processing unit 12 assigns a flag or the like to the image determined to have a moving object together with the image determined to have no moving object, and stores the image in the image DB 191. The image DB 191 overwrites the old image and always holds a certain number of new images.

  The far-infrared detector 13 periodically collects electromagnetic waves in the far-infrared wavelength region and performs A / D conversion to generate an electrical signal. The far-infrared detector 13 is realized by the CPU 106 in FIG. 3 controlling the far-infrared sensor 102.

  The temperature change determination unit 14 monitors the electrical signal generated by the far infrared ray detection unit 13 and determines whether or not there has been a temperature change. If it is determined that there is a temperature change, a moving object may be included in the image, so a flag or the like is assigned to the most recently captured image. This image is preferably one in which a moving object is detected by image processing (for example, when a moving object is detected from an image captured at the time when far-infrared light is detected, this can be achieved by adding a flag to the image. ). However, since a plurality of images before and after the flag given by the detection of the far-infrared are the warning image 6, even if a flag is attached to an image in which no moving object is detected by image processing, an image in which the moving object is detected is displayed. Including the camera device 10 can transmit the alarm image 6 to the information processing device 30.

  The communication unit 15 detects the image with the flag from the image DB 191 and transmits it to the information processing apparatus 30 as the alarm image 6. For example, several to several tens of images before and after a flagged image are transmitted. The communication unit 15 is realized by the CPU 106 of FIG.

<< Functional configuration of information processing equipment >>
The information processing apparatus 30 includes a communication unit 31, a display control unit 32, an operation reception unit 33, an alarm cause estimation unit 34, and a learning image automatic extraction unit 35. These functions are functions or means realized by the CPU 201 shown in FIG. 4 executing the program 209p and cooperating with the hardware of the information processing apparatus 30. It is not necessary to clearly distinguish functions implemented in hardware or software, and some or all of these functions may be implemented by a hardware circuit such as an IC.

  In addition, the information processing apparatus 30 includes a storage unit 39 constructed from the memory 202 or the HDD 209 illustrated in FIG. In the storage unit 39, an alarm image DB 391 and a learning DB 392 are constructed. These DBs do not have to be directly held by the information processing apparatus 30 and may be located at any location on the network accessible by the information processing apparatus 30.

  The communication unit 31 receives the alarm image 6 from the camera device 10. The received alarm image 6 is stored in the alarm image DB 391. The communication unit 31 is realized by the CPU 201 illustrated in FIG. 4 executing the program 209p to control the network driver 205, and the like.

  The display control unit 32 displays the alarm image 6 stored in the alarm image DB 391 on the LCD 206 by the operation of the monitor 7 for the input of the alarm cause or automatically. An example of the alarm cause input screen displayed by the display control unit 32 is shown in FIG. The display control unit 32 is realized by the CPU 201 illustrated in FIG. 4 executing the program 209p to control the graphics driver 204, for example.

  The operation accepting unit 33 accepts an operation of the monitor 7 for the information processing apparatus 30. In the present embodiment, an alarm cause input operation of the monitor 7 for the alarm image 6 displayed on the LCD 206 is received, and the alarm image 6 is registered in the alarm image DB 391 in association with the input alarm cause. The operation accepting unit 33 is realized, for example, by the CPU 201 illustrated in FIG. 4 executing the program 209p to control the input device 210.

  The alarm cause estimation unit 34 is an identification device that identifies a detection target from the alarm image 6. In the present embodiment, the recognition rate of the alarm cause estimating unit 34 can be improved. The alarm cause estimating unit 34 calculates the accuracy of the detection target shown in the alarm image 6 for each detection target by the identification function constructed by learning. The alarm cause estimating unit 34 stores the accuracy of each detection target in the alarm image DB 391 in association with the alarm image 6. The alarm cause estimating unit 34 is realized, for example, by the CPU 201 illustrated in FIG. 4 executing the program 209p.

  The learning image automatic extraction unit 35 reads each alarm cause registered in association with the alarm image 6 of the alarm image DB 391 and its accuracy, and first moves the alarm image 6 with the accuracy equal to or higher than the threshold value a to the learning DB 392. Further, the alarm image 6 having the alarm cause accuracy input by the monitor 7 and having a threshold value b or higher is moved to the learning DB 392. The learning image automatic extraction unit 35 is realized, for example, when the CPU 201 illustrated in FIG. 4 executes the program 209p.

表１は、警報画像ＤＢ３９１に記憶されている情報を模式的に示す。表１には警報画像６の画像番号に各警報原因の確度が対応付けられている。警報原因は、例えば、虫、鳥、小動物、及び、人等である。表１（ａ）は確度が閾値ａ以上の警報原因が写っている警報画像６を記憶する警報画像ＤＢ３９１を示す。表１（ｂ）は確度が閾値ａ以上の警報原因が写っている警報画像６を記憶せず、閾値ｂ以上の警報原因が写っている警報画像６を記憶する警報画像ＤＢ３９１を示す。一例として、監視員７が警報原因を入力することなく自動で警報画像６が学習ＤＢ３９２に記憶されるための閾値ａを０．７、監視員７が警報原因（例えば、小動物）を入力した場合に警報画像６が学習ＤＢ３９２に記憶されるための閾値ｂを０．３とする。閾値ａは警報原因が写っていることが９０〜１００％の確度に相当し、ほぼ間違いなく警報原因が写っている確度である。閾値ｂは警報原因が写っていることが５０〜９０％の確度に相当し、警報原因が写っていることは概ね当たるが完全ではない場合もある確度である。

Table 1 schematically shows information stored in the alarm image DB 391. In Table 1, the accuracy of each alarm cause is associated with the image number of the alarm image 6. The alarm causes are, for example, insects, birds, small animals, and people. Table 1 (a) shows an alarm image DB 391 that stores an alarm image 6 in which an alarm cause having an accuracy of a threshold value a or higher is shown. Table 1 (b) shows an alarm image DB 391 that stores an alarm image 6 in which an alarm cause with an accuracy greater than or equal to a threshold a is shown, and an alarm image 6 in which an alarm cause with an accuracy greater than or equal to the threshold b is shown. As an example, when the monitor 7 inputs a threshold value a for automatically storing the alarm image 6 in the learning DB 392 without inputting an alarm cause, and the monitor 7 inputs an alarm cause (for example, a small animal) The threshold value b for storing the alarm image 6 in the learning DB 392 is 0.3. The threshold value a corresponds to 90% to 100% accuracy that the cause of the alarm is shown, and is almost certainly the probability that the cause of the alarm is shown. The threshold value b corresponds to the accuracy of 50 to 90% that the cause of the alarm is shown, and the probability that the cause of the alarm is shown is almost correct but may not be perfect.

表２は、学習ＤＢ３９２に記憶されている情報を模式的に示す。学習ＤＢ３９２には画像番号に対応付けて、警報原因とその確度が登録されている。表２（ａ）は表１（ａ）に対応し、表２（ｂ）（ｃ）は表１（ｂ）に対応する。表２（ａ）は、確度が閾値ａ以上の警報原因が写っている警報画像が格納された学習ＤＢ３９２を示す。表１（ａ）の警報画像ＤＢ３９１で確度が０．７以上なのは小動物という警報原因である。小動物という警報原因に着目すると閾値ｂ以上の確度を有するのは画像番号が１〜３の警報画像６なので、画像番号が１〜３の警報画像６が学習ＤＢ３９２に記憶されている。

Table 2 schematically shows information stored in the learning DB 392. In the learning DB 392, an alarm cause and its accuracy are registered in association with the image number. Table 2 (a) corresponds to Table 1 (a), and Tables 2 (b) and (c) correspond to Table 1 (b). Table 2 (a) shows the learning DB 392 in which an alarm image in which an alarm cause having an accuracy of the threshold value a or higher is shown is stored. In the alarm image DB 391 of Table 1 (a), the accuracy of 0.7 or more is a cause of alarm of a small animal. Focusing on the alarm cause of the small animal, the alarm image 6 having the image numbers 1 to 3 has the accuracy equal to or higher than the threshold value b. Therefore, the alarm image 6 having the image numbers 1 to 3 is stored in the learning DB 392.

  Table 2 (b) shows the learning DB 392 in which the monitor 7 inputs a small animal as the cause of the alarm, and stores the alarm image 6 in which the cause of the alarm whose accuracy is less than the threshold a and b or more is shown. In Table 1 (b), since the alarm cause with the threshold value of 0.7 or more is not detected, the supervisor 7 inputs the alarm cause. If the input alarm cause is a small animal, the alarm image DB 391 in Table 1 (b) has an accuracy of 0.3 or more (less than 0.7) and the alarm cause is a small animal because the alarm image 6 has image numbers 1 to 3. The alarm images 6 having image numbers 1 to 3 are stored in the learning DB 392 in association with the alarm cause “small animal”.

  Table 2 (c) shows the learning DB 392 in which the supervisor 7 inputs a person as the cause of the alarm but the alarm image 6 is not stored. In Table 1 (b), since the alarm cause with the threshold value of 0.7 or more is not detected, the supervisor 7 inputs the alarm cause. When the input alarm cause is a person, the alarm image DB 391 in Table 1 (b) has an accuracy of 0.3 or more (less than 0.7) and the alarm cause 6 is not a person's alarm image. For this reason, the warning image 6 is not stored in the learning DB 392. Thereby, it is possible to suppress the extraction of the alarm image 6 that is not suitable for learning even if the supervisor 7 makes an input error.

<A circumscribed rectangle drawn by the camera device>
A circumscribed rectangle that is drawn on the alarm image 6 when the camera apparatus 10 detects a moving object will be described with reference to FIG. FIG. 6 shows an example of a circumscribed rectangle 301 drawn on the alarm image 6 by the camera device 10. In this alarm image 6, a person is detected as a moving object. A circumscribed rectangle 301 is drawn so as to surround the moving object. The monitor can determine the cause of the alarm early by paying attention to the circumscribed rectangle 301.

  Further, the cause of alarm by the information processing apparatus 30 can be performed only inside the circumscribed rectangle 301 by using the circumscribed rectangle 301. Thereby, the time required for identification can be shortened and the load can be reduced.

  However, since the circumscribed rectangle 301 does not always correctly surround the cause of the alarm, it is also effective for the information processing apparatus 30 to identify the entire alarm image 6. The circumscribed rectangle 301 is effective for the supervisor 7 to easily understand the location of the alarm cause, but the recognition by the image processing may hinder the improvement of the recognition rate. For this reason, when the information processing apparatus 30 identifies the entire warning image 6, it is preferable to delete the circumscribed rectangle 301 from the warning image 6. When the circumscribed rectangle 301 is drawn directly on the alarm image 6, the circumscribed rectangle 301 is drawn on a different layer from the alarm image 6 because deletion of the circumscribed rectangle 301 causes a part of the image to disappear. Is preferred.

<Accuracy>
The learning and accuracy of the alarm image 6 will be described with reference to FIG. FIG. 7 shows a model called a multilayer perceptron in a neural network which is an example of a learning model.

  FIG. 7A illustrates the learning phase. Each region image of the alarm image 6 divided into a mesh shape is input to the input layer 501, a feature amount is calculated between the intermediate layer 502 and the output layer 503, and the accuracy of the detection target is output from the output layer 503. Is done. In FIG. 7, the number of nodes in the input layer 501 is the number of area images, and the number of nodes in the output layer 503 is the number of types of detection targets. In FIG. 7, the number of intermediate layers 502 is 1. However, the number of intermediate layers 502 is appropriately designed, and may be 10 to 20 or more in a learning method called deep learning.

  Each region image input to the input layer 501 is feature-extracted by an image filter, and the region images are combined in the later stages. As a result, the global features of the alarm image 6 are extracted as the input layer 501 approaches the output layer 503.

  In the learning phase, “1” is set as a teacher signal in each node of the output layer 503 when the detection target corresponding to the node is captured, and “0” is set otherwise. For example, in FIG. 7, the detection target of the node y1 is an insect, the detection target of the node y2 is a bird, and the detection target of the node y3 is a person. Since the alarm cause of the input alarm image 6 is “insect”, “1” is set to the node y1, and “0” is set to the nodes y2 and y3.

  The difference between the node value of the output layer 503 and the teacher signal is propagated to the image filter based on the error back propagation method. Thereby, deep learning has a feature that an image filter suitable for extracting a feature of an alarm image is automatically optimized. Therefore, deep learning can automatically extract features.

  FIG. 7B illustrates the recognition phase. The neural network in FIG. 7B outputs the accuracy of the detection target shown in the alarm image 6. When learning is completed to some extent, when an alarm image showing an alarm cause is input from the input layer 501, each node of the output layer 503 outputs an accuracy close to 1 from the node corresponding to the alarm cause. The accuracy close to 0 is output from the node. For example, in FIG. 7B, since an insect is reflected as an alarm cause in the input alarm image, the node y1 outputs 0.8, the node y2 outputs 0.2, and the node y3 is 0.1. Is output. In this way, the accuracy of each detection target is obtained from one alarm image 6.

  Note that the learning method or the accuracy calculation method shown in FIG. 7 is merely an example, and feature extraction is performed using deep learning in the previous stage, and the cause of the alarm is identified using SVM (support vector machine) in the subsequent stage. Also good. Further, the cause of the alarm may be identified by inputting a feature amount extracted by SIFT (Scale-Invariant Feature Transform), GIST, or the like into the SVM.

  In addition to the accuracy estimated by the alarm cause estimation unit 34, information equivalent to the accuracy may be calculated by a method of calculating the probability of the alarm cause. However, by using the accuracy estimated by the alarm cause estimation unit 34, a step of calculating information equivalent to the accuracy can be omitted.

<User interface of cause of alarm input by supervisor 7>
Next, the alarm cause input by the supervisor 7 will be described with reference to FIG. FIG. 8 is an example of the alarm cause input screen 401 displayed on the LCD 206. The alarm cause input screen 401 is displayed on the LCD 206 of the information processing apparatus 30 automatically or upon operation of the monitor 7 when the alarm image 6 is received from the camera apparatus 10. The alarm cause input screen 401 includes an alarm image field 411, a return button 412, a send button 413, an image number display field 414, a plurality of alarm cause buttons 415, and an OK button 416.

  The alarm image column 411 is a column in which the alarm image 6 is displayed. A plurality of alarm images 6 before and after the detection target is detected are displayed in time series. The return button 412 is a button for the monitor 7 to display the previous alarm image 6, and the send button 413 is a button for the monitor 7 to display the next alarm image 6. The image number display column 414 indicates how many of the alarm images 6 currently displayed in the alarm image column 411 are. The alarm cause button 415 has a notation of insect, bird, small animal, and person, respectively, and the supervisor 7 looks at the alarm image column 411 to determine the cause of the alarm and presses the alarm cause button 415. The OK button 416 is a button for confirming the input result of the monitor 7, and the extraction process of the alarm image 6 extracted to the learning DB 392 is started when the OK button 416 is pressed.

  The monitoring staff 7 may input one alarm cause for the plurality of alarm images 6. By doing so, the burden on the supervisor 7 to determine the cause of the alarm can be reduced.

  Although it is not necessary to display the alarm image 6 of the threshold value a or higher on the alarm cause input screen 401, the supervisor 7 may confirm it just in case. For this reason, it is preferable that whether the alarm image 6 with the accuracy of the threshold value a or higher is displayed on the alarm cause input screen 401 can be switched by the setting of the monitor 7. When the alarm image 6 with the accuracy of the threshold value a or higher is displayed, the display control unit 32 displays that fact (for example, this image is determined to be an insect and stored in the learning DB). This allows you to verify your judgment. Alternatively, if the cause of alarm is not correct, it can be deleted from the learning DB.

<Alarm image extracted in the learning DB 392>
FIG. 9 is an example of a diagram illustrating a method for extracting the alarm image 6 used for learning. As described above, the monitor 7 inputs one alarm cause for the plurality of alarm images 6. The learning image automatic extraction unit 35 extracts the alarm image 6 having the accuracy of the alarm cause input by the monitor 7 from each alarm image 6 and having a threshold value b or more. For example, the case where the input alarm cause is an insect will be described as an example.

  The accuracy of the insect in the alarm image 61 is less than the threshold value b, the accuracy of the insect in the alarm image 62 is greater than or equal to the threshold value b, and the accuracy of the insect in the alarm image 63 is less than the threshold value b. In this case, the learning image automatic extraction unit 35 extracts only the alarm image 62 and stores it in the learning DB 392.

  When the monitor 7 inputs the correct cause of the alarm shown in the alarm image 6, only the alarm image 6 having a high accuracy of the alarm cause is extracted and used for learning. Therefore, the learning image automatic extraction unit 35 has a plurality of alarms. The alarm image 6 in which the cause of the alarm is shown in the image 6 can be extracted. Therefore, an appropriate alarm image 6 can be extracted in order to learn the relationship between the alarm image 6 and the alarm cause.

  If the monitor 7 inputs an incorrect alarm cause that is not shown in the alarm image 6, there is no alarm image 6 with high accuracy of the alarm cause, so the alarm image 6 associated with the incorrect alarm cause is Not extracted. Therefore, it is possible to suppress the information processing device 30 from learning an incorrect relationship between the alarm image 6 and the alarm cause. In other words, it is possible to cope with an erroneous input of the supervisor 7.

  Specifically, when the monitor 7 inputs an alarm cause different from the alarm cause shown in the alarm image 6, the following processing is performed. That is, if the accuracy of the alarm cause shown in the alarm image 6 is less than the threshold value a and the supervisor 7 inputs an alarm cause different from the alarm cause shown in the alarm image 6, the alarm cause 6 is shown in the alarm image 6. Even if the accuracy of the alarm cause is greater than or equal to the threshold value b, the accuracy of the alarm cause input by the monitor 7 is considered to be less than the threshold value b, and thus is not stored in the learning DB 392. Therefore, there is a possibility that the number of alarm images stored in the learning DB 392 may be reduced, but it is possible to suppress the information processing apparatus 30 from learning an incorrect relationship between the alarm image 6 and the alarm cause.

  The alarm image captured by the camera device 10 has a feature that a similar image is unlikely to exist because of a low frame rate, and there are many images in which the cause of the alarm is not captured because the image is several seconds before and after the alarm. For this reason, the accuracy improvement by learning can be expected by eliminating the alarm image in which the cause of the alarm is not shown.

<< Eliminating alarm images with unstable accuracy >>
Thus, by storing the alarm image with high accuracy of the alarm cause in the learning DB 392, the information processing apparatus 30 can learn the relationship between the alarm image 6 and the alarm cause. However, there may be an alarm image 6 that should be not included in the learning DB 392 even if the accuracy is high.

  FIG. 10 shows, in a time series, the accuracy of the alarm cause input by the monitor 7 out of the accuracy of each alarm cause calculated from the plurality of alarm images 6. The accuracy of this series of alarm images 6 may exceed the threshold value a or b, but the accuracy fluctuates extremely greatly. It is unlikely that the accuracy of the same alarm cause fluctuates extremely greatly, and there is a high possibility that the alarm cause estimation unit 34 has not correctly calculated the accuracy. Since such an alarm image 6 should not be used for learning, it is preferable that the learning image automatic extraction unit 35 does not move to the learning image DB.

  Specifically, the variance of the accuracy is calculated for each alarm cause of each alarm image 6, and the alarm image 6 having a variance exceeding the threshold is not moved to the learning DB 392. This makes it easier to improve the recognition rate.

<Operation procedure>
FIG. 11 is an example of a flowchart illustrating a process in which the information processing apparatus 30 extracts the alarm image 6 and stores it in the learning DB 392.

  First, the communication unit 31 of the information processing device 30 acquires a plurality of alarm images 6 transmitted from the camera device 10 when a detection target is detected (S10). The plurality of warning images 6 are stored in the warning image DB 391.

  Next, the alarm cause estimating unit 34 of the information processing device 30 performs image recognition on each alarm image 6 and calculates the accuracy of each alarm cause (S20). Thereby, as shown in Table 1, the accuracy of each alarm cause is registered for each alarm image 6 in the alarm image DB 391.

  Next, the learning image automatic extraction unit 35 of the information processing device 30 determines whether or not there is an alarm image 6 having an accuracy of the threshold value a or more (S30). This accuracy may be the accuracy of any alarm cause. This is because when the accuracy is high regardless of the cause of the alarm, it may be determined that the cause of the alarm is detected.

  If the determination in step S30 is Yes, there is a high possibility that the alarm cause is shown in the alarm image 6, so the learning image automatic extraction unit 35 pays attention to the alarm cause whose accuracy is determined to be greater than or equal to the threshold value a. The alarm image 6 whose accuracy of the alarm cause is equal to or greater than the threshold value b is extracted and stored in the learning DB 392 (S40). For example, the alarm image 6 in Table 1 (a) having an accuracy of the threshold value a (0.7) or more is a cause of alarm of a small animal. Therefore, the alarm image 6 in which the accuracy of the small animal is greater than or equal to the threshold value b (0.3) is extracted. The learning DB 392 in Table 2 (a) is created for the alarm image 6 in Table 1 (a), and the processing in FIG. 11 ends. Even if the alarm cause having the accuracy of the threshold a or higher is not confirmed by the supervisor 7, it is almost certain that the alarm cause is reflected in the alarm image 6. Therefore, the alarm image 6 having the same alarm cause accuracy of the threshold b or higher This is because it may be determined that the cause of the same alarm is shown. Thereby, more alarm images 6 can be automatically extracted.

  If the determination in step S30 is No, since there is no alarm image 6 with the alarm cause accuracy equal to or greater than the threshold value a, the operation accepting unit 33 accepts the alarm cause input of the monitor 7 (S50). That is, the display control unit 32 displays an alarm cause input screen 401 and the monitor 7 inputs one alarm cause for the plurality of alarm images 6.

  The learning image automatic extraction unit 35 of the information processing device 30 extracts the alarm image 6 having the alarm cause accuracy input by the monitor 7 and having a threshold value b or more (S60). By doing so, only the alarm image 6 suitable for learning can be extracted.

  The learning image automatic extraction unit 35 of the information processing device 30 stores the extracted alarm image 6 in the learning DB 392 (S70). Thus, only the alarm image 6 appropriate for learning is stored in the learning DB 392. When the supervisor 7 inputs the cause of alarm “small animal” to the alarm image 6 in Table 1 (b), the learning DB 392 in Table 2 (b) is created. When the supervisor 7 inputs an alarm cause “person” to the alarm image 6 in Table 1 (b), the learning DB 392 in Table 2 (c) is created.

  As described above, the image processing system 100 according to the present embodiment can automatically extract an alarm image in which the cause of an alarm is captured by one identification device without separately preparing a high-performance identification device.

  In this embodiment, the update of the identification device obtained by learning using the alarm image 6 stored in the learning DB 392 will be described.

  FIG. 12 is an example of a diagram illustrating an outline of evaluation of the identification device by the information processing device 30. First, the learning DB 392 stores the alarm image 6 extracted in the first embodiment. In addition, the configuration DB 393 stores the alarm image 6 used for constructing the alarm cause estimation unit 34 that is the current identification device. The current alarm cause estimation unit 34 is referred to as an alarm cause estimation unit (current) 34b, and the new alarm cause estimation unit 34 is referred to as an alarm cause estimation unit (new) 34a.

  (1) First, the learning unit 36 learns using both alarm images 6 of the configuration DB 393 and the learning DB 392. A new alarm cause estimator (new) 34a can be constructed by learning.

  (2) The evaluation unit 37 evaluates the performance of the alarm cause estimation unit (new) 34a using the evaluation image in the evaluation image DB 394.

  (3) When the alarm cause estimator (new) 34a recognizes the evaluation image that the alarm cause estimator (current) 34b can recognize, the alarm cause estimator (current) 34b becomes the alarm cause estimator (new). 34a is replaced.

  (4) When replaced, the warning image 6 in the learning DB 392 is moved to the configuration DB 393, and when not replaced, the warning image 6 in the learning DB 392 is deleted. Therefore, in any case, the alarm image 6 does not remain in the learning DB 392.

  The possibility that the recognition rate decreases even if learning is performed with the alarm image 6 extracted as in the first embodiment is not zero. However, in the update method of this embodiment, the alarm cause estimation unit (current) 34b can recognize Since it is guaranteed that the image can be recognized by the alarm cause estimating unit (new) 34a, the alarm cause estimating unit 34 can be updated only when the recognition rate is improved.

<Functional configuration example of image processing system>
FIG. 13 is an example of a functional block diagram illustrating each function provided in the image processing system 100 of the present embodiment. In FIG. 13, functions related to the update of the alarm cause estimating unit 34 will be described. In the description of FIG. 13, the components denoted by the same reference numerals in FIG. 5 perform the same functions, and therefore, only the main components of the present embodiment will be mainly described.

  First, the function of the camera device 10 is not changed from FIG. In contrast, the information processing apparatus 30 includes a learning unit 36, an alarm cause estimation unit (new) 34a, an evaluation unit 37, an alarm cause estimation unit (current) 34b, and an update unit 38. In the storage unit 39, a learning DB 392, a configuration DB 393, and an evaluation image DB 394 are built. The configuration DB 393 stores the alarm image 6 used to construct the alarm cause estimation unit (current) 34b. The evaluation image DB 394 stores an evaluation image prepared for evaluating the performance of the alarm cause estimating unit 34 and an alarm cause verified to be reflected in the evaluation image in association with each other. Note that the evaluation images are obtained by classifying the actual alarm image 6 for each scene (the images are classified based on what kind of images are captured at the monitoring place 8), and the occurrence frequency is high. Various alarm images 6 are prepared for higher scenes. Therefore, it is easy to evaluate the recognition rate of the actual alarm image 6.

  The learning unit 36 obtains the alarm image 6 of the learning DB 392 and the configuration DB 393, and calculates a probability that the detection target is reflected in the alarm image 6 by a machine learning technique such as neural network, SVM, or deep learning. Learn (sometimes called a learning model). The teacher signal is the alarm cause estimated by the alarm cause estimation unit (current) 34b in the alarm image 6 of the threshold value a or higher, and the alarm cause input by the monitor 7 in the alarm image 6 of the threshold value b or higher.

  The alarm cause estimation unit (new) 34a is an identification device generated by the learning unit 36 learning from the learning DB 392 and the alarm image 6 of the configuration DB 393. The alarm cause estimation unit (new) 34a calculates the accuracy of each alarm cause with respect to the input of the alarm image 6.

  The alarm cause estimation unit (current) 34 b is the current alarm cause estimation unit 34. The alarm cause estimating unit (current) 34b corresponds to the alarm cause estimating unit 34 of FIG.

  The evaluation unit 37 evaluates the performance of the alarm cause estimation unit (new) 34a, and determines whether or not the performance of the alarm cause estimation unit (new) 34a is improved as compared with the alarm cause estimation unit (current) 34b. For example, each of the alarm cause estimation unit (new) 34a and the alarm cause estimation unit (current) 34b performs processing for identifying the alarm cause from the evaluation image in the evaluation image DB 394.

  The update unit 38 updates the alarm cause estimation unit (current) 34b with the alarm cause estimation unit (new) 34a.

<Determination of identification device update>
There are two methods for determining whether or not the alarm cause estimation unit (current) 34b is updated by the alarm cause estimation unit (new) 34a. Each will be explained.
A. Judgment based on whether or not correctly identified The performance evaluation of the alarm cause estimating unit 34 will be described with reference to FIG. FIG. 14 shows the identification results when the alarm cause estimation unit (current) 34b and the alarm cause estimation unit (new) 34a identify the evaluation images. In FIG. 14, “◯” indicates that it can be correctly identified, and “x” indicates that it cannot be correctly identified. Being able to identify correctly means that the cause of the alarm attached to the evaluation image can be detected with a threshold value of 1 or more. Further, “not correctly identified” means that the cause of the alarm attached to the evaluation image is detected with a threshold value of less than 1. The threshold value 1 corresponds to the threshold value a, for example.

  FIG. 14A shows an evaluation result when the alarm cause estimating unit 34 is updated. The evaluation image 1 (image number 1) is correctly identified by both the alarm cause estimation unit (current) 34b and the alarm cause estimation unit (new) 34a. The evaluation image 2 (image number 2) cannot be correctly identified by the alarm cause estimating unit (current) 34b, but is correctly identified by the alarm cause estimating unit (new) 34a. The evaluation image 3 (image number 3) cannot be correctly identified by either the alarm cause estimation unit (current) 34b or the alarm cause estimation unit (new) 34a. In this way, in FIG. 14A, since the alarm cause estimation unit (new) 34a can also correctly identify the evaluation image that the alarm cause estimation unit (current) 34b can correctly identify, the evaluation unit 37 has improved performance. to decide.

  The performance of the present embodiment mainly refers to the recognition rate. This recognition rate includes being able to identify the cause of the alarm shown in the alarm image 6 and not identifying the cause of the alarm not shown in the alarm image 6.

  FIG. 14B shows an evaluation result when the alarm cause estimating unit 34 is not updated. The evaluation image 1 (image number 1) can be correctly identified by the alarm cause estimating unit (current) 34b, but cannot be correctly identified by the alarm cause estimating unit (new) 34a. On the other hand, the evaluation image 2 (image number 2) and the evaluation image 3 (image number 3) cannot be correctly identified by the alarm cause estimation unit (current) 34b, but can be correctly identified by the alarm cause estimation unit (new) 34a. As described above, if even one alarm cause estimation unit (new) 34a cannot be correctly identified by the alarm cause estimation unit (current) 34b, the evaluation unit 37 does not determine that the performance has improved. By doing so, it can be guaranteed that the performance of the alarm cause estimating unit 34 has been improved.

  However, when the recognition rate of the entire evaluation image in the evaluation image DB 394 is improved, it may be determined that the performance is improved.

B. Judgment based on whether or not the accuracy has been improved Evaluation of the performance of the alarm cause estimating unit 34 based on the accuracy will be described with reference to FIG. FIG. 15 shows the accuracy when the alarm cause estimation unit (current) 34b and the alarm cause estimation unit (new) 34a each identify an image for evaluation. This accuracy is the accuracy of the alarm cause attached to the evaluation image. Therefore, it is desirable that the alarm cause estimation unit (new) 34a is higher than the alarm cause estimation unit (current) 34b.

  FIG. 15A shows an evaluation result when the alarm cause estimating unit 34 is updated. In all of the evaluation images 1 to 3 (image numbers 1 to 3), the accuracy of the alarm cause estimation unit (new) 34a is larger than that of the alarm cause estimation unit (current) 34b. As described above, in FIG. 15A, the alarm cause estimation unit (new) 34a is higher than the alarm cause estimation unit (current) 34b in the accuracy of the alarm cause of all the evaluation images. Is judged to have improved.

  FIG. 15B shows an evaluation result when the alarm cause estimating unit 34 is not updated. On the other hand, in FIG. 15B, the accuracy of the alarm cause estimation unit (new) 34a is larger in the evaluation images 2 and 3 (image numbers 2 and 3) than the alarm cause estimation unit (current) 34b. However, in the evaluation image 1 (image number 1), the accuracy of the alarm cause estimation unit (new) 34a is lower than that of the alarm cause estimation unit (current) 34b.

  If even one evaluation image is found in which the accuracy of the alarm cause estimation unit (new) 34a is lower than the accuracy of the alarm cause estimation unit (current) 34b, the evaluation unit 37 does not determine that the performance has improved. By doing so, it can be guaranteed that the performance of the alarm cause estimating unit 34 has been improved.

  However, if the average accuracy of the alarm cause estimator (new) 34a is compared with the average accuracy of the alarm cause estimator (new) 34a, the average accuracy of the alarm cause estimator (new) 34a is higher. May be determined to have improved.

<Operation procedure>
FIG. 16 is an example of a flowchart illustrating a procedure for determining whether or not the information processing apparatus 30 evaluates and updates the performance of the alarm cause estimating unit 34. The process of FIG. 16 may be executed periodically, or may be executed when a certain amount of alarm image 6 is stored in the learning DB 392.

  First, the learning unit 36 learns using the alarm image 6 of the learning DB 392 and the configuration DB 393, thereby constructing an alarm cause estimation unit (new) 34a (S110).

  Next, the evaluation unit 37 identifies the image for evaluation using the alarm cause estimation unit (new) 34a (S120). Although the image for evaluation may be identified using the alarm cause estimating unit (current) 34b, the identification result of the alarm cause estimating unit (current) 34b may be the same as the previous evaluation.

  The evaluation unit 37 determines whether or not the performance of the alarm cause estimation unit (new) 34a is improved as compared with the alarm cause estimation unit (current) 34b as described with reference to FIGS. 14 and 15 (S130).

  When the determination in step S130 is Yes, the update unit 38 updates the alarm cause estimation unit (current) 34b with the alarm cause estimation unit (new) 34a (S140). The alarm cause estimation unit (current) 34b may be discarded or saved.

  And the update part 38 moves the warning image 6 of learning DB392 to composition DB393 (S150). Thus, the alarm image 6 that brings about the performance improvement can be stored in the configuration DB 393.

  When the determination in step S130 is No, the update unit 38 deletes the alarm image 6 in the learning DB 392 (S160). Therefore, it is possible to prevent the alarm image 6 that does not cause performance improvement from remaining in the learning DB 392.

  As described above, the image processing system 100 according to the present embodiment can improve the performance of the alarm cause estimation unit (identification device) using the alarm image 6 stored in the learning DB 392. Since the performance of the alarm cause estimation unit (identification device) is evaluated, it can be improved while suppressing a decrease in recognition performance that may occur due to learning.

<Other application examples>
The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  In the present embodiment, the cause of the alarm is determined by identifying the image. However, the information processing apparatus 30 can also determine the cause of the alarm by identifying the sound. In this case, the camera device 10 has a microphone or the like that collects sound. The sound data emitted from the detection target is transmitted to the information processing apparatus 30 together with the alarm image. The information processing apparatus can identify what kind of sound the sound data is (sound cause). For example, key operations, door opening / closing, human voice, animal voice, glass breaking sound, etc. are identified. In order to improve the sound recognition rate, the sound data having the accuracy of the threshold value a or higher is similarly stored in the learning DB 392. The supervisor listens to the other sound data, determines the content of the sound, and inputs the cause of the sound. The input sound data having the sound cause accuracy of the threshold value b or more is stored in the learning DB 392. Therefore, the sound data is also identified.

In the system configuration diagram of FIG. 2, the camera device 10 and the information processing device 30 are separate bodies, but the camera device 10 and the information processing device 30 may be integrated. In this case, the camera device 10 can identify the image alone and output the cause of the alarm. Moreover, the camera device 10 can improve the performance by itself. In this case, the camera device 10 operates as follows.
(1) When the cause of the alarm cannot be identified, an alarm image is transmitted to the monitoring center 9.
(2) The supervisor 7 inputs the cause of the alarm. The alarm image and the alarm cause are stored in the learning DB 392 in the same procedure as described above.
(3) The alarm cause estimation unit (new) whose performance is improved as a result of learning is transmitted to the camera device 10, and the camera device 10 updates the internal alarm cause estimation unit (current).

  In the system configuration diagram of FIG. 2, the monitoring center 9 may be a client server configuration. The supervisor 7 communicates with the information processing apparatus 30 by operating a terminal such as a PC and inputs an alarm cause. In this case, the information processing apparatus 30 may be on the cloud instead of the monitoring center 9.

  5 and 13 are divided according to main functions in order to facilitate understanding of processing by the camera apparatus 10 and the information processing apparatus 30. The present invention is not limited by the way of dividing the processing unit or the name. The processing of the camera device 10 and the information processing device 30 can be divided into more processing units according to the processing content. Moreover, it can also divide | segment so that one process unit may contain many processes.

  5 and 13, there is one information processing device 30, but there may be a plurality of the same information processing devices 30, and the functions of FIGS. 5 and 13 are distributed to the plurality of information processing devices 30. It may be.

  The communication unit 31 is an example of an image acquisition unit, the alarm cause estimation unit (current) 34b is an example of a first identification unit, the display control unit 32 is an example of a display processing unit, and the operation receiving unit 33 Is an example of a reception unit, the learning image automatic extraction unit 35 is an example of an image extraction unit, the learning unit 36 is an example of an identification unit construction unit, and the alarm cause estimation unit (new) 34a is a second identification unit. The evaluation unit 37 is an example of an evaluation unit, and the update unit 38 is an example of an update unit. The learning DB 392 is an example of an image storage unit.

6: Alarm image 7: Monitor 10: Camera device 30: Information processing device 31: Communication unit 32: Display control unit 33: Operation reception unit 34: Alarm cause estimation unit 35: Automatic learning image extraction unit 36: Learning unit 37: Evaluation unit 38: Update unit 100: Image processing system

Claims (9)

An image processing system for identifying a predetermined alarm cause from an image captured by an imaging device,
Image acquisition means for acquiring the image from the imaging device;
First identifying means for identifying the cause of the alarm in the image and calculating the accuracy with which the cause of the alarm is identified;
Display processing means for displaying the image on a display device;
Receiving means for receiving an input of the cause of the alarm shown in the image displayed on the display device;
An image extracting means for storing the image in an image storage unit, when the probability that the alarm cause received by the receiving means and calculated by the first identifying means is greater than or equal to a first threshold;
Identifying means constructing means for constructing a second identifying means using the image stored in the image storage unit;
An image processing system. The image acquisition means acquires a plurality of the images including before and after the time when an abnormality is detected from the imaging device,
The accepting means accepts one alarm cause input for the plurality of images,
2. The image processing according to claim 1, wherein the image extraction unit causes the image storage unit to store all the images having a probability of the alarm cause received by the reception unit that is greater than or equal to the first threshold among the plurality of images. system. The image extraction means, when the image of the plurality of images identified by the first identification means includes the image having a second threshold value greater than the first threshold value, the accuracy of the alarm cause,
The image having the accuracy of the alarm threshold having the accuracy equal to or higher than the second threshold is stored in the image storage unit in association with the alarm cause having the accuracy of the second threshold or higher. The image processing system according to claim 1 or 2.   When the plurality of images have the image with extremely large fluctuations in accuracy, the image extracting means stores the image with extremely large fluctuations in accuracy even if the accuracy is not less than the first threshold. The image processing system according to claim 1, wherein the image processing system is not stored in the unit. Evaluation means for evaluating the performance of the second identification means and the first identification means;
When the evaluation unit evaluates that the performance of the second identification unit is higher than that of the first identification unit, the update unit updates the first identification unit with the second identification unit;
The image processing system according to claim 1, comprising: The evaluation means includes
When all of the images that can be identified by the first identifying means can be identified by the second identifying means, it is evaluated that the performance of the second identifying means is improved compared to the first identifying means. The image processing system according to claim 5. The evaluation means includes
When the accuracy of identifying the image by the second identification unit is higher in all the images than the accuracy of identifying the image by the first identification unit, the performance of the second identification unit The image processing system according to claim 5, wherein the image processing device is evaluated as being improved over the first identification unit. An information processing apparatus for identifying a predetermined alarm cause from an image captured by an imaging apparatus,
Image acquisition means for acquiring the image from the imaging device;
First identifying means for identifying the cause of the alarm in the image and calculating the accuracy with which the cause of the alarm is identified;
Display processing means for displaying the image on a display device;
Receiving means for receiving an input of the cause of the alarm shown in the image displayed on the display device;
An image extracting means for storing the image in an image storage unit, when the probability that the alarm cause received by the receiving means and calculated by the first identifying means is greater than or equal to a first threshold;
An information processing apparatus comprising: an identification unit construction unit that constructs a second identification unit using the image stored in the image storage unit. An information processing apparatus for identifying a predetermined alarm cause from an image captured by the imaging apparatus,
Image acquisition means for acquiring the image from the imaging device;
First identifying means for identifying the cause of the alarm in the image and calculating the accuracy with which the cause of the alarm is identified;
Display processing means for displaying the image on a display device;
Receiving means for receiving an input of the cause of the alarm shown in the image displayed on the display device;
An image extracting means for storing the image in an image storage unit, when the probability that the alarm cause received by the receiving means and calculated by the first identifying means is greater than or equal to a first threshold;
A program for functioning as identification means construction means for constructing second identification means using the image stored in the image storage unit.

Images