JP7002009B2 - Monitoring parameter update system, monitoring parameter update method and program - Google Patents

Description

The present disclosure relates to a monitoring parameter update system, a monitoring parameter update method and a program.

In Patent Document 1, image data captured by each of a plurality of cameras and a slide bar in which the parameter values of the cameras are variable are displayed on a GUI (Graphical User Interface), and each image data has the same appearance. If not, the technique of changing the parameter value by the operation of the slide bar of the user for the camera selected by the operation of the user is disclosed.

Japanese Unexamined Patent Publication No. 2008-17476

However, in the configuration of Patent Document 1, whether or not to change the parameter value depends on the criterion of whether or not each image data has the same appearance. Therefore, when a predetermined object (for example, a person) appearing in the image data is erroneously recognized when the parameter value is set, the current parameter value cannot be changed. Therefore, even if a predetermined object, which is a monitoring target, appears in the image data, the appearance may be overlooked, and the monitoring accuracy may be insufficient.

The present disclosure has been devised in view of the above-mentioned conventional situation, and when a specific object that is a monitoring target appearing in a captured image is erroneously recognized, the monitoring parameters related to the recognition of the captured image are updated to recognize the specific object. It is an object of the present invention to provide a monitoring parameter update system, a monitoring parameter update method, and a program for improving accuracy and suppressing deterioration of monitoring accuracy.

The present disclosure is a monitoring parameter update system in which a first processing device connected to a camera installed so as to be able to image a monitoring area and a second processing device are connected so as to be communicable, and the first processing device is the first processing device. Based on the captured image of the monitoring area captured by the camera, the specific object appearing in the captured image is recognized, and the recognition result and the captured image are associated with each other and transmitted to the second processing device. When the second processing device recognizes an object that is not the specific object based on the recognition result and the captured image sent from the first processing device, the first processing device issues a monitoring parameter update instruction including a monitoring parameter. Provide a monitoring parameter update system to be sent to the processing device .

Further, the present disclosure is a monitoring parameter updating method using a monitoring parameter updating system in which a first processing device connected to a camera installed so as to be able to image a monitoring area and a monitoring parameter updating device connected to a second processing device so as to be communicable. Therefore, the first processing device recognizes a specific object appearing in the captured image based on the captured image of the monitoring area, and transmits the recognition result and the captured image to the second processing device in association with each other. Then, when the second processing device recognizes an object that is not the specific object based on the recognition result and the captured image sent from the first processing device, the first processing device issues a monitoring parameter update instruction including a monitoring parameter. Provides a monitoring parameter update method to be sent to the processing device .

Further, the present disclosure describes the captured image of the monitoring area captured by the camera on the first processing device, which is a computer connected to the camera installed so that the monitoring area can be imaged and the second processing device, respectively. Based on this, a step of recognizing a specific object appearing in the captured image, a step of associating the recognition result of the specific object and the captured image and transmitting the captured image to the second processing device, a recognition result of the specific object, and When an object other than the specific object is recognized by the second processing device based on the captured image, a step of receiving a monitoring parameter update instruction including a monitoring parameter sent from the second processing device is performed on a computer. Provides a program to run on.

Further, the present disclosure is captured by the camera on a second processing device, which is a computer, which is communicably connected to a first processing device connected to a camera installed so that the monitoring area can be imaged. When the specific object appearing in the captured image is recognized by the first processing device based on the captured image of the monitoring area, the recognition result and the captured image of the specific object sent from the first processing device. Based on the image, a step of recognizing an object that is not the specific object and a step of transmitting a monitoring parameter update instruction including a monitoring parameter to the first processing device when the object is recognized are executed on a computer . To provide a program for.

According to the present disclosure, when a specific object that is a monitoring object appearing in a captured image is erroneously recognized, the monitoring parameters related to the recognition of the captured image are updated to improve the recognition accuracy of the specific object and deteriorate the monitoring accuracy. It can be suppressed.

The figure which shows the outline example of the surveillance camera system which concerns on Embodiment 1. Block diagram showing a hardware configuration example of a surveillance camera Block diagram showing a hardware configuration example of an image recognition device Block diagram showing a hardware configuration example of a cloud server A sequence diagram showing an example of an operation procedure of the surveillance camera system according to the first embodiment. A diagram showing an example of the registered contents of the parameter table in which the parameters for recognition are registered. The figure which shows an example of the image taken by the surveillance camera which is displayed when the positive news is notified. The figure which shows an example of the image taken by the surveillance camera which is displayed when a false alarm is notified.

(Background to the idea of the embodiment)
For example, the entrance to a highway is a place where people cannot enter for safety reasons. Surveillance cameras are usually installed at the entrance of the highway. An image recognition device for recognizing a subject (object) reflected in an image captured by such a surveillance camera is a user (object) when a subject (object) that looks like a person is reflected in the image captured by the surveillance camera. For example, the system operator) is notified of the event. When the operator confirms that the subject shown in the image captured by the surveillance camera is a person, the operator takes various measures such as issuing an alarm.

The notification that the image recognition device notifies the operator when it is recognized that an object that seems to be a person is reflected is a positive report (so-called a message to inform the correct recognition result that the recognized subject is a person). ), There are many false alarms in which the shadow or sign reflected in the captured image is mistakenly recognized as a person. The operator does not always monitor and watch the images captured by one surveillance camera, but always watches the images captured by a plurality of surveillance cameras. If the operator is notified of the correct report based on the image captured by another surveillance camera while the operator is checking the image captured by the surveillance camera corresponding to the false report, the operator confirms the captured image corresponding to the correct report. Will be delayed. During this time, a dangerous event may occur, such as a person entering from the entrance of a highway going out to the shoulder. As described above, in the conventional technology, when the operator has to confirm a large number of captured images, it is assumed that each event that has occurred cannot be dealt with quickly and the response is delayed. In addition, when the number of false alarms increases, the operator is overwhelmed by checking the captured image, which leads to overlooking the captured image corresponding to the correct report. It is still difficult to solve such a problem concretely with the configuration of Patent Document 1.

Therefore, in the following embodiment 1, by reducing the misrecognition of the subject (in other words, the target) and reducing the number of false alarms notified to the operator, the delay in responding to the event to be dealt with quickly and the delay in responding to the event should be dealt with. An example of a surveillance camera system will be described in which the operator can quickly recognize the target and reduce the burden on the operator by eliminating the oversight of the captured image corresponding to the false alarm.

(Embodiment 1)
Hereinafter, an embodiment in which a surveillance camera system as an example of a surveillance parameter update system is specifically disclosed as a monitoring parameter update system, a monitoring parameter update method, and a program according to the present disclosure will be described in detail with reference to the drawings as appropriate. do. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

FIG. 1 is a diagram showing a schematic example of the surveillance camera system 5 according to the first embodiment. In the surveillance camera system 5, an exclusion zone (for example, the entrance of a highway) where people are prohibited from entering is set as an imaging area (in other words, an imaging area) so that no one can enter the imaging area. Monitor. Specifically, the surveillance camera system 5 has a configuration including a plurality of surveillance cameras 10, a plurality of image recognition devices 30 corresponding to the respective surveillance cameras, and a cloud server 40. The plurality of surveillance cameras 10 are installed at positions where, for example, a plurality of restricted areas can be imaged as surveillance areas. The exclusion zone may include a place where access is restricted irregularly or depending on the time of day, for example, in a railroad crossing where a barrier is dropped or in a park where entry is prohibited at night. Further, the exclusion zone may include, for example, a place where not only humans but also vehicles, dogs, cats and other small animals are prohibited from entering. The captured images captured by the surveillance camera 10 are transmitted to the corresponding image recognition devices 30.

Here, one surveillance camera 10 and one image recognition device 30 are combined as one set and can communicate via the network NW1. In addition, one surveillance camera 10 and one image recognition device 30 may be configured as a device housed in separate housings, or may be housed in the same housing and configured as an integrated device. You may. Further, a plurality of surveillance cameras may be connected to one surveillance recognition device.

Each image recognition device 30 analyzes the captured image captured by each surveillance camera 10 and performs image recognition processing, and performs processing such as person determination in the surveillance area, and is a cloud via the network NW2. It is communicably connected to the server 40. The cloud server 40 performs image recognition processing such as recognition processing of a subject (object) such as a person for each captured image constituting the captured image based on the person determination by the image recognition device 30 and the captured image by the surveillance camera 10. I do. Further, the cloud server 40 updates the recognition parameters (see below) for the corresponding image recognition device 30 or the surveillance camera 10 corresponding to the image recognition device 30 based on the result of the image recognition process. Yes, it is connected to a PC (personal computer) 50 via the network NW3. The PC 50 displays the alarm information transmitted from the cloud server 40 (see the positive report described later) and the captured image when the positive report is generated. The PC 50 issues an alarm to notify the operator when the alarm information acquired from the cloud server 40 contains a false alarm.

FIG. 2 is a block diagram showing a hardware configuration example of the surveillance camera 10. The surveillance camera 10 as an example of the camera is a PTZ (Pan Tilt Zoom) camera capable of pan / tilt / zoom. For example, the surveillance camera 10 takes an image at a predetermined home position (that is, one of the preset positions) so as to include the entire restricted area corresponding to the own camera within the angle of view. When an object such as a person appears in the captured image, the surveillance camera 10 changes the pan angle and tilt angle so that the object becomes larger within the angle of view, and even if the posture is aimed at the object and the zoom magnification is increased. good.

The surveillance camera 10 includes a CPU 11, a storage unit 12, an image pickup unit 13, a video processing unit 14, a communication unit 15, a pan / tilt drive unit 16, an acceleration sensor 18, and an IR irradiation unit 19. be.

The pan-tilt drive unit 16 drives the image pickup unit 13 in the pan direction and the tilt direction, and includes a tilt control unit 21, a pan control unit 22, a tilt motor 23, and a pan motor 24. The tilt control unit 21 controls the rotation angle of the tilt motor 23. The pan control unit 22 controls the rotation angle of the pan motor 24.

The accelerometer 18 outputs a signal corresponding to the posture of the surveillance camera 10 (for example, a pan angle and a tilt angle that determine the optical axis direction of the lens). The CPU 11 controls the posture of the surveillance camera 10 based on the signal output from the acceleration sensor 18. As the acceleration sensor 18, for example, a semiconductor type using a semiconductor piezoresistive effect or a mechanical type that opens and closes a movable contact and a fixed contact by a rotating mass body is used.

The image pickup unit 13 has a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) as an image pickup device. The image pickup device includes, for example, a plurality of pixel groups (for example, 1 million pixels) in which one pixel is formed by a Bayer array of R pixels, G pixels, B pixels, and IR pixels. Further, the image pickup unit 13 includes a focus lens whose focal length can be adjusted and a zoom lens whose zoom magnification can be changed. The image pickup unit 13 includes an amplifier that amplifies the charge signal output from the image pickup element, and a gain adjustment unit that can adjust the gain of the amplifier.

The image processing unit 14 performs various signal processing such as AD conversion, color correction, noise removal, and contour enhancement on the data signal of the image captured image captured by the image pickup unit 13.

The storage unit 12 includes, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory), and is required for programs and data (for example, focal length required for focus adjustment and gain adjustment) necessary for executing the operation of the surveillance camera 10. (Including parameters such as gain value), information or data generated during operation is temporarily saved. The storage unit 12 may store (pan angle, tilt angle, zoom magnification) representing the home position of the surveillance camera 10 as preset values. The RAM is, for example, a work memory used when the CPU 11 is operating. The ROM stores, for example, a program and data for controlling the CPU 11 in advance.

The CPU 11 controls the operation of each part of the surveillance camera 10. The CPU 11 functions as a control unit of the surveillance camera 10, controls processing for overall control of the operation of each unit of the surveillance camera 10, data input / output processing with and from each unit of the surveillance camera 10, and data calculation (data calculation (). Calculation) Processing and data storage processing are performed. The CPU 11 operates according to the execution of the program stored in the ROM in the storage unit 12. The CPU 11 controls the posture of the surveillance camera 10 based on the signal output from the acceleration sensor 18. The CPU 11 transmits the data of the captured image captured by the imaging unit 13 and signal-processed by the image processing unit 14 to the image recognition device 30 via the communication unit 15.

The communication unit 15 is communicably connected to the image recognition device 30 via a wired LAN (Local Area Network). The communication unit 15 may be connected to the PC via a wireless LAN so as to be able to communicate with the PC.

The IR irradiation unit 19 irradiates a subject with near-infrared light (IR (infrared radiation) light) in order to capture a black-and-white image when the surroundings are dark, such as at night. Here, the case where the light reaches a long distance at night and the IR light which is invisible light is irradiated is shown. However, the light is not limited to the IR light which is invisible light, and visible light may be irradiated.

FIG. 3 is a block diagram showing a hardware configuration example of the image recognition device 30. The image recognition device 30 receives the captured image captured by the surveillance camera 10 connected to the own device in real time and performs image recognition processing (for example, determination of a person, detection of the presence or absence of intrusion of a person). Further, when the image recognition device 30 detects the intrusion of a person as a result of the person determination, the image recognition device 30 cloud the result of the person determination (an example of the alarm information) and the captured image in which the person is determined (an example of the alarm image). Send to server 40. Further, the image recognition device 30 receives, for example, once a day from the cloud server 40 the optimum recognition parameters, updates the recognition parameters held by the own device, or recognizes the recognition held by the surveillance camera 10. The surveillance camera 10 is made to update the parameters for.

The image recognition device 30 as an example of the first processing device operates as a video processing device, and has a processor 31, a ROM (Read only memory) 32, a RAM (Random Access memory) 33, a hard disk drive (HDD) 34, and a communication unit. It is a configuration including at least 35.

The processor 31 is configured by using, for example, a CPU (Central Processing unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), and controls the operation of each part of the image recognition device 30. The processor 31 functions as a control unit of the image recognition device 30, and controls processing for overall control of the operation of each part of the image recognition device 30, data input / output processing with each part of the image recognition device 30, and data input / output processing. Performs data calculation processing and data storage processing. The processor 31 operates according to the program and data stored in the ROM 32. The processor 31 performs the above-mentioned image recognition process using the captured image captured by the surveillance camera 10 connected to the image recognition device 30.

The ROM 32 stores, for example, a program and data for controlling the processor 31 in advance. The RAM 33 is a work memory used when the processor 31 operates. The RAM 33 temporarily stores programs and data necessary for executing the operation of the processor 31, as well as information or data generated during the operation.

The HDD 34 is a large-capacity storage medium, and the image recognition determination result 34a of the captured image executed by the processor 31, the captured image data 34b received by the communication unit 35, and the recognition used in the image recognition process in the processor 31. Parameters 34c and the like for storage are stored.

The communication unit 35 is connected to the surveillance camera 10 via the network NW1 so as to be capable of data communication. The communication unit 35 is connected to the cloud server 40 via the network NW2 so as to be capable of data communication. The network NW1 and the network NW2 may be a wired LAN (Local Area Network) or a wireless LAN.

FIG. 4 is a block diagram showing a hardware configuration example of the cloud server 40. The cloud server 40 receives the result of the person determination (an example of the alarm information) and the captured image in which the person is determined (an example of the alarm image) transmitted from the image recognition device 30, and also receives the captured image. The image recognition is re-verified in order to confirm the accuracy of the image recognition process in the image recognition device 30. When the cloud server 40 performs re-verification, it confirms the similarity with the false alarm data (not shown) stored in the hard disk drive (HDD) 44, and if the similarity with the false alarm data is high, it determines that it is a false alarm. , The captured image determined to be a false alarm is not displayed on the PC 50. On the other hand, when the similarity with the false alarm data is low, the cloud server 40 determines that the report is correct, and displays the captured image determined to be the correct report on the PC 50. The cloud server 40 stores the false alarm data including the false alarm determination result and the captured image determined as the false alarm in the hard disk drive (HDD) 44 and treats it as learning data. The cloud server 40 accumulates a large amount of false alarm data. When the accumulated amount of false alarm data increases, the accuracy of confirming the similarity with the false alarm data determined by the cloud server 40 improves. The cloud server 40 updates the recognition parameter each time it determines that it is a false alarm. The cloud server 40 transmits, for example, once a day, the parameters for the optimum recognition at the present time to the image recognition device 30. As shown in FIG. 6, the optimum recognition parameters include, for example, black-and-white / color switching, focus adjustment, gain adjustment, and the like corresponding to illuminance, brightness, background color, color information, and the like.

The cloud server 40 as an example of the second processing device operates as a video processing device, and has a processor 41, a ROM (Read only memory) 42, a RAM (Random Access memory) 43, a hard disk drive (HDD) 44, and a communication unit 45. It is a configuration including.

The processor 41 is configured by using, for example, a CPU (Central Processing unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), and controls the operation of each part of the cloud server 40. The processor 41 functions as a control unit of the cloud server 40, and controls processing for overall control of the operation of each unit of the cloud server 40, data input / output processing with and from each unit of the cloud server 40, and data calculation. Performs (calculation) processing and data storage processing. The processor 41 operates according to the program and data stored in the ROM 42. The processor 41 uses the result of the person determination (an example of the alarm information) and the captured image in which the person is determined (an example of the alarm image) transmitted from the image recognition device 30 to repeat the above-mentioned image recognition. Perform verification.

The ROM 42 stores, for example, a program and data for controlling the processor 41 in advance. The RAM 43 is a work memory used when the processor 41 operates. The RAM 43 temporarily stores programs and data necessary for executing the operation of the processor 41, as well as information or data generated during the operation.

The HDD 44 is a large-capacity storage medium, and stores an image recognition determination result 44a of a captured image, captured image data 44b, a recognition parameter 44c, and the like.

The communication unit 45 is connected to the image recognition device 30 via the network NW2 so as to be capable of data communication. The communication unit 45 is connected to the PC 50 via the network NW3 so as to be capable of data communication. The network NW3 may be a wired LAN (Local Area Network) or a wireless LAN.

Next, an operation example of the surveillance camera system 5 according to the first embodiment will be described.

FIG. 5 is a sequence diagram showing an example of an operation procedure of the surveillance camera system 5 according to the first embodiment. The imaging unit 13 of the surveillance camera 10 captures an exclusion zone as an example of the surveillance area SA (see FIG. 7) (T1). The communication unit 15 transmits the image captured by the image pickup unit 13 to the image recognition device 30 via the network NW1 (T2).

The communication unit 35 of the image recognition device 30 receives the captured image from the surveillance camera 10. The processor 31 performs image recognition processing on the captured image received via the communication unit 35, and performs a person determination (T3).

In the image recognition process, for example, a person is determined using artificial intelligence. The processor 31 generates a trained model based on the training data of the person image accumulated by machine learning such as deep learning, inputs the captured image to the trained model, and determines the person using the trained model. (In other words, the presence or absence of the target person is determined as the output of the process of detecting and recognizing the person). Further, the processor 31 sets a frame surrounding the determined person, and moves the frame according to the movement of the person image.

In step T3, the person in the image is determined by using artificial intelligence, but the processor 31 performs template matching between the reference image of the person registered in advance and the image captured by the surveillance camera 10 to determine the person. You may. The processor 31 may perform template matching, determine a person when the similarity between the feature amount of the reference image and the captured image is higher than the default value, and similarly set a frame surrounding the person.

The communication unit 35 transmits the result of the person determination and the captured image in which the person determination is performed to the cloud server 40 (T4). Here, if it is determined that the person is a person, the captured image is transmitted, but if it is determined that the person is not a person, the captured image may be transmitted. In the image recognition device 30, even if it is determined that the person is not a person, it is assumed that the cloud server 40 performs the person determination, cancels the determination of the image recognition device, and determines that the person is a person. As a result, it is possible to eliminate an erroneous determination that the person is not a person. Therefore, the accuracy of person detection can be improved.

The communication unit 45 of the cloud server 40 receives the result of the person determination and the captured image after the person determination from the image recognition device 30. The processor 41 of the cloud server 40 performs object recognition processing (T5).

In the object recognition process, for example, a person is determined using artificial intelligence. The processor 41 generates a trained model based on the training data of the person image accumulated by machine learning such as deep learning, inputs the captured image to the trained model, and recognizes the object using the trained model. As the output of the process (that is, the process of detecting and recognizing a person), it is determined whether or not there is a target person. Further, the processor 41 sets a frame surrounding the determined person, and moves the frame according to the movement of the person image.

In step T5, the person in the image is determined by using artificial intelligence, but the processor 41 may perform template matching between the reference image of the person registered in advance and the image captured by the surveillance camera 10. The processor 41 determines a person when the similarity of the feature amount is higher than the default value in the template matching. The processor 41 sets a frame surrounding the person image in which the person is determined, and moves the frame according to the movement of the person image.

Although the same determination method was used in the determinations in steps T3 and T5, different determination methods may be used. For example, in step T3, a person may be determined using artificial intelligence, and in step T5, template matching may be performed to determine a person. On the contrary, in step T3, template matching may be performed to determine a person, and in step T5, artificial intelligence may be used to determine a person. In this case, if the same determination result can be obtained by different determination methods, the determination accuracy is improved. Further, when the judgment results are different, the suspicious judgment result may be displayed on the PC to ask the operator's judgment. Further, the determination result of the cloud server, which is assumed to have higher determination ability than the image recognition device, may be prioritized.

Further, in the determination of step T3 and step T5, it is assumed that the target invading the restricted area is a human being, but a vehicle, a small animal, garbage, or the like may be used.

When a correct report (that is, a result that the recognition result of the image recognition device 30 matches the recognition result of the cloud server 40) is obtained as a result of the person determination, the processor 41 tells the HDD 44 the result of the person determination. Is stored as the image recognition determination result 44a, and the captured image corresponding to the correct report is stored as the captured image data 44b. Further, the processor 41 stores the captured image corresponding to the positive report as learning data, and updates the recognition parameter 44c.

When the communication unit 45 of the cloud server 40 obtains a correct report (that is, the result that the recognition result of the image recognition device 30 matches the recognition result of the cloud server 40) as a result of the person determination, the determination is made. The result and the captured image are transmitted to the PC 50 (T6). The PC 50 receives a determination result and a captured image which are positive reports, and displays the captured image on a display screen of a display (not shown) built in the PC 50 or connected to the PC 50 (T7). The operator can visually recognize the captured image which is the correct report displayed on the PC 50. As a result, the captured image corresponding to the false alarm (that is, the result that the recognition result of the image recognition device 30 does not match the recognition result of the cloud server 40) is not displayed, so that the operator responds to the correct report. You can focus on checking the captured image.

If an erroneous report (that is, a result that the recognition result of the image recognition device 30 does not match the recognition result of the cloud server 40) is obtained as a result of the person determination in step T5, the processor 41 sends the HDD 44 to the HDD 44. The result of the person determination is stored as the image recognition determination result 44a, and the captured image corresponding to the erroneous report is stored as the captured image data 44b. Further, the processor 41 stores the captured image corresponding to the false alarm as learning data, and updates the recognition parameter 44c.

The communication unit 45 of the cloud server 40 recognizes when a false report (that is, a result that the recognition result of the image recognition device 30 does not match the recognition result of the cloud server 40) is obtained as a result of the person determination. The parameter 44c for is transmitted to the image recognition device 30, and an update instruction (an example of a monitoring parameter update instruction) including the parameter 44c for recognition is given (T8). The transmission of the recognition parameter 44c is performed to the image recognition device 30 that has transmitted the captured image in step T4.

FIG. 6 is a diagram showing an example of the registered contents of the parameter table Tb1 in which the recognition parameter 44c is registered. This parameter table Tb1 is stored in the HDD 44 of the cloud server 40. The recognition parameter registered in the parameter table Tb1 is a parameter that is updated in order to suppress an erroneous determination when performing a person determination on an object displayed in a captured image. In the parameter table Tb1, each condition of black-and-white / color switching, focus adjustment, and gain adjustment is registered as an example of the recognition parameter. Further, the recognition parameter may include, for example, the number of neurons in the intermediate layer constituting the neural network and a weighting coefficient when performing image recognition processing by artificial intelligence. Further, when performing image recognition processing by template matching, a reference image to be used is included. Moreover, the parameter for recognition may include a positive report image and a false report image.

Black-and-white / color switching is performed by the image recognition device 30. Black-and-white / color switching is to switch the captured image to a black-and-white image or a color image based on the date and time and the illuminance of the entire image. For example, a color image is adopted during the daytime (bright hours) or when the entire image is bright. A black-and-white image is adopted at night (dark hours) or when the entire image is dark. As an example, in the image pickup element of the surveillance camera 10, when one pixel is composed of a bayer arrangement of R pixel, G pixel, B pixel and IR pixel, a black-and-white image is generated by a signal output from the IR pixel and is colored. The image is generated by the signals output from the R pixel, the G pixel and the B pixel. When performing black-and-white / color switching, the processor 31 of the image recognition device 30 selects an IR pixel signal from among the images captured by the surveillance camera 10 input via the communication unit 35, or selects a black-and-white image. Select the RGB pixel signal and select the color image.

The focus adjustment is performed by the surveillance camera 10. Focus adjustment is to adjust the focus (adjust the focal length) based on the date and time and the degree of distortion (in other words, blur) of the entire image. The CPU 11 of the surveillance camera 10 adjusts the focal length by moving the focus lens arranged in front of the image pickup element in the image pickup unit 13 in the optical axis direction.

The gain adjustment is performed by the surveillance camera 10. Gain adjustment is to adjust the brightness of an image by increasing or decreasing the imaging signal when the amount of light is large due to the influence of weather or nighttime. The CPU 11 of the surveillance camera 10 increases or decreases the image pickup signal by adjusting the gain of the amplifier to which the charge signal from the image pickup element in the image pickup unit 13 is input.

When the communication unit 35 of the image recognition device 30 receives the recognition parameter 44c from the cloud server 40 in step T8, the communication unit 35 updates the recognition parameter 34c stored in the HDD 34 of its own device (T9). The processor 31 sets the black-and-white / color switching included in the recognition parameter 34c as a parameter of its own device. The communication unit 35 transmits one or both of the focus adjustment and gain adjustment parameters included in the recognition parameter 34c to the surveillance camera 10 (T10).

The communication unit 15 of the surveillance camera 10 receives one or both of the focus adjustment and gain adjustment parameters from the image recognition device 30. The CPU 11 stores one or both of the focus adjustment and the gain adjustment parameters in the storage unit 12, and performs one or both of the focus adjustment and the gain adjustment on the image pickup unit 13 (T11).

The HDD 44 of the cloud server 40 stores a parameter table Tb1 in which the optimum recognition parameter 44c, which is updated in real time based on the captured images of positive or false reports received from many image recognition devices 30, is registered. To. The processor 41 periodically (here, once a day as an example) transmits the recognition parameter 44c registered in the parameter table Tb1 stored in the HDD 44 to each image recognition device 30, and this recognition. A usage instruction (monitoring parameter usage instruction) including the parameter 44c for is given (T12).

When the communication unit 35 of each image recognition device 30 receives the recognition parameter 44c from the cloud server 40 in step T12, the communication unit 35 updates the recognition parameter 34c stored in the HDD 34 of its own device (T13). Each processor 31 sets black-and-white / color switching included in the recognition parameter 34c as a parameter of its own device. Each communication unit 35 transmits one or both of the focus adjustment and gain adjustment parameters included in the recognition parameter 34c to the respective surveillance cameras 10 (T14).

The communication unit 15 of each surveillance camera 10 receives one or both of the focus adjustment and gain adjustment parameters from the image recognition device 30. Each CPU 11 stores one or both of the focus adjustment and gain adjustment parameters in the storage unit 12, and performs one or both of the focus adjustment and the gain adjustment on the image pickup unit 13 (T15). .. After that, the surveillance camera 10 returns to the process of step T1, and the surveillance camera system 5 repeats the same process.

FIG. 7 is a diagram showing an example of the captured image GZ1 of the surveillance camera 10 displayed when the correct report is notified. The captured image GZ1 shown in FIG. 7 is displayed, for example, on the display screen of a display built in the PC 50 (not shown) or a display connected to the PC 50 (not shown). The surveillance camera 10 is installed above, for example, near the entrance of a highway. The captured image GZ1 shows the road from the entrance of the expressway to the tollhouse gate as the surveillance area SA, which is captured by the surveillance camera 10 so as to take a bird's-eye view. The captured image GZ1 shows three people, hm1, hm2, and hm3, who are determined to be people and are likely to be workers. Further, the frames w1, w2, w3 are superimposed and displayed on the captured image GZ1 so as to surround the upper bodies of the three people hm1, hm2, hm3, respectively. The operator visually recognizes three people hm1, hm2, hm3 surrounded by frames w1, w2, w3 displayed in the captured image GZ1 displayed on the display screen of the display (not shown) described above. In this case, since it is determined that the three persons hm1, hm2, hm3 are related to the road, the operator does not take any particular measures, for example. The captured image GZ1 is, for example, a positive image.

FIG. 8 is a diagram showing an example of the captured image GZ2 of the surveillance camera 10 displayed when a false alarm is notified. When a false alarm is notified, the captured image GZ2 by the surveillance camera 10 is not displayed on the PC50, but the captured image GZ1 shown in FIG. 8 is, for example, a display built in the PC50 (not shown) or a display connected to the PC50 (not shown). It may be displayed on the display screen (not shown). When the captured image GZ2 is transmitted from the image recognition device 30 to the cloud server 40, the processor 41 stores the captured image GZ2 as the captured image data 44b in the HDD 44. The captured image GZ2 is used as learning data of the parameter 44c for recognition. The captured image GZ2 shows the shadow e1 and the sign e2 of the pole, which are erroneously determined to be a person due to the shade. Further, the frames w4 and w5 are superimposed and displayed on the captured image GZ2 so as to surround a part of the shadow e1 of the pole and the marker e2, respectively. However, since the captured image GZ2 is not displayed on the PC 50, the operator does not notice the erroneously determined captured image GZ2. In this case, when the captured image is always displayed on the PC 50, the frames w4 and w5 are not displayed. If the operator is free, the erroneously determined captured image GZ2 may be displayed on the PC 50, and the result of the person determination can be confirmed. The captured image GZ2 is a false alarm image.

As described above, the surveillance camera system 5 according to the first embodiment includes an image recognition device 30 (an example of a first processing device) connected to a surveillance camera 10 installed so as to be able to capture a surveillance area SA, and a cloud server 40 (an example of a first processing device). An example of a second processing device) is communicably connected. The image recognition device 30 recognizes the person hm1, hm2, hm3 (an example of a specific object) appearing in the captured image based on the captured image of the surveillance area SA captured by the surveillance camera 10, and the recognition result and the image pickup. The images are associated and transmitted to the cloud server 40. When the cloud server 40 recognizes an object (an example of an object that is not a specific object) such as a shadow e1 or a marker e2 that is not a person based on the recognition result and the captured image sent from the image recognition device 30, the cloud server 40 holds the object. An instruction to use the monitoring parameter including the recognition parameter (an example of the monitoring parameter) is transmitted to the image recognition device 30. The recognition parameters held by the cloud server 40 are transmitted to the image recognition device 30 because the cloud server 40 is superior to each image recognition device 30 in the object recognition ability. That is, if the recognition result of the cloud server 40 is prioritized over the recognition result of the image recognition device 30, and the recognition result of the image recognition device 30 does not match the recognition result of the cloud server 40, the cloud server 40 holds the data. The recognition parameters are sent to the image recognition device 30 as correct answer data.

As a result, when the cloud server 40 determines that the person who is the target appearing in the captured image captured by the surveillance camera 10 is erroneously recognized by the image recognition device 30, the surveillance camera system 5 causes the image recognition device 30 to determine. The recognition parameters used to recognize the captured image can be updated appropriately. That is, as described above, the cloud server 40 is superior to the image recognition device 30 in the recognition ability of an object (that is, an object such as a person or an object) appearing in the captured image. The cloud server 40 receives the recognition result of a specific object such as a person sent from the image recognition device 30 and the captured image, and re-verifies the recognition result of the image recognition device 30 to recognize the recognition used by the image recognition device 30. Decide whether to update the parameters for. Therefore, according to the surveillance camera system 5, it is possible to improve the recognition accuracy of a person who may appear in the image captured by the surveillance camera 10, suppress the deterioration of the surveillance accuracy in the surveillance area SA, and reduce the burden on the observer. Be done.

Further, the image recognition device 30 uses the recognition parameter used when recognizing the captured image of the surveillance area SA captured by the surveillance camera 10 as an update instruction (an example of a surveillance parameter update instruction) sent from the cloud server 40. Update to the included recognition parameters. As a result, the cloud server 40 can transmit the recognition parameters held by its own device to all the image recognition devices 30 and update the recognition parameters registered in the image recognition device 30. Therefore, the recognition parameters registered in each image recognition device 30 can be shared, and the accuracy of image recognition can be made uniform. For example, when there is maintenance or the like, the image recognition device 30 can be temporarily replaced.

Further, the recognition parameter includes a switching condition for switching the captured image of the monitoring area SA to a black-and-white image or a color image. The image recognition device 30 uses this switching condition to switch the captured image of the monitoring area SA to a black-and-white image or a color image. As a result, the image recognition device 30 and the cloud server 40 obtain a black-and-white image or a color image suitable for image recognition as an image captured by the surveillance camera 10 even when the amount of light varies depending on the date and time or the illuminance of the entire image. be able to. Further, the recognition accuracy is improved by performing image recognition on the captured image suitable for image recognition.

Further, the recognition parameter includes the focus adjustment condition at the time of imaging by the surveillance camera 10. The image recognition device 30 transfers a usage instruction including this focus adjustment condition to the surveillance camera 10. The image recognition device 30 causes the surveillance camera 10 to update the focus adjustment condition at the time of imaging. The image recognition device 30 receives the captured image captured by the surveillance camera 10 using the updated focus adjustment condition. As a result, the image recognition device 30 and the cloud server 40 can adjust the focus of the image captured by the surveillance camera 10 to sharpen the captured image even when the image is blurred due to the date and time or the disturbance of the entire image. .. Further, by performing image recognition on the sharpened captured image, the recognition accuracy is improved.

Further, the recognition parameter includes a gain adjustment condition at the time of imaging by the surveillance camera 10. The image recognition device 30 transfers a usage instruction including a gain adjustment condition to the surveillance camera 10. The image recognition device 30 updates the gain adjustment conditions at the time of imaging by the surveillance camera 10. The image recognition device 30 receives the captured image captured by using the updated gain adjustment condition. As a result, the image recognition device 30 and the cloud server 40 adjust the gain of the image pickup signal output from the image pickup unit 13 of the surveillance camera 10, so that the brightness changes due to the influence of the weather, nighttime, or the like. , The brightness of the captured image can be adjusted appropriately. Further, the recognition accuracy is improved by performing image recognition on the captured image whose brightness is appropriately adjusted.

Further, the recognition parameter included in the usage instruction includes at least one of a positive image showing an image that has been recognized as a person and an erroneous image showing an image that has been erroneously recognized as a person. The image recognition device 30 recognizes the captured image of the surveillance area SA captured by the surveillance camera 10 by using the positive report image or the false report image. As a result, the image recognition device 30 and the cloud server 40 perform image recognition using the positive report image or the false report image, and the recognition accuracy is improved. In addition, image recognition may be performed using either a positive image or a false alarm image, which enhances the learning effect and leads to an improvement in recognition accuracy.

Further, when the cloud server 40 recognizes a person based on the recognition result and the captured image sent from the image recognition device 30, the recognition result in the image recognition device 30 or the cloud server 40 is connected to the cloud server 40. It is displayed on a display (an example of a monitor) built in or connected to the PC 50. As a result, when a person is recognized, the PC 50 connected to the cloud server 40 can display the image GZ1 captured by the surveillance camera 10 which is a positive image, and the frames w1, w2, and w3 representing the recognition result. Further, since the captured image GZ2 that erroneously recognizes a person is not displayed, the amount of the captured image visually recognized by the operator can be reduced. Therefore, the operator can quickly deal with an event that has occurred, such as a person invading an exclusion zone, and can reduce the oversight of the event.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications, modifications, substitutions, additions, deletions, and even examples within the scope of the claims. It is naturally understood that it belongs to the technical scope of the present disclosure. Further, each component in the various embodiments described above may be arbitrarily combined as long as the gist of the invention is not deviated.

For example, in the above embodiment, the case of monitoring the entrance of the expressway is shown, but the case is not limited to this example, and a place where people or things cannot enter, for example, a barrier that prohibits the entry of people or cars is dismounted. This disclosure is also applicable when monitoring railroad crossings, radiation controlled areas, zoos and parks that are not accessible at night.

In the above embodiment, the case where the target is a person is shown, but the target includes those who are not permitted to enter the restricted area, cars, small animals, garbage, and the like.

In the above embodiment, as the recognition result of the person, the frames w1, w2, w3 surrounding the person are displayed in the captured image, but a mark or a message indicating that the person has been recognized may be displayed.

In the above embodiment, the PC connected to the cloud server displays the captured image that recognizes the person, but a monitor may be connected to the image recognition device and the captured image that the monitor recognizes the person may be displayed. As a result, the operator can compare the captured image displayed on the PC and the captured image displayed on the monitor, and can also determine whether or not the person is the final person.

The present disclosure also covers a program in which a program that realizes the functions of the device of the above embodiment is supplied to the device via a network or various storage media, and is read and executed by a computer in the device.

The present disclosure updates the monitoring parameters related to the recognition of the captured image when the specific object that is the monitored object appearing in the captured image is erroneously recognized, improves the recognition accuracy of the specific object, and suppresses the deterioration of the monitoring accuracy. It is useful as a monitoring parameter update system, monitoring parameter update method, and program.

5 Surveillance camera system 10 Surveillance camera 11 CPU
12 Storage unit 13 Imaging unit 14 Image processing unit 15 Communication unit 16 Pan-tilt drive unit 18 Accelerometer 19 IR irradiation unit 30 Image recognition device 31 Processor 32 ROM
33 RAM
34 HDD
35 Communication unit 40 Cloud server 41 Processor 42 ROM
43 RAM
44 HDD
45 Communication unit 50 PC

Claims (10)

It is a monitoring parameter update system in which the first processing device connected to a camera installed so that the monitoring area can be imaged and the second processing device are connected so as to be communicable.
The first processing device is
Based on the captured image of the monitoring area captured by the camera, a specific object appearing in the captured image is recognized, and the recognition result and the captured image are associated with each other and transmitted to the second processing device.
The second processing device is
When an object other than the specific object is recognized based on the recognition result and the captured image sent from the first processing device, a monitoring parameter update instruction including a monitoring parameter is transmitted to the first processing device .
Monitoring parameter update system. The first processing device updates the monitoring parameters used when recognizing the captured image of the monitoring area captured by the camera to the monitoring parameters included in the monitoring parameter update instruction sent from the second processing device . ,
The monitoring parameter update system according to claim 1. The monitoring parameter is a switching condition for switching the captured image of the monitoring area to a black-and-white image or a color image.
The first processing device is
Using the switching condition, the captured image of the monitoring area is switched to a black-and-white image or a color image.
The monitoring parameter update system according to claim 2. The monitoring parameter is a focus adjustment condition at the time of imaging by the camera.
The first processing device transfers the monitoring parameter update instruction including the focus adjustment condition to the camera, updates the focus adjustment condition at the time of imaging of the camera, and is imaged using the updated focus adjustment condition. Receive the captured image,
The monitoring parameter update system according to claim 2. The monitoring parameter is a gain adjustment condition at the time of imaging by the camera.
The first processing apparatus transfers the monitoring parameter update instruction including the gain adjustment condition to the camera, updates the gain adjustment condition at the time of imaging of the camera, and is imaged using the updated gain adjustment condition. Receive the captured image,
The monitoring parameter update system according to claim 2. The monitoring parameter included in the monitoring parameter update instruction is at least a positive image showing an image that has been recognized as the specific object and a false alarm image showing an image that has been erroneously recognized as the specific object. Have one
The first processing apparatus recognizes the captured image of the monitoring area captured by the camera using the positive image or the false alarm image.
The monitoring parameter update system according to claim 2. When the second processing device recognizes the specific object based on the recognition result and the captured image sent from the first processing device, the second processing device obtains the recognition result in the first processing device or the second processing device. Displayed on the monitor connected to the second processing device ,
The monitoring parameter update system according to any one of claims 1 to 6. It is a monitoring parameter updating method using a monitoring parameter updating system in which a first processing device connected to a camera installed so that a monitoring area can be imaged and a second processing device are connected so as to be able to communicate with each other.
The first processing device is
Based on the captured image of the monitoring area, a specific object appearing in the captured image is recognized, and the recognition result and the captured image are associated with each other and transmitted to the second processing device.
The second processing device is
When an object other than the specific object is recognized based on the recognition result and the captured image sent from the first processing device , a monitoring parameter update instruction including a monitoring parameter is transmitted to the first processing device .
How to update monitoring parameters. To the first processing device, which is a computer, connected to the camera installed so that the monitoring area can be imaged and the second processing device, respectively.
A step of recognizing a specific object appearing in the captured image based on the captured image of the monitoring area captured by the camera , and
A step of associating the recognition result of the specific object and the captured image and transmitting the image to the second processing device ,
When an object that is not the specific object is recognized by the second processing device based on the recognition result of the specific object and the captured image, the monitoring parameter update instruction including the monitoring parameter sent from the second processing device is received . Steps to perform, and to run them on your computer,
program. To the second processing device, which is a computer, which is communicably connected to the first processing device connected to the camera installed so that the monitoring area can be imaged.
When the specific object appearing in the captured image is recognized by the first processing device based on the captured image of the monitoring area captured by the camera .
A step of recognizing an object that is not the specific object based on the recognition result and the captured image of the specific object sent from the first processing device .
When an object that is not the specific object is recognized, a step of transmitting a monitoring parameter update instruction including a monitoring parameter to the first processing device and a step of transmitting the monitoring parameter update instruction to the first processing device are executed on the computer .
program.

Images