JP5127883B2 - Network camera error detection and notification method - Google Patents

Description

  The present invention relates to a method for detecting and reporting a failure occurring in a monitoring camera or the like and an abnormality that has occurred artificially, and more particularly to a network camera abnormality detection and notification method applied to a network camera.

  2. Description of the Related Art Conventionally, as a crime prevention system, a surveillance camera that suppresses crime by performing surveillance with a captured image by a fixed camera is used. This monitoring camera uses the analog camera that is used, and many of them send video signals to the monitoring terminal side and decode them on the monitoring terminal side. Channel loss (video signal lost) is used as a method for detecting the failure or abnormality. A method of notifying an administrator, a security company, or the like that a failure or abnormality has occurred in the surveillance camera when the camera is detected.

  However, due to the remarkable expansion of TCP / IP network usage scenes in recent years, surveillance cameras are also connected to TCP / IP networks like network cameras, and photographed images or videos become viewable and can be viewed on surveillance terminals such as personal computers. It has been replaced with the one to send.

  For example, in Patent Document 1, a monitoring system using a network camera and a sensor detects an abnormality to be monitored by the sensor, transmits the abnormal data to the monitoring terminal by superimposing the abnormal data on the image data, and the monitoring terminal transmits the data. A configuration is disclosed in which display and abnormality notification are performed at a monitoring terminal by performing separate decoding.

  Also, in Patent Document 2, a configuration is adopted in which a change in video is detected as abnormal by a network camera, is added to video data as abnormal information and transmitted to the monitoring side, and the added abnormal information is alarmed to the monitoring side. It is disclosed.

JP 2005-354367 A JP 2005-268996 A

  However, although the method of Patent Document 1 can detect the failure of the monitoring target, it cannot detect the failure of the network camera itself, and even if a suspicious object or the like is installed around the monitoring target, it is detected if the sensor does not react. Can not.

  Further, the method of Patent Document 2 makes an abnormality when the image is changed, and is not suitable for outdoor monitoring in which the brightness varies depending on, for example, a place where people or vehicles are busy and time zones.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a network camera abnormality detection and notification system capable of appropriately detecting and reporting an abnormality in a monitoring location.

A network camera abnormality detection and notification system according to claim 1 is a network camera that captures a still image or a moving image to create image data, an image analysis device that is communicably connected to the network camera, and the image analysis device. a network camera anomaly detection and notification system constituted by the monitoring terminal that is communicably connected, the image analyzer, an image dividing means for dividing the captured image transmitted from the network camera, the image An image storage unit that stores the captured images obtained by the dividing unit, and a plurality of captured images stored in the image storage unit are divided into image comparison units (arbitrary units for partial comparison of images), Average image data, which is a set of average values for all image comparison units, calculated for the average values of various parameters for each image comparison unit The average image generating means to be generated, the average image data generated by the average image generating means in the past, and the average image data generated by the image data obtained within the most recent predetermined time are compared between the same image comparison units. and an image data comparing means the degree of change is determined whether more than a predetermined threshold value, if the degree of change in the data to which the image data comparing means comparing exceeds a predetermined threshold value, to the monitoring terminal An abnormality detection / notification means for performing abnormality notification is provided.

Network camera anomaly detection and reporting system according to claim 2 is the network camera anomaly detection and notification system of claim 1, wherein said image analyzing apparatus includes an average picture library to accumulate further average image data, the image A plurality of image data captured within a predetermined time by the image data comparison means of the analysis device are compared for each image comparison unit, a change rate in the image comparison unit is calculated, and the change rate exceeds a predetermined threshold value. If not, the average image data generated by the average image generation means is stored in the average image library with classification information attached thereto.

The network camera abnormality detection and notification system according to claim 3 is the network camera abnormality detection and notification system according to claim 2, wherein the classification information attached to the average image data stored in the average image library is a predetermined value. It is classified according to time zone, predetermined year / month, predetermined day of the week, predetermined season, and weather.

  The network camera abnormality detection and notification system according to claim 4 is the network camera abnormality detection and notification system according to any one of claims 1 to 3, wherein the image data comparison means calculates the degree of change of the image data. The image comparison unit to be compared with the threshold value is limited to an image comparison unit designated in advance.

According to the network camera abnormality detection and notification system according to claim 1, a network camera that captures a still image or a moving image and creates image data, an image analysis apparatus that is communicably connected to the network camera, and the image analysis a monitoring terminal that is communicably connected to the device, a network camera anomaly detection and notification system constituted by the image analyzer, an image dividing means for dividing the captured image transmitted from the network camera, dividing said image storing means for storing the photographed image obtained by the image dividing means, a plurality of captured images stored in the image storage means in the image comparison unit (arbitrary units in comparing the images partially) The average image data, which is a set of average values for all image comparison units, calculated for the average values of various parameters for each image comparison unit. Average image generation means for generating data, average image data generated by the average image generation means in the past, and average image data generated by image data obtained within the most recent predetermined time, in comparison, the image data comparing means the degree of change is determined whether more than a predetermined threshold value, if the degree of change in the data to which the image data comparing means comparing exceeds a predetermined threshold value, the monitoring terminal In addition to blocking communication with the network camera, it can detect the degree of change in video and perform detailed abnormality determination according to that level. it can.

According to the network camera abnormality detection and notification system according to claim 2, the image analysis device further includes an average image library that classifies and stores average image data according to various parameters, and the image data comparison of the image analysis device. A plurality of image data photographed within a predetermined time by the means are compared for each image comparison unit, a change rate in the image comparison unit is calculated, and if the change rate does not exceed a predetermined threshold, the average image generation Since the average image data at that time generated by the means is added to the average image library with classification information, the average image data to be compared with the current captured image is classified according to various conditions. By making a library, the reusability of the average image data can be improved.

According to the network camera abnormality detection and notification system according to claim 3, the classification information attached to the average image data stored in the average image library includes a predetermined time zone, a predetermined year and month, a predetermined day of the week, Since classification is performed for each predetermined season, it is easy to obtain average image data that matches the time and season of the captured image to be compared.

  According to the network camera abnormality detection and notification system according to claim 4, the image data comparison unit calculates a degree of change of the image data and sets an image comparison unit to be compared with a threshold as an image comparison unit designated in advance. Since the image data corresponding to the specific image comparison unit is excluded from the comparison target, a part of the image specified in the specific image comparison unit is not handled as the comparison target. For example, in a monitoring camera on a road with a large amount of traffic, erroneous detection can be prevented by performing settings such that image comparison units including sidewalks and roadways with large image fluctuations are not compared.

It is a block diagram which shows the system configuration | structure of a network camera abnormality detection and notification system in the Example of this invention. It is explanatory drawing which shows the production | generation method of the average image in a network camera abnormality detection and notification system same as the above. It is explanatory drawing which shows the acquisition operation | movement of the change degree of an image by performing block division of image data by a network camera abnormality detection and notification system same as the above. It is a flowchart figure which shows operation | movement of a network camera abnormality detection and notification system same as the above. It is a flowchart figure which shows the operation | movement of abnormality detection by the block division | segmentation of a monitoring image | video same as the above. It is a flowchart figure which shows the operation | movement of abnormality detection by the area specification of a monitoring image | video same as the above. It is a flowchart figure which shows the operation | movement of the abnormality detection by the whole image of a monitoring image as above.

  Hereinafter, an embodiment as the best mode for carrying out the present invention will be described with reference to FIGS. Of course, it goes without saying that the present invention can be easily applied to other than those described in the embodiments without departing from the spirit of the invention.

  Based on FIG. 1, the structure of the network camera abnormality detection and notification system in a present Example is demonstrated. Reference numeral 1 denotes a network camera abnormality detection and notification system that analyzes a captured image of a network camera described later. A network camera 10 records an object to be monitored as an image or video and transmits image data or moving image data to a recording device or the like. Reference numeral 20 denotes an IP network such as the Internet. Reference numeral 30 denotes an image analysis apparatus which acquires and analyzes image data or moving image data (hereinafter simply referred to as image data) captured by the network camera 10 via the IP network 20. Reference numeral 40 denotes an image storage device for storing image data used in the image analysis device 30. Reference numeral 50 denotes a monitoring terminal, which displays a comparison result between image data transmitted from the image analysis apparatus 30 and an average image library described later, and performs monitoring by the user viewing it.

The configuration of the image analysis device 30 will be described. Image analysis software 31 is installed in the image analysis device 30. Image analysis software 31 analyzes the image data transmitted from the network camera 10, the image dividing unit 31a, the divided image data to the image comparison unit taken within a predetermined period of time for dividing the image data into image comparison unit Average image generation means 31b for generating average image data for each image comparison unit from the image data comparison means for comparing with average image data stored in an average image library, which will be described later, and analyzing the degree of change in image comparison units 31c, an abnormality detection / notification unit 31d that compares the degree of change analyzed by the image data comparison unit 31c with a threshold value and reports to a monitoring terminal described later according to the comparison result.

  Further, the image analysis apparatus 30 includes an average image library 32, and the average image data generated by the average image generation means 31b is attached with classification information that can determine the generated season, year, month, time zone, and weather. The image data comparison unit 31d accumulated in the average image library 32 acquires average image data whose classification information matches the current time and the current weather, and uses them as comparison targets. In addition, a log recording device 33 for recording an operation log is provided, and it is possible to perform log maintenance as necessary to perform system maintenance or the like.

  All the image data transmitted to the image analysis device 30 is transmitted to the image storage device 40 when the analysis is completed, and is stored in the image storage HDD 41.

  The image data transmitted to the image analysis device 30 is transferred to the monitoring terminal 50 and displayed on the display device of the monitoring terminal 50 or the like. If the abnormality detection / notification means 31d determines that there is an abnormality, the notification information is also transmitted and an alarm is given to the monitoring terminal 50.

  Next, an example of the operation of the abnormality detection / notification unit 31d based on the average image data will be described with reference to FIG. In this description, it is assumed that the average image data is generated once every 5 minutes. In FIG. 2, the average image 1 includes one or more parameters (from information constituting image data such as luminance, color tone, and contrast) from image data captured at intervals of 1 minute between 10:00 and 10:05. Is selected as appropriate), and average image data obtained by taking the average value. Average image 2 is 10:05 to 10:10, average image 3 is 10:10 to 10:15, average image 4 is 10:15 to 10:20, and average image data is obtained in the same manner as average image 1. is there. Each of the average images 1, 2, and 3 includes a slightly different image in each case, but this is abnormal by using average image data obtained by averaging the image data taken for 5 minutes. Is no longer determined. However, the image taken at 10:15 to 10:20 corresponding to the average image 4 has undergone a great change in four of the five images, and when the average image data is generated, it is compared with the average images 1-3. However, it can be seen that a large change has occurred, and this can be determined as an error (abnormality). In this way, when judging by each image, a large amount of errors occur, and there is a possibility that a large error is buried in a small error. By doing this, it is possible to reliably catch large errors.

Next, comparison of image data for each image comparison unit by the image data comparison unit 31c of the image data divided into blocks by the image division unit 31a will be described with reference to FIG. When the network camera 10 captures a state in which a person is moving from right to left, as shown in the figure, when the images are compared for each of the six images, the object appears as a large change with a slight movement. End up. By averaging the six pieces of image data for each image comparison unit, the degree of change is moderated, and if the object only moves, it is not detected as abnormal. Conversely, if there is no significant change in the entire image, such as when there is condensation on only a part of the image, but a fatal failure occurs on a part of the image, the condensation that covers a part of the image will be In some cases, it is not detected as a large variation. However, when divided into image comparison units, a dew spot can be detected as having a large variation even if there is no large variation in most blocks.

  The network camera anomaly detection and notification system 1 in this embodiment performs image comparison unit detection for detecting camera anomalies divided into predetermined image comparison units, and anomaly detection for image comparison units within a pre-specified coordinate range. Which of the three detection modes of area detection and overall detection for detecting an abnormality in the entire video captured by the camera is set in advance, and the abnormality is detected based on the setting. The operation will be described with reference to FIG. It is determined whether or not the detection mode set in advance is the image comparison unit detection mode (S401). If the detection mode is the image comparison unit detection mode, a video comparison operation based on the image comparison unit detection mode described later is performed (S402). In the area detection mode (S403), a video comparison operation in the area detection mode described later is performed (S404). If none of these applies, a video comparison operation in the entire detection mode is performed (S405).

Next, the operation in the image comparison unit detection mode will be described with reference to FIG. First, image data e photographed from a network camera is received (S501). It is determined whether it is within a preset monitoring image detection time (S502). If it is within the detection time, the image dividing means 31a divides the received image data into blocks (S503). It is determined whether the degree of change of the image of any image comparison unit exceeds a predetermined threshold (S504). If the threshold value is exceeded, the image data e is copied as the notification image data a (S505). The latest (for example, 5 minutes) average image data (referred to as average image data aa) is generated (S506). The average image library is searched based on the ID of the network camera 10 that captured the image data and the date and time, and the corresponding average image data for comparison (referred to as average image data bb) is read (S507). The image data comparison unit 31c compares the average image data aa and the average image data bb for each divided image comparison unit (S508). In any image comparison unit, it is determined whether the degree of image change exceeds a predetermined threshold (S509). The number of image comparison units exceeding the threshold is measured (S510). The position of the image comparison unit exceeding the threshold is recorded (S511). A factor code c is generated based on the number of blocks exceeding the threshold and their positions and recorded in a log together with the image information f of the notification image data a (S512). The factor code c is, for example, a code indicating that the number of blocks is small and there is a block that exceeds the threshold at the bottom of the screen, indicating misplacement, stagnant, and suspicious objects. This code indicates a defect or dirt, and if there are a large number of image comparison units, a code such as monitoring obstruction or lens failure can be used, and the cause of the report can be roughly determined without the monitor checking the details of the image. Can be grasped. Subsequently, it is determined whether the number of image comparison units exceeding the threshold exceeds a predetermined number (S513). If the number exceeds the predetermined number, the monitoring terminal is added with camera information d of the network camera 10 (camera installation location, camera name, IP address, etc.), detection date and time g, notification image data a, and factor code c. 50 is notified (S514).

  If the degree of change in the image data divided into image comparison units in S504 does not exceed the threshold value, a predetermined time has elapsed from the time when the image divided in the previous image comparison unit changed beyond the threshold value. It is determined whether or not it has been done (S515). If the predetermined time has elapsed, camera information d and shooting date / time h are added to generate a new average image (S516). The generated average image data is stored in the average image library (S517). At this time, if there is already an average image of the same camera and the same date and time, overwriting is performed. If it is outside the detection time in step S502, or if a predetermined time has not elapsed since the time when the previous block divided image changed beyond the threshold in step S515, the received image data e is stored in the image. The image reception pointer is incremented by one for reception of the next photographed image stored in the HDD 41 (S518).

  Next, the operation in the area detection mode will be described with reference to FIG. First, image data e photographed from a network camera is received (S601). It is determined whether it is within a preset monitoring image detection time (S602). If it is within the detection time, it is determined whether or not there is a change that exceeds a predetermined threshold in the image comparison unit in the designated area set in advance by a rectangle or a closed curve designated by the user (S603). When the threshold value is exceeded, the image data e is copied as the notification image data a (S604). The latest (for example, 5 minutes) average image data (referred to as average image data aa) is generated (S605). The average image library is searched based on the ID of the network camera 10 that captured the image data and the date and time, and the corresponding average image data for comparison (referred to as average image data bb) is read (S606). Whether various parameters for each image comparison unit corresponding to the designated area of the image data e are compared with various parameters for each image comparison unit corresponding to the designated area of the average image data bb, and whether the rate of change exceeds a predetermined threshold Then, it is determined whether or not the real number of image comparison units whose rate of change exceeds the threshold exceeds a predetermined threshold (S607). If the predetermined threshold value is exceeded, the image information f is recorded in a log (S608), the camera information d of the network camera 10 (camera installation location, camera name, IP address, etc.), detection date and time g, notification The image data “a” and the factor code “c” are added, and the monitoring terminal 50 is notified (S609).

  If the degree of change of the image data in the designated area does not exceed the threshold value in S603, it is determined whether or not a predetermined time has elapsed from the time when the change exceeded the previous threshold value (S610). If the predetermined time has elapsed, the camera information d and the shooting date / time h are added to generate a new average image (S611). The generated average image data is stored in the average image library (S612). At this time, if there is already an average image of the same camera and the same date and time, overwriting is performed. If it is outside the detection time in step S602, or if the predetermined time has not elapsed since the previous time when the change exceeded the threshold in the designated area of the image data in step S607, the received image data e Are stored in the image storage HDD 41, and the image reception pointer is advanced by one for reception of the next photographed image (S613).

  Next, the overall mode operation will be described with reference to FIG. As in the above two modes, the image data e captured from the network camera is received (S701), and it is determined whether it is within the preset monitoring image detection time (S702). If it is within the detection time, it is determined whether there is a change in the entire image that exceeds a predetermined threshold (S703). When the threshold value is exceeded, the image data e is copied as the notification image data a (S704). The latest average image data aa is generated (S705). The average image library is searched based on the ID of the network camera 10 that captured the notification image data a and the date and time, and the corresponding average image data for comparison (referred to as average image data bb) is read (S706). The notification image data a and the average image data bb are compared, and it is determined whether or not the rate of change exceeds a threshold value (S707). If the predetermined threshold value is exceeded, the image information f is recorded in a log (S708), the camera information d of the network camera 10 (camera installation location, camera name, IP address, etc.), detection date and time g, notification information The image data “a” and the factor code “c” are added, and the monitoring terminal 50 is notified (S709).

  With the above three modes, for example, when it is desired to limit the monitoring target at roughly positions, the image comparison unit mode is selected, a complicated range that is difficult to specify in the image comparison unit mode, or the image comparison unit is the entire image data. If it is too small, it will be done in the area mode if each designation is complicated, and if it is desired to detect an abnormality in the whole image, it will be done in the whole mode. In addition, various abnormality detection methods can be selected.

  According to the network camera abnormality detection and notification system 1 that performs the operation as described above, the setting to cope with the environment to which the monitoring system is applied is compared with the system that discriminates the image data by comparing with the reference image itself. It can be done flexibly, and it is possible to strictly define the events that should not be detected and the events that should be detected, and prevent false detection in places where the image changes drastically and where the image changes little. it can. In addition, since average image data can be made into a library and managed by various classifications, for example, it may be bright or dark depending on the season even in the same time zone, but average image data can be extracted from the library according to the same conditions And the frequency of erroneous detection can be further suppressed.

  As mentioned above, although one Example of this invention was explained in full detail, this invention is not limited to the said Example, A various deformation | transformation implementation is possible within the range of the summary of this invention. For example, the image comparison unit division of the image data shown in FIG. 3 is set to 36 × 6 × 6. However, the present invention is not limited to this, and when it is desired to discriminate every pixel (pixel), the image comparison unit is set to one pixel. The number of divisions may be appropriately selected and used. Although image data is taken every minute and average image data is generated every 5 minutes, the shooting interval and the average image data generation interval are shortened according to the environment to which the network camera abnormality detection and notification system 1 is applied. There is no problem even if it is extended. In addition, a configuration for comparing images using an average image library and detecting an abnormality, and a configuration for dividing image data into image comparison units and comparing images are each independently established as an invention.

DESCRIPTION OF SYMBOLS 1 Network camera abnormality detection and notification system 30 Image analysis device 31 Image analysis software 31a Image division means 31b Average image generation means 31c Image data comparison means 31d Abnormality detection and notification means 32 Average image library 33 Log storage device 40 Image storage device 41 Image Storage HDD (Hard Disk Drive)
50 Monitoring terminal

Claims (4)

A network camera that captures a still image or a moving image and creates image data, an image analysis device that is communicably connected to the network camera, and a monitoring terminal that is communicably connected to the image analysis device. Network camera abnormality detection and notification system,
The image analysis device includes:
Image dividing means for dividing the captured image transmitted from the network camera into arbitrary image comparison units for partial comparison ;
An image storing means for storing the photographed image obtained by the image dividing means;
An average, which is a set data of average values for all image comparison units, obtained by calculating an average value of various parameters for each image comparison unit , for a plurality of captured images divided in the image comparison unit stored in the image storage unit Average image generating means for generating image data;
The average image data generated by the average image generating unit in the past, the average image data generated by the image data obtained in the most recent predetermined time, compared with the same image comparison unit between the degree of variation is predetermined Image data comparison means for determining whether the threshold value is exceeded;
An abnormality detection / notification means for notifying the monitoring terminal of an abnormality when the degree of change of the data compared by the image data comparison means exceeds a predetermined threshold, and a network camera abnormality Detection and notification system. Wherein the image analyzing apparatus further comprises an average picture library to accumulate the average image data, comparing the plurality of image data captured within a predetermined time in the image comparison each unit by the image data comparison unit of the image analyzer When the ratio of change in the image comparison unit is calculated and the ratio of change does not exceed a predetermined threshold value, the average image data at that time generated by the average image generating means is added with classification information and the average The network camera abnormality detection and notification system according to claim 1, wherein the network camera abnormality detection and notification system is stored in an image library. The classification information attached to the average image data stored in the average image library is classified according to a predetermined time zone, a predetermined year / month, a predetermined day of the week, a predetermined season, and weather. Item 3. The network camera abnormality detection and notification system according to item 2. 4. The image data comparison unit according to claim 1, wherein the image data comparison unit calculates a degree of change of the image data and limits an image comparison unit to be compared with a threshold to an image comparison unit designated in advance. The network camera abnormality detection and notification system according to item 1.

Images