JP7479535B1 - Person detection system and person detection method - Google Patents

Abstract

[Problem] To provide a human detection system and a human detection method capable of automatically detecting a human in a monitored area. [Solution] A human detection system including a surveillance camera equipped with at least a thermal camera that captures images of the monitored area at a predetermined time interval, a video analysis server 20 including a central processing unit that processes the images, and a video storage unit that stores the images, the central processing unit including a moving object detection unit and an AI determination unit for processing the images. The moving object detection unit uses a frame difference method to cut out three or more consecutive frames from the images, calculates the absolute value of the difference between two consecutive frames, repeats this for the next two frames, and calculates the logical product of the differences to extract an object that continues to move over three or more frames. The AI determination unit performs image analysis processing using artificial intelligence on the object to determine whether the object is a person. [Selected Figure] Figure 1

Description

The present invention relates to a human detection system and a human detection method, and more particularly to a human detection system and a human detection method that analyze images captured by a surveillance camera to detect people within a surveillance area.

There is a need for technology that can automatically detect people using surveillance cameras. In places such as factories where civilians are prohibited from entering and where there are few people coming and going at night, illegal intrusions and thefts occur at night. Therefore, there is a need for technology that can automatically detect whether a detected person is authorized to enter using surveillance cameras. Thefts by illegal intrusion often occur at night, so monitoring people's movements at night has become a major theme. For nighttime surveillance, thermal cameras and combinations of infrared lights and cameras are used.

Here, known conventional human detection systems and methods are, for example, those disclosed in Patent Documents 1 and 2. The human detection systems of Patent Documents 1 and 2 detect people from images acquired by an imaging device attached to construction machinery, from the viewpoint of preventing accidents caused by contact between people and construction machinery at construction sites.

The human detection system of Patent Document 1 converts an image (thermal image) acquired by a monocular far-infrared camera attached to a construction machine such as a wheel loader into a bird's-eye view image, and detects the grounding position of a person present around the construction machine by image processing using a trained model (see, for example, paragraphs [0006] and [0010] of Patent Document 1). Here, in Patent Document 1, the human detection unit of the image processing unit uses an image (thermal image) sent from the far-infrared camera, and detects a person present around the construction machine using a temperature distribution unique to a person (see, for example, paragraph [0022] of Patent Document 1). However, the human detection system of Patent Document 1 can detect a person, but cannot distinguish whether the person is a person authorized to enter.

Patent Document 2 relates to a human detection system for construction machinery that detects people around the construction machinery from images acquired by an imaging device attached to the construction machinery. Here, the imaging device in Patent Document 2 is a visible light camera (color camera) including an imaging element such as a CCD (
For example, see paragraph [0013] of Patent Document 2.) Therefore, in the human detection system of Patent Document 2 that employs a visible light camera as an imaging device, the function (or accuracy) of detecting a human is significantly reduced, for example, at night or in bad weather.

Generally, the larger the image size to be analyzed, the longer it takes to process the data. In particular, it is common to use a computer with relatively low processing power and artificial intelligence (AI) to detect people.
When I) is used, it takes more time, so it is difficult to detect people in real time.

JP 2021-163401 A Patent No. 6434507

The present invention provides a human detection system and a human detection method capable of automatically detecting a human.

Specifically, according to the present invention,
a surveillance camera including at least a thermal camera for acquiring images of a surveillance area at predetermined time intervals;
A human detection system comprising: a video analysis device including a central processing unit that processes the image received from the thermal camera, and a video storage unit that stores the image received from the thermal camera, the central processing unit including a moving object detection unit and an AI determination unit for processing the image;
the moving object detection unit extracts three or more consecutive frames from the image using a frame difference method, calculates the absolute value of the difference between two consecutive frames, repeats this process for the next two frames, and calculates the logical product of the differences to extract an object that continues to move across the three or more frames;
The AI determination unit performs image analysis processing using artificial intelligence (AI) on the object to determine whether the object is a person, thereby providing a human detection system.

Here, a thermal camera detects the temperature of an object by detecting far-infrared rays. Images captured by a thermal camera are called thermographs, and usually high-temperature areas are displayed in red and low-temperature areas in blue. Thermal cameras can obtain thermographs because they capture far-infrared light emitted by objects even in the absence of ambient light. This makes it possible to make measurements even in dark places at night. In addition, the use of far-infrared rays makes it possible to take images in places where visibility is poor due to fog or smoke. Conversely, far-infrared rays are not affected by ambient light, so they can be used during the day as well.
The AI determination unit preferably includes a program having an algorithm for determining whether the object is a person. More preferably, the AI determination unit determines whether the object is a person based on a ratio of the height and width of the object.
Furthermore, it is preferable that the central processing unit further includes a marking unit connected to the AI determination unit, which executes a process of adding a mark indicating the area surrounding the person to the image determined to be the person.
Additionally, the surveillance camera may further comprise at least one visible light camera.
It is preferable that the central processing unit identifies the person in an image obtained by a black and white mode of a visible light camera by irradiating an invisible near-infrared light based on the coordinates of the person in the image detected by the thermal camera and the time of shooting, and determines whether or not light is emitted within a predetermined range surrounding the person in the image. Since an authorized person carries an infrared light, it is more preferable that the central processing unit determines that the person is authorized to enter the surveillance area when light emission is detected within the predetermined range.
In addition, it is preferable that the above-mentioned human detection system further includes a display device that displays the images stored in the video storage unit, or further includes at least one terminal that controls the video analysis device via the Internet.

The present invention also provides a human detection method using the above-mentioned human detection system.
in particular,
a surveillance camera including at least a thermal camera for acquiring images of a surveillance area at predetermined time intervals;
A human detection method in a human detection system comprising: a video analysis device including a central processing unit that processes the image received from the thermal camera; and a video storage unit that stores the image received from the thermal camera, the central processing unit including a moving object detection unit and an AI determination unit for processing the image,
the moving object detection unit extracts three or more consecutive frames from the image using a frame difference method, calculates the absolute value of the difference between two consecutive frames, repeats this process for the next two frames, and calculates the logical product of the differences, thereby extracting an object that continues to move across the three or more frames;
the AI determination unit having been repeatedly trained to determine whether or not the object is a person using a plurality of images including the object as learning data, executes an image analysis process using artificial intelligence (AI) on the object, and determines whether or not the object is a person.

Here, it is preferable that the AI determination unit includes a program having an algorithm for determining whether the object is a person. It is also preferable that the AI determination unit determines whether the object is a person based on a ratio of the height and width of the object.
Furthermore, it is preferable that the central processing unit further includes a step of causing a marking unit connected to the AI determination unit to execute a process of adding a mark indicating an area surrounding the person to the image determined to be the person.
In addition, it is preferable that the surveillance camera further comprises at least one visible light camera, and the central processing unit identifies the person in an image obtained by the black and white mode of the visible light camera based on the coordinates of the person in the image in which the person is detected by the thermal camera and the time of shooting, and determines whether or not light is emitted within a predetermined range surrounding the person in the image. Here, it is more preferable that, when light is detected within the predetermined range, the central processing unit determines that the person is authorized to enter the surveillance area.
In addition, it is preferable that the method further includes a step of displaying the images stored in the video storage unit on a display device, or that the method further includes a step of at least one terminal controlling the video analysis device via the Internet.

Furthermore, the present invention can store the image in the image storage unit, and the central processing unit can
There is also provided a computer readable program for causing a computer to execute any of the human detection methods described above.

According to the human detection system and human detection method of the present invention, it is possible to automatically detect humans in a monitored area while reducing the influence of the environment when the monitored area is photographed.
In addition, according to the human detection system and the human detection method of the present invention, a conventional CNN (Convolutional Neural Network) known as a machine learning algorithm is used.
The proposed method uses a simpler structure than the Real-World Network (VRN) model structure, which enables faster image analysis when detecting people.
Furthermore, according to the human detection system and human detection method of the present invention, it is possible to determine whether or not a person detected in a monitored area is a person authorized to enter.

1A is a schematic diagram showing an overall configuration of a human detection system 1 according to an embodiment of the present invention, and FIG. 1B is a functional block diagram of the human detection system 1 of A. FIG. 1 is a schematic diagram showing wavelength regions and main applications in each wavelength region. A is a color image taken by a visible light camera, and B is a thermal image taken by a thermal camera at the same time and location as A. 4 is a flowchart showing a process for automatically detecting a person in the human detection system 1 according to an embodiment of the present invention. 11 is an image showing a process of detecting a moving object by the moving object detection unit 202a. 11 is a flowchart showing a first filtering process executed in a moving object detecting section 202a. 13 is a flowchart showing a second filtering process executed in the moving object detection unit 202a and the AI determination unit 202b. 13 is a schematic diagram showing processing by a PositionFilter executed in a moving object detection unit 202a. FIG. Image A is an image obtained when the moving object detection unit 202a determines that the moving object detected by the PositionFilter in Fig. 7 is a non-human object (a car) because the size of the moving object is horizontal. Image B is an image obtained when the moving object detection unit 202a determines that the moving object detected by the PositionFilter in Fig. 7 is a human candidate based on its size. 8 is an image acquired during a predetermined filtering process of the second filtering process of FIG. 7. This is an AI learning model structure for detecting people in the AI determination unit 202b of the human detection system 1 according to one embodiment of the present invention. 1 is an image showing a test environment in which a person was actually detected using a human detection system 1 according to an embodiment of the present invention. 13 shows data indicating that a person was automatically detected by the human detection system 1 according to an embodiment of the present invention in the test environment of FIG. 12. 1 is a thermal image (obtained by a thermal camera) for determining whether a person detected by the human detection system 1 of the present invention is an authorized entrant. 1 is a black and white mode image (obtained in the black and white mode of the visible light camera) for determining whether a person detected by the human detection system 1 of the present invention is an authorized trespasser. A is an image that indicates whether a detected individual is an authorized entrant or a trespasser depending on whether an infrared light is found emitting within a predetermined range of the image, and B is an image showing an example of a device that can be used as an infrared light to determine authorized entrants. FIG. 1 is a diagram showing the structure of a typical CNN model, which is one of conventional machine learning algorithms.

A human detection system 1 according to an embodiment of the present invention will be described with reference to FIG.
For example, when monitoring illegal intrusions into a factory, the human detection system 1 can be installed in a location where an intruder enters or an object that may be stolen can be photographed. The human detection system 1 includes a surveillance camera 10 including at least one thermal camera that captures images of a surveillance area, and a video analysis device (
The image analyzing device 20 includes a video analysis server (or image analyzing server) 20, and a display device 30 that displays the results of analysis by the image analyzing device 20. The thermal camera detects far-infrared rays (having a wavelength of 3 to 10 nm) emitted from an object.
The image analysis device 20 is a device for detecting a distance of 0.000 micrometers. The image analysis device 20 includes an image storage unit 201 for storing images from the surveillance camera 10, analysis programs, and various data for analysis and results, and a central processing unit (CCU) for performing calculations such as image analysis processing by communicating with the image storage unit 201.
The central processing unit 202 includes a moving object detection unit 202a
The moving object detection unit 202a, the AI determination unit 202b, and the marking unit 202c are a central processing unit 2
The analyzed image is then sent to the video analysis device 2.
The image is displayed on a display device 30 connected to the surveillance camera 10 and the image analysis device 20.
For example, the surveillance camera 10 and the video analysis device 20 may be connected by a communication cable, wirelessly, or both wired and wirelessly. Here, the surveillance camera 10 and the video analysis device 20 may be connected by an Internet line, for example. Similarly, the video analysis device 20 and the display device 30 may be connected by a communication cable, wirelessly, or both wired and wirelessly. Here, the video analysis device 20 and the display device 30 may be connected by an Internet line, for example. In this way, the human detection system 1 adopts a simple configuration in which the surveillance camera 10 including at least one thermal camera is connected to one video analysis device 20.

Figure 2 shows typical wavelength ranges and their applications. In particular, in the infrared region (wavelength 0.78 to 1000 micrometers) shown in Figure 2, uncooled thermal cameras typically detect far infrared rays with wavelengths of 8 to 14 micrometers, while cooled thermal cameras typically detect mid infrared rays with wavelengths of 3 to 5 micrometers. For example, by using a thermal camera as the surveillance camera 10, images of the fishing grounds, which are the surveillance area, can be obtained with good contrast not only during the day but also at night.

Generally, a thermal camera detects heat and visualizes temperature. Images captured by a thermal camera are called thermography (thermal images), and usually, high temperature areas are displayed in red and low temperature areas in blue (in the thermal camera used in this application, high temperature areas are displayed in white and low temperature areas in black). A thermal camera is equipped with a sensor that detects far-infrared rays in order to measure the temperature of an object without contact. The strength of the far-infrared rays is detected to measure the temperature, and the image is colored according to the measured temperature using image processing to display a thermograph that makes it easy to visually grasp the temperature distribution. On the other hand, an infrared camera is intended to capture images in the dark. Infrared cameras are usually equipped with near-infrared LEDs that function as light sources, and capture the infrared rays that are irradiated onto the subject and reflected. Therefore, it is necessary to note that although thermal cameras and infrared cameras use the same infrared rays, they use completely different light bands and have different configurations and functions.

Fig. 3A shows a color image captured by a visible light camera when a certain monitored area is photographed at night. Fig. 3B shows a color image captured by a visible light camera when the same monitored area as Fig. 3A is photographed at the same night.
A thermal image obtained by a thermal camera is shown. Comparing FIG. 3A and FIG. 3B, it can be seen that FIG. 3B provides a clearer image. This is because a thermal camera receives far-infrared rays (having a wavelength of 3 to 1000 micrometers) emitted from objects in the monitoring area, and is less affected by the brightness or darkness of visible light in the monitoring area than a visible light camera. In addition, theft by illegal intrusion is generally more likely to occur at night, when it is less noticeable than during the day. Therefore, compared to a visible light camera, a thermal camera has the advantage of being able to obtain clear images, especially at night.
1, a case is described in which, for example, an uncooled thermal camera is used as the surveillance camera 10. However, the present invention is not limited to this, and for example, a cooled thermal camera can also be used as the surveillance camera 10.

Next, each process executed by the video analysis device 20 of the human detection system 1 will be described with reference to the flowchart of FIG.
In step S1, an image frame of a monitoring area captured by the monitoring camera 10 is stored in the video storage unit (hard disk) 201 of the video analysis device 20.
The surveillance camera 10 may also store video (video data) such as a series of images continuously captured by the surveillance camera 10 in the video storage unit 201 .
In step S2, the moving object detection unit 202a executes a process of determining whether or not an object in the video is a moving object. If the moving object detection unit 202a determines that the object in the video is a moving object, the process proceeds to step S3. On the other hand, if the moving object detection unit 202a determines that the object in the video is not a moving object, the process proceeds to step S4.
If it is determined that the image includes only stationary objects, the process proceeds to step S4. Then, in step S4, the process returns to the normal flow without detection (i.e., returns to step S1), and the process for that image ends. Note that the process in step S2 includes a MOGBacksubFilter 60, an ErodeFilter 61, and a DilateFilter 62 in FIG. 6, which will be described later, and the subsequent PositionFilter 70, a MotionObjectTracker 71, a CandidateFilter 72, and a MaskValidaFilter 73 in FIG. 7.
This corresponds to the processing in tor73.
In step S3, the AI determination unit 202b performs image analysis processing using AI on the moving object in the video according to a predetermined model structure (for the predetermined model structure, see FIG. 11 described later). If the AI determination unit 202b determines that the moving object in the video is a person, the process proceeds to step S5. Then, the AI determination unit 202b performs a process of attaching a mark to the part of the moving object in the video that is determined to be a person (for example, a marking process such as surrounding it with a rectangular frame).
On the other hand, if the AI determination unit 202b determines that the moving object in the video is other than a person, the process proceeds to step S4. Then, in step S4, the process returns to the normal flow without detection (i.e., returns to step S1), and the process for the video is terminated. The process in step S3 corresponds to the process in DnnFilter 74 in FIG. 7, which will be described later.

The process of detecting a moving object by the moving object detection unit 202a will be described with reference to Fig. 5. Here, the moving object detection unit 202a detects a moving object using a frame difference method. This can be performed using image processing software called OpenCV, which is publicly available. First, the moving object detection unit 202a detects three consecutive images from among the images acquired by the surveillance camera 10.
The moving object detection unit 202a extracts three consecutive frames I1, I2, and I3. Next, the three consecutive frames (images) are converted to grayscale.
1 and I2, and a difference image I2′ between successive frames I2 and I3.
Then, the moving object detection unit 202a calculates the absolute value of the difference image I1' and the difference image I
2' to obtain an image I1'' of the object that is in motion. By this process, the moving object detection unit 202a can extract an image of the object that is in motion over three frames. The above process in the moving object detection unit 202a is performed in the "MO
This corresponds to the processing performed by "GBacksubFilter60".

6 is a flowchart showing the first filtering process executed by the moving object detection unit 202a of the human detection system 1. Note that the functions MOGBacksubFilter and Position
Filter and the like are available in OpenCV, for example, version 4.6.0. First, the first filter process is a combination of MOGBacksubFilter 60 and ErodeFilter
The filter includes a MOGBacksubFilter 60, which performs detection using a background subtraction method, and a DilateFilter 61, which removes noise where pixels are not concentrated.
In the ateFilter 62, a collection of pixels is dilated and extracted.

7 is a flowchart showing a second filtering process executed in the moving object detection unit 202a and the AI determination unit 202b of the human detection system 1. The second filtering process includes a PositionFilter 70, a MotionObjectTracker 71, a CandidateFilter 72, and a MaskVali
The MotionObjectTrac includes a data filter 73 and a DnnFilter 74. Here, the PositionFilter 70 removes objects that are too small or too large, and deletes objects that are horizontally long.
In the CandidateFilter71, IDs are assigned to the same moving object in the previous and next frames. In the CandidateFilter72, objects that were invalid in the previous frame or objects with large size changes are removed.
In step 73, it is judged whether the object is inside or outside the masked area in the image. Then, in step 74, a convolutional neural network (
The accuracy is determined using AI using a Convolutional Neural Network (CNN).
Here, the moving object detection unit 202a includes a MOGBacksubFilter 60, an ErodeFilter 61, and a DilateFilter 62.
ter62, PositionFilter70, MotionObjectTracker71, CandidateFilter72, and Mas
The AI determination unit 202b executes processing with the kValidator 73. Then, the AI determination unit 202b executes processing with the DnnFilter 74.

8 is a schematic diagram showing the processing by the PositionFilter 70 executed in the moving object detection unit 202a. Here, in the processing by the PositionFilter 70, the ratio (H/W) of the height (H) to the width (W) of an object in each image where a moving object has been detected is calculated. Here, in the processing by the PositionFilter 70, it is possible to set in advance to exclude objects with a height (H) of 1 pixel or less from the filtering process (for example, to avoid detecting small objects such as birds and insects, or objects moving at a distance outside the monitoring area). Then, the PositionFilter 70
In the case of processing in the above, when H/W>1.1 (i.e., when the object in each image is vertically long),
The object is determined to be a candidate for a person.
If it is 15 (i.e., the object in each image is significantly longer than the rest), then the object is eliminated as a candidate for a person.

Next, referring to FIG. 9A and FIG. 9B, the Positive-Negative Transformation executed by the moving object detection unit 202a is performed.
The process in onFilter70 will be described. In FIG. 9A, the moving object detection unit 202a executes PositionFilter
The process at 70 determines the height (H _A ), width (W _A ), and height-to-width ratio (H _A /W A ) for the object. For the object in FIG. 9A, the height-to-width ratio (H _A /W _A ) is
W _A ) is smaller than 1 (H _A /W _A <1). Therefore, in the subsequent processing by DnnFilter 74 executed in the AI determination unit 202b, the object is determined to be something other than a person (a car).
On the other hand, in FIG. 9B, the moving object detection unit 202a obtains the height (H _B ), width (W _B ), and height-to-width ratio (H
9B, the height to width ratio ( _{H B} /W _{B )} is greater than 1.1 (H _B /W _B >1.1). Therefore, in the subsequent process by DnnFilter 74 executed in the AI determination unit 202b, the object is determined to be highly likely to be a person.

10 shows an image undergoing filtering by the PositionFilter 70 and the DnnFilter 74 in the second filtering process of FIG. 7. Here, the PositionFilter 70 shown in FIG.
In DnnFilter 70, the moving object detection unit 202a excludes moving objects of 1 pixel or less and moving objects of a predetermined size or more, and excludes moving objects with a height-to-width ratio (H/W) of less than 1.1. In DnnFilter 74 shown in Fig. 10, the AI determination unit 202b determines whether a moving object in an image that has already been collected is a person by using an AI model determined by the algorithm in Fig. 11 described later.

FIG. 11 shows the DnnFilter7 executed by the AI determination unit 202b of the human detection system 1.
It can be used to determine whether a moving object is a person in the process of 4.
The model structure of the machine learning algorithm is shown below. This model structure includes six convolution layers and three fully connected layers. Here, the convolution layer is a layer created by repeatedly sliding over the image to compress (convolute) a small region on the image into a single feature.
The convolutional layer allows feature extraction by region, not by point.
This layer receives features extracted by convolutional operations and is used to perform classification based on the features.
Here, the model structure in FIG. 11 uses a part of the thermal image that is determined to be a moving object by the moving object detection unit 202a, and therefore is different from the general C shown in FIG. 17, which uses the entire image.
This is a simpler structure than the NN model structure.
The AI determination unit 202b can perform image processing at a higher speed by using the model structure of Fig. 11 (and, as described later in Figs. 12 and 13, the AI determination unit 202b of the human detection system 1 can detect people with high accuracy). The configuration of the convolution layer changes depending on the size of the thermal image.
Next, a procedure for learning images of people in advance for the model structure of FIG. 11 used for judgment in the AI judgment unit 202b will be briefly described. First, shooting is performed at multiple locations, and 15,000 images including people are collected. Next, an AI judgment image can be created from the images including people using a video extraction tool. Here, the number of AI judgment images can be increased from 15,000 to 20,000 by performing processing such as adding noise or inverting the images. Then, the 20,000 AI judgment images obtained in this way are used to perform AI judgment.
The model structure of Fig. 11 used for the judgment in the judgment unit 202b can be learned in advance. Note that, for example, the model structure of Fig. 11 can be learned using 20,000 or more AI judgment images.

On the other hand, FIG. 17 shows one of the neural networks that has been used mainly for image recognition.
The structure of the CNN model, which is one of the neural networks, is shown below. A neural network is a mathematical model that mimics the neural circuits of the brain and returns an output for a certain input.
The N model generally includes a convolutional layer, a pooling layer, and a fully connected layer. Here, the convolutional layer is a layer created by repeatedly setting a small region on an image and compressing (convolving) it into one feature while sliding across the image. The convolutional layer makes it possible to extract features for each region, not for each point. The pooling layer is a layer created by setting a small region in the convolutional layer and performing operations on the data within that frame. In particular, the pooling layer makes it possible to absorb the positional shift of the convolutional layer that extracted features in the task of classifying image categories. The fully connected layer is a layer that receives the features extracted by the convolutional operation or pooling operation and is used to classify by the features. And, at the end of the fully connected layer, Soft
By adding a max layer, the sum of the output values for each class can be made 1, so that the output result can be treated as a probability.
For details of the conventional CNN model shown in FIG. 17, please refer to the following URL:
Please refer to tps://newtechnologylifestyle.net/vgg16originalpicture/ .

12 shows an example of an image obtained by the surveillance camera 10 (thermal camera) of the human detection system 1 according to an embodiment of the present invention. This image was taken in the early morning of March 11th (00:00-09:00).
The image was obtained at 1:00 a.m. with a part of the first parking lot in the Kanuma factory of Denki Kogyo Co., Ltd. as a monitored area. At this time, human detection system 1 was installed on the roof of a building about 100 meters away from the first parking lot, and detected a person in the first parking lot.

Next, referring to FIG. 13, the location of the image shown in FIG. 12 (first parking lot) was photographed in the morning (00:00-09:00
13, a screen displayed on the display device 30 of the human detection system 1 according to an embodiment of the present invention when a human is detected in the monitoring area (i.e., the area in the vicinity of the camera 10) is described below.
The image (shown in FIG. 12) of the monitoring area captured by the monitoring camera 10 and the date and time of capturing the image are automatically displayed on the display unit of the display device 30. Here, the display device 30 can display, for example, the number of detected moving objects, the time when the image of the moving object was captured (image time), whether or not a person was detected, whether or not a vehicle was detected, and whether or not an object of detection is unknown. Specifically, as shown in FIG. 13, the human detection system 1 according to one embodiment of the present invention monitored a part of the first parking lot shown in FIG. 12 in the morning (00:00-09:00), and automatically obtained the results that the number of detected people was 177, the number of undetected people was 6, the number of detected vehicles was 2, and the number of unknown objects of detection was 0 (note that the total number of people counted by visual inspection of the image was 183). Therefore, the human detection rate (number of detected people÷total number of people) of the human detection system 1 in this case was 177÷183≒96.7%. And, the human detection system 1 in this case was
The overall detection rate (number of people detected ÷ total number of people detected) is 177 ÷ (177 + 6 + 2 + 0) ≒ 95.7
%. Thus, according to the human detection system 1 according to one embodiment of the present invention,
%, we were able to obtain a good human detection rate and overall detection rate of over 100%. Here, the "number of undetected humans" is the number of humans that the human detection system 1 was unable to detect within the monitored area, and the "number of unknown detection targets" is the number of objects other than humans or vehicles that the human detection system 1 detected within the monitored area.
In addition, the entry "0:00 - none" in the remarks column of Fig. 13 means that there was no detection in the range of 0:00 - 1:00. Similarly, the entry "1:00 - none" in the remarks column of Fig. 13 means that there was no detection in the range of 1:00 - 2:00, and the entry "3:00 - none" in the remarks column of Fig. 13 means that there was no detection in the range of 3:00 -
This means that there was no detection in the 4 o'clock range.

In addition, in the human detection system 1 according to an embodiment of the present invention, an image of a person that has been subjected to a predetermined marking process can be displayed on the display device 30. Also, by continuously marking the image of a person that has been subjected to a predetermined marking process (for example, from frame-in to frame-out), the trajectory can be visually confirmed. Therefore, it can be easily confirmed on the display device 30 whether or not a person that has been subjected to such a marking process was present, for example, in the monitoring area at a predetermined shooting date and time (i.e., the detection date and time of the person).

14 is a thermal image in which a person determined by the AI determination unit 202b is marked by the marking unit 202c in a rectangular frame and displayed on the display device 30. The dashed lines in FIG. 14 are auxiliary lines for making the coordinates easier to understand (i.e., the dashed lines in FIG. 14 are auxiliary lines described for the purpose of explanation in this application, and are not markings by the marking unit 202c).
The color of the square frame surrounding the person determined by the marking process in c can be displayed in green or red (depending on whether the person is permitted to enter, for example). Next, FIG. 15 shows a person (
That is, it is an image (black and white mode image or infrared mode image) taken in black and white mode by the visible light camera, in which the same person, at the same time, and at the same place as in Figure 14) is marked by the marking unit 202c so as to be surrounded by a rectangular frame and displayed on the display device 30. Here, in black and white mode, if an amount of light (i.e., the amount of infrared light) exceeding a certain threshold is detected within the range of the rectangular frame set from the setting panel, it can be determined that the person surrounded by the rectangular frame is a person who is permitted to enter. The color of the rectangular frame surrounding the person can then be displayed as green. On the other hand, if an amount of light (i.e., the amount of infrared light) exceeding a certain threshold is not detected within the range of the rectangular frame set from the setting panel,
The person surrounded by the square frame can be determined to be an intruder, and the square frame surrounding that person can be displayed in red.
Here, the thermal image in Fig. 14 and the black-and-white mode image in Fig. 15 have the same resolution. Furthermore, the human detection system 1 records the coordinates at which the human is detected in the thermal image in Fig. 14. The human detection system 1 then determines whether a predetermined light source (i.e., infrared light emission) is present within the same coordinate range in the black-and-white mode image in Fig. 15. The human detection system 1 then determines that the human is permitted to enter the monitored area, since it has detected infrared light emission within the range enclosed by the dashed rectangular frame in Fig. 15.

16A shows, as an example, a color image (black and white mode or infrared mode) including a marking process in which a person detected by the marking unit 202c is surrounded by several rectangular frames. Since the emission of infrared light is detected within a certain rectangular frame in the image of FIG. 16A,
It can be seen that the person inside this square frame is someone who is authorized to enter the monitored area.
Since no infrared light is detected in another square frame in FIG. 16A, it is understood that the person within this square frame is highly likely to be an intruder.
Next, Fig. 16B shows an example of an infrared light that can be used to determine whether or not a person is authorized to enter a monitored area. For example, by lending an infrared light shown in Fig. 16B to a person authorized to enter a monitored area of the human detection system 1 according to an embodiment of the present invention, it is possible to distinguish between a person authorized to enter the monitored area and an intruder.

In addition, the above-mentioned human detection system 1 can store a computer-readable program including predetermined instructions for the human detection method in the video storage unit 201, and cause the central processing unit 202 to execute the predetermined instructions.

So far, we have used a human detection system 1 as an example to monitor trespassers in a factory.
However, it will be understood by those skilled in the art that the human detection system 1 can be used for other purposes other than the purpose of monitoring a factory. For example, it will be understood by those skilled in the art that the human detection system 1 can be used to monitor people passing through a place where only authorized persons are allowed to enter.

Although the present invention has been particularly described herein with reference to certain embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention, and that those skilled in the art will therefore recognize that numerous modifications can be made to the illustrative embodiments and that other configurations can be adopted without departing from the spirit and scope of the present invention as defined by the appended claims.

REFERENCE SIGNS LIST 1 Person detection system 10 Surveillance camera 20 Video analysis device 201 Video storage unit 202 Central processing unit 202a Moving object detection unit 202b AI determination unit 202c Marking unit 30 Display device

Claims (17)

a surveillance camera including at least a thermal camera for acquiring images of a surveillance area at predetermined time intervals;
A human detection system comprising: a video analysis device including a central processing unit that processes the image received from the thermal camera, and a video storage unit that stores the image received from the thermal camera, the central processing unit including a moving object detection unit and an AI determination unit for processing the image;
the moving object detection unit extracts three or more consecutive frames from the image using a frame difference method, calculates the absolute value of the difference between two consecutive frames, repeats this process for the next two frames, and calculates the logical product of the differences to extract an object that continues to move across the three or more frames;
The AI determination unit performs image analysis processing using artificial intelligence on the object to determine whether the object is a person. The human detection system according to claim 1 , wherein the AI determination unit includes a program having an algorithm for determining that the object is a human. The human detection system according to claim 1 , wherein the AI determination unit determines whether the object is a human based on a ratio of the height and width of the object. The human detection system according to any one of claims 1 to 3, wherein the central processing unit further includes a marking unit connected to the AI determination unit that executes a process of adding a mark indicating an area surrounding the person in the image determined to be the person. 2. The human detection system of claim 1, wherein the surveillance camera further comprises at least one visible light camera, and the central processing unit identifies the person in an image obtained in black and white mode of the visible light camera by irradiating the person with near-infrared light based on the coordinates of the person in the image detected by the thermal camera and the time the person was photographed, and determines whether or not light is emitted within a predetermined range surrounding the person in the image. The human detection system according to claim 5 , wherein the central processing unit, when detecting light emission within the predetermined range, determines that the human is a person permitted to enter the monitored area. The human detection system according to claim 1 , further comprising a display device that displays the image stored in the video storage unit. The human detection system according to claim 1 , further comprising at least one terminal that controls the video analysis device via the Internet. a surveillance camera including at least a thermal camera for acquiring images of a surveillance area at predetermined time intervals;
a video analysis device including a central processing unit that processes the image received from the thermal camera, and a video storage unit that stores the image received from the thermal camera, the central processing unit including a moving object detection unit and an AI determination unit for processing the image,
the moving object detection unit extracts three or more consecutive frames from the image using a frame difference method, calculates the absolute value of the difference between two consecutive frames, repeats this process for the next two frames, and calculates the logical product of the differences, thereby extracting an object that continues to move across the three or more frames;
a step in which the AI determination unit, which has been repeatedly trained to determine whether or not the object is a person using a plurality of images including the object as learning data, executes an image analysis process using artificial intelligence (AI) on the object and determines whether or not the object is a person. The human detection method according to claim 9 , wherein the AI determination unit includes a program having an algorithm for determining that the object is a human. The human detection system according to claim 9 , wherein the AI determination unit determines whether the object is a human based on a ratio of a height to a width of the object. The human detection method according to any one of claims 9 to 11, further comprising a step in which the central processing unit causes a marking unit connected to the AI determination unit to execute a process of adding a mark indicating an area surrounding the person to the image determined to be the person. The human detection method of claim 9, wherein the surveillance camera further comprises at least one visible light camera, and the central processing unit identifies the person in an image obtained in black and white mode of the visible light camera by shining near-infrared light on the basis of the coordinates of the person in the image detected by the thermal camera and the time the person was photographed, and determines whether or not light is emitted within a predetermined range surrounding the person in the image. The human detection method according to claim 13 , wherein the central processing unit determines, when detecting light emission within the predetermined range, that the human is a person permitted to enter the monitored area. The human detection method according to claim 9 , further comprising the step of displaying the image stored in the video storage unit on a display device. The human detection method according to claim 9 , further comprising the step of: at least one terminal controlling the video analysis device via the Internet. A computer-readable program that can be stored in the video storage unit and causes the central processing unit to execute the human detection method according to claim 9.