KR20230062175A - Closed circuit television system for detecting fallen people in industrial sites and operation method thereof - Google Patents

Abstract

According to one embodiment of the present invention, provided are a surveillance camera system for detecting falls at industrial sites and an operating method thereof. The surveillance camera system includes: a camera module which captures an industrial site and generates an optical image; and a server which analyzes the optical image received from the camera module and determines whether a fallen person exists. If a person falls, the abnormal event of a person is notified to a manager, so as to prevent accidents in advance or quickly respond to accidents that occur.

Description

Closed circuit television system for detecting fallen people in industrial sites and operation method thereof}

The present invention relates to a surveillance camera system for detecting a fall in an industrial site and a method for operating the same.

Economic loss and human loss are occurring due to accidents at industrial sites. Recently, with the development of information and communication technology, there are attempts to automatically detect accidents in industrial settings. Conventional industrial site safety management technology analyzes data collected from sensors installed in facilities to detect accidents, and there is a problem in that available data is limited and the range of use is narrow.

On the other hand, conventional surveillance camera systems (CCTV) used in industrial sites have been used for simple purposes such as checking access through video recording and storage. Various accidents such as fire, explosion, leakage of toxic gas, high temperature, and smoke may occur in industrial sites. In order to be able to detect abnormalities in industrial sites and deal with them in advance before accidents occur, technology for detecting abnormalities in industrial sites is required.

KR 10-2019-0072703 A

An object according to an embodiment of the present invention is an industrial site fall detection surveillance camera that analyzes an image received from a camera photographing an industrial site, determines that an abnormal event has occurred if there is a person who has fallen, and transmits a notification to a manager. A system and its operating method are provided.

A surveillance camera system for detecting a fall at an industrial site according to an embodiment of the present invention includes a camera module that photographs an industrial site and generates an optical image, and analyzes the optical image received from the camera module to determine whether a fallen person exists. may include a server that

In addition, the server includes a storage unit for storing the optical image received from the camera module, analyzing the optical image to determine whether a person is in a fallen state, and outputting an abnormal person event when it is determined that the person has fallen. A judgment unit and a notification unit notifying a manager of the occurrence of the abnormal person event when the abnormal person determination unit outputs the occurrence of the abnormal person event.

In addition, the person abnormality determination unit extracts a frame from the optical image and inputs the frame to a fall recognition artificial intelligence model, and when the fall recognition artificial intelligence model determines that a person who has fallen in the frame exists, the person It may be determined that an abnormal event has occurred.

In addition, the fall recognition artificial intelligence model is learned using learning data in which an image of a person falling down is input data and information indicating a person who has fallen in the image is label data, and when a frame extracted from the optical image is input, the person It is possible to determine whether the person is in a collapsed state, display a box in the area where the person fell, and display and output the class matching score.

In addition, the fall recognition artificial intelligence model may be formed by adjusting network weights of a trained CNN model to determine a person's fall by applying deep transfer learning for the purpose of classifying people, vehicles, and animals.

In addition, a plurality of camera modules may be disposed at the industrial site so that at least a part of an effective area that varies depending on a collapsed posture and a positional relationship with the camera module in the surveillance area is continuous.

In addition, the person abnormality determining unit may determine that a person has fallen when a class concordance score output by the fall recognition artificial intelligence model is higher than a reference score.

In addition, since the area of the effective region of the camera module becomes narrow when the reference score increases, the direction the camera module faces changes so that at least a part of the effective region continues when the reference score increases, and the reference score increases. When the reference score is lowered, since at least a part of the valid region is continuous, the direction in which the camera module is directed may be changed so as to monitor an area as wide as possible.

According to an embodiment of the present invention, a method of operating a surveillance camera system for detecting a fall at an industrial site includes an image generation step in which a camera module photographs an industrial site, generates an optical image, and transmits the optical image to a server, A person abnormality determination step of extracting a frame from an optical image and inputting the frame to a fall recognition artificial intelligence model to determine that an abnormal person event has occurred if there is a person who has fallen, and outputting the occurrence of an abnormal person event in the abnormal person judgment unit, The notification unit of the server may include a notification step of notifying an administrator of the occurrence of an abnormal event.

In addition, an image of a person falling down is input data and information indicating a person falling down in the image is learned using a learning data set as label data, and when a frame extracted from the optical image is received, it is determined whether the person is in a fallen state The method may further include a model generation step of generating the fall recognition artificial intelligence model so that a box is displayed in the area where the person fell and a class matching score is displayed and output.

Also, the model generation step may be performed in advance before the human abnormality determination step.

In addition, in the method of operating the surveillance camera system for detecting a fall in an industrial site according to an embodiment of the present invention, when the reference score of the class matching score, which is the criterion for determining that a person has fallen, increases, the area of the effective region narrows. When the reference score is high, the direction in which the camera module faces is changed so that at least a part of the valid area is continuous, and when the reference score is low, the area of the valid area is widened. It may further include a reference score changing step in which the direction of the camera module is controlled to change so as to monitor a wide area as possible while continuing at least a part of the .

Features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

Prior to this, the terms or words used in this specification and claims should not be interpreted in a conventional and dictionary sense, and the inventor may appropriately define the concept of the term in order to explain his or her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is.

According to one embodiment of the present invention, it is determined that an abnormal event has occurred when there is a person who has fallen at an industrial site, and automatically notifies a manager of the occurrence of an abnormal event to support prompt response to the accident.

1 is a diagram showing a fall detection monitoring camera system and an industrial site in an industrial site according to an embodiment of the present invention.
2 is a block diagram showing a surveillance camera system for detecting a fall at an industrial site according to an embodiment of the present invention.
3 is a flowchart illustrating an operating method of a fall detection monitoring camera system at an industrial site according to an embodiment of the present invention.
4 is a diagram illustrating a fall detection artificial intelligence model of an industrial site fall detection surveillance camera system according to an embodiment of the present invention.
5 is a diagram showing a relationship between a camera module and a collapsed posture of an industrial site fall detection surveillance camera system according to an embodiment of the present invention.
6 is a diagram showing a fall recognition accuracy distribution according to posture 1;
7 is a diagram showing a fall recognition accuracy distribution according to posture 2;
8 is a diagram illustrating a state in which a plurality of camera modules are disposed in consideration of a fall recognition accuracy distribution and a step of changing a reference score.

Objects, specific advantages and novel features of an embodiment of the present invention will become more apparent from the following detailed description and preferred embodiments taken in conjunction with the accompanying drawings. In adding reference numerals to components of each drawing in this specification, it should be noted that the same components have the same numbers as much as possible, even if they are displayed on different drawings. In addition, terms such as "one side", "other side", "first", and "second" are used to distinguish one component from another, and the components are not limited by the above terms. no. Hereinafter, in describing an embodiment of the present invention, a detailed description of related known technologies that may unnecessarily obscure the subject matter of an embodiment of the present invention will be omitted.

In addition, it should be noted that the terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention, and singular expressions include plural expressions unless otherwise specified in context.

Hereinafter, with reference to the accompanying drawings, an embodiment of the present invention will be described in detail.

1 is a diagram showing a fall detection monitoring camera system 10 and an industrial site 1 according to an embodiment of the present invention.

The industrial site (1) may include various indoor facilities such as factories, research institutes, laboratories, chemical plants, oil refineries, automation facilities, and smart factories, as well as outdoor facilities such as ports, ports, logistics hubs, and loading docks. In the industrial site (1), various facilities (2) such as motors, heaters, air conditioners, pipes, distillers, etc. may be installed. In the industrial site (1), there may be various people (3) such as workers, managers, or visitors who work using the facility (2). An accident that occurs at an industrial site (1) may injure a person (3) or damage a facility (2). The surveillance camera system 10 for detecting falls in an industrial site according to an embodiment of the present invention prevents an accident that may occur in an industrial site 1 in advance, or in order to quickly cope with an accident when it occurs, a person 3 occurs It can detect abnormal events that occur and notify the manager of abnormal events. The manager may include a safety manager, a worker, and the like of the industrial site (1), and may include public officials such as a fire station, an ambulance, and a district office.

The abnormal event may include an abnormal human event. The abnormal event may include various accidents such as an abnormality in a facility or a fire. An abnormal person event may include a person 3 being knocked down. For example, when the person (3) falls down due to an accident such as a leak of toxic gas or electric shock, or when the person (3) falls down due to a disease such as a heart attack or stroke, and the person (3) is lying on the floor An abnormal event has occurred. According to an embodiment of the present invention, it is possible to detect whether or not a person has fallen due to a person abnormal event occurring due to various causes.

2 is a block diagram showing a fall detection surveillance camera system 10 in an industrial site according to an embodiment of the present invention. See Figures 1 and 2 together.

The industrial site fall detection surveillance camera system 10 according to an embodiment of the present invention includes a camera module 100 for generating an optical image by photographing an industrial site 1, and an optical image received from the camera module 100. It may include a server 200 that analyzes the video and determines whether the person 3 who has fallen is present.

The camera module 100 may be disposed in the industrial site 1 to photograph a space where facilities 2 or people 3 exist. A plurality of camera modules 100 may be disposed in the industrial site 1 . The camera module 100 may change a photographing direction according to a control signal received from the server 200 . The camera module 100 may include one or more optical cameras 110 . The camera module 100 may include an optical camera 110 and a thermal imaging camera 120 together in one module.

The optical camera 110 may include a general digital camera of the RGB method. The optical camera 110 may generate an optical image recognizable by the eyes of a person 3 by photographing the industrial site 1 in the visible ray region. The thermal imaging camera 120 may include a digital camera that captures an infrared region. The thermal imaging camera 120 may capture the industrial site 1 in the infrared region and generate a grayscale thermal image.

Server 200 may be installed in the industrial site (1). The server 200 may be connected to the camera module 100 through a wired or wireless network, receive an image generated by the camera module 100, and control the camera module 100. The server 200 is a computer device having an information processing function. The camera module 100 and the server 200 may be connected through a closed circuit or through an encrypted network using an IP address.

The server 200 analyzes the storage unit 210 for storing the optical image and thermal image received from the camera module 100 and the optical image to determine whether the person 3 is in a fallen state, and determines that the person 3 has fallen down. If it is determined, the abnormal person determination unit 240 outputs the occurrence of the abnormal human event, and if the abnormal human determination unit 240 outputs the occurrence of the abnormal human event, the notification unit 250 notifies the manager of the occurrence of the abnormal event can include The server 200 may further include a communication unit 220 connected to a wired or wireless network to transmit and receive data, and an interface unit 230 to receive a manager's command or provide data or information to the manager.

The storage unit 210 includes an optical image, a thermal image, a learned fall recognition artificial intelligence model 241, and an operating method of the industrial site fall detection surveillance camera system 10 according to an embodiment of the present invention implemented in software. Program codes and other necessary data can be stored. The storage unit 210 may include a hard disk, a memory, a cloud storage device, a database, and the like.

The communication unit 220 may be connected to a wired or wireless network to transmit and receive data between the camera module 100 , the manager terminal 300 , and the server 200 . The communication unit 220 may use known communication methods such as the World Wide Web (www), Ethernet, IPv4, IPv6, LAN, and WAN. The communication unit 220 may use a known short-range communication method such as wi-fi, bluetooth, zigbee, or the like.

The interface unit 230 may include input devices such as a keyboard, mouse, touch panel, and switch capable of receiving commands from a manager, and output devices such as a display, speaker, and printer capable of providing information to the manager. can include

The human anomaly determination unit 240 and the notification unit 250 may be written in program code and implemented in a manner that runs on the processor or graphics processing unit (GPU) of the server 200, or may be implemented within the server 200. It may be implemented as an independent computer device included.

The abnormal person determination unit 240 may determine whether the person 3 is in a fallen state by analyzing an optical image received from the camera module 100 in real time or an optical image stored in the storage unit 210 in real time. The abnormal person determination unit 240 extracts a frame from the optical image and inputs the frame to the fall recognition artificial intelligence model 241, so that the fall recognition artificial intelligence model 241 determines that the fallen person 3 exists in the frame. In the case of determination, it may be determined that an abnormal event has occurred to the person 3 .

The human abnormality determination unit 240 may include a fall recognition artificial intelligence model 241 . The fall recognition artificial intelligence model 241 is learned using learning data in which an image of a person 3 falling down is input data and information indicating the person 3 falling in the image is label data, and a frame extracted from an optical image. When is input, it is determined whether the person 3 is in a collapsed state, a box is displayed in the area where the person 3 has fallen, and a class matching score is displayed and output.

The class consistency score refers to the extent to which an object included in a box inferred and displayed by the fall recognition artificial intelligence model 241 corresponds to the fallen person (3). The class consistency score is one of the output values of the fall recognition artificial intelligence model 241 . The abnormal person determination unit 240 may determine that the person 3 has fallen when the class matching score output by the fall recognition artificial intelligence model 241 is higher than the reference score. The reference score can be changed by the manager.

The fall recognition artificial intelligence model 241 may display a box for each person 3 and display a class matching score when it is determined that a plurality of people 3 are in a collapsed state in one frame.

When the occurrence of an abnormal human event is output from the abnormal human determination unit 240, the notification unit 250 may notify the administrator of the occurrence of the abnormal event and notify the contents of the abnormal event. The contents of the abnormal event include the location of the camera module 100 that captured the abnormal event, the occurrence time of the abnormal event, the frame of the optical image in which the abnormal event occurred, guidelines for responding to the abnormal event, and other necessary information. can do. The notification unit 250 may automatically transmit a phone call, text message, e-mail, etc. to the administrator terminal 300, display the occurrence of an abnormal event on a display, or notify the occurrence of an abnormal event through a speaker.

When a person abnormality event occurs, the notification unit 250 may provide a manager with an image in which a box in which the fall recognition artificial intelligence model 241 displays the fallen person 3 is displayed in a frame of an optical image. For example, the notification unit 250 may automatically generate an email including an image of the fallen person 3 as a box and send it to the manager's email address.

The notification unit 250 may include the number of people 3 who have fallen in the notification provided to the manager. When the fall recognition artificial intelligence model 241 determines that a plurality of people 3 have fallen in one frame, the notification unit 250 may provide the manager with the number of people 3 who have fallen down.

3 is a flowchart illustrating an operating method of the fall detection monitoring camera system 10 at an industrial site according to an embodiment of the present invention. See Figures 2 and 3 together.

In the method of operating the surveillance camera system 10 for detecting a fall in an industrial site according to an embodiment of the present invention, the optical camera 110 of the camera module 100 photographs the industrial site 1 to generate an optical image, and the server 200, the thermal image camera 120 of the camera module 100 photographs the industrial site 1, generates a thermal image, and transmits the thermal image to the server 200 (S21), the server ( 200), in which the person abnormality determination unit 240 extracts frames from the optical image and inputs them to the fall recognition artificial intelligence model 241, and determines that a person abnormality event has occurred if there is a person who has fallen (3). S22), and a notification step (S24) in which the notification unit 250 of the server 200 notifies the manager of the occurrence of the abnormal person event when the abnormal person determination unit 240 outputs the occurrence of the abnormal person event. there is.

The image generation step (S21) and the human abnormality determination step (S22) may be included in the monitoring step (S20). The monitoring step (S20) is repeatedly performed in real time while the operation method of the industrial site fall detection monitoring camera system 10 according to an embodiment of the present invention is being performed. The notification unit 250 determines whether an abnormal event is output from the abnormal person determination unit 240 (S23), and performs a notification step (S24) when an abnormal event occurs.

The operation method of the surveillance camera system 10 for detecting falls at an industrial site according to an embodiment of the present invention may further include a model generation step (S11) of generating a fall recognition artificial intelligence model 241 using learning data. The reference score change step in which the manager changes the reference score that is the criterion for determining whether the person 3 is down in the person abnormality determination unit 240 and controls the direction of the camera module 100 accordingly ( S12) may be further included. The model generation step (S11) and the reference score change step (S12) may be included in the preparation step (S10). The preparation step (S10) is a process of setting the fall detection monitoring camera system 10 of the industrial site to perform the monitoring step (S20). The preparation step (S10) may be carried out before the monitoring step (S20). While the preparation step (S10) is being performed, the operation of the monitoring step (S20) may be stopped. The preparation step (S10) and the monitoring step (S10) may be written as a program code that can be driven in the processor of the server and stored in the storage unit.

The model generation step (S11) may be performed in advance before the human abnormality determination step (S22). That is, the model generation step (S11) may be performed before the fall detection monitoring camera system 10 of the industrial site is installed in the industrial site 1. The model generation step (S11) may be performed regularly or irregularly to improve the performance of the fall recognition artificial intelligence model 241 even after the fall detection surveillance camera system 10 of the industrial site is installed in the industrial site 1. . When the model generation step (S11) is performed to generate the fall recognition artificial intelligence model 241 with improved performance, the existing fall recognition artificial intelligence model 241 stored in the server 200 may be updated.

The reference score change step (S12) may be performed according to a manager's judgment or an update of the fall recognition artificial intelligence model 241. If the reference score is changed, the width of the effective area (EA) capable of determining whether the person 3 falls in the area monitored by the camera module 100 is changed, so the person abnormality determination unit 240 determines whether the person 3 has fallen down. It is possible to change the direction of the camera module 100 so that no blind spot is not recognized.

4 is a diagram illustrating a fall recognition artificial intelligence model 241 of the fall detection surveillance camera system 10 at an industrial site according to an embodiment of the present invention. See Figures 2 and 3 together.

In the model generation step (S11), a fall recognition artificial intelligence model 241 is generated. The model generation step (S11) may be performed in the server 200, or may be performed in another computer device remotely connected to the server 200.

The image used as learning data is an image of a state in which the person 3 is down. In the image acquisition step, images to be used as learning data are collected. Images for training data can be acquired through various paths. For example, public images may be collected using web crawling. Alternatively, an image in which the person 3 is in a fallen state may be selected from among images included in a dataset collected for artificial intelligence learning (eg, Fallen Person Dataset, Human Pose Dataset, etc.).

In the training data generation step, input data and label data are generated by pre-processing images for training data. Input data is created by adjusting the size of the image for training data. For example, the image for training data may be pre-processed to be processed into an image of an RGB method (3 channels) having a width of 608 pixels and a height of 608 pixels. The label data may be generated as a character string including a position of a box displaying the form in which the person 3 has collapsed in the input data and a class name. For example, in the label data, the class name is "Fall", and the box location indicating the state in which the person (3) has fallen is displayed as numbers (0.531, 0.529, 0.877, 0.286). can The location of the box may be a character string in which the coordinates of the center of the box in the width direction (0.531), the coordinates of the center of the box in the height direction (0.529), the width of the box (0.877), and the height of the box (0.286) are sequentially arranged as numbers. One input data image and one label data string may be generated by pre-processing one training data image. In the above manner, a plurality of label data strings corresponding to a plurality of input data images may be generated.

When the training data is generated, batch division is performed to divide the plurality of input data and label data included in the training data into a plurality of batches. In the batch division step, the learning speed can be improved by dividing the batch size by 64 and the mini-batch by 64.

After dividing the training data into a plurality of batches, the model training step is performed. In the model training stage, deep transfer learning can be used. The network configuration file has a network structure of 604x604 and 708x708 sizes. The fall recognition artificial intelligence model 241 is formed by adjusting the network weights of the trained CNN model to determine the fall of the person 3 by applying deep transfer learning for the purpose of distinguishing people 3, vehicles, and animals. can The network structure uses a convolutional neural network (CNN) structure used for classifying various objects, such as people 3, vehicles, and animals. Existing network weights are weights pre-learned in a CNN structure in order to classify people (3), vehicles, animals, and the like. After setting the CNN network structure with the existing network weights, additional learning is performed using the divided training data to fine-tune the weights to generate collapse weights. The fall weight is a weight generated by fine-tuning an existing network weight as a result of learning using learning data including a state in which the person 3 has fallen. In the fine-tuning step, the 604x604 network structure is set to the initial value of the learning rate of 0.01% and the number of training iterations is 6000, and learning is performed. And, the 708x708 network structure was set to an initial value of 0.01% of the learning rate and 6000 training iterations, and learning was performed to generate three learned fall recognition artificial intelligence models (241).

A fall recognition artificial intelligence model 241 is generated through learning, and then a performance test step is performed. In the performance test step, how well the fall recognition artificial intelligence model 241 judges the state in which the person 3 has fallen is evaluated. The performance test is performed by calculating loss, IoU, mAP, and class concordance score, and generating an accuracy distribution.

Loss is calculated according to Equation 1 below.

y _i : label value of the prediction model, t _i : label value of the actual model, i: training data number,

N: number of training data

The performance of the fall recognition artificial intelligence model 241 in a state in which the loss is minimized by calculating the loss is determined to be the best.

Intersection over Union (IoU) is calculated by Equation 2 below.

The box of the label data is a box indicating the part where the person (3) fell down in the input data image of the learning data, and the predicted box is the part where the person (3) fell after the fall recognition AI received the input data image of the training data. It is a box indicating the area determined to be . When the two boxes completely overlap, the IoU becomes 1, and in an embodiment of the present invention, the case where the IoU is 0.5 is used as a detection index.

mAP (mean average precision) is a method of generating a confusion matrix by comparing actual data and predicted results, obtaining an AP, and evaluating the model as having higher accuracy as the AP increases. The error matrix is calculated according to Table 1 below.

real data
(Ground Truth) prediction result Positive (target object) Negative (non-target object) Positive
(target object) TP(True Positive)
(Detect target object correctly) FN(False Negative)
(Target object not detected) Negative
(non-target object) FP(False Positive)
(Detecting non-target objects) TN (True Negative)
(Untargeted objects are not detected)

Precision refers to the ratio of correctly detected results out of the total detection results, and is calculated as Precision=TP/(TP+FP), and Recall refers to the ratio of predicted correct results among actual correct results, and Recall=TP/ It is calculated as (TP+FN). AP is obtained as the area under the graph where the vertical is the precision and the horizontal is the recall. The class confidence score is a value indicating how accurately the fall recognition artificial intelligence model 241 recognized the person 3 who fell down. The class concordance score is calculated by Equation 3 below.

The class concordance score is expressed as the product of conditional class probability and box confidence. The conditional class probability indicates the probability that the current object is class _i , and the box confidence means the similarity between the location of the box in the label data and the location of the box predicted by the fall recognition artificial intelligence model 241. Therefore, the class concordance score is a numerical value that reflects both the accuracy of whether the class name is correctly classified and how accurately the box is predicted.

The model trained with the 608x608 network structure has relatively faster average frame processing speed than the model trained with the 704x704 network structure. As the size of the network structure increases, the time required for learning and frame processing increases. However, the model trained with the 704x704 network structure has a higher class concordance score than the model trained with the 608x608 network structure. Although it has the same 704x704 network structure, the model with 20000 training iterations had a lower loss and the highest class concordance score of 98% or more than the model with 6000 training iterations. Therefore, the fall recognition artificial intelligence model 241 according to an embodiment of the present invention uses a model having a 704x704 network structure and 20000 training iterations.

The accuracy distribution depends on the relationship between the camera module and the person 3's fallen position and the camera module and the person 3's position when the fall recognition artificial intelligence model 241 determines that the person 3 has fallen. It is a graph showing that the accuracy varies according to the

5 is a diagram showing the relationship between the camera module 100 and the fallen posture of the fall detection surveillance camera system 10 at an industrial site according to an embodiment of the present invention.

Posture 1 (Form 1) is a posture in which the head and feet of the person 3 are directed in the direction (arrow B) perpendicular to the direction the camera shoots (arrow A), and pose 2 (Form 2) is the direction the camera shoots (arrow B). This is a posture in which the head and feet of the person 3 are directed in a direction parallel to arrow A) (arrow C). The posture in which the person 3 falls is various. The posture captured in the optical image taken by the camera module 100 is different according to the posture in which the person 3 falls down. Accordingly, the fall recognition artificial intelligence model 241 may have different class concordance scores for recognizing a fall according to the posture in which the person 3 has fallen.

5 shows that the person 3 who fell down is located in the central part of the monitoring area (MA) photographed by the camera module 100, but the person 3 who fell down at the edge of the monitoring area MA may be located. When the fallen person 3 is located at the edge of the monitoring area MA, there may be a problem such as not all of the person 3 being captured in the optical image. In this case, a problem may occur in which the fall recognition artificial intelligence model 241 cannot recognize the person 3 who has fallen. Therefore, the accuracy of falling recognition may vary depending on the location and direction of the camera module 100 and the fallen person 3 .

6 is a diagram showing a fall recognition accuracy distribution according to posture 1 (Form 1). 7 is a diagram showing a fall recognition accuracy distribution according to posture 2 (Form 2). 6 and 7, the accuracy distribution is displayed using lines and shading at 10% intervals from 20% to 90% based on the class concordance score. The camera module 100 was installed at a height of 1.8 m from the ground, and optical images were taken to test accuracy distribution. The surveillance area (MA) photographed by the camera module 100 was tested with a width of 16 m and a length of 12 m.

If the reference score is set to 90% in the fall recognition accuracy distribution according to the posture 1 (Form 1) of FIG. From a position approximately 4m away from the camera to a position 12m away from the end of the monitoring area (MA), up to a position 6m left and right around the camera in a direction perpendicular to the direction in which the camera shoots, and as a whole, the width of the side farther away from the camera appears as a broad trapezoid.

If the reference score is set to 90% in the fall recognition accuracy distribution according to the posture 2 (Form 2) of FIG. From a position about 5m away to a position 10m away, in a direction perpendicular to the direction the camera shoots, up to a position of 5m left and right with the camera as the center, and as a whole, the side closest to the camera is a fan shape centered on the camera, and the side in the far direction is It appears as a trapezoid with a wide area.

In summary, it can be seen that the fall recognition accuracy varies according to the posture and position of the camera module 100 and the person who fell down 3 . In the case of Form 1, where the head and feet of the person 3 are directed in a direction perpendicular to the direction in which the camera module 100 is photographed, the accuracy of fall recognition is high, and the camera module 100 is in a direction parallel to the photographed direction. In the case of the posture 2 (Form 2) in which the head and feet of the person (3) are directed, the fall recognition accuracy is low. And, compared to posture 1 (Form 1) in which the head and feet of the person 3 are directed in a direction perpendicular to the direction in which the camera module 100 is photographed, the person 3 in a direction parallel to the direction in which the camera module 100 photographs ), a semicircular blind spot exists in a portion close to the camera module 100, and the shape of the effective area EA is irregular as a whole. Also, as the fall recognition accuracy increases, the area occupied by the effective area EA in the monitoring area MA decreases, and as the fall recognition accuracy decreases, the area occupied by the effective area EA in the monitoring area MA increases.

Therefore, the camera module 100 of the surveillance camera system 10 for detecting a fall in an industrial site according to an embodiment of the present invention has an effective area that varies depending on the positional relationship between the falling posture and the camera module 100 in the surveillance area MA. A plurality of EAs may be arranged in the industrial site 1 such that at least a portion thereof is continuous. A plurality of camera modules 100 should be disposed at the industrial site 1 so that at least a portion of the effective area EA is continuous, so that a person who has fallen down 3 can be recognized without a blind spot. When the monitoring area MA of the camera module 100 is arranged so that it is continuous, the fall recognition accuracy is increased between the effective area EA of an arbitrary camera module 100 and the effective area EA of an adjacent camera module 100. Since a very low area exists, it is difficult to effectively monitor the industrial site (1). As in one embodiment of the present invention, if the camera module 100 is arranged so that at least a part of the effective area EA of adjacent camera modules 100 overlaps or continues, the person 3 who fell even at the boundary of the effective area EA can be effectively recognized.

8 is a diagram illustrating a state in which a plurality of camera modules 100 are disposed in consideration of a fall recognition accuracy distribution and a reference score change step (S12). FIG. 8 illustratively shows a state in which the industrial site 1 is a square and four camera modules 100 are disposed. Depending on the shape of the industrial site 1 and the arrangement of the facilities 2, the camera module 100 may be arranged in various ways.

In FIG. 8, (a) is a diagram showing the monitoring area MA and effective area EA of the camera module 100 when the reference score is 70%. The direction of the camera module 100 is fixed so that the effective areas EA of adjacent camera modules 100 partially overlap.

When the manager changes the reference score in the reference score change step (S12), the area of the effective area EA is changed. In FIG. 8 (b) is a diagram showing the monitoring area MA and effective area EA of the camera module 100 when the reference score is changed from 70% to 90%. The reference score is raised from 70% to 90%, the area of the effective area EA is reduced, and the effective areas EA of adjacent camera modules 100 do not overlap. Therefore, the human abnormality determination unit 240 of the server 200 controls the camera module 100 to change the photographing direction of the camera module 100 so that the effective areas EA of the camera module 100 partially overlap.

In FIG. 8, (c) shows a state in which the direction of the camera module 100 is changed so that a part of the effective area EA overlaps with a reference score of 90%. The direction of the camera module 100 is moved so that a part of the effective area EA overlaps. The positions of the surveillance areas MA of the camera module 100 are moved so that they overlap each other more. The camera module 100 may include a driving unit including a motor and the like to change the shooting direction. In the reference score change step (S12), when the manager performs a change to increase the reference score, as shown in (a), (b) and (c) of FIG. 8 in order, a portion of the reduced effective area overlaps the camera. By moving the shooting direction of the module 100, blind spots can be minimized.

In FIG. 8, (d) shows a state in which the reference score is changed from 90% to 70%. When the reference score decreases, the area of the effective area EA increases, and the effective areas EA of adjacent camera modules 100 overlap each other a lot. In order for the camera module 100 to monitor the widest possible area, the human abnormality determination unit 240 of the server 200 overlaps some of the effective areas EA of the adjacent camera modules 100 with a little more camera modules ( 100) can be changed.

When the manager performs a change to lower the reference score in the reference score change step (S12), as shown in (c), (d) and (a) of FIG. 8 in order, a part of the increased effective area overlaps The surveillance area MA may be maximized while minimizing blind spots by moving the photographing direction of the camera module 100 so as to monitor a wide area as possible.

In summary, since the area of the effective area EA of the camera module 100 becomes narrow when the reference score increases, the direction in which the camera module 100 faces is such that at least a part of the effective area EA continues when the reference score increases. change, and when the reference score is lowered, the area of the effective area (EA) widens. When the reference score is lowered, at least a part of the effective area (EA) continues and the camera module 100 is directed so as to monitor as wide an area as possible. It can be controlled to change direction. This control is performed in the reference score change step (S12). The reference score change step (S12) is performed by the person abnormality determination unit 240, and when the reference score of the class matching score, which is the criterion for determining that the person 3 has fallen, increases, the area of the effective area EA narrows. When the reference score increases, the direction in which the camera module 100 faces is changed so that at least a portion of the effective area EA continues, and when the reference score decreases, the area of the effective area EA widens, so that the reference score decreases. In this case, the direction of the camera module 100 is controlled to change so that at least a part of the effective area EA is continuous and monitors a wide area as possible.

Although the present invention has been described in detail through specific examples, this is for explaining the present invention in detail, the present invention is not limited thereto, and within the technical spirit of the present invention, by those skilled in the art It will be clear that the modification or improvement is possible.

All simple modifications or changes of the present invention fall within the scope of the present invention, and the specific protection scope of the present invention will be clarified by the appended claims.

1: Industrial site
2: Facility
3: people
10: Industrial site fall detection monitoring camera system
100: camera module
110: optical camera
120: thermal imaging camera
200: server
210: storage unit
220: communication department
230: interface unit
240: abnormal human judgment unit
241: Fall Recognition AI Model
250: notification unit
300: manager terminal
MA: Monitoring Area
EA: Effective Area

Claims (9)

A camera module for generating an optical image by photographing an industrial site; and
A fall detection surveillance camera system in an industrial site comprising a server that analyzes the optical image received from the camera module to determine whether a person who has fallen exists. The method of claim 1,
The server
a storage unit for storing the optical image received from the camera module;
a person abnormal determination unit that analyzes the optical image to determine whether a person is in a fallen state, and outputs an occurrence of a person abnormal event when it is determined that the person has fallen; and
A surveillance camera system for detecting falls in an industrial field, comprising a notification unit for notifying a manager of the occurrence of an abnormal event when the abnormal person determination unit outputs the occurrence of an abnormal person event. The method of claim 2,
The above-mentioned human judgment department
A frame is extracted from the optical image, the frame is input to a fall recognition artificial intelligence model, and when the fall recognition artificial intelligence model determines that a person who has fallen in the frame exists, it is determined that an abnormal event has occurred to the person; ,
The fall recognition artificial intelligence model
An image of a person falling down is input data, and information indicating a person falling down in the image is learned using learning data, which is label data. A surveillance camera system for detecting falls at industrial sites that displays a box in the area and displays and outputs the class matching score. The method of claim 3,
The fall recognition artificial intelligence model
An industrial fall detection surveillance camera system formed by adjusting the network weights of a CNN model trained for the purpose of classifying people, vehicles, and animals to determine the fall of a person by applying deep transfer learning. The method of claim 3,
The camera module is
A surveillance camera system for detecting a fall at an industrial site, wherein a plurality of surveillance cameras are disposed at the industrial site so that at least a part of an effective area that varies depending on a falling posture and a positional relationship with the camera module in the monitoring area is continuous. The method of claim 5,
The above-mentioned human judgment department
If the class matching score output by the fall recognition artificial intelligence model is higher than the reference score, it is determined that the person has fallen,
The camera module is
When the reference score increases, the area of the valid area narrows. Therefore, when the reference score increases, the direction in which the camera module faces is changed so that at least a part of the valid area continues, and when the reference score decreases, the valid area Since the area of is widened, when the reference score is lowered, at least a part of the effective area is continued and the direction in which the camera module is directed is changed so as to monitor as wide an area as possible. An image generation step in which the camera module photographs the industrial site, generates an optical image, and transmits the optical image to the server;
a human abnormality determination step in which a human abnormality determination unit of the server extracts a frame from the optical image and inputs the frame to a fall recognition artificial intelligence model, and determines that an abnormal human event has occurred if there is a fallen person; and
A method of operating a fall detection monitoring camera system in an industrial site, comprising a notification step of notifying a manager of the occurrence of an abnormal event by a notification unit of the server when the abnormal person determination unit outputs the occurrence of an abnormal person event. The method of claim 7,
An image of a person falling down is input data and information indicating a person falling down in the image is learning using a learning data set, which is label data. Further comprising a model generation step of generating the fall recognition artificial intelligence model to display a box in the fallen area and display and output a class matching score,
The method of operating a fall detection surveillance camera system in an industrial site, wherein the model generation step is performed in advance before the person abnormality determination step. The method of claim 7,
When the reference score of the class matching score, which is the criterion for determining that a person has fallen, increases, the area of the valid region narrows, so when the reference score increases, the direction in which the camera module faces changes so that at least a part of the effective region continues, , When the reference score is lowered, since the area of the effective area is widened, when the reference score is lowered, at least a part of the effective area is continuously controlled to change the direction in which the camera module is directed so as to monitor an area as wide as possible. A method of operating a fall detection surveillance camera system at an industrial site, further comprising a step of changing the reference score.

Images