KR20240057963A - Apparatus and method for detecting collision risk situation between a forklift and a worker - Google Patents

Abstract

The present invention relates to a method for detecting a collision risk situation between a forklift and a worker, comprising the steps of measuring the distance between a forklift and a worker through an AIoT device or an Edge device to predict or detect a collision risk situation between a forklift and a worker; And when a collision risk situation between the forklift and the worker is predicted or detected, a step of alarming a collision risk situation.

Description

Apparatus and method for detecting a collision risk situation between a forklift and a worker {APPARATUS AND METHOD FOR DETECTING COLLISION RISK SITUATION BETWEEN A FORKLIFT AND A WORKER}

The present invention relates to an apparatus and method for detecting a collision risk situation between a forklift and a worker, and more specifically, to an apparatus and method for detecting a collision risk situation between a forklift and a worker based on an image. It's about.

In general, forklifts are widely used in various industrial sites for cargo loading, logistics transportation, and product transportation.

Forklifts are recognized as having a low risk of accidents because their speed is not fast and their movements are relatively simple. However, every year, casualties occur due to forklift-related accidents, such as collisions with workers due to the forklift driver's poor visibility and forklifts falling over due to overloading. Damage continues to occur.

In order to prevent accidents caused by forklifts in industrial sites and to utilize them widely, the demand for industrial safety continues to increase. Accordingly, for industrial safety management, it is possible to detect collision risk situations between workers and forklifts and prevent safety accidents. There is a need for technology to enable this.

The present invention is intended to solve the above problems, and its purpose is to provide a device and method for detecting a collision risk situation between a forklift and a worker, which can detect a collision risk situation between a worker and a forklift based on images at an industrial site. There is.

The present invention measures the safety distance between forklifts and workers in real time at industrial sites to monitor collision risk situations between forklifts and workers in real time, and also predicts collision risk situations and promptly notifies workers to prevent safety accidents. The purpose is to provide a device and method for detecting a collision risk situation between a forklift and a worker.

A method for detecting a collision risk situation between a forklift and a worker according to some embodiments of the present invention measures the distance between a forklift and a worker using an AIoT (Artificial Intelligence of Things) device or an Edge device to predict a collision risk situation between a forklift and a worker. detecting; And when a collision risk situation between the forklift and the worker is predicted or detected, a step of alarming a collision risk situation.

In the present invention, the AIoT device or edge device predicts or detects a collision risk situation between a forklift and an operator through a safety management module, and the safety management module periodically collects data detected from the sensor module through a collection module. Collects and analyzes multiple industrial site situations based on the data collected through the collection module through the data analysis module, and uses the data learning module to generate an AI model that predicts or detects collision risk situations. It is characterized by learning and predicting a collision risk situation through the generated AI model through a collision risk situation prediction module.

In the present invention, the AIoT device or Edge device sets a threshold related to prediction or detection of a collision risk situation through a user interface module, and the user interface module sets the distance between the forklift and the worker through an image capture module. Captures images to set configuration elements related to calibration for measurement, and sets the safety distance violation threshold based on the user's selection to output an alarm for safety distance violations between the operator and forklift through the safety point setting module. It is characterized by:

In the present invention, the safety point setting module sets a straight line for measuring the safety distance in the captured image upon the user's selection, and automatically sets the safety distance violation threshold based on the coordinate values of the start and end of the straight line. It is characterized by setting.

In the present invention, the user interface module sets a rectangle for setting a search area in the captured image through the safety distance indicator module by receiving the user's selection, and sets the coordinate values corresponding to the four vertices forming the rectangle. It is characterized by automatically setting the safety distance violation threshold for the square.

In the present invention, the AIoT device or Edge device uses an AI model to detect data from a forklift based on the relative ratio of the actual worker's height to the worker's height reflected in the CMOS sensor based on the camera lens attached to the forklift. It is characterized by calculating the distance of a fallen object.

In the present invention, the AI model in the AIoT device or Edge device is based on the safety distance violation threshold set between the forklift and the worker, and as the collision risk situation based on the distance between the forklift and the worker changes, the AI model is captured through a camera. It is characterized by changing the color of the object bounding box detected in the image.

In the present invention, the AI model in the AIoT device or Edge device recognizes an object in an image captured through a camera attached to a forklift, extracts the object bounding box, calculates the ratio of the object bounding box, and calculates the ratio of the object bounding box. If the ratio is less than the specified first threshold value (Th _ratio ), the bottom position value of the object bounding box is extracted and checked to see if it is greater than the second threshold value (Th _space ), and the bottom position value of the object bounding box is the second threshold value. If it is more than (Th _space ), it is a collision risk situation and the distance between the worker and the forklift is judged to be very close, and an alarm and violation alarm message is generated and sent to the data collection connector, and the distance of the object is greater than the violation threshold (Th _violation ). In this case, alarm and violation alarm messages are generated and transmitted to the data collection connector, and the alarm module is driven and a screen is displayed. Additionally, alarm and violation images and video data are stored in the database according to the cycle set by the user, and are stored for later use. It is characterized by being implemented so that users can search.

In the present invention, the AI model in the AIoT device or Edge device calculates the distance of the object from the forklift if the lower position value of the object bounding box is not more than the second threshold (Th _space ), and the distance of the object is Check whether the distance of the object is greater than the specified alarm threshold (Th _alarm ), and if the distance of the object is greater than the specified alarm threshold (Th _alarm ), check whether the distance of the object is less than the violation threshold (Th _violation ), and check whether the distance of the object is greater than the specified alarm threshold (Th alarm). If it is greater than the alarm threshold (Th _alarm ) and less than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and transmitted to the data collection connector 410, and the distance of the object is greater than the violation threshold (Th _violation ). Even in larger cases, alarm and violation alarm messages are generated and sent to the data collection connector, the alarm module is driven and a screen is displayed, and alarm and violation images and video data are stored in the database according to the cycle set by the user. Therefore, it is characterized by being implemented so that users can query it later.

In the present invention, the AI model in the AIoT device or edge device recognizes the objects of the forklift and the worker from the captured image of a camera attached to the industrial site, extracts the coordinates of each object's bounding box, and sets each object's bounding box. Calculate the center point of the forklift and the center point of the worker, calculate the trajectory and movement speed of the forklift and worker based on the center point, predict the direction of movement of the object from the frame data of the video, and calculate the movement of the forklift and worker. Considering the speed and trajectory, the distance between the forklift and the worker for a collision risk situation is calculated, and the distance between objects is not greater than the alarm threshold (Th _alarm ), and the warning threshold (Th _alarm ) and violation threshold (Th _violation ) are calculated. ), an alarm and violation alarm message is generated and sent to the data collection connector. Also, if the distance between objects is greater than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector, and an alarm message is generated and sent to the data collection connector. It is characterized by being implemented to drive the module and display a screen, and to store alarm and violation images and video data in a database according to a cycle set by the user, so that the user can search them later.

An apparatus for detecting a collision risk situation between a forklift and a worker according to some embodiments of the present invention includes an AIoT device or Edge device that predicts or detects a collision risk situation between a forklift and a worker based on an image; A safety management module included in the AIoT device or Edge device to detect a collision risk situation and perform corresponding operations; and a user interface module that is connected to the AIoT device or Edge device and sets a threshold value that serves as a standard for predicting or detecting a collision risk situation in the safety management module.

In the present invention, the AIoT device or Edge device includes a safety management module for predicting a collision risk situation and performing a corresponding operation, and the safety management module periodically collects data detected from the sensor module. collection module; a data analysis module that analyzes a plurality of industrial site situations based on data collected through the collection module; A data learning module that learns designated data to create an AI model that predicts or detects collision risk situations; And a collision risk situation prediction module that predicts a collision risk situation through the generated AI model.

In the present invention, it further includes a user interface module that is connected to the AIoT device or edge device and sets a threshold value related to prediction of a collision risk situation, wherein the user interface module is configured to measure the distance between the forklift and the worker. An image capture module that captures images selected by the user to set configuration elements related to calibration; and a safety point setting module that sets a safety distance violation threshold selected by the user to output an alarm for violation of the safety distance between the worker and the forklift.

In the present invention, the safety point setting module sets a straight line for measuring the safety distance in the captured image upon the user's selection, and automatically sets the safety distance violation threshold based on the coordinate values of the start and end of the straight line. It is characterized by setting.

In the present invention, the user interface module further includes a safety distance indicator module, wherein the safety distance indicator module receives the user's selection and sets a rectangle for setting a search area in the captured image, forming the rectangle. It is characterized in that the safety distance violation threshold value for the square is automatically set according to the coordinate values corresponding to the four vertices.

In the present invention, the AI model in the AIoT device or Edge device is based on the safety distance violation threshold set between the forklift and the worker, and as the collision risk situation based on the distance between the forklift and the worker changes, the AI model is captured through a camera. It is characterized by changing the color of the object bounding box detected in the image.

In the present invention, the AI model in the AIoT device or Edge device recognizes an object in an image captured through a camera attached to a forklift, extracts the object bounding box, calculates the ratio of the object bounding box, and calculates the ratio of the object bounding box. If the ratio is less than the specified first threshold value (Th _ratio ), the bottom position value of the object bounding box is extracted and checked to see if it is greater than the second threshold value (Th _space ), and the bottom position value of the object bounding box is the second threshold value. If it is more than (Th _space ), it is a collision risk situation and the distance between the worker and the forklift is judged to be very close, and an alarm and violation alarm message is generated and sent to the data collection connector, and the distance of the object is greater than the violation threshold (Th _violation ). In this case, alarm and violation alarm messages are generated and transmitted to the data collection connector, and the alarm module is driven and a screen is displayed. Additionally, alarm and violation images and video data are stored in the database according to the cycle set by the user, and are stored for later use. It is characterized by being implemented so that users can search.

In the present invention, the AI model in the AIoT device or Edge device calculates the distance of the object from the forklift if the lower position value of the object bounding box is not more than the second threshold (Th _space ), and the distance of the object is Check whether the distance of the object is greater than the specified alarm threshold (Th _alarm ), and if the distance of the object is greater than the specified alarm threshold (Th _alarm ), check whether the distance of the object is less than the violation threshold (Th _violation ), and check whether the distance of the object is greater than the specified alarm threshold (Th alarm). If it is greater than the alarm threshold (Th _alarm ) and less than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and transmitted to the data collection connector 410, and the distance of the object is greater than the violation threshold (Th _violation ). Even in larger cases, alarm and violation alarm messages are generated and sent to the data collection connector, the alarm module is driven and a screen is displayed, and alarm and violation images and video data are stored in the database according to the cycle set by the user. Therefore, it is characterized by being implemented so that users can query it later.

In the present invention, the AI model in the AIoT device or edge device recognizes the objects of the forklift and the worker from the captured image of a camera attached to the industrial site, extracts the coordinates of each object's bounding box, and sets each object's bounding box. Calculate the center point of the forklift and the center point of the worker, calculate the trajectory and movement speed of the forklift and worker based on the center point, predict the direction of movement of the object from the frame data of the video, and calculate the movement of the forklift and worker. Considering the speed and trajectory, the distance between the forklift and the worker for a collision risk situation is calculated, and the distance between objects is not greater than the alarm threshold (Th _alarm ), and the warning threshold (Th _alarm ) and violation threshold (Th _violation ) are calculated. ), an alarm and violation alarm message is generated and sent to the data collection connector. Also, if the distance between objects is greater than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector, and an alarm message is generated and sent to the data collection connector. It is characterized by being implemented to drive the module and display a screen, and to store alarm and violation images and video data in a database according to a cycle set by the user, so that the user can search them later.

An apparatus for detecting a collision risk situation between a forklift and a worker according to some embodiments of the present invention includes an AIoT device or edge device that predicts or detects a collision risk situation between a forklift and a worker based on an image; And is connected to communicate with the AIoT device or Edge device, performs learning to create an AI model that predicts a collision risk situation based on data collected from the AIoT device or Edge device, and uses the created AI model. It includes a cloud that predicts or detects a collision risk situation, and the cloud predicts a collision risk situation depending on the lack of resources or policy needs to perform data learning and collision risk situation prediction in the AIoT device or edge device. It is characterized by predicting or detecting.

The present invention detects collision risk situations between forklifts and workers based on video and provides warnings and violation notifications to prevent collisions between forklifts and workers and respond quickly.

The present invention measures the safety distance between forklifts and workers in real time at industrial sites to monitor collision risk situations between forklifts and workers in real time, and also predicts collision risk situations and promptly notifies workers to prevent safety accidents. do.

Figure 1 is an example diagram shown to explain the basic concept for predicting or detecting a collision risk situation between a forklift and a worker related to an embodiment of the present invention.
Figure 2 is an example diagram showing the safety management module and user interface module included in AIoT devices and edge devices.
Figure 3 is an example diagram illustrating a method of setting the straight-line distance and search area, which are standards for determining a collision risk situation.
Figure 4 is an example diagram showing a cloud that is connected to AIoT devices and edge devices.
Figure 5 is an example diagram to explain a type 1 method of predicting in advance a collision risk situation between a forklift driver and worker using an AIoT module.
Figure 6 is an example diagram to explain a type 2 method of predicting in advance a collision risk situation between a forklift driver and worker using an edge device.
Figure 7 is a flow chart to explain the image capture operation at an industrial site.
Figure 8 is a flow chart to explain the calibration process for setting the safety distance.
Figure 9 is an example diagram to explain a method of calculating the distance from the camera using a mathematical equation based on the worker's height.
Figure 10 is a flow chart to explain a method of setting a search area for predicting or detecting a type 2 collision risk situation.
Figure 11 is a flow chart to overall explain a method for predicting or detecting a collision risk situation between a forklift and an operator according to an embodiment of the present invention.
Figure 12 is a flow chart to explain a detailed method for predicting or detecting a collision risk situation between a forklift and a worker for type 1 branched into ①.
Figure 13 is a flow chart to explain a detailed method for predicting or detecting a collision risk situation between a forklift and a worker for type 2 branched into ②.
Figure 14 is an example diagram shown to explain a method of recognizing an object and extracting an object bounding box based on AI from front/rear camera images of a forklift.
Figure 15 is an example diagram showing a collision risk situation detection screen by type 1.
Figure 16 is an example diagram showing a collision risk situation detection screen by type 2.

Hereinafter, an embodiment of an apparatus and method for detecting a collision risk situation between a forklift and a worker according to the present invention will be described with reference to the attached drawings. In this process, the thickness of lines or sizes of components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, definitions of these terms should be made based on the content throughout this specification.

Figure 1 is an example diagram shown to explain the basic concept for predicting or detecting a collision risk situation between a forklift and a worker related to an embodiment of the present invention.

Referring to Figure 1, methods for predicting or detecting collision risk situations between forklifts and workers that may occur in industrial sites can be classified into two types.

The first type (i.e., type 1) uses AIoT (Artificial Intelligence of Things) devices (or AIoT modules) attached to the front and rear of the forklift, as shown in (a) of FIG. 1, to communicate with the forklift driver. This is a method of predicting (or detecting) risk of collision between workers in advance.

Here, an AIoT device (or AIoT module) is a device that combines artificial intelligence (AI) and IoT and applies object intelligence convergence technology.

The second type (i.e., type 2) is a collision between a forklift and a worker at an edge device connected to sensors (e.g. cameras, temperature, humidity, etc.) installed in an industrial site, as shown in (b) of Figure 1. It is a method of predicting (or detecting) risky situations in advance.

Here, the edge device may include a server, closed circuit television (CCTV), a user's computer, or an AIoT device. Additionally, the Edge device is equipped with an AI model to control collision risk situations between workers and forklifts in real time and provides a function to immediately notify workers when a collision risk situation occurs.

Figure 2 is an example diagram showing the safety management module and user interface module included in AIoT devices and edge devices. FIG. 3 is an example diagram illustrating a method of setting a straight-line distance and a search area that are standards for determining a collision risk situation in FIG. 1.

In this embodiment, N or more AIoT devices and edge devices including the user interface module 100 and the safety management module 200 can be installed and operated at an industrial site. Here, the user interface module 100 and the safety management module 200 can be implemented with at least one processor, and the detailed internal modules can be integrated into one processor and driven, or can be driven independently by a separate processor. there is.

Therefore, in the following description, for convenience, each module is described as operating separately, but one processor may operate in an integrated manner.

Referring to FIG. 2, the environment setting module 210 inside the safety management module 200 configures the basic environment setting information of the safety management module 200 (e.g., IP address, device ID, data collection cycle, dangerous situation warning, and (violation thresholds, etc.) are stored and managed.

The second monitoring module 220 monitors situational information including designated high-risk elements at industrial sites.

The collection module 230 periodically collects data (e.g., image information, temperature, humidity, etc.) sensed (detected) from the sensor module 320 installed at the industrial site.

The collision risk situation prediction module 240 predicts a collision risk situation through the generated AI model 340.

The data control module 250 controls the alarm module 310, collects data detected through the sensor module 320, and stores it in the database 330.

The data analysis module 260 analyzes industrial field data (e.g., collision risk situation, worker compliance with safety management, etc.) based on data detected and collected through the sensor module 320.

The external interface module 270 is connected to an external system or cloud in a wired or wireless manner to transmit and receive designated data.

The data learning module 280 learns designated data to generate the AI model 340 (that is, an AI model that predicts or detects a collision risk situation).

Referring to FIG. 2, the image capture module 110 of the user interface module 120 captures configuration elements related to calibration (i.e., forklift) from image (i.e., video and still image) data collected at industrial sites. Images (e.g., reference images containing forklifts at industrial sites, still images) for setting environmental setting elements related to calibration to measure the exact distance between the user and the worker are captured upon the user's selection.

The safety point setting module 120 is a type 1 (i.e., a method of predicting in advance a collision risk situation between a forklift driver and a worker using an AIoT module) as shown in (a) of FIG. 1. The alarm and violation threshold values for the safety distance (i.e., the safety distance violation threshold value) are set based on the user's selection.

As shown in (a) of FIG. 3, the safety point setting module 120 creates one straight line (e.g., a virtual straight line) to measure the safety distance (or danger distance) from the captured image containing the forklift. Set based on the user's selection (i.e., draw a straight line).

For example, referring to (a) of FIG. 3, the safety point setting module 120 moves the user from the center of the forklift (e.g., one x, y point coordinate) to what the user considers a safe distance (or dangerous distance). Select and set a straight line (i.e. draw a straight line). At this time, this straight line is used to determine that there is a risk of collision if the worker is within the distance (or range) of this straight line, and the violation threshold (i.e., safety distance violation threshold) from the user is determined by the start and end coordinate values of this straight line. can be set.

At this time, the straight-line distance is set to the violation threshold (i.e., safety distance violation threshold), and the alarm threshold (i.e., safety distance violation threshold) is set to be smaller than the violation threshold (i.e., safety distance violation threshold) (i.e. , short so as to be close to the center of the forklift), can be set at the user's choice.

In the case of Type 2 (i.e., a method of predicting a collision risk situation between a forklift driver and worker using an edge device) as shown in (a) of FIG. 1, as shown in (b) of FIG. 3, In order to set a search area for predicting or detecting collision risk situations, a rectangle (i.e., a virtual rectangle) is created from captured images (e.g., images containing forklifts at industrial sites, still images) selected by the user. You can set the coordinates (i.e. x, y point coordinates) corresponding to the four vertices that form .

The safety distance indicator module 130 sets a violation threshold (i.e., safety distance violation threshold) selected by the user to detect a collision risk situation between the forklift and the worker.

The first monitoring module 140 monitors data collected from AIoT devices or edge devices (i.e., data detected and collected from sensors installed at industrial sites), and through this monitoring, collision risk situations are predicted (or detected) or In case of occurrence, a warning/alarm message is output and the user can view images (e.g. still image) and images (e.g. video) of a collision risk situation.

Figure 4 is an example diagram showing a cloud that is connected to AIoT devices and edge devices.

Referring to (a) of FIG. 4, the safety management module 200 mounted on the AIoT device detects and collects sensing data (e.g., image, temperature, humidity, etc.).

The data collection connector 410 manages AIoT devices at industrial sites and monitors data collected through AIoT devices.

Additionally, the data collection connector 410 transmits AIoT device control information (e.g., sensing data collection cycle, AI model automatic update, alarm image storage cycle, etc.) to the AIoT device.

The cloud 420 manages N data collection connectors 410.

In addition, based on data collected through N AIoT devices at industrial sites, the cloud 420 learns data to create an AI model (i.e., an AI model that predicts collision risk situations) and collision risk using the AI model. Perform situational predictions.

Data learning and collision risk situation prediction are performed when there are insufficient execution resources to perform data learning and collision risk situation prediction on AIoT devices and edge devices, or when service policy requires AI model learning in the cloud (420). can do.

Additionally, the cloud 420 manages AI models for management and optimization of production processes and collision risk situations required in industrial sites.

Referring to (b) of FIG. 4, the safety management module 200 mounted on the edge device performs the same function as the AIoT device shown in (a) of FIG. 4.

At this time, the edge device and the cloud 420 may be connected to communication through the data collection connector 410, or communication may be connected directly between the edge device and the cloud 420 without going through the data collection connector 410.

Figures 5 and 6 are examples to explain a method of predicting or detecting a collision risk situation between a forklift and a worker through an AIoT device or edge device.

Figure 5 is an example diagram to explain a type 1 method of predicting in advance a collision risk situation between a forklift driver and worker using an AIoT module, and Figure 6 is an example showing a method of predicting a collision risk situation between a forklift driver and worker using an edge device in advance. This is an example diagram to explain the type 2 prediction method.

As described in FIG. 1, there are two types of methods for predicting collision risk situations between forklifts and workers (e.g., Type 1 and Type 2).

The first type (i.e. Type 1) is a method in which an AIoT device (or AIoT module) is attached directly to a forklift and detects (or predicts) a risk of collision with surrounding workers as the forklift moves to an industrial site (Figure 5 reference).

The second type (i.e. type 2) is a method of detecting (or predicting) dangerous situations between forklifts and workers remotely in real time through an edge device (see Figure 6).

Referring to FIG. 5, an AIoT device is attached to a location where the front/rear situation of the forklift can be viewed.

Sensor (e.g. camera, temperature, humidity, etc.) and alarm (e.g. sound, LED, etc.) devices (or modules) are connected to an AIoT device (or AIoT module).

For example, an AIoT device attached to a forklift and a rear camera connected to it can capture the situation behind the forklift and transmit it to the forklift driver's front monitor via high-speed optical wireless communications (OWC).

In Type 1 (i.e., a method of predicting collision risk situations between forklift drivers and workers using an AIoT module), prediction of collision risk situations between forklift drivers and workers can be divided into an online process and an offline process.

The offline process sets and stores configuration elements related to calibration to measure the exact distance between the forklift and the worker through the AIoT device through the user interface module 100.

The online process performs safety management functions in real time based on information set and stored in the offline process (e.g., environmental setting elements related to calibration).

An AIoT device equipped with an AI model recognizes objects (e.g. workers, high-risk elements, etc.) from the front camera (not shown) and rear camera (not shown) of the forklift. In addition, data analysis and collision risk situations are predicted (or detected) in real time by detecting the situation (e.g. distance, direction, location, trajectory, etc.) between forklifts and workers in industrial sites to predict and detect collision risk situations.

Accordingly, when a collision risk situation is detected, the AIoT device immediately disseminates the collision risk situation to the surrounding area through an alarm (e.g., sound, alarm) that notifies the collision risk situation, and informs the data collection connector 410 of the collision risk situation. Send warning and violation messages (i.e. safety distance violation messages).

Referring to FIG. 6, the user performs calibration to measure the distance between the forklift and the worker in advance through the user interface module 100 (see (a) of FIG. 3).

Additionally, the user sets the safety management target area (or search area) at the industrial site through the user interface module 100 (see (b) in FIG. 3).

Environmental setting information such as the safety management target area (or search area) and the distance between the forklift and the worker are stored in the database, and the risk of collision between the forklift and the worker is detected through the distance measurement function between the forklift and the worker.

Figure 7 is a flow chart to explain the image capture operation at an industrial site.

The image capture module 110 of the user interface module 100 measures the image desired by the user (i.e., the exact distance between the forklift and the worker) through the sensor module 320 (e.g. camera) connected to the AIoT device or edge device. Capture images to set configuration elements related to calibration.

For example, in the case of the first type (i.e., type 1) that predicts a collision risk situation between a forklift and a worker, the image capture module 110 receives the image input from the sensor module 320 (e.g., camera) upon selection by the user. Among the videos, the image desired by the user (i.e., an image for setting configuration elements related to calibration to measure the exact distance between the forklift and the worker) is captured (S101, S102).

The environment setting module 210 of the AIoT device can convert the video into a frame and store it (S103, S104).

Figure 8 is a flow chart to explain the calibration process for setting the safety distance.

The safety distance indicator module 130 retrieves a pre-captured image (i.e., an image for setting configuration elements related to calibration to measure the accurate distance between the forklift and the worker) (reference image) from the database 330. It comes (loads) (S201).

If Type 1 (i.e., a method of predicting in advance a collision risk situation between a forklift driver and worker using an AIoT module) (example of S202), the worker height (Height) is calculated using an AI model from the captured image (reference image). ) is detected, and the pixels within the frame for the operator's actual key are calculated from the captured image (S203).

Then, the actual distance data between the forklift and the worker is retrieved (S204), and the focal distance (f) (or focal length) from the camera lens to the CMOS sensor is calculated and set (S205) (see FIG. 9). At this time, the focal length (f) may be given in advance as a specification (SPEC).

The safety distance indicator module 130 includes the type value of the method for predicting a collision risk situation between a forklift and a worker (i.e., type 1 or type 2), the actual worker's height for calculating the camera focal length, and the actual distance between the forklift and the worker. The values of environmental setting elements related to the distance apart and calibration of differences in pixel values within the frame are stored in the database (or file) 330 (S206).

Meanwhile, in the case of Type 2 (i.e., a method of predicting a collision risk situation between a forklift driver and worker using an edge device in advance) (No in S202), the safety distance indicator module 130 captures in advance the user's selection. To set the violation threshold (i.e., safety distance violation threshold) on the forklift in the image (i.e., image for setting configuration elements related to calibration to measure the correct distance between the forklift and the operator) (reference image). Set a straight line horizontally or vertically (i.e., set the pixel value of the distance (length) corresponding to the straight line) (S207).

This is a standard for judging that the forklift and the worker are in close proximity if it is within the straight-line distance, and for judging that the worker is safe from the forklift if it is outside the straight-line distance. This is the violation threshold (i.e., the safety distance violation threshold) and is the standard for determining that the worker is safe from the forklift. Rather than entering numbers, drawing a straight line on the captured image has the feature of allowing you to know the safety distance more intuitively.

In addition, the safety distance indicator module 130 sets calibration-related settings (e.g., pixel values corresponding to a straight line distance set horizontally or vertically (i.e., drawn) in order to set a violation threshold value (i.e., a safety distance violation threshold value). : Safety distance warning threshold, safety distance violation threshold, etc.) data is saved (S206).

For example, assuming that the pixel value corresponding to the straight-line distance set by the user is 600, the safety distance violation threshold and the safety distance warning threshold can be set in various ways within this pixel value of 600. At this time, in order to correct deviations depending on the angle and position of the camera, the safety distance violation threshold and safety distance warning threshold can be set directly from the user.

Figure 9 is an example diagram to explain a method of calculating the distance from the camera using a mathematical equation based on the worker's height.

Figure 9 (a) is to explain a method of calculating the height of the actual worker using the relative ratio of the actual worker's height based on the lens and the worker's height reflected on the CMOS sensor. .

The focal length (f) of a camera is the distance from the lens to the CMOS sensor.

As shown in the equation shown in (b) of Figure 9, the worker's height (c) on the CMOS sensor can be calculated as a relative ratio according to the focal distance (f) and the actual worker's height (h) and distance (d). and this is the tangent It is calculated as Also tangent Can be calculated according to the equation shown in (c) of FIG. 9, and the final distance (d) can be calculated according to the equation shown in (d) of FIG. 9.

However, the method shown in Figure 9 is an example to explain that the distance from the camera can be calculated through the relative ratio of the actual worker's height and the worker's height reflected on the CMOS sensor, and is not intended to necessarily limit this. Pay attention to

Figure 10 is a flowchart explaining a method of setting a search area for predicting or detecting a type 2 collision risk situation.

As explained with reference to (b) of FIG. 3, through the user interface module 100, the user can use Type 2 (i.e., a method of predicting in advance a collision risk situation between a forklift driver and worker using an edge device). You can set up a search area (i.e., a search area for predicting or detecting a collision risk situation).

The user interface module 110 receives the user's selection and retrieves a previously captured image (i.e., an image for setting configuration elements related to calibration to measure the accurate distance between the forklift and the worker) (reference image) ( S301).

If Type 2 (i.e., a method of predicting a collision risk situation between a forklift driver and worker using an edge device in advance) (example of S302), the search area, that is, the collision risk situation, is selected from the captured image (reference image). To set up a search area for prediction or detection, select and store four x, y point coordinates corresponding to the vertices of a square (i.e., a virtual square), draw a line connecting the four vertices, and Set and save the boundary line (S303 ~ S305).

A collision risk situation between a forklift and a worker is detected within the search area for predicting or detecting the collision risk situation set above.

Figure 11 is a flow chart to overall explain a method for predicting or detecting a collision risk situation between a forklift and an operator according to an embodiment of the present invention.

Referring to FIG. 11, an input image is acquired through the sensor module 320 (e.g., a camera) (S401), and in FIG. 11, the input image is acquired by the sensor module 320 (e.g., a camera), Type 1. In the case of (i.e., a method of predicting in advance the risk of collision between forklift drivers and workers using an AIoT module), the camera for acquiring input images is connected to the AIoT device, and type 2 (i.e., forklift drivers and workers using edge devices) In the case of (a method of predicting collision risk situations between workers in advance), the camera for acquiring input images is connected to the edge device.

In Figure 11, in the collection module 230 of the safety management module 200 mounted on the AIoT device, the input image is converted to each frame and resized (S402), and if type 2 (i.e., using an edge device) In the case of a method of predicting a collision risk situation between a forklift driver and worker in advance (example of S404), the AI model corresponding to type 2 is loaded and branches to ② (S407).

If it is type 1 (No in S404), the AI model corresponding to type 1 is loaded (S405), and the AI model corresponding to type 1 checks the preset focal length and then branches to number ① (S406). .

Figure 12 is a flow chart to explain a detailed method for predicting or detecting a collision risk situation between a forklift and a worker for type 1 branched into ①. Figure 13 is a flow chart to explain a detailed method for predicting or detecting a collision risk situation between a forklift and a worker for type 2 branched into ②.

Referring to FIG. 12, the AI model corresponding to Type 1 acquires images captured through cameras on the front/rear of the forklift (S408), and displays the image of the rear of the forklift on the operator monitor (not shown) at the front of the forklift. When transmitted (S409) (see (a) in Figure 14), AI-based objects (workers) are recognized from the front/rear camera images (S410), the object bounding box is extracted (S411), and the ratio of the object bounding box is calculated. Calculate (S412) (see (b) of FIG. 14).

Figure 14(b) shows coordinate values displayed as a bounding box when an object is detected. The size of the bounding box can be determined by measuring the horizontal and vertical lengths through the (x ₀ , y ₀ ) coordinates of the upper left corner and (x ₁ , y ₁ ) coordinates of the lower right corner of the bounding box. In other words, the bounding box is displayed as a square using the x and y axes with the object as the center, and the bounding box values are (X minimum value, Y minimum value) in the upper left corner (e.g. x ₀ , y ₀ ) and ( X maximum value, Y maximum value) (e.g. x ₁ , y ₁ ).

The bounding box ratio of the object can be calculated through the data analysis module 260.

The ratio of the object bounding box is compared with a specified first threshold value (Th _ratio ) (i.e., ratio threshold) (S413), and the ratio of the object bounding box is determined by the first threshold (Th _ratio ) (i.e., ratio threshold). If it is smaller than (example in S413), extract the lower position value of the object bounding box and check whether this value is greater than or equal to the second threshold Th _space (i.e. space threshold) (S414). If the lower position value of the object bounding box is more than the second threshold (Th _space ) (i.e., space threshold) (example in S414), it is a collision risk situation, and the distance between the worker and the forklift is judged to be very close (S418). , Alert and violation alarm messages are generated and transmitted to the data collection connector 410 (S420).

If the bounding box ratio of the object exceeds the threshold value of the bounding box ratio set by the user (No in S413), it is recognized as an abnormal object detection and branches back to the process of receiving front and rear camera images of the forklift (S408). .

On the other hand, when the coordinate value of the bottom of the object's bounding box exceeds the space threshold (Th _space ) set by the user (example in S414), that is, when the object image is close to the camera and the entire bounding box is not detected, the operator's distance This means that cannot be calculated normally.

Meanwhile, if the lower position value of the object bounding box is not more than the second threshold (Th _space ) (No in S414), the distance of the object (i.e. worker) from the forklift is calculated (S415), and the distance of the object is set to the specified alarm threshold. Check whether it is greater than the value (Th _alarm ) (i.e., safety distance alarm threshold) (S416).

If the distance of the object is greater than the specified alarm threshold (Th _alarm ) (i.e., the safe distance alarm threshold) (example in S416), the distance of the object is greater than the violation threshold (Th _violation ) (i.e., the safe distance violation threshold). Check if it is small (S417).

If the distance of the object is greater than the specified alarm threshold (Th _alarm ) (i.e., safe distance alarm threshold) (example of S416) and less than the violation threshold (Th _violation ) (i.e., safe distance violation threshold) ( Example of S417), an alarm and violation alarm message is generated and transmitted to the data collection connector 410 (S420).

If the distance of the object is not between the specified alarm threshold (Th _alarm ) (i.e., safe distance alarm threshold) and the violation threshold (Th _violation ) (i.e., safety distance violation threshold) (No in S417), the object If the distance is greater than the violation threshold (Th _violation ) (i.e., safety distance violation threshold) (example of S419), an alarm and violation alarm message is generated and sent to the data collection connector 410 (S420), and the alarm module In addition to driving 310, a screen is displayed (S421) (e.g., GUI display), and alarm and violation image (e.g., still image) and image (e.g., video) data are stored in the database 330 (S422) ).

At this time, alarm and violation image (e.g. still image) and image (e.g. video) data can be stored at a cycle set by the user.

Referring to FIG. 13, in type 2, the safety management module 200 recognizes an object (eg, worker, forklift) from the sensor module 320 (eg, camera) based on an AI model (S423).

In addition, the collision risk situation prediction module 240 recognizes the objects of the forklift and the worker from the camera image, and then sets the coordinates of the bounding box ((x ₀ , y ₀ ), (x ₁ , y ₁ )) for tracking. Extract (S424).

The data analysis module 260 calculates the center point of the forklift (Forklift(x,y)) and the center point of the worker (person(x,y)) from the bounding box (S425), and calculates the center point of the forklift and the worker based on the center point. Trajectory tracking and speed are calculated (S426).

Here, the trajectory tracking may be performed by connecting the center point of the object detected in previous frames and the center point of the object detected in frames displayed from the current image. Additionally, the speed calculates the time difference by measuring t1 time when an object (e.g., forklift and worker) is detected in the previous frame, and measuring t2 time when an object is detected in the current frame. Additionally, the distance moved is calculated by calculating the center point of the object's bounding box in the previous frame and the current frame, respectively. From this, the distance moved can be calculated based on the relative ratio between the pixel difference value and the actual distance value in the image through which the object moved, and the speed can be calculated by dividing the distance moved by time.

Then, the direction of movement of the object is predicted from the video frame data (S427).

For reference, if the object's bounding box moves up/down/left/right, the direction of movement can be predicted using the Kalman filter. That is, based on the (x,y) coordinates of the center point of the bounding box, the direction can be predicted by accumulating the motion information of the (x,y) coordinates of the current frame and the predicted frame.

For example, X can be calculated as X _t+1 -X _t , and Y can be calculated as Y _t+1 -Y _t .

Therefore, the overall direction of movement can be calculated as dx = dx + X _t+1 -X _t, dy = dy + Y _t+1 -Y _t . At this time, if dx is greater than 0, the object can be judged to have moved to the left, if it is less than 0, it can be judged to have moved to the right, if dy is greater than 0, it can be judged to have moved up, and if it is less than 0, it can be judged to have moved down.

Then, considering the speed and trajectory of the forklift and the worker, the distance between the forklift and the worker (i.e., the distance between objects) for a collision risk situation is calculated (S428).

For reference, the distance between the forklift and the worker is: It can be calculated by the formula.

If the distance between objects is greater than the alarm threshold (Th _alarm ) (Yes in S429), branch back to the object recognition process (S423), otherwise (No in S423), the distance between objects is greater than the alarm threshold (Th _alarm ). Compare whether it is within the violation threshold (Th _violation ) (S430).

If the distance between objects is between the alarm threshold (Th _alarm ) and the violation threshold (Th _violation ) (example in S430), an alarm and violation alarm message is generated and transmitted to the data collection connector 410 (S432).

If the distance between objects is not between the alarm threshold (Th _alarm ) and the violation threshold (Th _violation ), but is greater than the violation threshold (Th _violation ) (example of S431), data collection connector generates alarm and violation alarm messages transmits to 410 (S432), drives the alarm module 310 and displays a screen (S433) (e.g., GUI display), and alarm and violation images (e.g., still images) and images (e.g., video) ) Data is stored in the database 330 (S434).

At this time, alarm and violation image (e.g. still image) and image (e.g. video) data can be stored at a cycle set by the user.

Figure 15 is an example diagram showing a collision risk situation detection screen by type 1.

Figure 15 (a) is a screen displayed when the worker's distance from the forklift is normal and is not in a dangerous situation, and Figure 15 (b) shows that the worker's distance from the forklift is smaller than the distance that would trigger a violation alarm, and the warning alarm is not triggered. This is a screen displayed when the distance is greater than the distance to be generated. Figure 15(c) is a screen displayed when the worker's position from the forklift exceeds the alarm threshold value (Th _alarm ), and Figure 15(d) is a screen displayed when the operator's position from the forklift exceeds the alarm threshold (Th alarm). This screen is displayed when the worker's position in the forklift exceeds the violation threshold (Th _violation ). At this time, the color of the bounding box is gradually changing from green to red.

Figure 16 is an example diagram showing a collision risk situation detection screen by type 2.

Figure 16 (a) is a screen displayed when a collision risk situation detection area set by the user is set. Even if the distance between the forklift and the object is close, if it is outside the collision risk situation detection area, it is not judged to be a dangerous situation. At this time, the worker's trajectory, location, and predicted direction of progress are displayed.

Figure 16(b) is a screen displayed when the dangerous situation detection area is not set. It shows a screen where an alarm occurs and a violation alarm occurs depending on the distance between the forklift and the worker. And it can be performed in a complex manner by combining Type 1 and Type 2.

As described above, the present invention calculates data (e.g. distance, direction of movement, trajectory, location, etc.) in real time to predict or detect collision risk situations between forklifts and workers based on images, thereby reducing the risk of collisions between forklifts and workers in industrial sites. It is effective in detecting (or predicting) situations and sounding alarms and violation notifications to prevent accidents caused by collisions between forklifts and workers and to respond quickly.

In addition, the present invention mounts an AI model on an AIoT device that can be directly attached to a forklift and an edge device that can remotely monitor dangerous situations, thereby reducing the industrial accident rate by detecting or predicting various dangerous situations in advance, such as collision accidents between forklifts and workers. It has the effect of significantly reducing .

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those skilled in the art will recognize that various modifications and other equivalent embodiments are possible therefrom. You will understand. Therefore, the scope of technical protection of the present invention should be determined by the scope of the patent claims below. Implementations described herein may also be implemented as, for example, a method or process, device, software program, data stream, or signal. Although discussed only in the context of a single form of implementation (eg, only as a method), implementations of the features discussed may also be implemented in other forms (eg, devices or programs). The device may be implemented with appropriate hardware, software, firmware, etc. The method may be implemented in a device such as a processor, which generally refers to a processing device that includes a computer, microprocessor, integrated circuit, or programmable logic device.

100: User interface module
110: Image capture module
120: Safety point setting module
130: Safety distance indicator module
140: first monitoring module
200: Safety management module
210: Environment setting module
220: second monitoring module
230: collection module
240: Collision risk situation prediction module
250: data control module
260: Data analysis module
270: external interface module
280: Data learning module
310: Alarm module
320: sensor module
330: database
340: AI model
410: data collection connector
420: Cloud

Claims (20)

Predicting or detecting a risk of collision between a forklift and a worker by measuring the distance between a forklift and a worker using an AIoT (Artificial Intelligence of Things) device or edge device; and
A method of detecting a collision risk situation between a forklift and a worker, comprising the step of alarming a collision risk situation when a collision risk situation between a forklift and a worker is predicted or detected.
According to clause 1,
The AIoT device or Edge device predicts or detects collision risk situations between forklifts and workers through a safety management module,
The safety management module is,
Data detected from the sensor module is periodically collected through the collection module,
Through the data analysis module, multiple industrial site situations are analyzed based on the data collected through the collection module,
Through the data learning module, designated data is learned to create an AI model that predicts or detects collision risk situations,
A method for detecting a collision risk situation between a forklift and a worker, characterized in that the collision risk situation is predicted through the AI model generated through the collision risk situation prediction module.
According to clause 1,
The AIoT device or Edge device sets a threshold related to prediction or detection of a collision risk situation through a user interface module,
The user interface module is,
Through the image capture module, images are captured to set configuration elements related to calibration to measure the distance between the forklift and the worker,
A method of detecting a risk of collision between a forklift and a worker, characterized in that a safety distance violation threshold value for outputting an alarm for a safety distance violation between the worker and a forklift is selected by the user and set through a safety point setting module.
According to clause 3,
The safety point setting module is,
Collision risk between forklifts and workers, characterized in that a straight line for measuring the safety distance in the captured image is selected by the user and the safety distance violation threshold is automatically set based on the coordinate values of the start and end of the straight line. How to detect a situation.
According to clause 3,
The user interface module is,
A rectangle for setting a search area in the captured image is selected and set by the user through the safety distance indicator module, and the safety distance violation threshold for the rectangle is determined by the coordinate values corresponding to the four vertices forming the rectangle. A method for detecting a collision risk situation between a forklift and a worker, characterized by automatic setting.
According to clause 1,
The AIoT device or Edge device,
A forklift that uses an AI model to calculate the distance of an object away from a forklift based on the relative ratio of the actual worker's height to the worker's height reflected in the CMOS sensor based on the camera lens attached to the forklift. Method for detecting conflict risk situations between workers.
According to clause 1,
The AI model in the AIoT device or edge device is,
Based on the safety distance violation threshold set between the forklift and the worker,
A method for detecting a collision risk situation between a forklift and a worker, characterized in that the color of an object bounding box detected in an image captured by a camera changes as the collision risk situation based on the distance between the forklift and the worker changes.
According to clause 1,
The AI model in the AIoT device or edge device is,
Objects are recognized in images captured through a camera attached to a forklift, the object bounding box is extracted, and the ratio of the object bounding box is calculated.
If the ratio of the object bounding box is less than the specified first threshold value (Th _ratio ), extract the lower position value of the object bounding box and check whether it is greater than the second threshold value (Th _space ),
If the lower position value of the object bounding box is more than the second threshold (Th _space ), it is a collision risk situation and the distance between the worker and the forklift is judged to be very close,
Alarm and violation alarm messages are generated and sent to the data collection connector, and even when the distance of the object is greater than the violation threshold (Th _violation ), alarm and violation alarm messages are created and sent to the data collection connector, and the alarm module is driven. A method for detecting a risk of collision between a forklift and a worker, characterized in that it displays a screen and stores warning and violation images and video data in a database according to a cycle set by the user, so that the user can search them later.
According to clause 10,
The AI model in the AIoT device or edge device is,
If the lower position value of the object bounding box is not more than a second threshold (Th _space ), calculate the distance of the object from the forklift, and check whether the distance of the object is greater than the specified alarm threshold (Th _alarm ),
If the distance of the object is greater than the specified alarm threshold (Th _alarm ), check whether the distance of the object is less than the violation threshold (Th _violation ). If the distance of the object is greater than the specified alarm threshold (Th alarm), check whether the distance of the object is less than the violation threshold (Th _alarm ). If it is less than the value (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector 410,
Even if the distance of the object is greater than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector, and the alarm module is driven and the screen is displayed, and alarm and violation images and video data are also generated. A method of detecting a risk of collision between a forklift and a worker, characterized in that the data is stored in a database according to a cycle set by the user and can be searched later by the user.
According to clause 1,
The AI model in the AIoT device or edge device is,
After recognizing the objects of the forklift and worker from images captured by cameras attached to industrial sites, the coordinates of each object's bounding box are extracted,
Calculate the center point of the forklift and the center point of the worker from each object bounding box,
Calculate the trajectory tracking and movement speed of the forklift and worker based on the center point,
Predict the direction of movement of an object from the frame data of the video,
Calculate the distance between the forklift and the worker for a collision risk situation by considering the movement speed and trajectory of the forklift and the worker.
If the distance between objects is not greater than the alarm threshold (Th _alarm ) and is between the alarm threshold (Th _alarm ) and the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector. Even if the distance between objects is greater than the violation threshold (Th _violation ), an alarm and violation alarm message are generated and sent to the data collection connector, and the alarm module is driven and a screen is displayed.
A method of detecting a risk of collision between a forklift and a worker, characterized in that alarm and violation images and video data are stored in a database according to a cycle set by the user and can be viewed by the user later.
An AIoT device or edge device that predicts or detects collision risk situations between forklifts and workers based on video;
A safety management module included in the AIoT device or Edge device to detect a collision risk situation and perform corresponding operations; and
A collision risk situation detection device between a forklift and a worker, characterized in that it includes a user interface module that is connected to the AIoT device or Edge device and sets a threshold value that serves as a standard for predicting or detecting a collision risk situation in the safety management module. .
According to clause 11,
The AIoT device or Edge device includes a safety management module for predicting collision risk situations and performing corresponding operations,
The safety management module is,
a collection module that periodically collects data detected from the sensor module;
a data analysis module that analyzes a plurality of industrial site situations based on data collected through the collection module;
A data learning module that learns designated data to create an AI model that predicts or detects collision risk situations; and
A collision risk situation detection device between a forklift and a worker, comprising a collision risk prediction module that predicts a collision risk situation through the generated AI model.
According to clause 11,
It further includes a user interface module that is connected to the AIoT device or Edge device and sets a threshold related to prediction of a collision risk situation,
The user interface module is,
An image capture module that captures a capture image selected by the user to set environmental setting elements related to calibration for measuring the distance between the forklift and the worker; and
A collision risk situation detection device between a forklift and a worker, comprising a safety point setting module that sets a safety distance violation threshold selected by the user to output an alarm for a violation of the safety distance between the worker and the forklift.
According to clause 13,
The safety point setting module is,
Set a straight line to measure the safety distance in the captured image based on the user's selection,
A collision risk situation detection device between a forklift and a worker, characterized in that automatically sets the safety distance violation threshold based on the coordinate values of the start and end of the straight line.
According to clause 13,
The user interface module further includes a safety distance indicator module,
The safety distance indicator module is,
Set a rectangle for setting the navigation area in the captured image based on the user's selection.
A collision risk situation detection device between a forklift and a worker, characterized in that automatically setting a safety distance violation threshold value for the square based on coordinate values corresponding to the four vertices forming the square.
According to clause 11,
The AI model in the AIoT device or edge device is,
Based on the safety distance violation threshold set between the forklift and the worker,
A collision risk situation detection device between a forklift and a worker, characterized in that the color of the object bounding box detected in the image captured by the camera changes as the collision risk situation based on the distance between the forklift and the worker changes.
According to clause 11,
The AI model in the AIoT device or edge device is,
Objects are recognized in images captured through a camera attached to a forklift, the object bounding box is extracted, and the ratio of the object bounding box is calculated.
If the ratio of the object bounding box is less than the specified first threshold value (Th _ratio ), extract the lower position value of the object bounding box and check whether it is greater than the second threshold value (Th _space ),
If the lower position value of the object bounding box is more than the second threshold (Th _space ), it is a collision risk situation and the distance between the worker and the forklift is judged to be very close,
Alarm and violation alarm messages are generated and sent to the data collection connector, and even when the distance of the object is greater than the violation threshold (Th _violation ), alarm and violation alarm messages are created and sent to the data collection connector, and the alarm module is driven. A collision risk situation detection device between a forklift and a worker, characterized in that it is implemented to display a screen and store warning and violation images and video data in a database according to a cycle set by the user so that the user can view them later.
According to clause 17,
The AI model in the AIoT device or edge device is,
If the lower position value of the object bounding box is not more than a second threshold (Th _space ), calculate the distance of the object from the forklift, and check whether the distance of the object is greater than the specified alarm threshold (Th _alarm ),
If the distance of the object is greater than the specified alarm threshold (Th _alarm ), check whether the distance of the object is less than the violation threshold (Th _violation ). If the distance of the object is greater than the specified alarm threshold (Th alarm), check whether the distance of the object is less than the violation threshold (Th _alarm ). If it is less than the value (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector 410,
Even if the distance of the object is greater than the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector, and the alarm module is driven and the screen is displayed, and alarm and violation images and video data are also generated. A collision risk situation detection device between a forklift and a worker, characterized in that it is implemented by storing it in a database according to a cycle set by the user so that the user can search it later.
According to clause 11,
The AI model in the AIoT device or edge device is,
After recognizing the objects of the forklift and worker from images captured by cameras attached to industrial sites, the coordinates of each object's bounding box are extracted,
Calculate the center point of the forklift and the center point of the worker from each object bounding box,
Calculate the trajectory tracking and movement speed of the forklift and worker based on the center point,
Predict the direction of movement of an object from the frame data of the video,
Calculate the distance between the forklift and the worker for a collision risk situation by considering the movement speed and trajectory of the forklift and the worker.
If the distance between objects is not greater than the alarm threshold (Th _alarm ) and is between the alarm threshold (Th _alarm ) and the violation threshold (Th _violation ), an alarm and violation alarm message is generated and sent to the data collection connector. Even if the distance between objects is greater than the violation threshold (Th _violation ), an alarm and violation alarm message are generated and sent to the data collection connector, and the alarm module is driven and the screen is displayed.
A collision risk situation detection device between a forklift and a worker, characterized in that it is implemented by storing warning and violation images and video data in a database according to a cycle set by the user so that the user can search them later.
An AIoT device or edge device that predicts or detects collision risk situations between forklifts and workers based on video; and
It is connected to communicate with the AIoT device or Edge device, performs learning to create an AI model that predicts a collision risk situation based on data collected from the AIoT device or Edge device, and uses the created AI model to perform collision Includes a cloud that predicts or detects risky situations,
The cloud is,
A collision risk situation detection device between a forklift and a worker, characterized in that it predicts or detects a collision risk situation according to a lack of resources or policy needs to perform data learning and collision risk situation prediction in the AIoT device or Edge device.