KR102366590B1 - Safety management system for heavy equipment vehicle - Google Patents

Abstract

The present invention relates to a heavy equipment vehicle safety management system using a deep learning algorithm, and the heavy equipment vehicle safety management system of the present invention is an image photographing device for photographing a plurality of images of a heavy equipment vehicle, a plurality of images captured by the image photographing device , and creates a program based on the learned results based on the accident occurrence data that occurred on the existing heavy equipment vehicles, and based on multiple images and programs, the type of heavy equipment vehicle, work contents, movement speed, and movement route In consideration of the deep learning algorithm driving device and deep learning algorithm driving device that determines whether an accident occurs before and during the operation of the heavy equipment vehicle in advance, and generates notification data according to the determination result, the notification data is received and , a notification device that displays an accident occurrence point or generates a warning sound based on the received notification data. Accordingly, the present invention estimates the occurrence of an accident of a heavy equipment vehicle using a deep learning algorithm and informs the driver of the heavy equipment vehicle before the accident occurs, thereby reducing the accident rate of the heavy equipment vehicle in the work environment.

Description

Heavy Equipment Vehicle Safety Management System {SAFETY MANAGEMENT SYSTEM FOR HEAVY EQUIPMENT VEHICLE}

The present invention relates to a heavy equipment vehicle safety management system, and more particularly, to a heavy equipment vehicle safety management system using a deep learning algorithm.

In general, various types of heavy equipment vehicles such as forklifts, dump trucks, ready-mixed concrete and forklifts are widely used in various industrial sites such as construction sites. Most heavy-duty vehicles are generally large and heavy and generate enormous power, so drivers who operate heavy equipment must always pay attention for safe driving.

However, as described above, the heavy equipment vehicle has a large body and is operated by one driver, and since the position of the driver's seat for operating the heavy equipment is high, it is impossible to recognize all the surrounding conditions of the heavy equipment vehicle.

Accordingly, there is a trend of frequent accidents due to stenosis or collision during the forward or backward movement of the heavy-duty vehicle, which generally occurs because the driver of the heavy-duty vehicle does not recognize all the surrounding conditions within the danger radius.

Accordingly, there is an increasing need to comply with the Industrial Safety and Health Act and to establish a work system in which a driver who drives a heavy-duty vehicle can safely operate a heavy-duty vehicle.

The present invention has been devised to solve the above problems, and uses an image captured by an image capturing device installed in the workplace and a deep learning algorithm to notify the driver when a hazardous element occurs within the working radius of a heavy-duty vehicle. is to provide a heavy equipment vehicle safety management system.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A heavy equipment vehicle safety management system according to an embodiment of the present invention receives an image photographing device for photographing a plurality of images of a heavy equipment vehicle, a plurality of images photographed by the image photographing device, and an accident occurring for a heavy equipment vehicle in the past A program is created based on the results learned based on the data, and accidents before and during heavy equipment vehicle operation and heavy equipment vehicle operation in consideration of the type of heavy equipment vehicle, work content, movement speed and movement route based on multiple images and programs A deep learning algorithm driving device that determines whether an occurrence has occurred, and generating notification data according to the result of the determination, and receiving notification data generated from the deep learning algorithm driving device, displaying the accident occurrence point or sounding an alarm based on the received notification data It may include a notification device that is generated.

According to various embodiments of the present disclosure, the image capturing apparatus may include a first image capturing unit mounted on a heavy equipment vehicle and a second image capturing unit installed in a work environment in which the heavy equipment vehicle works.

According to various embodiments, the deep learning algorithm driving device receives a plurality of images captured by the image capturing device, and based on the plurality of images, predicts accidents that may occur during operation of heavy equipment vehicles and signs of accidents of heavy equipment vehicles. The communication unit that transmits the notification data generated based on the analysis result to the notification device, and a plurality of images received from the image capturing device are analyzed to determine the expected accident that can occur during heavy equipment vehicle operation before operation of the heavy equipment vehicle program, which can determine the signs of an accident that may occur while working with heavy equipment vehicles It may include a storage unit for storing a risk factor determination program capable of including a control unit for controlling the overall operation of the deep learning algorithm driving device by executing the predicted accident determination program stored in the storage unit, the accident indication determination program and the risk factor determination program there is.

According to various embodiments, the control unit executes a predicted accident determination program and analyzes the images received from the image capturing device to determine an expected accident that may occur during the operation of the heavy equipment vehicle before the heavy equipment vehicle operates, the accident The accident symptom determination unit that determines whether an accident has occurred in the work environment of heavy equipment vehicles in advance by executing the occurrence sign determination program, and executes the risk factor determination program to determine whether or not if there are signs of an accident caused by a person, , generating notification data according to the results determined by the risk factor determination unit and the expected accident occurrence determination unit, the accident sign determination unit and the risk factor determination unit to determine whether a risk factor exists within the danger radius during the operation of the heavy equipment vehicle It includes a hazard notification data generation unit, the predicted accident occurrence determination unit determines the expected accident in advance by considering the type and work details of the heavy equipment vehicle, and the accident symptom determination unit considers the moving speed, movement route and work details of the heavy equipment vehicle In this way, it is possible to determine in advance whether an accident has occurred.

According to various embodiments, the deep learning algorithm driving device may be any one of a cloud server and an edge device installed in a heavy equipment vehicle.

The heavy equipment vehicle safety management method according to an embodiment of the present invention includes the steps of learning an accident pattern of a heavy equipment vehicle based on an accident image of an existing heavy equipment vehicle, and generating an accident occurrence symptom program and a risk factor determination program based on the learning result Step, receiving a plurality of images taken by an image capturing device, a predicted accident that may occur in a heavy-duty vehicle based on the received plurality of images and an expected accident determination program, an accident occurrence symptom program and a risk factor determination program; The stage of judging the signs of an accident and the risk factors that exist around the heavy equipment vehicle before the accident. It may include generating the notification data and transmitting the generated notification data.

According to various embodiments, in the step of pre-determining the expected accident of the heavy equipment vehicle, the signs of the accident, and the risk factors existing around the heavy equipment vehicle, the determination is made in consideration of the type of the heavy equipment vehicle, the movement speed, the movement route, and the contents of the work can do.

Details of other embodiments are included in the detailed description and drawings.

In addition to the first image capturing unit such as a plurality of cameras mounted on the heavy equipment vehicle, the present invention acquires an image of the surrounding environment of the heavy equipment vehicle by using a second image capturing unit such as a closed circuit television or digital image storage and transmission equipment installed in a work environment By doing so, it is possible to estimate in advance whether an accident has occurred in a heavy equipment vehicle with a wider radius, and it is possible to estimate in advance whether an accident has occurred in a heavy equipment vehicle in real time in consideration of the movement path of the heavy equipment vehicle.

The present invention can reduce the accident rate of heavy equipment vehicles in the work environment by estimating the occurrence of accidents of heavy equipment vehicles three times using a deep learning algorithm and notifying the drivers who drive the heavy equipment vehicles.

The present invention estimates the occurrence of an accident of a heavy equipment vehicle and informs the driver of the heavy equipment vehicle whether the reason for the estimation is caused by a person or an object, so that it is possible to respond more quickly and clearly to the accident estimation.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present invention.

1 is a diagram schematically showing the configuration of a heavy equipment vehicle safety management system according to a first embodiment of the present invention.
2 is a diagram schematically showing the configuration of an image capturing apparatus of a heavy equipment vehicle safety management system according to a first embodiment of the present invention.
3 is a diagram schematically showing the configuration of an edge device of the heavy equipment vehicle safety management system according to the first embodiment of the present invention.
4 is a flowchart illustrating a method for safety management of a heavy equipment vehicle of the heavy equipment vehicle safety management system according to the first embodiment of the present invention.
5 is a diagram schematically showing the configuration of a heavy equipment vehicle safety management system according to a second embodiment of the present invention.
6 is a diagram schematically showing the configuration of the atcloud server of the heavy equipment vehicle safety management system according to the second embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them, will become apparent with reference to the detailed description in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, it can be implemented in various different forms, and it is understood to include all transformations, equivalents and substitutes included in the spirit and scope of the present invention. should be The embodiments presented below are provided to complete the disclosure of the present invention, and to fully inform those of ordinary skill in the art to the scope of the invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Various embodiments of the present invention may be implemented as software (eg, a program) including instructions stored in a machine-readable storage medium (eg, a computer). The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, a server) according to the disclosed embodiments. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

According to an example, the method according to various embodiments disclosed in the present invention may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a device-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

Each of the components (eg, a module or a program) according to various embodiments may be composed of a singular or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be It may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component are sequentially, parallel, repetitively or heuristically executed, or at least some operations are executed in a different order, are omitted, or other operations are added. can be

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically showing the configuration of a heavy equipment vehicle safety management system according to a first embodiment of the present invention.

Referring to FIG. 1 , the heavy equipment vehicle safety management system 1000 according to the first embodiment of the present invention may include an image capturing device 100 , an edge device 200 , and a notification device 300 .

The image photographing apparatus 100 may photograph a plurality of images of a surrounding environment of a heavy equipment vehicle working in a work environment. The image capturing apparatus 100 may be installed in a heavy equipment vehicle, or may be installed in a work environment in which the heavy equipment vehicle works. The image capturing apparatus 100 may include a first image capturing unit 110 and a second image capturing unit 120 .

The first image capturing unit 110 may be a type of black box or a plurality of cameras for photographing a heavy equipment vehicle and an environment close to the heavy equipment vehicle. The first image capturing unit 110 may photograph the surroundings of the heavy equipment vehicle by dividing the area in all directions around it. The plurality of images photographed in this way is transmitted to the edge device 200 .

The second image capturing unit 120 is installed in the work environment, and at least one of a closed circuit television (CCTV) and a digital video recorder (DVR) for photographing a work site where a heavy equipment vehicle works. can be one The images captured by the second image capturing unit 120 are transmitted to the edge device 200 . In this way, the heavy equipment vehicle safety management system 1000 according to the first embodiment of the present invention uses the image capturing apparatus 100 to photograph the surroundings of the heavy equipment vehicle in addition to a plurality of cameras mounted on the heavy equipment vehicle in the existing work environment. By using the installed closed circuit television or digital image storage and transmission equipment to manage the safety of heavy equipment vehicles, accidents of heavy equipment vehicles can be estimated in advance by considering not only the occurrence of accidents around the heavy equipment vehicles but also the movement routes.

The edge device 200 may be mounted in a heavy-duty vehicle working in a work environment. The edge device 200 analyzes the images transmitted from the image capturing device 100 and transmits the analysis result to the notification device 300 . The edge device 200 can determine whether an accident has occurred in advance by using a deep learning algorithm when analyzing the image, and can analyze people and objects separately, and collect notification data based on the analysis results. can create The notification data generated in this way may be transmitted to the notification device 300 . On the other hand, the edge device 200 may be referred to as a deep learning algorithm driving device. A detailed configuration of the edge device 200 will be described in more detail with reference to FIG. 3 to be described later.

The notification device 300 may notify the driver of the heavy equipment vehicle of the occurrence of a risk factor based on the notification data received from the edge device 200 . The notification device 300 may be installed in a heavy equipment vehicle. The notification device 300 may include a sound unit 310 and a display unit 320 .

The sound unit 310 is a device that notifies a driver of a heavy equipment vehicle whether a dangerous element has occurred or not through sound. The sound unit 310 may, for example, generate an alarm to notify the driver of whether a dangerous element has occurred in advance. In this case, a plurality of sounds stored in the sound unit 310 are stored, and accordingly, different alarm sounds may be generated according to circumstances. For example, if the sound unit 310 generates a first alarm sound when there is a possibility that a risk factor may occur to the driver in advance, the driver's view is blocked, such as when the heavy-duty vehicle lifts an object while the heavy-duty vehicle is running. In this case, a second warning sound may be generated.

The display unit 320 is a device that intuitively informs a driver of a heavy equipment vehicle whether a risk factor has occurred in advance. When the analysis result of the edge device 200 indicates that the heavy equipment vehicle has an accident potential, the display unit 320 may output a map or the like in advance to notify it.

2 is a diagram schematically showing the configuration of an image capturing apparatus of a heavy equipment vehicle safety management system according to a first embodiment of the present invention.

Referring to FIG. 2 , the image capturing apparatus 100 of the heavy equipment vehicle safety management system 1000 according to the first embodiment of the present invention may include a camera 101 , a communication unit 102 , and a control unit 103 . . As described above, the image photographing apparatus 100 is mounted on a heavy equipment vehicle and is not installed to photograph only the first image photographing unit 110 and the heavy equipment vehicle to photograph the environment close to the heavy equipment vehicle, but the heavy equipment vehicle works. A second image capturing unit 120 for capturing the inside of the working environment is included, and both the first image capturing unit 110 and the second image capturing unit 120 include a camera 101 , a communication unit 102 and a control unit 103 . may include

The camera 101 generates a plurality of images by photographing the heavy equipment vehicle and the heavy equipment vehicle proximity environment as well as the work environment in which the heavy equipment vehicle works. In this case, the images generated by the camera 101 may be still images and moving images.

The communication unit 102 includes an interface circuit for communication of the edge device 200 . More specifically, the communication unit 102 transmits a plurality of images generated by the camera 101 to the edge device 200 .

The controller 103 controls the overall operation of the image capturing apparatus 100 . More specifically, the controller 103 allows the images captured by the image capturing device 100 to be transmitted to the edge device 200 through the communication unit 102 in real time or to the edge device through the communication unit 102 at predetermined time intervals. It can be controlled to be transmitted to (200).

3 is a diagram schematically showing the configuration of an edge device of the heavy equipment vehicle safety management system according to the first embodiment of the present invention.

Referring to FIG. 3 , the edge device 200 of the heavy equipment vehicle safety management system 1000 according to the first embodiment of the present invention includes an edge communication unit 210 , an edge storage unit 220 , and an edge control unit 230 . can do. The edge communication unit 210, the edge storage unit 220, and the edge control unit 230 may be connected to each other through a bus, and the bus may include a circuit for transmitting various signals such as control messages, status information and/or data. can

The edge communication unit 210 includes an interface circuit for communication between the image capturing device 100 and the notification device 300 . For example, the edge communication unit 210 receives a plurality of images obtained by photographing the working environment of the heavy equipment vehicle from the image photographing apparatus 100 , and based on the plurality of images received from the image photographing apparatus 100 , around the heavy equipment vehicle A result of analyzing whether a risk factor has occurred may be transmitted to the notification device 300 .

The edge storage 220 may store a plurality of images received from the image capturing apparatus 100 , heavy equipment vehicle information, deep learning learning results, analysis results, and the like.

The edge storage 220 may include a volatile memory and/or a non-volatile memory. In the edge storage unit 220 , commands or data related to components, one or more programs and/or software, an operating system, etc. may be stored in order to implement and/or provide operations and functions provided by the edge device 200 . there is.

The edge storage unit 220 analyzes a plurality of images received from the heavy equipment vehicle information and a plurality of images received from an expected accident determination program that determines an accident that may occur based on the heavy equipment vehicle information and a plurality of images before driving the heavy equipment vehicle, and the image photographing device 100 to analyze the heavy equipment vehicle An accident symptom determination program that can determine the signs of an accident that may occur in the vicinity of a heavy-duty vehicle while driving before an accident, and a risk factor that can determine whether a person or an object exists within a danger radius around a heavy-duty vehicle A judgment program can be stored. Here, the predicted accident determination program, the accident occurrence sign determination program, and the risk factor determination program stored in the edge storage unit 220 are programs that are generated based on the results of learning based on the accident occurrence data that has occurred previously in the work environment of the heavy equipment vehicle. can be

The edge controller 230 may control the overall operation of the edge device 200 by executing one or more programs stored in the edge storage 220 . The edge control unit 230 receives a plurality of images received from the image capturing apparatus 100 and uses all of the plurality of images received by a deep learning-based algorithm to determine an expected accident determination program, an accident occurrence symptom determination program, and risk factors. You can create and update a judgment program, and use the predictive accident judgment program, the accident sign judgment program, and the risk factor judgment program to determine the expected accident, judge the signs of an accident, and determine the presence of risk factors, or view some of the plurality of images. By extracting it, it is possible to determine whether there is an expected accident, the signs of an accident, and the presence of risk factors. The edge control unit 230 may include an expected accident occurrence determination unit 231 , an accident occurrence symptom determination unit 232 , a risk factor determination unit 233 , and a risk factor notification data generation unit 234 .

The predicted accident occurrence determination unit 231 may execute the predicted accident determination program stored in the edge storage unit 220 to determine an expected accident that may occur in the work environment of the heavy equipment vehicle before the heavy equipment vehicle starts work. That is, the predicted accident occurrence determination unit 231 may determine the expected accident that may occur based on the heavy equipment vehicle information stored in the edge storage unit 220 and the image photographed inside the work environment from the image capturing device 100 . . In this case, the predicted accident occurrence determining unit 231 may determine the expected accident based on the plurality of images captured by the second image capturing unit 120 . In more detail, for example, when the heavy-duty vehicle is an excavator, a plurality of images captured by the second image capturing unit 120, that is, the work performed by the previously learned excavator, is displayed in the work environment in which the excavator will work. It is possible to determine the expected accident that may occur by matching the contents of the work. Here, the predicted accident determination program may be a program learned based on images and data of an accident occurring in the past.

The accident indication determination unit 232 may determine whether an accident has occurred in the work environment of the heavy equipment vehicle by executing the accident indication determination program stored in the edge storage unit 220 . That is, the accident occurrence symptom determination unit 232 determines whether an accident has occurred in consideration of the moving speed, the moving route, and the work contents of the heavy equipment vehicle in the work environment based on the plurality of images received from the image capturing apparatus 100 . can do. In this case, the accident occurrence sign determining unit 232 may determine the accident occurrence sign based on the plurality of images captured by the first image capturing unit 110 and the second image capturing unit 120 . For example, the accident sign determination unit 232 determines if there are dangerous equipment or people in the current moving path of the heavy equipment vehicle, if the moving speed is fast and it may collide with dangerous equipment or people, etc. Signs that an accident may occur It can be judged that there is Here, the accident occurrence symptom program may be a program learned based on an accident occurrence image and data that have occurred in the past.

The risk factor determination unit 233 may determine whether the risk factor existing around the heavy equipment vehicle is a person or an object by executing the risk factor determination program stored in the edge storage unit 220 when there is a sign of occurrence of an accident. In this case, the risk factor determination program classifies the case where the previously occurring accident data is human-caused accident data and object-caused accident data in consideration of the current moving speed and moving route of the heavy equipment vehicle, and the It can be a program. For example, if it is determined that there is a person in the moving path of the current heavy equipment vehicle, the risk factor determination unit 233 may determine that an accident caused by a human will occur, and consider the current moving speed of the heavy equipment vehicle. Therefore, if it is estimated that a situation in which an object can collide after 3 minutes will occur, it can be determined that an accident caused by an object will occur.

The risk factor determining unit 233 may determine the heavy equipment vehicle as a risk factor when the height of the object increases to a preset height or higher due to the heavy equipment vehicle work. This is to prevent accidents that may occur by obscuring the driver's view. Accordingly, the risk factor determination unit 233 may set a height lower than the height at which the heavy equipment vehicle can block the driver's view during work in advance, and generate notification data when an object exists above the set height.

Accordingly, the safety management system 1000 of the heavy equipment vehicle according to the first embodiment of the present invention analyzes the type and work environment of the heavy equipment vehicle to predict an accident that may occur in the heavy equipment vehicle before driving the heavy equipment vehicle, so that the driver in advance In addition, the heavy equipment vehicle receives an image from the image capturing device 100 while the vehicle is working, and the accident that may occur in consideration of the current work content and the moving speed of the heavy equipment vehicle can be secondarily informed to the driver in advance When it is judged that an accident will occur due to the presence of dangerous factors around the heavy-duty vehicle, such as blocking the driver's view while working on the heavy-duty vehicle, it is tertiary to notify the driver in advance to prevent accidents that may occur while the heavy-duty vehicle is working. can

The risk factor notification data generation unit 234 may generate notification data according to the respective determination results by the expected accident occurrence determination unit 231 , the accident occurrence symptom determination unit 232 , and the risk factor determination unit 233 . . At this time, the notification data includes first notification data, which is notification data when an accident is expected, second notification data, which is notification data when there is a sign of an accident, and second notification data, which is notification data when the sign of an accident is caused by a person. 3 may include notification data, fourth notification data, which is notification data when the sign of an accident is caused by an object, and fifth notification data, which is notification data when the driver's view can be blocked due to work of a heavy equipment vehicle . Here, the notification data may be image data and/or audio data. The notification data generated in this way may be transmitted to the notification device 300 .

Meanwhile, in the heavy equipment vehicle safety management system 1000 according to the first embodiment of the present invention, the deep learning algorithm is driven by the edge device 200 , but may also be driven by a cloud server. A more detailed description thereof will be described with reference to FIGS. 4 and 5 below.

4 is a diagram schematically showing the configuration of a heavy equipment vehicle safety management system according to a second embodiment of the present invention, and FIG. 5 is a diagram schematically illustrating the configuration of a cloud server of a heavy equipment vehicle safety management system according to a second embodiment of the present invention. It is a drawing shown as

First, referring to FIG. 4 , the heavy equipment vehicle safety management system 1000 ′ according to the second embodiment of the present invention may include an image capturing device 100 , a notification device 300 , and a cloud server 500 . . Among the configurations of the heavy equipment vehicle safety management system 1000 ′ according to the second embodiment of the present invention, the image photographing device 100 and the notification device 300 excluding the cloud server 500 are the present invention of FIG. 1 described above. Since it has the same configuration as the image capturing apparatus 100 and the notification apparatus 300 of the heavy equipment vehicle safety management system 1000 according to the first embodiment, a detailed description thereof will be omitted. Accordingly, the heavy equipment vehicle safety management system 1000 ′ according to the second embodiment of the present invention will be described with a focus on the cloud server 500 .

The cloud server 500 of the heavy equipment vehicle safety management system 1000 ′ according to the second embodiment of the present invention analyzes the images transmitted from the image capturing apparatus 100 , and transmits the image analysis result to the notification device 300 . do. The cloud server 500 can determine in advance whether an expected accident that may occur before starting work of a heavy equipment vehicle and an accident that may occur during operation of a heavy equipment vehicle using a deep learning algorithm during image analysis, and It is possible to classify and analyze the object and the object, and based on the analysis result, it is possible to generate notification data. The notification data generated in this way may be transmitted to the notification device 300 . On the other hand, the cloud server 500 may be referred to as a deep learning algorithm driving device.

The cloud server 500 may include a cloud communication unit 510 , a cloud storage unit 520 , and a cloud control unit 530 . The cloud communication unit 510, the cloud storage unit 520, and the cloud control unit 530 may be connected to each other through a bus, and the bus may include a circuit for transmitting various signals such as control messages, status information and/or data. can

The cloud communication unit 510 includes an interface circuit for communication between the image capturing apparatus 100 and the notification apparatus 300 . For example, the cloud communication unit 510 receives a plurality of images obtained by photographing the working environment of the heavy equipment vehicle from the image photographing apparatus 100 , and based on the plurality of images received from the image photographing apparatus 100 , around the heavy equipment vehicle A result of analyzing whether a risk factor has occurred may be transmitted to the notification device 300 .

The cloud storage 520 may store a plurality of images received from the image capturing apparatus 100 , a deep learning learning result, an analysis result, and the like.

The cloud storage unit 520 may include volatile memory and/or non-volatile memory. The cloud storage unit 520 may store commands or data related to components, one or more programs and/or software, an operating system, etc. in order to implement and/or provide operations and functions provided by the cloud server 500 . there is.

The cloud storage unit 520 is an expected accident program that determines an accident that may occur based on a plurality of images received from the heavy equipment vehicle information and the image capturing device 100 before driving the heavy equipment vehicle, received from the image capturing device 100 . An accident sign determination program that can analyze multiple images to determine the signs of an accident that may occur around a heavy-duty vehicle, and a risk that can determine whether a person or an object exists within a danger radius around a heavy-duty vehicle It is possible to store an element judgment program. Here, the predicted accident determination program, the accident occurrence symptom determination program, and the risk factor determination program stored in the cloud storage unit 520 are programs generated based on the results of learning based on the accident occurrence data that has occurred previously in the work environment of the heavy equipment vehicle. can be

The cloud control unit 530 may control the overall operation of the cloud server 500 by executing one or more programs stored in the cloud storage unit 520 . The cloud control unit 530 receives a plurality of images received from the image photographing device 100 and uses all of the plurality of images received by a deep learning-based algorithm to determine an expected accident determination program, an accident symptom determination program, and risk factors. You can create and update a judgment program, and use the predicted accident judgment program, the accident sign judgment program, and the risk factor judgment program to determine the presence or absence of an expected accident, the signs of an accident, and the presence of risk factors, or to view some of the plurality of images. By extracting it, it is possible to determine whether there is an expected accident, the signs of an accident, and the presence of risk factors. The cloud control unit 530 may include an expected accident occurrence determination unit 531 , an accident occurrence symptom determination unit 532 , a risk factor determination unit 533 , and a risk factor notification data generation unit 534 .

The predicted accident occurrence determination unit 531 may execute the predicted accident determination program stored in the cloud storage unit 520 to determine an expected accident that may occur in the work environment of the heavy equipment vehicle before the heavy equipment vehicle starts work. That is, the expected accident occurrence determination unit 531 may determine the expected accident that may occur based on the heavy equipment vehicle information stored in the cloud storage unit 520 and the image photographed inside the work environment from the image capturing apparatus 100 . . In this case, the predicted accident occurrence determining unit 531 may determine the expected accident based on the plurality of images captured by the second image capturing unit 120 . In more detail, for example, when the heavy-duty vehicle is an excavator, a plurality of images captured by the second image capturing unit 120, that is, the work performed by the previously learned excavator, is displayed in the work environment in which the excavator will work. By matching the contents of the work, it is possible to first determine the expected accident that may occur. Here, the predicted accident determination program may be a program learned based on images and data of an accident occurring in the past. The accident occurrence sign determination unit 532 may execute the accident occurrence symptom determination program stored in the cloud storage unit 520 to determine in advance whether an accident has occurred in the work environment of the heavy equipment vehicle while the heavy equipment vehicle is running. That is, the accident symptom determination unit 532 determines whether an accident has occurred in consideration of the current moving speed, moving route, and work contents of the heavy equipment vehicle in the work environment based on the plurality of images received from the image capturing apparatus 100 . can be determined in advance. In this case, the accident sign determining unit 532 may determine the accident occurrence sign based on the plurality of images captured by the first image capturing unit 110 and the second image capturing unit 120 . For example, if there are dangerous equipment or people on the moving path of the current heavy equipment vehicle, the accident sign determination unit 532 may detect a sign that an accident may occur, such as when a dangerous equipment or person may collide with a high moving speed. It can be judged that there is Here, the accident occurrence symptom program may be a program learned based on an accident occurrence image and data that have occurred previously.

The risk factor determination unit 533 may determine whether the risk factor existing around the heavy equipment vehicle is a person or an object by executing the risk factor determination program stored in the cloud storage unit 520 when there is a sign of an accident occurring. In this case, the risk factor determination program classifies the case where the previously occurring accident data is human-caused accident data and object-caused accident data in consideration of the current moving speed and moving route of the heavy equipment vehicle, and the It can be a program. For example, if it is determined that there is a person in the current moving path of the heavy equipment vehicle, the risk factor determination unit 533 may determine that an accident caused by a human will occur, and consider the current moving speed of the heavy equipment vehicle. Therefore, if it is estimated that a situation in which an object can collide after 3 minutes will occur, it can be determined that an accident caused by an object will occur.

The risk factor determining unit 533 may determine that when the height of the object increases to a predetermined height or more due to the heavy equipment vehicle work during the operation of the heavy equipment vehicle, it may be determined as a risk factor. This is to prevent accidents that may occur by obscuring the driver's view. Accordingly, the risk factor determination unit 533 may set a height lower than the height at which the heavy equipment vehicle can block the driver's view during work in advance, and generate notification data when an object exists above the set height.

The risk factor notification data generation unit 534 may generate notification data according to respective determination results by the expected accident occurrence determination unit 231 , the accident occurrence symptom determination unit 532 , and the risk factor determination unit 533 . . At this time, the notification data includes first notification data, which is notification data when an accident is expected, second notification data, which is notification data when there is a sign of an accident, and second notification data, which is notification data when the sign of an accident is caused by a person. 3 may include notification data, fourth notification data, which is notification data when the sign of an accident is caused by an object, and fifth notification data, which is notification data when a driver's field of view can be obtained due to work of a heavy equipment vehicle . Here, the notification data may be image data and/or audio data. The notification data generated in this way may be transmitted to the notification device 300 .

As described above, the heavy equipment vehicle safety management system 1000 according to the first and second embodiments of the present invention uses a deep learning algorithm in the edge device 200 and the cloud server 500 to indicate and risk occurrence of a heavy equipment vehicle accident. It is possible to prevent accidents of heavy equipment vehicles in advance by territorially determining factors and notifying the driver of the heavy equipment vehicle before an accident occurs.

6 is a flowchart illustrating a method for safety management of a heavy equipment vehicle according to an embodiment of the present invention. The operations of FIG. 6 may be performed by the edge device 200 or the cloud server 500 .

Referring to FIG. 6 , the edge device 200 or the cloud server 500 may learn an accident pattern of a heavy equipment vehicle based on an existing accident image of the heavy equipment vehicle ( S601 ). For example, the edge device 200 or the cloud server 500 determines whether accidents that occur frequently depending on the type and work content of the heavy-duty vehicle, whether many accidents occur within a radius of several meters centered on the heavy-duty vehicle, and the Considering the movement speed and movement route, it is possible to learn about the occurrence of an accident in any situation.

Thereafter, the edge device 200 or the cloud server 500 may generate and update an expected accident determination program, an accident occurrence symptom program, and a risk factor determination program based on the learning result ( S602 ). The predicted accident determination program, the accident occurrence symptom program, and the risk factor determination program generated and updated in this way are controlled to be stored in the edge storage unit 220 of the edge device 200 or the cloud storage unit 520 of the cloud server 500 . can

Thereafter, the edge device 200 or the cloud server 500 may receive images photographed by the image photographing apparatus 100 from the image photographing apparatus 100 to determine an expected accident ( S603 ). In this case, the edge device 200 or the cloud server 500 may receive the captured image from the second image capturing unit 120 installed in the work environment.

Thereafter, the edge device 200 or the cloud server 500 may analyze the received image and the expected accident determination program to determine whether an expected accident that may occur during heavy equipment vehicle work occurs ( S604 ). Here, the predicted accident refers to an accident that may occur before the heavy equipment vehicle works in the work environment photographed by the image capturing apparatus 100 according to the type and work details of the heavy equipment vehicle. For example, assuming that the type of heavy equipment vehicle is an excavator, the edge device 200 or the cloud server 500 identifies the working environment through the image captured by the second image capturing unit 120 and then When the excavator works within the identified working environment, it is possible to pre-determine an accident that may occur based on the learning contents.

As a result of the determination, when it is determined that an expected accident occurs (S605), first notification data is generated and transmitted to the display device 320 mounted on the heavy equipment vehicle (S606). In this case, the first notification data may be image data. For example, the first notification data may be data in the form of notifying the driver of the expected accident by displaying the work environment as a map on the display device 320 so that the predicted point where an accident may occur is displayed on the map. there is. Accordingly, the driver may be able to recognize in advance the expected accident according to the work contents of the heavy equipment vehicle before driving the heavy equipment vehicle.

Thereafter, the edge device 200 or the cloud server 500 may receive images photographed by the image photographing apparatus 100 from the image photographing apparatus 100 in order to determine an accident occurrence sign (S607). In this case, the edge device 200 or the cloud server 500 may receive both the captured images from the first image capturing unit 110 and the second image capturing unit 120 .

Thereafter, the edge device 200 or the cloud server 500 analyzes the received images and the accident occurrence symptom program to determine whether an accident occurs (S608), and as a result of the determination, if it is determined that there is an accident occurrence sign (S609) ) A second notification message may be generated and transmitted to the notification device 300 (S610). In this case, the step of determining whether an accident has occurred is performed while the heavy equipment vehicle is running, and it is possible to determine in advance the accident that may occur in consideration of the moving speed of the heavy equipment vehicle, the moving route, and the contents of the work. In this case, the second notification data may be image data and/or audio data. For example, the image data of the second notification data may be data capable of displaying the work environment as a map and a point where there is a sign of an accident, similar to the first notification data, and the voice data of the second notification data is an alarm. can be negative

On the other hand, when it is determined that there are no signs of an accident occurring, the edge device 200 or the cloud server 500 analyzes the plurality of images received from the image capturing device 100 and the accident sign program once again to determine whether there are any signs of an accident. It can be done (S608).

Thereafter, the edge device 200 or the cloud server 500 may analyze the received plurality of images and the risk factor determination program to determine whether the accident occurrence sign is caused by a human ( S611 ).

After that, if it is determined that the sign of the accident is caused by a person as a result of the analysis (S612), a third notification message is generated and transmitted (S613), and when it is determined that the sign of the accident is not caused by a person, the sign of an accident caused by a thing It is determined that , a fourth notification message may be generated and transmitted (S614). In this case, the third notification data and the fourth notification data may be voice data. Also, the third notification data and the fourth notification data may be alarm sounds having different sounds, and when the third notification data and the fourth notification data are alarm sounds, they may be alarm sounds having different sounds from the second notification data.

Thereafter, the edge device 200 or the cloud server 500 may receive images captured by the image capturing apparatus 100 from the image capturing apparatus 100 to determine the occurrence of a risk factor (S615). In this case, the edge device 200 or the cloud server 500 may receive both the captured images from the first image capturing unit 110 and the second image capturing unit 120 .

Thereafter, the edge device 200 or the cloud server 500 analyzes the received images and the risk factor determination program to determine whether a risk factor has occurred (S616), and as a result of the determination, if it is determined that a risk factor has occurred (S617) A fifth notification message may be generated and transmitted to the notification device 300 (S618). In this case, in the step of determining whether a risk factor occurs, a case in which the driver's field of view may be obstructed may occur in advance, which is performed while the heavy equipment vehicle is being operated, and the operation contents of the heavy equipment vehicle are taken into consideration. For example, when lifting an object using a heavy-duty vehicle, the height that can be determined as a risk factor is set in advance, and when a heavy-duty vehicle lifts an object by the preset height, it is determined that a risk factor has occurred and removed. 5 Notification data can be created and transmitted. In this case, the preset height may be the height before the height of the object lifted by the heavy equipment vehicle blocks the driver's view, and the fifth notification data may be voice data. In this case, the alarm sound of the fifth notification data may be an alarm sound having a different sound from the alarm sound of the second to fourth notification data.

On the other hand, the edge device 200 or the cloud server 500 analyzes the images received from the image capturing device 100 when it is determined that a risk factor does not occur as a result of the determination to determine whether there are any signs of an accident. Can be (S609).

As described above, in the safety management system 1000 of the heavy equipment vehicle according to an embodiment of the present invention, the determination of the occurrence of an accident sign and the determination of the occurrence of a risk factor while the heavy equipment vehicle is running are repeatedly performed. Accordingly, when it is determined that there are signs and risk factors of an accident occurring while the heavy equipment vehicle is operating, the accident rate of the heavy equipment vehicle can be reduced by notifying the driver in advance before the accident occurs.

Thereafter, the edge device 200 or the cloud server 500 terminates the safety management of the heavy equipment vehicle when it is determined that the operation of the heavy equipment vehicle is completed.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1000, 1000': Heavy-duty vehicle safety management system
100: image capturing device 110: first image capturing unit
120: second image capturing unit 101: camera
102: communication unit 103: control unit
200: edge device 210: edge communication unit
220: edge storage unit 230: edge control unit
231: predictive accident occurrence determination unit 232: accident occurrence symptom determination unit
233: risk factor determination unit 234: risk factor notification data generation unit
300: notification device 500: cloud server
510: cloud communication unit 520: cloud storage unit
530: cloud control unit

Claims (7)

an image photographing device for photographing a plurality of images of a heavy equipment vehicle;
Receives the plurality of images captured by the image capturing device, generates a program based on the learned result based on accident occurrence data that has occurred in the existing heavy equipment vehicle, and based on the plurality of images and the program Driving a deep learning algorithm to determine in advance whether an accident has occurred before driving the heavy equipment vehicle and during operation of the heavy equipment vehicle in consideration of the type of heavy equipment vehicle, work content, movement speed, and movement route, and to generate notification data according to the determination result Device; and
a notification device that receives the notification data generated by the deep learning algorithm driving device, and displays an accident point or generates a warning sound based on the received notification data;
The deep learning algorithm driving device primarily determines an expected accident that may occur in the heavy equipment vehicle before driving the heavy equipment vehicle by analyzing the type and work environment of the heavy equipment vehicle, and from the image capturing device during the operation of the heavy equipment vehicle After receiving an image, the present work content, detection of risk factors around the heavy-duty vehicle, and the movement speed of the heavy-duty vehicle are taken into consideration to secondarily determine the signs of an accident, and there is a risk factor around the heavy-duty vehicle during the operation of the heavy-duty vehicle. A heavy equipment vehicle safety management system that tertiarily determines an accident that may occur due to the presence or the height of an object being raised above a preset height during the operation of the heavy equipment vehicle. The method of claim 1,
The image capturing device includes a first image capturing unit mounted on the heavy equipment vehicle and a second image capturing unit installed in a work environment in which the heavy equipment vehicle works. According to claim 1, wherein the deep learning algorithm driving device,
Receiving a plurality of images captured by the image capturing device, and based on the plurality of images, a notification generated based on an analysis result of an expected accident that may occur during operation of the heavy equipment vehicle and an accident occurrence sign of the heavy equipment vehicle a communication unit for transmitting data to the notification device;
A predicted accident determination program capable of analyzing a plurality of images received from the image photographing device to determine an expected accident that may occur during the heavy equipment vehicle operation before the heavy equipment vehicle operation, and a sign of an accident that may occur during the heavy equipment vehicle operation A storage unit for storing a risk factor determination program capable of determining whether there is a risk factor existing within a danger radius around the heavy equipment vehicle and a risk factor determination program capable of determining whether the accident occurrence sign is caused by a person ; and
A control unit for controlling the overall operation of the deep learning algorithm driving device by executing the expected accident determination program, the accident occurrence symptom determination program, and the risk factor determination program stored in the storage unit; Heavy equipment vehicle safety management system comprising a. According to claim 3, wherein the control unit,
a predicted accident occurrence determination unit that executes the predicted accident determination program and analyzes the images received from the image capturing device to determine an expected accident that may occur during operation of the heavy equipment vehicle prior to operation of the heavy equipment vehicle;
an accident sign determination unit that executes the accident sign determination program to determine in advance whether an accident has occurred in the work environment of the heavy equipment vehicle;
a risk factor determination unit that executes the risk factor determination program to determine whether the accident is caused by a person if there is a sign of the accident, and determines whether a risk factor exists within a danger radius during the operation of the heavy equipment vehicle; and
A risk factor notification data generation unit for generating notification data according to the results determined by the expected accident occurrence determination unit, the accident occurrence symptom determination unit, and the risk factor determination unit,
The expected accident occurrence determination unit determines in advance the expected accident in consideration of the type and work details of the heavy equipment vehicle,
A heavy equipment vehicle safety management system for determining in advance whether an accident has occurred in consideration of the moving speed, moving route, and work contents of the heavy equipment vehicle, the accident sign determining unit. 5. The method of claim 4,
The deep learning algorithm driving device is a heavy equipment vehicle safety management system that is any one of a cloud server and an edge device installed in the heavy equipment vehicle. Learning an accident pattern of a heavy equipment vehicle based on an image of an accident of an existing heavy equipment vehicle;
generating an accident occurrence symptom program and a risk factor determination program based on the learning result;
receiving a plurality of images captured by an image capturing apparatus;
Based on at least one of the plurality of images received in the above step, the expected accident determination program, the accident occurrence symptom program, and the risk factor determination program, the predicted accident that may occur in the heavy equipment vehicle before operation of the heavy equipment vehicle is 1 secondary judgment;
generating and transmitting first notification data when it is determined that the expected accident exists as a result of the predicted accident determination;
receiving an image from the image capturing device during the operation of the heavy equipment vehicle, and secondarily determining whether an accident has occurred in consideration of the current work content and the moving speed of the heavy equipment vehicle;
generating and transmitting second notification data when it is determined that there is a sign of an accident as a result of the determination of the indication of the accident;
determining whether the sign of the accident is caused by a person;
generating and transmitting third notification data when it is determined that the sign of the accident is caused by a person, and generating and transmitting fourth notification data when it is determined that the accident is not caused by a person;
receiving an image from the image capturing device during the operation of the heavy equipment vehicle and tertiarily determining an accident that may occur because a dangerous element exists around the heavy equipment vehicle or the height of an object is higher than a preset height; and
As a result of determining the presence of the risk factor, if it is determined that the risk factor exists, 5th notification data Generating and transmitting the; heavy equipment vehicle safety management method comprising a.
delete

Images