KR20230086189A - Automatic safety braking system of forklifts for avoiding collision - Google Patents

Abstract

Disclosed is an automatic safety braking device for a forklift truck to prevent collision. A camera module that recognizes an object; A lidar module for recognizing 3D space and objects; a sensor fusion module that fuses the object recognized by the camera module and the three-dimensional space and object recognized by the LIDAR module according to the corresponding space and direction; a 3D space object creation module for generating a 3D space by fusion of the sensor fusion module and creating a 3D object in the created 3D space; an AI human recognition module for recognizing a person among the 3D objects created by the 3D space/object generation module by AI analysis of the corresponding object recognized by the camera module; A forklift/person distance recognition module that recognizes a distance to a person recognized by the AI person recognition module using the distance recognized by the lidar module is configured. According to the above-described automatic safety braking device for a forklift truck for collision prevention, it is configured to converge sensing information of a camera and a rider, recognize a person through AI analysis, give an alarm, and automatically apply the brake, so that the operator can secure the field of view. It is effective in predicting and coping with a collision with a person on the route in advance even in an unpredictable situation. In addition, by predicting the path of the forklift and the path of a person and anticipating whether or not to collide and the point of collision, the automatic brake is operated, thereby preventing the operation of the forklift from being stopped too frequently.

Description

Automatic safety braking system for forklift trucks for collision avoidance

The present invention relates to a safety braking device for preventing collision, and more particularly, to an automatic safety braking device for a forklift truck to prevent collision.

Since cargo is loaded at the front end of the forklift, there is a problem in that the operator's view is blocked from the driver's seat at the rear end.

Since it becomes difficult for workers to secure a view while driving, there is a high possibility of a collision accident, and in particular, a high risk of collision with people around them.

By attaching radar or other sensors to the forklift, the brake of the forklift can be forcibly operated in a dangerous situation.

However, due to the nature of the work of the forklift, there are many situations in which other workers must work together. Therefore, if the brake is operated whenever a person or object is detected in the vicinity, there is a problem in that work efficiency is too low.

That is, too often there are many situations where it is impossible to stop, and it is impossible to operate such a system, especially in narrow aisles.

Accordingly, there is a demand for a means capable of preventing a collision accident by reflecting the work characteristics of a forklift.

Registered Patent Publication 10-2325652 Registered Patent Publication 10-2117046

An object of the present invention is to provide an automatic safety braking device for a forklift to prevent collision.

Another object of the present invention is to provide an automatic safety braking method for a forklift to prevent a collision.

An automatic safety braking device for a forklift truck for collision prevention according to the object of the present invention described above includes a camera module for recognizing an object; A lidar module for recognizing 3D space and objects; a sensor fusion module that fuses the object recognized by the camera module and the three-dimensional space and object recognized by the LIDAR module according to the corresponding space and direction; a 3D space object creation module for generating a 3D space by fusion of the sensor fusion module and creating a 3D object in the created 3D space; an AI human recognition module for recognizing a person among the 3D objects created by the 3D space/object generation module by AI analysis of the corresponding object recognized by the camera module; It may be configured to include a forklift/person distance recognition module that recognizes a distance to a person recognized by the AI person recognition module using the distance recognized by the lidar module.

Here, it may be configured to further include a forklift/person collision estimation module that predicts a collision between the forklift and the person using the distance to the person recognized by the forklift/person distance recognition module.

and a safety level calculation module for calculating a safety level according to a distance to a person recognized by the forklift/person distance recognition module and a collision with a person predicted by the forklift/person collision prediction module; It may be configured to further include a safety level output module that outputs the safety level calculated by the safety level calculation module.

and a speaker module outputting a corresponding alarm sound according to the safety level output from the safety level output module. It may be configured to further include a display module displaying a corresponding image according to the safety level output from the safety level output module.

And it may be configured to further include an automatic brake control module for automatically controlling the brake device according to the safety level output from the safety level output module.

An automatic safety braking method for a forklift truck for collision prevention according to another object of the present invention described above includes recognizing an object by a camera module and recognizing a 3D space and an object by a lidar module; fusing, by a sensor fusion module, the object recognized by the camera module and the three-dimensional space and object recognized by the LIDAR module according to the corresponding space and direction; generating, by a 3D space object generation module, a 3D space by fusion of the sensor fusion module, and generating a 3D object on the created 3D space; Recognizing a person among the 3D objects created by the 3D space/object generation module by AI-analyzing the corresponding object recognized by the camera module by the AI human recognition module; The forklift/person distance recognition module may be configured to include recognizing a distance to a person recognized by the AI person recognition module using the distance recognized by the lidar module.

Here, the forklift/person collision estimation module may be configured to further include predicting a collision between the forklift and the person using the distance to the person recognized by the forklift/person distance recognition module.

and calculating, by a safety level calculation module, a safety level according to a distance to a person recognized by the forklift/person distance recognition module and a collision with a person predicted by the forklift/person collision prediction module; The safety level output module may be configured to further include outputting the safety level calculated by the safety level calculation module.

and outputting, by a speaker module, a corresponding alarm sound according to the safety level output from the safety level output module, and displaying, by a display module, a corresponding image according to the safety level output from the safety level output module. can

The automatic brake control module may further include automatically controlling the brake device according to the safety level output from the safety level output module.

According to the above-described automatic safety braking device for a forklift truck for collision prevention, it is configured to converge sensing information of a camera and a rider, recognize a person through AI analysis, give an alarm, and automatically apply the brake, so that the operator can secure the field of view. It is effective in predicting and coping with a collision with a person on the route in advance even in an unpredictable situation.

In addition, by predicting the path of the forklift and the path of a person and anticipating whether or not to collide and the point of collision, the automatic brake is operated, thereby preventing the operation of the forklift from being stopped too frequently.

also, The automatic safety braking device 100 for a forklift to prevent collision may be applied not only to a forklift but also to heavy equipment such as a bulldozer, a poclain, and an excavator.

1 is a block diagram of an automatic safety braking device for a forklift truck for collision prevention according to an embodiment of the present invention.
2 is a flowchart of an automatic safety braking method for a forklift to prevent collision according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in specific contents for practicing the invention. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an automatic safety braking device for a forklift truck for collision prevention according to an embodiment of the present invention.

Referring to FIG. 1 , an automatic safety braking device 100 for a forklift for collision prevention according to an embodiment of the present invention includes a camera module 101, a lidar module 102, and a sensor fusion module 103. , 3D space object generation module 104, AI (artificial intelligence) human recognition module 105, human movement path tracking module 106, human movement speed tracking module 107, forklift/human distance recognition module 108 , wheel encoder module 109, handling monitoring module 110, forklift moving path tracking module 111, forklift moving speed tracking module 112, forklift path/speed database 113, forklift iterative path extraction module 114, forklift/human collision prediction module 115, safety level calculation module 116, safety level output module 117, speaker module 118, display module 119, automatic brake control module 120 , a safe mode lock/unlock module 121, and a black box module 122.

Hereinafter, a detailed configuration will be described.

The camera module 101 may be configured to recognize objects. Here, the object may include both people and objects.

LiDAR module 102 may be configured to recognize three-dimensional space and objects. The lidar module 102 may be configured to recognize the position of an object in a 3D space at a certain distance around the forklift. Objects here may include both people and objects.

The sensor fusion module 103 may be configured to fuse an object recognized by the camera module 101 and a 3D space and object recognized by the LIDAR module 102 according to the corresponding space and direction. That is, each sensing information may be fused corresponding to the positions of the camera module 101 and lidar module 102 attached to the forklift.

The 3D space object creation module 104 may be configured to generate a 3D space by fusion of the sensor fusion module 103 and create a 3D object on the created 3D space.

The AI human recognition module 105 may be configured to recognize a person among the 3D objects created by the 3D space/object creation module 104 by AI-analyzing the corresponding object recognized by the camera module 101 . The AI human recognition module 105 may recognize a person in the camera image through AI image object analysis.

The human movement path tracking module 106 may be configured to track the movement path of a person recognized by the AI human recognition module 105 using the fusion result of the sensor fusion module 103 . The person movement path tracking module 106 may track a person's movement path in a 3D space.

The human movement speed tracking module 107 may be configured to track the movement speed of a person recognized by the AI human recognition module 105 using the fusion result of the sensor fusion module 103 . The human movement speed tracking module 107 may track the movement speed of a person in a 3D space.

The forklift/person distance recognition module 108 may be configured to recognize a distance to a person recognized by the AI person recognition module 105 using the distance recognized by the lidar module 102 .

The wheel encoder module 109 may be configured to measure wheel rotation of the forklift. Specifically, the wheel encoder module 109 can measure the angle of rotation of the wheel.

The handling monitoring module 110 may be configured to monitor handling information of the forklift. Specifically, the handling monitoring module 110 may monitor the handling angle.

The forklift moving path tracking module 111 may be configured to track the moving path of the forklift using wheel rotation measured by the wheel encoder module 109 and handling information monitored by the handling monitoring module 110 . The moving distance of the forklift can be known through the wheel rotation angle, and the moving path can be known by adding handling information.

The forklift moving speed tracking module 112 may be configured to track the moving speed of the forklift using wheel rotation measured by the wheel encoder module 109 and handling information monitored by the handling monitoring module 110 . Calculating the wheel rotation angle per hour gives the forklift moving speed.

The forklift path/speed database 113 may be configured such that the moving path tracked by the forklift moving path tracking module 111 and the moving speed tracked by the forklift moving speed tracking module 112 are synchronized and stored. That is, the movement path and the movement speed at each position on the movement path may be synchronized and stored.

The forklift iterative path extraction module 114 may be configured to extract a past iterative path and a corresponding moving speed using the moving path and moving speed of the forklift truck stored in the forklift path/speed database 113 . Due to the nature of the forklift's work, the main task is to load cargo from point A and move it to point B. Accordingly, the forklift repeated path extraction module 114 may track the repeated path through past moving paths that repeatedly appear, and may extract the moving speed at each location on the repeated path. The movement speed of each position in the iterative path may have a similar movement speed. This is because the moving speed may appear similar in a straight section, a left/right turn section, etc. of a movement path.

The forklift/person collision prediction module 115 includes the distance to the person recognized by the forklift/person distance recognition module 108, the person's moving path tracked by the person moving path tracking module 106, and the person's moving speed tracking module 107 ), the forklift moving path tracked by the forklift moving path tracking module 111, and the forklift moving speed tracked by the forklift moving speed tracking module 112 are used to determine the collision between the forklift and the person. It can be configured to anticipate.

Specifically, it is as follows.

First, the forklift/person collision prediction module 115 calculates the distance to the person recognized by the forklift/person distance recognition module 108, the person's moving path tracked by the person's moving path tracking module 106, and the person's moving speed tracking module ( In step 107, an expected movement path and expected movement speed of the person may be calculated using the movement speed of the person tracked.

The forklift/person collision estimation module 115 uses the forklift movement path tracked by the forklift movement path tracking module 111 and the forklift movement speed tracked by the forklift movement speed tracking module 112 to determine the forklift's expected movement path. and an expected movement speed.

Then, the forklift/person collision prediction module 115 may be configured to predict a collision between the forklift and the person and a collision point using the expected movement path and expected movement speed of the person and the expected movement path and expected movement speed of the forklift. .

That is, the forklift/person collision prediction module 115 can primarily predict a collision at the current location through the current distance between the forklift and the person in the 3D space, and secondarily, the direction and movement of the current movement path of the forklift Whether or not the collision occurs and the location of the collision can be predicted using the speed, the direction of the current movement path of the person, and the movement speed.

Meanwhile, the forklift/person collision prediction module 115 may be configured to calculate an expected movement path and an expected movement speed of the forklift by referring to the repeated path of the forklift extracted by the forklift repeated path extraction module 114 and the corresponding movement speed. . That is, after checking where the current moving path, position, and moving speed of the forklift are on the past iterative path and moving speed, the subsequent moving path and moving speed can be predicted by referring to the repeated path and corresponding moving speed .

The forklift/person collision prediction module 115 may be configured to output a predicted collision point between the forklift and the person through the display module 119 .

The safety level calculation module 116 calculates a safety level according to predetermined criteria according to the distance to a person recognized by the forklift/person distance recognition module 108 and a collision with a person predicted by the forklift/person collision prediction module 115. It can be configured to calculate.

The safety level calculation module 116 calculates safety level 4 when the distance to the person recognized by the forklift/person distance recognition module 108 is less than a predetermined threshold value, and the forklift/person collision prediction module 115 calculates the distance to the person as safety level 4. When a collision is expected, safety level 3 is calculated, when a collision with a non-human object is expected using an object detected by the rider 102, safety level 2 is calculated, and when a collision is not expected, a safety level is calculated. It can be configured to yield 1.

The safety level output module 117 may be configured to output the safety level calculated by the safety level calculation module 116 .

The speaker module 118 may be configured to output a corresponding alarm sound according to the safety level output from the safety level output module 117 . For example, in safety level 4, an emergency alarm with a higher volume than in safety level 3 can be output.

The display module 119 may be configured to display a corresponding image according to the safety level output from the safety level output module 117 . For example, in safety level 3, collision prediction and collision prediction points may be indicated.

The automatic brake control module 120 may be configured to automatically control the brake device 10 according to the safety level output from the safety level output module 117 .

Specifically, the automatic brake control module 120 may control the brake device 10 to be automatically executed in the safety level 4 and safety level 3 output from the safety level output module 117 .

The safe mode lock/unlock module 121 may be configured to lock or unlock the safe mode which causes the safe level output module 117 to output a safe level. Due to the nature of the forklift operation, when working with other workers close to the forklift, the brake is inevitably operated from time to time in the safety mode operating as above. In this case, it can be configured to work by unlocking safe mode.

The black box module 122 tracks the moving path of the person tracked by the person moving path tracking module 106, the moving speed of the person tracked by the person moving speed tracking module 107, and the moving path of the forklift tracked by the tracking module 111. It can be configured to generate and store black box information using the moving path of the forklift, the moving speed of the forklift tracked by the forklift moving speed tracking module 112, and the image of the corresponding object recognized by the camera module 101. there is.

On the other hand, the automatic safety braking device 100 for a forklift for collision prevention according to the present invention is implemented for a forklift, but can be applied to a bulldozer, a forklift, an excavator, etc. in addition to a forklift.

2 is a flowchart of an automatic safety braking method for a forklift to prevent collision according to an embodiment of the present invention.

Referring to FIG. 2 , a camera module 101 recognizes an object, and a lidar module 102 recognizes a 3D space and an object (S101).

Next, the sensor fusion module 103 fuses the object recognized by the camera module 101 and the 3D space and object recognized by the LIDAR module 102 according to the corresponding space and direction (S102).

Next, the 3D space object creation module 104 creates a 3D space by fusion of the sensor fusion module 103 and creates a 3D object on the created 3D space (S103).

Next, the AI human recognition module 105 AI-analyzes the object recognized by the camera module 101 to recognize a person among the 3D objects created by the 3D space/object generation module 104 (S104) .

Next, the forklift/person distance recognition module 108 recognizes the distance to the person recognized by the AI person recognition module 105 using the distance recognized by the lidar module 102 (S105).

Next, the forklift/person collision estimation module 115 predicts a collision between the forklift and the person using the distance to the person recognized by the forklift/person distance recognition module 108 (S106).

Next, the safety level calculation module 116 calculates a safety level according to the distance to a person recognized by the forklift/person distance recognition module 108 and a collision with a person predicted by the forklift/person collision prediction module 115 Do (S107).

Next, the safety level output module 117 outputs the safety level calculated by the safety level calculation module 116 (S108).

Next, the speaker module 118 outputs a corresponding alarm sound according to the safety level output from the safety level output module 117, and the display module 119 outputs a corresponding alarm sound according to the safety level output from the safety level output module 117. The corresponding image is displayed (S109).

Next, the automatic brake control module 120 automatically controls the brake device 10 according to the safety level output from the safety level output module 117 (S110).

Although described with reference to the above embodiments, those skilled in the art can understand that the present invention can be variously modified and changed without departing from the spirit and scope of the present invention described in the claims below. There will be.

101: camera module
102: lidar module
103: sensor fusion module
104: 3D space object generation module
105: AI human recognition module
106: human movement path tracking module
107: human movement speed tracking module
108: forklift/person distance recognition module
109: wheel encoder module
110: handling monitoring module
111: forklift movement path tracking module
112: forklift moving speed tracking module
113 Forklift path/speed database
114: Forklift repeated path extraction module
115: forklift/person collision prediction module
116: safety level calculation module
117: safety level output module
118: speaker module
119: display module
120: brake automatic control module
121: safe mode lock/unlock module
122: black box module

Claims (10)

In the automatic safety braking device for a forklift for preventing collision,
A camera module that recognizes an object;
A lidar module for recognizing 3D space and objects;
a sensor fusion module that fuses the object recognized by the camera module and the three-dimensional space and object recognized by the LIDAR module according to the corresponding space and direction;
a 3D space object creation module for generating a 3D space by fusion of the sensor fusion module and creating a 3D object in the created 3D space;
an AI human recognition module for recognizing a person among the 3D objects created by the 3D space/object generation module by AI analysis of the corresponding object recognized by the camera module;
An automatic safety braking device for a forklift truck for collision prevention including a forklift/person distance recognition module that recognizes a distance to a person recognized by the AI person recognition module using the distance recognized by the lidar module.
According to claim 1,
Automatic forklift for collision avoidance, characterized in that it is configured to further include a forklift/person collision prediction module that predicts a collision between the forklift and the person using the distance to the person recognized by the forklift/person distance recognition module. Safety Braking Device.
According to claim 2,
a safety level calculation module for calculating a safety level according to a distance to a person recognized by the forklift/person distance recognition module and a collision with a person predicted by the forklift/person collision estimation module;
The automatic safety braking device for a forklift truck for collision prevention, characterized in that it is configured to further include a safety level output module that outputs the safety level calculated by the safety level calculation module.
According to claim 3,
a speaker module outputting a corresponding alarm sound according to the safety level output from the safety level output module;
The automatic safety braking device for a forklift truck for collision prevention, characterized in that it is configured to further include a display module for displaying a corresponding image according to the safety level output from the safety level output module.
According to claim 4,
The automatic safety braking device for a forklift truck for collision prevention, characterized in that it is configured to further include an automatic brake control module that automatically controls the brake device according to the safety level output from the safety level output module.
Recognizing an object by a camera module and recognizing a 3D space and an object by a lidar module;
fusing, by a sensor fusion module, the object recognized by the camera module and the three-dimensional space and object recognized by the LIDAR module according to the corresponding space and direction;
generating, by a 3D space object generation module, a 3D space by fusion of the sensor fusion module, and generating a 3D object on the created 3D space;
Recognizing a person among the 3D objects created by the 3D space/object generation module by AI-analyzing the corresponding object recognized by the camera module by the AI human recognition module;
An automatic safety braking method for a forklift truck for collision prevention comprising the step of recognizing, by a forklift/person distance recognition module, a distance to a person recognized by the AI person recognition module using the distance recognized by the lidar module.
According to claim 6,
Forklift for collision avoidance, characterized in that the forklift/person collision prediction module is configured to further include predicting a collision between the forklift and the person using the distance to the person recognized by the forklift/person distance recognition module. For automatic safety braking method.
According to claim 7,
calculating, by a safety level calculation module, a safety level according to a distance to a person recognized by the forklift/person distance recognition module and a collision with a person predicted by the forklift/person collision prediction module;
The automatic safety braking method for a forklift truck for collision prevention, characterized in that the safety level output module is configured to further include outputting the safety level calculated by the safety level calculation module.
According to claim 8,
The speaker module outputs a corresponding alarm sound according to the safety level output from the safety level output module, and the display module displays the corresponding image according to the safety level output from the safety level output module. An automatic safety braking method for forklift trucks for collision avoidance.
According to claim 9,
The automatic safety braking method for a forklift truck for collision prevention, characterized in that it is configured to further include the step of automatically controlling the brake device according to the safety level output from the safety level output module by the automatic brake control module.

Images