KR20220161899A - Monitoring device to detect the risk of forward collision of forklift - Google Patents

Abstract

Disclosed is a forklift forward collision risk detection and monitoring device. According to the present invention, the position of a part covered by a lifted moving object and a dangerous object can be displayed based on a forward image and an image taken by a time of flight (TOF) camera. The forklift forward collision risk detection and monitoring device comprises: a first camera (110) installed on an upper end of a mast assembly (210) of a forklift (200) to photograph the anterior site; a second camera (120) installed at a lower end of the mast assembly (210) of the forklift (200) to photograph the anterior site; a control unit (130) for overlaying at least a portion of a captured image (500) of the second camera (120) on a captured image (400) of the first camera (110) to generate an output image (600) in which a portion covered by a moving object (230) lifted from a viewpoint of the first camera (110) is displayed; and a display unit (140) displaying the generated output image (600).

Description

Forward collision risk detection monitoring device of forklift {MONITORING DEVICE TO DETECT THE RISK OF FORWARD COLLISION OF FORKLIFT}

The present invention relates to a device for detecting and monitoring the risk of forward collision of a forklift, and more particularly, based on an image taken from the front and an image taken by a TOF camera, a part covered by a moving target object lifted and the position of the dangerous object are detected. It relates to a front collision risk detection and monitoring device for a forklift.

In general, a forklift truck is a special type of industrial truck used to lift or unload a load. It is a vehicle.

The structure of such a forklift truck basically has a vehicle body capable of being driven, and a mast assembly capable of being manipulated and driven to load cargo is installed in front of the vehicle body.

Here, the mast assembly is composed of an outer mast and multi-stage inner masts installed so as to overlap each other inside the outer mast, and a carriage is assembled to be movable vertically on the inner mast. The carriage and inner mast are moved up and down by primary and secondary lift cylinders.

In addition, a pair of forks are installed in the carriage of the mast assembly, and in the case of the pair of forks, intervals in the width direction can be adjusted on the carriage so that cargo can be directly lifted.

Here, instead of the fork, other attachments such as a hinged bucket, a side shift, a load stabilizer, and a rotating fork may be mounted.

At this time, the attachment is configured to receive hydraulic fluid through the attachment line.

However, since the mast and fork are installed in front of the driver's seat in the process of loading cargo using a forklift, it is difficult for the operator to secure the front view for the overall operation of raising or lowering the cargo as well as the operation of the worker moving while the cargo is loaded. There was a problem.

In particular, there is a problem that there is always a risk of collision in the workplace as the saya is covered by the cargo and the mast in the process of lifting and transporting the cargo.

Republic of Korea Patent Publication No. 76199 (2016.06.30.) discloses a forklift vehicle body, a mast assembly provided in front of the vehicle body, a carriage that moves up and down along the mast rail of the mast assembly, and provided in front of the carriage A forklift comprising a pair of forks, a camera provided at the lower part of the carriage to detect the position of the fork, and a monitor installed in the driver's seat, A camera bracket having a camera at the bottom and a slot formed in the vertical direction And, as configured to include a guide bracket assembled to slide with the camera bracket by a guide member, and an elastic member connecting the camera bracket and the guide bracket, the camera is always located below the fork during work at height, and the efficiency of work and A fork camera mounting structure of a forklift that can improve convenience is known.

However, since the camera for taking a front image is located at the lower center of the vehicle body corresponding to the mast or on both sides of the vehicle body, it is still difficult to secure a front view for the driver in a state where cargo is loaded on both forks.

Korean Patent Laid-open Publication No. 10-2016-0076199 (2016.06.30.)

In order to solve this problem, the present invention is a front collision risk detection monitoring device for a forklift that displays the position of the dangerous object and the part covered by the lifted moving object based on the image taken by the front and the TOF camera is intended to provide

In order to achieve the above object, an embodiment of the present invention is an apparatus for detecting and monitoring a risk of forward collision of a forklift, comprising: a first camera installed on top of a mast assembly of a forklift and photographing the front; a second camera installed at the lower end of the mast assembly of the forklift to photograph the front side; a control unit that overlays at least a portion of an image captured by a second camera on an image captured by the first camera to generate an output image such that a portion covered by the moving object lifted from the viewpoint of the first camera is displayed; and a display unit displaying the generated output image.

In addition, the output image according to the above embodiment is characterized in that a virtual fork image indicating a position of a fork covered by the lifted moving target object is displayed.

In addition, the second camera according to the embodiment is characterized in that it is a Time of Flight (TOF) camera.

In addition, the control unit according to the embodiment is characterized in that it detects the height of the fork and the collision risk object in front from the image taken by the second camera.

In addition, the control unit according to the embodiment detects a plane region formed by points through RANSAC (RANdom Sample Consensus)-based clustering of images taken by a TOF camera, and sets a predetermined ROI (Region Region) based on coordinate information of the plane region. It is characterized in that a dangerous object is detected from points corresponding to the object of interest, and a dangerous object display image is generated and displayed on an output image as the dangerous object is detected.

The present invention has the advantage of preventing the risk of collision by displaying the position of the dangerous object and the part covered by the lifted moving object based on the image taken from the front and the image taken by the TOF camera.

1 is an exemplary view illustrating an apparatus for detecting and monitoring a risk of forward collision of a forklift truck according to an embodiment of the present invention;
2 is a block diagram illustrating a configuration of an apparatus for detecting and monitoring a risk of forward collision of a forklift truck according to an embodiment of the present invention.
3 is an exemplary view illustrating an operation of a monitoring device for detecting and monitoring a risk of forward collision of a forklift truck according to an embodiment of the present invention;
4 is an exemplary view showing a first camera-captured image of the device for detecting and monitoring a risk of forward collision of a forklift truck according to the embodiment of FIG. 3;
5 is an exemplary diagram illustrating an image captured by a second camera of the device for detecting and monitoring a risk of forward collision of a forklift truck according to the embodiment of FIG. 3;
6 is an exemplary view illustrating a composite of images taken by first and second cameras of the apparatus for detecting and monitoring a risk of forward collision of a forklift truck according to the embodiment of FIG. 3;
FIG. 7 is an exemplary view showing a dangerous subject displayed based on images captured by first and second cameras of the apparatus for detecting and monitoring a risk of forward collision of a forklift truck according to the embodiment of FIG. 3;

Hereinafter, the present invention will be described in detail with reference to preferred embodiments of the present invention and accompanying drawings, but the same reference numerals in the drawings will be described on the premise that they refer to the same components.

Prior to describing specific details for the implementation of the present invention, it should be noted that configurations not directly related to the technical subject matter of the present invention are omitted within the scope of not disturbing the technical subject matter of the present invention.

In addition, the terms or words used in this specification and claims are meanings and concepts consistent with the technical idea of the invention based on the principle that the inventor can define the concept of appropriate terms to best describe his/her invention. should be interpreted as

In this specification, the expression that a certain part "includes" a certain component means that it may further include other components, rather than excluding other components.

In addition, terms such as ".. unit", ".. unit", and ".. module" refer to units that process at least one function or operation, which may be classified as hardware, software, or a combination of the two.

In addition, the term "at least one" is defined as a term including singular and plural, and even if at least one term does not exist, each component may exist in singular or plural, and may mean singular or plural. would be self-evident.

In addition, the singular or plural number of each component may be changed according to embodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an apparatus for detecting and monitoring a risk of forward collision of a forklift truck according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary view illustrating a front collision risk detection and monitoring device of a forklift according to an embodiment of the present invention, and FIG. 2 is a configuration of a forklift front collision risk detection and monitoring device according to an embodiment of the present invention. Figure 3 is a block diagram shown for explanation, Figure 3 is an exemplary diagram shown to explain the operation of the monitoring device for detecting the risk of forward collision of a forklift according to an embodiment of the present invention, Figure 4 is a forklift according to the embodiment of Figure 3 5 is an exemplary view showing an image captured by a second camera of the front collision risk detection and monitoring device of a forklift according to the embodiment of FIG. 3, and FIG. 6 3 is an exemplary diagram showing the synthesis of images taken by the first and second cameras of the device for detecting and monitoring the risk of forward collision of a forklift truck according to the embodiment of FIG. 3, and FIG. It is an exemplary view showing a dangerous object based on the images captured by the first and second cameras of the device.

1 to 7, the monitoring device 100 for detecting a risk of forward collision of a forklift truck according to an embodiment of the present invention detects a moving target object lifted based on an image taken from the front and an image taken by a TOF camera. It may be configured to include a first camera 110, a second camera 120, a controller 130, and a display unit 140 so that the position of the portion covered by the screen and the dangerous object may be displayed.

The first camera 110 is installed on the top of the mast assembly 210 of the forklift 200 and photographs the front direction of the forklift 200, and a camera using a CCD sensor, a camera using a CMOS sensor, or an optical signal It can be composed of a camera using an arbitrary sensor that converts and outputs an electrical signal.

The second camera 120 is installed at the lower end of the mast assembly 210 of the forklift 200 to photograph the front direction of the forklift 200, and is preferably a Time of Flight (TOF) camera. have.

The TOF camera measures the distance (d) through a value converted into a distance by measuring the flight time of IR from the TOf sensors arranged in an array form.

The controller 130 overlays at least a part of the second output image 500 taken by the second camera 120 on the first output image 400 taken by the first camera 110 to obtain an output image 600. Create and control to be displayed.

That is, when the moving target object 230 is lifted on the forklift 200 as shown in FIG. 3 and there is a dangerous object 300 or 310 in the forward direction of the forklift 200, the first camera 110 is set to FIG. 4 As such, the first output image 400 in which the dangerous objects 300 and 310 are covered by the moving target object 230 is displayed.

In addition, in a state in which the moving object 230 is lifted by the forklift 200, the second camera 120 may display the second output image 500 including the dangerous objects 300 and 310 as shown in FIG. 5. have.

When the moving target object 230 lifted through the mast assembly 210 covers the first camera 110 from the viewpoint of the first camera 110, the controller 130 controls the second camera to display the covered portion. Overlaying at least a part of the image taken in 120 with the image taken by the first camera 110, the position of the fork 220 covered by the lifted moving object 230 is the output image 600 As shown in FIG. 6 , an output image 600 in which the virtual fork image 610 is additionally displayed at the bottom of the moving target object 230 may be generated.

In addition, the control unit 130 can detect the height of the fork 220 and the collision risk object in front from the image captured by the second camera 120 .

That is, the control unit 130 recognizes an area corresponding to a fork and an area with a dangerous object such as a person in the captured image of the TOF camera.

The control unit 130 multiplies the light (IR) stem unit vector of the corresponding pixel position by the depth value to calculate the position of each pixel in the 3D space of the captured image, and synthesizes the 3D point cloud data generate

In addition, the control unit 130 detects a two-dimensional plane area formed by points through RANSAC (RANdom SAmple Consensus)-based clustering of images captured by the TOF camera.

In addition, the control unit 130 detects a dangerous object from points corresponding to a preset Region of Interest (ROI) based on coordinate information of a 2D flat area.

That is, the control unit 130 assumes that the Z coordinates of all points are '0' and uses a circle centered on an arbitrarily extracted point as a model.

However, it may be a cylinder standing vertically in a 3D space.

In addition, for all points projected on the XY plane, the controller 130 counts, for example, '1' if the distance from the center point is less than or equal to the preset 'IN area', and the distance from the center point is preset 'OUT area'. area' or less, for example, '-2' is counted for clustering.

In addition, the control unit 130 classifies the highest point of the point extracted by RANSAC as, for example, a key, measures and classifies the width (eg, head width) from the key to a certain distance below, and divides it into another The width (for example, shoulder width) of the range below the distance is measured and classified, and if the divided head width and shoulder width fall within the predetermined range, it is recognized as a person.

At this time, if the radius of the circle used for human recognition is large, arm recognition may be difficult because the arm is treated as the body and removed, so only a range narrower than the radius used in the cluster ring may be recognized as the body and removed.

In addition, as the controller 130 detects a dangerous object in the second output image 500 through RANSAC, the detected dangerous object 300 and the dangerous object 1 310 may be displayed on the output image 600. As shown in FIG. 7 , a risk object display image 620 and a risk object display image 1 630 are generated and displayed on the output image 600 to be displayed.

The display unit 140 displays the output image 600 generated by the controller 130 so that it can be visually checked.

Accordingly, it is possible to prevent a risk of collision by displaying the position of the dangerous object and the part covered by the lifted moving object based on the image taken by the TOF camera and the image taken from the front.

As described above, although it has been described with reference to the preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

In addition, the drawing numbers described in the claims of the present invention are only described for clarity and convenience of explanation, but are not limited thereto, and in the process of describing the embodiments, the thickness of lines or the size of components shown in the drawings, etc. may be exaggerated for clarity and convenience of description.

In addition, the above-mentioned terms are terms defined in consideration of functions in the present invention, which may change according to the intention or custom of the user or operator, so the interpretation of these terms should be made based on the contents throughout this specification. .

In addition, even if it is not explicitly shown or described, a person skilled in the art to which the present invention belongs can make various modifications from the description of the present invention to the technical idea according to the present invention. Obviously, it is still within the scope of the present invention.

In addition, the above embodiments described with reference to the accompanying drawings are described for the purpose of explaining the present invention, and the scope of the present invention is not limited to these embodiments.

100: monitoring device
110: first camera
120: second camera
130: control unit
140: display unit
200: forklift
210: mast assembly
220: fork
230: moving target object
300: Dangerous target
310: risk target 1
400: first output image
500: second output image
600: output image
610: virtual fork image
620: Danger target display image
630: Danger target display image 1

Claims (5)

a first camera 110 installed on top of the mast assembly 210 of the forklift 200 to photograph the front;
a second camera 120 installed at the lower end of the mast assembly 210 of the forklift 200 to photograph the front;
By overlaying at least a portion of the captured image 500 of the second camera 120 on the captured image 400 of the first camera 110, the moving target object 230 lifted from the viewpoint of the first camera 110 a control unit 130 generating an output image 600 so that the obscured portion is displayed; and
A front collision risk detection and monitoring device for a forklift including; a display unit 140 for displaying the generated output image 600. According to claim 1,
The output image 600 is a forklift forward collision risk detection monitoring device, characterized in that a virtual fork image 610 indicating the position of the fork 220 covered by the lifted moving target object 230 is displayed. According to claim 2,
The second camera 120 is a time of flight (TOF) camera, characterized in that the forward collision risk detection monitoring device of the forklift. According to claim 3,
The control unit 130 detects the height of the fork 220 and the forward collision risk object in the image taken by the second camera 120. According to claim 4,
The control unit 130 detects a plane region formed by points through RANSAC (RANdom SAmple Consensus)-based clustering of an image taken by a TOF camera,
Detecting a risk object from points corresponding to a preset Region of Interest (ROI) based on the coordinate information of the plane area;
An apparatus for detecting and monitoring a risk of forward collision of a forklift, characterized in that when a dangerous object is detected, dangerous object display images (620, 630) are generated and displayed on the output image (600).

Images