KR20220064111A - Ship block transportation equipment based on spatial information and method for detecting obstacle using the same - Google Patents

Abstract

The present invention relates to a ship block transportation apparatus based on spatial information for detecting an obstacle. According to the present invention, the ship block transportation apparatus based on spatial information for detecting an obstacle comprises: a Lidar unit attached to the ship block transportation apparatus and transmitting a Lidar signal to the surroundings to acquire one or more point clouds through a reflected signal; a grouping unit grouping the detected one or more point clouds into clusters and measuring an azimuth and distance of the point clouds; a camera unit attached to the ship block transportation apparatus and photographing a driving image; an identification unit applying the driving image to a pre-learned deep learning algorithm and identifying the obstacle included in the driving image as a person or a thing; and a determination unit determining whether there is a possibility to collide with the obstacle or not by using a preset variable safety area, the azimuth and distance information of the point clouds, and the identification information on the person or the thing. As such, according to the present invention, the ship block transportation apparatus is able to detect the azimuth and distance of point clouds acquired from the Lidar signal, apply the driving image photographed by the camera to a learned deep learning algorithm, and identify the obstacle, thereby determining whether there is a possibility that the ship block transportation apparatus can collide with the obstacle or not. In addition, when there is a possibility to collide with the obstacle, the ship block transportation apparatus provides an alert on the collision to the driver of the ship block transportation apparatus, and thus, prevents a collision and jamming accident with the obstacle.

Description

A ship block transport device based on spatial information and an obstacle detection method using the same

The present invention relates to an apparatus for transporting a ship block based on spatial information and an obstacle detection method using the same, and more particularly, to detecting an obstacle by fusing detection information obtained from a lidar signal and driving image information photographed using a camera. It relates to a ship block transport device based on spatial information and an obstacle detection method using the same.

Recently, a technique using a combination of distance and shape information acquisition devices is mainly used. Here, the process of integrating the 2D/3D distance and shape information obtained from each sensor can be viewed as one of the essential pre-processing processes for driving route calculation and vehicle control.

In general, a large number of field workers in various fields are deployed to perform work, and safety accidents occur very frequently. Accordingly, although safety management for field workers is very important, the number of managers who manage many field workers is very limited, so there is a limit in preventing safety accidents in advance by accurately understanding all the work situations that occur in the field.

In particular, the ship block transport device travels in the shipyard yard in a state in which a large load is loaded in the shipyard, and has a different appearance and operation method than a general vehicle. The ship block transport device has a very long length, and since the driver's seat is located at a lower height than the load, it is difficult to secure a view to the front. Therefore, one ship block transport device works with 2 to 4 signalers, and communicates collision-related information through a radio through the preceding signal number. However, there is a limit to relying on the ship block transport device to secure a movement route through the number of signals. In addition, the ship block transport device has a high probability of collision with surrounding obstacles and stenosis accidents.

In order to prevent such safety accidents, there are Lidar, Radar, 2D camera, 3D scanner, stereo vision, etc. sensors that recognize the surrounding environment, and distance and shape information can be obtained from these sensors. there is.

The technology underlying the present invention is disclosed in Korean Patent Publication No. 10-2020-0076989 (published on June 30, 2020).

The technical problem to be achieved by the present invention is to provide a spatial information-based ship block transport apparatus that detects obstacles by fusing detection information obtained from a lidar signal and driving image information photographed using a camera, and an obstacle detection method using the same it is for

According to an embodiment of the present invention for achieving this technical problem, in the spatial information-based ship block transport device for detecting obstacles, it is attached to the ship block transport device and transmits a lidar signal to the surroundings to collect the reflected signal. A lidar unit that acquires one or more point clouds through a clustering unit that clusters one or more detected point clouds into clusters, respectively, and detects an azimuth and distance to the point cloud, is attached to the ship block transport device, A camera unit for photographing a driving image, an identification unit for identifying an obstacle included in the driving image as a person or an object by applying the driving image to a previously learned deep learning algorithm, and a preset variable safety area and the point cloud and a determination unit that determines whether a collision with an obstacle is possible using the azimuth and distance information and the identification information on the person or thing.

A variable safety area is set using the braking distance and safe turning radius according to the traveling speed of the ship block transport device, and fixed safety using shape information about the weight, center of gravity, length, width, height and loading position of the load It may further include a safe area setting unit for setting the area.

When there is a possibility of a collision between the load and the obstacle, it may further include an alarm unit for providing a collision notification to the driver of the ship block transport device.

The variable safety area may be set using a braking distance for preventing collision with the obstacle when an obstacle in the driving direction of the ship block transport device exists within a reference distance.

The camera unit and the lidar unit may be integrally attached to the front of the ship block transport device.

According to another embodiment of the present invention, in the method of detecting an obstacle using a ship block transport device based on spatial information, a lidar signal is transmitted to the surroundings through a lidar attached to the ship block transport device and reflected through the signal. Acquiring one or more point clouds, grouping the detected one or more point clouds into clusters, respectively, detecting an azimuth and distance to the point cloud, photographing a driving image through a camera attached to the ship block transport device Step, applying the driving image to a pre-learned deep learning algorithm to identify obstacles included in the driving image as a person or an object, and azimuth and distance information of a preset variable safety area and the point cloud, the person or determining whether there is a possibility of collision with an obstacle by using the identification information on the object.

As described above, according to the present invention, by detecting the azimuth and distance to the point cloud obtained from the LIDAR signal, and identifying obstacles by applying the driving image captured through the camera to the learned deep learning algorithm, the ship block transportation device and It is possible to determine whether there is a possibility of collision with an obstacle.

In addition, when there is a possibility of collision with an obstacle, a collision with an obstacle and a stenosis accident can be prevented by providing a notification whether a collision exists to the driver of the ship block transport apparatus.

1 is a block diagram showing the configuration of a ship block transport apparatus based on spatial information according to an embodiment of the present invention.
Figure 2a is a plan view of a ship block transport apparatus to which a safety area according to an embodiment of the present invention is applied.
Figure 2b is a side view of the ship block transport device to which the safety area according to the embodiment of the present invention is applied.
3 is a flowchart illustrating an obstacle detection method using a space information-based ship block transport apparatus according to an embodiment of the present invention.
FIG. 4 is a diagram for explaining step S320 of FIG. 3 .
FIG. 5 is a diagram for explaining step S330 of FIG. 3 .
FIG. 6 is a view for explaining step S340 of FIG. 3 .
FIG. 7 is a view for explaining step S350 of FIG. 3 .
FIG. 8 is a diagram for explaining step S360 of FIG. 3 .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Then, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them.

Hereinafter, a ship block transport apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 1 .

1 is a block diagram showing the configuration of a ship block transport apparatus based on spatial information according to an embodiment of the present invention.

As shown in FIG. 1 , the ship block transport apparatus 100 according to an embodiment of the present invention includes a lidar unit 120 , a clustering unit 130 , a camera unit 140 , an identification unit 150 , and a determination unit 160 . ), a safe area setting unit 110 and a notification unit 170 .

Ship block transport apparatus 100 according to an embodiment of the present invention means a transporter for loading and disembarking at a loading site in a shipyard.

First, the safety area setting unit 110 may set a variable safety area and a fixed safety area by using the previously stored driving-related information and loading block information of the transportation device.

That is, the safety area setting unit 110 sets a variable safety area using the braking distance and the safe turning radius according to the traveling speed of the ship block transport device 100, and sets the weight and center of gravity, length, width, and height of the load. and setting the fixed safety area using the shape information about the loading position.

Next, the lidar unit 120 is attached to the front of the ship block transport device 100, and searches the front, left and right in consideration of the variable safety area. In this case, the lidar unit 120 may transmit a lidar signal to the surroundings and acquire one or more point clouds through the reflected signal.

Next, the clustering unit 130 may cluster one or more point clouds obtained from the lidar unit 120 into clusters, respectively, and detect an azimuth and a distance to the point cloud.

Next, the camera unit 140 takes a driving image using a camera attached to the ship block transport apparatus 100 . In this case, the camera unit 140 may be installed in front of the ship block transport device 100 , and may be integrated with the lidar unit 120 and attached thereto.

Next, the identification unit 150 may recognize a person or an object included in the driving image as an obstacle by applying the captured driving image to a pre-learned deep learning algorithm.

Next, the determination unit 160 determines whether there is a possibility of collision with an obstacle by using the preset variable safety area, azimuth and distance information of the point cloud, and identification information on a person or thing.

And, when it is determined that there is a possibility of a collision between the load 200 and the obstacle, the notification unit 170 may provide a collision notification to the driver of the ship block transport apparatus 100 .

Hereinafter, the setting of the variable safety region 111 and the fixed safety region 112 by the safety region setting unit 110 will be described with reference to FIGS. 2A and 2B .

Figure 2a is a plan view of a ship block transport device to which a safety area is applied according to an embodiment of the present invention, Figure 2b is a side view of a ship block transport device to which a safety zone is applied according to an embodiment of the present invention.

As shown in FIGS. 2A and 2B , the ship block transport apparatus 100 according to the embodiment of the present invention is illustrated as loading the load 200 .

Here, the load 200 loaded on the ship block transport device 100 loads from several tens of tons to a maximum of one thousand tons and runs in the shipyard yard, so to prevent the load 200 from pouring, the center of gravity of the load 200 is held Since it is important, the safe area setting unit 110 receives information on the length, width, height and loading position of the load 200 in advance, and sets the fixed safe area 112 based on the information.

When an obstacle located around the ship block transport device 100 deviates from the fixed safety area 112 , the collision probability is low, and when the obstacle is located within the fixed safety area 112 , the collision probability is high.

Therefore, the safety area setting unit 110 knows in advance the weight, length, width, height information of the load 200 loaded in the ship block transport device 100 and the loading position and center of gravity of the load 200, and drives, The fixed safety area 112 may be set based on the shape information of the load 200 .

In addition, the ship block transport device 100 changes the braking distance and the safe turning radius according to the traveling speed. Based on the braking distance and the safe turning radius, the safety area setting unit 110 sets the variable safety area 111 can be set.

At this time, the braking distance means the distance the vehicle moves from the moment the brake is applied until the vehicle is completely stopped. Therefore, the braking distance is inevitably long due to the load. That is, when the speed of the ship block driving device 100 increases, the braking distance becomes longer. In general, the running speed of the ship block driving device 100 including the load 200 in the shipyard is maintained at 10 to 15 km/h. .

At this time, in the case of the ship block driving device 100, only the main road is allowed to pass, and since collision accidents with obstacles frequently occur due to the size of the load 200, information about the turning radius is required. The turning radius is the size of the circle drawn while turning. The slower the speed, the smaller the turning radius.

Accordingly, the safety area setting unit 110 sets the variable safety area 111 based on the driving information of the ship block transportation device 100 , and an obstacle in the traveling direction of the ship block transportation device 100 exists within the reference distance. In this case, the variable safety area 111 is set as shown in FIGS. 2A and 2B because it is necessary to set a safety area equivalent to the braking distance in order to prevent collision with an obstacle.

Hereinafter, an obstacle detection method using the ship block transport apparatus 100 according to an embodiment of the present invention will be described with reference to FIGS. 3 to 8 .

3 is a flowchart illustrating an obstacle detection method using a space information-based ship block transport apparatus according to an embodiment of the present invention.

First, the safety area setting unit 110 provides information about the driving-related transportation device for the braking distance and safe turning radius according to the previously stored driving speed, and the weight and center of gravity of the load 200, length, width, height, and loading position. A safety area is set using the load shape information (S310).

Here, the safe area includes a variable safety area 111 and a fixed safe area 112 , and the method for the safe area setting unit 110 to set the safe area has been described with reference to FIGS. 2A and 2B , and overlapping descriptions is omitted.

Next, the lidar unit 120 searches the surrounding environment in consideration of the variable safety area 111 and acquires one or more point clouds through the transmitted/received signal (S320).

FIG. 4 is a diagram for explaining step S320 of FIG. 3 .

As shown in FIG. 4 , the lidar unit 120 searches forward, left, and right within the variable safety area 111 . In this case, the lidar unit 120 uses an 800-2000 nm laser, emits a laser pulse, measures the time for the light to be reflected from a surrounding target object and returns to measure the distance to the object and position coordinates.

Here, the lidar unit 120 generates at least one point cloud by transmitting a lidar signal in the moving direction, and receiving a signal reflected by the obstacle when the obstacle exists in the variable safety area 111 . In this case, the detection range of the lidar unit 120 may be 100 m, and a signal that is not received and a point having a distance of 100 m or more may be excluded.

Then, the clustering unit 130 clusters one or more point clouds obtained from the lidar unit 120 into clusters, respectively, and detects an azimuth and a distance to the point cloud ( S330 ).

FIG. 5 is a diagram for explaining step S330 of FIG. 3 .

The signal received from the lidar unit 120 in step S320 forms at least one point cloud, and is clustered into a plurality of points as shown in FIG. 5 . At this time, the plurality of clustered points are clustered in units of one or more clusters 300a, ??, and 300d, and the clustering unit 130 can measure the azimuth and distance for each cluster 300a, ??, and 300d. there is.

For example, as shown in FIG. 5 , the cluster 300a is located at a point spaced apart by about 15.2m at an angle of 40° with respect to the ship block transport device 100 , and the cluster 300b is 28.9m at an angle of 33° It is detected as being located at a point spaced apart as much as possible. In addition, it may be detected that the cluster 300c is positioned at a point spaced apart by about 22 m at an angle of -11°, and the cluster 300d is positioned at a point spaced apart by about 9.3 m at an angle of -18°. At this time, the distance between the ship block transport device 100 and each cluster is determined as the shortest distance.

That is, the clustering unit 130 may precisely detect the location coordinates of the cluster by using the distance and the azimuth between the lidar unit 120 and the cluster.

Next, the camera unit 130 takes a driving image (S340). In this case, the camera unit 130 may be attached to the front of the ship block transport device 100 , and may be installed integrally with the lidar unit 120 .

FIG. 6 is a view for explaining step S340 of FIG. 3 .

As shown in FIG. 6 , the camera unit 130 captures a forward driving image of the ship block transport apparatus 100 .

As shown in FIG. 6 , for convenience of description, it is assumed that one or more obstacles 400b , 400c , and 400d exist in the image captured by the camera unit 130 .

Next, the identification unit 140 applies the obstacle included in the captured driving image to the pre-learned deep learning algorithm to identify a person or an object with respect to the obstacle ( S350 ).

FIG. 7 is a view for explaining step S350 of FIG. 3 .

As shown in FIG. 7 , the captured driving image is applied to a pre-learned deep learning algorithm. In this case, the deep learning algorithm is a model trained to identify the type of obstacle. Accordingly, the identification unit 140 identifies a person or an object for each of the obstacles 400b , 400c , and 400d included in the driving image.

Also, the identification unit 140 may determine whether a person or thing is movable based on the learned algorithm.

For example, the identification unit 140 may determine the obstacle 400c identified by the learned deep learning algorithm as a movable obstacle when it is determined as a person (worker). In addition, when the identified obstacle 400b is determined to be a load, the identification unit 140 may determine that it is an obstacle that cannot be moved.

Next, the determination unit 150 determines whether a collision with the recognized obstacle is possible using the variable safety area set in step S310 and the azimuth and distance information of the point cloud obtained in step S320, and identification information on a person or object. do (S360).

FIG. 8 is a diagram for explaining step S360 of FIG. 3 .

As shown in FIG. 8 , the determination unit 150 fuses the variable safety area 111 and the location information and driving image information of each of the obstacles 400b, 400c, and 400d to the ship block transport apparatus 100 and the obstacle 400b. , 400c, 400d), it is determined whether the collision is possible, and the possibility of collision between the load 200 and the obstacles 400b, 400c, and 400d loaded in the ship block transport apparatus 100 is determined.

For example, when each obstacle 400b, 400c, 400d is adjacent to the variable safety area 111 in FIG. 8, the load 200 and each obstacle 400b, 400c, 400d) can be judged to be highly likely to collide.

And the notification unit 160 provides a collision notification to the driver of the ship block transport device 100 when there is a possibility of a collision between the load 200 loaded on the ship block transport device 100 and an obstacle (S370).

That is, the ship block transport apparatus 100 provides a collision detection result to the driver, thereby preventing collisions and stenosis accidents between the load 200 and the obstacles 400b, 400c, and 400d.

As described above, according to an embodiment of the present invention, by detecting the azimuth and distance to the point cloud obtained from the LiDAR signal, and applying the driving image photographed through the camera to the learned deep learning algorithm to identify obstacles, the ship block It may be determined whether there is a possibility of collision between the transport apparatus 100 and the obstacle.

In addition, when there is a possibility of a collision with an obstacle, a collision with an obstacle and a stenosis accident can be prevented by providing a notification whether a collision exists to the driver of the ship block transport apparatus 100 .

Although the present invention has been described with reference to the embodiment shown in the drawings, which is merely exemplary, it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: ship block transport device, 110: safety area setting unit,
111: variable safety zone, 112: fixed safety zone;
120: lidar unit, 130: clustering unit,
140: camera unit, 150: identification unit,
160: judgment unit, 170: notification unit,
200: load, 300a, 300b, 300c, 300d: cluster;
400b, 400c, 400d: obstacles

Claims (10)

In the spatial information-based ship block transport device for detecting obstacles,
A lidar unit that is attached to the ship block transport device and transmits a lidar signal to the surroundings to acquire one or more point clouds through the reflected signal;
A clustering unit that clusters the detected one or more point clouds into clusters, respectively, and detects an azimuth and a distance to the point cloud;
A camera unit attached to the ship block transport device and photographing a driving image;
An identification unit that applies the driving image to a previously learned deep learning algorithm to identify obstacles included in the driving image as a person or an object, and
Ship block transport apparatus comprising a determination unit for determining whether a collision with an obstacle is possible using a preset variable safety area, azimuth and distance information of the point cloud, and identification information on the person or thing. According to claim 1,
A variable safety area is set using the braking distance and safe turning radius according to the traveling speed of the ship block transport device, and fixed safety using shape information about the weight, center of gravity, length, width, height and loading position of the load Ship block transport apparatus further comprising a safety area setting unit for setting an area. 3. The method of claim 2,
Ship block transport device further comprising an alarm unit for providing a notification whether collision to the driver of the ship block transport device when there is a possibility of a collision between the load and the obstacle. 3. The method of claim 2,
The variable safety region is
When an obstacle exists within a reference distance in the driving direction of the ship block transport device, the ship block transport device is an area set using a braking distance to prevent collision with the obstacle. According to claim 1,
The camera unit and the lidar unit is a ship block transport device that is integrally attached to the front of the ship block transport device. In a method for detecting an obstacle using a ship block transport device based on spatial information,
Transmitting a lidar signal to the surroundings through the lidar attached to the ship block transport device and acquiring one or more point clouds through the reflected signal;
clustering the detected one or more point clouds into clusters, respectively, and detecting an azimuth and a distance to the point cloud;
Taking a driving image through a camera attached to the ship block transport device;
applying the driving image to a pre-learned deep learning algorithm to identify obstacles included in the driving image as a person or an object; and
An obstacle detection method comprising the step of determining whether a collision with an obstacle is possible using a preset variable safety area, azimuth and distance information of the point cloud, and identification information on the person or thing. 7. The method of claim 6,
A variable safety area is set using the braking distance and safe turning radius according to the traveling speed of the ship block transport device, and fixed safety using shape information about the weight, center of gravity, length, width, height and loading position of the load Obstacle detection method further comprising setting an area 7. The method of claim 6,
When there is a possibility of a collision between the load and the obstacle, the method further comprising the step of providing a collision notification to the driver of the ship block transport device. 8. The method of claim 7,
The variable safety region is
When an obstacle exists within a reference distance in the driving direction of the ship block transport device, the obstacle detection method is an area set using a braking distance to prevent a collision with the obstacle. 7. The method of claim 6,
The camera and the lidar obstacle detection method is integrally attached to the front of the ship block transport device.

Images