KR20230017595A - Management Robot and Method for Underground Facility - Google Patents

Abstract

The present invention relates to a facility management robot and a method thereof. In accordance with the present invention, an autonomous driving management robot photographs the inside of a tunnel through a camera, generates a three-dimensional point cloud based on the photographed image and depth information to recognize an area in which the robot cannot be driven on a three-dimensional space based on the point cloud, and sets a driving route based thereon to control a main body such that the main body is moved. Therefore, the present invention can have an effect of controlling the management robot by easily analyzing an internal environment and setting a driving route to effectively avoid an obstacle, and an effect of preventing an accident and improving safety by inspecting a facility which is confined and difficult to access.

Description

Facility management robot and its method {Management Robot and Method for Underground Facility}

The present invention relates to a facility management robot that monitors and manages the state of a facility while moving inside a facility installed underground and a method thereof.

Facilities such as power facilities or transmission lines are generally buried underground or installed in tunnels formed underground.

In the case of a power facility of a certain size, it can be installed in a space of a certain size or more in the basement, but in the case of a transmission line, it is often installed in a power outlet with a narrow width and a low ceiling.

In addition, in the case of a transmission line, since it is a facility for transmitting and supplying power from a specific point to another point, a power outlet is formed as much as the distance for supplying power.

In order to prevent breakdowns of transmission lines in power outlets, periodic inspections are required.

In order to inspect the transmission line of the power outlet, manpower directly entered the power outlet and inspected it, but since the power outlet is located underground, it is difficult to access and the internal environment is not good, so it is difficult for manpower to stay for a certain period of time. In particular, since the power outlet is narrow, it is difficult for manpower to enter. It is difficult to move within the power tunnel, so there is a limit to inspecting the transmission line.

In order to solve this problem, Korean Patent Registration No. 10-1480543 discloses a mobile monitoring unit and monitoring system for managing underground facilities. Disclosed are a mobile monitoring unit in which a rail is installed on the ceiling of an underground facility and moves through the rail to detect a state of the underground facility, and a monitoring system connected to the mobile monitoring unit to receive and manage information.

However, such a mobile monitoring unit has a problem in that it is difficult to apply when it is not initially installed in that monitoring through the mobile unit is possible only when a rail is installed in the underground space where the facility is installed.

In addition, as the mobile unit is connected to the rail of the ceiling, the distance to the facility is different depending on the size or shape of the underground facility and the height of the ceiling, and it is difficult to approach the facility installed on the floor and check its condition in detail. It is difficult, and since the mobile monitoring unit installed on the ceiling cannot access and check the facility, there is a limit to checking and monitoring the state of the facility.

Accordingly, there are increasing cases of introducing autonomous robots, but there is difficulty in driving due to obstacles located in waterways or passages formed in power outlets.

Republic of Korea Patent No. 10-1480543

The present invention was created by the above necessity, and the self-driving robot is put into a narrow power outlet, and the captured image is analyzed to analyze the internal environment of the power outlet to set a path to the target point while avoiding obstacles. Its purpose is to provide a facility management robot and its method that checks the transmission line while doing so.

In addition, the present invention provides a facility management robot and method for controlling driving in consideration of the location and size of obstacles by dividing waterways and passages based on three-dimensional data in analyzing the internal environment of a power outlet, and a method therefor. There is a purpose.

In order to achieve the above object, a facility management robot according to the present invention includes a camera that captures an image inside a facility and obtains depth information; an image processing unit which analyzes the image captured by the camera, divides the area, and detects a waterway located in the facility; a point cloud unit that converts the image and the depth information into a 3D point cloud; a calculation unit calculating a boundary line for the waterway from the point cloud and detecting a passage in the facility; an obstacle recognizing unit that detects an obstacle located on the passage and calculates its position; a driving control unit that controls driving by setting a driving path on the passage based on obstacle information; and a control unit that analyzes the image while driving and checks the state of equipment within the facility.

The point cloud unit converts the image into a 3D point cloud based on the image, the image of the waterway, and the depth information, and converts the image into a point cloud, a left channel point cloud, and a right channel point cloud, respectively. It is characterized by generating and removing misrecognized points.

The point cloud unit may remove points located higher than the waterway among points recognized as the waterway based on the coordinates of the waterway.

The calculation unit may calculate a linear equation for a boundary line of the waterway based on the point cloud, calculate a boundary line between the waterway and the passage, and detect the passage.

The operation unit may calculate an outer boundary of the waterway, then calculate an inner boundary of the waterway based on the width of the waterway, and set it as a boundary between the waterway and the passage.

The obstacle recognizing unit may set an obstacle recognition area on the passage based on a height of the management robot, a width of the passage, and a distance to be detected, and detect an obstacle located within the obstacle recognition region.

A control method of a facility management robot according to the present invention includes the steps of photographing an image through a camera and acquiring depth information; detecting a waterway located in a facility by dividing an area of the image; converting the image and the depth information into a 3D point cloud; detecting a passage in the facility by calculating a boundary line for the waterway from the point cloud; detecting an obstacle located on the passage and calculating its position; setting a driving path on the passage based on the information of the detected obstacle and driving; and checking the state of equipment in the facility by analyzing the image while driving.

The converting into a point cloud may include converting the image into a 3D point cloud based on the image, the image of the waterway, and the depth information; generating a point cloud for the image, a point cloud for the left number, and a point cloud for the right number, respectively; The method may further include removing an erroneously recognized point higher than the waterway among points recognized as the waterway based on the coordinates of the waterway.

When detecting the obstacle, setting an obstacle recognition area on the passage based on the height of the management robot, the width of the passage, and a distance to be detected to detect an obstacle located within the obstacle recognition region; and determining whether passing is possible in response to the position of the obstacle detected in the obstacle recognition area.

In the case of detecting the obstacle, obstacles located in an area other than the passage, which is a non-running area, are ignored, obstacles located on the passage are detected, and obstacles located higher than the height of the management robot are judged to be passable. to be characterized

The step of inspecting the state of the facility may include: detecting the facility by analyzing the image, and detecting an abnormality of the facility by analyzing the state of the facility; and generating and transmitting inspection data on the current location and state of the facility when an abnormality occurs in the facility.

According to one aspect of the present invention, the facility management robot and its method can easily recognize a waterway lower than a driving passage and analyze obstacles in a designated area, thereby setting an easy driving route.

According to one aspect, the present invention provides a barrier as well as an obstacle located on the floor. By recognizing the obstacles located on the upper part and analyzing the obstacles in the 3D space, the optimal path through which the robot can pass can be set and driven.

In addition, the present invention can inspect and manage narrow and difficult-to-access facilities through a robot, and has an effect of preventing accidents and improving safety through management of transmission lines.

1 is a block diagram briefly showing the configuration of a facility management robot according to an embodiment of the present invention.
2 is a diagram referenced to describe a data flow for environment analysis of a facility management robot according to an embodiment of the present invention.
3 is a diagram referenced to explain waterway detection through environment analysis of a facility management robot according to an embodiment of the present invention.
FIG. 4 is a diagram referenced to explain passage detection through environment analysis of a facility management robot according to an embodiment of the present invention.
5 is a diagram referenced to describe an obstacle recognition area of a facility management robot according to an embodiment of the present invention.
6 is a diagram referenced to describe obstacle detection of a facility management robot according to an embodiment of the present invention.
FIG. 7 is a reference diagram for explaining driving determination according to an obstacle analysis of a facility management robot according to an embodiment of the present invention.
8 and 9 are diagrams referenced to explain an environment analysis method of a facility management robot according to an embodiment of the present invention.
10 is a flowchart referenced to describe a control method of a facility management robot according to an embodiment of the present invention.

Hereinafter, the present invention will be described with reference to the accompanying drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

1 is a block diagram briefly showing the configuration of a facility management robot according to an embodiment of the present invention.

As shown in FIG. 1, the management robot according to the present invention includes a camera 170, an image processing unit 130, a point cloud unit 140, a calculation unit 150, a driving control unit 120, and an obstacle recognition unit 160. ), a data unit 190, a communication unit 210, an input/output unit 220, a driving unit 230, and a control unit 110 that controls overall operations.

In addition, the management robot 100 includes a battery (not shown) and drives by using the charging power of the battery. The management robot 100 may include a plurality of wheels (not shown) and rotation means for traveling forward, backward, left, and right.

The data unit 190 includes images captured by the camera, RGB images generated by the image processing unit, point cloud data, obstacle information, body location information, data on the structure inside the power outlet, data on the driving route, and driving Records and transmission line inspection data are stored.

In addition, the data unit 190 stores data necessary for the management robot to drive, reference data for determining failure, and data for communication with the outside.

The communication unit 210 may communicate with the management server by including at least one communication module. In some cases, the communication unit 210 may communicate with a facility provided inside the power outlet using short-range communication and receive data from the facility.

The communication unit 210 transmits the image data and the data collected from the power outlet to the outside, for example, the management server according to the control command of the control unit 110.

In addition, the communication unit 210 may transmit a notification signal when an accident occurs while the management robot is driving or the driving is disabled.

The input/output unit 220 includes an input means for inputting a user's requirements to the management robot 100 and an output means for outputting information about an operating state of the management robot 100 and a set mode. The input unit may include a touch pad, a button, a switch, and the like, and the output unit may include a display (not shown), a speaker (not shown), a lamp, and the like.

The input/output unit 220 may output a warning when the management robot 100 cannot drive or does not reach a destination. As for the warning, at least one of effect sounds, warning sounds, and voice guidance may be output through a speaker, and a message composed of a combination of at least one of letters, numbers, images, and emoticons may be displayed.

The driving unit 230 operates a motor by operating power applied from a battery and transmits driving force to a plurality of wheels through a driving shaft so that the robot drives.

The driving unit 230 transmits driving force so that a plurality of wheels for moving the management robot 100 forward, backward, left, and right rotate. The driving unit 230 may include at least one motor (not shown), a driving shaft for transmitting driving force generated from the motor to the wheel, and a plurality of other driving force transmitting means, rotating means, etc. are provided, but descriptions thereof are given below. to be omitted from

The camera 170 captures an image of the driving direction. The camera 170 may be an RGB-D camera that captures images and includes depth information. The camera 170 not only generates an image using incident light, but also generates depth information using the time it takes for light or signal to be transmitted and returned.

At least one camera 170 is provided, and may further include an adjustment means to be able to adjust the angle up and down, left and right, or to be rotatable.

In some cases, in addition to the camera 170, the robot 100 may further include lidar, radar, and ultrasonic sensors for detecting obstacles located within a certain distance around the main body to detect obstacles together with the camera. However, when minimizing the size of the robot 100, it is preferable to detect an obstacle only with the camera 170.

The image processing unit 130 converts the image of the camera 170 to generate image data. The image processing unit 130 extracts depth information from the image and generates an RGB image.

The image processing unit 130 generates image data by removing noise from images captured by each camera and adjusting brightness. The image processing unit 130 generates image data by automatically adjusting color temperature and automatically adjusting exposure by performing auto white balance and auto exposure functions.

Also, the image processor 130 may generate image data as video data or image data according to settings.

The image processing unit 130 may create region labels and object boxes by dividing regions from the image.

The image processing unit 130 extracts a channel based on the region label and the object box. The image processing unit 130 generates a left channel image and a right channel image for the extracted channel, respectively.

The point cloud unit 140 combines depth information and image data (RGB image) and converts them into a 3D point cloud. The point cloud unit 140 converts 2-dimensional image data into 3-dimensional data by combining it with depth information.

The point cloud unit 140 generates an RGB point cloud, a point cloud with left numbers, and a point cloud with right numbers through point cloud conversion. The point cloud unit 140 removes misrecognized points after converting the point cloud.

A point cloud is a set of points in a 3D space, and not only expresses the basic structure of the space as points, but also objects located in the space are expressed as points.

Since the point cloud includes information on the structure of a three-dimensional space and objects inside, the management robot 100 can easily grasp the structure of a facility such as a power tunnel and obstacles located inside.

The calculation unit 150 calculates a straight line equation for the waterway. The calculation unit 150 calculates a straight line parameter for the left number and a straight line parameter for the right number using an intuition equation based on the point cloud as the left number and the point cloud as the right number.

The obstacle recognizing unit 160 detects an object located within a certain distance from the main body of the robot 100 . The obstacle recognizing unit 160 calculates the position of the obstacle, the distance to the obstacle, and the size of the obstacle. In some cases, the obstacle recognizing unit 160 may determine the type of obstacle.

The obstacle recognizing unit 160 detects the obstacle by determining whether or not the obstacle is located in the driving direction based on the point cloud of the image, the height of the robot, and the target driving distance.

In addition, the obstacle recognizing unit 160 analyzes the detected obstacle in detail to determine whether the robot can drive or cannot drive due to the obstacle.

The travel controller 120 sets a travel route to a target point.

The driving control unit 120 sets a driving route to the target point based on the recognized passage, but changes the driving route to avoid or pass through the obstacle in response to an obstacle detected through the obstacle recognizing unit 160. .

The driving controller 120 controls the driving unit 230 based on the set driving path to control the robot to move to the target point.

When an obstacle is detected during driving, the driving control unit 120 controls the driving unit 230 based on the obstacle analysis result of the obstacle recognizing unit to avoid or pass through the obstacle to drive.

In addition, the driving control unit 120 applies a signal to the control unit 110 when an impossible driving situation occurs and controls the driving unit 230 to stop the movement of the main body.

The controller 110 controls driving of the camera 170, sets the main body to drive according to settings input through the input/output unit 220, and outputs information about the driving state.

The control unit 110 stores data on facilities in the power outlet in the data unit 190 based on data on captured images, information on detected obstacles, and images captured while driving.

The control unit 110 checks the equipment in the power outlet based on the image and determines whether there is an abnormality. The control unit 110 detects facilities in the electric power outlet based on the image analyzed by the image processing unit 130, determines the state of the facilities, and compares them with pre-stored data to detect abnormalities.

The control unit 110 stores the inspection data in the data unit 190 when the equipment is abnormal, and transmits the location information and inspection data through the communication unit 210.

The controller 110 communicates with an external server through the communication unit 210 to transmit data about the situation in the power outlet when an abnormality occurs while driving, for example, when the vehicle cannot drive to the destination before the inspection is completed.

The controller 110 transmits information about the current location through the communication unit 210 when driving is impossible.

In addition, the control unit 110 applies a control signal to the driving control unit 120 to return to the designated position.

Therefore, the management robot 100 calculates a drivable route by utilizing the RGB image information captured by the RGB-D camera, and converts the drivable route into a 3D point cloud by combining with the depth information image.

Based on the point cloud, the management robot 100 recognizes an area where the robot cannot travel in a 3D space, sets a travel path based on this, and controls the main body to move according to the set travel path.

Accordingly, the management robot 100 moves to the target point, checks the state of facilities in the power district, and returns to a designated point according to the environment in the power district.

2 is a diagram referenced to describe a data flow for environment analysis of a facility management robot according to an embodiment of the present invention.

As shown in FIG. 2 , the management robot 100 takes an image, processes data step by step, sets a route to a destination, and performs inspection.

As shown in (a) of FIG. 2, the management robot 100 recognizes the channel from the RGB image taken through the camera 170, and as shown in (b) of FIG. 2, the three-dimensional Converting to a point cloud, recognizing obstacles and setting a route as shown in (c) of FIG.

Referring to (a) of FIG. 2, the steps of recognizing a waterway are as follows.

The management robot 100 generates an RGB image from the image captured by the camera 170 (S11), and the image processing unit 130 divides an area within the image based on the RGB image (S12). The image processing unit 130 designates a region label based on the divided region and sets an object box (S13). The image processing unit 130 extracts a channel based on the region label and the object box (S14). The image processing unit 130 generates a left channel image for the left channel and a right channel image for the right channel with respect to the extracted channel based on the center point of the region (S15).

Referring to (b) of FIG. 2, the step of converting the point cloud is as follows.

The point cloud unit 140 converts the RGB image, the left channel image, the right channel image, and the depth image into a 3D point cloud (S21). Through conversion, the point cloud unit 140 generates an RGB point cloud, a point cloud with left numbers, and a point cloud with right numbers (S23). The point cloud unit 140 removes misrecognized points from among the generated point clouds.

The calculation unit 150 calculates a straight line equation for the waterway using the point cloud as the left number and the point cloud as the right number (S25). Accordingly, the calculation unit 150 calculates the linear parameter as the left number and the linear parameter as the right number (S26).

Steps for recognizing an obstacle and setting a path as shown in (c) of FIG. 2 are as follows.

The obstacle recognizing unit 160 determines the area where the robot will travel based on the RGB point cloud generated by the point cloud unit 140, the linear parameters of the left channel, the linear parameters of the right channel, the height of the robot, and the target driving distance. Obstacles located at are firstly detected (S32). The obstacle recognizing unit 160 determines whether or not there is an obstacle.

If there is an obstacle, the obstacle recognizing unit 160 analyzes the obstacle in detail (S34) based on the RGB point cloud, the linear parameters of the left number, the linear parameters of the right number, the shape of the robot, and the obstacle distance (S35). .

The obstacle recognizing unit 160 determines whether the robot can avoid or pass based on the specific position of the obstacle, and sets a path to the target point in response thereto (S36).

3 is a diagram referenced to explain waterway detection through environment analysis of a facility management robot according to an embodiment of the present invention.

As shown in (a) of FIG. 3 , the management robot 100 enters the facility and photographs the inside of the facility through the camera 170 . 2(a) is an RGB image.

In facilities such as power outlets, facilities such as transmission lines are installed, and passages 1 for moving inside are formed in the facilities. It is also possible that the passage 1 is installed in the center of the facility or formed on either side of the left or right side.

Since the facility is located underground, waterways 2 and 3 through which water flows may be formed. The waterways 2 and 3 are installed on the left and right sides of the passage 1, and may be installed on either side depending on the case.

In addition, a structure for supporting the facility, a tool or device for installing or managing the facility, and the like are disposed in the facility.

As shown in (b) of FIG. 3, the management robot 100 divides an area from an RGB image.

Since the management robot 100 inspects facilities such as power transmission lines while moving within the facility, the passage within the facility is set as a driving area, and the waterways 2 and 3 are set as non-running areas.

The image processing unit 130 detects the channels 2 and 3 from the RGB image. The waterways 2 and 3 are formed along the passage and are formed in a straight line or a curve depending on the structure of the facility. The image processing unit 130 divides the area by analyzing the RGB image, detects the left waterway 2 and the right waterway 3, which are non-driving areas, and divides the area.

The image processing unit 130 may perform object region segmentation by applying various object region segmentation algorithms in order to recognize passages and waterways in the RGB image.

Accordingly, as shown in (c) of FIG. 3 , the image processing unit 130 generates an image including the left waterway 2 among the detected waterways, that is, an image of the left waterway. In addition, as shown in (d) of FIG. 3 , the image processing unit 130 generates an image of the right water channel, which is an image including the right water channel 3, among the detected water channels.

FIG. 4 is a diagram referenced to explain passage detection through environment analysis of a facility management robot according to an embodiment of the present invention.

The point cloud unit 140 converts a left channel image, a right channel image, an RGB image, and a depth image (depth information) into a 3D point cloud. Depth image is the depth information obtained when capturing an image by an RGB-D camera.

The point cloud unit 140 may convert an RGB image and a depth image into a 3D point cloud as shown in Equation 1 using a pinhole camera model.

,

는 영상에서의 pixel 값,

,

는 영상 원점 정보와 카메라 광학축 원점과의 이동변환 값,

,

는 초점 값, depth는 깊이 영상에 기록된 값이다.here

,

is the pixel value in the image,

,

is the movement conversion value between the video origin information and the camera optical axis origin,

,

is a focus value, and depth is a value recorded in the depth image.

Since this value is a coordinate in the camera coordinate system, coordinate conversion is performed in the robot coordinate system as shown in Equation 2 below.

는 로봇 좌표계에서 i번째 포인트 클라우드, T는 카메라 좌표계에서 로봇 좌표계로 좌표변환 행렬,

는 카메라 좌표계에서 i번째 포인트 클라우드다. here,

is the ith point cloud in the robot coordinate system, T is the coordinate transformation matrix from the camera coordinate system to the robot coordinate system,

is the ith point cloud in the camera coordinate system.

Accordingly, the point cloud unit 140 generates a converted point cloud as shown in FIG. 4 .

The point cloud unit 140 generates an RGB point cloud for an RGB image, a point cloud for a left channel for a left channel, and a point cloud for a right channel for a right channel.

The point cloud unit 140 sets the previously detected left channel 2 and right channel 3 based on the point cloud. The point cloud unit 140 sets the X-axis, Y-axis, and Z-axis using the position of the robot as a coordinate axis, and processes each point.

The point cloud unit 140 removes misrecognized points to eliminate misrecognized information.

For example, the point cloud unit removes points located higher than the waterway among points recognized as the waterway based on the coordinates of the waterway. Since the coordinates of the waterways have a Z-axis value less than 0, the point cloud unit 140 generates a point cloud with numbers having a Z-axis coordinate value of 0 or more, from a point cloud with a left number having a Z-axis coordinate value of 0 or more and a point cloud with a right-hand number having a Z-axis coordinate value of 0 or more. Remove.

The calculation unit 150 detects a boundary line between the left waterway 2 and the right waterway 3 based on the processing result of the point cloud unit 140.

The calculation unit 150 calculates an equation of a straight line for the left waterway 2 and the right waterway 3 based on the left waterway point cloud and the right waterway point cloud. At this time, since the calculation unit 150 requires analysis in the X-Y plane in the robot coordinate system, the Z coordinate may be ignored and the equation of the straight line may be calculated.

, 해당 단위벡터를 지나는 한점을

, 우측수로의 직선의 R과 평행한 단위벡터를

, 해당 단위백터를 지나는 한점을

이라고 설정한다. The operation unit 150 generates a unit vector parallel to the straight line L in the left channel.

, a point passing through the unit vector

, the unit vector parallel to R of the straight line in the right channel

, a point passing through the unit vector

set as

Although there is not much difference in the slope of the straight line between the left and right sides of the actual waterway, a slight error may occur in the process of recognizing it from the image and estimating the equation of the straight line.

와, 오른쪽 직선 R의 평행한 단위 벡터

의 평균단위벡터를 최종적으로 직선 L 및 R과 평행한 단위벡터로 변경한다. For simplicity, the operation unit 150 is a unit vector parallel to the left straight line L.

and unit vector parallel to the right-hand straight line R

The average unit vector of is finally changed to a unit vector parallel to the straight lines L and R.

The average unit vector is expressed by Equation 3 below.

At this time, in the case of the equation of the calculated straight line, it represents the outer boundary lines 13 and 14 outside the waterway, not the boundary line between the waterway and the passage.

Since the width of the channel is determined by standard, the calculation unit 150 changes the equation of the straight line using Equation 4 below for the obtained straight line.

The calculated linear equation is as follows, and the graph thereof is shown in (a) of FIG.

The calculation unit 150 calculates the inner boundary lines 11 and 12 on the passage side among the boundary lines of the waterway using the calculated linear equation. In addition, since the inner boundary lines 11 and 12 are the boundary lines between the channel and the passage, the calculation unit 150 sets the passage 1 based on them.

Accordingly, the calculation unit 150 may calculate a straight line parameter as a left number and a straight line parameter as a right number for the boundary lines 11 and 12 .

5 is a diagram referenced to describe an obstacle recognition area of a facility management robot according to an embodiment of the present invention.

As shown in FIG. 5 , the management robot 100 detects an obstacle located in a passage.

When the passage 1 is set based on the right boundary 11 of the left waterway 2 and the left boundary 12 of the right waterway 3, the management robot 100 is located on the passage 1 obstacles (21, 22) to be detected.

Since a plurality of supports for supporting the power transmission line are installed inside the facility, the management robot 100 detects obstacles within a certain distance since the plurality of supports may be detected as obstacles when detecting obstacles in the entire area. do.

In addition, since the management robot 100 travels along the aisle 1 and does not travel to other areas, obstacles located in areas other than the aisle are ignored.

The obstacle recognizing unit 160 sets an obstacle recognition area 15 within a certain distance from the main body on the passage 1 and detects obstacles included in the corresponding area.

The obstacle recognition area 15 can be set to the maximum robot height (b) vertically, the length (a) of the passage horizontally, and a certain distance (d) from the robot in thickness.

That is, the management robot 100 detects an obstacle located within a height of the robot within a range of a certain distance d from the robot on the passage.

When the obstacle recognition area 15 is expressed in the X-Y plane, it is as shown in FIG. 8(b).

When an obstacle is not located within the obstacle recognition area 15, the obstacle recognizing unit 160 applies a signal indicating that there is no obstacle to the controller 110, and the controller 110 controls the driving controller 120 to reach the target point. Set the driving route to drive.

In addition, the obstacle recognizing unit 160 may detect an obstacle by setting a new obstacle recognition area based on point T of FIG. .

The obstacle recognizing unit 160 continuously sets an obstacle recognition area while driving to detect obstacles located on the passage.

6 is a reference diagram for explaining obstacle detection of a facility management robot according to an embodiment of the present invention, and FIG. 7 is a reference for explaining driving determination according to an obstacle analysis of a facility management robot according to an embodiment of the present invention. It is the way to become

When an obstacle is located in the obstacle recognizing area 15, the obstacle recognizing unit 160 analyzes the obstacle in detail to calculate its position.

As shown in (a) of FIG. 6 , the obstacle recognizing unit 160 detects the first obstacle 21 .

The obstacle recognizing unit 160 analyzes the first obstacle 21 in detail and calculates its position and size. The obstacle recognizing unit 160 analyzes the first obstacle on the plane where the obstacle is located.

As shown in (a) of FIG. 9, the plane at the distance d0 to the first obstacle 21 is detected as a reference.

In addition, the obstacle recognizing unit 160 determines whether the management robot 100 can drive through or avoid the obstacle.

As shown in (a) of FIG. 7, when the first obstacle 21 is located, the obstacle recognizing unit 160 determines the size of the area excluding the obstacle and the management robot since the first obstacle 21 is located on the right side of the passage. By comparing with the cross-sectional area 50 of , it is possible to determine whether driving is possible.

As shown in (b) of FIG. 6 , the obstacle recognizing unit 160 detects the second obstacle 22 .

The obstacle recognizing unit 160 determines the distance of the second obstacle 22, analyzes the position of the second obstacle 22, and determines whether passing is possible.

As shown in (b) of FIG. 7, the obstacle recognizing unit 160 is located on either side of the passage 1 of the second obstacle 22, and considering the cross-sectional area 50 of the management robot 100, Since the second obstacle 22 is located on the driving path of the management robot 100 and the collision area 31 occurs, it is determined that the passage is impossible.

On the other hand, as shown in (c) of FIG. 7 , when the third obstacle 23 is detected, the obstacle recognizing unit 160 analyzes the position of the third obstacle 23 and determines the height b of the robot. When located at a higher place, it is determined that the management robot 100 can pass through the lower part of the third obstacle 23 .

The obstacle recognizing unit 160 applies information about the detected obstacle and whether it is possible to pass the obstacle to the driving control unit 120 .

The driving controller 120 sets a driving route to the target point based on obstacle information, the target point, and information on passages.

The travel control unit 120 sets a path that travels within the range of the recognized passage 1 and avoids or passes through the recognized obstacle.

In addition, when an obstacle is detected by the obstacle recognizing unit 160 while driving, the driving controller 120 corrects the driving path based on information about the new obstacle.

The driving control unit 120 may apply various driving equations to set a driving route and control the driving unit accordingly. For example, the driving control unit 120 may control driving by using a differential driving method in which two signals are used, such as a straight ahead signal and a turn signal.

The equation for calculating the driving signal is schematized as shown in (b) of FIG.

라고 할 때, 차동 구동 방식에 따라, 원 운동을 실시하므로, 직진 속도를 v, 회전속도를 w라고 하면 아래의 수학식5와 같다. The driving control unit 120 determines the driving target point of the management robot 100.

When it is said, since circular motion is performed according to the differential drive method, if the straight speed is v and the rotational speed is w, Equation 5 is shown below.

where r represents the radius of rotation. By calculating the r value, the rotational speed can be calculated for an arbitrary straight speed.

와 r과의 관계는 다음 수학식6과 같다.At this time,

The relationship between and r is shown in Equation 6 below.

Here, when organized using the trigonometric formula, the rotational speed w is as shown in Equation 7 below.

Accordingly, the driving control unit 120 sets a driving route and controls the driving unit 230 according to the calculated rotational speed so that the main body moves.

8 and 9 are diagrams referenced to explain an environment analysis method of a facility management robot according to an embodiment of the present invention.

(a) of FIG. 8 is a linear equation for a waterway set by the calculation unit 150 described in FIG. 4 .

After calculating the left boundary line 13 of the left waterway 2 and the right boundary line 14 of the right waterway 3, the calculation unit 150 calculates the left waterway (14) based on the width of the designated waterway. The linear equation for the right boundary line 11 of 2) and the left boundary line 12 of the right channel 3 can be calculated as shown in FIG. 8(a).

In addition, the calculation unit 150 sets the passage 1 based on the boundary lines 11 and 12 .

8(b) shows the obstacle recognition area 15 described in FIG. 5 in the X-Y plane.

The width of the obstacle recognition area 15 is the width (a) of the passage, the length is the height of the management robot 100, and the thickness is the distance (d) to check the obstacle from the management robot 100.

In this case, the distance d may be set based on an area that can be detected by a camera or sensor.

9(a) is an X-Y plane at the position of an obstacle detected by the obstacle recognizing unit 160.

The obstacle recognizing unit 160 analyzes the obstacle based on a plane at a distance d0 to the first obstacle 21 .

9(b) is a diagram for calculating a driving signal in order to set a driving route to a target point of the driving control unit 120.

라고 할 때, 차동 구동 방식에 따라, 원 운동을 실시하므로, 직진 속도 v, 회전속도 w를 산출하여 주행을 제어한다. The driving control unit 120 determines the driving target point of the management robot 100.

In this case, since the circular motion is performed according to the differential driving method, the driving is controlled by calculating the straight speed v and the rotational speed w.

10 is a flowchart referenced to describe a control method of a facility management robot according to an embodiment of the present invention.

As shown in FIG. 10 , the management robot 100 enters the facility and captures an image through a camera 170 (S310).

The image processing unit 130 analyzes the captured image, divides the image area (S320), and extracts the waterways 2 and 3 located in the facility from the image (S330).

The point cloud unit 140 converts the captured image (RGB image), the channel image, and the depth image into a 3D point cloud (S340).

Also, the point cloud unit 140 removes erroneously recognized points (S350).

Accordingly, the management robot 100 acquires three-dimensional information about structures within the facility.

The calculation unit 150 calculates a linear equation for the waterway based on the 3D point cloud, and detects the passage 1 based on this.

The calculation unit 150 detects the boundary line of the waterway based on the 3D point cloud, calculates the boundary line between the passage and the waterway from the boundary line of the waterway, and calculates a linear parameter for the boundary line of the passage (S360).

The obstacle recognizing unit 160 sets an obstacle recognition area 15 on the passage and detects an obstacle located in the corresponding area (S370).

The obstacle recognizing unit 160 determines whether or not there is an obstacle (S380), and if there is an obstacle, analyzes the obstacle in detail to calculate its location (S390).

The obstacle recognizing unit 160 determines whether it is possible to pass through the obstacle in consideration of the position of the obstacle and the size of the management robot 100 (S400).

The obstacle recognizing unit 160 determines whether it is possible to pass through the obstacle and applies the obstacle information to the driving control unit 120 .

When there is no obstacle or it is possible to pass through the obstacle, the driving controller 120 calculates a target point (S410), and sets a driving path that avoids or passes the obstacle to the target point (S420).

In addition, the driving control unit 120 controls the driving unit 230 by calculating the straight speed v and the rotational speed w of the wheels based on the set driving path.

Accordingly, the management robot 100 travels along the passage (S430).

The management robot 100 checks the condition of the track through an image captured by the camera 170 while driving (S440).

On the other hand, when it is determined that passing through the obstacle is impossible, the travel controller 120 controls the driving unit 230 to stop the robot from traveling (S450). The driving control unit 120 applies a signal for disabling driving to the control unit 110 .

The control unit 110 outputs a warning about a driving stop and an obstacle, and transmits information therefor through the communication unit 210 (S460).

The control unit 110 applies a control signal to the driving control unit 120 so that the management robot 100 returns to a starting point or a designated position.

Accordingly, the driving controller 120 controls the driving unit 230 so that the management robot returns (S470).

Meanwhile, if the detected obstacle moves after the warning is output, the obstacle recognizing unit 160 may re-detect the obstacle to re-determine whether passing is possible.

When the driving is possible according to the movement of the obstacle, the driving control unit 120 may control the driving by recalculating the driving path.

Therefore, the facility management robot according to the present invention generates a 3D point cloud based on the image captured by the camera and the depth information, recognizes an area where the robot cannot drive in the 3D space based on the point cloud, Based on this, a driving route can be set and driving can be controlled so that the main body moves according to the set driving route.

Accordingly, the present invention can easily inspect and manage facilities that are difficult to access and have poor internal environments.

Although the present invention has been described with reference to the embodiments shown in the drawings, it should be noted that this is only exemplary and various modifications and equivalent other embodiments are possible from those skilled in the art to which the technology pertains. will understand Therefore, the true technical protection scope of the present invention should be defined by the claims below.

1: aisle 1, 3: waterway
11, 12, 13, 14: boundary line 15: obstacle recognition area
21, 22, 23: Obstacles
100: management robot
110: control unit 120: driving control unit
130: image processing unit 140: point cloud unit
140: calculation unit 160: obstacle recognition unit

Claims (19)

a camera that captures an image within the facility and obtains depth information;
an image processing unit which analyzes the image captured by the camera, divides the area, and detects a waterway located in the facility;
a point cloud unit that converts the image and the depth information into a 3D point cloud;
a calculator configured to calculate a boundary line for the waterway from the point cloud and detect a passage in the facility;
an obstacle recognizing unit that detects an obstacle located on the passage and calculates its position;
a driving control unit that controls driving by setting a driving path on the passage based on obstacle information; and
A management robot comprising: a controller that analyzes the image while driving and checks the state of equipment within the facility. According to claim 1,
The image processing unit divides the area from the image, detects the area of the passage, which is a drivable area, and the area of the waterway, which is a non-drivable area, and generates a left waterway image and a right waterway image for the waterway. Characterized in that management robot. According to claim 1,
The point cloud unit converts the image into a 3D point cloud based on the image, the image of the waterway, and the depth information, and converts the image into a point cloud, a left channel point cloud, and a right channel point cloud, respectively. A management robot characterized by generating and removing misrecognized points. According to claim 1,
The management robot according to claim 1 , wherein the point cloud unit removes a point located higher than the waterway among the points recognized as the waterway based on the coordinates of the waterway. According to claim 1,
The management robot according to claim 1 , wherein the calculation unit calculates a linear equation for a boundary line of the waterway based on the point cloud, calculates a boundary line between the waterway and the passage, and detects the passage. According to claim 4,
The management robot, characterized in that the operation unit calculates an outer boundary of the waterway, then calculates an inner boundary of the waterway based on the width of the waterway, and sets it as a boundary between the waterway and the passage. According to claim 1
The management robot according to claim 1 , wherein the obstacle recognizing unit ignores obstacles located in an area other than the passage, which is a non-running area, and detects an obstacle located on the passage. According to claim 1,
The obstacle recognizing unit sets an obstacle recognition area on the passage based on a height of the management robot, a width of the passage, and a distance to be detected, and detects an obstacle located within the obstacle recognition region. . According to claim 8,
The obstacle recognizing unit determines whether passing is possible in response to the position of the detected obstacle, and determines that an obstacle at a position higher than the height of the management robot is passable. According to claim 1,
The driving control unit sets a driving path based on the information of the obstacle and the target point within the range of the passage, and applies a straight ahead signal and a turn signal to the driving unit according to the driving path to control driving in a differential driving method. , A management robot characterized in that it corrects the driving path to avoid or pass through obstacles detected during driving. According to claim 1,
The control unit analyzes the image to determine a change in the state of the facility, outputs a warning, and transmits related data when an abnormality is detected or when driving is impossible due to the obstacle. photographing an image through a camera and acquiring depth information;
detecting a waterway located in a facility by dividing an area of the image;
converting the image and the depth information into a 3D point cloud;
detecting a passage in the facility by calculating a boundary line for the waterway from the point cloud;
detecting an obstacle located on the passage and calculating its position;
setting a driving path on the passage based on the information of the detected obstacle and driving; and
A control method of a management robot comprising: checking the state of equipment in the facility by analyzing the image while driving. According to claim 12,
In the step of detecting the waterway,
A method for controlling a management robot, characterized in that, from the image, an area of the passage, which is a drivable area, and an area of the waterway, which is a non-drivable area, are detected, and an image of the area of the waterway is generated. According to claim 12,
The step of converting the point cloud,
converting the image into a 3D point cloud based on the image, the image of the waterway, and the depth information;
generating a point cloud for the image, a point cloud for the left number, and a point cloud for the right number, respectively; and
The method of controlling a management robot further comprising the step of removing an erroneously recognized point higher than the waterway among the points recognized as the waterway based on the coordinates of the waterway. According to claim 12,
In case of detecting the above obstacle,
Setting an obstacle recognition area on the passage based on the height of the management robot, the width of the passage, and a distance to be detected to detect an obstacle located within the obstacle recognition region; and
The management robot control method further comprising determining whether passage is possible in response to the position of the obstacle detected in the obstacle recognition area. According to claim 12,
In case of detecting the above obstacle,
Control of the management robot, characterized in that ignoring obstacles located in an area other than the passage, which is a non-running area, detecting obstacles located on the passage, and determining that obstacles located higher than the height of the management robot are passable method. According to claim 12,
The driving step is
Within the range of the passage, the driving route is set based on the information of the obstacle and the position of the target point, and the driving is controlled by a differential driving method by applying a straight ahead signal and a turn signal to the driving unit according to the driving route, A control method of a management robot, characterized in that for modifying the driving path to avoid or pass through obstacles detected during driving. According to claim 12,
The driving step is
outputting a warning when the obstacle cannot be passed;
Transmitting data about the location and driving state of the obstacle; and
A control method of a management robot further comprising the step of returning to a starting point or a designated location. According to claim 12,
The step of checking the status of the facility,
detecting the facility by analyzing the image, and detecting an abnormality in the facility by analyzing the state of the facility; and
The method of controlling a management robot further comprising generating and transmitting inspection data for a current location and a state of the facility when an abnormality occurs in the facility.

Images