KR102446104B1 - Self-driving stair climbing transport robot using deep learning and its operation method - Google Patents

Abstract

Disclosed are an autonomous stair climbing transport robot using deep learning and an operating method thereof. The stair climbing robot according to an embodiment includes a main body, a camera sensor attached to the front of the main body to collect image data, a moving unit, a first gyro sensor and a processor attached to the moving unit to collect tilt data of the moving unit, The processor searches the stairs by inputting the image data into the pre-trained neural network, and controls the moving unit to move the robot to a predetermined position based on the position movement signal determined based on the position between the searched stairs and the robot, Controls the moving unit so that the robot can climb the stairs based on the climbing signal, determines whether the robot arrives on a flat ground based on the inclination data, and controls the moving unit to stop the robot based on the flat land arrival signal.

Description

Self-driving stair climbing transport robot using deep learning and its operation method

The following embodiments relate to an autonomous driving stair climbing transport robot using deep learning and an operation method thereof, and more specifically, to a method for recognizing various types of stairs based on deep learning and for the robot to go up to a target floor.

As 'non-face-to-face' enters the daily life beyond the trend, 'courier service' is also growing rapidly along with the growth of the e-commerce market. As the actual deployment of service robots that deliver goods and food instead of people is accelerated, Broad Branch Market, a grocery store in Washington, DC, has closed its stores and only operates robot delivery and pick-up orders, while delivering to local customers with Starship Technology’s delivery robot. is doing

Various robots are being activated in accordance with the era of non-face-to-face service, but their biggest problem is that they are only suitable for driving on flat ground, so it is difficult to stably run on steep slopes, bumps, and indoor/outdoor stairs of buildings. Due to the nature of the country's hilly terrain, houses are densely located on sloping areas and there are many stair terrains in various places.

Embodiments are to provide a method for recognizing various types of stairs based on deep learning and for a robot to go up to a target floor.

Embodiments are intended to provide a robot capable of stable driving not only on flat ground, but also on stairs and uneven terrain.

Embodiments are to provide a robot driving process by transmitting the image of the camera attached to the front part of the robot to the web server in real time.

Embodiments are intended to stably transport the goods even while climbing stairs by making the article loading box located at the top of the robot always parallel.

The technical problem of the present invention is not limited to those mentioned above, and another technical problem not mentioned will be clearly understood by those skilled in the art from the following description.

A method of operating a stair climbing robot according to an embodiment includes: inputting image data received from a camera sensor into a pre-trained neural network to search for stairs; controlling the moving unit to move the robot to a predetermined position based on a position movement signal determined based on a position between the searched stairs and the robot; controlling the moving unit so that the robot can climb the stairs based on the stair climbing signal; determining whether the robot arrives on a flat ground based on the inclination data received from the first gyro sensor; and controlling the moving unit so that the robot can stop based on a flat land arrival signal.

The step of controlling the moving unit so as to climb the stairs includes controlling the balance maintaining motor to maintain the level of the product loading box based on the inclination data of the product loading box attached to the robot received from the second gyro sensor. may include.

The method of operating the stair climbing robot according to an embodiment may further include controlling the moving unit to align the robot using distance data between the robot and a specific subject received from a first distance sensor. .

The step of controlling the moving unit to align the robot may include calculating an angle between the robot and the subject based on the distance data; and controlling the moving unit so that the angle becomes a predetermined angle.

The step of searching for the stairs may include searching for the stairs based on each of a plurality of step search signals, and the position movement signal may correspond to each of the step search signals.

The method of operating the stair climbing robot according to an embodiment further comprises the step of storing distance data between the robot and the object located in the first direction and the object located in the second direction of the robot using a second distance sensor, , The position movement signal includes rotation information and movement distance information of the robot, and the step of controlling the movement unit so that the robot can move to the predetermined position is to determine the movement distance information based on the distance data may include steps.

The step of controlling the moving unit so as to climb the stairs may include: determining whether the robot is stopped using a second distance sensor; And based on the determination that the robot stopped while climbing the stairs, it may include the step of controlling the moving unit to provide additional power to the robot.

The operation method of the stair climbing robot according to an embodiment comprises: receiving a target number of floors from a web server; calculating the number of floors of the stairs climbed by the robot based on the flat land arrival signal; and comparing the number of floors of the climbed stairs with the target number of floors to determine whether to further climb the robot.

The method of operating the stair climbing robot according to an embodiment may further include transmitting the image data received from the camera sensor to a web server in real time.

Stair climbing robot according to an embodiment includes a main body; a camera sensor attached to the front of the main body to collect image data; moving part; a first gyro sensor attached to the moving unit to collect tilt data of the moving unit; and a processor, wherein the processor inputs the image data to a pre-trained neural network to search for stairs, and based on a position movement signal determined based on a position between the searched stairs and the robot, the robot is pre-trained. Control the moving unit to move to a predetermined position, control the moving unit so that the robot can climb the stairs based on a stair climbing signal, determine whether the robot arrives on a flat ground based on the inclination data, and arrive on a flat ground The moving part is controlled so that the robot can stop based on the signal.

Stair climbing robot according to an embodiment includes an article loading box attached to the body; and a second gyro sensor attached to the main body to collect inclination data of the article loading box, wherein the processor controls the balance motor to maintain the horizontality of the article loading box based on the inclination data of the article loading box can do.

The stair climbing robot according to an embodiment further comprises a first distance sensor attached to the front surface of the main body to collect distance data between the main body and a specific subject, and the processor aligns the robot using the distance data It is possible to control the moving unit to do this.

The processor may calculate an angle between the robot and the subject based on the distance data, and control the moving unit so that the angle becomes a predetermined angle.

The processor may search for the stairs based on each of the plurality of stair search signals, and the position movement signal may correspond to each of the plurality of stair search signals.

The stair climbing robot according to an embodiment further comprises a second distance sensor attached to both sides of the main body to collect distance data between both sides of the main body and a specific subject, and the position movement signal includes rotation information of the robot and including movement distance information, and the processor uses the second distance sensor to store distance data between the robot and the object located in the first direction and the object located in the second direction of the robot, and based on the distance data Thus, it is possible to determine the movement distance information.

The stair climbing robot according to an embodiment further includes a second distance sensor attached to both sides of the main body to collect distance data between both sides of the main body and a specific subject, the processor using the second distance sensor It is determined whether the robot is stopped, and based on the determination that the robot has stopped while climbing the stairs, it is possible to control the moving unit to provide additional power to the robot.

The processor receives the target number of floors from the web server, calculates the number of floors of the stairs climbed by the robot based on the flat land arrival signal, and compares the number of floors of the climbed stairs with the target number of floors of the robot. You can decide whether or not to climb further.

The processor may transmit the image data received from the camera sensor to a web server in real time.

Embodiments may provide a method for recognizing various types of stairs based on deep learning and for a robot to climb to a target floor.

Embodiments may provide a robot capable of stable driving not only on flat ground but also on stairs and uneven terrain.

Embodiments may provide a robot driving process by transmitting an image of a camera attached to the front of the robot to a web server in real time.

In embodiments, the article loading box located at the top of the robot is always parallel, so that the articles can be stably transported even while climbing the stairs.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a diagram illustrating a configuration of a stair climbing robot according to an embodiment.
FIG. 2 is a view for explaining a specific structure of a stair climbing device implemented in a locker-bogie structure according to an embodiment.
3 is a diagram for explaining a specific structure of a processor according to an embodiment.
4 is a flowchart for explaining a method of operating a stair climbing robot according to an embodiment.
5 is a view for explaining a method of searching for a staircase and moving in front of the searched staircase according to an embodiment.
6 is a view for explaining a method of aligning a stair climbing robot according to an embodiment.

Specific structural or functional descriptions disclosed in this specification are merely illustrative for the purpose of describing embodiments according to technical concepts, and the embodiments may be embodied in various other forms and are limited to the embodiments described herein. doesn't happen

Terms such as first or second may be used to describe various elements, but these terms should be understood only for the purpose of distinguishing one element from another element. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Expressions describing the relationship between elements, for example, “between” and “between” or “neighboring to” and “directly adjacent to”, etc. should be interpreted similarly.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the specified feature, number, step, operation, component, part, or a combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

The embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like. Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

1 is a diagram illustrating a configuration of a stair climbing robot according to an embodiment.

Referring to FIG. 1 , the stair climbing robot according to an embodiment includes a main body 100 and a plurality of sensors attached to the main body 100 , a moving unit 120 , an article loading box 140 , a communication unit, a memory and a processor. may include

The plurality of sensors according to an embodiment may include a camera sensor 110 , a gyro sensor 130 , and distance sensors 150 - 1 , 150 - 2 , 150 - 3 and 150 - 4 . However, the type and number of sensors according to an embodiment are not necessarily limited to those shown in FIG. 1 , and various types of sensors may be applied.

The camera sensor 110 is a sensor that is attached to the front of the main body 100 to collect image data, and may be, for example, an RGB camera. Hereinafter, the terms indicating the direction, such as the front and the side, are based on the moving direction of the stair climbing robot. That is, the direction in which the stair climbing robot proceeds is the front, and the right and left sides of the moving direction are the sides.

The gyro sensor 130 may measure a value of an angular velocity that is a rotational speed of an object. Although the drawing shows that the stair climbing robot includes only one gyro sensor 130, the stair climbing robot may include a plurality of gyro sensors. As an example, the stair climbing robot is attached to the first gyro sensor (eg, the gyro sensor 130) and the main body 100 that are attached to the moving unit 120 to collect the tilt data of the moving unit and are attached to the article loading box (140). It may include a second gyro sensor that collects the tilt data of the.

The distance sensors 150-1, 150-2, 150-3, and 150-4 are sensors that measure the distance between the sensor and the object. It could be an infrared distance sensor that calculates time and calculates distance through it. The distance sensors 150-1, 150-2, 150-3, and 150-4 are the first distance sensors 150-3 and 150-4 and the second distance sensors 150-1 and 150- according to the attached positions. 2) can be distinguished. The first distance sensors 150 - 3 and 150 - 4 may be front distance sensors attached to the front surface of the main body 100 to collect distance data between the main body 100 and a specific subject. The second distance sensors 150 - 1 and 150 - 2 may be lateral distance sensors attached to both sides of the main body 100 to collect distance data between both sides of the main body 100 and a specific subject.

The moving unit 120 according to an embodiment may include a driving unit 121 , a frame 122 , and a plurality of wheels 123 . The moving unit 120 may be implemented in a rocker-bogie structure to climb stairs. The specific structure of the stair climbing device implemented in the rocker-bogie structure will be described in detail with reference to FIG. 2 below.

The driving unit 121 may generate power energy of the robot under the control of the processor. The driving unit 121 according to an embodiment may be divided into a first driving unit and a second driving unit according to the type of power energy to be provided. The first driving unit may be a dynamixel motor that transmits rotational kinetic energy to the plurality of wheels 123 . The second driving unit may be a linear motor that converts electric energy into linear kinetic energy as a linearly unfolded rotational structure in order to obtain direct linear driving. As will be described in detail below, the second motor may serve to overcome the difficulty in climbing stairs due to insufficient thrust of the wheels, such as when all three wheels contact the stair wall.

The article loading box 140 is attached to the main body and is a basket capable of accommodating an object. The stair climbing robot may control the balance motor to keep the article loading box 140 level even on the inclined stairs based on the inclination data of the second gyro sensor of the article loading box 140 .

The communication unit is connected to the sensors, the moving unit, the processor, and the memory to transmit and receive data. The communication unit may be connected to another external device to transmit/receive data. Hereinafter, the expression “transmitting and receiving “A” may indicate transmission and reception of “information or data representing A”.

The communication unit may be implemented as circuitry in the stair climbing robot. For example, the communication unit may include an internal bus and an external bus. As another example, the communication unit may be an element connecting the stair climbing robot and an external device. The communication unit may be an interface. The communication unit may receive data from an external device and transmit the data to the processor and the memory.

The processor processes the data received by the communication unit and the data stored in the memory. A “processor” may be a data processing device implemented in hardware having circuitry having a physical structure for performing desired operations. For example, desired operations may include code or instructions included in a program. For example, a data processing device implemented as hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

A processor executes computer readable code (eg, software) stored in memory (eg, memory) and instructions issued by the processor.

The memory stores data received by the communication unit and data processed by the processor. For example, the memory may store a program. The stored program may be a set of syntaxes that are coded so that the stair climbing robot can autonomously drive and can be executed by the processor. A detailed structure and operation of the processor according to an embodiment will be described in detail with reference to FIG. 3 below.

According to one aspect, memory may include one or more of volatile memory, non-volatile memory and random access memory (RAM), flash memory, hard disk drives, and optical disk drives.

The memory stores a set of instructions (eg, software) for operating the stair climbing robot. The instruction set that operates the stair climbing robot is executed by the processor.

FIG. 2 is a view for explaining a specific structure of a stair climbing device implemented in a locker-bogie structure according to an embodiment.

Referring to FIG. 2 , the moving unit 200 according to an embodiment may be implemented in a locker-bogie structure. The stair climbing robot according to an embodiment can always move with six wheels in contact with the ground because the moving part 200 is implemented in a rocker-bogie structure and flexibly adapts according to the shape of the ground.

The moving unit 200 according to an embodiment includes a front wheel 210 , a rear wheel 2100 , a front frame 230 , a rear frame 240 , and a connection frame 250 , but the front wheel 210 is a first sub It may include a front wheel 211 and a second sub front wheel 212 . In addition, a first sub front frame 231 extending in an upward direction is connected to the first sub front wheel 211 , and a second sub front frame 232 extending in an upward direction is connected to the second sub front wheel 212 . The first sub front frame 231 and the second sub front frame 232 may be connected by a sub connection frame 233 .

The front frame 230 is connected to the sub-connection frame 233 , and in this case, the sub-connection frame 233 may be hinge-connected to be rotatable with respect to the front frame 230 . The length of the rear frame 240 is the sum of the length of the front frame 230 and the length of the first sub front frame 231 , or the sum of the length of the front frame 230 and the length of the second sub front frame 232 . can be implemented in the same way as

Each of the first sub front wheel 211 , the second sub front wheel 212 , and the rear wheel 2100 may include one or more wheels.

3 is a diagram for explaining a specific structure of a processor according to an embodiment.

Referring to FIG. 3 , a processor according to an embodiment may include a first module 310 , a second module 320 , and a third module 330 . However, in the embodiment of FIG. 3, each component of the processor is separately indicated in the drawing to indicate that it can be separated functionally and logically, which means that it is necessarily a separate component physically or is implemented as a separate code. it is not

The first module 310 according to an embodiment may embed ROS robot operating system-based software for the operation sequence of the robot, and thus may control the operation of the stair climbing robot. For example, the first module 310 may be a Jetson nano module. Robot operating system ROS can provide hardware abstraction, sub-device control, implementation of commonly used functions, message passing between processes, package management, libraries and various development and debugging tools required for developing robot applications. . ROS is a software platform for supporting robot application software development and has the same functions as an operating system that can be used on heterogeneous hardware. The first module 310 (eg, Jetson nano) is heterogeneous between the second module 320 (eg, OpenCM 9.04) and the third module 330 (eg, Arduino DUE) using ROS. A system can be built to support communication in hardware and control the movement of the robot by aligning the received sensors and data. In addition, one module 310 may make a web server for robot control by message passing as the ROS Web Server (3), so that it can be controlled through a computer and a smart phone.

The first module 310 may include a pre-learned step recognition artificial neural network. Here, including the artificial neural network may mean including the weight of the learned artificial neural network.

The artificial neural network can learn about a target task and build an inference model. Also, the artificial neural network may output an inference result for an external input value based on the constructed inference model.

The artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer includes one or more neurons, and the artificial neural network may include neurons and synapses connecting neurons. In the artificial neural network, each neuron may output a function value of an activation function for input signals, weights, and biases input through synapses.

The model parameter means a parameter determined through learning, and may include a weight of a synaptic connection and a bias of a neuron. In addition, the hyperparameter refers to a parameter that must be set before learning in a machine learning algorithm, and may include a learning rate, the number of iterations, a mini-batch size, an initialization function, and the like.

The purpose of learning the artificial neural network can be seen as determining the model parameters that minimize the loss function. The loss function may be used as an index for determining optimal model parameters in the learning process of the artificial neural network.

The second module 320 according to an embodiment may control the moving unit. The second module 320 may receive a control signal from the first module 310 and transmit a signal to the moving unit to control the movement of the robot. The second module 320 and the mobile unit communicate with the UART method and may transmit a control signal based thereon. For example, the second module 320 may be an OpenCM 9.04 module.

The third module 330 according to an embodiment may control a plurality of sensors. The third module 330 may obtain sensor data from the distance sensor and the gyro sensor and transmit it to the first module 310 . For example, the third module 330 may be an Arduino DUE module.

Hereinafter, a detailed operation method of the stair climbing robot will be described with reference to FIGS. 4 to 6 .

4 is a flowchart for explaining a method of operating a stair climbing robot according to an embodiment.

Referring to FIG. 4 , steps 410 to 450 according to an embodiment may be performed by the stair climbing robot described above with reference to FIGS. 1 to 3 . More specifically, it may be performed by the processor of the stair climbing robot. However, the operations of FIG. 4 may be performed in the illustrated order and manner, but the order of some operations may be changed or some operations may be omitted without departing from the spirit and scope of the embodiment illustrated in FIG. 4 . Many of the operations shown in FIG. 4 may be performed in parallel or concurrently.

In step 410, the stair climbing robot searches for the stairs by inputting the image data received from the camera sensor into the pre-trained neural network. The stair climbing robot may be equipped with a pre-trained artificial neural network to recognize various stairs. For example, step recognition through a camera can be performed by the YOLOv4 deep learning system.

In step 420, the stair climbing robot controls the moving unit to move the robot to a predetermined position based on a position movement signal determined based on the position between the searched stairs and the robot. The position movement signal may mean a signal for moving the stair climbing robot in front of the searched stair.

In step 430, the stair climbing robot controls the moving unit so that the stair climbing robot can climb the stairs based on the stair climbing signal. The stair climbing robot may control the balance motor to maintain the level of the article stacker even on the inclined stairs based on the inclination data received from the second gyro sensor of the article stacker. More specifically, the second gyro sensor may be mounted on the second module, and the stair climbing robot is balanced so that the loading box can be maintained horizontally based on the data related to the angle of the x-axis among the sensor data collected by the second gyro sensor. The holding motor can be controlled.

The stair climbing robot may use the second distance sensor to determine whether the robot has stopped while climbing the stairs. When a stair climbing robot is climbing a stair, there may be a case in which the robot does not receive thrust due to the slope due to the reason that the wheel touches the stair wall. In this case, the stair climbing robot may control the moving unit to provide additional power to the robot based on the determination that the robot has stopped while climbing the stairs. More specifically, when the corresponding case occurs, the stair climbing robot may increase the thrust by reducing the length of the rear frame by operating the second driving unit (eg, a linear motor) coupled to the rocker of the frame. For example, when the stair climbing robot reduces the length of the rear frame, the middle wheel and the end wheel come into contact with each other. , it is also possible to increase the power of the motor to increase the stair climbing force.

In step 440, the stair climbing robot determines whether the robot arrives on a flat surface based on the inclination data received from the first gyro sensor. The first gyro sensor may be attached to the moving part of the stair climbing robot to collect tilt data of the moving part. The first gyro sensor collects inclination data that distinguishes the stairs from the flat ground, and may be attached near the midpoint of the moving part so as to be positioned parallel to the floor.

In step 450 , the stair climbing robot controls the moving unit so that the robot can stop based on the flat land arrival signal. Here, the flat land may include a landing that is a space made of flat land between the stairs and the stairs. When the stair climbing robot arrives on a flat surface, it can decide whether or not to climb the next stair further. More specifically, the stair climbing robot may receive the target number of floors from the web server, and may climb the stairs until it reaches the target number of floors. To this end, the stair climbing robot may calculate the number of floors of the stairs that have been climbed so far, based on the arrival signal on the flat ground. The stair climbing robot can determine whether to climb further by comparing the number of floors of the climbed stairs with the target number of floors.

The web server can operate the robot non-face-to-face by providing functions such as providing a robot control panel (for example, a control panel for input of a target floor) and real-time confirmation of camera images. In addition, the web server provides a strong security function because it can access the server only if it shares the same Wi-Fi.

If additional climbing is required, the stair climbing robot can navigate to the next stair after stopping once at the landing. However, if the robot does not stop in the forward direction at the landing due to a structural defect in the robot or stairs, the alignment can be performed in advance. A specific method of aligning the robot will be described with reference to FIG. 6 below. A stair climbing robot that requires additional climbing can navigate to the next stair and move forward after it has been navigated. A specific method of searching for the next stair and moving in front of the found next stair will be described below with reference to FIG. 5 .

5 is a view for explaining a method of searching for a staircase and moving in front of the searched staircase according to an embodiment.

Referring to FIG. 5 , the stair climbing robot arriving at the landing according to an exemplary embodiment may search for the next stair after stopping once at the landing.

Referring to the drawing 510 , the stair climbing robot may store distance data between the robot and the object located in the first direction and the object located in the second direction of the robot by using the second distance sensor. For example, the stair climbing robot measures the distance between the robot reference left and right walls and the robot using the second distance sensors (eg, lateral distance sensors) attached to both sides of the main body, and stores only a small value among them. can

In addition, the stair climbing robot may search for the next stair based on each of the plurality of stair search signals. For example, the stair climbing robot is rotated by a predetermined angle (eg, 135 degrees) in the first direction (eg, left direction) based on the first stair search signal to navigate the stairs, the next stair can explore. Alternatively, the stair climbing robot is rotated by a predetermined angle (eg, 135 degrees) in the second direction (eg, the right direction) and based on the second stair search signal to search for the stairs, to search for the next stair. can Alternatively, the stair climbing robot may search for the next stair based on the third stair search signal to search whether a stair exists in front of the stair climbing robot without rotating.

Referring to the drawing 520 , the stair climbing robot that has searched for the next stair may rotate to move in front of the next stair. In this case, the direction of rotation may be determined according to the stair search signal for searching for the stair. For example, if the stairs are recognized while rotating by a predetermined angle (eg, 135 degrees) in the first direction (eg, the left direction), the stair climbing robot moves in the first direction (eg, the left side) direction) by a predetermined angle (eg, 90 degrees). Alternatively, if the stairs are recognized while rotating by a predetermined angle (eg, 135 degrees) in the second direction (eg, the right direction), the stair climbing robot moves in the second direction (eg, the right direction) can be rotated by a predetermined angle (eg, 90 degrees). Alternatively, when the stairs are recognized from the front without rotating, the stair climbing robot may remain as it is without rotating.

Referring to the drawing 530 , the stair climbing robot may move in front of the next stair based on the position movement signal. Specifically, the stair climbing robot may determine movement distance information based on pre-stored distance data. For example, for a stair climbing robot, the distance between the robot and a subject (eg, a wall) existing in the moving direction of the robot is a pre-stored distance (eg, the smaller of the distance between the left and right walls and the robot) You can move forward until you get enough.

Referring to the drawing 540 , the stair climbing robot may rotate for stair climbing. In this case, the direction of rotation may be determined according to the stair search signal for searching for the stair. For example, if the stairs are recognized while rotating by a predetermined angle (eg, 135 degrees) in the first direction (eg, the left direction), the stair climbing robot moves in the first direction (eg, the left side) direction) by a predetermined angle (eg, 90 degrees). Alternatively, if the stairs are recognized while rotating by a predetermined angle (eg, 135 degrees) in the second direction (eg, the right direction), the stair climbing robot moves in the second direction (eg, the right direction) can be rotated by a predetermined angle (eg, 90 degrees). Alternatively, when the stairs are recognized from the front without rotating, the stair climbing robot may remain as it is without rotating.

6 is a view for explaining a method of aligning a stair climbing robot according to an embodiment.

Referring to FIG. 6 , after the stair climbing robot according to an embodiment arrives on a flat ground, if the robot is not stopped in the forward direction at the landing due to a structural defect of the robot or the stairs, the stair climbing robot may perform alignment in advance. Alternatively, the stair climbing robot according to an embodiment may not be stopped in the forward direction in front of the next stair as it moves to climb the next stair, and in this case, alignment may be required.

The stair climbing robot may control the moving unit to align the robot using distance data between the robot and a specific subject received from the first distance sensor. More specifically, the stair climbing robot may calculate the angle formed between the robot and the subject (eg, a wall or stairs) based on the output data of two first distance sensors (eg, a front distance sensor), and the calculated The moving unit may be controlled so that the angle becomes a predetermined angle (eg, 0 degrees). More specifically, the two front distance sensors can measure the distance between themselves and the wall, respectively, and the stair climbing robot can measure the distance between the robot and the subject (for example, based on the difference between the two distances and the separation distance between the two front distance sensors). , walls or stairs) can be calculated. Alternatively, the stair climbing robot may calculate an angle based on a change in the position of the motor according to the rotation time of the motor.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (8)

In the operation method of the stair climbing robot,
inputting the image data received from the camera sensor into a pre-trained neural network to search for stairs;
controlling the moving unit to move the robot to a predetermined position based on a position movement signal determined based on a position between the searched stairs and the robot;
controlling the moving unit so that the robot can climb the stairs based on the stair climbing signal;
determining whether the robot arrives on a flat ground based on the inclination data received from the first gyro sensor; and
Comprising the step of controlling the moving unit so that the robot can stop based on the arrival signal on the flat ground,
The step of controlling the moving unit so as to climb the stairs,
determining whether the robot is stopped using a second distance sensor; and
Based on the determination that the robot stopped while climbing the stairs, the operation method of the stair climbing robot comprising the step of controlling the moving unit to provide additional power to the robot.
According to claim 1,
The step of controlling the moving unit so as to climb the stairs
Controlling the balance maintenance motor to maintain the level of the article loading box based on the inclination data of the product loading box attached to the robot received from the second gyro sensor
Including, the operation method of the stair climbing robot.
According to claim 1,
controlling the moving unit to align the robot using distance data between the robot and a specific subject received from a first distance sensor
Further comprising, the method of operation of the stair climbing robot.
4. The method of claim 3,
The step of controlling the moving unit to align the robot
calculating an angle between the robot and the subject based on the distance data; and
controlling the moving unit so that the angle becomes a predetermined angle
Including, the operation method of the stair climbing robot.
According to claim 1,
The step of navigating the stairs
Searching for the stairs based on each of the plurality of stairs search signals
including,
The position movement signal is
Corresponding to each of the stair search signals, the operation method of the stair climbing robot.
According to claim 1,
Storing distance data between the robot and the object located in the first direction and the object located in the second direction by using a second distance sensor
further comprising,
The position movement signal is
Including rotation information and movement distance information of the robot,
The step of controlling the moving unit so that the robot can move to the predetermined position
determining the moving distance information based on the distance data
Including, the operation method of the stair climbing robot.
delete In the operation method of the stair climbing robot,
inputting the image data received from the camera sensor into a pre-trained neural network to search for stairs;
controlling the moving unit to move the robot to a predetermined position based on a position movement signal determined based on a position between the searched stairs and the robot;
controlling the moving unit so that the robot can climb the stairs based on the stair climbing signal;
determining whether the robot arrives on a flat ground based on the inclination data received from the first gyro sensor;
controlling the moving unit to stop the robot based on a flat land arrival signal;
receiving a target number of floors from a web server;
calculating the number of floors of the stairs climbed by the robot based on the flat land arrival signal; and
Comprising the step of determining whether to further climb the robot by comparing the number of floors of the climbed stairs with the target number of floors, the operation method of the stair climbing robot.

Images