KR102330607B1 - Artificial intelligence Moving Robot and controlling method - Google Patents

Abstract

The artificial intelligence mobile robot for learning obstacles of the present invention and its control method store a plurality of image data captured by shooting an image while driving through an image acquisition unit, and the obstacle is determined to be located within a predetermined distance by the obstacle detection unit At a time point, at least one response motion is started according to an obstacle determined based on the image data, and a dangerous situation is determined in response to the driving condition, and at least one It is an artificial intelligence mobile robot that transmits image data to the server to learn about obstacles and dangerous situations. It sets the response motion to avoid the obstacles in which dangerous situations occur, and prevents dangerous situations from occurring so that driving continues. do.

Description

Artificial intelligence Moving Robot and controlling method that learns obstacles

The present invention relates to a mobile robot and a method for controlling the same, and to a mobile robot for performing cleaning by running in a cleaning area, and a method for controlling the same.

In general, a mobile robot is a device that automatically cleans by sucking foreign substances such as dust from a floor while driving in an area to be cleaned without user's manipulation.

This mobile robot senses the distance to obstacles such as furniture, office supplies, and walls installed in the cleaning area, and performs an obstacle avoidance operation.

The mobile robot is a means for detecting an obstacle, and is provided with an obstacle detecting means such as infrared rays or a laser. The mobile robot avoids the obstacle by detecting an obstacle within a predetermined distance based on a detection signal by a sensor such as infrared or laser.

If the path is changed immediately upon detecting an obstacle, cleaning of the corresponding area cannot be performed, so the path is changed after approaching or colliding with the obstacle as much as possible.

Although these mobile robots are basically designed to run while avoiding obstacles, there is a limit to recognizing all types of obstacles. As a result, driving may be obstructed by some obstacles, and in severe cases, you may be in a restrained situation where you can no longer drive.

As a result, recently, as tests are conducted through various experiments before products are released, information about obstacles is taught in advance.

However, since the test environment cannot accumulate information on all obstacles through prior learning, there is a limit to the response of the mobile robot to obstacles.

In addition, since the user's residential environment in which the actual mobile robot is used is different from house to house, and dangerous situations that may occur are also different, there is a need to continuously add information about obstacles according to each environment.

The problem to be solved by the present invention is to obtain information on new obstacles by periodically capturing images while driving and analyzing the characteristics of obstacles through a plurality of images, thereby avoiding obstacles with various response motions or escaping from dangerous situations An object of the present invention is to provide a mobile robot and a method for controlling the same.

A mobile robot according to an embodiment of the present invention includes a movable body, an image acquisition unit that captures an image of a traveling direction, an obstacle detection unit that detects an obstacle located in the traveling direction, and a plurality of images periodically photographed through the image acquisition unit. A data unit in which image data of the image data is stored, determines whether the main body is restrained by the obstacle in response to the driving state, and transmits at least one image data previously photographed from the time of determining restraint to the server, the server and a control unit configured to avoid the obstacle by setting a corresponding motion to prevent restraint of the main body in response to the obstacle information received from the .

The control unit starts a corresponding motion at a predetermined time when it is determined by the obstacle detection unit that the obstacle is located within a predetermined distance, and responds according to the obstacle determined based on the image data obtained before the predetermined time. It is characterized by determining the motion.

When a restraint situation occurs, the control unit transmits to the server at least one image data photographed during a previous predetermined time or a previous predetermined distance from the time when the restraint of the main body is determined.

The control unit performs a designated response motion based on the obstacle information received from the server to escape from the restraint situation.

When the obstacle is an obstacle causing the restraint, the control unit may perform any one of the corresponding motions designated among the plurality of corresponding motions to avoid the obstacle.

The control unit is characterized in that, as the corresponding motion, it avoids the obstacle after approaching a certain distance or immediately avoids it when the obstacle is detected.

When the obstacle is included in a candidate in which a restraint situation may occur, the controller performs any one of the corresponding motions to output a warning about restraint among the plurality of corresponding motions.

The present invention includes the steps of periodically capturing an image for the driving direction through an image acquisition unit while driving and storing image data, analyzing the image data to recognize an obstacle located in the driving direction, and using the obstacle detection unit to detect the obstacle. Detecting an obstacle, when detecting the obstacle, performing any one of a plurality of corresponding motions in response to the type of the obstacle previously recognized from the image data, corresponding to the driving state, Determining whether the main body is restrained, if it is determined that the restraint is determined, transmitting at least one of the image data previously photographed from the time of determining restraint to the server, and in response to the obstacle information received from the server, and avoiding the obstacle by setting a corresponding motion to prevent restraint.

According to the present invention, an image is taken periodically while driving, a plurality of image data is stored, an obstacle is recognized from the image data, and any one corresponding motion is performed according to the recognized obstacle when an obstacle within a predetermined distance is detected, and driving In the case of being restrained by the obstacle, a mobile robot that transmits a plurality of image data captured before it to a server, and analyzes the plurality of image data to analyze the characteristics of the obstacle causing the restraint, and and a server for setting a corresponding motion for relieving restraint or avoiding the obstacle.

The server analyzes the at least one image data, recognizes an obstacle in the vicinity from the image data taken before the main body is restrained, and determines an obstacle that causes restraint.

The server generates a recognition model for the obstacle by extracting the characteristics of the obstacle that is the cause of the restraint, and generates the obstacle information including a corresponding motion for avoiding the obstacle or a corresponding motion for escaping from a restrained situation characterized in that

In response to the characteristics of the obstacle, the server generates the obstacle information by setting other obstacles having a similar shape to the obstacle as candidates for which restraint can occur.

The mobile robot and its control method of the present invention periodically photograph an image while driving, and when an obstacle is located within a predetermined distance, it analyzes a pre-photographed image and selects any one of a plurality of motions set for the obstacle. , even when the same detection signal is input when detecting an obstacle for a plurality of different obstacles, it is possible to respond to various obstacles by determining the obstacle through the image and performing different operations according to the obstacle.

The present invention determines whether or not to be restrained according to the driving state, analyzes at least one image taken before that from the time of restraint, extracts an obstacle that causes restraint, and avoids the obstacle that causes restraint. By setting the corresponding motion, restraint does not occur during the next driving.

The present invention can solve the restraint situation by analyzing at least one image taken before that based on the time when the movement of the main body is restrained and setting a corresponding motion to escape the restraint situation.

The present invention sets candidates for constraint generation with respect to other obstacles having the same or similar shape as the obstacle causing the constraint, and sets a corresponding motion for the other obstacle so that the constraint does not occur.

The present invention prevents a situation in which the body cannot travel and is constrained in advance, so that the body can continuously travel.

According to the present invention, through the analysis and learning of obstacles through the server, the mobile robot acquires information on new obstacles and updates information on existing obstacles, thereby enabling various responses to obstacles and avoiding dangerous situations. can

1 is a perspective view illustrating a mobile robot of a mobile robot system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a horizontal angle of view of the mobile robot of FIG. 1 .
FIG. 3 is a front view of the mobile robot of FIG. 1 .
FIG. 4 is a view showing a bottom surface of the mobile robot of FIG. 1 .
5 is a diagram schematically illustrating a system for recognizing a mobile robot and an obstacle according to an embodiment of the present invention.
6 is a block diagram illustrating main parts of a mobile robot according to an embodiment of the present invention.
7 is a diagram referenced for explaining a signal flow between a mobile robot and a device for recognizing an obstacle according to an embodiment of the present invention.
8 is a diagram referenced for explaining image transmission for obstacle recognition of a mobile robot according to an embodiment of the present invention.
9 is a diagram referenced for explaining a method for determining a restraint situation through an image of a mobile robot according to an embodiment of the present invention.
10 is a flowchart referenced to explain a method for controlling a mobile robot according to an embodiment of the present invention.
11 is a flowchart referenced to explain an error determination method of a mobile robot according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout. The control configuration of the present invention may consist of at least one processor.

1 is a perspective view of a mobile robot according to an embodiment of the present invention. FIG. 2 illustrates a horizontal angle of view of the mobile robot of FIG. 1 . 3 is a front view of the mobile robot of FIG. 1 . 4 is a view showing a bottom surface of the mobile robot of FIG. 1 .

1 to 4, the mobile robot 1 according to an embodiment of the present invention moves along the floor of the cleaning area, and the main body 10 for sucking foreign substances such as dust on the floor, and the main body 10 ) is disposed on the front side and includes a sensing means for detecting an obstacle.

The main body 10 forms an exterior and a casing 11 which forms a space in which the parts constituting the main body 10 are accommodated, and a suction unit disposed in the casing 11 to suck foreign substances such as dust or garbage. 34 , and a left wheel 36 (L) and a right wheel 36 (R) rotatably provided in the casing 11 . As the left wheel 36(L) and the right wheel 36(R) rotate, the main body 10 moves along the floor of the cleaning area, and foreign substances are sucked through the suction unit 34 in this process.

The suction unit 34 may include a suction fan (not shown) that generates a suction force, and a suction port 10h through which an airflow generated by the rotation of the suction fan is sucked. The suction unit 34 may include a filter (not shown) for collecting foreign substances from the airflow sucked through the suction port 10h, and a foreign substance collecting container (not shown) in which the foreign substances collected by the filter are accumulated.

In addition, the main body 10 may include a driving unit for driving the left wheel 36 (L) and the right wheel 36 (R). The driving unit may include at least one driving motor. The at least one driving motor may include a left wheel driving motor rotating the left wheel 36(L) and a right wheel driving motor rotating the right wheel 36(R).

The operation of the left wheel drive motor and the right wheel drive motor is independently controlled by the driving control unit of the control unit, so that the main body 10 can move forward, backward, or turn. For example, when the main body 10 travels straight, the left wheel drive motor and the right wheel drive motor rotate in the same direction, but when the left wheel drive motor and the right wheel drive motor rotate at different speeds or rotate in opposite directions, The traveling direction of the main body 10 may be switched. At least one auxiliary wheel 37 for stable support of the main body 10 may be further provided.

A plurality of brushes 35 positioned on the front side of the bottom surface of the casing 11 and having a brush composed of a plurality of radially extending wings may be further provided. The dust is removed from the floor of the cleaning area by the rotation of the plurality of brushes 35, and the dust separated from the floor is sucked through the suction port 10h and collected in the collection container.

A control panel including a manipulation unit 160 that receives various commands for controlling the mobile robot 1 from a user may be provided on the upper surface of the casing 11 .

The sensing means includes a sensor unit 150 for detecting an obstacle by using a plurality of sensors, as shown in FIG. 1A , and an image acquisition unit 170 for capturing an image.

In addition, the detection means, as shown in Fig. 1 (b), is disposed on the front of the main body 10, it may include an obstacle detection unit 100 for detecting an obstacle, and an image acquisition unit 170 for taking an image. have. The obstacle detection unit 100 irradiates a light pattern and detects an obstacle through a photographed image. The obstacle detecting unit 100 may include a pattern acquisition unit 140 , and may also include a sensor unit 150 .

The image acquisition unit 170 is provided to face forward, respectively, to photograph the driving direction, and, in some cases, may be provided to face the ceiling. In some cases, two image acquisition units 170 facing the front and the ceiling may be provided. When two image acquisition units are provided, they are installed on the front and upper portions of the main body, respectively, so that images of the front and the ceiling can be captured, respectively.

The obstacle detecting unit 100 may be disposed on the front side of the main body 10 .

The obstacle detecting unit 100 is fixed to the front surface of the casing 11 , and includes a first pattern irradiation unit 120 , a second pattern irradiation unit 130 , and a pattern acquisition unit 140 . In this case, the pattern acquisition unit 140 may be installed under the pattern irradiation unit as shown or disposed between the first and second pattern irradiation units.

In FIG. 2 , the displayed irradiation angle θh indicates the horizontal irradiation angle of the first pattern light P1 irradiated from the first pattern irradiation unit 120 , and both ends of the horizontal line form the first pattern irradiation unit 120 . It represents an angle, and it is preferable to be set in the range of 130˚ to 140˚, but it is not necessarily limited thereto. The dotted line indicated in FIG. 2 is directed toward the front of the mobile robot 1 , and the first pattern light P1 may be configured to be symmetrical with respect to the dotted line.

The main body 10 is provided with a rechargeable battery 38 , and the charging terminal 33 of the battery 38 is connected to commercial power (eg, a power outlet in the home) or a separate charging station connected to commercial power. The main body 10 is docked at 400 , the charging terminal 33 is electrically connected to commercial power through contact with the terminal 410 of the charging station, and the battery 38 can be charged. Electrical components constituting the mobile robot 1 may be supplied with power from the battery 38 . Therefore, in a state in which the battery 38 is charged, the mobile robot 1 has a magnetic force in a state electrically separated from commercial power. driving is possible

5 is a diagram schematically illustrating a system for recognizing a mobile robot and an obstacle according to an embodiment of the present invention.

The mobile robot 1 receives a cleaning command for a predetermined cleaning area among the traveling areas H and performs cleaning. The mobile robot 1 analyzes an image captured while driving to determine the type of obstacle.

The mobile robot 1 detects a plurality of obstacles 2 and 3 existing in the traveling area H, recognizes the obstacles, and performs any one of approach, passage, and avoidance according to the type of obstacle. At this time, the mobile robot 1 may avoid without approaching the obstacle, or may perform a predetermined operation after approaching a predetermined distance, may avoid after approaching the obstacle, and may pass through the obstacle depending on the shape of the obstacle. have.

The mobile robot 1 takes a driving direction through an image acquisition unit 170 provided while driving, and detects an obstacle located within a predetermined distance through the obstacle detecting unit 100 . The mobile robot 1 may continuously photograph the driving direction through the image acquisition unit 170 and may photograph images at predetermined time intervals according to settings.

The mobile robot 1 stores an image taken while driving, and when an obstacle is detected through the obstacle detection unit 100, selects a corresponding corresponding motion, and then, according to the obstacle determined through image analysis, especially of the obstacle. It operates by determining the response motion to the obstacle according to the type.

Even when the same detection signal is input by the obstacle detection unit 100 for different obstacles, the mobile robot 1 analyzes the obstacle based on the pre-photographed image, sets a corresponding motion according to the obstacle, and accordingly Perform actions in response to obstacles.

In addition, when an image is captured, the mobile robot 1 analyzes the image at a predetermined time interval to recognize an obstacle included in the image, and when it is detected that an obstacle is located within a predetermined distance by the obstacle detection unit 100, the Based on the determined type of obstacle, an operation for the corresponding motion may be performed.

For example, depending on the obstacle, the mobile robot 1 avoids a dangerous obstacle without approaching it, and in the case of a general obstacle, it approaches up to a close distance or avoids it after colliding with the obstacle.

The mobile robot 1 may determine the type of obstacle based on data stored therein, and may be connected to the server 90 or the terminal 80 to determine the type of the obstacle. The mobile robot 1 may be connected to a separate communication device (not shown) in order to communicate with the server 90 through the network N.

The mobile robot 1 filters each image data for an image taken while driving and transmits it to a server or a terminal at predetermined time intervals to request obstacle information, thereby transmitting information on obstacles included in the image data to the server or received from the terminal. When it is determined by the obstacle detecting unit 100 that the obstacle is located within a predetermined distance, the mobile robot 1 can immediately identify the obstacle through the image data.

The terminal 80 includes an application for controlling the mobile robot 1, displays a map for the traveling area to be cleaned by the mobile robot 1 through the execution of the application, and selects an area to clean a specific area on the map can be specified. In addition, the position of the mobile robot is displayed based on the data received from the mobile robot, and information about the cleaning state is displayed.

The terminal is equipped with a communication module to allow network connection and is a device in which a program for controlling a mobile robot or an application for controlling a mobile robot is installed, and devices such as a computer, a laptop, a smart phone, a PDA, and a tablet PC may be used. In addition, the terminal may also use a wearable device such as a smart watch.

The terminal 80 may be interconnected with the mobile robot through a network built in the home. The terminal and the mobile robot are not only connected through WIFI, but also can communicate with each other using short-range wireless communication methods such as Bluetooth, infrared communication, and Zigbee. It is specified that the communication method between the terminal and the mobile robot is not limited thereto.

Also, the terminal 80 may determine the type of obstacle in response to the image received from the mobile robot 1 . The terminal 80 may determine the type of obstacle based on pre-stored data, or may be connected to the server to determine the type of obstacle. For example, when the mobile robot 1 cannot connect to a server through a network connection, the terminal 80 may receive data of the mobile robot and transmit it to the server.

In addition, the terminal 80 may display an image on the screen, input the type of obstacle through the user's input, and transmit it to the mobile robot, and may designate an operation for a specific obstacle through the menu of the application.

The server 90 analyzes the image data received from the mobile robot 1 connected through a predetermined network N to determine the type of obstacle, and transmits a response thereto to the mobile robot 1 . When there is a request from the terminal, the server 90 may respond by determining the type of obstacle.

The server 90 accumulates image data received from the plurality of mobile robots 1 and analyzes the plurality of image data to learn about obstacles.

The server 90 includes a database (not shown) for obstacle recognition based on a plurality of image data, and determines the type of obstacle by recognizing the characteristics of the obstacle extracted from the image data.

The server 90 accumulates and stores obstacle information and analyzes the obstacle information to determine the type of obstacle. In addition, the server 90 may classify obstacles according to types and set the motion of the mobile robot 1 to the obstacles as at least one corresponding motion.

The server 90 analyzes the new obstacle information and updates the previously stored obstacle information. In addition, the server 90 receives and stores the motion information of the mobile robot for the obstacle set or changed by the terminal 80, and matches the pre-stored obstacle information to set the motion of the mobile robot for the obstacle. update

6 is a block diagram illustrating main parts of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 6 , the mobile robot 1 includes an obstacle detection unit 100 , an image acquisition unit 170 , a cleaning unit 260 , a driving unit 250 , a data unit 180 , and an output unit 190 . , an operation unit 160 , a communication unit 280 , and a control unit 110 for controlling overall operations.

The operation unit 160 includes at least one button, a switch, and a touch input means, and receives an on/off or various commands necessary for the overall operation of the mobile robot 1 and inputs it to the control unit 110 .

The output unit 190 has a display such as LED and LCD, and displays the operation mode, reservation information, battery state, operation state, error state, etc. of the mobile robot 1 . In addition, the output unit 190 is provided with a speaker or a buzzer, and outputs a predetermined sound effect, warning sound, or voice guidance corresponding to the operation mode, reservation information, battery state, operation state, and error state.

The communication unit 280 communicates with the terminal 80 in a wireless communication method. Also, the communication unit 280 may be connected to an Internet network through an in-home network and communicate with the external server 90 .

The communication unit 280 transmits the generated map to the terminal 80 , receives a cleaning command from the terminal, and transmits data on the operation state and cleaning state of the mobile robot to the terminal. The communication unit 280 transmits and receives data including communication modules such as Wi-Fi and WiBro, as well as short-range wireless communication such as Zigbee and Bluetooth.

The communication unit 280 transmits information on the obstacle detected by the obstacle detecting unit 100 to the server 90 through the communication unit 280 and receives data on the obstacle from the server. In addition, the communication unit 280 receives information on obstacles existing in the driving zone and operation information corresponding thereto from the terminal 80 , and transmits operation data of the mobile robot to the terminal 80 .

The traveling unit 250 includes at least one driving motor to allow the mobile robot to travel according to a control command of the traveling control unit 113 . As described above, the driving unit 250 may include a left wheel driving motor rotating the left wheel 36(L) and a right wheel driving motor rotating the right wheel 36(R).

The cleaning unit 260 operates a brush to make it easy to suck dust or foreign substances around the mobile robot, and operates a suction device to suck the dust or foreign substances. The cleaning unit 260 controls the operation of the suction fan provided in the suction unit 34 for sucking foreign substances such as dust or garbage, so that the dust is put into the foreign material collecting bin (dust bin) through the suction port.

In addition, the cleaning unit 260 is installed at the rear of the bottom surface of the main body and in contact with the floor surface to wet the mop cleaning unit (not shown) for wiping the floor, water bucket (not shown) for supplying water to the wet mop cleaner may further include. .

The battery (not shown) supplies power required for the overall operation of the mobile robot 1 as well as the driving motor. When the battery is discharged, the mobile robot 1 may travel to return to the charging station 400 for charging, and during this return driving, the mobile robot 1 may detect the position of the charging station by itself. The charging station 400 may include a signal transmitting unit (not shown) for transmitting a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not necessarily limited thereto.

The data unit 180 stores a detection signal input from the obstacle detection unit 100 or the sensor unit 150, stores reference data for determining an obstacle, and stores obstacle information on the detected obstacle.

The data unit 180 stores obstacle data 181 for determining the type of obstacle, image data 182 in which a photographed image is stored, and map data 183 for an area. The map data 183 includes obstacle information, and the basic map for the drivable area searched by the mobile robot, the cleaning map in which the area is separated from the basic map, and the shape of the area of the cleaning map are organized so that the user can check it. The generated user map and a guide map in which the cleaning map and the user map are overlapped and displayed may be stored.

The obstacle data 181 is data for recognizing an obstacle and determining the type of the obstacle, and includes motion information on the motion of the mobile robot with respect to the recognized obstacle, for example, running speed, driving direction, avoidance, stop, etc., and a speaker. Data for sound effects, warning sounds, and voice guidance output through (173) are included. The image data 182 includes the captured image and recognition information for obstacle recognition received from the server.

In addition, the data unit 180 stores control data for controlling the operation of the mobile robot, data according to the cleaning mode of the mobile robot, and detection signals such as ultrasonic/laser by the sensor unit 150 .

The data unit 180 stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, and a CD. - May contain ROM, magnetic tape, floppy disk, optical data storage device.

The obstacle detecting unit 100 includes a first pattern irradiation unit 120 , a second pattern irradiation unit 130 , and a pattern acquisition unit 140 . When the pattern irradiation unit is not provided as shown in FIG. 1 (a) described above, the image acquisition unit may detect an obstacle from the captured image without the irradiated pattern.

In addition, a plurality of image acquisition units may be provided. When the pattern irradiation unit is provided, an image acquisition unit for photographing an image including a pattern and a separate image acquisition unit for photographing the driving direction or the upper part of the front may be further provided.

The obstacle detecting unit 100 includes a first pattern irradiation unit 120 , a second pattern irradiation unit 130 , and a pattern acquisition unit 140 . In addition, the obstacle detecting unit 100 may include a sensor unit 150 composed of at least one sensor. In some cases, the obstacle detecting unit 100 may be configured as a sensor unit.

As described above, the obstacle detection unit 100 is installed on the front side of the main body 10, irradiates the first and second patterns of light P1 and P2 in front of the mobile robot, and emits the light of the irradiated pattern. An image containing the pattern is obtained by photographing. The obstacle detection unit 100 inputs the acquired image as an obstacle detection signal to the controller 110 .

The first and second pattern irradiation units 120 and 130 of the obstacle detection unit 100 may include a light source and a pattern generator (OPPE: Optical Pattern Projection Element) that generates a predetermined pattern by transmitting the light irradiated from the light source. can The light source may be a laser diode (LD), a light emitting diode (LED), or the like. Laser light is superior to other light sources in monochromaticity, straightness, and connection characteristics, enabling precise distance measurement. Since there is a problem that is large, a laser diode is preferable as a light source. The pattern generator may include a lens and a diffractive optical element (DOE). Various patterns of light may be irradiated according to the configuration of the pattern generator provided in each of the pattern irradiation units 120 and 130 .

The pattern acquisition unit 140 may acquire an image of the front of the main body 10 . In particular, pattern lights P1 and P2 appear in the image acquired by the pattern acquisition unit 140 (hereinafter, referred to as an acquired image), and hereinafter, the images of the patterned lights P1 and P2 appearing in the acquired image are light. It is called a pattern, and since the pattern light P1 and P2 incident on the real space is the image formed on the image sensor, the same reference numerals are given to the pattern light P1 and P2, and the first pattern light P1 ) and images corresponding to the second pattern light P2 will be referred to as a first light pattern P1 and a second light pattern P2 .

When the pattern irradiation unit is not provided, the pattern acquisition unit 140 acquires an image of the front of the main body without pattern light.

The pattern acquisition unit 140 may include a camera that converts the image of the subject into an electrical signal and then converts the image into a digital signal and stores it in a memory device. The camera includes at least one optical lens, an image sensor (eg, CMOS image sensor) including a plurality of photodiodes (eg, pixels) that are imaged by light passing through the optical lens (eg, CMOS image sensor); It may include a digital signal processor (DSP) that configures an image based on the signals output from the diodes. The digital signal processor may generate a still image as well as a moving picture composed of frames composed of still images.

An image sensor is a device that converts an optical image into an electrical signal, and is composed of a chip in which a plurality of photodiodes are integrated, for example, a pixel as the photodiode. Charges are accumulated in each pixel by the image formed on the chip by the light passing through the lens, and the charges accumulated in the pixels are converted into electrical signals (eg, voltage). As an image sensor, CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor), etc. are well known.

The image processing unit generates a digital image based on the analog signal output from the image sensor. The image processing unit includes an AD converter that converts an analog signal into a digital signal, a buffer memory that temporarily records digital data according to a digital signal output from the AD converter, and a buffer memory that stores the information recorded in the buffer memory. It may include a digital signal processor (DSP: Digital Signal Processor) configured to process the digital image.

The obstacle detection unit 100 analyzes the pattern through the acquired image to detect the obstacle according to the shape of the pattern, and the sensor unit 150 detects the obstacle through the sensor provided at the sensing distance of each sensor.

The sensor unit 150 detects an obstacle including a plurality of sensors. The sensor unit 150 detects an obstacle in front of the main body 10 , that is, in a driving direction, using at least one of a laser, an ultrasonic wave, and an infrared ray. In addition, the sensor unit 150 may further include a cliff detection sensor for detecting whether a cliff exists on the floor in the driving zone. When the transmitted signal is reflected and incident, the sensor unit 150 inputs information on whether an obstacle exists or a distance to the obstacle as an obstacle detection signal to the control unit 110 .

The image acquisition unit 170 captures continuous images when the mobile robot operates. Also, the image acquisition unit 170 may capture an image at a predetermined cycle. The image acquisition unit 170 captures an image even when the obstacle is not detected by the obstacle detecting unit 100 , in a driving or cleaning state.

For example, if the image acquisition unit 170 moves in the same driving direction after shooting once, the size of the obstacle captured in the image taken once changes while the driving direction is maintained without changing the same obstacle. Since it is photographed, images are taken periodically. The image acquisition unit 170 may capture an image in units of a predetermined time or a predetermined distance. In addition, when the driving direction is changed, the image acquisition unit 170 takes a new image.

The image acquisition unit 170 may set a shooting period according to the moving speed of the mobile robot. In addition, the image acquisition unit 170 may set the shooting period in consideration of the sensing distance by the sensor unit and the moving speed of the mobile robot.

The image acquisition unit 170 stores the image captured while the main body is driving as image data 182 in the data unit 180 .

Meanwhile, the obstacle detecting unit 100 detects an obstacle located in the driving direction while driving and inputs a detection signal to the controller. The obstacle detecting unit 100 inputs information on the position or movement of the detected obstacle to the controller 110 . The pattern acquisition unit 140 inputs an image including the pattern irradiated by the pattern irradiation unit as a detection signal to the control unit, and the sensor unit 150 inputs a detection signal for an obstacle detected by the provided sensor to the control unit. .

The control unit 110 controls the traveling unit 250 so that the mobile robot travels within a specified traveling area of the traveling area H.

The controller 110 controls the driving unit 250 and the cleaning unit 260 while driving to absorb dust or foreign substances around the mobile robot, thereby cleaning the driving area. Accordingly, the cleaning unit 260 operates the brush to make it easy to suck dust or foreign substances around the mobile robot, and operates the suction device to suck the dust or foreign substances. Controls the cleaning unit to perform cleaning by sucking foreign substances while driving.

The control unit 110 checks the charging capacity of the battery to determine the time to return to the charging station. When the charging capacity reaches a predetermined value, the control unit 110 stops the operation being performed and starts searching for the charging station to return to the charging station. The controller 110 may output a notification about the charging capacity of the battery and a notification about returning to the charging station.

The control unit 110 processes the data input by the operation of the operation unit 160 to set the operation mode of the mobile robot, outputs the operation state through the output unit 190, and detects the operation state, error state, or obstacle. Alerts, sound effects, and voice guidance are output through the speaker.

In addition, the controller 110 recognizes the obstacle with respect to the obstacle detected by the image acquisition unit 170 or the obstacle detecting unit 100, and sets any one of a plurality of corresponding motions in response to the obstacle to be performed.

The control unit 110 determines an obstacle from the image captured by the image acquisition unit 170, and when the obstacle is detected to be located within a predetermined distance by the obstacle detection unit 100, sets a corresponding motion for the obstacle and operates let it do

Before the obstacle is detected by the obstacle detecting unit 100 , the controller 110 may analyze the image captured by the image acquisition unit 170 , that is, the image data to determine the obstacle included in the image data. The control unit 110 may determine the obstacle through data provided by itself, and may also transmit image data to the server or terminal through the communication unit to determine the type of the obstacle.

After the image is captured, when the obstacle detection unit 100 detects that the obstacle is located within a predetermined distance, the controller 110 transmits the detection signal to the corresponding obstacle by the sensor unit for setting a corresponding motion according to the type of the obstacle. When an obstacle is detected, the main body performs a specified operation according to the type of the obstacle recognized in advance.

The control unit 110 may perform an operation corresponding to the obstacle when the obstacle is detected according to the sensing distance of the obstacle sensing unit, and also, even if the obstacle is detected by the obstacle sensing unit, the obstacle is located within a specified distance. In this case, an operation corresponding to the obstacle may be performed.

For example, when the distance at which the obstacle is initially detected from the detection signal by the pattern acquisition unit is different from the distance detected by the ultrasonic sensor, the timing at which the detection signal for the obstacle is input is different. Accordingly, when a plurality of means for detecting an obstacle or a distance capable of detecting an obstacle such as a laser sensor is greater than a predetermined distance, the control unit 110 may Based on the distance, when the obstacle is located at a specified distance, for example, 30 cm, any one of a plurality of corresponding motions is performed.

The control unit 110 selects a corresponding motion that can be performed among a plurality of corresponding motions according to the shape of the detection signal based on the detection signal for the obstacle, analyzes the image data, recognizes the obstacle included in the image, and recognizes any one according to the motion. Select the corresponding motion of to control the operation of the main body.

For example, when the detection signal is an image included in a pattern, the controller may determine that the detection signal is a different detection signal according to the shape of the pattern, and may classify an obstacle according to the detection signal, ie, the shape of the pattern.

When an obstacle is recognized through an image, the controller 110 may perform it before a detection signal is input by the obstacle detection unit at a predetermined period with respect to the photographed image, and also determines when an obstacle detection signal is input. may be The controller 110 may transmit data to the server 90 or the terminal 80 as necessary to receive data about the obstacle.

For example, when the same detection signal is input to different obstacles, thresholds, fans, and tables, the control unit 110 selects a plurality of corresponding motions that can be operated with respect to the corresponding detection signals, and uses the image data to select a threshold, a fan, and a table. The corresponding motion is set for each to control the operation.

The control unit 110 includes an obstacle recognition unit 111 , a map generation unit 112 , and a driving control unit 113 .

The map generator 112 generates a map for the cleaning area based on the obstacle information during the initial operation or when the map for the cleaning area is not stored while driving in the cleaning area. In addition, the map generator 112 updates the previously generated map based on obstacle information acquired while driving.

The map generation unit 112 generates a basic map based on information obtained by the obstacle recognition unit 111 while driving, and generates a cleaning map by dividing an area from the basic map. In addition, the map generating unit 112 creates a user map and a guide map by organizing an area with respect to the cleaning map and setting properties for the area. The basic map is a map in which the shape of the cleaning area obtained through driving is displayed as an outline, and the cleaning map is a map in which the areas are divided in the basic map. The basic map and the cleaning map include the drivable area and obstacle information of the mobile robot. The user map is a map with visual effects added by simplifying the area of the cleaning map and arranging and processing the outlines. The guide map is a map in which the cleaning map and the user map are superimposed. Since the cleaning map is displayed on the guide map, a cleaning command can be input based on the area in which the mobile robot can actually travel.

After generating the base map, the map generator 112 divides the cleaning area into a plurality of areas, includes a connection passage connecting the plurality of areas, and generates a map including information on obstacles in each area. The map generation unit 112 generates a map in which regions are divided by dividing a small region to set a representative region for region classification on the map, setting the separated small region as a separate detailed region, and merging the divided subregion into the representative region. .

The map generating unit 112 processes the shape of each divided region. The map generator 112 sets properties for the divided areas, and processes the shape of the area according to the properties for each area.

The obstacle recognition unit 111 determines an obstacle through data input from the image acquisition unit 170 or the obstacle detection unit 100 , and the map generation unit 112 generates a map for the driving area and detects the obstacle. information is included in the map. In addition, the driving controller 113 controls the driving unit 250 to pass through or avoid the obstacle by changing the moving direction or the driving route in response to the obstacle information.

The traveling control unit 113 controls the traveling unit 250 to independently control the operation of the left wheel drive motor and the right wheel drive motor so that the main body 10 travels in a straight line or rotates. The traveling control unit 113 controls the traveling unit 250 and the cleaning unit 260 according to the cleaning command so that the main body 10 sucks foreign substances while traveling in the cleaning area to perform cleaning.

The obstacle recognition unit 111 analyzes data input from the obstacle detection unit 100 to determine an obstacle. The obstacle recognition unit 111 calculates the direction of the obstacle or the distance to the obstacle according to the detection signal of the obstacle detection unit, for example, a signal such as an ultrasonic wave or a laser, and also analyzes the acquired image including the pattern to determine the pattern. It extracts and analyzes the shape of the pattern to determine the obstacle. When using an ultrasonic wave or infrared signal, the obstacle recognition unit 111 determines the obstacle based on the difference in the shape of the ultrasonic wave received and the ultrasonic wave reception time depending on the distance to the obstacle or the position of the obstacle.

In addition, when the obstacle recognition unit 111 receives image data obtained by photographing the obstacle by the image acquisition unit 170 , it is stored in the data unit 180 . Since the image acquisition unit 170 captures the obstacle in front a plurality of times, a plurality of image data is also stored. In addition, when the image acquisition unit 170 continuously shoots images for the driving direction, the obstacle recognition unit 111 stores the input video as image data or divides the video into frame units and stores it as image data. The obstacle recognition unit 111 analyzes the video frame by frame, removes unnecessary frames, that is, frames in which the object is shaken or out of focus, or empty frames (frames in which obstacles are not photographed), save it as

The obstacle recognition unit 111 analyzes a plurality of image data, and determines whether a photographed object, that is, an obstacle can be recognized. At this time, the obstacle recognition unit 111 determines whether the image data is recognizable by analyzing the image data. For example, the obstacle recognizing unit 111 separates and discards a shaken image, an out-of-focus image, and an image that cannot distinguish an obstacle because it is dark.

The obstacle recognition unit 111 analyzes the image data and extracts the characteristics of the obstacle, determines the obstacle based on the shape (form), size, and color of the obstacle, and determines the position thereof. The obstacle recognizing unit 111 analyzes an obstacle from a plurality of pre-photographed images, and analyzes an image captured before a preset time based on a point in time when it is determined that the obstacle is located at a specified distance to determine the obstacle.

Although it is described that obstacle determination is performed through image data after obstacle determination by a detection signal, the order is not limited, and obstacle determination by a detection signal can be performed after performing obstacle determination through an image, and , can be performed simultaneously.

Since only a part of the obstacle can be photographed while approaching the obstacle within the specified distance, the obstacle is determined using the image photographed before the preset time, that is, photographed at a distance greater than the specified distance, and the entire shape of the obstacle is photographed. The obstacle recognition unit 111 may determine a specific type of obstacle, and may determine only the shape and size in some cases.

The obstacle recognition unit 111 may determine the type of obstacle by excluding the background of the image from the image data, and extracting the characteristics of the obstacle based on pre-stored obstacle data. The obstacle data 181 is updated by new obstacle data received from the server. The mobile robot 1 may store obstacle data on the detected obstacle and receive data on the type of obstacle from the server for other data.

The obstacle recognition unit 111 detects a feature such as a point, a line, or a plane with respect to predetermined pixels constituting an image, and detects an obstacle based on the detected feature.

The obstacle recognition unit 111 extracts the outline of the obstacle, recognizes the obstacle based on the shape, and determines the type of the obstacle. The obstacle recognition unit 111 may determine the type of obstacle according to the color and size of the obstacle based on the shape. Also, the obstacle recognition unit 111 may determine the type of obstacle based on the shape and movement of the obstacle.

The obstacle recognition unit 111 classifies a person, an animal, and an object based on the obstacle information. The obstacle recognition unit 111 classifies the types of obstacles into general obstacles, dangerous obstacles, biological obstacles, and floor obstacles, and may determine the detailed types of obstacles for each classification.

In addition, the obstacle recognition unit 111 transmits the recognizable image data to the server 90 through the communication unit 280 to determine the type of obstacle. The communication unit 280 transmits at least one image data to the server 90 .

When the image data is received from the mobile robot 1, the server 90 analyzes the image data to extract an outline or shape of the photographed object, and compares it with pre-stored data on the obstacle to determine the type of obstacle. . The server 90 determines the type of obstacle by first searching for obstacles of a similar shape or color, extracting features from the corresponding image data and comparing them.

After determining the type of the obstacle, the server 90 transmits data on the obstacle to the mobile robot 1 .

The obstacle recognizing unit 111 stores data on obstacles received from the server through the communication unit as obstacle data in the data unit 180 . When the type of obstacle is determined by the server, the obstacle recognition unit 111 performs an operation corresponding thereto. The traveling control unit 113 controls the traveling unit to avoid, approach, or pass through the obstacle in response to the type of the obstacle, and in some cases, a predetermined sound effect, warning sound, or voice guidance is output through the speaker.

As described above, the obstacle recognition unit 111 determines whether the image data is recognizable and transmits the image data to the server according to the stored obstacle data, thereby recognizing the type of obstacle according to the response of the server.

In addition, the obstacle recognition unit can recognize obstacles based on the obstacle recognition data without transmitting image data to the server by storing obstacle data for obstacle recognition for the obstacle selected from among the plurality of obstacles.

Since the storage capacity of the data unit 180 is limited, the controller 110 may store information on some selected obstacles as obstacle recognition data. For example, the controller 110 may store obstacle recognition data in the data unit for an obstacle selected through the terminal 80 or an obstacle with a large number of detections based on the number of detections.

Accordingly, the obstacle recognizing unit 111 stores obstacles existing in the cleaning area and repeatedly sensed obstacles in the data unit, so that when an obstacle is detected, an operation corresponding thereto can be performed immediately.

When the type of obstacle is recognized from the image data, the traveling control unit 113 controls the traveling unit 250 in response to the type of obstacle so that the main body 10 performs a predetermined operation.

When it is determined that the obstacle is located within a predetermined distance according to the detection signal of the obstacle detection unit 100, the driving control unit 113 determines the corresponding motion that can be executed according to the type or shape of the detection signal based on the image data. An operation is performed by setting any one of a plurality of corresponding motions according to the type and size of the obstacle.

In response to the obstacle recognized by the obstacle recognition unit 111, the driving control unit 113 determines whether driving is possible or enters the driving path / so as to approach the obstacle and drive, pass the obstacle, or avoid the obstacle. The driving unit 250 is controlled by setting a cleaning path.

For example, in response to the obstacle, the driving control unit 113 causes the main body 10 to stop, decelerate, accelerate, reverse run, U-turn, and change the driving direction, not to approach the obstacle more than a certain distance, and for a predetermined time. can make you wait. Also, the driving control unit 113 may output a sound designated according to an obstacle through a speaker and output the sound together with a designated operation.

When an obstacle is located within a designated distance through the obstacle detection unit 100, the driving control unit 113 sets the avoidance, approach, and approach distance according to the detection signal, and stops, decelerates, accelerates, reverse run, U-turn, and travel direction. A plurality of corresponding motions such as change are set, and any one corresponding motion is set according to an obstacle determined from pre-photographed image data to control the driving unit.

That is, the driving control unit 113 sets a plurality of corresponding motions according to the input detection signal, but selects any one of the plurality of corresponding motions in response to the obstacle determined from the pre-photographed image data of the time before the detection signal is input. The driving unit is controlled so that the corresponding motion is performed. When the type of obstacle is specifically determined through the image data, the driving control unit 113 sets the corresponding motion according to the type of the obstacle, and even when the exact type cannot be determined, the corresponding motion is set according to the shape or size of the obstacle. can be set. For example, the specific type is unknown, but if there is a space of a certain size or more from the floor, that is, a space of a height and a width that the mobile robot can pass through, the body can be set to pass through the obstacle.

The driving control unit 113 classifies dangerous situations with respect to various situations that occur during driving, and transmits the image data taken before that to the server based on the time when the dangerous situation occurs to prevent or prepare for obstacles. You can request information.

For example, when a restrained situation occurs by an obstacle after entering a predetermined obstacle, the driving control unit 113 may control an image taken for a predetermined period of time or images taken while moving a certain distance based on the time determined as the restrained situation. At least one image data may be transmitted to the server. If the moving distance for a predetermined time is less than the set distance, the driving control unit 113 may determine that it is in a restrained state.

The constrained situation refers to a state in which movement of the main body is restricted, and movement is restricted because it is impossible to move more than a predetermined distance by an obstacle.

In addition, when a dangerous situation in which a specific object falls in front of the main body or the main body occurs while driving, the driving control unit 113 may transmit image data captured before that to the server. For example, a case in which a vase or the like falls due to a collision with an obstacle while driving may occur.

The driving control unit 113 may output an error and, if necessary, output a predetermined warning sound or voice guidance.

When a dangerous situation occurs, the driving control unit 113 receives obstacle information generated based on the image data transmitted to the server, updates the obstacle information in the data unit, and determines that the obstacle is an obstacle that causes a dangerous situation with respect to the obstacle. Make a decision and avoid driving afterwards.

The server 90 analyzes a plurality of image data for a predetermined time or movement distance received to determine an obstacle that is a cause of a dangerous situation, creates a recognition model for the obstacle, and performs a response motion so as not to approach or enter. After setting, the obstacle information is updated and transmitted to the mobile robot 1 .

The server 90 may determine the obstacle causing the constraint situation to occur, set obstacle information including the characteristics of the obstacle and the corresponding motion to avoid it without entering the obstacle, and transmit it to the mobile robot. In addition, when the vase falls when the obstacle collides, the server 90 analyzes the image to determine the obstacle mounted on the vase, and sets a corresponding motion so as not to approach the obstacle to generate obstacle information.

The driving controller 113 may drive while avoiding the obstacle according to the motion corresponding to the restraint condition received from the server. When a restrained situation occurs in the detected obstacle, the driving control unit 113 may control the driving unit to avoid the obstacle after approaching the obstacle for a predetermined distance, or to avoid by changing the driving direction immediately upon detection.

7 is a diagram referenced for explaining a signal flow between a mobile robot and a device for recognizing an obstacle according to an embodiment of the present invention.

As shown in FIG. 7 , the mobile robot 1 performs cleaning while traveling in a traveling area.

The mobile robot 1 takes an image while driving and stores the image as image data.

The mobile robot 1 continuously captures images for a predetermined time or an interval of a predetermined movement distance while driving.

The mobile robot 1 may take an image regardless of whether an obstacle is detected and recognize an obstacle located in the driving direction from the image.

When determining the obstacle through the image, the controller 110 determines the obstacle by analyzing the image based on the stored obstacle information (S1).

The control unit 110 stores image data, and when an obstacle is detected by the obstacle detection unit, when the obstacle is located within a specified distance, the image data is pre-photographed to remove the background and extract features. The obstacle recognition unit 111 may determine the shape (form), size, and type of the obstacle. In some cases, before the detection signal is input by the obstacle detection unit 100, the control unit 110 analyzes, filters, and extracts the image data in a predetermined unit to determine the shape, size, or type of the obstacle. have.

When it is detected that the obstacle is located within a predetermined distance with respect to the obstacle recognized from the image, the mobile robot 1 may run while avoiding the obstacle according to the type of the obstacle previously recognized through the image.

When a detection signal is input by the obstacle detection unit 100 in a state where the determination of the obstacle is completed, the controller sets a corresponding motion for the obstacle based on the detection signal and the obstacle determination through the image and performs a designated operation.

Meanwhile, the controller 1110 may transmit a plurality of image data to the server and request the server to determine an obstacle. When data on the obstacle is received from the server, the obstacle information (S16) is stored, and the obstacle is determined accordingly.

The control unit 110 sets a corresponding motion based on the detection signal of the obstacle detection unit 100 and the obstacle determination through the image, and operates accordingly.

In addition, the mobile robot 1 may transmit the image data to the server 90 or the terminal 80 to request obstacle confirmation (S2).

The server 90 analyzes the image data to extract features of the obstacle, and determines the type of the obstacle based on the shape. The server 90 stores data on the accumulated obstacles in the database and utilizes them for obstacle determination.

Also, when the server 90 recognizes a new obstacle from the image, it creates a recognition model for the new obstacle and updates the obstacle information. The recognition model includes information on the characteristics of the obstacle, the environment in which it occurs, and the like so that the type of the obstacle can be determined from the image. In addition, the recognition model sets a corresponding action for the obstacle, and when an image of a similar obstacle is received, the characteristics are analyzed based on the pre-created obstacle model to determine the type of the obstacle, and the corresponding action is used as an avoidance action. can be set. For example, when a specific obstacle is detected, a response motion may be set for whether to immediately avoid without approaching the obstacle, to avoid it after approaching a certain distance, or to output a predetermined warning sound.

When the mobile robot 1 cannot access the server, the terminal 80 may transmit image data received from the mobile robot to the server. Also, the terminal 80 may determine the type of the obstacle based on the received image data, and may determine the type of the obstacle through a user input.

The server 90 or the terminal 80 transmits data on the type of obstacle to the mobile robot 1 in response to the request of the mobile robot (S3). In addition, the server 90 or the terminal 80 may transmit data on the avoidance operation corresponding to the type of obstacle to the mobile robot 1 .

The mobile robot 1 determines the type of obstacle according to the received data, and performs an avoidance operation corresponding thereto. The avoidance operation may also be set from the server or the terminal, and in some cases, may be operated by selecting any one of a plurality of operations. If necessary, any one of a plurality of operations may be selectively input through the terminal.

8 is a diagram referenced for explaining image transmission for obstacle recognition of a mobile robot according to an embodiment of the present invention.

The mobile robot 1 takes a plurality of images through the image acquisition unit 170 while driving. The control unit 110 allows an image to be captured while driving even if an obstacle is not detected through the obstacle detecting unit.

As shown in FIG. 8 , the mobile robot 1 captures a plurality of images 301 to 304 while driving. The mobile robot 1 may store images taken at predetermined time intervals as image data, and transmit the image data to the server 90 to request information about obstacles.

The mobile robot 1 may periodically photograph an image as a still image, and may also photograph it as a moving image through continuous shooting. The image acquisition unit 170 stores the captured image as a plurality of image data 301 to 304 in the data unit.

The image processing unit provided in the image acquisition unit 170 filters the captured image so that the image data is stored at a predetermined time or interval of a predetermined movement distance, and when a video is captured, it analyzes it frame by frame, removes unnecessary frames, and then removes the image data. save the

The mobile robot 1 may transmit image data captured while driving to the server according to time sequence, and may selectively transmit some of the plurality of image data.

For example, when the mobile robot 1 recognizes an obstacle in the driving direction from the image, the mobile robot 1 may transmit the image data to the server.

The server 90 analyzes the type of obstacle and transmits obstacle information to the mobile robot 1 , and the mobile robot 1 performs an avoidance operation on the obstacle in response to the received obstacle information. The avoidance operation may be avoided by changing the driving direction after approaching in response to the type of obstacle, or may be avoided after approaching only a specified distance, and may be driven by avoiding the obstacle as soon as the obstacle is detected. In addition, the mobile robot 1 may output a predetermined warning sound, sound effect, and voice guidance in response to the type of obstacle, and may run again after stopping for a predetermined time.

Also, when the same image is repeatedly captured within a set time, the mobile robot 1 may transmit a plurality of image data captured within a predetermined time to the server.

For example, the mobile robot 1 outputs an error when it is constrained under a chair or table and cannot move, and also transmits image data photographed within a certain time or image data photographed while moving a certain distance to the server to protect against obstacles. By receiving the information, it is possible to determine the current detention situation. The mobile robot 1 may determine that movement is impossible due to the constraint situation and output an error accordingly.

Meanwhile, the server 90 may analyze the received image data to generate a recognition model for the restraint situation of the mobile robot.

The server 90 analyzes not only the location where the mobile robot is restrained, but also a plurality of image data captured by the mobile robot for a certain distance or for a certain time to analyze the image before the restraint situation occurs, and the mobile robot is the corresponding image through the image. A recognition model is created to avoid obstacles. The server 90 generates obstacle information based on the generated recognition model and transmits it to the mobile robot.

The server 90 may analyze the cause of the restraint situation based on information about the obstacles around the restraint situation and the information about the obstacles around the restraint situation before the occurrence. The server 90 may determine that a restraint situation may occur with respect to an obstacle having a shape and size similar to that of the obstacle, and may update the obstacle information by additionally setting a corresponding motion for the restraint situation.

In response to the obstacle information received from the server, the mobile robot 1 avoids and runs before approaching the obstacle in which the restraint situation has occurred so that the restraint situation does not occur.

In addition, the server 90 may calculate statistics by analyzing the obstacles in which the restraint situation occurred, and designate a candidate group for the obstacles in which the restraint situation may occur so that a warning is output.

For example, by analyzing multiple image data for a table, the image of the table where the restraint situation occurred and the image of the table where the restraint situation did not occur are classified and compared and analyzed, and when the type of obstacle is a table, the restraint situation occurs. It is possible to designate a candidate group for the restraint situation by extracting the characteristics of the obstacles.

For example, if the height of the table is 20 cm or less, if the table legs are less than 35 cm apart, if there are other obstacles under the table, if there are 5 or more legs of the table, etc. can The server may designate a response motion for the constraint situation for the candidate group.

If necessary, the server may set a response motion to avoid the obstacle by creating a candidate group for the obstacle in which the restraint situation may not occur, but the restraint situation occurs in another similar type of obstacle. In addition, the server enters the obstacle, but outputs a warning sound or voice guidance before entering, and waits for a certain period of time, then may set a response motion to enter.

Accordingly, the mobile robot 1 performs avoidance driving by changing the driving direction after approaching a certain distance or immediately upon detecting an obstacle in which the restraint situation has occurred. The mobile robot 1 may output a voice guidance for the obstacle avoidance. For example, the mobile robot 1 may output a voice guidance such as 'It runs while avoiding obstacles that cause a restraint situation.'

In addition, when the mobile robot (1) detects an obstacle similar to an obstacle in which a restrained situation has occurred based on the received obstacle information, it is temporarily stopped and voice guidance such as 'If you continue driving, a restrained situation may occur.' After printing, you can drive. When a predetermined command is input by the user after the voice guidance, the mobile robot may continue to travel or change the traveling direction in response.

In addition, when an object falls while driving, the mobile robot 1 may determine that it is a dangerous situation and transmit a plurality of image data captured before that to the server.

The server 90 analyzes a plurality of image data to determine an obstacle that causes the object to fall, extracts it as a feature of the obstacle, and sets a corresponding motion to generate obstacle information.

For example, if a mobile robot collides with a table while placed in a vase on a table, the vase may fall. The server may generate a new recognition model for the table on which the vase is placed, and generate a corresponding motion so as not to approach it more than a certain distance with respect to it, and transmit obstacle information to the mobile robot.

Accordingly, when the mobile robot 1 detects the table on which the vase is placed, it can avoid and run without approaching more than a certain distance.

9 is a diagram referenced for explaining a method for determining a restraint situation through an image of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 9 , the mobile robot 1 captures images 311 to 315 at predetermined time intervals and stores a plurality of image data. The mobile robot 1 may store image data together with time information.

The image acquisition unit 170 may shoot images continuously or repeatedly according to a predetermined period. When the mobile robot 1 moves a certain distance, the obstacle is photographed in an enlarged form as it approaches the obstacle.

The mobile robot 1 may recognize an obstacle by analyzing the captured image, and may also receive obstacle information by transmitting it to the server 90 .

The mobile robot 1 is operated at the 0th time (t0), the 11th time (t11), the 12th time (t12), the 13th time ( t13) and at the 14th time t14, the first to fifth images 311 to 315 are captured and stored, respectively. For example, the mobile robot 1 may take an image every 10 cm moving distance. In addition, the mobile robot 1 may take images at 10-second intervals while driving.

The mobile robot 1 may determine a restrained situation when the distance moved for a certain period of time is less than a set distance while driving. For example, if the mobile robot 1 does not move more than 5 cm for a certain period of time, it may be determined as a restrained situation.

When it is determined that the mobile robot 1 is in a restrained state, it transmits image data taken before a predetermined time or a predetermined moving distance from the time P11 determined as a restrained state to the server 90 .

Based on the moving distance, the mobile robot 1 may transmit images taken before a predetermined moving distance and the twelfth to fourteenth images 312 to 314 to the server 90 . Also, the mobile robot 1 may transmit images for the eleventh time t11 to the thirteenth time t13 , that is, the twelfth to fourteenth images 312 to 314 to the server 90 .

In addition, the mobile robot 1 may transmit image data and error information to a terminal or the like, so that information on an error and a current position according to a restraint condition may be output through the terminal.

The server 90 may recognize an obstacle from each image, analyze the obstacle causing the constraint situation to occur, and generate a recognition model so as not to approach or enter the obstacle. The server 90 analyzes the characteristics of the obstacle so that the same recognition model is applied to the similar obstacle.

As shown in the figure, when a constraint situation occurs under the table, the server 90 detects the corresponding table and generates a recognition model in which a corresponding motion is set so as not to enter under the table. The server 90 may regenerate the recognition model by adding information on the restraint situation when the obstacle information for the corresponding table has been previously generated.

The mobile robot runs while avoiding obstacles so that a restraint situation does not occur by updating new obstacle information or pre-stored obstacle information. In addition, it is possible to set the constraint occurrence location on a pre-stored map (map).

Accordingly, when an image is taken as in the twelfth image 312 during the next driving, the mobile robot 1 detects the table through the image, and approaches the table by a certain distance without entering the table according to the corresponding motion for the table. You can avoid driving. When necessary, the mobile robot 1 may output a voice guidance regarding the restraint situation during evasive driving.

Also, when an obstacle having a shape similar to that of a table is detected, the mobile robot 1 may run while avoiding the same.

10 is a flowchart referenced to explain a method for controlling a mobile robot according to an embodiment of the present invention.

As shown in FIG. 10 , the mobile robot 1 acquires image information by periodically photographing an image of the driving direction while driving ( S450 ) ( S460 ). The image acquisition unit 170 takes an image, processes the image, and stores it in the data unit as recognizable image data.

The obstacle detecting unit 100 detects an obstacle positioned in the driving direction while driving and applies a predetermined detection signal to the obstacle recognition unit. The pattern irradiation units 120 and 130 irradiate a pattern of a predetermined shape in the driving direction, photograph the pattern through the pattern acquisition unit 140 , and input the pattern into the obstacle recognition unit 111 .

The obstacle recognition unit 111 analyzes the image input from the image acquisition unit 170, extracts features, and determines the type of the photographed obstacle based on the obstacle data pre-stored in the data unit 180 (S470).

Also, the controller 110 may request obstacle information by transmitting the image data to the server. The control unit 110 determines the type of the obstacle photographed in the image based on the data received from the server. The control unit 110 may receive and set information on at least a corresponding motion that can be performed with respect to the obstacle as well as the type of the obstacle from the obstacle information.

The obstacle recognition unit 111 detects whether the obstacle is located within a predetermined distance by the obstacle detecting unit 100 with respect to the previously determined obstacle (S480).

When the driving control unit 113 is located within a predetermined distance from the obstacle recognition unit 111, it is determined that the obstacle is recognized in advance from the image (S490), and corresponding to any one of a plurality of corresponding motions corresponding to the type of the obstacle. to control the driving unit 250 to avoid obstacles by setting (S500).

The driving controller 113 may perform a corresponding motion composed of a combination of at least one of approach, entry, passage, and avoidance with respect to the obstacle in response to the pre-determined type of the obstacle. In addition, the control unit 110 may output a predetermined warning sound, sound effect, and voice guidance through the output unit 190 .

When the corresponding motion is set to enter the detected obstacle, the driving controller 113 controls the driving unit 250 so that the main body continues to travel.

For example, when the table is detected, the driving control unit 113 may change the driving direction after the main body approaches the table, and may also allow the body to continue driving by entering under the table.

The obstacle recognition unit 111 continuously analyzes the image taken while driving to determine the type of the obstacle, and when the obstacle is detected within a predetermined distance by the obstacle detecting unit 100, the driving control unit 113 is configured for the obstacle. Control the driving unit by setting the corresponding motion.

In addition, the controller 110 transmits the image data to the server 90 to request information on the obstacle (S510).

The server 90 analyzes the image data received from the plurality of mobile robots to determine the type of obstacle, updates information on the previously determined obstacle, and creates a new obstacle recognition model. The server 90 transmits obstacle information based on the newly created recognition model or updated obstacle information to the mobile robot according to the request of the mobile robot.

The mobile robot 1 stores data received from the server and updates pre-stored obstacle information. The mobile robot 1 may determine an obstacle based on the new data, and transmit information about the obstacle to the server.

When a restraint situation occurs in a previously recognized obstacle, the mobile robot 1 may transmit a plurality of image data previously photographed based on the restraint state to the server to request new information on the restraint state.

The server 90 may analyze a plurality of image data to determine the surrounding obstacles before the constraint situation occurs, classify the obstacles causing the constraint situation, and set a corresponding motion therefor.

The server 90 may newly create a recognition model for an obstacle in which a restraint situation has occurred, set a corresponding motion thereto, and transmit it to the mobile robot as obstacle information.

Accordingly, the mobile robot 1 may change the pre-stored data to perform a corresponding motion for the obstacle according to new obstacle information. In addition, when a constraint situation occurs, the mobile robot may display that a constraint situation has occurred at the corresponding position and set a virtual wall as necessary.

When the obstacle recognized from the image while driving is an obstacle in which a restraint situation has occurred, the mobile robot 1 avoids the obstacle according to the corresponding motion set when the obstacle is detected by the obstacle detection unit 100. .

11 is a flowchart referenced to explain an error determination method of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 11 , while driving (S550), the mobile robot 1 periodically captures an image of the driving direction through the image acquisition unit 170 (S560). The image acquisition unit stores image data by capturing images at intervals of a predetermined period or at intervals of a predetermined distance.

The mobile robot 1 recognizes the obstacle from the photographed image data and determines the type of the obstacle. The image acquisition unit 170 stores image data captured while driving, and the control unit analyzes the image data according to a predetermined period to determine an obstacle or to detect an obstacle as being located at a specified distance by the obstacle detection unit 100 . Then, the obstacle can be determined using the pre-photographed image data. Also, the controller 110 may transmit image data to the server to receive obstacle information, and determine the type of the photographed obstacle based on the obstacle information.

Since the control unit determines information on the type, shape, size, etc. of the obstacle through the image data, when the obstacle is detected to be located within a predetermined distance by the obstacle detecting unit while driving, the control unit controls the driving unit to perform a corresponding motion in response thereto.

For example, when a detection signal for an obstacle is input to the table, the control unit 110 selects two corresponding motions of approach, avoidance, and entry, and analyzes the image data to analyze any one depending on whether it is possible to enter under the table. It can operate by determining the corresponding motion of

In addition, the traveling control unit 113 calculates the moving distance based on the number of rotations or the moving speed of the wheels of the traveling unit during driving.

The driving control unit 113 applies a control command so that the image acquisition unit 170 captures an image in units of the first moving distance when moving the set first moving distance (S570) when capturing an image based on the moving distance (S570). can do. In addition, the driving control unit 113 may control the image acquisition unit 170 when moving for a predetermined time based on the moving time.

The driving controller 113 determines whether a moving distance for a predetermined time, ie, a first time, is less than a set distance (a second moving distance) while driving ( S580 ).

When the moving distance for the first time period is less than the second moving distance, the driving controller 113 determines that it is an error because the normal driving is not performed. The driving control unit 113 may determine that it is not possible to move due to an obstacle in the vicinity and may determine it as a restrained state (S590).

In this case, after the main body enters the obstacle, the driving controller 113 may determine whether the main body is in a restrained state based on the moving distance per hour. For example, if the table is too low, the distance between the legs of the table is narrow, or there are other kinds of obstacles under the table, a restraint situation may occur in which the body cannot run normally.

If the distance moved during the set time is less than the set distance, the driving control unit 113 may determine that the main body is in a restrained state. The driving controller 113 performs avoidance driving in response to the type of obstacle.

When it is determined that the driving control unit 113 is in a restrained state, it calls a plurality of image data for a predetermined time or a predetermined distance from the data unit and transmits it to the server 90 through the communication unit (S600). Since the moving distance of the mobile robot 1 during the first time is less than the second moving distance, the mobile robot 1 transmits the image data before the occurrence of the constraint to the server.

The server 90 may analyze a plurality of received images, classify the types of obstacles by analyzing image data received from other mobile robots, calculate necessary information according to the types of obstacles, and set corresponding motions, respectively.

As image analysis and learning through a plurality of image data are performed by the server 90, recognition models are respectively generated to perform different corresponding motions depending on the shape and size of the table, even if it is a table, and accordingly, obstacle information is newly created. The generated or pre-stored obstacle information may be updated. The server may set a plurality of corresponding motions executable by the mobile robot to provide obstacle information so that any one of the corresponding motions is performed by the mobile robot.

The control unit 110 stores the obstacle information received from the server through the communication unit in the data unit (S610).

The mobile robot can avoid or escape the restraint situation based on the received data and run (S620). When escape is impossible, the control unit outputs an error and outputs a warning sound. In addition, the control unit may transmit the current location and the pre-captured image to the terminal to allow the user to release the restraint condition of the main body.

When the table is photographed, the obstacle recognizing unit may detect the table in which the restraint situation is generated through an image based on pre-stored obstacle information.

When it is determined that the obstacle is located within a predetermined distance, the driving controller avoids the obstacle table based on the changed obstacle information and drives the vehicle.

That is, when the main body is initially detected, the response motion is set to enter and then, when a restrained situation occurs, when the next detection is made, the response motion is set to either avoid or avoid the obstacle after approaching the obstacle to avoid or escape. so you can drive it.

Even if the table is of the same type, the size is different, the size of the space under the table is different, and the size of the mobile robot may also be different. Based on the detection signal, a plurality of response motions, for example, avoiding after approaching, entering or passing through, are set, but multiple responses are made according to the type, shape (shape), and size of the obstacle through image analysis. It can be determined to perform any one of the motions by designating a corresponding motion.

In addition, when another table that is not identical and similar is detected, the control unit 110 may determine whether a constraint situation will occur due to the table and set it to avoid without entering. In addition, the mobile robot may output a predetermined warning sound.

Therefore, even when the same detection signal is input, the mobile robot 1 can perform different motions depending on the obstacle through image analysis, and can perform a corresponding motion according to the size or shape of the obstacle even with the same detection signal and type of obstacle. can be done differently.

The present invention can respond immediately to obstacles through pre-recorded images, and when a restraint situation occurs, by classifying obstacles that cause restraint situations through images captured while moving for a predetermined time or a predetermined distance, When the obstacle is detected, it is possible to prevent a restraint situation from occurring by avoiding the vehicle in advance.

In addition, by creating a new recognition model for the obstacle through the server and updating the obstacle information, the obstacle is avoided in a similar situation, thereby preventing the occurrence of a restrained situation and allowing the vehicle to continue driving.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains.

1: mobile robot 10: main body
100: obstacle detection unit 110: control unit
111: obstacle recognition unit 113: driving control unit
120, 130: pattern irradiation unit 140: pattern acquisition unit
150: sensor unit 170: image acquisition unit
180: data part
250: driving unit 260: cleaning unit

Claims (24)

movable body;
an image acquisition unit that captures an image of the driving direction;
an obstacle detecting unit for detecting an obstacle located in a driving direction;
a data unit storing a plurality of image data periodically photographed through the image acquisition unit;
If the moving distance for the set time during driving is less than the set distance, it is determined that the main body is constrained by an obstacle, and if it is determined that the main body is constrained, from the time when the constraint of the main body is determined, it is taken for a predetermined time or Or, at least one of the image data captured during the previous movement is transmitted to the server, and the first obstacle that causes the restraint is determined from the server based on the image data before the main body is restrained among the obstacles in the vicinity. , a control unit for controlling to escape from the restraint situation in response to the obstacle information on the first obstacle;
includes,
The control unit sets a corresponding motion to prevent restraint of the main body in response to the obstacle information received from the server,
A mobile robot configured to operate in the corresponding motion with respect to the first obstacle and output a warning about restraint when the detected obstacle is included in a candidate that can cause a restraint situation when driving next. delete delete delete The method of claim 1,
The control unit analyzes the image data based on the previously stored obstacle information to determine the type of the obstacle located in the driving direction, and when it is determined by the obstacle detecting unit that the obstacle is located within a predetermined distance, the A mobile robot, characterized in that it performs any one of a plurality of corresponding motions for obstacles. delete The method of claim 1,
The control unit avoids the first obstacle after approaching a predetermined distance according to the corresponding motion, or immediately avoids the first obstacle when the first obstacle is detected. delete The method of claim 1,
The control unit is a mobile robot, characterized in that for outputting a warning sound or voice guidance for restraint in the corresponding motion. The method of claim 1,
The server analyzes the at least one image data, recognizes surrounding obstacles from the image data taken before the main body is restrained, and determines the first obstacle as a cause of restraint. The method of claim 1,
The server, mobile robot, characterized in that for generating the obstacle information, including a corresponding motion for avoiding the first obstacle or a corresponding motion for escaping from a restrained situation by the first obstacle. The method of claim 1,
The server generates a recognition model for the first obstacle by extracting the characteristics of the first obstacle, and in response to the characteristics of the first obstacle, restraint occurs on other obstacles having a similar shape to the first obstacle. A mobile robot, characterized in that it generates the obstacle information by setting it as a possible candidate. The method of claim 1,
The mobile robot, characterized in that the image acquisition unit captures the image at a predetermined time or a predetermined moving distance interval. The method of claim 1,
The controller is a mobile robot, characterized in that it controls to perform the corresponding motion in which at least one of stop, deceleration, acceleration, reverse running, waiting, avoidance, short-range approach prohibition, and voice guidance is combined. The method of claim 1,
The obstacle detecting unit is a mobile robot comprising at least one of an ultrasonic sensor, a laser sensor, an infrared sensor, and a sensor using pattern light. During driving, the image acquisition unit periodically captures an image of the driving direction and stores the image data;
recognizing an obstacle located in a driving direction by analyzing the image data;
detecting the obstacle by an obstacle detecting unit;
performing any one of a plurality of corresponding motions in response to the type of the obstacle previously recognized from the image data when the obstacle is detected;
In response to the driving condition, the moving distance during the set time is compared with the set distance,
determining that the main body is constrained when the moving distance during the set time is less than the set distance, and transmitting at least one image data photographed during the moving for a predetermined time or a previous predetermined distance from the time of determining the restraint to the server;
receiving, from the server based on the image data, obstacle information on a first obstacle that is a cause of restraint among nearby obstacles before the main body is restrained;
escaping from a restraint situation in response to the obstacle information;
setting a corresponding motion with respect to the first obstacle to prevent restraint of the main body in response to the obstacle information; and
In the next driving, avoiding the first obstacle by operating in the corresponding motion with respect to the first obstacle;
When the obstacle detected by the obstacle detection unit is included in a candidate for which a restraint situation can occur, performing any one of the plurality of corresponding motions to output a warning about restraint Control method of mobile robot.
delete delete delete delete delete delete 17. The method of claim 16,
and generating, by the server, the obstacle information including at least one of a corresponding motion for avoiding the first obstacle and a corresponding motion for escaping from a restrained situation. 24. The method of claim 23,
generating, by the server, a recognition model for the obstacle by extracting the features of the first obstacle;
setting, in response to the characteristic of the first obstacle, as a candidate in which restraint can occur with respect to another obstacle having a shape similar to that of the first obstacle; and
The control method of the mobile robot further comprising the step of generating the obstacle information for the candidate.

Images