KR20230158668A - Robot and controlling method thereof - Google Patents

Abstract

The robot starts. The robot includes a communication interface, a camera, a sensor module including a plurality of sensors, a driving unit, and a processor that controls the driving unit based on control commands received from the user terminal through the communication interface. The processor acquires image data through a camera and acquires driving information of the robot based on sensing data and image data obtained through at least one sensor among a plurality of sensors. Additionally, the processor identifies driving information within a threshold time based on the point in time at which the control command is received among the driving information, and identifies whether the operation corresponding to the control command was normally performed based on the identified driving information. Furthermore, when the processor identifies that an operation corresponding to a control command has been performed abnormally, it transmits guide UI (User Interface) information related to the abnormal operation to the user terminal through a communication interface.

Description

Robot and its control method {ROBOT AND CONTROLLING METHOD THEREOF}

The present invention relates to robots and their control methods.

Recently, technology development for robots that are deployed in indoor spaces and provide services to users has become active. In particular, the demand for robots that provide services while driving in indoor spaces is increasing, and recently, the method of allowing users to remotely control robots and receive services has become common.

However, problems may arise where the robot does not operate according to the user's commands due to various reasons, and accordingly, there is a continuous demand for robots that provide notifications regarding robots that do not operate according to the user's commands and further solve the above-mentioned problems on their own. There was.

The present disclosure is in response to the above-described need, and identifies whether a robot has normally performed an operation corresponding to a user's control command based on a plurality of data acquired through cameras and sensors, and when an abnormal operation is identified, the corresponding operation is performed. The aim is to provide a guide UI related to the robot and its control method that performs alternative movements for abnormal movements.

To achieve the above object, a robot according to an embodiment of the present invention includes a communication interface, a camera, a sensor module including a plurality of sensors, a driving unit, and a control command received from a user terminal through the communication interface. and a processor that controls the driving unit, wherein the processor acquires image data through the camera and drives the robot based on the sensing data and the image data acquired through at least one sensor among the plurality of sensors. Obtain information, identify driving information within a threshold time based on the point at which the control command is received among the driving information of the robot, and determine whether the operation corresponding to the control command was normally performed based on the identified driving information. is identified, and if the operation corresponding to the control command is identified as being performed abnormally, guide UI (User Interface) information related to the abnormal operation may be transmitted to the user terminal through the communication interface.

Here, the sensor module includes a distance sensor and an odometry sensor, and the processor determines a first movement distance of the robot based on the image data acquired within the threshold time and the distance within the threshold time. Identify the moving distance of the robot during the critical time based on the second moving distance of the robot obtained through the sensor and the third moving distance of the robot obtained through the odometry sensor within the critical time, and By comparing the identified movement distance of the robot and the movement distance corresponding to the control command, it is possible to identify whether the operation corresponding to the control command was normally performed.

Here, the processor may identify a middle value among the first movement distance, the second movement distance, and the third movement distance as the movement distance of the robot.

Here, the processor identifies a first distance deviation and a second distance deviation by comparing the identified movement distance with each of the remaining movement distances among the first movement distance, the second movement distance, and the third movement distance, When at least one of the first distance deviation or the second distance deviation is greater than or equal to a first threshold value, it can be identified that an operation corresponding to the control command has been performed abnormally.

In addition, the sensor module includes an inertial sensor and an odometry sensor, and the processor includes a first rotation angle of the robot based on the image data acquired within the threshold time, the threshold Based on the second rotation angle of the robot obtained through the inertial sensor within the time and the third rotation angle of the robot obtained through the odometry sensor within the critical time, the rotation angle of the robot during the critical time is determined. It is possible to identify whether the operation corresponding to the control command was normally performed by comparing the rotation angle of the identified robot and the rotation angle corresponding to the control command.

Here, the processor may identify a middle value among the first rotation angle, the second rotation angle, and the third rotation angle as the rotation angle of the robot.

Here, the processor compares the identified rotation angle with each of the remaining rotation angles among the first rotation angle, the second rotation angle, and the third rotation angle to identify a first angle deviation and a second angle deviation, When at least one of the first angle deviation or the second angle deviation is greater than or equal to a second threshold value, it can be identified that the operation corresponding to the control command has been performed abnormally.

In addition, the sensor module includes an inertial sensor, and the processor is configured to obtain first shake information of the robot based on the image data and the first shake information obtained through the inertial sensor for the same time as the time at which the first shake information was acquired. Compare the second shaking information of the robot based on the shaking data to identify whether the operation of the inertial sensor is performed normally, and if it is identified that the operation of the inertial sensor is performed abnormally, the inertial sensor is reset (Reset) ), and guide UI (User Interface) information related to the abnormal operation can be transmitted to the user terminal through the communication interface.

In addition, the processor may identify the type of the abnormal operation, obtain solution information related to the identified type, and transmit the guide UI information including the solution information to the user terminal through the communication interface. there is.

In addition, it further includes a battery, wherein the processor acquires the driving information based on a first number of data among a plurality of data acquired through the camera and the plurality of sensors when the power amount of the battery is less than a threshold value, and , when the amount of power of the battery is greater than or equal to a threshold value, the driving information may be obtained based on the second number of data that is greater than the first number of the plurality of data.

Meanwhile, according to an embodiment of the present invention, a method of controlling a robot driven based on a control command received from a user terminal includes acquiring image data through a camera, and obtaining image data through at least one sensor among a plurality of sensors. Obtaining driving information of the robot based on the sensing data and the image data, identifying driving information within a threshold time based on the time when the control command is received among the driving information of the robot, and the identified driving information Based on the step of identifying whether the operation corresponding to the control command was performed normally, and if the operation corresponding to the control command is identified as being performed abnormally, providing guide UI (User Interface) information related to the abnormal operation It may include transmitting to the user terminal.

Here, the step of identifying whether the operation corresponding to the control command was normally performed includes the first movement distance of the robot based on the image data acquired within the threshold time, and the first movement distance of the robot obtained through a distance sensor within the threshold time. Identifying the moving distance of the robot during the critical time based on the second moving distance of the robot and the third moving distance of the robot obtained through an odometry sensor within the critical time, and the identified It may include comparing the movement distance of the robot and the movement distance corresponding to the control command to identify whether the operation corresponding to the control command was normally performed.

Here, the step of identifying the moving distance of the robot may identify a middle value among the first moving distance, the second moving distance, and the third moving distance as the moving distance of the robot.

Here, the step of identifying whether the operation corresponding to the control command was performed normally includes comparing the identified movement distance with each of the remaining movement distances among the first movement distance, the second movement distance, and the third movement distance. identifying a first distance deviation and a second distance deviation, and identifying that an operation corresponding to the control command is abnormally performed when at least one of the first distance deviation or the second distance deviation is greater than or equal to a first threshold value. May include steps.

In addition, the step of identifying whether the operation corresponding to the control command was normally performed includes the first rotation angle of the robot based on the image data acquired within the threshold time, and the first rotation angle of the robot obtained through an inertial sensor within the threshold time. Identifying a rotation angle of the robot during the critical time based on a second rotation angle of the robot and a third rotation angle of the robot obtained through an odometry sensor within the critical time, and the identified It may include comparing the rotation angle of the robot and the rotation angle corresponding to the control command to identify whether the operation corresponding to the control command was normally performed.

Here, the step of identifying the rotation angle of the robot may identify a middle value among the first rotation angle, the second rotation angle, and the third rotation angle as the rotation angle of the robot.

Here, the step of identifying whether the operation corresponding to the control command was normally performed includes comparing the identified rotation angle with each of the remaining rotation angles among the first rotation angle, the second rotation angle, and the third rotation angle. identifying a first angle deviation and a second angle deviation, and identifying that an operation corresponding to the control command is abnormally performed when at least one of the first angle deviation or the second angle deviation is greater than or equal to a second threshold value. May include steps.

In addition, the first shake information of the robot based on the image data is compared with the second shake information of the robot based on shake data acquired through an inertial sensor for the same time as the time at which the first shake information was acquired. Identifying whether the operation of the inertial sensor is performed normally, and if it is identified that the operation of the inertial sensor is performed abnormally, resetting the inertial sensor and providing guide UI (User Interface) information related to the abnormal operation. The step of transmitting to the user terminal may be further included.

In addition, the step of transmitting guide UI information to the user terminal includes identifying the type of the abnormal operation, obtaining solution information related to the identified type, and transmitting the guide UI information including the solution information to the user. It may include the step of transmitting to the terminal.

In addition, the step of acquiring the driving information of the robot includes acquiring the driving information based on a first number of data among a plurality of data obtained through the camera and the plurality of sensors when the power amount of the battery is less than a threshold value. It may include obtaining the driving information based on the second number of data that is greater than the first number of the plurality of data when the power amount of the battery is greater than or equal to a threshold value.

According to various embodiments of the present disclosure, the user's convenience can be improved because the user can not only receive notifications about the abnormal operation of the robot, but also the robot can perform alternative actions to solve the problem on its own.

1 is a diagram schematically illustrating the operation of a robot remotely controlled by a user.
Figure 2 is a block diagram for explaining the configuration of a robot according to an embodiment of the present disclosure.
Figure 3 is a diagram for explaining driving information according to an embodiment of the present disclosure.
Figure 4 is a diagram for step-by-step explaining the operation of a robot according to an embodiment of the present disclosure.
Figure 5 is a diagram for explaining in more detail the operation of a robot according to an embodiment of the present disclosure.
6A and 6B are diagrams for explaining an alternative operation of a robot according to an embodiment of the present disclosure.
7A and 7B are diagrams for explaining a sensor reset operation of a robot according to an embodiment of the present disclosure.
Figure 8 is a diagram for explaining the operation of a robot including a head according to an embodiment of the present disclosure.
Figure 9 is a flowchart for explaining the operation of a robot considering the battery state according to an embodiment of the present disclosure.
Figure 10 is a block diagram for specifically explaining the configuration of a robot according to an embodiment of the present disclosure.
Figure 11 is a flowchart for explaining a control method according to an embodiment of the present disclosure.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

The terms used in the embodiments of the present disclosure have selected general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the art, the emergence of new technology, etc. . In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description part of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

In the present disclosure, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

The expression at least one of A or/and B should be understood as referring to either “A” or “B” or “A and B”.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as “connected to,” it should be understood that a certain component can be connected directly to another component or connected through another component (e.g., a third component).

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “consist of” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented by at least one processor (not shown), except for “modules” or “units” that need to be implemented with specific hardware. It can be.

In this disclosure, 'user' may refer to a person who receives services from a robot, but is not limited thereto.

1 is a diagram schematically illustrating the operation of a robot remotely controlled by a user.

The robot 100 according to an embodiment of the present disclosure can provide a service to the user 10 while traveling in a specific space. For example, the robot 100 may provide a cleaning service to the user 10, but is not limited to this. Here, the robot 100 can operate and provide services based on control commands received from the user terminal 200.

The user 10 can remotely control the robot 100 through the user terminal 200. The user 10 may transmit a control command for driving the robot 100 to the robot 100 through the user terminal 200. Here, the user terminal 200 may be implemented as an electronic device. For example, the user terminal 200 may be a smartphone, tablet PC, mobile phone, video phone, e-book reader, desktop PC, laptop PC, netbook computer, workstation, server, PDA, PMP (portable multimedia player), MP3 It can be implemented in various forms such as players, medical devices, cameras, or wearable devices, but is not limited to these.

Meanwhile, while the robot 100 is providing a service, an error may occur in the operation of the robot 100. These motion abnormalities may be related to at least one of problems with the hardware configuration included in the robot 100, problems with the software mounted on the robot 100, or problems caused by the driving environment or surrounding objects of the robot 100. there is.

For example, if the condition of the road surface on which the robot 100 runs is not good, the robot 100 may not accurately perform an operation corresponding to a control command received from the user terminal 200. For example, when a control command instructs the robot 100 to move forward a certain distance, the robot 100 running on a slippery road cannot move forward and performs an abnormal motion of sliding on the road surface.

In this case, the robot 100 may identify that the above-described abnormal operation has occurred and transmit guide UI (User Interface) information related to the identified abnormal operation to the user terminal 200. The user 10 can receive guide UI information through the user terminal 200, through which the user can take appropriate measures for abnormal operation of the robot 100.

Separately from this, the robot 100 can identify alternative movements related to abnormal movements. Here, the alternative operation may mean an operation that the robot 100 must perform in order to cope with the cause of the abnormal operation. The robot 100 can perform an operation corresponding to the control command received from the user terminal 200 through the identified alternative operation.

Hereinafter, various embodiments will be described that identify whether the operation corresponding to the user's control command has been performed normally, provide a guide UI related to the operation when an abnormal operation is identified, and perform an alternative operation for the abnormal operation. Let me explain in more detail.

Figure 2 is a block diagram for explaining the configuration of a robot according to an embodiment of the present disclosure.

According to FIG. 2, the robot 100 may include a communication interface 110, a camera 120, a sensor module 130, a driver 140, and a processor.

The communication interface 110 can input and output various types of data. For example, the communication interface 110 includes AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired/wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES/EBU (Audio Engineering Society/ European Broadcasting Union), Optical , external devices (e.g., source devices), external storage media (e.g., USB memory), and external servers (e.g., web hard drives) and various types of data through communication methods such as coaxial, etc. Can send and receive.

The camera 120 is configured to acquire image data corresponding to the surrounding environment of the robot 100. The camera 120 may include a lens that focuses visible light and other optical signals reflected by an object and received into an image sensor, and an image sensor that can detect visible light and other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels.

The processor 150 receives optical signals of various wavelengths, such as visible light or infrared light, through the lens of the camera 120, and processes the received optical signals through the image sensor of the camera 120 to obtain image data. . Additionally, the camera 120 according to one example may be implemented as a depth camera, and in this case, the processor 150 may acquire image data including a depth image through the camera 120.

Meanwhile, the camera 120 may be implemented in the form of a camera module including a plurality of lenses and a plurality of image sensors. According to one example, the camera 120 may be a stereo camera including two lenses and two image sensors, but is not limited thereto.

The sensor module 130 is configured to include a plurality of sensors. Here, the plurality of sensors include a distance sensor, an odometry sensor, an inertial sensor, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, a grip sensor, a proximity sensor, and a color sensor (e.g., RGB (red, green, blue) sensor), a biometric sensor, a temperature/humidity sensor, an illumination sensor, a UV (ultra violet) sensor, or a microphone, but is not limited thereto.

The driving unit 140 is a component that drives the robot 100. The driving unit 140 can adjust the traveling direction and traveling speed according to the control of the processor 150. Additionally, the driving unit 140 may change the posture of the robot 100 by driving independently moving members of the robot 100.

The driving unit 140 according to one example is a power generator that generates power for the robot 100 to travel (e.g., a gasoline engine, a diesel engine, or a liquefied petroleum gas (LPG) depending on the fuel (or energy source) used. ) engines, electric motors, etc.), steering devices to control the driving direction (e.g., manual steering, hydraulics steering, electronic control power steering (EPS), etc.), robots depending on power. It may include a traveling device (e.g., wheels, propeller, etc.) that drives (100). Here, the driving unit 134 may be modified according to the driving type of the robot 100 (e.g., wheel type, walking type, flying type, ball type, etc.).

The processor 150 generally controls the operation of the robot 100. Specifically, the processor 145 is connected to each component of the robot 100 and can generally control the operation of the robot 100. For example, the processor 150 may be connected to the communication interface 110, the camera 120, the sensor module 130, and the driver 140 to control the operation of the robot 100.

According to one embodiment, the processor 150 includes a digital signal processor (DSP), a microprocessor, a central processing unit (CPU), a micro controller unit (MCU), and a micro processing unit (MPU). It may be named by various names such as unit), NPU (Neural Processing Unit), controller, and application processor (AP), but in this specification, it is referred to as processor 150.

The processor 150 may be implemented as a System on Chip (SoC), large scale integration (LSI), or may be implemented as a Field Programmable Gate Array (FPGA). Additionally, the processor 150 may include volatile memory such as SRAM.

The processor 150 may control the driver 140 based on a control command received from the user terminal through the communication interface 110. Here, the control command may include at least one of information on the distance at which the robot 100 should move or information on the angle at which the robot 100 should rotate.

Hereinafter, the operation of the robot 100, which is performed as the processor 150 controls the driving unit 140 and corresponds to the control command received from the user terminal, will be expressed by the term 'request operation'.

The processor 150 may acquire image data including an image corresponding to the surrounding environment of the robot 100 through the camera 120. Additionally, the processor 150 may acquire sensing data through at least one sensor among the plurality of sensors included in the sensor module 130.

Next, the processor 150 may obtain driving information of the robot 100 based on the image data and sensing data. Here, the driving information of the robot 100 may include, but is not limited to, at least one of movement distance information, rotation angle information, or shaking information of the robot 100.

In this regard, the processor 150 determines the first movement distance of the robot 100 based on the image data acquired within the critical time, the distance of the robot 100 acquired through the distance sensor included in the sensor module 130 within the critical time. The second movement distance and the third movement distance of the robot 100 obtained through the odometry sensor included in the sensor module 130 within the critical time can be identified, respectively.

In addition, the processor 150 may set a first rotation angle of the robot 100 based on image data acquired within a critical time, and a second rotation angle of the robot 100 acquired through an inertial sensor included in the sensor module 130 within the critical time. The third rotation angle of the robot 100 obtained through the odometry sensor included in the sensor module 130 within the rotation angle and critical time can be identified.

Furthermore, the processor 150 may identify driving information within a threshold time based on the time when the control command is received. The threshold time according to one example may be determined based on setting information included in the control command, or may be determined based on the control command received by the processor 150. In the above, it has been described that the processor 150 identifies driving information within a threshold time based on the time the control command is received. However, according to another example, the processor 150 obtains the driving information within the critical time based on the time the control command is received. It is also possible to obtain driving information based on image data and sensing data.

The processor 150 may identify whether the requested operation corresponding to the control command was normally performed based on the identified driving information. According to one example, the processor 150 compares the distance information to be moved by the robot 100 included in the control command with the movement distance information included in the driving information, or the processor 150 compares the distance information to be moved by the robot 100 included in the control command. It is also possible to identify whether the requested operation was performed normally by comparing the angle information and the rotation angle information included in the driving information. Here, even if the robot 100 did not perfectly perform the requested action based on the driving information, the processor 150 normally performs the requested action if the action performed by the robot 100 has a similarity level higher than a critical level compared to the requested action. It can be identified as having been performed.

In this regard, the processor 150 uses the middle value among the first movement distance of the robot 100, the second movement distance of the robot 100, and the third movement distance of the robot 100 as the movement distance of the robot 100. It is possible to identify whether the requested operation was performed normally by comparing the movement distance of the identified robot 100 with the distance to be moved by the robot 100 included in the control command.

In addition, the processor 150 identifies the middle value among the first rotation angle of the robot 100, the second rotation angle of the robot 100, and the third rotation angle of the robot 100 as the rotation angle of the robot 100, , it is possible to identify whether the requested operation was performed normally by comparing the rotation angle of the identified robot with the angle at which the robot 100 should rotate included in the control command.

According to another example, the processor 150 may compare information obtained from different sensors in relation to various types of information included in the driving information to identify whether the requested operation was performed normally. Here, the processor 150 may identify that the requested operation has been performed normally when information obtained from different sensors has a similarity level greater than or equal to a threshold level.

In this regard, the processor 150 identifies the middle value among the first movement distance, the second movement distance, and the third movement distance as the movement distance of the robot 100, and the movement distance of the robot 100 and the remaining two movement distances. The first distance deviation and the second distance deviation can be identified by comparing each. The processor 150 may identify that the requested operation has been performed abnormally when at least one of the first distance deviation or the second distance deviation is greater than or equal to the first threshold value.

Additionally, the processor 150 identifies the middle value among the first rotation angle, the second rotation angle, and the third rotation angle as the rotation angle of the robot 100, and uses the rotation angle of the robot 100 and each of the remaining two rotation angles as the rotation angle of the robot 100. By comparison, the first angle deviation and the second angle deviation can be identified. The processor 150 may identify that the requested operation has been performed abnormally when at least one of the first angle deviation or the second angle deviation is greater than or equal to the second threshold value.

If the requested operation is identified as being performed abnormally, the processor 150 may transmit guide UI (User Interface) information related to the identified abnormal operation to the user terminal through the communication interface 110. Here, the guide UI information may include, but is not limited to, information about at least one of the occurrence of abnormal operation, type of abnormal operation, cause of occurrence of abnormal operation, or user action tips for abnormal operation.

Furthermore, the processor 150 may obtain first shaking information of the robot 100 based on image data. In addition, the processor 150 may acquire second shaking information of the robot 100 based on shaking data acquired through the inertial sensor included in the sensor module 130 for the same time as the time at which the first shaking information was acquired. You can.

The processor 150 may identify whether the inertial sensor is operating normally by comparing the first shaking information and the second shaking information. If the operation of the inertial sensor is identified as abnormal, the processor 150 may reset the inertial sensor. Additionally, the processor 150 may transmit guide UI information related to the abnormal operation of the inertial sensor to the user terminal through the communication interface 110.

When the processor 150 identifies that the operation of the robot 100 is performed abnormally, the processor 150 may identify the type of the abnormal operation. Furthermore, the processor 150 may obtain solution information related to the identified type. Here, the solution information includes at least one of the problem cause corresponding to the type of abnormal operation, user action tips corresponding to the type of abnormal operation, or information about an alternative operation of the robot 100 corresponding to the type of abnormal operation. It may include, but is not limited to this. The processor 150 may transmit guide UI information including solution information to the user terminal through the communication interface 110.

Additionally, the robot 100 may further include a battery. If the power amount of the battery is less than a threshold value, the processor 150 may obtain driving information based on a first number of data among a plurality of data acquired through the camera 120 and a plurality of sensors. When the amount of power of the battery is greater than or equal to the threshold, the processor 150 may obtain driving information based on a second number of data that is greater than the first number of the plurality of data. Here, the threshold value may be a value determined according to characteristic information of the camera 120 and each of the plurality of sensors.

Figure 3 is a diagram for explaining driving information according to an embodiment of the present disclosure.

According to FIG. 3 , the processor 150 may identify driving information 300 of the robot 100 including movement distance information 310, rotation angle information 320, and shaking information 330. Here, the movement distance information 310 includes information about the distance the robot 100 has moved on the running surface within a critical time after receiving a control command through the communication interface 110, and the rotation angle information 320 is It includes information about the angle at which the robot 100 rotated on the running surface within a critical time after the control command is received, and the shaking information 330 indicates that the robot 100 rotates on a preset axis within the critical time after the control command is received. It may contain information about the degree of rotation relative to the reference.

Specifically, the moving distance information 310 includes moving distance information 311 calculated based on image data acquired through the camera 120, and moving distance information calculated based on distance data acquired through a distance sensor. (312) and may include movement distance information (313) calculated based on odometry data acquired through an odometry sensor.

The processor 150 compares the image data acquired when receiving the control command with the image data acquired when the threshold time elapses after receiving the control command, and determines how many pixels (or groups of pixels) matching each image data are before and after the threshold time. It is possible to identify a plurality of vector information regarding movement, and obtain image data-based movement distance information 311 based on the average value of the sizes of the plurality of vectors.

The processor 150 may obtain distance data-based movement distance information 312 based on Simultaneous Localization and Mapping (SLAM) method. For example, the processor 150 identifies the location of the robot 100 when receiving a control command based on distance data acquired through a distance sensor when receiving a control command, and detects the location of the robot 100 through the distance sensor when a threshold time has elapsed after receiving the control command. Based on the acquired distance data, the position of the robot 100 may be identified after a threshold time has elapsed, and distance data-based movement distance information 312 may be obtained as the distance between the two identified locations.

The processor 150 may obtain odometry data related to the movement of the driver 140 through the odometry sensor from the time the control command is received until the critical time elapses. For example, when the driving unit 140 includes a left wheel and a right wheel, the processor 150 may include first odometry data related to the degree to which the left wheel rotated during the critical time and the degree to which the right wheel rotated during the critical time. Second odometry data related to can be obtained. Additionally, the processor 150 may obtain odometry data-based movement distance information 313 based on first odometry data, second odometry data, and characteristic information about the driving unit 140.

The processor 150 according to one example may identify the middle value among the movement distance information 311, 312, and 313 as the movement distance information of the robot 100. According to FIG. 3, the image data-based moving distance information 311 indicates 3m, the distance data-based moving distance information 312 indicates 2.9m, and the odometry data-based moving distance information 313 indicates 5m, so the processor ( 150 may identify the image data-based movement distance information 311 as movement distance information of the robot 100.

Rotation angle information 320 includes rotation angle information 321 calculated based on image data acquired through the camera 120, and rotation angle information 322 calculated based on inertial data acquired through an inertial sensor. And it may include rotation angle information 323 calculated based on odometry data acquired through an odometry sensor.

The processor 150 compares the image data acquired when receiving the control command with the image data acquired when the threshold time elapses after receiving the control command, and determines how many pixels (or groups of pixels) matching each image data are before and after the threshold time. A plurality of vector information regarding movement can be identified, and image data-based rotation angle information 321 can be obtained based on the average value of the directions of the plurality of vectors.

The processor 150 may acquire inertial data-based rotation angle information 322 based on inertial data acquired through an inertial sensor from the time a control command is received until the threshold time elapses. For example, since the rotation angle information 320 is information indicating the degree to which the robot 100 rotates based on the running surface of the robot 100, the processor 150 selects the robot ( Inertial data-based rotation angle information 322 may be obtained based on inertial data regarding the degree to which the robot 100 rotates (yawing) relative to an axis perpendicular to the running surface of the robot 100, but the inertial data-based rotation angle information 322 is not limited thereto.

The processor 150 is based on the first odometry data, second odometry data, and characteristic information about the driver 140 obtained through the odometry sensor from the time the control command is received until the critical time elapses. Odometry data-based rotation angle information 323 can be obtained. For example, the processor 150 may operate the robot 100 when the movement of the driving unit 140 identified by the first odometry data is greater than the movement of the driving unit 140 identified by the second odometry data. It can be identified as having rotated in the right direction, and in this case, the rotation angle can be determined based on the difference value of the movement of the drive unit 140 indicated by the first odometry data and the second odometry data, respectively.

The processor 150 according to an example may identify the middle value among the rotation angle information 321, 322, and 323 as the rotation angle information of the robot 100. According to FIG. 3, the image data-based rotation angle information 321 indicates 30 degrees, the inertial data-based rotation angle information 322 indicates 50 degrees, and the odometry data-based rotation angle information 323 indicates 40 degrees, The processor 150 may identify the odometry data-based rotation angle information 323 as rotation angle information of the robot 100.

The shaking information 330 may include shaking information 331 calculated based on image data acquired through the camera 120 and shaking information 332 calculated based on inertial data acquired through an inertial sensor. You can. Here, the shaking information 330 may include information about the degree of rolling, pitching, and yawing of the robot 100, but is not limited thereto.

The processor 150 compares the image data acquired when receiving the control command with the image data acquired when the threshold time elapses after receiving the control command, and determines how many pixels (or groups of pixels) matching each image data are before and after the threshold time. A plurality of vector information regarding movement can be identified, and image data-based shake information 331 can be obtained based on the variance value for the size of the plurality of vectors.

The processor 150 may obtain inertial data-based shaking information 332 based on inertial data acquired through an inertial sensor from the time a control command is received until the threshold time elapses. For example, the processor 150 may acquire inertial data-based shaking information 332 based on inertial data obtained through an inertial sensor and corresponding to the degree to which the robot 100 shakes based on a preset axis. . When the preset axes are three axes, the processor 150 according to an example may obtain inertial data-based shaking information 332 based on the average value of rolling, pitching, and yawing values.

The processor 150 according to one example may identify the average value of each shaking information 331 and 332 as shaking information of the robot 100. According to FIG. 3, since the image data-based shaking information 331 indicates 200 and the inertial data-based shaking information 332 indicates 300, the processor 150 can identify 250 as the degree of shaking of the robot 100.

Figure 4 is a diagram for step-by-step explaining the operation of a robot according to an embodiment of the present disclosure.

According to FIG. 4, the sensor module 130 may include a distance sensor 131, an odometry sensor 132, and an inertial sensor 133. Additionally, the robot 100 may be additionally equipped with a battery 160. The processor 150 performs the requested operation indicated by the user through a control command based on information obtained from the camera 120, a plurality of sensors 131, 132, and 133 included in the sensor module 130, and the battery 160. It can be determined whether this was performed normally.

The processor 150 may analyze image data obtained from the camera 120 as the robot 100 operates (S410). For example, the processor 150 acquires movement distance information of the robot 100 based on the image data, acquires rotation angle information of the robot 100 based on the image data, and determines the robot (100) based on the image data. 100) of shaking information can be obtained.

According to one example, when the robot 100 is composed of a head and a body, the processor 150 processes the robot 100 based on inertial data acquired through the inertial sensor 133 provided on the head of the robot 100. Shaking information about the movement of the head can be obtained (S421). Additionally, the processor 150 may obtain shaking information about the movement of the body of the robot 100 based on inertial data acquired through the inertial sensor 133 provided in the body of the robot 100 (S422 ).

The processor 150 may calculate the change in position of the robot 100 by processing the distance data acquired through the distance sensor 131 using the SLAM method (S430). Through this, the processor 150 can obtain movement distance information based on distance data.

The processor 150 may calculate the expected movement of the robot 100 through the movement of the driving unit 140 based on odometry data acquired through the odometry sensor 132 (S440). Through this, the processor 150 can obtain odometry data-based movement distance information and odometry data-based rotation angle information.

The processor 150 can determine whether the robot 100 is in a low-power state based on the state of the battery 160 and whether each sensor operates normally under the identified battery state (S450).

The processor 150 may determine whether to perform the requested operation based on information obtained from the camera 120, the plurality of sensors 131, 132, and 133, and the battery 160 (S460). If the requested operation is not sufficiently performed, the processor 150 may identify the type of abnormal operation of the robot 100 and provide solution information related to the identified type (S470). Here, the processor 150 may provide solution information by transmitting guide UI information including solution information to the user terminal through the communication interface 110, but the robot 100 itself provides UI corresponding to the solution information. You may.

Next, the processor 150 determines whether an alternative operation of the robot 100 corresponding to the type of abnormal operation can be performed based on the solution information (S480). For example, the processor 150 may determine whether the robot 100 can solve the problem by executing an alternative operation for the abnormal operation on its own without user intervention based on the type of abnormal operation.

If execution of the alternative operation is possible (S480: Y), the processor 150 may control the driver 140 to execute the alternative operation corresponding to the solution information (S491). If it is impossible to execute an alternative operation (S480: N), the processor 150 may stop the operation of the driving unit 140 and then control the robot 100 in a standby state until a subsequent control command is received from the user terminal. (S492).

Figure 5 is a diagram for explaining in more detail the operation of a robot according to an embodiment of the present disclosure.

The processor 150 may obtain driving information based on information obtained from the camera 120, the plurality of sensors 131, 132, and 133, and the battery 160 (510). For example, the processor 150 may calculate the moving distance 511 based on image data, the degree of shaking 514 based on image data, and the rotation angle 516 based on image data through image data analysis (S410). Additionally, the processor 150 may calculate a moving distance based on distance data through SLAM-based position change calculation of the robot 100 (S430) (512).

In addition, the processor 150 calculates the movement of the robot 100 (S440), which is expected through the movement of the driving unit 140, and calculates the odometry data-based movement distance 513 and the odometry data-based rotation angle 518. It can be calculated by obtaining shaking information about the movement of the head of the robot 100 (S421) and acquiring shaking information about the movement of the body of the robot 100 (S422) to determine the degree of shaking based on inertial data (515) and rotation based on inertial data. The angle 517 can be calculated.

Subsequently, the processor 150 determines whether to perform the requested operation based on the driving information including the moving distance of the robot 100, the rotation angle of the robot 100, and the degree of shaking of the robot 100 calculated in the above-described manner. And, if the requested operation is not performed normally, guide UI information corresponding to the abnormal operation may be provided (520).

In this process, the processor 150 can identify information to be used to determine whether to perform the requested operation within the driving information obtained through determining whether each sensor is operating abnormally (S450). For example, if the camera 120 does not operate normally under the identified battery state, the processor 150 determines the moving distance 511 based on image data, the degree of shaking 514 based on image data, and the rotation angle 516 based on image data. It can be determined whether to perform the requested operation based on driving information excluding ).

Additionally, the processor 150 may identify whether the robot 100 is in a low power state (519). If the robot 100 is identified as being in a low-power state (519: Y), the processor 150 includes low-power state warning information and robot 100 operation interruption information in the guide UI information, and the robot 100 is driven. The driving unit 140 can be controlled so as not to do so (530).

If the robot 100 is identified as not being in a low power state (519: N), the processor may determine whether to perform the requested operation based on the driving information (520).

The processor 150 according to an example may identify a middle value among the identified movement distances 511, 512, and 513, and calculate a deviation ratio from the identified middle value and the remaining movement distances excluding the middle value. The processor 150 may identify the performance level of the requested operation based on whether the deviation rate is greater than or equal to the threshold rate and obtain guide UI information that outputs the identified performance level (521).

The processor 150 according to an example may identify the average value of the identified shaking degrees 514 and 515 and calculate a deviation ratio between the average value and the identified shaking degrees 514 and 515, respectively. The processor 150 may identify the performance level of the requested operation based on whether the deviation rate is greater than or equal to the threshold rate and obtain guide UI information that outputs the identified performance level (522).

The processor 150 according to an example may identify a middle value among the identified rotation angles 516, 517, and 518, and calculate a deviation ratio from the identified middle value and the remaining rotation angles excluding the middle value. The processor 150 may identify the performance level of the requested operation based on whether the deviation rate is greater than or equal to the threshold rate and obtain guide UI information that outputs the identified performance level (523).

Finally, the processor 150 may transmit the obtained guide UI information to the user terminal (540). When the robot 100 is identified as being in a low power state (519: Y), the processor 150 may transmit UI information including low power state warning information and robot 100 operation interruption information to the user terminal.

6A and 6B are diagrams for explaining an alternative operation of a robot according to an embodiment of the present disclosure. The robot 100 according to one example may be provided with a driving unit 140 including a right wheel 141 and a left wheel 142.

The processor 150 may calculate the movement distance of the right wheel 141 and the movement distance of the left wheel 142, respectively, based on the odometry data. The processor 150 according to an example may identify that the requested operation has not been performed normally when the difference between the movement distance of the right wheel 141 and the movement distance of the left wheel 142 is determined to be greater than or equal to the first threshold.

The processor 150 performs a request operation based on the ratio of the movement distance of the right wheel 141 and the movement distance of the left wheel 142, rather than the difference between the movement distance of the right wheel 141 and the movement distance of the left wheel 142. It may be determined whether this was performed normally, but for convenience of explanation, in relation to FIGS. 6A and 6B, the processor 150 determines whether the requested operation is performed normally based on the difference between the movement distance of the right wheel 141 and the movement distance of the left wheel 142. It is assumed and explained that it is judged whether it has been carried out.

Meanwhile, the processor 150 calculates the distance to be moved by the robot 100 (5 m) based on the control command 610 and the rotation amount (7 m, 5.5 m) of both wheels 141 and 142 based on odometry data. By comparison, it can be identified that the requested operation was not performed properly. For example, the processor 150 determines that any one of the rotation amounts (7 m, 5.5 m) of both wheels 141 and 142 is greater than the distance (5 m) that the robot 100 must move based on the control command 610. 3 If it is identified as having a value greater than the threshold (e.g. 1.5m), it can be identified that the requested operation was not performed properly.

In addition, the processor 150 identifies the type of abnormal operation as 'slipping' when the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 is less than a second threshold value that is greater than the first threshold value. And, if the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 is greater than the second threshold value that is greater than the first threshold value, the type of abnormal operation can be identified as 'foreign matter caught'.

According to FIG. 6A, the robot 100 that receives a control command 610 from the user terminal can move on a slippery running surface. The processor 150 according to an example may identify that the right wheel 141 has rotated 7 m (611) and that the left wheel 142 has rotated 5.5 m (612). In this case, the processor 150 may identify that the requested operation was not performed normally based on the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 being greater than or equal to a first threshold value (e.g., 1 m). You can.

In addition, the processor 150 determines that the type of abnormal operation is 'sliding' based on the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 being less than a second threshold value (e.g., 3 m). and can obtain solution information 620 corresponding to the identified type. For example, solution information 620 for coping with slippage may include information to reduce the moving speed of the robot 100 to 0.1 m/s, which is slower than 0.2 m/s corresponding to the control command 610. It is not limited to this.

According to FIG. 6B, the robot 100 that receives the control command 630 from the user terminal can move through the foreign matter 60 on the running surface. Here, the foreign matter 60 may be an object (eg, a cloth) that may interfere with the operation of the driving unit 140.

According to one example, the right wheel 141 of the processor 150 may be identified as having rotated 5 m (631), and the left wheel 142 may be identified as having rotated 0.5 m (632). In this case, the processor 150 normally performs the requested operation based on the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 being greater than or equal to a first threshold value (e.g., 1 m). It can be identified as not being done.

In addition, the processor 150 determines that the type of abnormal operation is 'foreign matter caught' based on the difference between the moving distance of the right wheel 141 and the moving distance of the left wheel 142 being more than a second threshold value (e.g., 3 m). , and solution information 640 corresponding to the identified type can be obtained. For example, the solution information 640 for coping with foreign matter being caught is to rotate the wheel 142 with foreign matter caught in the direction opposite to the rotation direction according to the control command 630, and then rotate the wheel 142 in the rotation direction according to the control command 630. It may include information that controls rotation, but is not limited to this.

7A and 7B are diagrams for explaining a sensor reset operation of a robot according to an embodiment of the present disclosure. The processor 150 may obtain shaking information of the robot 100 based on inertial data acquired from the inertial sensor 133.

According to FIG. 7A, the processor 150 may detect periodic shaking of the robot 100 (710). If rolling, pitching, or yawing is periodically identified during the driving process of the robot 100, this is not shaking caused by the condition of the running surface, but shaking caused by an error in the control algorithm of the robot 100 or the inertial sensor 133. Since there is a high possibility, the processor 150 may stop running the robot 100 (721) and reset the inertial sensor 133 (722).

In addition, the processor 150 may reset the control algorithm of the robot 100, and may transmit guide UI information indicating an error in the control algorithm or the inertial sensor 133 to the user terminal through the communication interface 110. there is.

According to FIG. 7B, the processor 150 can detect intermittent shaking of the robot 100 (730). If rolling, pitching, or yawing is identified intermittently during the robot's driving process, this is likely to be shaking caused by the condition of the running surface, so the processor 150 controls the driving unit 140 to decelerate the robot 100. Can (741).

At the same time, the processor 150 may adjust (or change) the analysis method of image data acquired through the camera 120 (742). For example, the processor 150 may improve the sensitivity involved in identifying obstacles or irregularities within image data. Through this, the robot 100 can move more smoothly on a running surface that is not in good condition.

Figure 8 is a diagram for explaining the operation of a robot including a head according to an embodiment of the present disclosure. The robot 100 according to an example consists of a head 100-1 and a body 100-2, and a head driving unit located between the head 100-1 and the body 100-2 ( The head portion 100-1 may be driven by 143). In this case, the processor 150 operates on the head 100-1 of the robot 100 based on inertial data acquired through the inertial sensor 133-1 provided on the head 100-1 of the robot 100. ) Obtain shaking information about the movement of the body portion 100 of the robot 100 based on the inertial data acquired through the inertial sensor 133-2 provided in the body portion 100-2 of the robot 100. -2) Shaking information about the movement can be obtained.

The processor 150 may obtain driving information within a first threshold time based on the time when the control command is received. The acquired driving information includes shaking information about the head 100-1 and shaking information about the body 100-2, as well as shaking information about the robot 100 based on image data acquired through the camera 120. may be included.

For example, the processor 150 identifies the degree of shaking of the robot 100 identified based on the image data as 'weak' (811), and the degree of shaking of the head 100-1 identified based on the inertial data is The degree of shaking can be identified as 'weak' (812), and the degree of shaking of the body portion 100-2 identified based on inertial data can be identified as 'strong' (813). Based on a situation where the identified degree of shaking is not the same, the processor 150 may identify that an error has occurred in at least one of the inertial sensors 133-1 and 133-2.

The processor 150 controls the driving unit 143 so that the head 100-1 does not move separately from the body 100-2 in order to identify the inertial sensor in which an error occurred (821), and the robot 100 The driving units 141 and 142 can be controlled to stop driving (822). In addition, the processor 150 based on data acquired from the camera 120 and the inertial sensors 133-1 and 133-2 within a second threshold time based on the control timing for the driving units 141, 142, and 143. The shaking information of the robot 100 can be acquired again.

Because the running of the robot 100 has stopped, the processor 150 may not be able to identify the shaking of the robot 100 based on the image data (831). If an error does not occur in the inertial sensor 133-1 provided in the head 100-1, shaking of the head 100-1 based on inertial data may not be identified. On the other hand, if an error occurs in the inertial sensor 133-2 provided in the body 100-2, the shaking of the body 100-2 based on the inertial data may be identified as 'strong'.

In this case, the processor 150 identifies that an error has occurred in the inertial sensor 133-2 provided in the body portion 100-2, and sends guide UI information related to the error of the inertial sensor 133-2 to the user terminal. The communication interface 110 can be controlled to transmit.

Furthermore, the processor 150 may identify that a failure has occurred in the inertial sensor 133-2 based on the reset history of the inertial sensor 133-2 in which an error occurred. For example, if the inertial sensor 133-2 in which an error occurred is identified as having been reset more than a threshold number of times within a critical period, the processor 150 identifies that a failure has occurred in the corresponding sensor 133-2 and provides a guide regarding the failure. The communication interface 110 can be controlled to transmit UI information to the user terminal.

Figure 9 is a flowchart for explaining the operation of a robot considering the battery state according to an embodiment of the present disclosure.

According to FIG. 9, the processor 150 can continuously monitor the state of the battery (S910). In the process of acquiring driving information within a threshold time based on the point in time at which the control command is received, the processor 150 may determine whether the amount of power in the battery is greater than or equal to the threshold (S920). Here, the threshold value may be the amount of battery power required for the robot 100 to run normally, but is not limited thereto.

If the battery power amount is identified as being less than the threshold (S920: N), the processor 150 may stop driving the robot 100 and provide guide UI information instructing to stop driving the robot 100 ( S930). Accordingly, the processor 150 can block problems that may occur as the robot 100 runs in a battery state with a power amount below the threshold.

If the battery power amount is identified as being equal to or greater than the threshold (S920: Y), the processor 150 may obtain driving information based on data acquired through the camera 120 and a plurality of sensors. In this case, the processor 150 may reflect only reliable data values under the identified battery state in obtaining driving information. Hereinafter, the amount of power required for the camera 120 or each of the plurality of sensors to acquire and output reliable data values will be expressed in terms of the 'required power amount' of the camera 120 or each of the plurality of sensors.

For example, the processor 150 determines whether the amount of power of the battery is greater than or equal to the amount of power required by the camera 120 (S940), and only if the amount of power of the battery is greater than the amount of power required by the camera 120 (S940: Y) Data can be reflected in driving information (S941).

In addition, the processor 150 determines whether the amount of power of the battery is greater than or equal to the amount of power required by the distance sensor (S950), and reflects the distance data to the driving information only when the amount of power of the battery is greater than the amount of power required by the distance sensor (S950: Y). You can do it (S951).

In addition, the processor 150 determines whether the power amount of the battery is more than the power amount required by the odometry sensor (S960), and only if the power amount of the battery is more than the power amount required by the odometry sensor (S960: Y) Data can be reflected in driving information (S961).

In addition, the processor 150 determines whether the amount of power in the battery is greater than or equal to the amount of power required by the inertial sensor (S970), and reflects the inertial data in the driving information only when the amount of power in the battery is greater than the amount of power required by the inertial sensor (S970: Y). You can do it (S971).

Subsequently, the processor 150 may obtain driving information within a threshold time based on the time when the control command is received based on the data reflected through the above process (S980). Additionally, if the processor 150 identifies that the requested operation was performed abnormally based on the acquired driving information, it may provide guide UI information related to the abnormal operation (S990).

Figure 10 is a block diagram for specifically explaining the configuration of a robot according to an embodiment of the present disclosure.

According to FIG. 10, the robot 100 includes a communication interface 110, a camera 120, a distance sensor 131, an odometry sensor 132, a sensor module 130 including an inertial sensor 133, and a drive unit. It may include 140, processor 150, battery 160, memory 170, and user interface 180. Among the configurations shown in FIG. 10, detailed descriptions of those that overlap with those shown in FIG. 2 will be omitted.

The distance sensor 131 may acquire distance data. Specifically, the distance sensor 131 may measure the distance between the position of the robot 100 and the position of the object and obtain distance data based on the measurement result. The distance sensor 131 according to one example may be implemented as a LIDAR (Light Detection And Ranging) or TOF (Time of Flight) sensor, but is not limited thereto.

The odometry sensor 132 may be placed adjacent to the driving unit 140. The odometry sensor 132 may acquire odometry data corresponding to the movement of the drive unit 140 by sensing the movement of the drive unit 140 using a physical or electronic method. According to one example, when a plurality of driving units 140 are provided, the odometry sensors 132 may also be provided in the same number as the driving units 140. According to another example, if the drive unit 140 is implemented as a ball type, a plurality of odometry sensors 132 may be provided, each corresponding to a preset rotation direction of the ball type drive unit 140.

The inertial sensor 133 is a component that acquires inertial data. The inertial sensor 133 according to one example may combine a 3-axis acceleration sensor and a 3-axis gyro sensor and then output inertial data based on sensing data acquired through each sensor. In this case, the inertial sensor 133 may output inertial data regarding the degree of rolling, pitching, and yawing.

The battery 160 is configured to supply power to components included in the robot 100. Battery 160 may be implemented as a rechargeable battery. For example, the battery 160 may be implemented as at least one of a lithium-ion battery or a lithium polymer battery. The processor 150 may perform Battery Management System (BMS)-related functions to monitor and manage the state of the battery 160.

The memory 170 may store data necessary for various embodiments of the present disclosure. For example, the memory 170 includes a threshold time related to obtaining driving information, a threshold value related to determining whether to perform the requested operation, and a battery 160 in which the camera 120 and the plurality of sensors 131, 132, and 133 are respectively operating normally. ) Information about the amount of power, etc. may be stored.

The memory 170 may be implemented as a memory embedded in the robot 100 or as a memory detachable from the robot 100 depending on the data storage purpose. For example, data for driving the robot 100 is stored in a memory embedded in the robot 100, and data for expansion functions of the robot 100 is stored in a memory that is detachable from the robot 100. It can be. Meanwhile, in the case of memory embedded in the robot 100, volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (e.g. : OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g. NAND flash or NOR flash, etc.) , a hard drive, or a solid state drive (SSD). In addition, in the case of memory that is removable from the robot 100, a memory card (e.g., compact flash (CF), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g. For example, it may be implemented in a form such as USB memory).

The user interface 180 is a component that helps the robot 100 interact with a user. For example, the user interface 180 may include at least one of a touch screen, a button, a jog dial, a switch, a microphone, or a speaker, but is not limited thereto. The processor 150 may provide guide UI information through the user interface 180 in addition to transmitting guide UI information to the user terminal through the communication interface 110. In this case, the processor 150 may execute an alternative operation based on the user's feedback input through the user interface 180.

Figure 11 is a flowchart for explaining a control method according to an embodiment of the present disclosure.

The control method according to an embodiment of the present disclosure acquires image data through a camera (S1110).

Next, the robot's driving information is obtained based on the sensing data and image data obtained through at least one sensor among the plurality of sensors (S1120).

Next, among the robot's driving information, the driving information within a threshold time is identified based on the time when the control command is received (S1130).

Next, it is identified whether the operation corresponding to the control command was normally performed based on the identified driving information (S1140).

Lastly, if the operation corresponding to the control command is identified as being performed abnormally, guide UI (User Interface) information related to the abnormal operation may be provided (S1150).

Here, the step of identifying whether the operation corresponding to the control command was performed normally (S1140) includes the first moving distance of the robot based on image data acquired within a critical time, and the second moving distance of the robot obtained through a distance sensor within the critical time. Identifying the moving distance of the robot during a critical time based on the moving distance and the third moving distance of the robot acquired through an odometry sensor within the critical time, and corresponding to the moving distance and control command of the identified robot. It may include comparing the movement distance to identify whether the operation corresponding to the control command was performed normally.

Here, in the step of identifying the moving distance of the robot, the middle value among the first moving distance, the second moving distance, and the third moving distance may be identified as the moving distance of the robot.

Here, the step of identifying whether the operation corresponding to the control command was performed normally (S1140) compares the identified movement distance with each of the remaining movement distances among the first movement distance, the second movement distance, and the third movement distance to determine the first movement distance. It may include identifying a distance deviation and a second distance deviation, and identifying that an operation corresponding to the control command is abnormally performed when at least one of the first distance deviation or the second distance deviation is greater than or equal to a first threshold value. there is.

In addition, the step of identifying whether the operation corresponding to the control command was performed normally (S1140) includes the first rotation angle of the robot based on the image data acquired within the critical time, and the second rotation angle of the robot acquired through the inertial sensor within the critical time. Identifying a rotation angle of the robot during a critical time based on the rotation angle and a third rotation angle of the robot acquired through an odometry sensor within the critical time, and corresponding to the rotation angle and control command of the identified robot. It may include comparing rotation angles to identify whether the operation corresponding to the control command was normally performed.

Here, in the step of identifying the rotation angle of the robot, the middle value among the first rotation angle, the second rotation angle, and the third rotation angle may be identified as the rotation angle of the robot.

Here, the step of identifying whether the operation corresponding to the control command has been performed normally (S1140) is to compare the identified rotation angle with each of the remaining rotation angles among the first rotation angle, the second rotation angle, and the third rotation angle. It may include identifying the angle deviation and the second angle deviation, and identifying that the operation corresponding to the control command is abnormally performed when at least one of the first angle deviation or the second angle deviation is greater than or equal to a second threshold value. there is.

In addition, the control method compares the first shake information of the robot based on image data and the second shake information of the robot based on shake data acquired through an inertial sensor for the same time as the time at which the first shake information was acquired. A step of identifying whether the operation of the inertial sensor is performed normally, and if the operation of the inertial sensor is identified as being performed abnormally, a step of resetting the inertial sensor and transmitting guide UI (User Interface) information related to the abnormal operation to the user terminal. More may be included.

In addition, the step of transmitting guide UI information to the user terminal (S1150) includes identifying the type of abnormal operation, obtaining solution information related to the identified type, and transmitting guide UI information including the solution information to the user terminal. It may include steps.

In addition, the step of acquiring driving information of the robot (S1120) includes acquiring driving information based on a first number of data among a plurality of data acquired through a camera and a plurality of sensors when the power amount of the battery is less than a threshold value; When the power amount of the battery is greater than or equal to the threshold, the method may include obtaining driving information based on a second number of data that is greater than the first number of the plurality of data.

Meanwhile, the methods according to various embodiments of the present disclosure described above may be implemented in the form of applications that can be installed on existing robots.

Additionally, the methods according to various embodiments of the present disclosure described above can be implemented only by upgrading software or hardware for an existing robot.

Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in a robot or at least one external server.

For example, various embodiments of the present disclosure may be implemented through cloud computing. That is, various embodiments of the present disclosure may be implemented through a cloud server connected to the robot 100 or other electronic devices connected to the cloud server, in addition to the processor 150 mounted on the robot 100 itself.

Accordingly, when the robot 100 operates abnormally, the cloud server connected to the robot 100 may identify the abnormal operation of the robot 100 and transmit related guide UI information to the user terminal, and the cloud server Another electronic device (e.g., TV, air conditioner, etc.) connected to may identify abnormal operation of the robot 100 and transmit related guide UI information to the user terminal.

Meanwhile, the various embodiments described above may be implemented in a recording medium that can be read by a computer or similar device using software, hardware, or a combination thereof. In some cases, embodiments described herein may be implemented by the processor 150 itself. According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing processing operations of the robot 100 according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. . Computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the robot 100 according to the various embodiments described above.

A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specific examples of non-transitory computer-readable media may include CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field pertaining to the disclosure without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those with the knowledge, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: robot 110: communication interface
120: camera 130: sensor module
140: driving unit 150: processor

Claims (20)

In robots,
communication interface;
camera;
A sensor module including a plurality of sensors;
driving part; and
It includes a processor that controls the driving unit based on a control command received from the user terminal through the communication interface,
The processor,
Obtain image data through the camera,
Obtaining driving information of the robot based on sensing data and the image data obtained through at least one sensor among the plurality of sensors,
Among the driving information of the robot, driving information within a threshold time is identified based on the time when the control command is received,
Identify whether the operation corresponding to the control command was normally performed based on the identified driving information,
When the operation corresponding to the control command is identified as being performed abnormally, the robot transmits guide UI (User Interface) information related to the abnormal operation to the user terminal through the communication interface. According to paragraph 1,
The sensor module is,
distance sensor; and
Includes an odometry sensor,
The processor,
A first moving distance of the robot based on the image data acquired within the critical time, a second moving distance of the robot obtained through the distance sensor within the critical time, and a second moving distance obtained through the odometry sensor within the critical time Identifying the moving distance of the robot during the threshold time based on the third moving distance of the robot,
A robot that compares the identified movement distance of the robot and the movement distance corresponding to the control command to identify whether the operation corresponding to the control command was normally performed. According to paragraph 2,
The processor,
A robot that identifies a middle value among the first movement distance, the second movement distance, and the third movement distance as the movement distance of the robot. According to paragraph 3,
The processor,
Identifying a first distance deviation and a second distance deviation by comparing the identified movement distance with each of the remaining movement distances among the first movement distance, the second movement distance, and the third movement distance,
A robot that identifies that an operation corresponding to the control command is abnormally performed when at least one of the first distance deviation or the second distance deviation is greater than or equal to a first threshold value. According to paragraph 1,
The sensor module is,
inertial sensor; and
Includes an odometry sensor,
The processor,
The processor,
A first rotation angle of the robot based on the image data acquired within the threshold time, a second rotation angle of the robot obtained through the inertial sensor within the threshold time, and a second rotation angle obtained through the odometry sensor within the threshold time. Identifying the rotation angle of the robot during the threshold time based on the third rotation angle of the robot,
A robot that compares the identified rotation angle of the robot and the rotation angle corresponding to the control command to identify whether the operation corresponding to the control command was normally performed. According to clause 5,
The processor,
A robot that identifies a middle value among the first rotation angle, the second rotation angle, and the third rotation angle as the rotation angle of the robot. According to clause 6,
The processor,
Identifying a first angle deviation and a second angle deviation by comparing the identified rotation angle with each of the remaining rotation angles among the first rotation angle, the second rotation angle, and the third rotation angle,
A robot that identifies that an operation corresponding to the control command is abnormally performed when at least one of the first angle deviation or the second angle deviation is greater than or equal to a second threshold value. According to paragraph 1,
The sensor module is,
Includes an inertial sensor;
The processor,
By comparing the first shake information of the robot based on the image data and the second shake information of the robot based on shake data acquired through the inertial sensor for the same time as the time at which the first shake information was acquired, the inertia Identify whether the sensor is operating normally,
When the operation of the inertial sensor is identified as abnormal, the robot resets the inertial sensor and transmits guide UI (User Interface) information related to the abnormal operation to the user terminal through the communication interface. According to paragraph 1,
The processor,
Identifying a type of abnormal operation, obtaining solution information related to the identified type, and
A robot that transmits the guide UI information including the solution information to the user terminal through the communication interface. According to paragraph 1,
It further includes a battery;
The processor,
When the amount of power of the battery is less than a threshold value, obtain the driving information based on a first number of data among a plurality of data obtained through the camera and the plurality of sensors,
A robot that acquires the driving information based on the second number of data that is greater than the first number of the plurality of data when the amount of power of the battery is greater than a threshold value. In the control method of a robot driven based on a control command received from a user terminal,
Acquiring image data through a camera;
Obtaining driving information of the robot based on sensing data and the image data obtained through at least one sensor among a plurality of sensors;
Identifying driving information within a threshold time among the driving information of the robot based on the time when the control command is received;
Identifying whether an operation corresponding to the control command was normally performed based on the identified driving information; and
If the operation corresponding to the control command is identified as being performed abnormally, transmitting guide UI (User Interface) information related to the abnormal operation to the user terminal. According to clause 11,
The step of identifying whether the operation corresponding to the control command was performed normally,
A first movement distance of the robot based on the image data acquired within the threshold time, a second movement distance of the robot obtained through a distance sensor within the threshold time, and an odometry sensor within the threshold time. identifying a moving distance of the robot during the threshold time based on the obtained third moving distance of the robot; and
A control method including; comparing the identified movement distance of the robot and the movement distance corresponding to the control command to identify whether the operation corresponding to the control command was normally performed. According to clause 12,
The step of identifying the moving distance of the robot is,
A control method that identifies a middle value among the first movement distance, the second movement distance, and the third movement distance as the movement distance of the robot. According to clause 13,
The step of identifying whether the operation corresponding to the control command was performed normally,
identifying a first distance deviation and a second distance deviation by comparing the identified movement distance with each of the remaining movement distances among the first movement distance, the second movement distance, and the third movement distance; and
A control method comprising; identifying that an operation corresponding to the control command is abnormally performed when at least one of the first distance deviation or the second distance deviation is greater than or equal to a first threshold value. According to clause 11,
The step of identifying whether the operation corresponding to the control command was performed normally,
A first rotation angle of the robot based on the image data acquired within the critical time, a second rotation angle of the robot acquired through an inertial sensor within the critical time, and an odometry sensor within the critical time. identifying a rotation angle of the robot during the threshold time based on the obtained third rotation angle of the robot; and
Comparing the identified rotation angle of the robot and the rotation angle corresponding to the control command to identify whether the operation corresponding to the control command was normally performed. A control method comprising a. According to clause 15,
The step of identifying the rotation angle of the robot is,
A control method that identifies a middle value among the first rotation angle, the second rotation angle, and the third rotation angle as the rotation angle of the robot. According to clause 16,
The step of identifying whether the operation corresponding to the control command was performed normally,
identifying a first angle deviation and a second angle deviation by comparing the identified rotation angle with each of the remaining rotation angles among the first rotation angle, the second rotation angle, and the third rotation angle; and
A control method comprising; identifying that an operation corresponding to the control command is abnormally performed when at least one of the first angle deviation or the second angle deviation is greater than or equal to a second threshold value. According to clause 11,
The inertial sensor compares the first shake information of the robot based on the image data and the second shake information of the robot based on shake data acquired through an inertial sensor for the same time as the time at which the first shake information was acquired. Identifying whether the operation is performed normally; and
If it is identified that the operation of the inertial sensor is abnormal, resetting the inertial sensor and transmitting guide UI (User Interface) information related to the abnormal operation to the user terminal; a control method further comprising: . According to clause 11,
The step of transmitting guide UI information to the user terminal is:
Identifying a type of the abnormal operation and obtaining solution information related to the identified type; and
A control method comprising: transmitting the guide UI information including the solution information to the user terminal. According to clause 11,
The step of acquiring driving information of the robot is,
When the amount of battery power is less than a threshold, acquiring the driving information based on a first number of data among a plurality of data acquired through the camera and the plurality of sensors; and
A control method comprising; acquiring the driving information based on the second number of data that is greater than the first number of the plurality of data when the amount of power of the battery is greater than or equal to a threshold value.

Images