KR20190070385A - Robot hand for grasping unknown object and control method thereof - Google Patents

Abstract

The present invention relates to a robot hand and a control method thereof and, more specifically, to a robot hand and a control method thereof which use visual and tactile sensors without detailed modeling or programming for an object to handle the object. According to an aspect of the present invention, the robot hand comprises: a camera to acquire an image for at least one first object; a control unit to use the acquired image to determine at least one among position coordinates, a size, and a center of the first object; and a hand to use a factor if at least one factor among the position coordinates, the size, and the center is determined to grip the first object. If the hand fails to grip the first object, the control unit determines at least one among the position coordinates, the size, and the center of the first object again, and the hand uses the factor determined again to grip the first object again.

Description

Technical Field [0001] The present invention relates to a robot hand for grasping an object having no information,

The present invention relates to a robot hand and a control method thereof. More particularly, the present invention relates to a robot hand capable of handling a visual and tactile sensor without detailed modeling or programming operations, and a control method thereof.

A robot hand refers to a hand of a machine that restrains or moves an object by a plurality of fingers and a robot arm makes an approximate positioning in a wide working area while a robot hand has a limited area And is used for fine manipulation or grasping of an object.

In addition, the artificial intelligence and robotic technology is spreading in various fields and can be utilized as a core technology for breakthrough of handling of components in assembling and assembling in a manufacturing environment.

However, most of the operations performed in the actual manufacturing environment based on the above-described robot hand are handling operations for holding and assembling parts, and most of the handling operations except the simple repetitive operations in a formal environment .

Therefore, it is urgent to develop a gripping / assembling technique equipped with an artificial intelligence in order to solve the above problems and to increase the production efficiency in response to a change in the manufacturing environment.

(1) Korean Intellectual Property Office Registration No. 10-1323217 (2) Korean Intellectual Property Office Registration No. 10-1048762

The present invention relates to a robot hand and a control method thereof, and it is an object of the present invention to provide a robot hand and a control method thereof that can be handled by using a visual and tactile sensor without detailed modeling or programming of an object.

In addition, the present invention provides a handling technique for grasping an approximate position, shape, and the like of an object through a vision and attempting grasping, and for failing to perform a task, learning a failed event according to an algorithm and attempting regression , And a technique for further enhancing the efficiency of the learning algorithm by utilizing the tactile sensor.

In addition, the present invention overcomes the drawbacks of long learning time by developing the existing model-free and model-based methods for developing gripping and manipulation methods through conventional machine learning, And to provide a handling technique that can be utilized even in situations.

Another object of the present invention is to provide a handling technique capable of classifying and learning objects by measuring a shear force, a conductivity of an object, and the like, as well as a conventional normal force measurement by further utilizing a tactile sensor.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.

According to an aspect of the present invention, there is provided a robot hand comprising: a camera for acquiring an image of at least one first object; A controller for determining at least one of a position coordinate, a size and a center of the first object using the obtained image; And a hand for holding the first object using the determined factor when at least one factor of the position coordinates, size and center is determined, wherein the hand grasps the first object In case of failure, the control unit re-determines at least one of the coordinates, size and center of the first object based on the failure event, and the hand can again grasp the first object using the determined factor have.

In addition, the controller may control the hand to grasp the first object using only the determined factor in the absence of prior information about the first object.

The control unit may further determine at least one of a position coordinate, a size and a center of the first object based on the re-failure event when the hand re-fails to grasp the first object, And controlling to grasp the first object again using the again determined factor.

In addition, when the hand grasps the first object again using the determined factor, the control unit determines the grip stability of the first object held by the hand, and when the grip stability is equal to or less than a preset reference , It is possible to control the hand to release the first object.

A tactile sensor for sensing at least one tactile factor of a normal force, a shear force and a conductive of the first object when the hand is in contact with the first object for gripping; And when the hand grasps the first object again using the determined factor, at least one tactile factor sensed by the tactile sensor may be used together.

Further, the hand includes a plurality of fingers, each of the plurality of fingers includes a distance sensor for sensing a distance distanced from the first object, and when the first object is gripped, Based on the acquired information, each of the plurality of fingers can move close to the first object at the same speed.

According to another aspect of the present invention, there is provided a method of controlling a robot hand, the method comprising: a first step of a camera acquiring an image of at least one first object; A second step of the controller determining at least one of a position coordinate, a size and a center of the first object using the obtained image; A third step of the hand grasping the first object using the determined factor when at least one factor of the position coordinate, the magnitude and the center is determined; A fourth step of the hand failing to grasp the first object; A fifth step of the controller determining again at least one of a position coordinate, a size and a center of the first object based on the failure event; And a sixth step of the hand grasping the first object again using the determined factor.

In addition, in the third step, in the state where there is no prior information about the first object, the hand may grasp the first object using only the determined factor.

Also, in the sixth step, when the hand fails to grasp the first object, the first through sixth steps may be repeatedly performed.

A seventh step of determining, after the sixth step, the grip stability of the first object held by the hand; And an eighth step of releasing the first object by the hand if the gripping stability is less than a predetermined standard.

In addition, in the third step, when the hand is brought into contact with the first object for gripping, a normal force, a shear force, and a conductive state of the first object through the tactile sensor Wherein at least one tactile factor sensed by the tactile sensor is used together when the hand grasps the first object again using the again determined factor have.

The present invention relates to a robot hand and a control method thereof, and it is possible to provide a robot hand and a control method thereof that can handle a visual and tactile sensor by using a visual and tactile sensor without finely modeling or programming the object.

According to the present invention, it is possible to overcome weak points such as long learning time and develop a model-free method and a model-based method to develop a gripping and manipulating method through machine learning, It can be a complementary development.

In addition, according to the present invention, a tactile sensor capable of detecting not only a normal force but also a shear force and measuring the conductivity of a target object can be developed. It is possible.

The present invention also relates to a real-time position / attitude / state recognition technology for parts using visual and tactile information, an optimum grip type intelligence technology for stable gripping of parts, an intelligent grip technology based on recognition information and experience , More than 30 kinds of objects), in-hand technology of parts using vision and tactile information, assembly strategy learning technology of various parts based on experience, The present problems which are being solved by the water process of the present invention can be solved.

It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

1 is a block diagram illustrating a configuration of a robot hand system proposed by the present invention.
2 is a view for explaining a configuration of a robot hand proposed by the present invention.
FIG. 3 illustrates a specific example of gripping an object according to the vision, control, and machine learning of the present invention.
4 is a block diagram illustrating a configuration of a robot hand system according to the present invention for gripping an object based on a vision system.
FIG. 5 is a flowchart illustrating a method of grasping an approximate position, shape, and the like of an object through a vision according to the present invention and learning a failed event according to an algorithm when a task fails to execute.
6 is a block diagram illustrating the configuration of a robot hand system according to the present invention for gripping an object by additionally using a tactile sensor in addition to a vision system.
FIGS. 7A and 7B show an example of a specific configuration of a robot finger using vision information and tactile information together with respect to the present invention.
FIG. 8 is a flowchart for explaining the operation of a robot finger using vision information and tactile information together with respect to the present invention.
Fig. 9 shows an example of a small-sized model and three-dimensional position fitting used in the gripping of the present invention.
FIG. 10 shows an example of fitting to a three-dimensional position according to the present invention and applying the same to a grip.
11 is a flowchart for explaining the overall operation of the robot phage using the vision information and the tactile information together with the present invention.
Fig. 12 shows an example of a flowchart for explaining an object grasp using learning in connection with the present invention.
13 shows an example in which the gripping method is changed according to the change of the direction and the position of the object in relation to the present invention.
14 is a diagram for explaining the necessity of repagination according to the present invention.
FIG. 15 shows an example of a repagment learning model according to the present invention.
FIG. 16 is a diagram for explaining a re-fingerprinting method according to the fingerprint success probability of the present invention.
FIG. 17 is a diagram for explaining the performance of a task to which repargement is applied, according to the present invention. FIG.
FIG. 18 is a flowchart illustrating a method for completing a specific task using regrasping when a manipulation according to a given purpose can not be performed in the current grasping form according to the present invention.

Prior to a specific description of the present invention, a configuration of a robot hand system applied to the present invention will be described with reference to the drawings.

1 is a block diagram illustrating a configuration of a robot hand system proposed by the present invention.

1, the robot hand system 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A memory 160, an interface unit 170, a control unit 180, a power supply unit 190, a robot hand 200, and the like.

However, the components shown in Fig. 1 are not essential, and a robot hand system having components having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the robot hand system and the wireless communication system or between the device and the network in which the device is located.

For example, the wireless communication unit 110 may include a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, a location information module 115, and the like.

The mobile communication module 112 transmits and receives a radio signal to at least one of a base station, an external device, and a server on a mobile communication network.

And various types of data according to transmission / reception of text / multimedia messages.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built in or enclosed in a robot hand system. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. (Bluetooth), Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, WiFi Can be used.

The position information module 115 is a module for acquiring the position of the robot hand system, and a representative example thereof is a Global Position System (GPS) module.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [

Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal by a microphone in a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for controlling the operation of the robot hand system by the user. The user input unit 130 may include a key pad dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the robot hand system such as the opening / closing state of the robot hand system, the position of the robot hand system, the presence of the user, the orientation of the robot hand system, And generates a sensing signal for controlling the operation of the sensor.

The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

Meanwhile, the sensing unit 140 may include a proximity sensor (not shown).

The output unit 150 is for generating an output relating to visual, auditory or tactile sense and includes a display unit 151, an acoustic output module 152, an alarm unit 153, a haptic module 154, 155, and the like.

The display unit 151 displays (outputs) information processed in the robot hand system.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type.

There may be two or more display units 151 depending on the implementation of the robot hand system. For example, in the robot hand system, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

(Hereinafter, referred to as a 'touch screen') in which a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') form a mutual layer structure, It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor (not shown) may be disposed within an interior region of the robotic hand system that is enclosed by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

In addition, the robot hand system 100 according to the present invention may include a gyro sensor 141.

The gyro sensor 141 is a sensor for measuring the azimuth change of an object by using the property of maintaining the predetermined constant direction set at the first time with high accuracy irrespective of the rotation of the earth and the gyroscope is an optical type have.

In addition, the robot hand system 100 according to the present invention may include an acceleration sensor 142.

The acceleration sensor 142 processes the output signal to measure dynamic forces such as acceleration, vibration, and impact of the object.

The acceleration sensor 142 is classified into an inertia type, a gyro type, and a silicon semiconductor type, and the acceleration system, the tilt system, and the like can be regarded as one type of acceleration sensor.

In addition, the robot hand system 100 according to the present invention may include a pressure sensor 143.

The pressure sensor 143 refers to an apparatus and a device that detect the pressure of a liquid or a gas and convert it into an electric signal that is easy to use for measurement or control and transmit.

Recently, a strain gauge type pressure sensor made of silicon material has been developed and used for precise pressure measurement. Integrated pressure sensors have also been developed that incorporate integrated circuits on the same substrate to process signals.

The robot hand system 100 according to the present invention may also include a tactile sensor 144. [

The tactile sensor (144, Tactile Sensor) is divided into a touch sensor, a pressure sensor, a slip sensor, and a temperature sensor to realize a human tactile sense artificially in a robot. Technology.

A tactile sensor array is a sensor for obtaining two-dimensional information by arranging several hundreds of contact angle sensors 144 or an elevation angle sensor in a planar shape, and can also be used for detection of shape or motion.

Many of these sensors constitute an array type sensor by dividing any one of the electrodes of the conductive rubber, the piezoelectric polymer, and the decompression polymer, and the two-dimensional pressure distribution is detected from the resistance change or the voltage output between the opposing electrodes.

In particular, the tactile sensor 144 according to the present invention can sense at least one of the normal force, the shear force, and the conductivity of the object with which the robot hand 200 contacts.

Meanwhile, the audio output module 152 can output audio data received from the wireless communication unit 110 or stored in the memory 160 in a recording mode, a voice recognition mode, a broadcast receiving mode, and the like. The sound output module 152 also outputs an acoustic signal related to a function performed in the robot hand system. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the robot hand system.

The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration.

The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that they may be classified as a part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled.

For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. More than two haptic modules 154 may be provided according to the configuration of the robot hand system.

The projector module 155 is a component for performing an image project function using the robot hand system and is configured to have the same or at least the same image displayed on the display unit 151 in accordance with a control signal of the controller 180 Some can display other images on an external screen or wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided longitudinally on the side, front or back side of the robotic hand system. Needless to say, the projector module 155 may be provided at any position of the robot hand system, if necessary.

The memory unit 160 may store a program for processing and controlling the control unit 180 and temporarily store input / output data (e.g., message, audio, still image, For example. The frequency of use of each of the data may be stored in the memory unit 160 as well. In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The robotic hand system may operate in association with a web storage that performs the storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the robot hand system. The interface unit 170 receives data from an external device, receives power from the external device, transfers the data to each component in the robot hand system, or transmits data in the robot hand system to external devices. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the usage right of the robot hand system and includes a user identification module (UIM), a subscriber identity module (SIM), a universal user authentication module Identity Module, USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Therefore, the identification device can be connected to the robot hand system through the port.

When the robot hand system is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the robot hand system, or various command signals input from the cradle by a user are transmitted to the mobile device It can be a passage. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile device is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the robotic hand system.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, controllers, micro-controllers, microprocessors, and other electronic units for performing other functions. In some cases, The embodiments described may be implemented by the control unit 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

In addition, the system 100 according to the present invention may include a robotic hand 200.

The robot hand 200 mainly includes a palm part 210 and a finger part 200.

The finger portion 200 can be configured with a structure having at least one node, and it is also possible to move at the same speed for an object through a distance sensor (not shown).

The object according to the present invention can be an object in a state in which no information exists at all, an object in a state in which information received from the outside is present, an object in a state in which information in which information is accumulated through a failure event, and the like .

The state in which the information on the object according to the present invention exists includes a state in which complete information on the object is constructed as well as a state in which information is accumulated to some extent through the failure event and the function.

One side of the robot hand 200 may be provided in a plurality of systems 100 and may perform a task having a specific order and combination direction as well as a simple gripping operation through a plurality of robot hands 200 It is possible.

Based on the above-described system 100 configuration of the present invention, specific techniques to be proposed in the present specification will be described with reference to the drawings.

First Embodiment - A technique of handling an unknown object using a visual sensor

Most of the operations performed in a real manufacturing environment based on the conventional robot hand are a handling task of holding and assembling parts, and most of the handling operations are performed by a human water process, except for a simple repetitive operation in a formal environment There was a problem.

Therefore, the present invention proposes a gripping / assembling technique equipped with an artificial intelligence to solve the above problems and to increase production efficiency in response to a change in a manufacturing environment.

That is, the present invention grasps the approximate position, shape, and the like of an object based on the above-described configuration of the system 100 through a vision, and when failing to perform a task, And provides a technique for regaling, and a technique for further improving the efficiency of the learning algorithm by using a tactile sensor.

In addition, the present invention overcomes the drawbacks of long learning time by developing the existing model-free and model-based methods for developing gripping and manipulation methods through conventional machine learning, And to provide a handling technique that can be utilized even in situations.

2 is a view for explaining a configuration of a robot hand proposed by the present invention.

2, a robot hand 200 for gripping an object 10 is shown.

A plurality of cameras 121 are arranged in the system 1 according to the present invention.

Here, the controller 180 can perform the operation of gripping the object 10 by controlling the robot hand 200 in a state in which there is no information about the object 10 at all.

That is, in the present invention, since there is no acquired information or learned information about the object, it is possible to grasp the approximate position and shape of the object based on the vision system through the plurality of cameras 121, The user can grasp the object 10 by controlling the hand 200.

Specifically, in this case, the controller 180 determines at least one of the coordinates, the size, and the center of the object 10 using the acquired image, and based on the grasped information, the robot hand 200 directly grasps the object 10, As shown in Fig.

In this case, the controller 180 may cause the robot hand 200 to grasp the object 10 based on power grasping rather than precise grasping.

Since the robot hand 200 does not grasp the object 10 based on the information, it may generate an event to release the object 10 after the object 10 can not be grasped or grasped .

In the present invention, a machine learning algorithm for failed events is applied.

That is, when the robot hand 200 fails to grip the object 10, the controller 180 determines at least one of the coordinates, the size, and the center of the object 200 based on the failure event.

Thereafter, the robot hand 200 again attempts to grasp the object 10 again using the determined factor.

This machine learning algorithm can be repeatedly performed when the object 10 can not be grasped and eventually it is possible to use much less trial, computation time, and time than the method of grasping the object 10 after acquiring specific information So that the object 10 can be gripped.

FIG. 3 illustrates a specific example of gripping an object according to the vision, control, and machine learning of the present invention.

3, the robot hand 200 according to the present invention includes one palm 210 and three fingers 220, and each finger 220 is composed of a plurality of nodes.

3 (a), the robot hand 200 according to the present invention can grasp the object 10 through the unfolding operation of the plurality of fingers 220 with reference to the palm 210 have.

That is, the control unit 180 determines at least one of the coordinates, the size, and the center of the object 10 based on the image obtained through the camera 121, and the robot hand 200 directly So as to grasp the object 10.

At this time, when the robot hand 200 does not grasp the object 10 or grasps the object 10 and then releases the object 10, the controller 180 controls the coordinates of the object 200 , Size, and center, and the robotic hand 200 attempts to retouch the object 10 again using the determined factor.

The manner in which the robot hand 200 grasps the object 10 according to such a machine learning algorithm can be changed.

That is, an attempt can be made to grip the right side of the object 10 as shown in (a) of FIG. 3, and an attempt can be made to grip the left side as shown in (b) It is also possible to grasp the lower end of the object 10 through the other fingers 220 on the left side as shown in FIG.

4 is a block diagram illustrating a configuration of a robot hand system according to the present invention for gripping an object based on a vision system.

Referring to FIG. 4, a tactile sensor 144 may additionally be used in addition to the configuration using the vision system of the system 100, which will be described in detail later with reference to the drawings.

Also, in gripping the object 10, the controller 180 may determine the grasping safety of the currently unknown unknown object 10 by using the vision information (tactile sensor information can be used together).

Specifically, the control unit 180 can check the safety by measuring the force / torque required as a whole while moving the holding body 10.

In addition, the control unit 180 may estimate the required force / torque in the current grasping mode in consideration of the expected physical properties of the object 10.

At this time, if it is determined that the grasping safety of the robot hand 200 is lower than a predetermined value, the controller 180 releases the object 10 and controls the robot hand 200 to retake the object 10 based on the learned information It is possible.

In order to regress, it is possible to grasp the optimum condition by the vision and tactile information, and to arrange the object in the regrasing process to suit the purpose of holding the object.

FIG. 5 is a flowchart illustrating a method of grasping an approximate position, shape, and the like of an object through a vision according to the present invention and learning a failed event according to an algorithm when a task fails to execute.

Referring to FIG. 5, first, the camera 121 acquires an image of at least one first object 10 (S10).

Here, the present invention attempts to grasp the first object 10 based only on information obtained through the camera 121 without any prior information about the first object 10.

Then, the control unit 180 determines at least one of the coordinates, the size, and the center of the first object 10 using the obtained image (S11).

In step S11, when at least one factor of the coordinate, size, and center is determined rather than obtaining all the information about the coordinates, size, and center through the image, the controller 180 determines the hand 200 to directly grasp the first object 10 (S12).

In step S12, since the robot hand 200 does not have accurate information, it may fail to grip the first object 10 (S13).

For example, if the first object 10 is not present at a place where the robot hand 200 is moved or the degree of the finger 220 of the robot hand 200 is smaller than the width of the first object 10 An event that can not grasp the first object 10 can be generated by an element such as the movement of the robot hand 200 toward the central part of the expected one object 10 and the other central part.

At this time, the controller 180 determines again at least one of the coordinates, the size, and the center of the first object based on the failure event (S14), and the robot hand 200 determines the first object 10 using the determined factor again. (S15).

Until the object 10 is successfully gripped, steps S13 to S15 according to the present invention can be repeatedly performed.

In addition, even if the object 10 is successfully gripped, as described above, the object 10 can be gripped and re-gripped if it is determined through the stability test that the grip 10 is unstable.

Therefore, according to the present invention, it is possible to overcome the weak points such as long learning time and develop the model-free method and the model-based method in order to develop a gripping and manipulating method through machine learning, Can be complementary to each other.

SECOND EMBODIMENT - Technique for handling an object by using a camera and a tactile sensor together

In the present invention, in order to improve the efficiency of the machine learning described in the first embodiment, a technique of trying to grasp the robot hand 200 by using the tactile sensor 144 in addition to the camera 121 is proposed.

That is, the present invention overcomes the shortcomings of long learning time by developing the existing model-free and model-based methods for developing gripping and manipulation methods through conventional machine learning, And to provide a handling technique that can be utilized even in situations.

In addition, in the second embodiment, a tactile sensor 144 is further utilized to provide a handling technique capable of classifying and learning objects by measuring a shear force, a conductivity of a target object, as well as a conventional normal force measurement .

6 is a block diagram illustrating the configuration of a robot hand system according to the present invention for gripping an object by additionally using a tactile sensor in addition to a vision system.

The configuration shown in Fig. 6 is the same as that shown in Fig. 2 described above, but additionally uses the tactile sensor 144. Fig.

The tactile sensor 144 according to the present invention can sense at least one of the normal force, the shear force, and the conductivity of the object 10 held by the robot hand 200.

6, a plurality of cameras 121 are arranged in the system 1 according to the present invention, and a plurality of tactile sensors 144 are provided at least in a part of the finger unit 220 of the robot hand 200 .

Since the present invention does not have acquired or learned information about an object, it is possible to grasp the approximate position and shape of an object on the basis of a vision system through a plurality of cameras 121, The control unit 180 determines at least one of the coordinates, the size, and the center of the object 10 by using the obtained image, It is possible to control the robot hand 200 to grasp the object 10 directly.

Since the robot hand 200 does not grasp the object 10 based on the information, it can generate an event to release the object 10 after the object 10 is not grasped or grasped , The present invention applies a machine learning algorithm for failed events.

That is, when the robot hand 200 fails to grip the object 10, the controller 180 determines at least one of the coordinates, the size, and the center of the object 200 based on the failure event.

6, when the finger 220 of the robot hand 200 is brought into contact with the object 10 for gripping, the object 10 obtained through the tactile sensor 144 is moved to a position At least one of the normal force, the shear force, and the conductive force of the second electrode is additionally utilized.

In other words, since the robot hand 200 grasps the object 10 again using the determined factor and at least one tactile factor sensed by the tactile sensor 144, , It is possible to grasp the object 10 using time.

FIGS. 7A and 7B show an example of a specific configuration of a robot finger using vision information and tactile information together with respect to the present invention.

7A, a robot hand 200 according to an embodiment of the present invention includes a palm 210 and two fingers 220, and each finger 220 is divided into a plurality of segments .

The robot hand 200 according to the present invention is configured such that the plurality of fingers 220 are unfolded and tilted with respect to the palm 210 as shown in Figures 7A to 7D, 10).

That is, the control unit 180 determines at least one of the coordinates, the size, and the center of the object 10 based on the image obtained through the camera 121, and the robot hand 200 directly So as to grasp the object 10.

In this case, when the robot hand 200 does not grasp or grasp the object 10 in FIG. 7A and then releases the object 10, the controller 180 generates a failure event The robot hand 200 determines again at least one of the coordinates, the size and the center of the object 200, and the robot hand 200 tries again to grasp the object 10 using the determined factor again. A normal force, a shear force, and a conductive state of the object 10 obtained through the tactile sensor 144 when the finger portion 220 comes into contact with the object 10 for gripping. At least one is utilized for re-digging.

According to the machine learning algorithm, the robot hand 200 grasping the object 10 may be used to perform a specific task in addition to simple grasp.

That is, as shown in FIG. 7B, two robot hands 200 are provided, and the first robot hand 200a includes a first palm 210a and a first finger 220a, The robot hand 200b may include a first palm 210b and a first finger 220b.

7B, the first robot hand 200a and the second robot hand 200b are used to insert the second object 10b into the first object 10a.

It is necessary to perform the operation of inserting the second object 10b into the first object 10a rather than the simple gripping operation. Therefore, the two robot hands 200 may fail to grasp continuously, and at this time, Learning algorithm is repeatedly used.

Furthermore, at least one of the normal force, shear force, and conductivity of the object 10 obtained through the tactile sensor 144 is further utilized for re-grasping.

For example, as in the case of manipulating a water bottle 10 containing half of water, the tactile information can be used to grasping and manipulating the object while adjusting the force according to the disturbance of the object (the effect of changing the center of gravity).

In addition, in the present invention, the shape and the like of the hand 200 holding the object 10 with the learned information can be calculated and applied in advance.

Also, as described above, in order to perform a specific task, an algorithm for changing a method, direction, and the like of holding the object 10 in accordance with the purpose of holding the object 10 can be applied.

The tactile information used in the second embodiment includes information such as normal force, shear force, and electric conductivity, and additionally information that can be caught while being in contact with an external environment (such as a floor surface) Can be additionally obtained.

Therefore, by precisely measuring force (including shear force) by using both vision information and tactile information, it becomes possible to perform precise assembly work.

FIG. 8 is a flowchart for explaining the operation of a robot finger using vision information and tactile information together with respect to the present invention.

Referring to FIG. 8, first, the camera 121 acquires an image of at least one first object 10 (S20).

Thereafter, the controller 180 determines at least one factor among the coordinates, sizes, and centers of the first object using the obtained image (S21), and determines whether the hand 200 is a first The object 10 is grasped (S22).

5, in the second embodiment, when the hand 200 is brought into contact with the first object 10 for gripping, the normal force, shear force (force) of the first object 10, a step S22 is performed in which the tactile sensor 144 senses at least one of shear force, conductive, and conductive.

If the robot hand 200 fails to grasp the first object 10 due to insufficient information after step S22 (S23), the controller 180 determines whether the coordinate, size, and center of the first object At least one factor among the factors is determined again (S24).

In the case where the hand 200 desires to grasp the first object again in step S24, unlike the method in FIG. 5, in the second embodiment, by using the determined factor and at least one tactile factor sensed by the tactile sensor And the first object is again gripped (S25).

Therefore, according to the present invention, a tactile sensor capable of detecting a shear force as well as a normal force measurement is developed, which can classify an object by measuring the conductivity of the object, Do.

In addition, it provides real time position / attitude / status recognition technology of parts using visual and tactile information, optimal grip type intelligence technology for stable grip of parts, intelligent grip technology based on recognition information and experience (single gripper / In-Hand technology using visual and tactile information, assembly strategy learning techniques of various parts based on experience, and most of the handling work except for simple repetitive tasks, The present problem that is being solved by the present invention can be solved.

Third Embodiment - A method of more precisely grasping an object based on learned information

In the third embodiment according to the present invention, when information on the object 10 is learned on the basis of the first and second embodiments described above, an additional method of gripping the object 10 faster and more accurately .

An additional method applied in the third embodiment is as follows.

1) a method of resolving the difference between a reference image stored in advance and an input image through an error function

2) Determine the small phase model by determining the two-dimensional point in the gravitational part determination, and apply it to the three-dimensional positioning

First, a method of resolving the difference between a reference image stored in advance and an input image through an error function will be described.

In the first method, the system 100 may pre-store information related to the object 10 through the memory 160 or receive it from the outside through the wireless communication unit 110. [

In addition, information about the object 10 can be obtained by attempting to grasp the object 10 through the camera 121 or the tactile sensor 144.

At this time, the controller 180 compares the information about the object 10 stored in advance or received from the outside with the information obtained through the camera 121 or the like to determine the degree of error, And a method of correcting information.

Here, the error function for comparing the information about the object 10 stored in advance or received from the outside with the information obtained through the camera 121 or the like is as follows.

The coverage error is εcover (u, v, x, y) and the range error is range (u, v, x, y, z). These error terms are evaluated for each pixel in the coordinates (u, v) of the input range image. The pixel translation value (x, y, z) of the reference range image determines its position with respect to the input range image.

The function is summed over all image pixels (u, v) using the weight [lambda] (e.g., [lambda] = 10). The normalization factors Ncover and Nrange separate the error from the object and the image size. The error can be minimized when the image R is aligned with the partially shielded object in the input image I.

Next, a method of determining a small-sized model by determining a two-dimensional point in the gravitational part determination, and fitting it to a three-dimensional position and applying it to the grip will be described.

FIG. 9 shows an example of a small-sized model and three-dimensional position fitting used in the gripping of the present invention, and FIG. 10 shows an example of fitting to a three-dimensional position to be applied to a grip according to the present invention.

Referring to Fig. 9, the control unit 180 receives the designation of a range to be made with the " recognition area " in the photographed image through the input device. Here, the " recognition area " is an area set by the control unit 180 in the photographed image in order to designate a portion to be gripped by the hand 200. [ The recognition area is DT which can be used as a target area for distance measurement.

Referring to FIG. 9A, recognition areas 11, 12, and 13 indicating areas that the control unit 180 desires to grasp in the object 10 are shown.

The recognition area 11 designates the body part of the cup, the recognition area 12 designates the rim part of the cup opening, and the recognition area 13 designates the handle part.

9 (b), an icon list 20 indicating a two-dimensional graphic form associated with the recognition area 13 can be further displayed on the display unit 151. [

Icons 21 to 24 shown in FIG. 9 (b) each show an example of a miniature image model.

That is, the icon 41 is a square column model, the icon 42 is a flat plate model, the icon 43 is a cylinder model, and the icon 43 is a frustum model.

Each miniature model has configuration parameters to define its shape and size, and placement parameters to specify its position and orientation.

In addition, the type of the small-sized model is not limited to that shown in Fig. 9B, and various shapes such as a secondary ellipsoid, an L-shaped footstep, and a C-shaped cylindrical shape can be further utilized.

9 (c) is a conceptual diagram showing the acquisition of the three-dimensional position data 30 of the work space corresponding to the recognition areas 11, 12, and 13.

The three-dimensional position data 30 in (c) of FIG. 9 shows the depth seen from the robot 200 in the work space together with the recognition area.

9 (d), the small-sized model (specifically, the cylindrical model 40) is displayed in a superimposed manner, and the fitting result when the user selects the cylindrical model is shown.

10, the data structure is a concrete example and shows the contents of data relating to the grip pattern applicable to the cylindrical model 40. FIG.

If the types of the robot hands 200 are different, the executable grasp patterns are different and the grasp patterns applicable to the same small-sized model may be different.

That is, in FIG. 10, the hand 200 is a grip pattern applicable when the hand type is a "plane three-joint 2-finger hand" and the small-sized model is a cylindrical model, and four grip patterns are recorded together with the application conditions.

Specifically, the four grasping patterns can be used as side grasping grasping, cross-sectional grasping grasping, grasping grasping, and edge grasping grasping.

Accordingly, in the present invention, the controller 180 determines a two-dimensional model corresponding to a point the robot hand 200 wants to grasp based on the acquired image, determines the two-dimensional model as a three-dimensional model through the depth information, The robot hand 200 can grasp the object 10 by determining an attitude optimized for the grasp of the determined three-dimensional model.

Thereby, the robot hand 200 can quickly grip the object 10 without obtaining additional information based on the obtained information.

Fourth Example  - vision  A method of performing a task through an operation of gripping based on information and tactile information

In the present invention, a method of holding the object 10 by the robot hand 200 and performing a specific task will be described on the basis of the above-described first to third embodiments.

FIG. 11 is a flow chart for explaining an overall operation of a robot phage using vision information and tactile information together in connection with the present invention. FIG. 12 shows an example of a flowchart for explaining an object grasp using learning in accordance with the present invention. It is.

11, based on the information obtained from the tactile sensor 144 and the nonmetal sensor 145 and the information obtained through the camera 122, the controller 180 determines the properties such as the physical properties of the object 10 And attempts to grasp the object 10 based on the prediction.

At this time, the gripping of the object may fail, and the controller 180 adjusts the factor for re-gripping based on the information obtained through the camera 122 every time a failure event is generated, (S30) based on the information formed through the touch operation.

That is, in step S30, the control signal is fed back, pressure, roughness information is provided, or physical property information about the object 10 is transmitted to the controller 180.

At this time, the control unit 180 learns information about the object 10 based on the information obtained through the camera and the information obtained through the step S30 (S31), and based on the learned information, (Step S32), and a specific task can be performed by repeatedly grasping the object 10 based on the reflected reset condition.

Referring to FIG. 12, a process 1000 for learning based on information obtained through the camera 122 and a process 1100 for learning based on information obtained through the tactile sensor 144 are shown. A process 1200 of a robot hand performing a gripping operation based on input information and tactile input information is shown.

Fifth Embodiment - A method of performing a task through a re-finger operation

On the other hand, in order for the robot hand 200 to perform a specific task, it may be necessary to use regrasping when the manipulation according to a given purpose can not be performed in the present grasping form.

Therefore, in the present invention, it is desired to propose a technique capable of completing a specific task using a regression when a robot hand can not perform a manipulation for a given purpose in the present grasping form in order to perform a specific task .

That is, according to the present invention, when the two-arm robot hand 200 assembles an object, when an assembling operation can not be carried out while holding an object at present, an optimal algorithm And the like.

In addition, in the present invention, the tactile sensor 144 is additionally utilized in the re-gripping operation to measure a normal force, which is widely used in the past, as well as a handling technique capable of classifying and learning objects by measuring shear force, The purpose is to provide.

13 shows an example in which the gripping method is changed according to the change of the direction and the position of the object in relation to the present invention.

Referring to Fig. 13 (a), the position or direction of the object 10 can be changed by an external force or an operation to grasp.

In order to perform a specific task in which a plurality of objects 10 exist, an order in which a plurality of objects 10 are combined is determined. In an event in which an object 10 May occur.

In this case, as shown in FIG. 13 (b), the gripping of the object 10 must be performed in consideration of the change of the direction and the position of the changed object 10.

13B, there are a plurality of objects 10, and in order to perform a specific task, the order in which a plurality of objects 10 are combined is determined, A method of releasing the gripped object 10 and gripping the object 10 newly should be applied when an event of gripping the object 10 not corresponding to the object 10 occurs.

In the rephasing, a method of gripping a point of the object 10 optimized for the next process or gripping the number object 10 in which the position, direction, etc. of the object 10 is changed to suit the next process .

14 is a diagram for explaining the necessity of repagination according to the present invention.

In FIG. 14, a plurality of objects 10 exist, and a specific task of assembling a plurality of objects 10 is performed.

At this time, the controller 180 may store information on the order of assembling the plurality of objects 10 or the like in advance in order to perform a specific task, or may receive instructions from the outside through the communication unit 110. [

As shown in FIG. 14A, the object 10 can be completed when the first object 10a, the second object 10b and the third object 10c are combined in order, After the object 10a and the second object 10b are combined, the third object 10c must be combined with the second object 10b so that the task can be performed.

However, at this time, as shown in (b) of FIG. 14, when a plurality of robot hands 200 generate an event to perform an operation of holding and combining the second object 10b and the third object 10c first .

14 (b), the second object 10b and the third object 10c are combined and positioned on the left side as shown in FIG. 14 (c), and the first object 10a ) Can be located on the right side.

At this time, as shown in (d) of FIG. 14, an event in which the first object 10a is coupled to the bottom of the form in which the second object 10b and the third object 10c are combined is generated, (10) can not be assembled.

Therefore, in this case, the first object 10a and the second object 10b are combined in a predetermined order, and then the third object 10c is combined with the second object 10b so that the task can be performed This can be done through re-grasping.

That is, in FIG. 14 (b), two robot hands 200 need to grasp the first object 10a and the second object 10b, and the second object 10b and the third object 10c It is possible to release the objects held by the robot hand 200 and again grasp the correct object.

In addition, in the present invention, when an erroneous event occurs, it is also possible to perform re-grasping by changing object objects and procedures to be held in correspondence with the event.

That is, when an incorrect event is generated by holding and combining the second object 10b and the third object 10c in FIG. 14 (c), the second object 10b and the third object 10c ) To move the combined structure to the left or to the right or to move and rotate the first object 10a to the retracted position and then to engage the combined structure with the first object 10a A specific task can be performed.

FIG. 15 shows an example of a repagment learning model according to the present invention.

Referring to FIG. 15, in (1), the robot hand 200 succeeds in grasping the object 10 vertically.

However, in performing a specific task, the control unit 180 can determine that the form (1) is undesirable and release the object 10 in the gripping operation as in (2) and (3).

In addition, since the controller 180 knows the next process for completing a specific task, the controller 180 can change the position and direction of the object 10 when the object 10 is released in the releasing operations of (2) and (3).

Then, according to the control of the controller 180, the robot hand 200 performs the operation of holding the object 10 at the left upper end at an angle and lifting up the object.

(4) In the performance of the operation, (5) operation is performed in which the information acquired through the tactile sensor 144 and the information acquired through the camera 122 are grasped together based on the operation (1).

Thereafter, the operation (6) of moving the object 10 held by the operation (5) according to a predetermined order and process can be performed.

FIG. 16 is a diagram for explaining a re-fingerprinting method according to the fingerprint success probability of the present invention.

Referring to FIG. 16 (a), a kernel density estimation method can be utilized in the present invention.

That is, it is determined whether the success rate of a certain gripping operation is high based on the grasped success probability P (grasped | s, a) after the re-grasping operation and the object state probability P (s_new | s, a, grasped ), It is possible to judge what action is good to judge whether or not to catch the object center.

As shown in FIG. 16B, in re-grasping the object 10, a kernel density estimation method is used, and based on the grasp success probability P (grasped | s, a) after the re-grasping operation, As shown in FIG. 16 (c), if it is necessary to center the object on the basis of the object state probability P (s_new | s, a, grasped) after successful re-filling, It can be judged whether or not the operation is good.

FIG. 17 is a diagram for explaining the performance of a task to which repargement is applied, according to the present invention. FIG.

17 (a) shows a case where, when a plurality of objects 10 are moved according to a predetermined process to perform a specific task, when an event that is inconsistent with a predetermined process is generated, re- FIG.

17 (b) shows a case where, when a plurality of objects 10 are moved according to a predetermined process to perform a specific task, when an event that is inconsistent with a predetermined process is generated, A process of newly performing the most suitable process to be described below is determined by the control unit 180, and the process of performing repetition of hitching and grating is shown.

In the present invention, all of the contents shown in FIGS. 17A and 17B can be applied.

FIG. 18 is a flowchart illustrating a method for completing a specific task using regrasping when a manipulation according to a given purpose can not be performed in the current grasping form according to the present invention.

Referring to FIG. 18, the camera 122 acquires an image for a plurality of objects 10 (S40).

Thereafter, the robot hand 200 grasps and moves the first object among the plurality of objects (S41) in accordance with a preset order on the basis of the control of the control unit 180. [

Thereafter, the control unit 180 determines whether or not the task can be performed (S41).

Here, in the case where the task can not be performed, at least one of the position and the direction of the first object is changed so that the first object can not be gripped as desired, or the wrong gripping and combining are performed and the task can not be performed according to the predetermined process And the like.

If it is determined in step S41 that the task can not be performed, the control unit 180 controls the robot hand 200 to release the first object held (S42).

After step S42, the controller 180 newly grasps the first object according to the original plan or newly constructs a plan suitable for the wrong process, and controls the first object based on the image obtained through the camera so as to grasp the first object. And determines at least one of the changed direction and the position (S43).

Further, the robot hand 200 proceeds to step S44 of re-grasping the first object based on the changed direction and position.

The control unit 180 can control so that steps S41 to S44 are repeatedly learned and performed until a specific task is performed.

Since the present invention is known as regrasping in assembling, information on the movement shape and assembly order and the like is known in advance, the following process is known, and the object is placed at the time of releasing from the grip so as to be in a good arrangement with respect to the next work, .

According to the present invention, when an event is generated in which the object 10 can not be grasped differently from the expected shape, the corresponding part of the object 10 is rotated through the robot hand 200, the time information is acquired again, Operation may be performed.

In the present invention, based on the acquired information, a geometric characteristic reflection file for assembling each part is generated (hole orientation direction, size, etc.), permutation is performed, and a primary assembly operation sequence is secured through combination of all parts Afterwards, you can confirm that all the holes of each part to be assembled are used. In addition, searching the assembly sequence and solving the problem with Constraint satisfaction problem (checking whether inter-part interference occurs in assembly direction during assembly process).

Accordingly, the present invention can provide a technique for completing a specific task by using a regression when a robot hand performs a specific task and can not perform a manipulation for a given purpose in a current grasping form.

In addition, according to the present invention, a tactile sensor capable of detecting a shear force as well as a conventional normal force measurement and measuring a conductivity of a target object and developing a tactile sensor can be used. .

In addition, it provides real time position / attitude / status recognition technology of parts using visual and tactile information, optimal grip type intelligence technology for stable grip of parts, intelligent grip technology based on recognition information and experience (single gripper / In-Hand technology using visual and tactile information, assembly strategy learning techniques of various parts based on experience, and most of the handling work except for simple repetitive tasks, The present problem that is being solved by the present invention can be solved.

Meanwhile, the embodiments of the present invention described above can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs) , FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The foregoing description of the preferred embodiments of the invention disclosed herein has been presented to enable any person skilled in the art to make and use the present invention. While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. For example, those skilled in the art can utilize each of the configurations described in the above-described embodiments in a manner of mutually combining them. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention. The present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, claims that do not have an explicit citation in the claims may be combined to form an embodiment or be included in a new claim by amendment after the filing.

Claims (11)

A camera for acquiring an image of at least one first object;
A controller for determining at least one of a position coordinate, a size and a center of the first object using the obtained image; And
And a hand for gripping the first object using the determined factor when at least one factor of the position coordinates, size, and center is determined,
When the hand fails to grasp the first object,
Wherein the controller determines again at least one of a position coordinate, a size, and a center of the first object based on the failure event,
Wherein the hand grasps the first object again using the again determined factor. The method according to claim 1,
Wherein,
Wherein the controller controls the hand to grasp the first object using only the determined factor in the absence of dictionary information for the first object. The method according to claim 1,
Wherein,
When the hand re-fails to grasp the first object,
Determining again at least one of a position coordinate, a size and a center of the first object based on the re-failure event, and controlling the hand to retouch the first object using the again determined factor Wherein the robot hand is a robot hand. The method according to claim 1,
When the hand grasps the first object again using the determined factor,
Wherein,
The hand determines the gripping stability of the first object held by the hand,
And controls the hand to release the first object when the gripping stability is equal to or less than a preset reference. The method according to claim 1,
A tactile sensor for sensing a tactile factor of at least one of a normal force, a shear force and a conductive of the first object when the hand is in contact with the first object for gripping; Further included,
Wherein the at least one tactile factor detected by the tactile sensor is used together when the hand grasps the first object again using the determined factor. The method according to claim 1,
The hand includes a plurality of fingers,
Each of the plurality of fingers including a distance sensor for sensing a distance from the first object,
When the first object is gripped,
Wherein each of the plurality of fingers moves close to the first object at the same speed based on information obtained through the plurality of distance sensors. A first step of the camera acquiring an image for at least one first object;
A second step of the controller determining at least one of a position coordinate, a size and a center of the first object using the obtained image;
A third step of the hand grasping the first object using the determined factor when at least one factor of the position coordinate, the magnitude and the center is determined;
A fourth step of the hand failing to grasp the first object;
A fifth step of the controller determining again at least one of a position coordinate, a size and a center of the first object based on the failure event; And
And a sixth step of the hand grasping the first object again using the again determined factor. 8. The method of claim 7,
In the third step,
Wherein the hand grasps the first object using only the determined factor in the absence of prior information about the first object. 8. The method of claim 7,
If the hand fails to grasp the first object in the sixth step,
Wherein the first to sixth steps are repeatedly performed. 8. The method of claim 7,
After the sixth step,
A seventh step of determining the gripping stability of the first object held by the hand; And
Further comprising the step of releasing the first object by the hand when the gripping stability is less than a predetermined standard. 8. The method of claim 7,
In the third step,
Sensing a tactile factor of at least one of a normal force, a shear force and a conductivity of the first object through the tactile sensor when the hand is in contact with the first object for gripping, ,
In the sixth step,
Wherein the at least one tactile factor sensed by the tactile sensor is used together when the hand grasps the first object again using the determined factor.

Images