JP6948164B2 - Work robot arm attitude control system and method - Google Patents

Description

The present invention relates to a technique for controlling the posture of a robot, and more particularly to a technique for controlling an arm posture of a working robot.

As an example of a conventional working robot, Patent Document 1 proposes a hydraulically driven actuator having a large handling weight and a muscle robot incorporating the hydraulic actuator.

Generally, in the operation of the working robot, it is conceivable that the operator directly visually observes the robot or operates the robot while observing the movement of the robot in a remote place photographed by a camera or the like. In addition, the information processing device can be assisted in determining the actual operation of the robot.

Patent Document 2 discloses, as a robot motion planning method, a method of sampling innumerable postures that interpolate an initial posture and a target posture and determining a trajectory so that the cost is minimized based on a predetermined cost function. NS.

Japanese Unexamined Patent Publication No. 2016-203359 Japanese Unexamined Patent Publication No. 2015-160253

In the decommissioning work of a nuclear reactor, the work is assumed under the condition that the surrounding structures are unknown and it is difficult for people to enter in a high radiation environment. Therefore, in principle, work by a remote-controlled robot is assumed.

When a robot works, it is necessary to control interference with surrounding structures in the working environment in order to avoid damage to the robot itself and damage to surrounding facilities. Therefore, it is required to accurately control the position and posture of the robot.

However, sensors that use electronic components such as encoders that directly measure joint angles, such as those mounted on general industrial robots, may malfunction in high radiation environments. Therefore, it may not be possible to directly grasp the posture of the robot by the sensor.

Further, when an image is acquired by a video camera and an attempt is made to grasp the posture of the robot based on the image, high radiation may generate noise in the image of the camera. Further, in an environment of high temperature and high humidity, the image of the camera may be blurred due to the influence of water vapor and the like. Furthermore, there are restrictions on the placement of video cameras and the installation of lighting equipment. If the clarity of the image of the video camera cannot be sufficiently ensured, the work efficiency of the operator who operates the robot by looking at the image may be lowered.

Therefore, an object of the present invention is to improve operability by an operator of a remote-controlled robot that can be used in an environment of high radiation.

One aspect of the present invention is an arm attitude control system for a working robot including an arm having a movable portion. The system takes a camera image of a marker attached to the arm as an input and performs processing. The system components include an image analysis device that extracts markers from the camera image and generates three-dimensional coordinates of the markers, and arm posture data that indicates the posture of the arm based on the three-dimensional coordinates of the markers and the structural model of the arm. It is provided with an arm posture estimation device for generating an image and an arm posture control device for controlling the posture of the arm based on the arm posture data. Further, when a predetermined error is detected in the generation of the arm posture data, the posture of the arm before the detection of the predetermined error or the basic posture is restored, or the situation where the predetermined error is detected is stored. In the subsequent control of the posture of the arm, avoidance control is performed so as to avoid the situation.

Another aspect of the present invention is an arm posture control method for a working robot including an arm having a movable portion. This method includes an input step of inputting a camera image of a marker attached to an arm, an image analysis step of extracting a marker from the camera image and generating a three-dimensional coordinate of the marker, and a three-dimensional coordinate of the marker and the arm. Based on the structural model of the arm posture estimation step that generates arm posture data indicating the arm posture, the arm minute movement target amount is obtained based on the arm posture data, and the arm minute movement target amount is obtained. Arm to control posture Perform a posture control step. Further, when the arm posture data indicating the posture of the arm cannot be generated, the posture of the arm controlled based on the arm posture data or the return control for returning to the basic posture is performed, or the arm posture data is generated. The situation that could not be done is memorized, and in the subsequent control of the posture of the arm, avoidance control is performed so as to avoid the situation.

The operability of the remote-controlled robot can be improved.

The perspective view which shows the whole structure of the hardware of the arm posture control system of the robot of an Example. The flow chart which shows the flow of the whole processing of the arm posture control system of the robot of an Example. The block diagram which shows the functional block of the arm posture control system of a robot. The flow chart which shows the processing flow of an image analyzer. The flow chart which shows the processing flow of the arm posture estimation processing in the arm posture estimation apparatus. The flow chart which shows the processing flow of the arm attitude control processing in the arm attitude control device.

In one example of the embodiment described below, a robot system including a robot having an arm (working robot) uses a non-contact sensor installed in a working environment or the robot itself. As a non-contact sensor, for example, there is a stereo camera. A plurality of markers are arranged on the arm, and the robot system grasps the posture of the arm by detecting the markers with a non-contact sensor. According to this configuration, the posture of the robot can be grasped by the configuration having excellent radiation resistance.

Then, when the non-contact sensor cannot detect the arm, the posture returns to the previous posture or the basic posture. The basic posture is, for example, the posture when the hydraulic pressure is maximized or minimized in the work robot disclosed in Patent Document 1 described above. The basic posture can be defined as a posture that is uniquely determined when a control parameter such as hydraulic pressure is set to a predetermined condition or a predetermined value. According to this configuration, even if the non-contact sensor cannot detect the arm and the posture cannot be recognized, it is possible to return to the recognizable posture.

In addition, the position where the arm cannot be detected is memorized, and in the subsequent operation, the arm is driven while avoiding that position. If the non-contact sensor cannot detect the arm, it is assumed that the marker cannot be seen from the camera due to, for example, an obstacle. Therefore, by controlling the posture of the arm by avoiding such a position in advance, it is possible to continuously grasp the posture of the arm.

(1. Overall hardware configuration of the system)
FIG. 1 is a perspective view showing an overall configuration of hardware of a robot arm posture control system according to an embodiment of the present invention.

The robot 1 having the arm 1001 can be moved by, for example, the moving mechanism 1002. The arm 1001 includes one or more joints 1003, which can be angled or rotated. The posture of the arm 1001 can be changed by an instruction from an operator or the like in a remote place, and the work object 7 can be operated by the holding unit 1004 or the like. The mechanical structure of the arm 1001 is disclosed in, for example, Patent Document 1. The work robot disclosed in Patent Document 1 has a structure that operates by hydraulic pressure, and in particular, water pressure can be expected to have a small effect on the work environment due to oil leakage from the robot.

One or more markers 3 are arranged on the arm 1001. In this embodiment, the marker has a configuration with good visibility. The marker 3 is, for example, a white disk having a predetermined size and is attached to each part of the arm 1001. The marker 3 may be made to emit light by a light emitting diode or the like. Further, the size and color of the marker may be changed for each part. As the configuration of the marker 3, it is desirable to select a color or shape having a clear contrast with respect to the background so that the marker 3 can be sufficiently detected in the working environment of the robot.

The marker 3 attached to the arm 1001 is photographed by the stereo camera 2. The stereo camera 2 is, for example, two video cameras. The image taken by the stereo camera 2 is transmitted to the image analysis device 6. The output of the image analysis device 6 is input to the arm posture estimation device 5, and the output of the arm posture estimation device 5 is input to the arm posture control device 4. The output from the arm posture control device 4 is input to the arm 1001 and controls the posture of the arm 1001.

For example, in the technique of Patent Document 1, hydraulic pressure is used to control the posture of the arm. When this method is adopted, the arm attitude control device 4 includes a pump for controlling the pressure of the liquid, and the liquid itself delivered from the pump may be considered as an output from the arm attitude control device 4.

(2. Overall system processing flow)
FIG. 2 is a diagram showing an overall processing flow of the arm posture control system of the robot of this embodiment.

In the process S101, the stereo camera 2 photographs the arm 1001 and the marker 3 of the robot 1 to acquire a stereo camera image.

In the process S102, the image analysis device 6 performs the image analysis process. This will be described in detail later with reference to FIG.

In the process S103, the arm posture estimation device 5 performs the arm posture estimation process. By this process, the operator or the system can grasp the posture of the robot 1 or the arm 1001. This will be described in detail later with reference to FIG.

In the process S104, the arm attitude control device 4 performs the arm attitude control process. This process is a process in which the operator or the system instructs the robot 1 or the arm 1001 to take the next posture. This will be described in detail later with reference to FIG.

In the process S105, the arm attitude control device 4 outputs an arm operation instruction to the robot 1.

(3. System functional block configuration)
FIG. 3 is a block diagram showing a functional block of a robot arm posture control system, which is an embodiment of the present invention. The arm attitude control device 4, the arm attitude estimation device 5, and the image analysis device 6 are connected by a signal cable or a wireless link, and can transmit or receive signals.

In this embodiment, the arm posture control device 4, the arm posture estimation device 5, and the image analysis device 6 are realized by a normal server including a processor, a memory, an input interface, and an output interface. Functions such as calculation and control of each device are realized in cooperation with other hardware by executing a program stored in memory by a processor. A program executed by a computer or the like, its function, or a means for realizing the function may be referred to as a "function", a "means", a "part", a "unit", a "module", or the like. The above configuration may be configured by a single computer, or any part of the processor, memory, input interface and output interface may be configured by other computers connected to each other.

Although the arm posture control device 4, the arm posture estimation device 5, and the image analysis device 6 are described as separate devices in FIGS. 1 and 3, they may be a single device. That is, it may be configured by three servers or one server having a plurality of functions.

Of the configurations shown in FIG. 3, the robot 1 and the stereo camera 2 that captures the robot 1 may be arranged in a high-radiation work environment. In this case, since the components of a normal server, particularly an electronic device such as a SRAM (Static Random Access Memory), have poor radiation resistance, the arm posture control device 4, the arm posture estimation device 5, and the image analysis device 6 are used. It is desirable to install the server to be configured in a remote location from the work environment. A signal cable for transmitting image data can be arranged between the server and the stereo camera 2 to secure a distance.

Further, a signal line for sending an arm operation instruction signal to the robot 1 can be arranged between the server and the robot 1 to maintain a distance. Alternatively, as described above, when the arm 1001 operates under the pressure of a liquid such as water or oil, a pipe for sending the liquid can be installed. In this case, a pump or the like (not shown) is controlled by a signal from the arm operation control unit 703, and the pressure of the liquid supplied to each part of the arm 1001 is controlled by the action of the pump or the like.

(3-1. Stereo camera)
In general, the smaller the element scale of an electronic component, the lower the radiation resistance of the stereo camera 2. Therefore, the stereo camera 2 only needs to have a resolution that can be recognized by the marker 3, and a high-definition image is not necessary. The stereo camera 2 generates a stereo camera image signal V and transmits it to the image analysis device 6. Further, the stereo camera 2 shall start, end, or change the shooting angle by the instruction signal C from the image analysis management unit 601.

The number of stereo cameras is one or more. Ideally, there should be no blind spots in the image of the arm 1001, so three or more may be installed. The relative positional relationship and optical axis direction of the stereo camera shall be converted into data by a method such as setting at the time of installation, actual measurement after setting, or estimation from the camera image.

(3-2. Image analyzer (overview))
FIG. 4 shows a processing flow of the image analysis apparatus 6, and the processing contents will be described with reference to FIG. In the image analysis device 6 of this embodiment, the image analysis management unit 601 controls the overall operation. When the image analysis management unit 601 of the image analysis device 6 receives the stereo camera image signal V from the stereo camera 2, the stereo camera image signal V is input to the marker detection unit 602 (S201). Stereo camera images shall include images from two or more cameras.

The marker detection unit 602 detects the marker portion from the stereo camera image signal V by image processing, and detects the three-dimensional coordinates of each marker according to the principle of the stereo camera. Further, if necessary, the marker coordinates are complemented by using the marker three-dimensional predicted coordinates 3DE (S202). The marker three-dimensional coordinate 3D obtained as a result of the process S202 is sent to the image analysis management unit 601. The coordinate conversion unit 603, the marker three-dimensional predicted coordinate 3DE, and the complementation of the marker coordinates will be described later.

In the image processing for detecting the marker, for example, when the marker 3 is a white circle, a high-contrast circular portion is detected. The color and reflectance of the marker may be changed to detect the color or the reflected light. Then, the marker two-dimensional coordinates indicating the coordinates of the marker 3 in the two-dimensional image are generated.

The image analysis device 6 has sensor arrangement information 604 as data. The sensor arrangement information 604 is a sensor, for example, a position coordinate and an angle of view of a camera (direction of the optical axis of the camera and a viewing angle). The marker detection unit 602 converts the two-dimensional coordinates viewed from the camera into three-dimensional coordinates using the sensor arrangement information 604, and outputs the marker three-dimensional coordinates 3D to the image analysis management unit 601. At this time, as described above, the marker three-dimensional coordinates are complemented as necessary.

As a method of restoring three dimensions from a plurality of images, there is a method known as SfM (Structure from Motion). The origin and coordinate system of the three-dimensional coordinates may be arbitrary. The image analysis management unit 601 transmits the marker three-dimensional coordinate 3D to the arm posture estimation device 5 (S203).

(3-3. Arm posture estimation device)
FIG. 5 shows a processing flow of the arm posture estimation process in the arm posture estimation device 5, and the processing contents will be described with reference to FIG. As shown in FIG. 5, the posture update unit 501 of the arm posture estimation device 5 receives the marker three-dimensional coordinate 3D from the image analysis management unit 601 of the image analysis device 6. The stereo camera image signal V includes a plurality of frame images in a time series, and the movement of the marker 3 can be tracked by generating the marker three-dimensional coordinates 3D from each frame.

The posture update unit 501 of the arm posture estimation device 5 performs a process of estimating the arm posture and updating it as the arm posture data AP. The arm posture estimation device 5 accumulates the arm posture data AP as arm posture history data 503 in time series. The content of the estimation process of the arm posture data AP will be described later, but in this embodiment, the arm predicted from the marker 3D coordinates 3D based on the detected marker and the arm posture history data 503 (past arm posture data AP). Both attitude prediction data APE are used.

As shown in FIG. 3, the arm posture history data 503 is input to the posture prediction unit 504. The posture prediction unit 504 generates arm posture prediction data APE from the arm posture history data 503 using the arm movement model 505 (S301). The movement model 505 of the arm is a model that follows a constraint condition that each joint of the arm operates at a predetermined equiangular velocity, for example. Such a prediction can be made using, for example, a Kalman filter.

Marker three-dimensional coordinates 3D are input to the posture update unit 501 from the image analysis management unit 601 (S302). The posture update unit 501 calculates the posture of the arm from the marker three-dimensional coordinates 3D using the arm model 502. Here, the arm model 502 includes information on the joint-to-joint connection relationship of the arm 1001, the joint-to-joint distance, the joint movable range, the position of the marker with respect to each joint axis, and the like. When the arm is driven by the pressure of the liquid, the pressure of the liquid corresponding to the posture may be used.

The posture update unit 501 calculates a parameter for uniquely specifying the posture of the arm from the arm model 502 and the marker three-dimensional coordinate 3D. This parameter is, for example, the angle of each joint of the arm, or the pressure of the liquid to take that angle. If the arm 1001 has an expansion / contraction function, it may include an expansion / contraction length. In this embodiment, the arm 1001 is driven by the pressure of the liquid, and the posture of the arm and the control signal for driving the arm are defined by the pressure of the liquid supplied to each part of the arm.

The posture update unit 501 estimates the arm posture from the arm posture calculated from the marker three-dimensional coordinates 3D and the arm model 502, and the arm posture prediction data APE predicted by the posture prediction unit 504, and uses it as the arm posture data AP. Input to the arm posture estimation management unit 506.

As a specific method for estimating the arm posture, for example, the likelihood function is evaluated from the posture of the arm calculated from the marker 3D coordinates 3D, the parameters of the Kalman filter are corrected, and the arm posture data AP is estimated and updated. (S303). The arm attitude data AP is transmitted from the arm attitude estimation management unit 506 to the arm attitude control device 4.

While the arm posture history data 503 is not sufficiently accumulated, the arm posture calculated from the marker three-dimensional coordinates 3D may be used as the arm posture data AP as it is.

On the other hand, in this embodiment, as shown in FIG. 3, the arm posture prediction data APE generated by the posture prediction unit 504 is fed back to the image analysis management unit 601 of the image analysis device 6. By using the arm posture prediction data APE, it is possible to supplement the marker detection omission.

(3-4. Image analyzer (complementing marker coordinates))
Returning to FIG. 4, the complement processing of the marker detection omission will be described. The arm posture prediction data APE is input to the coordinate conversion unit 603 (S204), and is converted into the marker three-dimensional prediction coordinates 3DE based on the arm model 605 (which may have the same contents as the arm model 502) (S205). The marker three-dimensional predicted coordinates 3DE are sent to the marker detection unit 602 via the image analysis management unit 601.

The marker detection unit 602 complements the marker 3D coordinates based on the signal detected by the image of the stereo camera 2 with the marker 3D predicted coordinates 3DE to generate the marker 3D coordinates 3D. That is, the marker 3D predicted coordinates 3DE are the coordinates of the marker that should be based on the posture of the arm predicted by the posture prediction unit 504, and even if the marker 3 is hidden by an obstacle or water vapor, it is caused by the lack of data. It is possible to supplement the detection omission of the marker. For example, when two markers are detected by the image of the stereo camera 2 and no marker between them is detected, the marker three-dimensional predicted coordinate 3DE can suggest the existence of the marker between them. However, for example, complementing the marker at the tip of the arm is difficult in a situation where the arm angle cannot be known.

As described above, as shown in FIG. 4, the marker detection unit 602 complements the marker information obtained from the image of the stereo camera 2 by using the marker 3D predicted coordinates 3DE (S202). Then, the marker three-dimensional coordinate 3D is generated and output to the image analysis management unit 601 (S203). In this embodiment, the arm posture prediction data APE in the posture prediction unit 504 is used for complementing the marker detection in the image analysis device 6 as described above, but the complement processing may be omitted in some cases.

In the working environment of the robot 1, even when the above complementation is performed, it may not be possible to sufficiently recognize a part or all of the posture of the robot 1 due to surrounding obstacles, water vapor, or the like. For example, when the image analysis management unit 601 cannot recognize the coordinates of more than a predetermined number of markers, or when the markers at the tip of the arm are lost, the image analysis management unit 601 is added to or instead of the marker three-dimensional coordinates 3D. Then, an error signal indicating that the posture of the robot cannot be recognized is transmitted to the arm posture estimation device 5. The arm posture estimation device 5 that has received the error signal stores the error signal in the arm posture history data 503 in place of or in addition to the arm posture data AP, and also transmits the error signal to the arm posture control device 4. At this time, the marker 3D coordinate 3D immediately before or the arm posture data AP immediately before may be further added to the error signal.

Further, instead of being generated by the image analysis management unit 601 as described above, the error signal may be generated when the posture update unit 501 cannot estimate the arm posture. The case where the arm posture cannot be estimated is, for example, a case where the posture of the arm calculated from the marker three-dimensional coordinates 3D and the arm model 502 and the arm posture prediction data APE predicted by the posture prediction unit 504 are different from each other by a predetermined value or more.

(3-5. Arm attitude control device)
FIG. 6 shows a processing flow of the arm attitude control processing in the arm attitude control device 4, and the processing contents will be described with reference to FIG. The arm motion planning unit 701 of the arm attitude control device 4 receives the arm attitude data AP from the arm attitude estimation device 5. The arm motion planning unit 701 sends the arm posture data AP to the final target setting unit 702.

In the initial setting of the attitude control of the arm, the final target setting unit 702 sets the final target attitude FP which is the final target of the arm 1001 (S401). For the final target posture FP, for example, the posture of the arm 1001 is reproduced as an image from the arm posture data AP and displayed on an image display device (not shown), and the operator confirms the position where the holding unit 1004 is to be moved, for example. It may be specified on the image. Alternatively, other known methods may be used. At this time, the image obtained by the stereo camera image signal V may also be displayed. For the sake of explanation, it is assumed that the arm 1001 is visible at the time when the final target posture FP is set.

The final target posture FP set as the data handled by the system is defined by, for example, the angle and hydraulic pressure of each joint of the arm 1001 for taking that posture. In this embodiment, the final target posture FP is defined by the pressure of the liquid supplied to each part of the arm 1001 as well as the posture of the arm and the control signal for driving the arm. At this time, the posture of the arm instructed by the operator may be converted into the pressure of the liquid supplied to each part of the arm in order to take that posture. The set final target posture FP is transmitted to the arm motion planning unit 701.

The arm motion planning unit 701 generates a motion plan from the current arm posture data AP and the final target posture FP (S402). In robot control, there is known a technique called a motion planning method that dynamically generates a trajectory according to the environment, initial posture, and target posture in order to respond to changes in the environment and changes in the target posture. Motion planning is a technology that determines the trajectory of a robot's motion based on some criteria by inputting information on the initial posture, target posture, and surrounding environment. The motion planning problem is a problem of finding how to move the arm to obtain a specified posture in an industrial arm-type robot. An example of such a motion planning method is also disclosed in Patent Document 2.

The arm motion planning unit 701 inputs the latest arm posture data AP (S403). Then, it is determined whether or not the posture of the arm 1001 can be grasped (S404). This determination can be made based on the content of the arm posture data AP generated by the image analysis management unit 601 or the posture update unit 510, or the presence or absence of an error signal (described above). If it is guaranteed that the arm is visible at the time of setting the final target posture FP, this step may be omitted before the first minute movement.

The arm motion planning unit 701 determines whether or not the posture of the arm 1001 is the final target posture FP based on the latest arm posture data AP (S405). If the posture of the arm is known and the current posture of the arm is the final target posture, the arm motion control is terminated.

The arm motion planning unit 701 can grasp the posture of the arm, and if the current posture of the arm has not reached the final target posture, the arm motion planning unit 701 continues the minute movement of the arm. The arm motion planning unit 701 obtains the arm minute movement target amount TA based on the arm motion plan (S406). The arm minute movement target amount corresponds to, for example, the trajectory of the arm 1001 determined based on the arm motion plan divided by a predetermined amount.

The arm motion planning unit 701 instructs the arm motion control unit 703 of the arm minute movement target amount TA, and the arm motion control unit 703 issues an arm motion instruction CONT according to the arm minute movement target amount TA to the robot 1 (S407). ). By these controls, the minute movement of the arm 1001 is executed (S408). In this embodiment, the arm operation instruction CONT is the hydraulic pressure supplied to each part of the arm 1001.

After executing the minute movement, the latest arm posture data AP reflecting the minute movement is input (S403), it is determined whether the posture of the arm 1001 can be grasped (S404), and the minute movement is continued.

On the other hand, when the arm is not recognized in the determination S404, that is, when an error signal is input, the arm motion planning unit 701 slightly moves the arm so as to return to the previous state (for example, the immediately preceding state). By setting the target amount TA, converting the arm minute movement target amount TA into a control variable (for example, hydraulic pressure), and instructing the arm operation, the arm 1001 returns to the previous state (S409).

By returning the arm 1001 to the state immediately before, it is possible to return to the state before losing the posture of the arm 1001, and stable control becomes possible. Further, other than returning to the state immediately before, the state before the predetermined step may be returned. Further, the basic posture of the arm 1001 (described above) may be restored.

When the posture of the arm 1001 is controlled by the pressure of the liquid as described above, in order to return to the immediately preceding state, for example, the difference in the liquid pressure or the liquid amount before and after the error signal is detected, and the difference is obtained. It can be controlled based on it. Further, in order to return to the basic posture, for example, it can be controlled by minimizing the hydraulic pressure or the liquid amount of all the cylinders of the joint driven by the hydraulic pressure. In this case, the arm 1001 takes, for example, the most contracted state. Further, in order to return to the basic posture, for example, it can be controlled by maximizing the hydraulic pressure or the liquid amount of all the cylinders of the joint driven by the hydraulic pressure. In this case, the arm 1001 takes, for example, the most extended state. Further, in order to return to the basic posture, for example, the hydraulic pressure or the liquid amount of one or more joint cylinders driven by hydraulic pressure is maximized, and the hydraulic pressure or liquid amount of one or more cylinders is minimized. It can be controlled by. In this case, the arm 1001 takes, for example, a bent state.

On the other hand, the arm minute movement target amount TA generated by the arm motion planning unit 701 is accumulated as the arm minute movement target amount history 705. When the arm 1001 is lost as a result of being controlled by a certain arm minute movement target amount TA, that is, when an error signal is received, the history is recorded in the arm minute movement target amount history 705 (S410).

If the arm is not recognized in the determination S404, the arm operation plan is corrected through the processes S409 and S410. That is, for example, in the case of returning to the immediately preceding state in the process 409, the operation plan for taking the final target posture FA set in the process S401 is recreated starting from the posture returned in the process 409. However, in this motion plan, a motion plan is created so as to avoid a state in which the arm 1001 is lost. For that purpose, the arm minute movement target amount history 705 is used to control so that the arm minute movement target amount TA immediately before the arm 1001 is lost is not used. Alternatively, control may be performed so as to avoid the posture of the immediately preceding arm posture data AP.

Alternatively, the arm 1001 can be controlled so as to avoid the area by recording the corresponding 3D coordinate 3D of the marker immediately before the reception of the error signal and mapping the outer edge of the area where the marker cannot be seen. ..

According to this embodiment, it is possible to grasp the posture of the arm without using a sensor such as an encoder that directly measures the angle of the joint, which is mounted on a conventional industrial robot. In addition, due to the influence of radiation and water vapor, it is possible to grasp the posture of the arm even in an environment where high-quality camera images cannot be expected to be acquired. Further, even in a situation where the robot or the arm may be lost due to an unknown surrounding structure or the like, it is possible to avoid interference with the surrounding structure and control the arm posture safely and surely.

In this embodiment, functions equivalent to the functions configured by software can also be realized by replacing them with hardware such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit).

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace the configurations of other examples with respect to a part of the configurations of each embodiment.

Robot 1, arm posture control device 4, arm posture estimation device 5, image analysis device 6, arm 1001, movement mechanism 1002, joint 1003

Claims (13)

An arm attitude control system for a work robot equipped with an arm having a movable part.
A camera image of the marker attached to the arm is used as an input.
An image analysis device that extracts the marker from the camera image and generates three-dimensional coordinates of the marker.
An arm posture estimation device that generates arm posture data indicating the posture of the arm based on the three-dimensional coordinates of the marker and the structural model of the arm.
An arm posture control device for controlling the posture of the arm based on the arm posture data is provided.
When the image analysis device cannot extract a predetermined number of the markers, when the image analysis device cannot extract the predetermined markers, and when the arm posture estimation device fails to extract the three-dimensional coordinates of the markers. When the arm posture data based on the structural model of the arm and the posture of the arm estimated from the past arm posture data are different from each other by a predetermined value or more, the arm posture data cannot be generated, a predetermined error indicating at least one of the above. When is detected
Whether to return to the posture of the arm before the detection of the predetermined error or the basic posture
or,
The situation in which a predetermined error is detected is memorized, and in the subsequent control of the posture of the arm, avoidance control is performed so as to avoid the situation.
Arm attitude control system for working robots. The arm attitude control device controls the hydraulic pressure in order to control the attitude of the arm.
When returning to the posture of the arm before the detection of the predetermined error,
The difference in hydraulic pressure before and after the predetermined error is detected, and the arm is controlled based on the difference.
The arm attitude control system for a working robot according to claim 1. The arm attitude control device controls the hydraulic pressure in order to control the attitude of the arm.
When returning to the basic posture,
Controlled by minimizing the liquid pressure involved in all of the plurality of drive positions in which the arm has,
The arm attitude control system for a working robot according to claim 1. The arm attitude control device controls the hydraulic pressure in order to control the attitude of the arm.
When returning to the basic posture,
Controlled by the maximum hydraulic pressure related to all of the plurality of drive positions in which the arm has,
The arm attitude control system for a working robot according to claim 1. The arm attitude control device controls the hydraulic pressure in order to control the attitude of the arm.
When returning to the basic posture,
Wherein the hydraulic pressure associated with at least one of the plurality of drive portions of the arm has a minimum, controlled by the maximum hydraulic pressure related to at least one of the plurality of drive positions in which the arm has,
The arm attitude control system for a working robot according to claim 1. The arm posture control device generates an motion plan from the current arm posture data and the final target posture, and when the current arm posture does not reach the final target posture, the arm posture control device is determined based on the motion plan. The arm minute movement target amount obtained by dividing the trajectory of the arm by a predetermined amount is obtained, and the minute movement is continued based on the arm minute movement target amount.
In the avoidance control,
Control is performed so as not to use the arm minute movement target amount immediately before the detection of the predetermined error.
The arm attitude control system for a working robot according to claim 1. In the avoidance control,
Control is performed so as to avoid the posture corresponding to the arm posture data immediately before the detection of the predetermined error.
The arm attitude control system for a working robot according to claim 1. In the avoidance control,
The three-dimensional coordinates of the marker corresponding to at least one immediately before and immediately after the detection of the predetermined error are mapped, an area to be avoided is defined, and control is performed so as to avoid the area.
The arm attitude control system for a working robot according to claim 1. This is an arm attitude control method for a work robot having an arm having a movable part.
An input step for inputting a camera image of a marker attached to the arm, and
An image analysis step of extracting the marker from the camera image and generating three-dimensional coordinates of the marker.
An arm posture estimation step that generates arm posture data indicating the posture of the arm based on the three-dimensional coordinates of the marker and the structural model of the arm.
Based on the arm posture data, the arm minute movement target amount is obtained, and based on the arm minute movement target amount, an arm posture control step for controlling the posture of the arm is executed.
When a predetermined number of the markers could not be extracted in the image analysis step, when a predetermined number of the markers could not be extracted in the image analysis step, and in the arm posture estimation step, the three-dimensional coordinates of the markers. And at least one of the cases where the arm posture data indicating the posture of the arm cannot be generated because the posture of the arm based on the structural model of the arm and the posture of the arm estimated from the past arm posture data are different from each other by a predetermined value or more. When the specified error shown is detected ,
Whether to perform return control to return to the arm posture controlled based on the arm posture data or the basic posture,
or,
The situation in which the predetermined error is detected is memorized, and in the subsequent control of the posture of the arm, avoidance control is performed so as to avoid the situation.
How to control the arm posture of a working robot. Control the hydraulic pressure to control the posture of the arm,
When returning to the arm posture controlled based on the arm posture data,
The difference in hydraulic pressure before and after the timing at which the arm posture data cannot be generated is detected, and the arm is controlled based on the difference.
The arm posture control method for a working robot according to claim 9. Control the hydraulic pressure to control the posture of the arm,
When returning to the basic posture,
Controlled by the hydraulic pressure to the plurality of drive positions in which the arm has a predetermined set value,
The arm posture control method for a working robot according to claim 9. Control the hydraulic pressure to control the posture of the arm,
When returning to the basic posture,
Controlled by minimizing the liquid pressure involved in all of the plurality of drive positions in which the arm has,
The arm posture control method for a working robot according to claim 11. Control the hydraulic pressure to control the posture of the arm,
When returning to the basic posture,
Controlled by the maximum hydraulic pressure related to all of the plurality of drive positions in which the arm has,
The arm posture control method for a working robot according to claim 11.

Images