TWI734867B - Teaching system and method for operation path of robot arm - Google Patents

Abstract

The invention is to disclose a teaching system and method for the operation path of a robot arm. A 3D camera and projector are disposed on the robot arm. A finger moving along the operation path is detected by the 3D camera, and the projector projects the detected operation path on a workpiece at real time. A preset page starts to set when the preset segment of the operation path is detected to be pressed. When amending, the pressed segment of the operation path is set to move following the finger to promote teaching efficiency.

Description

Teaching system and method of robot arm operating trajectory

The present invention relates to a robot arm, in particular to a teaching system and method for controlling the robot arm to detect the fingertips to form an operation track when the robot arm is programming an operation track.

With the rapid development of production technology, factory automation uses cameras to locate objects, and guides robotic arms to automatically grab objects for assembly and manufacturing, so as to improve factory production speed and efficiency. And simple and correct teaching of the robot arm's operating trajectory, so that the robot arm can quickly enter the ranks of production, has become the main topic of improving production efficiency.

In order to improve the teaching efficiency of the robotic arm, the prior art US20140201112 patent published in the United States provided buttons on the teaching tool, and used a wired line to connect the teaching tool to the control device of the robotic arm. The control signal was generated by pressing the button and transmitted via the wired line. To the control device, the camera on the arm of the robotic arm is controlled to shoot the hand-held teaching tool, and the captured image is transmitted to the control device for image processing, and the moving track of the teaching tool is positioned to generate the operation track. Due to the different sizes and shapes of the processed workpieces, the teaching tool is restricted by the winding and connection of the wired circuit in some directions, and it is difficult to place the teaching tool for teaching, which consumes too much teaching time. Although there is another patent of US5748854 published by the United States of the prior art, an infrared transmitter is installed on the teaching tool, and the communication receiver on the robot arm is connected wirelessly, and the wireless transmission teaching tool is used to press down. The signal of the button controls the camera of the robot arm to shoot the positioning mark of the teaching tool, which reduces the restriction of the wire line and increases the convenience of teaching the robot arm's trajectory.

However, the aforementioned prior art robotic arm still needs to be equipped with wireless transmission transmitters and receivers in the teaching tool and robotic arm, respectively, which not only increases the cost of parts and installation space of the robotic arm, but also increases the control complexity of the wireless transmission technology. . In addition, the prior art robot arm cannot immediately refer to the trajectory that has been taught during the teaching process, and it is difficult to maintain the consistency of the trajectory before and after the operation. At the same time, the robot arm of the prior art is also unable to modify the operation trajectory and set the operation parameters immediately during the teaching process. Therefore, there are still problems to be solved urgently for the robot arm in the teaching system and method of the operating trajectory.

The object of the present invention is to provide a robot arm operating trajectory teaching system. A depth camera and a projector are installed on the robot arm, and the depth camera is projected by the projector to detect the fingertip operating trajectory, so that the teaching process can be referenced to maintain the teaching consistency.

Another object of the present invention is to provide a robot arm operating trajectory teaching system that uses a projector to project a setting page, and uses a depth camera to detect the selected setting page, so that the teaching process can quickly set operating parameters to simplify the teaching operation.

Another object of the present invention is to provide a method for teaching a robot arm's operating trajectory, which uses a depth camera to detect a specific line segment of the projected operating trajectory of a pressed projector, activates the projector's projection setting page, and makes corrections or settings to improve the efficiency of the teaching operation.

Another object of the present invention is to provide a method for teaching the operation trajectory of a robot arm. When setting the correction operation trajectory, a depth camera is used to detect the line segment of the operation trajectory projected by the pressing projector, so that a preset part of the operation trajectory at the pressing position can be moved to facilitate the operation. Fix.

In order to achieve the purpose of the foregoing invention, the robot arm operating trajectory teaching system of the present invention includes one end of the robot arm as a fixed end, and the other end as a movable end end. The depth camera and projector are fixed on both sides of the end end, and the robot arm is connected to the control. Device, the control device controls the depth camera and the projector carried at the end of the robot arm, and controls the depth camera. The depth camera continuously detects the operator's fingertips moving on the workpiece. The expected operation trajectory is formed by the image processing of the control device. , The projector instantly projects the part of the work track to the workpiece to complete the work track, and uses the depth camera to detect the completed work track pressed by the fingertip, and then starts the projector to project the setting page around the work piece to perform the set work track as Taught work trajectory

In the teaching system of the robot arm of the present invention, the specific line segment of the start projection setting page is marked on the completed operation track, and the specific line segment is marked with a star mark or a different color. For the selection of the setting page, the control device controls the projector to project the adjustment page of the operation parameters or the edit adjustment page of the operation track. The adjustment page of the operating parameters displays the preset parameter unit quantities and the adjustment buttons for adjusting the parameters. The edit adjustment page displays correction and deletion options to edit the completed job track.

The method for teaching the operation trajectory of the robot arm of the present invention uses the robot arm to move the depth camera to shoot the workpiece and establish a 3D model of the workpiece. The operator’s fingertip is used to move the expected operation trajectory on the surface of the workpiece, and the depth camera continuously detects the movement of the fingertip. Position, forming part of the work trajectory, use the projector on the robot arm to project the part of the work trajectory to the surface of the workpiece in real time, the depth camera detects the movement of the fingertip to complete the work trajectory, and the completed work trajectory is projected by the projector to the surface of the workpiece , The depth camera detects the fingertip pressing the completed operation track, starts the projector to project the setting page, performs the setting operation of the completed operation track, checks that the completed operation track is satisfied, and records the completed operation track as the teaching operation track .

In the teaching method of the robot arm operating track of the present invention, the fingertip continues to move and the expected action is Work trajectory until the depth camera detects that the fingertip has left the workpiece and judges the completion of the work trajectory. Press the specific line segment marked by the completed task track with your fingertip to activate the projection setting page. When setting the setting page, use the depth camera to detect the fingertip and press the setting page to control the projector to project the preset parameter adjustment page for adjusting the operation parameters of the completed job track, or project the preset editing adjustment page for correction Or delete the completed job track.

In the teaching method of the robot arm operating trajectory of the present invention, when the completed operating trajectory is corrected, the depth camera detects that the fingertip is pressed on the part of the operating trajectory that needs to be corrected, and the preset length line segment of the pressed operating trajectory is marked as a movable condition. Set the length of the line segment and move it with the movement position of the fingertip. When the check is not satisfied with the completed job trajectory, press the delete option on the edit adjustment page to delete the completed job trajectory and teach the job trajectory again.

The present invention uses the depth camera to detect that the fingertip leaves the parameter adjustment page or edit adjustment page, and controls the projector to close the projection parameter adjustment page or edit the adjustment page. When the depth camera detects that the fingertip is pressed against the blank space around the workpiece, control the projector to close the projection setting page.

10‧‧‧Teaching system

11‧‧‧Robot arm

12‧‧‧Depth camera

13‧‧‧Projector

14‧‧‧Control device

15‧‧‧Tools

16‧‧‧Fixed end

17‧‧‧End

18‧‧‧Shaft section

19‧‧‧Actuating motor

20‧‧‧Workpiece

21‧‧‧Fingertips

22‧‧‧Operation trajectory

23‧‧‧Specific line segment

30‧‧‧Setting page

31‧‧‧Adjust the page

32‧‧‧Movement speed parameter adjustment page

33‧‧‧Edit adjustment page

34‧‧‧Line segment with preset length

Figure 1 is a schematic diagram of the teaching system of the robotic arm of the present invention.

Fig. 2 is a schematic diagram of the teaching operation trajectory of the robot arm of the present invention.

Figure 3 is a schematic diagram of the robot arm of the present invention completing the work trajectory.

Fig. 4 is a schematic diagram of the setting page of the work track of the present invention.

Fig. 5 is a schematic diagram of the modified operation trajectory of the present invention.

Fig. 6 is a flow chart of the teaching method of the robot arm operating trajectory according to the present invention.

With regard to the technical means adopted by the present invention in order to achieve the above-mentioned objects and their effects, preferred embodiments are described below in conjunction with the drawings.

Please refer to FIGS. 1 to 3 at the same time. FIG. 1 is a schematic diagram of the teaching system of the robotic arm of the present invention, FIG. 2 is a schematic diagram of the teaching operation trajectory of the robotic arm of the present invention, and FIG. In FIG. 1, the teaching system 10 of the present invention mainly includes a robotic arm 11, a depth camera 12, a projector 13, a control device 14 and a tool 15. The robot arm 11 can be in the form of multiple shafts, one end is a fixed fixed end 16, the other end is a movable end 17, and each shaft 18 between the fixed end 16 and the end 17 is provided with an actuating motor 19. The end portion 17 is provided with a tool 15, and the depth camera 12 and the projector 13 are fixed on both sides of the tool 15. The robotic arm 11 is further connected to the control device 14, and the control device 14 controls the actuating motor 19 that rotates each shaft section 18 to move the end portion 17 of the robotic arm 11, the carrying tool 15, the depth camera 12 and the projector 13 to move. The control device 14 controls the depth camera 12 to capture a depth image of the workpiece 20, and then transmits the image captured by the depth camera 12 to the control device 14 for image processing, and uses the projector 13 to project the image to the surface of the workpiece 20 and its vicinity.

The teaching system of the robot arm of the present invention takes teaching the operation trajectory of shoe-making glue as an example, but the present invention includes but is not limited to teaching shoe-making glue, and can also be used to teach the operation track of machining such as workpiece grinding. When teaching the operation trajectory in the present invention, firstly, the control device 14 is used to control the moving end 17 of the robot arm 11, carry the depth camera 12, and control the depth camera 12 to shoot the depth image of the workpiece 20 such as the shoe upper, and transmit the captured image to The control device 14 performs image processing to establish a 3D model of the workpiece 20. The teaching system of the robot arm of the present invention takes teaching the trajectory of shoe-making glue as an example, but the present invention includes and is not limited to teaching shoe-making glue, and can also be used in Teach the job trajectory of machining such as workpiece grinding. When teaching the operation trajectory in the present invention, firstly, the control device 14 is used to control the moving end 17 of the robot arm 11 to carry the depth camera 12, And control the depth camera 12 to take a depth image of the workpiece 20 such as a shoe upper, and transmit the taken image to the control device 14 for image processing, to establish a 3D model of the workpiece 20.

In FIG. 2, in the teaching system 10 of the present invention, the operator uses the fingertip 21 to move on the surface of the workpiece 20 along the expected work trajectory 22 to perform the teaching work trajectory 22. The moving end 17 of the robot arm 11 carries the depth camera 12 , Continuously tracking the moving position of the shooting fingertip 21, and transmitting the captured image to the control device 14. After image processing, the coordinates of the moving position of the fingertip 21 are calculated and recorded to form a part of the operation track 22, and the control device 14 then transmits a part of the operation track 22 to The projector 13 instantly projects a part of the work trajectory 22 onto the surface of the workpiece 20 by the projector 13. Therefore, the operator can instantly see part of the completed operation trajectory 22 during the process of teaching the operation trajectory 22, and can refer to it at any time during the subsequent teaching operation trajectory 22, for example, refer to the distance between the operation trajectory 22 and the edge to maintain consistency. In order to avoid uneven coating.

When the operator continues to move the fingertip 21 until the operation track 22 is completed, the operator will move the fingertip 21 away from the workpiece 20. Once the depth camera 12 detects that the fingertip 21 leaves the workpiece 20, it will determine that the operation track 22 is completed, as shown in Figure 3 , The control device 14 controls the projector 13 to project the completed work trajectory 22 onto the surface of the workpiece 20. The operator can press the projected work trajectory 22 to modify the work trajectory 22 and the setting of the work parameters, or mark a specific line 23 on the projected work trajectory 22 to prompt the operator to set or correct the work parameters Track 22. Although the specific line segment 23 is marked with an asterisk in this embodiment, it includes but is not limited to marking with a mark, and may also be marked with a different color or line type.

Please refer to FIG. 4 and FIG. 5 at the same time. FIG. 4 is a schematic diagram of the setting page of the work track of the present invention, and FIG. 5 is a schematic diagram of the correction work track of the present invention. In FIG. 4, when the projector 13 projects the completed work track 22 onto the surface of the workpiece 20, the operator uses the fingertip 21 to press on the work track 22 The line specific segment 23, the depth camera 12 continuously shoots images and transmits it to the control device 14. Once it detects that the fingertip 21 presses the line specific segment 23, the control device 14 will control the projector 13 to project the preset setting page 30 around the workpiece 20. The setting page 30 includes setting options such as the setting of operation parameters and the correction or deletion of the expected operation trajectory 22. When the operator uses the fingertip 21 to press, for example, the glue volume parameter setting A, once the depth camera 12 detects that the fingertip 21 is pressed on the setting A, the control device 14 will control the projector 13 to project the preset glue volume parameter adjustment page 31 , The glue amount parameter adjustment page 31 displays the preset glue amount in units, and the depth camera 12 detects the pressing of the fingertip 21 by pressing the up and down adjustment buttons to adjust the appropriate amount of glue. After the adjustment, the fingertip 21 leaves The glue amount parameter adjustment page 31 has a preset time, and the control device 14 will control the projector 13 to close the glue amount parameter adjustment page 31.

Similarly, when the operator uses the fingertip 21 to press, for example, the moving speed parameter setting B, once the depth camera 12 detects that the fingertip 21 is pressed on the setting B, the control device 14 will control the projector 13 to project the preset moving speed parameter The adjustment page 32, the movement speed parameter adjustment page 32 displays the preset movement speed, and the depth camera 12 detects the pressing of the fingertip 21 by pressing the up and down adjustment buttons to adjust the appropriate movement speed. After the adjustment, the fingertip 21 leaves The movement speed parameter adjustment page 32 has a preset time, and the control device 14 will control the projector 13 to close the movement speed parameter adjustment page 32.

In addition, when the operator thinks that the job trajectory 22 is not perfect enough and needs to be edited again, press the setting C of the editing function with the fingertip 21, once the depth camera 12 detects that the fingertip 21 is pressed on the setting C, the control device 14 will control the projection The machine 13 projects the preset edit adjustment page 33, the edit adjustment page 33 displays correction and deletion options, when the delete option is pressed, the depth camera 12 detects the fingertip 21 to press the deletion, and the control device 14 deletes the completed operation track 22. And control the projector 13 to close the projection of the edit adjustment page 33, the setting page 30, and the work trajectory 22, and teach the work trajectory 22 again.

When the correction option is pressed, the depth camera 12 detects the pressing correction of the fingertip 21, and the control device 14 controls the projector 13 to project the completed work track 22. In FIG. 5, the operator uses the fingertip 21 to press the part of the completed work track 22 that needs to be corrected. Once the depth camera 12 detects that the fingertip 21 is pressed against the work track 22, the control device 14 presses the work track 22 adjacent to the preset length The line segment 34 is set to indicate the movable condition, and according to the depth camera 12 by detecting the moving position of the fingertip 21, the projector 13 is controlled to project the continuous operation track 22 to move the line segment 34 of a preset length to the moving position of the fingertip 21. After the adjustment, the fingertip 21 is away from the preset length line segment 34 for a preset time, and the control device 14 will control the projector 13 to close the edit adjustment page 33. If there is no need to adjust and correct, just use the fingertip 21 to press the blank area around the workpiece 20. Once the depth camera 12 detects that the fingertip 21 is pressed against the blank area, it will control the projector 13 to close the setting page 30 and record the adjustment and correction. The work trajectory 22 is a teaching work trajectory 22 to complete the teaching robot arm 11.

As shown in Fig. 6, it is a flow chart of the teaching method of the robot arm operating trajectory of the present invention. The detailed steps of the teaching method of the robot arm operating trajectory of the present invention are described as follows: first, step S1, start the teaching operation of the robot arm; in step S2, the robot arm moves the depth camera to shoot the workpiece, and builds a 3D model of the workpiece; step S3, use the finger The tip moves the expected operation trajectory on the surface of the 3D model of the workpiece; in step S4, the depth camera continuously detects the position of the fingertip movement to form part of the operation trajectory; step S5, use the projector on the robotic arm to project part of the operation in real time Trajectory to the surface of the work piece; step S6, move the fingertip to complete the work trajectory according to the projected part of the work trajectory, and project the completed work trajectory onto the surface of the work piece; step S7, press the specific line segment of the completed work trajectory; step S8, from the depth The camera detects that the fingertip presses a specific line segment to start the projector projection setting page; step S9, the depth camera detects the fingertip and selects to press the setting page to correct the trajectory or set the operation parameters; step S10, check whether the completed operation trajectory needs to be deleted ? like If the job trajectory is unsatisfactory, return to step S3, repeat the previous steps, and re-teach the job trajectory. If the job trajectory is satisfied, go to step S11, record the completed job trajectory as the taught job trajectory, and complete the teaching robot arm.

Therefore, the robot arm operating trajectory teaching system and method of the present invention use a depth camera and a projector on the robot arm, and the depth camera can be projected by the projector to detect the expected operating trajectory of the fingertip, so that the teaching process can be used as a reference. To achieve the purpose of maintaining the consistency of teaching. The present invention uses fingertips instead of tools, and the projector projects the setting page, and the depth camera detects the selected setting page, so that the teaching process can quickly set the operating parameters, so as to achieve the purpose of simplifying the teaching operation. In addition, the teaching method of the robot arm operating trajectory of the present invention uses a depth camera to detect the specific line segment of the projected operating trajectory by pressing the projector, activates the projector to project the setting page, corrects or sets the operating trajectory, and uses it when setting the corrected operating trajectory The depth camera detects and presses the projector to project the job trajectory, so that the job trajectory can move with the fingertip by pressing the preset length line segment to facilitate correction and achieve the purpose of improving the teaching efficiency.

The above are only for the convenience of describing the preferred embodiments of the present invention. The scope of the present invention is not limited to these preferred embodiments. Any changes made in accordance with the present invention will not depart from the spirit of the present invention. , All belong to the scope of the patent application of the present invention.

11‧‧‧Robot arm

12‧‧‧Depth camera

13‧‧‧Projector

14‧‧‧Control device

20‧‧‧Workpiece

21‧‧‧Fingertips

22‧‧‧Operation trajectory

23‧‧‧Specific line segment

30‧‧‧Setting page

31‧‧‧Adjust the page

32‧‧‧Movement speed parameter adjustment page

33‧‧‧Edit adjustment page

Claims (13)

A robot arm operating trajectory teaching system, including: a robot arm, one end is a fixed end, the other end is a movable end, and the depth camera and projector are fixed on both sides of the end; a control device, connected to the robot arm, controls the movement of the robot arm The end part carries a depth camera and a projector, and controls the depth camera to shoot a depth image. After image processing, the projector is used to project the image to the workpiece; where the depth camera continuously detects the operator’s fingertip moving on the workpiece and the expected operation trajectory. The image processing of the control device successively forms part of the operation trajectory, and the projector instantly projects the part of the operation trajectory to the workpiece to complete the operation trajectory, and uses the depth camera to detect the completed operation trajectory by pressing the fingertip, and then starts the projector to project the setting page. Set the completed work trajectory as the teaching trajectory. The robot arm operating trajectory teaching system as described in item 1 of the scope of patent application, wherein the completed operating trajectory indicates a specific line segment of the start projection setting page. As described in item 2 of the scope of patent application, the robot arm operating trajectory teaching system, wherein the specific line segment is marked with a mark or a different color. The robot arm operating trajectory teaching system as described in the first item of the scope of patent application, wherein the projector projects a preset setting page around the workpiece. For example, the robot arm operating trajectory teaching system described in item 4 of the scope of patent application, wherein the setting page is selected by the control device to control the projector to project the adjustment page of the operating parameters or the editing adjustment page of the operating trajectory, and the adjustment page of the operating parameters Displays the preset parameter unit amount and the adjustment button for adjusting the parameters, and the edit adjustment page displays the correction and deletion options to edit the completed operation track. A method for teaching the operating trajectory of a robotic arm, the steps include: the robotic arm moves a depth camera to shoot the workpiece to establish a 3D model of the workpiece; the operator's fingertip is used to move the expected operating trajectory on the surface of the workpiece; the depth camera continuously detects the movement of the fingertip The position of the robot arm forms part of the work trajectory; the projector on the robot arm is used to project the part of the work trajectory to the surface of the workpiece in real time; the depth camera detects the movement of the fingertip to complete the work trajectory, and the completed work trajectory is projected by the projector to the workpiece Surface; the depth camera detects the fingertip pressing the completed operation track, starts the projector to project the setting page, and performs the setting operation of the completed operation track; checks that the completed operation track is satisfied, and records the completed operation track as a teaching operation Trajectory. The method for teaching the robot arm's operating trajectory as described in item 6 of the scope of patent application, wherein the fingertip continues to move the expected operating trajectory until the depth camera detects that the fingertip leaves the workpiece and judges the completion of the operating trajectory. As described in item 6 of the scope of patent application, the robot arm's operation trajectory teaching method, wherein the fingertip presses a specific line segment marked by the completed operation trajectory to activate the projection setting page. For example, the method for teaching the robot's operating trajectory as described in item 6 of the scope of patent application, in which the depth camera is used to detect the fingertip and press the setting page to control the projector to project the preset parameter adjustment page for adjustment when the setting page is set. The job parameters of the completed job trajectory, or the preset editing adjustment page is projected for correction or deletion of the completed job trajectory. For example, in the teaching method of the robot arm's operation trajectory described in item 9 of the scope of patent application, when the completed operation trajectory is corrected, the depth camera detects that the fingertip is pressed on the part of the operation trajectory that needs to be corrected. The preset length line segment of the pressing operation track is marked as a movable condition, and the preset length line segment moves with the movement position of the detection fingertip. For example, the method for teaching the robot arm's operation trajectory as described in item 9 of the scope of patent application, in which it is checked that the completed operation trajectory is unsatisfactory, press the delete option on the edit adjustment page, delete the completed operation trajectory, and teach the operation trajectory again. For example, in the method for teaching the operating trajectory of a robotic arm described in the scope of patent application, the depth camera detects that the fingertip leaves the parameter adjustment page or edit adjustment page, and controls the projector to close the projection parameter adjustment page or edit adjustment page. According to the method for teaching the operating trajectory of the robotic arm described in the scope of patent application, the depth camera detects that the fingertip is pressed against the blank space around the workpiece, and controls the projector to close the projection setting page.

Images