JP2021003782A - Object recognition processing device, object recognition processing method and picking apparatus - Google Patents

Abstract

To provide an object recognition processing device, an object recognition processing method and picking apparatus which can suppress the reduction in the recognition processing speed while increasing the recognition accuracy.SOLUTION: An object recognition processing device comprises: a storage unit which stores object information of an identification object; an image generation unit which generates a simulation image in which the plurality of identification objects are arranged; a recognition accuracy calculation unit which performs image recognition by comparing the object information with the identification object in the simulation image generated by the image generation unit and obtains the recognition accuracy of the image recognition for each posture of the identification object; a user interface unit which notifies a user of the recognition accuracy and receives permission/rejection of the recognition accuracy from the user for each posture of the identification object; and a discriminator generation unit which generates a discriminator for each posture of the identification object that is rejected when the user interface unit receives the rejection.SELECTED DRAWING: Figure 2

Description

The present invention relates to an object recognition processing device, an object recognition processing method, and a picking device.

The simulation device described in Patent Document 1 includes a display device that displays a three-dimensional virtual space on a screen, an imaging range specified by an operator, optical feature information of an imaging camera used, and required measurement. A camera position determining unit that determines the installation position of the imaging camera based on accuracy, and a virtual image to be acquired by the imaging camera based on the position of the imaging camera in a three-dimensional virtual space and its optical feature information are generated. Equipped with a virtual image generation unit, it is possible to easily determine an appropriate position of the imaging camera and adjust detection parameters.

JP-A-2008-21092

However, with such a simulation device, recognition processing with excessive accuracy is performed, and it is difficult to improve the processing speed.

The object recognition processing device of the present invention includes a storage unit that stores object information to be identified and a storage unit.
An image generation unit that generates a simulation image in which a plurality of identification targets are arranged,
A recognition accuracy calculation unit that performs image recognition by comparing the identification target in the simulation image generated by the image generation unit with the object information, and obtains the recognition accuracy of the image recognition for each posture of the identification target. When,
A user interface unit that notifies the user of the recognition accuracy and accepts whether or not the recognition accuracy is allowed from the user for each posture of the identification target.
It is characterized by having a classifier generation unit that generates a classifier for each posture of the discriminating target that is rejected when the user interface unit accepts the rejection.

It is a figure which shows the whole structure of the picking apparatus which concerns on a preferable embodiment of this invention. It is a block diagram which shows the structure of the object recognition processing apparatus. It is a flowchart which shows the method of teaching work. It is a figure which shows an example of the simulation image P1. It is a figure which shows an example of the screen displayed on a monitor. It is a figure which shows an example of the screen displayed on a monitor. It is a flowchart which shows the picking method of a picking apparatus.

Hereinafter, the object recognition processing device, the object recognition processing method, and the picking device of the present invention will be described in detail based on the embodiments shown in the accompanying drawings.

FIG. 1 is a diagram showing an overall configuration of a picking device according to a preferred embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of an object recognition processing device. FIG. 3 is a flowchart showing a method of teaching work. FIG. 4 is a diagram showing an example of the simulation image P1. 5 and 6 are diagrams showing an example of a screen displayed on the monitor, respectively. FIG. 7 is a flowchart showing a picking method of the picking device.

The picking device 1 shown in FIG. 1 includes a camera 200 that images a plurality of objects X as identification targets that are irregularly or randomly arranged on a mounting table 170, and an object that performs object recognition processing based on the imaging results of the camera 200. It has a recognition processing device 300 and a robot 100 including a manipulator 151 that picks one or more objects X from a mounting table based on the object recognition processing result of the object recognition processing device 300.

First, the robot 100 will be briefly described. As shown in FIG. 1, the robot 100 is a horizontal articulated robot, that is, a SCARA robot, and is used in each work such as holding, transporting, assembling, and inspecting a work such as an electronic component. The use of the robot 100 is not particularly limited.

The robot 100 has a base 110 and an arm 120 connected to the base 110. Further, the arm 120 has a first arm 121 whose base end is connected to the base 110 and is rotatable around the first axis J1 with respect to the base 110, and a base end is the tip of the first arm 121. It has a second arm 122 which is connected to the first arm 121 and is rotatable around a second axis J2 which is parallel to the first axis J1. A work head 130 is provided at the tip of the second arm 122.

The base 110 is fixed to, for example, a floor surface (not shown) with bolts or the like. Further, a drive device 141 for rotating the first arm 121 around the first axis J1 with respect to the base 110 is provided in the base 110, and with respect to the first arm 121 in the second arm 122. A drive device 142 for rotating the second arm 122 around the second axis J2 is provided. The drive devices 141 and 142 include a motor M as a drive source, a controller C for controlling the drive of the motor M, an encoder E for detecting the amount of rotation of the motor M, and the like, respectively.

The work head 130 has a spline nut 131 and a ball screw nut 132 coaxially arranged at the tip of the second arm 122, and a spline shaft 133 inserted through the spline nut 131 and the ball screw nut 132. The spline shaft 133 is the central axis of the second arm 122, is rotatable around the third axis J3 parallel to the first and second axes J1 and J2, and is along the third axis J3. It can be raised and lowered in the vertical direction.

In the second arm 122, a drive device 143 that rotates the spline nut 131 to rotate the spline shaft 133 around the third axis J3, and a direction in which the ball screw nut 132 is rotated to rotate the spline shaft 133 along the third axis J3. There is a drive device 144 that moves up and down. The drive devices 143 and 144 include a motor M as a drive source, a controller C for controlling the drive of the motor M, an encoder E for detecting the amount of rotation of the motor M, and the like, respectively.

A payload 150 for mounting an end effector is provided at the lower end of the spline shaft 133. The end effector attached to the payload 150 is not particularly limited, but in the present embodiment, a manipulator 151 for picking or gripping the object X is used. The picking method is not particularly limited, and the picking method may be sandwiched between a plurality of claws, or may be attracted by an air chuck, an electrostatic chuck, or the like.

Further, in the base 110, a robot control device 160 that controls the drive of the drive devices 141, 142, 143, and 144 based on a command from the object recognition processing device 300 is provided. The robot control device 160 includes, for example, a processor (CPU) composed of a computer and processing information, a memory communicatively connected to the processor, and an external interface. In addition, various programs that can be executed by the processor are stored in the memory, and the processor can read and execute various programs and the like stored in the memory.

The overall configuration of the robot 100 has been briefly described above. However, the configuration of the robot 100 is not particularly limited. For example, the arm 120 may be configured such that the first arm 121 is omitted and the second arm 122 is connected to the base 110. Between the first arm 121 and the second arm 122, at least one arm that can rotate around an axis parallel to the first and second axes J1 and J2 may be further interposed. Further, instead of the horizontal articulated robot, an articulated robot such as a 6-axis robot or a dual-arm robot in which the rotation axes of a plurality of arms are twisted may be used.

Next, the camera 200 will be described. The camera 200 has a function of capturing a plurality of objects X arranged on the mounting table 170 and obtaining an image including the objects X. Further, the camera 200 has a fixed relative positional relationship with the mounting table 170. As a result, the object recognition process of the object X can be easily performed from the image captured by the camera 200. The camera 200 is not particularly limited, but in the present embodiment, an RGB camera is used. As the camera 200, for example, a grayscale camera, an infrared camera, or the like can be used in addition to the RGB camera. Further, instead of the camera 200, for example, a depth sensor that can acquire point cloud data of the object X may be used.

Next, the object recognition processing device 300 will be described. The object recognition processing device 300 is connected to the robot control device 160, the camera 200, and the keyboard K. Such an object recognition processing device 300 includes, for example, a processor (CPU) composed of a computer and processing information, a memory communicably connected to the processor, and an external interface. In addition, various programs that can be executed by the processor are stored in the memory, and the processor can read and execute various programs and the like stored in the memory.

As shown in FIG. 2, the object recognition processing device 300 includes a simulation unit 310, a classifier generation unit 320, a user interface unit 330, an image acquisition unit 340, a recognition unit 350, a learning unit 360, and a processing time. It has an estimation unit 370, a communication unit 380, and a control unit 390 that controls each of these units. Of these units, the simulation unit 310, the classifier generator 320, the user interface unit 330, the learning unit 360, and the processing time estimation unit 370 perform the recognition processing teaching work, and the image acquisition unit 340 and the image acquisition unit 340 The recognition unit 350 and the learning unit 360 perform recognition processing of the actual object X.

First, the teaching work of the recognition process will be described. As shown in FIG. 3, the teaching work is performed in step S11 to learn the information of the object X and obtain the object information D1, step S12 to simulate the recognition process of the object X using the basic recognition algorithm, and step S12. Step S13 to generate an additional discriminator for improving the recognition accuracy based on the result of the simulation, and step S14 to obtain the recognition processing time required when the discriminator generated in step S13 is added. Has.

[Step S11]
The learning unit 360 creates object information D1 using CAD data of the object X to be recognized, that is, data created by using CAD (computer-aided Design), and learns to use it for recognition processing of the object X. Get results. By creating the object information D1 from the CAD data, the object information D1 becomes more accurate. Examples of the object information D1 include templates of the object X viewed from various angles of 360 degrees. The learning unit 360 has a storage unit 361 that stores information, and stores the learning result in the storage unit 361. The number of templates used for the recognition processing of the object X is not particularly limited, and the larger the number, the better the recognition processing accuracy, but the lower the recognition processing speed. Therefore, it can be appropriately set according to the balance between the recognition processing accuracy and the recognition processing speed.

[Step S12]
The simulation unit 310 simulates the recognition process of the object X by using the object information D1 generated by the learning unit 360. The simulation unit 310 includes an image generation unit 311 and a recognition accuracy calculation unit 312.

The image generation unit 311 virtually prepares the same mounting table as the mounting table 170 in the real world on a computer, and randomly arranges a plurality of objects X on the mounting table using the CAD data of the object X. Then, as shown in FIG. 4, the image generation unit 311 generates an image of the object X on the mounting table captured by the same virtual camera as the camera 200 in the real world as the simulation image P1. As a result, an image similar to the image captured by the camera 200 in the real world can be generated on the computer as the simulation image P1. The image generation unit 311 stores the position and orientation of each object X used to generate the simulation image P1.

It is preferable that the virtual camera on the computer has the same specifications as the camera 200 in the real world, and the relative positional relationship with the mounting table is also the same as in the real world. The above-mentioned "specifications" include, for example, the focal length of the lens, the F value, the resolution of the image sensor, ISO (sensitivity), noise, and the like. As a result, it is possible to generate a simulation image P1 that is closer to the real world.

Further, it is preferable that the image generation unit 311 uses a real-world environment when generating the simulation image P1. That is, it is preferable that the image generation unit 311 reproduces information on the environment in which the object X is placed in the real world, particularly the number, position, amount of light, and color of the light source to generate the simulation image P1. As a result, a shadow, that is, a shadow can be projected on the object X in the simulation image P1, so that the simulation image P1 closer to the real world can be generated.

The recognition accuracy calculation unit 312 recognizes the object X by comparing the object X in the simulation image P1 generated by the image generation unit 311 with the object information D1 stored in the storage unit 361 using a basic recognition algorithm. The processing is performed, and the recognition accuracy is obtained for each posture of the object X. According to such a method, the recognition accuracy can be obtained more easily. The method of recognition processing is not particularly limited, and for example, the following method can be used.

First, as shown in FIG. 4, the recognition accuracy calculation unit 312 finds the object X by setting a region S having a window size in which the object X is sufficiently accommodated and moving this region S with the simulation image P1. Next, the recognition accuracy calculation unit 312 acquires an image of the object X and performs recognition processing using the basic recognition algorithm to store the position and orientation of the object X reflected in the image in the storage unit 361. It is estimated from the object information D1 stored in. The estimation method is not particularly limited, and for example, the recognition accuracy calculation unit 312 can estimate the position and orientation of the object X reflected in the image by template matching using the object information D1. Next, the recognition accuracy calculation unit 312 determines whether the estimation result of the posture of the object X is a correct answer or an incorrect answer. As described above, the posture of each object X in the simulation image P1 is stored in the image generation unit 311. Therefore, it is possible to determine whether the estimation result is correct or incorrect based on this memorized posture.

The recognition accuracy calculation unit 312 performs the above-mentioned work in order for all the objects X shown in the simulation image P1. When the recognition accuracy calculation unit 312 finishes the above-mentioned work for all the objects X, the image generation unit 311 generates a new simulation image P1 in which the arrangement of the objects X is different from that of the simulation image P1. The new simulation image P1 can be generated as an image in which a plurality of objects X arranged to generate the previous simulation image P1 are randomly stirred.

The recognition accuracy calculation unit 312 estimates the postures of all the objects X for the newly generated simulation image P1 as in the previous time, and determines whether the estimation result is a correct answer or an incorrect answer. In this way, the simulation unit 310 repeats the generation of the simulation image P1 by the image generation unit 311 and the estimation of the posture of the object X by the recognition accuracy calculation unit 312 a sufficient number of times. As a result, it is possible to learn a large amount of estimation results of the posture of the object X and its correctness. The image generation unit 311 arranges objects X in various postures in a well-balanced manner in a large number of simulation images P1 to be generated so that the recognition accuracy calculation unit 312 can determine the correctness of the estimation result in a well-balanced manner for various postures. It is preferable to do so.

Next, the recognition accuracy calculation unit 312 obtains the recognition accuracy for each posture of the object X based on the learned information. Specifically, for each posture of the learned object X, the correct answer rate and the incorrect answer rate of the estimation result are obtained, and further, in the case of an incorrect answer, which other posture is mistaken for is obtained together with the probability. That is, information is obtained such that the correct answer rate is x%, the wrong postures are the A posture and the B posture, the misrecognition probability of the A posture is y%, and the misrecognition probability of the B posture is z%.

The user interface unit 330 notifies the user of a posture whose recognition accuracy is lower than a predetermined probability among the postures obtained by the recognition accuracy calculation unit 312. In the present embodiment, as shown in FIGS. 1 and 5, the user is notified by displaying the display on the monitor 400 connected to the object recognition processing device 300. However, the notification method is not particularly limited. Further, when the posture with low recognition accuracy does not exist, the user interface unit 330 notifies that fact. The monitor 400 may be built in the object recognition processing device 300.

In the example shown in FIG. 5, two postures whose recognition accuracy is lower than a predetermined probability are displayed. Further, the correct answer rate, the erroneously recognized posture, and the erroneously recognized posture are displayed for each posture. Further, the user interface unit 330 has a reception unit 331 that accepts from the user whether or not to allow such misrecognition. In the illustrated configuration, the receiving unit 331 is displayed on the monitor 400 and has a check column for allowing or not allowing erroneous recognition. The user determines whether or not to allow misrecognition for each posture, and if misrecognition is tolerated, uses an operating device such as a keyboard K or a mouse to check the check box corresponding to that posture. .. Then, the user interface unit 330 receives the signal corresponding to this selection.

For example, No. In the posture of 1, the erroneously recognized posture is the posture upside down, and if the robot 100 picks up the posture upside down, the subsequent work may be hindered. Therefore, No. The misrecognition of the posture of 1 cannot be tolerated, and the check box is not checked. On the other hand, No. In the posture of 2, the misrecognized posture is shifted around the central axis, and even if the robot 100 picks up the posture in the shifted posture, there is almost no hindrance to the subsequent work. Therefore, No. Misrecognition of posture 2 can be tolerated, and the check box is checked. In this way, it is possible to determine whether or not to allow erroneous recognition for each posture in consideration of individual circumstances such as whether or not the subsequent work is hindered.

[Step S13]
The classifier generation unit 320 improves the recognition accuracy for each posture selected as "not tolerating erroneous recognition" when the result received by the user interface unit 330 includes "not tolerating erroneous recognition". Generate the classifiers individually. The classifier consists of an algorithm for improving recognition accuracy, and this algorithm is generated using, for example, a neural network. As a result, the classifier can be easily generated. As an algorithm, it is particularly preferable to use a classifier based on a convolutional neural network (CNN) among neural networks.

The classifier can be generated, for example, by learning a large amount of images for which image recognition is correct and images for which image recognition is incorrect for a posture selected as "not tolerating erroneous recognition". According to such a classifier, instead of finding one correct posture from all the postures of the object X stored in the object information D1, the correct answer is obtained from some postures that are likely to be incorrect. Since one posture is classified, the time required for the processing can be significantly reduced. The classifier is not particularly limited, and may be, for example, an image recognition algorithm using a support vector machine or an image recognition algorithm using AdaBoost.

As described above, by adding a discriminator that enhances the recognition accuracy only to a specific posture with low recognition accuracy, only the processing speed for the posture to which the discriminator is added becomes longer than the basic recognition algorithm, and other The processing speed for the pose, that is, the pose for which no classifier is added, is the same as the basic recognition algorithm. Therefore, while improving the recognition accuracy of the object X, the decrease in processing speed can be suppressed to be smaller than, for example, as compared with the configuration in which the recognition accuracy must be improved for all postures. Therefore, the object recognition processing device 300 can perform the recognition processing of the object X more accurately in a shorter time.

[Step S14]
The processing time estimation unit 370 does not add the classifier generated by the classifier generation unit 320, and performs the recognition process of the object X using only the basic recognition algorithm. The average processing time per object X is T1 [sec]. / Object X] and the classifier generated by the classifier generator 320 are added, and the average processing time T2 [sec / object X] per object X required when recognition by the classifier is further performed after the basic recognition algorithm. And ask. The average processing time T2 becomes longer than the average processing time T1 due to the addition of recognition by the classifier.

The user interface unit 330 notifies the user of the average processing times T1 and T2 obtained by the processing time estimation unit 370. In the present embodiment, as shown in FIG. 6, the user is notified by displaying the average processing times T1 and T2 side by side on the monitor 400. By displaying the average processing times T1 and T2 side by side, the delay time due to the addition of the classifier becomes easy to understand. In addition, the user can make a final judgment as to whether or not the recognition accuracy should be improved based on the average processing times T1 and T2 displayed on the monitor 400. That is, it is possible to select whether the recognition accuracy should be increased even if the processing time becomes long, or conversely, whether the processing time should be shortened even if the processing time is given up.

As shown in FIG. 6, the user interface unit 330 has a reception unit 332 that accepts whether or not to add a classifier. The reception unit 332 is displayed on the monitor 400 together with the average processing times T1 and T2, and in the illustrated configuration, the user can select "add" or "do not add". When "add" is accepted, the high-precision recognition algorithm with a classifier added to the basic recognition algorithm is used as the actual recognition algorithm, and when "do not add" is accepted, the basic recognition algorithm is used as the actual recognition algorithm. Use. When "Do not add" is accepted, the user interface unit 330 may return to the screen shown in FIG. 5 and again allow the user to select whether to allow the erroneous recognition or not. Then, the user interface unit 330 receives the signal corresponding to this selection.

The teaching work of image recognition has been described above. Next, the picking operation of the object X by the robot 100 will be described based on the flowchart shown in FIG.

First, in step S21, the camera 200 images the object X, and the image acquisition unit 340 acquires the image data P2 obtained by the imaging. Next, in step S22, the recognition unit 350 sets a region S having a window size in which the object X is sufficiently accommodated, and sets this region S as the initial position on the image data P2.

Next, in step S23, the recognition unit 350 determines whether or not the object X exists in the area S. When the object X is found in step S23, as step S24, the recognition unit 350 estimates the position and orientation of the object X using the basic recognition algorithm. Next, as step S25, the recognition unit 350 takes a posture in which the estimated posture requires additional recognition by the above-mentioned classifier, that is, a posture in which the user determines in the above-mentioned teaching work that "misrecognition is not allowed". Determine if it exists. When additional recognition by the classifier is required, in step S26, the recognition unit 350 performs additional recognition by the classifier and estimates the posture of the object X more accurately.

After the completion of step S26, or when the object X is not found in step S23, or when additional recognition by the classifier is not required in step S25, the recognition unit 350 performs the recognition unit 350 on the image data P2 as step S27. The position of the area S of is changed. Next, in step S28, the recognition unit 350 determines whether the entire area of the image data P2 has been scanned by the area S. If the scanning of the entire area of the image data P2 has not been completed, the process returns to step S23 and repeats steps S23 to S28.

On the contrary, when the scanning of the entire area of the image data P2 is completed, as step S29, the recognition unit 350 determines one object X to be picked by the robot 100 from the found objects X. Then, in step S30, the communication unit 380 transmits a command for picking the selected object X to the robot 100, and executes the picking operation of the object X by the robot 100. When the picking work of the object X by the robot 100 is completed, the process returns to step S21, and S21 to S30 are repeated until the picking work of the object X is completed.

The picking device 1 has been described above. Such a picking device 1 includes an object recognition processing device 300 that recognizes an object X to be identified as an image, and a manipulator 151 that picks the object X based on the recognition result of the object recognition processing device 300. Further, the object recognition processing device 300 includes a storage unit 361 in which the object information D1 of the object X is stored, an image generation unit 311 that generates a simulation image P1 in which a plurality of objects X are arranged, and an image generation unit 311. The recognition accuracy calculation unit 312 that performs image recognition by comparing the object X in the generated simulation image P1 with the object information D1 and obtains the recognition accuracy of the image recognition for each posture of the object X, and the recognition accuracy to the user. The user interface unit 330 that notifies and accepts whether or not the recognition accuracy is allowed from the user for each posture of the object X, and when the user interface unit 330 accepts the rejection, that is, when "the recognition accuracy is not allowed", no It has a classifier generator 320 that generates a classifier for each posture of the object X.

According to such a configuration, it is possible to add a discriminator that enhances the recognition accuracy only in a specific posture having a low recognition accuracy. Therefore, only the processing speed for the posture to which the classifier is added is longer than that for the basic recognition algorithm, and the processing speed for the other postures, that is, the postures to which the classifier is not added is the same as the basic recognition algorithm. Therefore, while improving the recognition accuracy of the object X, the decrease in processing speed can be suppressed to be smaller than, for example, as compared with the configuration in which the recognition accuracy must be improved for all postures. As a result, the picking device 1 is capable of performing the recognition process of the object X more accurately in a shorter time.

Further, as described above, the object recognition processing device 300 generates an image generation unit 361 in which the object information D1 of the object X to be identified is stored and a simulation image P1 in which a plurality of objects X are arranged. Image recognition is performed by comparing the object X in the simulation image P1 generated by the image generation unit 311 with the object information D1, and the recognition accuracy calculation for obtaining the recognition accuracy of the image recognition for each posture of the object X. The unit 312, the user interface unit 330 that notifies the user of the recognition accuracy and accepts whether or not the recognition accuracy is allowed from the user for each posture of the object X, and the case where the user interface unit 330 accepts the rejection, that is, "recognition accuracy". In the case of "not allowed", it has a discriminator generation unit 320 that generates a discriminator for each posture of the object X that is rejected.

According to such a configuration, it is possible to add a discriminator that enhances the recognition accuracy only in a specific posture having a low recognition accuracy. Therefore, only the processing speed for the posture to which the classifier is added is longer than that for the basic recognition algorithm, and the processing speed for the other postures, that is, the postures to which the classifier is not added is the same as the basic recognition algorithm. Therefore, while improving the recognition accuracy of the object X, the decrease in processing speed can be suppressed to be smaller than, for example, as compared with the configuration in which the recognition accuracy must be improved for all postures. As a result, the object recognition processing device 300 can perform the recognition processing of the object X more accurately in a shorter time.

Further, as described above, the object recognition processing device 300 has a processing time estimation unit 370 that estimates the image processing time when the image processing is performed by adding the classifier generated by the classifier generation unit 320. The user interface unit 330 notifies the average processing time T2, which is the image processing time estimated by the processing time estimation unit 370. This allows the user to determine whether or not the recognition accuracy should be improved for a particular posture. That is, it is possible to select whether the recognition accuracy should be increased even if the processing time becomes long, or conversely, whether the processing time should be shortened even if the processing time is given up.

Further, as described above, the object information D1 is generated from the CAD data. As a result, the object information D1 becomes more accurate.

Further, as described above, the image generation unit 311 generates the simulation image P1 by using the information of the environment in which the object X exists. As a result, it is possible to generate a simulation image P1 that is close to the real world.

Further, as described above, the recognition accuracy calculation unit 312 obtains the recognition accuracy by collating the information of the object X acquired from the simulation image P1 with the object information D1 stored in the storage unit 361. As a result, the recognition accuracy can be obtained more easily.

Further, as described above, the discriminator generation unit 320 generates a discriminator using a neural network. As a result, the classifier can be easily generated.

Further, as described above, the object recognition processing method includes a step of storing the object information D1 of the object X to be identified, a step of generating a simulation image P1 in which a plurality of objects X are arranged, and an image generation unit 311. Image recognition is performed by comparing the object X in the simulation image P1 generated by the object X with the object information D1, and the step of obtaining the recognition accuracy of the image recognition for each posture of the object X and the recognition accuracy are notified to the user. The step of accepting whether or not the recognition accuracy is allowed from the user for each posture of the object X, and the case where the user interface unit 330 accepts the rejection, that is, when "the recognition accuracy is not allowed", the object X is rejected. It has a step of generating a classifier for each posture.

According to such a configuration, it is possible to add a discriminator that enhances the recognition accuracy only in a specific posture having a low recognition accuracy. Therefore, only the processing speed for the posture to which the classifier is added is longer than that for the basic recognition algorithm, and the processing speed for the other postures, that is, the postures to which the classifier is not added is the same as the basic recognition algorithm. Therefore, while improving the recognition accuracy of the object X, the decrease in processing speed can be suppressed to be smaller than, for example, as compared with the configuration in which the recognition accuracy must be improved for all postures. As a result, it becomes an object recognition processing method capable of performing the recognition processing of the object X more accurately in a shorter time.

The object recognition processing device, the object recognition processing method, and the picking device of the present invention have been described above based on the illustrated embodiments, but the present invention is not limited to this, and the configurations of the respective parts have the same functions. It can be replaced with any configuration that has. Moreover, other arbitrary components may be added.

1 ... Picking device, 100 ... Robot, 110 ... Base, 120 ... Arm, 121 ... 1st arm, 122 ... 2nd arm, 130 ... Working head, 131 ... Spline nut, 132 ... Ball screw nut, 133 ... Spline shaft, 141-144 ... drive device, 150 ... payload, 151 ... manipulator, 160 ... robot control device, 170 ... mount, 200 ... camera, 300 ... object recognition processing device, 310 ... simulation unit, 311 ... image generation unit, 312 ... Recognition accuracy calculation unit, 320 ... Discriminator generation unit, 330 ... User interface unit, 331, 332 ... Reception unit, 340 ... Image acquisition unit, 350 ... Recognition unit, 360 ... Learning unit, 361 ... Storage unit, 370 ... Processing time Estimating unit, 380 ... Communication unit, 390 ... Control unit, 400 ... Monitor, C ... Controller, D1 ... Object information, E ... Encoder, J1 ... 1st axis, J2 ... 2nd axis, J3 ... 3rd axis, K ... Keyboard, M ... motor, P1 ... simulation image, P2 ... image data, S ... area, S11 to S14, S21 to S30 ... steps, T1 ... average processing time, T2 ... average processing time, X ... object

Claims (8)

A storage unit that stores object information to be identified, and
An image generation unit that generates a simulation image in which a plurality of identification targets are arranged,
A recognition accuracy calculation unit that performs image recognition by comparing the identification target in the simulation image generated by the image generation unit with the object information, and obtains the recognition accuracy of the image recognition for each posture of the identification target. When,
A user interface unit that notifies the user of the recognition accuracy and accepts whether or not the recognition accuracy is allowed from the user for each posture of the identification target.
An object recognition processing apparatus comprising: a classifier generating unit that generates a classifier for each posture of the discriminating target that is rejected when the user interface unit accepts the rejection. It has a processing time estimation unit that estimates the image processing time when the image processing is performed by adding the classifier generated by the discriminator generation unit.
The object recognition processing device according to claim 1, wherein the user interface unit notifies the image processing time estimated by the processing time estimation unit. The object recognition processing device according to claim 1 or 2, wherein the object information is generated from CAD data. The object recognition processing device according to any one of claims 1 to 3, wherein the image generation unit generates the simulation image by using the information of the environment in which the identification target exists. The recognition accuracy calculation unit obtains the recognition accuracy by collating the object information of the identification target acquired from the simulation image with the object information stored in the storage unit. The object recognition processing apparatus according to any one item. The object recognition processing device according to any one of claims 1 to 5, wherein the classifier generation unit generates the classifier using a neural network. The step of recording the object information to be identified and
A step of generating a simulation image in which a plurality of the identification targets are arranged, and
A step of performing image recognition on the identification target in the simulation image by comparing it with the object information and obtaining the recognition accuracy of the image recognition for each posture of the identification target.
A step of notifying the user of the recognition accuracy and receiving from the user whether or not to allow the recognition accuracy for each posture of the identification target.
An object recognition processing method comprising a step of generating a classifier for each posture of the identification target that is rejected when the user interface unit accepts the rejection. An object recognition processing device that recognizes the identification target as an image,
It has a manipulator that picks the identification target based on the recognition result of the object recognition processing device.
The object recognition processing device includes a storage unit in which the object information to be identified is stored and a storage unit.
An image generation unit that generates a simulation image in which a plurality of identification targets are arranged,
A recognition accuracy calculation unit that performs image recognition by comparing the identification target in the simulation image generated by the image generation unit with the object information, and obtains the recognition accuracy of the image recognition for each posture of the identification target. When,
A user interface unit that notifies the user of the recognition accuracy and accepts whether or not the recognition accuracy is allowed from the user for each posture of the identification target.
A picking device including a classifier generating unit that generates a classifier for each posture of the discriminating target that is rejected when the user interface unit accepts the rejection.

Images