JP7328173B2 - Picking control device - Google Patents

Description

The present invention relates to a picking control device.

2. Description of the Related Art In recent years, various proposals have been made for automating a picking operation for picking out a specified specific item from a large number of stored items at a distribution site such as a distribution warehouse using a robot.
For example, the conventional robot control device shown in FIG. The master image extracting unit 1 is supplied with the product number data of the product to be taken out from the outside. The master image database 2 stores the correspondence between the master image, which is the image of the product stored in the warehouse, and the product number. In the drawing, the database is described as "DB".

Then, the master image extraction unit 1 extracts the master image associated with the input product number from the master image database 2, and extracts the area of the product that matches the corresponding master image from the image captured by the camera of the robot. . Then, the master image extracting unit 1 determines the position where the corresponding article is arranged from the coordinate position of the extracted image, and provides the robot with data for picking the article at the determined position. send.
As shown in FIG. 10, the data for performing the picking work includes, for example, work recognition data d1 that instructs the work to be performed by the robot, trajectory planning data d2 that instructs the movement of the arm of the robot, and data for controlling the robot. It contains control data d3 indicating a command for

FIG. 11 shows an example of the data structure of the master image database.
The master image database stores correspondence between individual product numbers and master images associated with the product numbers.
The product number corresponds to the type of product handled by the physical distribution site. For example, one physical distribution site has tens of thousands of product numbers.
A master image is an image of a box containing products handled by a physical distribution site. Although the number of master images is smaller than the number of product numbers, there are, for example, tens to hundreds of types of master images. For example, even if the products are different, the packaging box may be the same for products of the same brand and of almost the same size. is less than the number of product numbers.

Conventionally, the task of associating the product number and the master image required for such picking work was manually performed by a worker using a terminal each time a product with a new product number was registered.

In Patent Document 1, in order to automate the picking work at the logistics site, the data of the three-dimensional shape that is the reference of the object to be picked is stored in a database, and the shape of the object measured by the measuring device is compared. technique is described. Further, in Patent Document 1, when the result of matching with the three-dimensional shape of an object measured by a measuring device is outside a predetermined range, the three-dimensional shape of the measured object is converted into a three-dimensional shape as a reference. It also describes that a recommendation process is performed to prompt registration as shape data.

JP 2019-38047 A

As described above, the work of associating the product number and the master image required for the picking work is currently performed manually, so the registration work for the prior association requires a great deal of time and effort. There was a problem. In other words, one physical distribution warehouse handles a huge number of tens of thousands of products, and it has been necessary to associate an extremely large number of products. In addition, when dealing with clothing, shoes, sporting goods, accessories, and other products in an industry that changes rapidly according to trends and seasons, it is necessary to associate tens of thousands of items every few months. It was a big obstacle in automating the work.

As described in Japanese Patent Laid-Open No. 2002-200013, it has been known that when a condition such as the shape of an item to be picked differs from the shape of an item that has already been registered, a process for prompting the registration of the item is performed. It is However, even if the user is prompted to register an article with a new shape, it is necessary for the operator to perform the work itself of registering the product number of the article. Therefore, the work of associating products is still necessary, and the problem of the heavy burden of registration work has not been improved.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a picking control device capable of reducing the burden of associating product numbers with master images and improving the efficiency of picking work by a robot.

In order to solve the above problems, for example, the configurations described in the claims are adopted.
The present application includes a plurality of means for solving the above problems, and to give one example, the present invention is the product number of the article under management and the image of the article or the image of the box containing the article. a master image extracting unit that acquires the item number of an item to be picked and extracts the master image associated with the acquired item number from the master image database; and the master image. a workpiece recognition unit that recognizes the gripping position of the product based on the , a trajectory planning unit that plans the trajectory of the robot based on the gripping position of the product, and a robot control unit that causes the robot to perform picking based on the planned trajectory. If the master image extraction unit cannot extract the master image associated with the product number of the product to be picked from the master image database, the product of the product to be picked is extracted from the master image stored in the master image database. a master image management unit that extracts a candidate for a master image inferred from the number, associates the extracted candidate for the master image with the article number of the article to be picked, and registers the extracted candidate in the master image database.

According to the present invention, when a master image associated with a product number does not exist, an operation of automatically associating an appropriate master image is performed, and data for picking operation can be properly registered.
Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the outline|summary structure of the whole picking system by one embodiment of this invention. 1 is a block diagram showing a configuration example of a picking control device according to an embodiment of the present invention; FIG. 1 is a block diagram showing a hardware configuration example of a picking system according to an embodiment of the present invention; FIG. 4 is a flowchart (example 1) showing the flow of control operation of the picking system according to one embodiment of the present invention; 9 is a flowchart (example 2) showing the flow of control operation of the picking system according to an embodiment of the present invention; It is a figure which shows the example (example 1) of the goods list by the database of one embodiment of this invention. FIG. 10 is a diagram showing an example (example 2) of a product list in the database according to one embodiment of the present invention; FIG. 10 is a diagram showing an example (example 3) of a product list in the database according to one embodiment of the present invention; 1 is a configuration diagram showing an example in which an embodiment of the present invention is extended to a system with a plurality of robots; FIG. It is a figure which shows the outline|summary of the conventional picking system. FIG. 10 is a diagram showing the correspondence between product numbers and master images in a conventional picking system;

An embodiment of the present invention (hereinafter referred to as "this example") will now be described with reference to FIGS. 1 to 9. FIG.
[Overall configuration of picking system]
FIG. 1 shows an outline of the overall configuration of the picking system of this example.
The picking system of this example includes a physical distribution control device 10 , a picking control device 20 and a robot 30 .
The physical distribution control device 10 controls the entire physical distribution in the physical distribution warehouse, and supplies the product number information of the product to be taken out to the picking control device 20 when it becomes necessary to take out the product from the physical distribution warehouse.
The picking control device 20 extracts a master image corresponding to the product number when information on the product number is supplied from the distribution control device 10, and performs a picking operation of picking out the product indicated by the master image. 30.

The robot 30 has an arm 31 that can freely move within a predetermined range, and under the control of the picking control device 20, uses a gripper 32 at the tip of the arm 31 to grip and take out the transported product.
For example, as shown in FIG. 1, it is assumed that a transport box 50 containing a large number of articles 51, 52, . Here, it is assumed that the item number of the item 51 is instructed to the picking control device 20 . At this time, the robot 30 picks up the instructed article 51 from the transport box 50 with the grasping portion 32 at the tip of the arm 31 and places the picked article 51 on a predetermined pallet 60 or the like.

A camera 41, a LiDAR sensor 42, and a millimeter wave radar 43 are arranged in the vicinity of the robot 30. The camera 41 photographs the conveyed article and the like, and the LiDAR sensor 42 and the millimeter wave radar 43 detect the position of the article and the like. Measure the shape. Images captured by the camera 41 and data measured by the LiDAR sensor 42 and the millimeter wave radar 43 are supplied to the picking control device 20 . A sensor for measuring the position and shape of an article may be part of the sensors described above, or may be combined with other sensors.

The picking control device 20 acquires a master image associated with the product number when supplied with the product number of the product to be picked. The acquired master image is an image of the surface of the box containing the item 51, and the picking control device 20 compares the image captured by the camera 41 with the master image to determine whether the item 51 is included in the box 50. Determine location.
After determining the position where the article 51 is stored, the picking control device 20 sends the robot 30 information on a trajectory plan for moving the arm 31 to the position where the article 51 is stored. Based on the trajectory plan sent from the picking control device 20 , the robot 30 operates the arm 31 of the robot 30 for picking to perform the task of picking up the article 51 .

That is, based on a command from the picking control device 20, the robot 30 uses the gripper 32 at the tip of the arm 31 to pick up the specified article 51 from the transport box 50, and picks up the picked article 51. The work of placing on the instructed pallet 60 or the like is performed.

[Configuration of picking control device]
FIG. 2 shows a configuration example of the picking control device 20. As shown in FIG.
The picking control device 20 includes a master image extraction unit 21 , a master image database 22 , a master image management unit 23 , a workpiece recognition unit 24 , a trajectory planning unit 25 , a robot control unit 26 and a new master image generation unit 27 .
When a product number is transmitted from the distribution control device 10 , the master image extraction unit 21 extracts the master image data associated with the product number from the master image database 22 . The master image data extracted by the master image extraction unit 21 is sent to the master image management unit 23 . Further, when the data of the master image cannot be extracted, the master image extraction unit 21 transmits information indicating that the data of the master image cannot be extracted to the master image management unit 23 .

The master image database 22 stores a product list indicating a list of product numbers handled by the distribution warehouse in which the picking system of this example is installed, and master image data associated with each product number in the product list. . A specific example of the product list stored in the master image database 22 will be described later.

A master image stored in the master image database 22 is an image of a box containing a product (article) or an image of the product itself. In the case of an image of a box containing products, the master image may be an image of a specific side that needs to be recognized at the time of picking, or may have images of a plurality of sides of the box as the master image. Furthermore, the box does not have to be square in shape and may be an article of irregular shape such as a bag.
It should be noted that the specific side that needs to be recognized at the time of picking is preferably a side that describes, for example, the manufacturing company, brand, type, etc. of the product, that is, a side that has a large amount of features. Also, the master image may be a parallax image or a point cloud image instead of the image of the box containing the article or the article itself. Further, data such as the shape data (three-dimensional shape data, etc.) of the box or article, and the holding position may be added to the master image.

First, the case where the master image extraction unit 21 can extract the master image associated with the product number in the above-described process will be described. In this case, the master image management section 23 transmits the master image to the workpiece recognition section 24 .

The workpiece recognition unit 24 recognizes the position of the article based on the master image and the image captured by the camera 41 and calculates the position to be gripped by the gripping unit 32 of the robot 30 . The data of the gripping position of the article calculated by the workpiece recognition unit 24 is supplied to the trajectory planning unit 25 .

The trajectory planning unit 25 plans a trajectory for moving the arm 31 of the robot 30 based on the data of the gripping position of the article, and sends the created trajectory planning data to the robot control unit 26 . Based on the trajectory planning data sent from the trajectory planning section 25 , the robot control section 26 controls the movement of the arm 31 and controls the grasping of the article by the grasping section 32 of the robot 30 .

Next, the case where the master image extraction unit 21 cannot extract the master image in the above-described process, that is, the case where the master image and the product number are not associated and stored will be described. When the master image management unit 23 receives the information indicating that the master image data cannot be extracted from the master image extraction unit 21, the master image management unit 23 sends a request for master image candidates to the master image database 22, and extracts the candidate master image data. do. Then, the extracted candidates for the master image are transmitted to the workpiece recognition unit 24 . The details of the process of extracting candidate master image data from the master image database 22 will be described later.

The workpiece recognition unit 24 recognizes the gripping position of the article based on the master image candidate and the image captured by the camera 41 . If the gripping position can be recognized, the trajectory planning unit 25 and the robot control unit 26 control the arm 31 based on the result of the gripping position in the same manner as described above, and the gripping unit 32 of the robot 30 grips the article. to control.
Then, the workpiece recognition unit 24 notifies the master image management unit 23 that the gripping position has been recognized successfully, that is, that the article number of the handled article corresponds to the candidate of the master image. The master image management unit 23 associates the product number with the master image and registers them in the master image database 22 .

Next, a case will be described in which the workpiece recognition unit 24 attempts to recognize the gripping position of the article by collating the candidates for the master image and the image captured by the camera 41, but fails to recognize the gripping position. In this case, the work recognizing section notifies the master image managing section 23 that it could not be recognized by the master image candidates. The master image management unit 23 requests another master image candidate from the master image database 22 . If there is another master image candidate, the master image candidate is sent to the workpiece recognition section 24, and the workpiece recognition section 24 repeats the same processing as described above. If there is no other master image candidate that has not been collated in the database, the master image management unit 23 instructs the new master image generation unit 27 to generate a master image. The new master image generation unit 27 generates a master image based on the image of the camera 41 , the lidar point group information, the millimeter wave shape recognition result, and the like, and sends the new master image to the master image management unit 23 . The master image management unit 23 transmits the new master image to the workpiece recognition unit, and registers the target product number and the new master image in the master image database 12 in association with each other.

Although not shown in FIG. 2, the picking control device 20 includes a display unit 20g and an input unit 20f as shown in FIG. In addition to doing so, it is also possible for the operator to input operations. For example, in response to the display on the display section 20g, the input section 20f accepts an operation input from the worker so that the worker can register the association between the product number and the master image. Alternatively, a worker registers the association between the product number and the master image using another terminal device capable of communicating with the picking control device 20, and the picking control device 20 acquires the data of the registration work, It may be registered in the image database 22 .

[Example of hardware configuration of picking control device]
The picking control device 20 can be composed of, for example, a computer.
FIG. 3 shows the hardware configuration when the picking control device 20 is configured by a computer.
A computer functioning as the picking control device 20 includes a CPU (Central Processing Unit) 20a, a ROM (Read Only Memory) 20b, and a RAM (Random Access Memory) 20c, which are respectively connected to a bus. Furthermore, the picking control device 20 includes a nonvolatile storage 20d, a network interface 20e, an input section 20f, and a display section 20g.

The CPU 20a executes arithmetic processing on the master image in the master image extraction unit 21, the master image management unit 23, and the new master image generation unit 27, and arithmetic processing related to robot control in the workpiece recognition unit 24, the trajectory planning unit 25, and the like. It is an arithmetic processing unit that reads and executes a program code of software to be executed from the ROM 20b. The RAM 20c is temporarily written with variables, parameters, etc. generated during the arithmetic processing.

A large-capacity information storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) is used for the nonvolatile storage 20d. The nonvolatile storage 20d constitutes a master image database 22 in which data such as master images and product lists are stored.

For example, a NIC (Network Interface Card) or the like is used for the network interface 20e. The network interface 20e receives product number data transmitted from the physical distribution control device 10, for example. Further, instructions to the robot 30 are given via the network interface 20e.
The input unit 20f performs input processing of various types of data by a user who performs work such as registration of a master image in performing processing as the picking control device 20 .
The operation status of the picking control device 20 is displayed on the display section 20g. Further, the display unit 20g displays candidate images and displays various warnings when registering a master image.

It should be noted that the configuration of the picking control device 20 with the computer shown in FIG. 3 is an example, and may be configured with a device that performs arithmetic processing other than the computer. For example, some or all of the functions performed by the picking control device 20 may be realized by hardware such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).
The configuration in which the picking control device 20 includes the input unit 20f and the display unit 20g is also an example. may be configured as

[Flow of processing performed by the picking control device]
FIG. 4 is a flow chart showing the flow of processing operations performed by the picking control device 20. As shown in FIG.
First, upon receiving a product number from the distribution control device 10, the master image extraction unit 21 performs extraction processing of a master image associated with the product number (step S11). Then, the master image extraction unit 21 determines whether or not the master image associated with the product number exists in the master image database 22 (step S12).

When it is determined in step S12 that there is a master image associated with the product number (Yes in step S12), the workpiece recognition unit 24 receives the master image extracted by the master image extraction unit 21 via the master image management unit 23. , and the camera image photographed based thereon and the master image are collated to recognize the gripping position of the article (step S13). Then, when the workpiece recognition unit 24 recognizes the gripping position of the article, in step S23, the picking control device 20 executes the process of gripping the article at the recognized position and transports the article to the desired position.

When it is determined in step S12 that the master image associated with the product number does not exist (No in step S12), the master image extraction unit 21 notifies the master image management unit that the master image associated with the product number cannot be extracted. 23. The master image management unit 23 that has received this sends a master image request to the master image database 22, and extracts candidate master image data from the master image database 22 (step S14).

When extracting candidate master image data in step S14, the master image management unit 23 extracts, for example, the master images in descending order of frequency of use. Alternatively, a master image that is highly likely to be related to the product number determined in step S11 is selected and extracted. As a master image that is highly likely to be related to the product number, for example, the manufacturer and brand are guessed from the product number, and the master image corresponding to the guessed manufacturer and brand is used as a candidate.

Then, the master image management unit 23 sends the obtained master image candidates to the workpiece recognition unit 24 . The work recognizing unit 24 compares the image of the box 50 (FIG. 1) containing the article to be taken out, which is photographed by the camera 41, with the candidates of the master image, and recognizes the gripping position of the article (step S15). In the collation at step S15, the workpiece recognition unit 24 determines whether or not there is a matching article in the image captured by the camera 41 and whether the gripping position of the matching article can be recognized (step S16).

If it is determined in step S16 that the gripping position of the article that matches the master image candidate can be recognized (Yes in step S16), the master image management unit 23 selects the corresponding master image candidate as the product of this time. It is registered in the master image database 22 as a master image associated with the number (step S17). Then, the picking control device 20 executes processing for gripping the article associated as the master image at the gripping position recognized in step S15 (step S23).

If the master image management unit 23 determines in step S16 that there is no image that matches the candidate for the master image in the image captured by the camera 41 (No in step S16), the master image management unit 23 determines whether or not there is another master image candidate that has not been collated in the master image database 22 (step S18).

If it is determined in step S18 that there is another candidate for the master image that has not been collated (Yes in step S18), the master image management unit 23 returns to the process of step S14, and performs data of another candidate master image. are extracted from the master image database 22, and the processes after step S14 already described are executed.

If it is determined in step S18 that there is no candidate for another master image (No in step S18), the master image management section 23 issues an instruction to the new master image generation section 27. FIG. The new master image generation unit 27 generates a master image based on the sensor information of the camera 41 , lidar 41 , and millimeter wave 43 , and outputs the master image to the master image management unit 23 . The master image management unit 23 registers the master image in the master image database 22 in association with the target product number (step S19). The registration of the master image may be performed automatically as described above. However, a message indicating that new registration of the master image is necessary is displayed on the display unit 20g, and the new registration of the master image is performed by the operator. may be executed.
At this time, for example, among the camera images captured by the camera 41, a specific article designated by the operator's operation may be registered as a master image. Alternatively, an image other than the camera image may be registered as the master image.
After the registration process of step S19 is performed, the workpiece recognition unit 24 recognizes the gripping position of the article associated as the master image (step S20), and executes the process of gripping the corresponding article (step S23).

Then, after the process of step S13, S17 or S20 is executed and the article gripping process of step S23 is executed, the process from step S11 is repeatedly executed by inputting the next product number.
When the candidates for the master image are extracted in step S14, it is preferable to notify the worker by displaying the fact that candidates are being extracted on the display unit 20g (FIG. 3) of the picking control device 20. . At this time, the display unit 20g may display candidate master images. Further, it is preferable to notify the operator through the display unit 20g of the picking control device 20 that the association process is being performed even when the process of associating the master image with the product number is executed in step S17. At the time of these notifications, the master image may be associated with the product number after the operator accepts the registration operation.

FIG. 5 shows a second example of a flow chart showing the control operation flow of the picking system according to one embodiment of the present invention.
The processing from steps S11 to S18 in the flow chart of FIG. 5 is the same as that of the flow chart of FIG. 4, so description thereof will be omitted.
In the flowchart of FIG. 4, after newly registering the master image in step S19, the process of gripping the corresponding article is performed in step S20.
On the other hand, in the second example shown in the flowchart of FIG. 5, in step S18, when the master image management unit 23 determines that there is no candidate for another master image (No in step S18), the operator's operation , the gripping position of a specific article in the work area of the robot 30 is indicated so that the work recognizing unit 24 first performs the task of recognizing the gripping position (step S21).

In this case, the new master image generating unit 27 newly registers the camera image of the article whose gripping position is recognized as a master image after performing the picking operation of taking out the article at the recognized gripping position (step S22). After the master image is newly registered in step S22, the process from step S11 is repeated by inputting the next product number.
By executing the processing shown in the flowchart of FIG. 4 or the processing shown in the flowchart of FIG. 5, most of the registration work for associating the master image with the product number is automated, and the pick-up work by the robot can be executed very efficiently. Become. In other words, the product numbers handled by distribution warehouses often change significantly at certain times, and even in such cases, workers do not need to manually register the associations, and they can be used for pickup work. This will greatly reduce the time required for registration.

[Example of product list associated with product number]
Next, specific examples of the product list, which is a list of product numbers, are shown in FIGS. 6 to 8. FIG.
FIG. 6 shows an example of the correspondence relationship between the product list La associated with the product number and the product list Lb associated with the master image.
The product list La associated with the product number contains product characteristics such as product number, type, brand, product size, box size, logo type, logo size, logo position, box color, hole position, etc. have information on However, it is not necessary to have all the information shown in the product list La for each product number item.

The product list Lb associated with the master image has characteristic information of the product having the master image, such as the master image, brand, product size, box size, and logo type. However, it is not necessary to have all the information shown in the product list Lb for each master image.

Here, for example, when the master image associated with the product number AAA-1 shown in the product list La associated with the product number is not registered, the candidate for the master image associated with the product number is associated with the master image. is extracted from the product list Lb obtained.
For example, if product number AAA-1 is registered as logo type "AAA" on product list La, in product list Lb associated with the master image, a master image with the same logo type "AAA" A master image of No. 1 and a master image of No. 2 are candidates. Also, master images with different logo types such as "BBB" are excluded from the candidates.

Then, among the two candidate master images (master images of No. 1 and No. 2), the optimum master image is selected by comparison with the camera image or by an operator's instruction, for example, product number AAA- A master image associated with 1 is chosen.

FIG. 7 shows a case where three product numbers, ie, product numbers AAA-1, AAA-2, and AAA-3, are not associated with a master image as a product list Lc associated with product numbers.
Assume that the product numbers AAA-1, AAA-2, and AAA-3 have the same type, brand, and product name, but differ only in size.
At this time, the picking control device 20 of the present example associates one product number AAA-1 with the master image B by the processing described in the flowchart of FIG. 4 and the like.
When the master image B is associated with this product number AAA-1, the picking control device 20 of this example also handles the product numbers AAA-2 and AAA-3, which differ only in size from the product number AAA-1. , registration processing is performed to associate the same master image B with the product number.

Here, the case where only the size is different has been described, but even if items other than the size are different, the product number with which the master image is associated and the product number that has a predetermined similarity relationship, The same master image may be associated. For example, product numbers with different product colors may be associated with the same master image.

Further, in the case of FIG. 7, the registration process for associating the master image B with the first product number AAA-1 is the automatic registration process described in the flowchart of FIG. 4 or FIG. It can also be applied to the case where the master image B registration process is performed manually. That is, when a worker manually associates a master image with a specific product number, the product number associated with the master image and another product having a predetermined similarity relationship such as size and color For product numbers, the same master image is registered collectively.
By registering the same master image for a plurality of product numbers having a predetermined similarity relationship in this way, it is possible to improve the efficiency of the master image registration work.

FIG. 8 shows a case where the product list Ld associated with the product number has information on the master image group for each product number.
When the picking control device 20 has this master image group information, when a master image is associated with one product number, the picking control device 20 associates the same master image with other product numbers in the same group.
That is, when a plurality of product numbers having a common master image are stored in advance as a group, and the master image is associated with the product number assigned to each product, the picking control device 20 combines the group information and the master image. Based on this, the same master image can be registered for another product number in the same group.

In this way, even when the product list Ld associated with the product number has group information, the master images of a plurality of product numbers can be collectively registered, thereby improving the efficiency of the master image registration work. can be done.
Even when such group information is available, the master image registration process may be performed manually by an operator in addition to the automatic registration process described in the flow chart of FIG. 4 or FIG. also applies in case

[Example applied to a system with multiple robots]
FIG. 9 shows an example of a configuration including a plurality of robots as the picking system of this example.
The picking system shown in FIG. 9 has three robots 110, 120 and 130 in addition to the robot 30 described in the system configuration of FIG. Robot 30 is controlled by picking controller 20, and robots 110, 120, 130 are controlled by picking controllers 210, 220, 230, respectively.
Each picking control device 210, 220, 230 has the same configuration as the picking control device 20 shown in FIG. However, in the case of the system configuration shown in FIG. 9, the master image database 71 is commonly used by all the picking control devices 20, 210, 220, 230. FIG.

The picking system shown in FIG. 9 also includes a sorter control device 70 . The sorter control device 70 sorts the transported products into lanes corresponding to the respective robots 30 , 110 , 120 , 130 based on the product number information supplied from the physical distribution control device 10 .
The sorter control device 70 also sorts the product numbers supplied from the physical distribution control device 10 to the robots 30, 110, 120, and 130 in conjunction with the lane sorting.

Here, the sorter control device 70 refers to the master image database 71 for the product number supplied from the distribution control device 10 and determines whether or not there is an associated master image. Then, the products having the product numbers with associated master images are sorted into lanes of one of the three robots 110, 120, 130, and the picking control devices 210, 220, 230 of the respective robots 110, 120, 130 , to carry out the picking of the corresponding goods. The robots 110, 120, and 130 may be equipped with only the camera 41 as a sensor for detecting products, and the LiDAR sensor 42 and millimeter wave radar 43 (both shown in FIG. 2) may be omitted. .

On the other hand, the sorter control device 70 sorts into the lanes of the robot 30 products with product numbers that do not have associated master images. Then, the picking control device 20 of the robot 30 picks the corresponding product. At this time, the picking control device 20 executes the master image registration process described in the flowchart of FIG. 4 or 5 . The robot 30 has a configuration in which the LiDAR sensor 42 and the millimeter wave radar 43 shown in FIG. 2 are arranged.

With this configuration, the three robots 110, 120, and 130 sequentially perform the picking process using the registered master image for a product having a master image associated with the product number. throughput can be improved.

On the other hand, a product for which there is no master image associated with the product number is assigned to one specific robot 30, and the robot 30 executes the picking process while performing the registration process of the master image. be registered. Further, when registering the master image, the LiDAR sensor 42 and the millimeter wave radar 43 are preferably provided so that details such as the shape of the box (product) can be measured. , need only be placed in the robot 30, which contributes to the simplification of the system configuration.

[Modification]
It should be noted that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, in the above-described embodiment, an example of a product list is a box containing shoes, but it may be applied to picking processing of various other products. In this case, in the above-described embodiment, an example of picking boxes containing products has been described, but the robot may pick products from a state in which the products themselves are directly arranged.

Moreover, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to those having all the described configurations.
Furthermore, in the above-described embodiments, the device or system configuration may be changed, and some of the processing procedures may be omitted or replaced without changing the gist of the present invention.
Information such as a program for performing the picking process can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or an optical disc.

Furthermore, in block diagrams such as FIGS. 2 and 3, only control lines and information lines that are considered necessary for explanation are shown, and not all control lines and information lines are necessarily shown on the product. do not have. In practice, it may be considered that almost all configurations are interconnected. In addition, in the flowcharts shown in FIGS. 4 and 5, a plurality of processes may be executed simultaneously or the order of the processes may be changed as long as the processing results are not affected.

DESCRIPTION OF SYMBOLS 1... Master image extraction part 2... Master image database 10... Distribution control apparatus 20... Picking control apparatus 20a...Central processing unit (CPU) 20b...ROM, 20c...RAM, 20d...Non-volatile storage, 20e... Network interface 20f Input unit 20g Display unit 21 Master image extraction unit 22 Master image database 23 Master image search unit 24 Work recognition unit 25 Trajectory planning unit 26 Robot control unit , 27... New master image generation unit, 30... Robot, 31... Arm, 32... Holding unit, 41... Camera, 42... LiDAR sensor, 43... Millimeter wave radar, 50... Box, 51, 52... Goods, 60... Pallet , 70... Sorter control device, 71... Master image database, 110 to 140... Robot, 210 to 260... Picking system

Claims (9)

a master image database in which product numbers of managed products and master images, which are images of products or boxes containing products, are stored;
a master image extraction unit that acquires the product number of an item to be picked and extracts a master image associated with the acquired product number from a master image database;
a workpiece recognition unit that recognizes the gripping position of the product based on the master image;
a trajectory planning unit that plans a trajectory of the robot based on the gripping position of the product;
a robot control unit that causes the robot to perform picking based on the planned trajectory;
When the master image extraction unit cannot extract the master image associated with the product number of the product to be picked from the master image database, the product of the product to be picked is extracted from the master image stored in the master image database. A picking control device, comprising: a master image management unit that extracts master image candidates estimated from numbers, associates the extracted master image candidates with product numbers of articles to be picked, and registers the extracted master image candidates in a master image database. 1. When the extracted candidate for the master image is included in the image obtained by photographing the range in which the picking operation is performed, the master image management unit associates the candidate for the extracted master image with the article number of the article to be picked. The picking control device according to . The master image database stores the frequency of use of each master image,
The picking control device according to claim 1, wherein the master image management unit extracts candidates for the master image based on the frequency of use of the master images stored in the master image database. The picking control device according to claim 1, wherein the master image management unit acquires property information of an article based on the article number of the article to be picked, and extracts master image candidates from the acquired property information. If the master image management unit cannot extract from the master image database a candidate for a master image inferred from the product number of the item to be picked, the master image management unit generates the image based on the result of measuring and/or photographing the item to be picked. The picking control device according to claim 1, wherein the picked image is registered as a master image associated with the article to be picked. the robot has at least a first robot and a second robot;
The robot control unit causes the first robot to perform a picking operation for an item having a master image associated with the item number of the item to be picked, and instructs the second robot to perform picking. 2. The picking control device according to claim 1, for executing a picking operation for a product for which a master image associated with the product number of the target product does not exist. Furthermore, it has a display unit,
7. Any one of claims 1 to 6, wherein, when the master image management unit extracts candidates for the master image estimated from the item number of the item to be picked, the display unit displays that candidates are being extracted. Picking control device according to paragraph. Furthermore, it has a display unit,
7. When the master image management unit associates the extracted master image candidate with the product number of the item to be picked, the display unit displays execution of processing for associating with the product number. or the picking control device according to item 1. a master image database in which product numbers of managed products and master images, which are images of products or boxes containing products, are stored;
a master image extraction unit that acquires the product number of an item to be picked and extracts a master image associated with the acquired product number from a master image database;
a workpiece recognition unit that recognizes the gripping position of the product based on the master image;
a trajectory planning unit that plans a trajectory of the robot based on the gripping position of the product;
a robot control unit that causes the robot to perform picking based on the planned trajectory;
Performing registration processing for associating the product number of the article to be picked with the master image stored in the master image database, and registering the product number that is not associated with the master image when the registration processing is performed. a master image management unit that associates a product number having a predetermined relationship with the product number of the product to be picked, with the master image stored in the master image database.

Images