JP6523883B2 - Estimation device, object position recognition device and estimation program - Google Patents

Description

  The present invention relates to an estimation device, an object position recognition device, and an estimation program.

  Conventionally, as a technique for inspecting the object using binary image data obtained by binarizing image data obtained by photographing the object with a camera, a camera is used to photograph an imprint made on the surface of the object, There has been a technique for inspecting the shape of the stamp by processing the obtained image.

  In this technique, for the input image captured by the camera, the engraved portion is masked to obtain the luminance distribution of the background portion, and the threshold of the binarization process is set based on the luminance distribution. Then, in this technology, binarization processing is performed on the input image based on the set threshold value, and the quality of the marking shape is determined using the bright and dark binary image data obtained by this.

JP, 2006-118895, A

  By the way, in the state where a plurality of objects are accommodated in a container, there are cases where it is desired to take out each object from the container by a robot. For example, there is a case of a delivery center which receives a large number of breads from a baking factory and distributes and delivers the required number of breads for each store to convenience stores and supermarkets and other stores. That is, at the delivery center, the bread is taken out one by one from the order number for arrival received from the baking factory by the picking robot, and finally the number of bread requested from each store is shipped for each store It is when you want to store in a stack.

  In this case, it is necessary to recognize the position of each object in the container, and as this technique, a technique to recognize the position of each object using image data obtained by photographing the plurality of objects can be considered. .

  However, with this technique, it has been difficult to accurately recognize the position of each of a plurality of objects having holes or recesses of various shapes such as an annular shape and a rectangular shape in the center. This is because it is difficult to distinguish the area of the hole or the recess and the area of the gap between the objects. In addition, as an object having a hole or a recess in the central part, in addition to food products such as annular (donut-like) bread manufactured in the above-described baking factory, donuts, bread in which the central part is recessed, etc., tires, nuts And annular members, etc. are exemplified.

  In order to solve this problem, it is also conceivable to recognize the position of each object by masking the position corresponding to the hole or recess of the object with respect to the image of the object using the above-mentioned conventional technique. However, in this case, if the position of each object deviates from the expected position for some reason, the accurate position can not be masked, and as a result, the recognition accuracy of the position of each object is significantly reduced. Do.

  The above problems occur not only in the case where it is desired to take out each object from the container by the robot in a state in which the plurality of objects are accommodated in the container, but also occurs in applications in which the positions of other objects are recognized. It is a problem to gain.

  An object of the present invention is to enable the position of each of a plurality of objects having a hole or a recess in the center to be recognized with high accuracy as one aspect.

  In one aspect, all the objects accommodated in the container in a state in which a predetermined number of objects having holes or recesses provided at the central portion in the container are accommodated in a predetermined arrangement direction intersecting the direction of plan view And an acquisition unit configured to acquire image data of an image obtained by photographing an object group region including the objects of Here, the container is configured to be capable of accommodating a plurality of the objects in the arrangement direction in a direction in which the holes or the recesses can be visually recognized in a plan view. The image processing apparatus further includes a detection unit that detects the positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit. Further, a deriving unit is provided which derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detection unit by the predetermined number. Further, an estimation unit is provided which estimates the position of the central portion in the arrangement direction in the plurality of divided regions derived by the derivation unit as the position of the hole or the recess in each of the objects.

  As one aspect, the position of each of a plurality of objects having a hole or a recess in the center can be recognized with high accuracy.

It is a top view (partial block diagram) which shows an example of a structure of the object distribution system which concerns on embodiment. It is a functional block diagram of an object position recognition device concerning an embodiment. It is a block diagram showing a schematic structure of a computer concerning an embodiment. It is a flow chart which shows an example of object position recognition processing concerning an embodiment. It is a schematic diagram showing an example of the initial information input screen concerning an embodiment. It is a flow chart which shows an example of the order number processing concerning a embodiment. FIG. 8 is a diagram provided for explaining the regular number stacking process according to the embodiment, and is a diagram showing an example of a binary image indicated by binary number data of the regular number double. It is a figure which is provided to the explanation of the regular stacking processing which concerns on embodiment, and is a figure which is provided to the explanation of the extraction method of a boundary. It is a figure which is provided to the explanation of the regular stacking process concerning an embodiment, and is a figure showing the example of a circumscribed rectangle frame and a division field. FIG. 8 is a diagram provided for explanation of regular number stacking processing according to the embodiment, and is a diagram provided for explanation of adjustment of divided regions in the row direction; FIG. 8 is a diagram provided for explanation of regular number stacking processing according to the embodiment, and is a diagram provided for explanation of adjustment of divided regions in the column direction; FIG. 8 is a diagram provided for explanation of regular stacking processing according to the embodiment, and showing an example of an arrangement position of mask data. FIG. 8 is a diagram for explaining regular stacking processing according to the embodiment, and showing a state of mask data superposition and an example of binary image data after superposition of mask data; FIG. It is a flow chart which shows an example of round number processing concerning a embodiment. FIG. 7 is a diagram for explaining the round numbering process according to the embodiment, and is a view showing an example of a binary image indicated by the binary number data of the round number double. It is a figure which is provided to the explanation of the round and square processing according to the embodiment, and shows the example of the temporary arrangement position of the mask data. It is a figure which is provided to the explanation of the fraction serial processing which concerns on embodiment, and is a figure which is provided to the explanation of the unnecessary mask data removal method. It is a figure which is provided for explanation of the round and square processing according to the embodiment, and shows an example of a state in which unnecessary mask data is removed. It is a figure which is provided for explanation of the round numbering process concerning an embodiment, and is a figure showing an example of binary image data after superposition of mask data. It is a flowchart which shows an example of the final stacking process which concerns on embodiment. It is a figure where it uses for explanation of the last loading process concerning an embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for explanation of the 2nd object position recognition concerning an embodiment. It is a figure where it uses for explanation of the 2nd object position recognition concerning an embodiment.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, the present invention is received from a bread factory in a state in which a large number of breads (corresponding to the "object" of the present invention) are accommodated in a receiving number, and for a plurality of shops. The case where it applies to the delivery center which distributes and distributes bread by the number requested for every store is explained.

  FIG. 1 is a plan view (partial block diagram) showing an example of the overall configuration of an object sorting system 90 for sorting pans in a delivery center according to the present embodiment.

  As shown in FIG. 1, in the object distribution system 90 according to the present embodiment, an object position recognition device 10, a picking robot 30, a distance image camera (hereinafter simply referred to as "camera") 32, a table 40 and a belt conveyor 46 It is equipped.

  In order to deliver the required number of breads 44 received from the baker's factory to each store, the belt conveyor 46 carries a shipping weight 48 for storing the required number of breads 44 from each store in a predetermined direction ( In this embodiment, the sheet is moved and conveyed in the direction of arrow D in FIG. In the present embodiment, a doughnut-shaped bread in which a through hole (hereinafter simply referred to as a "hole") 44A is opened in the center as the bread 44 is applied, but the present invention is not limited to this. For example, a pan or the like in which a recess is formed in the central portion may be applied.

  On the other hand, in the table 40, one end side (the lower side in FIG. 1) is a place (hereinafter referred to as a "stacking place") 40A where a plurality of receiving bunds 42 containing a plurality of pans 44 are stacked. It is done. Further, in the table 40, the intermediate portion is moved one by one from the plurality of incoming shipment weights 42 stacked one by one from the plurality of incoming shipment weights 42 stacked in the stacking place 40A, and the bread 44 from the incoming shipment weights 42. A place to be taken out (hereinafter referred to as "picking place") 40B. The table 40 is positioned so that the other end side (upper side in FIG. 1) opposite to the one end side approaches the intermediate portion of the belt conveyor 46. The picking robot 30 is installed on the other end side of the table 40. The picking robot 30 takes out the pan 44 from the inside of the receiving sheave 42 moved to the picking place 40 B, and accommodates it in the shipping sheave 48 transported by the belt conveyor 46.

  In the object sorting system 90 according to the present embodiment, the stocking bundling 42, which is a regular weight, in which the upper limit number of pans 44 that can be accommodated is accommodated is stacked at the stacking location 40A of the table 40. In the object sorting system 90 according to the present embodiment, if there is a fractional weight which is a number with which the bread 44 serving as a fraction is accommodated according to the number of articles received from the baking factory, the fractional weight is determined. Stacking bunks 42 are stacked on the top layer of the stacking station 40A. Then, in the object distribution system 90 according to the present embodiment, the plurality of receiving sheaves 42 stacked in the stacking place 40A are sequentially moved one by one from the receiving shelving pile 42 of the lowermost layer to the picking place 40B.

  In the object distribution system 90 according to the present embodiment, the first process of stacking the arrival number 42 in the stacking area 40A of the table 40 is performed by a worker of the delivery center (hereinafter simply referred to as "worker"). It is assumed. Further, in the object distribution system 90 according to the present embodiment, the second process of moving the arrival number 42 from the stacking place 40A to the picking place 40B is automatically and sequentially performed by a moving mechanism (not shown). Furthermore, in the object sorting system 90 according to the present embodiment, the worker performs the third step of removing all the received bundling weights 42 from the picking place 40B after all the pans 44 are sorted by the picking robot 30. It is assumed. However, each process of these 1st processes and 3rd processes is not limited to the form performed by the worker. These steps may be performed automatically by a transfer robot, a moving mechanism, or the like. Further, the second step is not limited to the mode automatically performed by the moving mechanism or the like, and may be manually performed by an operator.

  On the other hand, the object position recognition device 10 according to the present embodiment has an estimation device 20. The estimation device 20 according to the present embodiment estimates the positions of the individual holes 44A of the pan 44 contained in the arrival number 42 provided at the picking place 40B. Then, the object position recognition apparatus 10 according to the present embodiment recognizes the individual positions of the pan 44 in the arrival number 42 based on the positions, etc. of the individual holes 44A of the pan 44 estimated by the estimation apparatus 20. .

  As shown in FIG. 1, a picking robot 30 and a camera 32 are connected to the object position recognition apparatus 10. The camera 32 according to the present embodiment is provided above the picking place 40B, the imaging direction is directed downward, and the distance image in the entire area of the placement area of the stocking bundling 42 provided at the picking place 40B. Get data Here, the distance image data is data in which the value of each pixel of the image data indicates the distance to the subject for each pixel in the shooting angle of view by the camera 32, and an example of the camera 32 is Kinect (Kinect (registered trademark)) is exemplified.

  Thus, in this embodiment, although the case where a distance image camera is applied as camera 32 is described, it is not limited to this. For example, a commercially available camera for capturing a color image or a monochrome image may be applied as the camera 32.

  The estimation apparatus 20 according to the present embodiment sets “0” (white) as a pixel indicating the distance in the range from the upper end to the center in the height direction of the pan 44 in the distance image data obtained by the camera 32. Binary image data is generated by setting the other pixels to “1” (black). By binarizing the distance image data, the image area of the pan 44 and another image area can be divided by different values. In the present embodiment, as described above, the image area of the pan 44 is “0” and the other image areas are “1”. However, the present invention is not limited to this, and the image area of the pan 44 is “1”. The other image area may be "0".

  In addition, the estimation device 20 estimates the position of the hole 44A of the pan 44 contained in the arrival number 42 provided at the picking place 40B, using the generated binary image data.

  On the other hand, as described above, the object position recognition apparatus 10 according to the present embodiment is based on the position of the hole 44A of the pan 44 estimated by the estimation apparatus 20, etc. Recognize Then, the object position recognition device 10 transmits position information indicating the position of the recognized pan 44 to the picking robot 30.

  When the picking robot 30 receives the position information from the object position recognition device 10, the picking robot 30 takes out the pan 44 from the arrival number 42 provided at the picking place 40B based on the position information. Then, the picking robot 30 stores the necessary number of pans 44 taken out of the receiving number 42 in the shipping number 48 positioned at the object storage position 46A by the belt conveyor 46. When the picking robot 30 completes the storage of the required number of pans 44 in the shipping number 48, the belt conveyor 46 moves and transports the shipping number 48 to a predetermined location such as a shipping port at the delivery center.

  In addition, in the delivery number 42 according to the present embodiment, the predetermined alignment direction (in the present embodiment, an example) which intersects the direction of the plan view in a direction in which the pan 44 can visually recognize the holes 44A of the pan 44 in plan view. As shown in FIG. 1, a plurality of pieces can be accommodated in two directions, row direction and column direction. As shown in FIG. 1, the stocking weights 42 according to the present embodiment have a total of 12 pans 44 at a maximum of 4 (row direction) × 3 (column method), and the holes 44A of each pan 44 are It is to be understood that although it is intended to be flat in a visible orientation, the maximum number of these is merely an example.

  FIG. 2 shows an object position recognition apparatus 10 according to the present embodiment and an estimation apparatus 20 included in the object position recognition apparatus 10. As shown in FIG. 2, the estimation device 20 according to the present embodiment includes an acquisition unit 11, a detection unit 12, a derivation unit 13, a reception unit 14, and an estimation unit 15. Further, the object position recognition device 10 according to the present embodiment includes an estimation device 20, a superimposing unit 17, and a recognition unit 18.

  The acquisition unit 11 uses the camera 32 to capture an imaging area including an object group area including all the pans 44 of the receiving number 42 for reception, in which the upper limit number of pans 44 that can be accommodated are accommodated in a planar shape and made into a regular stack. By imaging, distance image data of the imaging area is acquired. Then, the acquiring unit 11 generates binary image data indicating an image including the object group area by binarizing the acquired distance image data as described above.

  Further, the detection unit 12 detects the positions of both ends of the arrangement direction of the pan 44 in the object group region in the binary image represented by the binary image data generated by the acquisition unit 11. Further, the deriving unit 13 derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detecting unit 12 by the upper limit number. Furthermore, the estimation unit 15 estimates the position of the central portion in the plurality of divided areas derived by the derivation unit 13 as the position of the hole 44A in each of the pans 44. The receiving unit 14 receives the input of the upper limit number, and the deriving unit 13 uses the upper limit number received by the receiving unit 14 to derive the divided area.

  By the way, in the object distribution system 90 according to the present embodiment, a plurality of pans 42 for receiving can be accommodated in each of the pan 44 in two directions of the row direction and the column direction as the arrangement direction. For this reason, the detection unit 12 according to the present embodiment detects the positions of the both ends in the object group region separately for each of the row direction and the column direction. Further, the deriving unit 13 derives a plurality of divided areas by equally dividing between both ends detected by the detecting unit 12 by the corresponding upper limit number separately for each of the row direction and the column direction. Then, the estimation unit 15 estimates the position of the central portion in the plurality of divided areas derived by the derivation unit 13 as the position of the hole 44A along each direction of the pan 44 separately for each of the row direction and the column direction. Do.

  In particular, the detection unit 12 according to the present embodiment selects one of the plurality of divided regions derived by the derivation unit 13 for each of the plurality of divided region groups along the arrangement direction in either the row direction or the column direction. The positions of both ends are further detected. Further, the detection unit 12 sets the positions of both end portions of each of the plurality of divided area groups along the other arrangement direction in the row direction or the column direction among the plurality of divided areas derived by the derivation unit 13 Further detect. Further, the derivation unit 13 sets the upper limit number between the respective end portions of the plurality of divided area groups along the row direction and the plurality of divided area groups along the vertical direction detected by the detection unit 12 to each other. The plurality of divided areas are derived again by equally dividing. Then, the estimation unit 15 estimates the position of the central portion in the plurality of divided areas derived again by the derivation unit 13 as the position of the hole 44A in each of the pans 44 separately for each of the row direction and the column direction.

  In addition, when the number of pans 44 accommodated in the arrival number 42 is smaller than the upper limit number, the estimation unit 15 according to the present embodiment estimates the hole 44A when the arrival number 42 is a true number. Position is assumed to be a temporary position of the hole 44A of the pan 44 accommodated in the arrival number 42. Then, the estimation unit 15 estimates that the position surrounded by an image other than the image of the pan 44 in the temporary position is a position less than a predetermined value as the position of the hole 44A in each of the pans 44.

  In the object distribution system 90 according to the present embodiment, the camera 32 acquires distance image data. Therefore, the estimation unit 15 and the like according to the present embodiment use binary image data obtained by binarizing the distance image data acquired by the camera 32 as described above. However, it is not limited to this. For example, when a camera for capturing a color image, a camera for capturing a monochrome image, or the like is applied as the camera 32, image data of multiple gradations acquired by the camera 32 is binarized with a predetermined threshold 2 Value image data may be used.

  On the other hand, the superimposing unit 17 superimposes mask data for masking the hole 44A on the position estimated by the estimating unit 15 in the binary image data, and the recognition unit 18 makes the mask data The placement position of the pan 44 is recognized using the superimposed binary image data. The recognition result of the arrangement position of the pan 44 by the recognition unit 18 is transmitted to the picking robot 30, and the recognition result received from the recognition unit 18 by the picking robot 30 is used, and the individual accommodated in the arrival number 42 is used. Picking of the bread 44 is performed.

  The acquisition unit 11 in the estimation device 20 is an example of an acquisition unit according to the present invention, the detection unit 12 is an example of a detection unit according to the present invention, and the derivation unit 13 is an example of a derivation unit according to the present invention. Moreover, the reception part 14 in the estimation apparatus 20 is an example of the reception part which concerns on this invention, and the estimation part 15 is an example of the estimation part which concerns on this invention. The recognition unit 18 of the object position recognition apparatus 10 is an example of a recognition unit according to the present invention.

  The object position recognition device 10 and the estimation device 20 can be realized, for example, by a computer 50 shown in FIG. The computer 50 includes a central processing unit (CPU) 51, a memory 52, a storage unit 53, an input unit 54, a display unit 55, and an interface (I / F) unit 56. The CPU 51, the memory 52, the storage unit 53, the input unit 54, the display unit 55, and the I / F unit 56 are connected to one another via a bus 59. The picking robot 30 and the camera 32 described above are connected to the I / F unit 56. In the present embodiment, the picking robot 30 and the camera 32 are connected to the computer 50 by wire, but the present invention is not limited to this, and a wireless connection may be made.

  In addition, the storage unit 53 can be realized by a hard disk drive (HDD), a flash memory, or the like. An object position recognition program 53A for causing the computer 50 to function as the object position recognition device 10 and the estimation device 20 is stored in the storage unit 53 as a recording medium. The CPU 51 reads the object position recognition program 53A from the storage unit 53, develops it in the memory 52, and sequentially executes the processes possessed by the object position recognition program 53A.

  The object position recognition program 53A has an acquisition process 53A1, a detection process 53A2, a derivation process 53A3, a reception process 53A4, an estimation process 53A5, a superposition process 53A6 and a recognition process 53A7. The CPU 51 operates as the acquisition unit 11 illustrated in FIG. 2 by executing the acquisition process 53A1, and operates as the detection unit 12 illustrated in FIG. 2 by executing the detection process 53A2, and executes the derivation process 53A3. Operates as the derivation unit 13 shown in FIG. Further, the CPU 51 operates as the reception unit 14 illustrated in FIG. 2 by executing the reception process 53A4, and operates as the estimation unit 15 illustrated in FIG. 2 by executing the estimation process 53A5. Furthermore, the CPU 51 operates as the superimposing unit 17 illustrated in FIG. 2 by executing the superposition process 53A6, and operates as the recognition unit 18 illustrated in FIG. 2 by executing the recognition process 53A7. Thus, the computer 50 executing the object position recognition program 53A functions as the object position recognition device 10 and the estimation device 20. The object position recognition program 53A includes an example of the estimation program of the present invention.

  On the other hand, the storage unit 53 is provided with an image data storage area 53B. The CPU 51 develops the distance image data stored in the image data storage area 53 </ b> B in the memory 52 to create distance image data.

  The object position recognition device 10 may be a personal computer or a smart terminal or the like which is a portable device equipped with a PDA (Personal Digital Assistant) function. The object position recognition apparatus 10 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an application specific integrated circuit (ASIC) or the like.

  Next, the operation of the present embodiment will be described. In the present embodiment, the worker of the distribution center stacks a plurality of receiving bunds 42 to be processed with respect to the stacking place 40A of the table 40 as described above. Thereafter, the worker causes the computer 50 to execute the object position recognition program 53A, whereby the object position recognition process shown in FIG. 4 is performed. Here, in order to avoid confusion, it is assumed here that distance image data continuously obtained by photographing with the camera 32 is stored in the image data storage area 53B of the storage unit 53 in real time.

  In step 100 of the object position recognition process, the reception unit 14 controls the display unit 55 to display an initial information input screen predetermined as a screen for inputting various initial information, and in the next step 102, The reception unit 14 waits for input of predetermined information.

  FIG. 5 shows an example of the initial information input screen according to the present embodiment. As shown in FIG. 5, on the initial information input screen according to the present embodiment, an input area 55A for inputting the maximum number of accommodation of the pan 44 to the arrival number 42, and an object to be processed in the current object position recognition process An input area 55B for inputting the number of incoming stocking weights 42 is displayed. Here, the maximum accommodation number corresponds to the upper limit number. In the object sorting system 90 according to the present embodiment, as described above, a processing target is one in which the pan 44 can be accommodated in a two-dimensional shape of the row direction and the column direction with respect to one receiving number 74. Therefore, in the initial information input screen according to the present embodiment, as the input area 55A for inputting the maximum accommodation number, the input area 55A1 for inputting the maximum accommodation number in the row direction, and the maximum accommodation number in the column direction An input area 55A2 for input is displayed.

  When the initial information input screen shown in FIG. 5 is displayed on the display unit 55, the operator uses the input unit 54 to set the maximum number of accommodation of the pan 44 in the row direction of the receiving number 42 to the input area 55A1. Input is performed, and the maximum accommodation number in the column direction is also input to the input area 55A2. In addition, the worker uses the input unit 54 to input the number of the delivery number 42 to be processed in the current object position recognition process in the input area 55B.

  When the operator completes the above input, the operator designates the “end of input” button 55C displayed on the initial information input screen by the input unit 54. When the "input end" button 55C is designated, step 102 becomes affirmation determination, and the process shifts to step 104.

  In step 104, the acquisition unit 11 determines whether or not the receiving number 42 for receiving the bread 44 is disposed at the picking location 40B, and the receiving number 42 for receiving the bread 44 is at the picking location 40B. If it is placed in the case, the determination is affirmative and the process proceeds to step 106. Note that when executing the processing of step 104, the acquisition unit 11 reads the latest distance image data stored in the image data storage area 53B. The acquisition unit 11 also generates binary image data by binarizing the read distance image data as described above. Then, the acquisition unit 11 determines the determination in step 104 by determining whether the object group region is included in the image (hereinafter referred to as “processing target image”) indicated by the generated binary image data. Do.

  In step 106, the detecting unit 12 determines that the delivery number 42 (hereinafter referred to as "processing target number") placed at the picking location 40B is the target number in the current object position recognition process. It is determined whether or not it is the last stocking weight 42 out of 42. Then, if this determination is negative, that is, if the process target load is not the last arrival load weight 42, the process proceeds to step 108. It should be noted that it is determined whether or not the number of movements of the arrival number 42 from the stacking place 40A to the picking place 40B is the initial information input, for example, as to whether or not the processing object number in step 106 is the last arrival number 42. This can be done by determining whether or not the number of incoming stocking weights 42 entered via the screen has been reached. In addition, the determination in step 106 may be performed based on whether or not the arrival number 42 has disappeared from the stacking location 40A of the table 40.

  In step 108, the detection unit 12 determines whether or not the processing object number is a regular number, and when an affirmative determination is made, the process proceeds to step 110. In addition, the determination in step 108 determines that the processing object number is positive if the processing in step 110 (regular number processing) to be described later is not executed even once, and the processing object number is positive. It is performed by determining that the order is not the correct order when the order process is performed.

  At step 110, the CPU 51 carries out the serial number processing shown in FIG.

  In step S200, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, and binarizes the distance image data as described above to generate binary image data. In the next step 202, the detection unit 12 extracts the boundary between the area of the pan 44 and the other area from the generated binary image data.

  FIG. 7 is a diagram showing an image of a plan view of the stock number 42 made to be an ideal regular stack at the left end, and a binary represented by binary image data acquired by the processing of step 200. Three examples of images (processing target images) are shown.

  As shown in the left end of FIG. 7, in the present embodiment, a total of 12 doughnut-shaped pans 44 capable of receiving four (row direction) × 3 (column direction) can be accommodated by the delivery number 42. It is done. However, as shown in the second image from the left in FIG. 7 as an example, the actual image to be processed is generally contained in the pan 44 housed in the second figure from the left due to vibration, inclination and the like at the time of transfer of the receiving number 42. It may be in contact with the Further, as shown in the second drawing from the right in FIG. 7 as an example, the processing target image may be entirely biased to one position. Furthermore, the image to be processed may be in a state where the individual pans 44 are separated as shown in the right end of FIG. 7 as an example. In the following, the processing target image shown in the second drawing from the left in FIG. 7 is called the “first image”, and the processing target image shown in the second drawing from the right in FIG. 7 is called the “second image”. The processing target image shown in the right end of FIG. 7 is referred to as a “third image”.

  With respect to such an image, in the extraction of the boundary line in step 202, as shown in the left end of FIG. 8 as an example, the pixel values from the four sides around the processing target image toward the inside of the image are A point changing from 1 "to" 0 "is detected as each point constituting the boundary. In addition, when detecting each of these points, when the noise image is included outside the area of the image of the pan 44 in the processing target image, an accurate boundary line can not be obtained. For this reason, the detection unit 12 according to the present embodiment performs noise removal processing on binary image data prior to extraction of boundary lines. In addition, as a noise removal process applied here, the conventionally known techniques, such as the method by a median filter and the method by a moving average filter, are illustrated.

  By the process of extracting the boundary line, for example, the boundary line shown in the second figure from the left in FIG. 8 is obtained for the first image. Similarly, for the second image, the boundary shown in the second figure from the right in FIG. 8 is obtained, and for the third image, the boundary shown in the right end of FIG. 8 is obtained.

  In the next step 204, the detection unit 12 specifies a circumscribed rectangular frame for the boundary obtained by the process of step 202. By this processing, for example, a circumscribed rectangular frame shown in the left end of FIG. 9 is obtained for the first image. Similarly, the circumscribed rectangular frame shown in the center of FIG. 9 is obtained for the second image, and the circumscribed rectangular frame shown in the right end of FIG. 9 is obtained for the third image.

  In the next step 206, the detection unit 12 detects the positions of both ends in each of the row direction and the column direction from the circumscribed rectangular frame obtained by the process of step 204. As described above, in the present embodiment, since the pan 44 can be accommodated in two dimensions in the row direction and the column direction in the receiving number 42, in step 206, both ends in each direction of the row direction and the column direction Although the position of a part is detected, it is not limited to this. For example, in the case where only one row of pans 44 can be accommodated in the receiving number 42, in step 206, the positions of both ends only in the row direction (the arrangement direction of the pans 44) may be detected.

  In the next step 208, the derivation unit 13 divides the distance between both ends in the row direction detected by the process of step 206 evenly by the maximum accommodation number in the row direction corresponding to the upper limit number. Derive multiple division regions for Further, the derivation unit 13 divides a plurality of divisions in the vertical direction by dividing equally between both ends in the vertical direction detected by the process of step 206 by the maximum accommodation number in the vertical direction corresponding to the upper limit number. Deriving the area. By this processing, for example, the divided area shown in the left end of FIG. 9 is obtained for the first image. Similarly, the divided area shown in the center of FIG. 9 is obtained for the second image, and the divided area shown in the right end of FIG. 9 is obtained for the third image.

  By the way, for example, when the outline of the boundary is substantially rectangular like the first image and the second image, each of the pans 44 is arranged in the divided area obtained by the above processing. It can be roughly estimated that the However, if the outline of the boundary is not rectangular (for example, trapezoidal or parallelogram) as in the third image, the divided area obtained by the above processing is It can not be estimated that each of the pans 44 is an area where it is arranged.

  Therefore, the detection unit 12 according to the present embodiment is a plurality of the plurality of divided regions derived by the derivation unit 13 along any one of the row direction and the column direction (in the present embodiment, the row direction). The positions of both ends of each of the divided area groups are further detected. In addition, the detection unit 12 selects one of a plurality of divided areas derived from the derivation unit 13 in a plurality of divided area groups along the other direction (in the present embodiment, the vertical direction) of the row direction and the column direction. The position of each end is further detected.

  Then, the derivation unit 13 according to the present embodiment detects the distance between each end of each of the plurality of divided area groups in the row direction and the plurality of divided area groups in the vertical direction detected by the detection unit 12 The plurality of divided areas are derived again by dividing equally by the corresponding upper limit number (maximum accommodation number).

  Of the processes of the detection unit 12 and the derivation unit 13 described above, the process on the divided area group along the row direction is performed as the first step, and the row direction is, for example, shown in FIG. The division area for is adjusted. Further, the processing of the divided area group along the vertical direction among the processes of the detection unit 12 and the derivation unit 13 described above is performed as a second stage on the divided areas that have undergone the first stage processing. Thereby, as shown in FIG. 11, in addition to the division area in the row direction, the division area in the column direction is adjusted.

  The divided area obtained by the above processing is a divided area that also takes into account the deviation of the arrangement position by each of the plurality of pans 44. Therefore, if the divided area is an area in which each of the pans 44 is arranged, It can be estimated with accuracy.

  In the next step 210, the estimation unit 15 sets the position of the central portion in the plurality of divided areas re-derived by the derivation unit 13 separately for each of the row direction and the column direction as the position of the hole 44A in each of the pans 44. Estimate.

  In the next step 212, the estimation unit 15 stores, in the memory 52, position information indicating the positions of the holes 44A in each of the estimated pans 44. In the next step 214, the superimposing unit 17 superimposes mask data for masking the hole 44A on the position estimated by the estimating unit 15 in the binary image data indicating the processing target image. Finish the regular stacking process. When the regular stacking process is finished, the process proceeds to step 116 of the object position recognition process shown in FIG. In the present embodiment, as mask data, a circular image of a predetermined size (diameter) is set to a value (“0” in the present embodiment) similar to the image of the pan 44 in binary image data. The data to be formed is used.

  The pan 44 corresponding to the small-sized divided area may have a large degree of compression, and the diameter of the hole 44A may be smaller. Taking this into consideration, the size of the mask data (diameter of circle) may be changed according to the size of each divided area.

  In FIG. 12, when the processing target image is the first to third images, the positions masked by the process of step 214 are shown together with the boundary lines and the divided areas. Further, in the upper part of FIG. 13, when the processing target image is the first to third images, positions to be masked by the processing of step 214 are shown together with the processing target image. Further, the lower part of FIG. 13 shows the processing target image after the mask data is superimposed on the corresponding processing target image.

  As shown in FIG. 12 and FIG. 13, although the small mask leakage is seen for the image of some pans 44 due to the superposition of the mask data of step 214, the image corresponding to the hole 44 A for the image of most of the pans 44. It can be seen that is lost (masked). Here, "mask leakage" means that a part of the area to be masked (the area of the hole 44A) can not be masked. Therefore, by performing object position recognition that recognizes the position of the pan 44 on the processing target image after the mask data is superimposed, the influence of the image of the hole 44A or the influence of the image of the hole 44A is reduced. Recognition can be performed. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  On the other hand, if a negative determination is made in step 108 of the object position recognition shown in FIG. 4, the processing object number is regarded as a fractional weight and processing proceeds to step 112, and the CPU 51 performs the fractional number processing shown in FIG. I do.

  In step 300 of round number processing, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, and binarizes the distance image data as described above to generate binary image data. In the next step 302, the detection unit 12 reads out the position information stored by the processing of step 212 of the regular number processing performed most recently.

  In the next step 304, the superimposing unit 17 superimposes the mask data on the position indicated by the position information read out in the process of step 302 on the binary image data generated by the process of step 300.

  FIG. 15 shows three examples of binary images (processing target images) indicated by binary image data acquired by the process of step 300.

  The image at the left end of FIG. 15 is an image in a state in which two pans 44 in the right half of the first row are picked with respect to the processing target weight which is the processing target image which is the first image. (Hereafter, it is called "4th image."). Further, in the central diagram of FIG. 15, the processing target image in the regular stacking state is all four in the first row and two in the central portion of the second row with respect to the processing row number being the second image. An image (hereinafter referred to as a “fifth image”) in a state where a total of six pans 44 have been picked up is shown. Furthermore, in the rightmost drawing of FIG. 15, the processing target image in the regular stacking state is the third pan 44 on the right side of the processing target stacking which is the third image, and the second pan An image (hereinafter referred to as a “sixth image”) in a state where two pans 44 in total in the left half have been picked is shown. Further, in FIG. 16, when the processing target image is the fourth to sixth images, the position to be masked by the process of step 304 is shown together with the processing target image.

  As shown in FIG. 16, in the process of step 304, the mask data is superimposed to the area where the pan 44 does not exist actually, so in this area the area where the pan 44 does not exist and the area where the mask data is superimposed. There is a high possibility that the pan 44 is misjudged as being located. Therefore, in the next step 306, the superimposing unit 17 determines, for each of the arranged mask data, whether or not the arrangement of the mask data performed in the process of step 304 is appropriate.

  In the present embodiment, it is determined as follows whether or not the arrangement of mask data is appropriate. That is, of the circumference of the area of the mask according to the arranged mask data, a pixel (in the present embodiment, “1” in the present embodiment) other than the image of the pan 44 in the processing target image It is determined that the ratio enclosed by the black pixel) is appropriate if it is less than a predetermined value. In the present embodiment, 70% is applied as the predetermined value. Therefore, for example, as shown in FIG. 17, when the ratio is 100% and 80%, it is determined that the arrangement of the mask data is not appropriate, and the ratio is 60%, 20% and 0% ( In the case of no contact with the black pixel at all), it is determined that the arrangement of mask data is appropriate.

  In the next step 308, the superimposing unit 17 removes from the binary image data the mask data determined to be invalid in the arrangement by the process of step 306, and then ends the number-n-th step process. When the round number processing is completed, the process proceeds to step 116 shown in FIG.

  In FIG. 18, when the processing target image is the fourth to sixth images, positions to be finally masked by the processing of steps 304 to 308 are shown together with the processing target image. Further, FIG. 19 shows the processing target image after the mask data is superimposed on the processing target images of the fourth to sixth images. In FIG. 18, the position of the mask according to the mask data removed by the process of step 308 is indicated by a white broken line.

  As shown in FIGS. 18 and 19, the fourth to sixth images are removed from the mask data of the area where the image of pan 44 does not exist by the processing of steps 304 to 308, and the image of pan 44 is The mask data is superimposed only on the area. As a result, although the area where there is no pan 44 is not masked and a small mask leak is seen for the image of some pans 44, the image corresponding to the hole 44A is lost for the image of most pans 44 ( It is understood that it is masked. Therefore, by performing object position recognition that recognizes the position of the pan 44 on these processing target images, recognition is performed that is not influenced by the image of the hole 44A or that the influence of the image of the hole 44A is reduced. Can. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  On the other hand, if a positive determination is made in step 106 of object position recognition shown in FIG. 4, the processing target number is the last arrival number among the arrival number 42 targeted by the current object position recognition processing. 42 (hereinafter referred to as "final weight") and the process moves to step 114.

  At step 114, the CPU 51 performs the final stacking process shown in FIG. The following describes the final stacking process. In addition, in order to avoid confusion, in the present embodiment, a case where an ideal arrangement position of the pan 44 in the final stack (hereinafter, referred to as “final stack arrangement position”) is set in advance will be described.

  In step 400 of the final stacking process, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, and binarizes the distance image data as described above to generate binary image data. In the next step 402, the detection unit 12 reads out the position information stored by the processing of step 212 of the regular number processing performed most recently.

  In the next step 404, the superimposing unit 17 superimposes the mask data on the position indicated by the position information read out in the process of step 402 on the binary image data generated by the process of step 400.

  In the next step 406, the superimposing unit 17 removes mask data corresponding to a position different from the final stacking position from the mask data arranged by the processing of step 404, and then ends the final stacking process. Do. When the final stacking process is completed, the process proceeds to step 116 of the object position recognition process shown in FIG.

  FIG. 21 is a diagram for explaining the final stacking process in the case where the processing target image in the regular number stacking process most recently executed is the third image. In this case, the processing target image in the nearest rank process performed most recently is the image shown at the left end of FIG. 21 and the mask data is in the position shown in the second figure from the left in FIG. It is superimposed.

  In this case, the processing target image indicated by the binary image data acquired by the process of step 400 of the final stacking process is, for example, the second image from the right in FIG. The second drawing from the right in FIG. 21 shows the case where there is no pan 44 of the first stage in the final gear, and a total of eight pans 44 of the second to third stages are accommodated. ing. In this case, in the process of step 406 of the final stacking process, as shown in the right end of FIG. 21, all mask data superimposed in the first stage is removed, and the second to third stages are arranged. Mask data remains. As a result, although an area without the pan 44 is not masked and a small mask leak occurs in the image of a part of the pan 44, an image corresponding to the hole 44A disappears in the image of the most pan 44. Therefore, by performing object position recognition for recognizing the position of the pan 44 with respect to the processing target image, it is possible to perform recognition that is not influenced by the image of the hole 44A or that the influence of the image of the hole 44A is reduced. it can. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  In step 116 of the object position recognition process shown in FIG. 4, the recognition unit 18 performs binary image data (hereinafter referred to as “masked”) on which the mask data is superimposed by the process of any of the above steps 110, 112 and 114. Image position recognition is performed to recognize the position of the pan 44 contained in the processing target stack using the image data. In the present embodiment, as the object position recognition, conventionally known object position recognition is performed to recognize the position of the pan without the hole 44A or the recess in the central portion. Note that in the object position recognition according to the present embodiment, histograms indicating the accumulated values of the number of white pixels in each of the horizontal and vertical directions of the processing target image indicated by the masked image data are created for each direction. Then, the position of each pan 44 is recognized as a boundary area between the adjacent pans 44 in which the valleys of the horizontal and vertical histograms are adjacent to each other. However, object position recognition is not limited to this method, and any method may be applied as long as it is a method of recognizing the position of each of a plurality of objects using binary image data.

  By the way, in object position recognition, the position of the pan 44 may be erroneously recognized due to various causes. Hereinafter, a specific example of the cause of erroneous recognition in object position recognition will be described.

  For example, as shown in the left drawing of FIG. 22 as an example, if the area of the mask indicated by the mask data is smaller than the area of the image of the hole 44A of the pan 44, erroneous recognition may occur in object position recognition. . This is because an area between the image of the pan 44 and the image by the mask data may be erroneously recognized as the space between the adjacent pans 44. Also, as shown in the right drawing of FIG. 22 as an example, if the area of the mask indicated by the mask data is larger than the area of the image of the pan 44, erroneous recognition may occur in object position recognition. This is because the space between adjacent pans 44 may not be distinguishable.

  In order to solve this problem, the size of the mask indicated by the mask data may be specified by the operator. As an example of this form, the size of the mask is input by the operator via the input unit 54, and the size of the mask input by the operator via the input unit 54 is received by the reception unit 14 Is illustrated. In this example, mask data corresponding to the size of the mask received by the receiving unit 14 is generated by the superimposing unit 17 and applied to the regular number processing, the fractional number counting processing, and the final number counting processing.

  In addition, as a specific example of the cause of erroneous recognition in object position recognition, as shown in FIG. 23 as an example, due to the movement of the arrival number 42, only a part of the pan 44 in the row direction or the column direction Even when a large positional deviation occurs, an erroneous recognition may occur in object position recognition. Furthermore, as shown in FIG. 24 as an example, even when positional deviation occurs in only a part of the pans 44 due to the previous picking, the number of mask data may be greater than the number of pans 44 in the fractional serial processing described above Also in this case, there is a possibility of misrecognition in object position recognition.

  In order to cope with such erroneous recognition to some extent, in the object position recognition processing according to the present embodiment, when an erroneous recognition occurs in the object position recognition in step 116, unlike the object position recognition, the distance image data Perform object position recognition directly using

  That is, in the next step 118, the recognition unit 18 determines whether or not an erroneous recognition has occurred in the object position recognition in step 116. If a negative determination is made, the process proceeds to step 120. Note that the recognition unit 18 according to the present embodiment determines whether the determination as to whether or not an erroneous recognition has occurred in step 118 is not identical to the number of pans 44 where the number of positions of the recognized pan 44 is assumed. It is done by deciding whether or not. Here, the assumed number of pans 44 is the number of pans 44 picked up by the picking robot 30 from the number of the processing target so far from the number of pans 44 when the processing target number is the normal number. It is obtained by subtracting the number 44.

  In step 120, the recognition unit 18 transmits position information indicating the position of the pan 44 recognized by the object position recognition in step 116 to the picking robot 30, and then proceeds to step 132. When the position information is received from the object position recognition apparatus 10, the picking robot 30 takes out one pan 44 from the processing target stack based on the position of the pan 44 indicated by the position information, and sets the pan 44 to the shipping stack weight 48. To accommodate.

  On the other hand, if an affirmative determination is made in step 118, it is considered that an erroneous recognition has occurred in the object position recognition in step 116, and the process proceeds to step 122, and the acquisition unit 11 obtains the latest distance image data from the image data storage area 53B. Read out. In addition, in step 122, the detection unit 12 reads out the position information stored by the processing of step 212 of the nearest number process performed most recently. Then, in step 122, as shown in FIG. 25 as an example, the superimposing unit 17 superimposes mask data on the read distance image data as follows. That is, cylindrical shapes having different upper and lower surface areas of a predetermined size (including the area and height when viewed in plan) centering on the position indicated by the position information of the distance image indicated by the distance image data Distance image data showing a shape obtained by cutting the upper side of the cone horizontally and removing it is superimposed as mask data.

  In the mask data used for the distance image data, as in the case of the mask data used for the binary image data, erroneous recognition occurs in the object position recognition depending on the size of the hole of the pan to be recognized. There is a possibility that Therefore, the size of mask data used for distance image data may also be specified by the operator.

  In the next step 124, the recognition unit 18 performs object position recognition (hereinafter referred to as "second object position recognition") on the distance image data on which the mask data is superimposed by the above processing. In the second object position recognition according to the present embodiment, as shown in FIG. 26 as an example, a histogram in the horizontal direction and a histogram in the vertical direction defined below are created, and the pan 44 has valleys adjacent to each other. The position of each pan 44 is recognized as a boundary area between Here, the histogram in the horizontal direction is obtained by averaging or integrating the pieces of distance information in the horizontal direction at corresponding vertical positions. Further, the histogram in the vertical direction is obtained by averaging or integrating the pieces of distance information in the vertical direction at the corresponding horizontal position. Here, the distance from the bottom surface (the surface in contact with the lower surface of the pan 44) is applied as the distance information, based on the bottom surface (the surface in contact with the lower surface of the pan 44). However, the second object position recognition is not limited to this method, and any method may be applied as long as it is a method of recognizing the positions of a plurality of objects using distance image data.

  In the next step 126, the recognition unit 18 determines whether or not an erroneous recognition has occurred in the second object position recognition in step 124. If a negative determination is made, the process proceeds to step 128. Note that the recognition unit 18 according to the present embodiment also determines whether or not the number of recognized positions of the pan 44 is not identical to the expected number of pans 44 whether or not the erroneous recognition in step 126 has occurred. It is done by deciding whether or not.

  In step 128, the recognition unit 18 transmits position information indicating the position of the pan 44 recognized by the second object position recognition in step 124 to the picking robot 30, and then proceeds to step 132. The fact that the picking robot 30 performs the picking operation of one pan 44 in response to this is as described in the description of step 120.

  In step 132, the recognition unit 18 determines whether picking has ended for all the pans 44 of the receiving number 42 stacked at the stacking location 40A, and if a negative determination is made, the process returns to step 104. . Further, when the determination is affirmative here, the present object position recognition processing is ended.

  On the other hand, if an affirmative determination is made in step 126, it is considered that an erroneous recognition has occurred in the second object position recognition, and the process proceeds to step 130, and the recognition unit 18 executes a predetermined error process, The object position recognition process ends. In the object position recognition process according to the present embodiment, the process of causing the display unit 55 to display information indicating that an erroneous recognition has occurred is applied as the error process of step 130, but the present invention is limited thereto. is not. For example, another form may be applied to notify that an erroneous recognition has occurred, such as ringing a buzzer.

  As described above, in the present embodiment, the upper limit number of objects in the arrangement direction is stored in the container (the delivery number 42) that can accommodate a plurality of objects (pans 44) having holes in the center in the predetermined arrangement direction. The image data of the image which image | photographed the object group area | region containing all the objects in the accommodated state is acquired. Further, in the present embodiment, the positions of both ends in the arrangement direction of the object group region indicated by the acquired image data are detected. Further, in the present embodiment, a plurality of divided areas are derived by equally dividing between the detected both ends by the above upper limit number. Furthermore, in the present embodiment, the position of the central portion in the arrangement direction in the plurality of derived divided regions is estimated as the position of the hole in each of the objects. Therefore, according to the present embodiment, the position of the object can be recognized in consideration of an unexpected deviation or spread of the object. As a result, the positions of a plurality of objects having holes in the central portion can be recognized with high accuracy. can do.

  In particular, in the present embodiment, a plurality of containers can be accommodated in each of the two directions, the row direction and the column direction, with the objects in the arrangement direction. Further, in the present embodiment, the positions of the both ends of the object group area are detected separately for each of the row direction and the column direction. Further, in the present embodiment, a plurality of divided areas are derived by equally dividing the detected both ends by the corresponding upper limit number separately for each of the row direction and the column direction. And in this embodiment, the position of the central part in the row direction and the central part in the column direction in the plurality of derived divided regions is estimated to be the position of the hole in each of the objects separately for each of the row direction and the column direction. Therefore, according to the present embodiment, the present invention can be applied to a form in which an object can be accommodated two-dimensionally in a plan view with respect to a container.

  In particular, in the present embodiment, the positions of the both ends of each of the plurality of divided area groups along one of the row direction and the column direction among the plurality of derived divided areas are further detected. There is. Further, in the present embodiment, the positions of the both ends of each of the plurality of divided area groups along the other arrangement direction in the row direction or the column direction among the plurality of derived divided areas are further detected. There is. Further, in the present embodiment, between the two ends of each of a plurality of divided area groups along the row direction and a plurality of divided area groups along the vertical direction detected are equally divided according to the number of corresponding objects. Thus, a plurality of divided areas are derived again. And in this embodiment, the position of the central part in the row direction and the central part in the column direction in the plurality of divided regions derived again is estimated as the position of the hole in each of the objects separately for each of the row direction and the column direction. Therefore, according to the present embodiment, even when the position of a part of the objects is shifted with respect to at least one of the row direction and the column direction, each position of the plurality of objects is recognized with high accuracy. can do.

  Further, in the present embodiment, when the number of objects contained in the container is less than the upper limit number, the position of the hole estimated when the above-described upper limit number of objects are contained in the container is other than the image of the object The position where the proportion enclosed by the image of is less than a predetermined value is estimated as the position of each hole of the object. Therefore, according to the present embodiment, even when the number of objects contained in the containing body is smaller than the upper limit number, the positions of the plurality of objects can be recognized with high accuracy.

  Further, in the present embodiment, distance image data is generated by photographing, and based on the distance image data, binary image data for distinguishing an area of an object from another area is acquired as the image data. Therefore, according to the present embodiment, it is possible to recognize each position of a plurality of objects with high accuracy using binary image data based on distance image data.

  Further, in the present embodiment, the input of the upper limit number is accepted, and the divided area is derived using the accepted upper limit number. Therefore, according to the present embodiment, a plurality of types of containers with different upper limit numbers can be targeted.

  Furthermore, in the present embodiment, an annular object is applied as the object. Therefore, according to the present embodiment, it is possible to recognize the position of each of a plurality of objects with high accuracy for a toroidal object.

  In the above-described embodiment, it is assumed that the weight is in the normal weight state (the state in which the upper limit number of objects is stored) when performing the normal number processing, but the present invention is not limited to this. . Even if the weight is less than the true weight, the objects within the weight are offset in the row direction and the column direction, and if the number of objects is known, the true weight processing is feasible. is there.

  Moreover, although the said embodiment demonstrated the case where the pan 44 which has the hole 44A in the center was applied as an object of this invention, it is not limited to this. For example, a tire, a nut, a washer, an O-ring, or the like may be applied as the object of the present invention, in addition to a food product such as a donut or a bread in which a central portion is recessed. Also in this case, the same effects as the above embodiment can be obtained.

  Moreover, although the case where the pan 44 was accommodated in the two-dimensional form in the delivery number 42 was described in the said embodiment, it is not limited to this. For example, the pan 44 may be accommodated in a one-dimensional shape, that is, only one row in the receiving number 42.

  Further, in the above embodiment, in the processing of step 208 of the regular number processing, after the division areas shown in FIG. 9 are derived, adjustment of the division areas shown in FIGS. 10 and 11 is described. However, it is not limited to this. For example, as in the case of the first image and the second image, the adjustment of the divided area may not be performed when the outline of the boundary is substantially rectangular.

  Moreover, although the case where the camera which acquires distance image data was applied as the camera 32 was demonstrated in the said embodiment, it is not limited to this. For example, as the camera 32, a camera for capturing a commercially available color image or monochrome image may be applied. In this case, since data of a plurality of gradations is acquired as image data by the camera 32, data obtained by binarizing the data of the plurality of gradations with a predetermined threshold value is used as image data. In this case, as the camera 32 can be used a camera for capturing relatively inexpensive color images and monochrome images, the object sorting system can be constructed at low cost.

  In the above embodiment, after the mask data is superimposed on all the positions estimated by the estimation unit 15 by the superposition unit 17, it is determined whether or not the arrangement of the mask based on the mask data is appropriate. Although explained, it is not limited to this. For example, the superimposing unit 17 determines in advance whether or not the arrangement of the mask is appropriate for each of the positions estimated by the estimating unit 15, and superimposes mask data only on the positions determined to be appropriate. May be

  In the above embodiment, the camera 32 is provided above the picking place 40B, and the picking robot 30 picks the pan 44 from the receiving number 42 provided at the picking place 40B. However, the present invention is not limited thereto. It is not a thing. For example, the camera 32 may be provided above the stacking place 40A, and the picking robot 30 may pick the pan 44 from the receiving number 42 provided at the stacking place 40A. Furthermore, the camera 32 is provided above both the stacking station 40A and the picking station 40B, and the picking robot 30 selectively picks the pan 44 from the stocking weights 42 provided at any of the stacking station 40A and the picking station 40B. You may do it.

  Further, in the above embodiment, the aspect in which the object position recognition program 53A is stored (installed) in the storage unit 53 in advance has been described, but the present invention is not limited to this. For example, the object position recognition program 53A may be read from a recording medium such as a compact disc read only memory (CD-ROM) or a digital versatile disc ROM (DVD-ROM) into the storage unit 53 of the computer 50 via the medium read / write device. It may be downloaded from the external device to the storage unit 53 via the network.

  All documents, patent applications and technical standards described herein are as specific as when each document, patent application and technical standard is specifically and individually indicated to be incorporated by reference. Incorporated by reference in the book.

  Further, the following appendices will be disclosed regarding the above embodiment.

(Supplementary Note 1)
The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view An acquisition unit configured to acquire image data of an image obtained by photographing an object group area including all the objects contained in the container in a state in which the predetermined number of objects are accommodated in the arrangement direction;
A detection unit that detects positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A deriving unit that derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detection unit by the predetermined number;
An estimation unit configured to estimate the position of the central portion in the arrangement direction in the plurality of divided areas derived by the derivation unit as the position of the hole or the recess in each of the objects;
Estimator including.

(Supplementary Note 2)
A plurality of the containers can be accommodated in each of the row direction and the column direction as the predetermined arrangement direction.
The detection unit detects positions of the both ends of the object group area separately for each of the row direction and the column direction,
The deriving unit derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detecting unit by the corresponding predetermined number separately for each of the row direction and the column direction.
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in the plurality of divided areas derived by the derivation unit separately for each of the row direction and the column direction as the holes in each of the objects Or estimate the position of the recess,
The estimation device according to appendix 1.

(Supplementary Note 3)
The detection unit is a position of the both ends of each of a plurality of divided area groups along one of the row direction and the column direction among the plurality of divided areas derived by the derivation unit. And the positions of the both ends of each of a plurality of divided area groups along the arrangement direction in the row direction or the column direction among the plurality of divided areas derived by the derivation unit are further detected And
The deriving unit corresponds to the object corresponding between the both ends of each of a plurality of divided area groups along the row direction and a plurality of divided area groups along the vertical direction detected by the detection unit. Deriving the plurality of divided areas again by equally dividing by the number of
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in each of the plurality of divided regions derived again by the derivation unit separately for each of the row direction and the column direction in each of the objects. Estimate the position of the hole or recess
The estimation device according to appendix 2.

(Supplementary Note 4)
When the number of the objects contained in the container is smaller than the predetermined number, the estimation unit estimates the position of the hole or the recess estimated when the predetermined number is stored in the container. Among them, a position at which the proportion surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects.
The estimation device according to any one of supplementary notes 1 to 3.

(Supplementary Note 5)
The imaging unit generates distance image data by imaging.
The acquisition unit acquires, as the image data, binary image data that distinguishes the area of the object from another area based on the distance image data.
The estimation device according to any one of supplementary notes 1 to 4.

(Supplementary Note 6)
The system further includes a receiving unit that receives the predetermined number of inputs,
The derivation unit derives the divided area using the predetermined number received by the reception unit.
The estimation device according to any one of supplementary notes 1 to 5.

(Appendix 7)
The object is annular,
The estimation device according to any one of supplementary notes 1 to 6.

(Supplementary Note 8)
The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view An acquisition unit configured to acquire image data of an image obtained by photographing an object group area including all the objects contained in the container in a state in which the predetermined number of objects are accommodated in the arrangement direction;
A detection unit that detects positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A deriving unit that derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detection unit by the predetermined number;
An estimation unit configured to estimate the position of the central portion in the arrangement direction in the plurality of divided areas derived by the derivation unit as the position of the hole or the recess in each of the objects;
A recognition unit that recognizes the arrangement position of the object using the image data obtained by masking the hole or the recess based on the position estimated by the estimation unit in the image data;
An object position recognition device including:

(Appendix 9)
A plurality of the containers can be accommodated in each of the row direction and the column direction as the predetermined arrangement direction.
The detection unit detects positions of the both ends of the object group area separately for each of the row direction and the column direction,
The deriving unit derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detecting unit by the corresponding predetermined number separately for each of the row direction and the column direction.
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in the plurality of divided areas derived by the derivation unit separately for each of the row direction and the column direction as the holes in each of the objects Or estimate the position of the recess,
The object position recognition device according to appendix 8.

(Supplementary Note 10)
The detection unit is a position of the both ends of each of a plurality of divided area groups along one of the row direction and the column direction among the plurality of divided areas derived by the derivation unit. And the positions of the both ends of each of a plurality of divided area groups along the arrangement direction in the row direction or the column direction among the plurality of divided areas derived by the derivation unit are further detected And
The deriving unit corresponds to the object corresponding between the both ends of each of a plurality of divided area groups along the row direction and a plurality of divided area groups along the vertical direction detected by the detection unit. Deriving the plurality of divided areas again by equally dividing by the number of
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in each of the plurality of divided regions derived again by the derivation unit separately for each of the row direction and the column direction in each of the objects. Estimate the position of the hole or recess
An object position recognition device according to appendix 9.

(Supplementary Note 11)
When the number of the objects contained in the container is smaller than the predetermined number, the estimation unit estimates the position of the hole or the recess estimated when the predetermined number is stored in the container. Among them, a position at which the proportion surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects.
An object position recognition device according to any one of supplementary notes 8 to 10.

(Supplementary Note 12)
The imaging unit generates distance image data by imaging.
The acquisition unit acquires, as the image data, binary image data that distinguishes the area of the object from another area based on the distance image data.
An object position recognition device according to any one of supplementary notes 8 to 11.

(Supplementary Note 13)
The system further includes a receiving unit that receives the predetermined number of inputs,
The derivation unit derives the divided area using the predetermined number received by the reception unit.
An object position recognition device according to any one of supplementary notes 8 to 12.

(Supplementary Note 14)
The object is annular,
The object position recognition device according to any one of appendices 8 to 13.

(Supplementary Note 15)
The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view Image data of an image obtained by photographing an object group area including all the objects contained in the container in a state of being accommodated in a predetermined number in the arrangement direction is acquired,
Detecting positions of both ends in the arrangement direction of the object group area indicated by the acquired image data;
A plurality of divided areas are derived by equally dividing between the detected both ends by the predetermined number,
Estimating the position of the central portion in the arrangement direction in the plurality of derived divided regions as the position of the hole or recess in each of the objects;
Program for making a computer execute processing including

(Supplementary Note 16)
A plurality of the containers can be accommodated in each of the row direction and the column direction as the predetermined arrangement direction.
The position of the both ends of the object group area is detected separately for each of the row direction and the column direction,
A plurality of divided areas are derived by equally dividing between the detected both ends by the corresponding predetermined number separately for each of the row direction and the column direction.
In each of the row direction and the column direction, the position of the row direction central portion and the column direction central portion in the plurality of derived divided regions is estimated as the position of the hole or the recess in each of the objects.
A presumption program given in appendix 15.

(Supplementary Note 17)
The positions of the both ends of each of the plurality of divided area groups in the row direction and the column direction among the plurality of divided areas which are derived, and the plurality of divided areas which are derived And detecting the positions of the both ends of each of a plurality of divided area groups along the arrangement direction in the row direction or the column direction.
By dividing equally between the two ends of each of the plurality of divided area groups along the row direction and the plurality of divided area groups along the vertical direction, the detected number of objects is the same. Derive multiple divisions again,
In each of the row direction and the column direction, the positions of the row direction central portion and the column direction central portion in the plurality of divided regions derived again are estimated to be the positions of the holes or recesses in each of the objects.
The estimation program according to appendix 16.

(Appendix 18)
When the number of the objects contained in the container is smaller than the predetermined number, the position of the object in the hole or the recess estimated when the predetermined number of the objects are contained in the container Estimating the position of the hole or the recess in each of the objects as the position where the proportion surrounded by images other than the image is less than a predetermined value,
APPENDIX 15. The estimation program according to any one of appendixes 15 to 17.

(Appendix 19)
The imaging unit generates distance image data by imaging.
The binary image data for distinguishing the area of the object from the other area is acquired as the image data based on the distance image data.
Appendix 15. The estimation program according to any one of appendixes 15 to 18.

(Supplementary Note 20)
Accept the predetermined number of inputs,
Deriving the divided area using the received predetermined number
The estimation program according to any one of appendices 15 to 19.

(Supplementary Note 21)
The object is annular,
Appendix 15. The estimation program according to any one of appendices 15 to 20.

DESCRIPTION OF SYMBOLS 10 object position recognition apparatus 11 acquisition part 12 detection part 13 derivation part 14 reception part 15 estimation part 17 superimposition part 18 recognition part 20 recognition device 20 picking robot 32 camera 42 delivery number 44 pan 44A hole 48 shipping number 50 computer 51 CPU
52 memory 53 storage unit 53A object position recognition program 53A1 acquisition process 53A2 detection process 53A3 derivation process 53A4 reception process 53A5 estimation process 53A6 superposition process 53A7 recognition process 53B image data storage area 54 input unit 55 display unit 90 object distribution system

Claims (8)

The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view An acquisition unit configured to acquire image data of an image obtained by photographing an object group area including all the objects contained in the container in a state in which the predetermined number of objects are accommodated in the arrangement direction;
A detection unit that detects positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A deriving unit that derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detection unit by the predetermined number;
An estimation unit configured to estimate the position of the central portion in the arrangement direction in the plurality of divided areas derived by the derivation unit as the position of the hole or the recess in each of the objects;
Estimator including. A plurality of the containers can be accommodated in each of the row direction and the column direction as the predetermined arrangement direction.
The detection unit detects positions of the both ends of the object group area separately for each of the row direction and the column direction,
The deriving unit derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detecting unit by the corresponding predetermined number separately for each of the row direction and the column direction.
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in the plurality of divided areas derived by the derivation unit separately for each of the row direction and the column direction as the holes in each of the objects Or estimate the position of the recess,
The estimation device according to claim 1. The detection unit is a position of the both ends of each of a plurality of divided area groups along one of the row direction and the column direction among the plurality of divided areas derived by the derivation unit. And the positions of the both ends of each of a plurality of divided area groups along the arrangement direction in the row direction or the column direction among the plurality of divided areas derived by the derivation unit are further detected And
The deriving unit corresponds to the object corresponding between the both ends of each of a plurality of divided area groups along the row direction and a plurality of divided area groups along the vertical direction detected by the detection unit. Deriving the plurality of divided areas again by equally dividing by the number of
The estimation unit determines the positions of the central portion in the row direction and the central portion in the column direction in each of the plurality of divided regions derived again by the derivation unit separately for each of the row direction and the column direction in each of the objects. Estimate the position of the hole or recess
The estimation device according to claim 2. When the number of the objects contained in the container is smaller than the predetermined number, the estimation unit estimates the position of the hole or the recess estimated when the predetermined number is stored in the container. Among them, a position at which the proportion surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects.
The estimation device according to any one of claims 1 to 3. The imaging unit generates distance image data by imaging.
The acquisition unit acquires, as the image data, binary image data that distinguishes the area of the object from another area based on the distance image data.
The estimation apparatus in any one of Claims 1-4. The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view An acquisition unit configured to acquire image data of an image obtained by photographing an object group area including all the objects contained in the container in a state in which the predetermined number of objects are accommodated in the arrangement direction;
A detection unit that detects positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A deriving unit that derives a plurality of divided areas by equally dividing the distance between the both ends detected by the detection unit by the predetermined number;
An estimation unit configured to estimate the position of the central portion in the arrangement direction in the plurality of divided areas derived by the derivation unit as the position of the hole or the recess in each of the objects;
A recognition unit that recognizes the arrangement position of the object using the image data obtained by masking the hole or the recess based on the position estimated by the estimation unit in the image data;
An object position recognition device including: The image processing apparatus further includes a superimposing unit that performs masking with predetermined mask data at a position estimated by the estimation unit and at a position where it is determined that masking is appropriate.
The recognition unit recognizes the arrangement position of the object using the image data in which the hole or the recess is masked by the overlapping unit.
The object position recognition device according to claim 6. The object is accommodated in a container capable of accommodating a plurality of objects provided with holes or recesses in the central portion in a predetermined arrangement direction intersecting with the direction of the plan view in a direction in which the holes or the recesses can be visually recognized in plan view Image data of an image obtained by photographing an object group area including all the objects contained in the container in a state of being accommodated in a predetermined number in the arrangement direction is acquired,
Detecting positions of both ends in the arrangement direction of the object group area indicated by the acquired image data;
A plurality of divided areas are derived by equally dividing between the detected both ends by the predetermined number,
Estimating the position of the central portion in the arrangement direction in the plurality of derived divided regions as the position of the hole or recess in each of the objects;
Program for making a computer execute processing including