JP2017054199A - Estimation apparatus, object position recognition apparatus, and estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize positions of a plurality of objects each having a hole or a recess at the center, with high accuracy.SOLUTION: An acquisition unit 11 of an estimation apparatus 20 acquires image data of an image formed by a camera 32 imaging a bread group area including all breads when a predetermined number of breads are stored in a predetermined arrangement direction in a supply food tray which can store a plurality of breads each having a hole at the center in the arrangement direction. A detection unit 12 detects positions of both ends in the arrangement direction of the bread group area represented by the acquired image data. A deriving unit 13 derives a plurality of sub areas by equally dividing a part between the detected both ends by the predetermined number. An estimation unit 15 estimates a position of the center in the arrangement direction of each of the derived sub areas, as a position of the hole in each of the breads.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an estimation device, an object position recognition device, and an estimation program.

  Conventionally, as a technique for inspecting the object using binary image data obtained by binarizing image data obtained by photographing an object with a camera, a mark applied to the object surface is photographed with a camera, There has been a technique for inspecting the shape of a stamp by processing the obtained image.

  According to this technique, a marking portion is masked from an input image photographed by a camera to obtain a luminance distribution of a background portion, and a threshold value for binarization processing is set based on the luminance distribution. In this technique, the input image is binarized according to the set threshold value, and the quality of the stamped shape is determined using the light and dark binary image data obtained thereby.

JP 2006-118895 A

  By the way, in a state where a plurality of objects are accommodated in the container, there are cases where it is desired to take out each object from the container by a robot. For example, there is an example of a delivery center that receives a large number of breads from a bakery factory and distributes and distributes the breads required for each store to a store such as a convenience store or a supermarket. That is, at the delivery center, breads are picked up one by one from the incoming weight received from the bakery factory by the picking robot, and finally the number of breads requested from each store is shipped for each store. This is the case when you want to keep it in a safe place.

  In this case, it is necessary to recognize the position of each object in the container. As this technique, a technique for recognizing the position of each object using image data obtained by photographing the plurality of objects can be considered. .

  However, with this technique, it has been difficult to accurately recognize the position of each of a plurality of objects having holes or recesses of various shapes such as an annular shape and a rectangular shape at the center. This is because it is difficult to distinguish the region of the hole or recess from the region of the gap between the objects. In addition, as an object which has a hole or a recessed part in a center part, in addition to foods, such as an annular | circular (doughnut-shaped) bread manufactured in the bread-making factory mentioned above, a donut, a bread | pan with a hollow center part, a tire, a nut An annular member or the like is exemplified.

  In order to solve this problem, it may be possible to recognize the position of each object by masking the position corresponding to the hole or recess of the object with respect to the image of the object by using the conventional technique. However, in this case, if the position of each object deviates from the assumed position for some reason, masking cannot be performed at the correct position, and as a result, the recognition accuracy of the position of each object is significantly reduced. To do.

  Note that the above problems occur not only in the case where a plurality of objects are accommodated in the container, but also in applications where the positions of other objects are recognized, not only when each object is to be removed from the container by the robot. It is a problem to get.

  An object of the present invention is, as one aspect, to make it possible to recognize each position of a plurality of objects having a hole or a recess in a central portion with high accuracy.

  In one aspect, all the objects accommodated in the container in a state in which a predetermined number of objects having holes or recesses provided in the central part are accommodated in a predetermined arrangement direction intersecting the direction in plan view are accommodated in the container. An acquisition unit that acquires image data of an image obtained by imaging the object group region including the object by the imaging unit. Here, the container is configured to be capable of accommodating a plurality of the objects in the arrangement direction in such a direction that the holes or the recesses are visible in a plan view. Moreover, the detection part which detects the position of the said arrangement direction both ends of the said object group area | region which the image data acquired by the said acquisition part shows is provided. In addition, a derivation unit that derives a plurality of divided regions by equally dividing the end portions detected by the detection unit by the predetermined number is provided. Moreover, the estimation part which estimates the position of the said array direction center part in the some division area derived | led-out by the said deriving part as the position of the said hole or recessed part in each of the said object is provided.

  As one aspect, there is an effect that the position of each of a plurality of objects having a hole or a recess at the center can be recognized with high accuracy.

It is a top view (partial block diagram) showing an example of the composition of the object distribution system concerning an embodiment. It is a functional block diagram of the object position recognition apparatus concerning an embodiment. It is a block diagram which shows schematic structure of the computer which concerns on embodiment. It is a flowchart which shows an example of the object position recognition process which concerns on embodiment. It is the schematic which shows an example of the initial stage information input screen which concerns on embodiment. It is a flowchart which shows an example of the normal number weight process which concerns on embodiment. FIG. 5 is a diagram for explaining a positive weight process according to the embodiment, and is a diagram illustrating an example of a binary image indicated by a positive weight binary image data. It is a figure with which it uses for description of the normal number weight process which concerns on embodiment, and is a figure where it uses for description of the extraction method of a boundary line. It is a figure with which it uses for description of the normal number weight process which concerns on embodiment, and is a figure which shows the example of a circumscribed rectangular frame and a division area. It is a figure with which it uses for description of the normal number weight process which concerns on embodiment, and is a figure where it uses for description of adjustment of the division area of a row direction. It is a figure with which it uses for description of the normal number weight process which concerns on embodiment, and is a figure where it uses for description of adjustment of the division area of a column direction. FIG. 5 is a diagram for explaining a normal number weight process according to the embodiment, and is a diagram illustrating an example of an arrangement position of mask data. FIG. 5 is a diagram for explaining the normal number weight processing according to the embodiment, and is a diagram illustrating an example of superposition of mask data and binary image data after superposition of mask data. It is a flowchart which shows an example of the fraction number process which concerns on embodiment. It is a figure for explanation of fraction number processing according to the embodiment, and is a diagram illustrating an example of a binary image indicated by binary image data of fraction number. FIG. 6 is a diagram for explaining the fraction number processing according to the embodiment and is a diagram illustrating an example of a temporary arrangement position of mask data. It is a figure with which it uses for description of the fraction number processing which concerns on embodiment, and is a figure where it uses for description of the removal method of unnecessary mask data. FIG. 10 is a diagram for explaining the fractional number processing according to the embodiment, and is a diagram illustrating an example of a state in which unnecessary mask data is removed. FIG. 6 is a diagram for explaining the fraction number processing according to the embodiment and is a diagram illustrating an example of binary image data after superimposition of mask data. It is a flowchart which shows an example of the last number process which concerns on embodiment. It is a figure with which it uses for description of the last weight process which concerns on embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for description of the misrecognition of the object position which concerns on embodiment. It is a figure where it uses for description of the 2nd object position recognition which concerns on embodiment. It is a figure where it uses for description of the 2nd object position recognition which concerns on embodiment.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, the present invention receives a large number of breads (corresponding to the “objects” of the present invention) from a bakery factory in a state of being accommodated in the arrival weight, for a plurality of stores, A case will be described in which the present invention is applied to a distribution center that distributes and distributes bread according to the number requested for each store.

  FIG. 1 is a plan view (partial block diagram) showing an example of the overall configuration of an object sorting system 90 that sorts bread in the distribution center according to the present embodiment.

  As shown in FIG. 1, the object sorting system 90 according to the present embodiment includes an object position recognition device 10, a picking robot 30, a distance image camera (hereinafter simply referred to as “camera”) 32, a table 40, and a belt conveyor 46. Is provided.

  The belt conveyor 46 carries a shipping weight 48 for storing the required number of breads 44 from each store in order to deliver the required number of breads 44 received from the bakery factory to each store in a predetermined direction ( In this embodiment, it is moved and conveyed in the direction of arrow D in FIG. In this embodiment, a donut-shaped pan having a through-hole (hereinafter simply referred to as “hole”) 44 </ b> A in the center is applied as the pan 44. However, the present invention is not limited to this. For example, you may apply the bread | pan etc. in which the recessed part was formed in the center part.

  On the other hand, one end of the table 40 (the lower side in FIG. 1) is a place (hereinafter referred to as a “stacking place”) 40A where a plurality of receiving loads 42 each containing a plurality of pans 44 are stacked. Has been. In addition, the intermediate portion of the table 40 is moved one by one from the plurality of receiving loads 42 stacked in the stacking place 40A one by one, and the pan 44 is moved from the receiving load 42. A place to be taken out (hereinafter referred to as “picking place”) 40B. Further, the other end side (the upper side in FIG. 1) of the table 40 opposite to the one end side is positioned at a position close to the intermediate portion of the belt conveyor 46. A picking robot 30 is installed on the other end side of the table 40. The picking robot 30 takes out the pan 44 from the arrival number 42 moved to the picking place 40 </ b> B and stores it in the shipment number 48 conveyed by the belt conveyer 46.

  In the object sorting system 90 according to the present embodiment, a loading weight 42 that is a normal weight, which is a weight that accommodates the upper limit number of breads 44 that can be accommodated, is stacked in the stacking place 40 </ b> A of the table 40. In the object sorting system 90 according to the present embodiment, when there is a fraction weight that is a weight in which the bread 44 that is a fraction is accommodated according to the number of arrivals from the bakery factory, the fraction weight is determined. The incoming load 42 is stacked on the uppermost layer of the stacking place 40A. In the object sorting system 90 according to the present embodiment, the plurality of arrival weights 42 stacked in the stacking place 40A are sequentially moved one by one from the lowermost arrival weight 42 to the picking place 40B.

  In the object sorting system 90 according to the present embodiment, a worker at the delivery center (hereinafter, simply referred to as “worker”) performs the first step of stacking the incoming load 42 on the stacking place 40A of the table 40. It is said that. Further, in the object sorting system 90 according to the present embodiment, the second step of moving the incoming load 42 from the stacking place 40A to the picking place 40B is automatically and sequentially performed by a moving mechanism (not shown). Furthermore, in the object sorting system 90 according to the present embodiment, all the breads 44 are sorted by the picking robot 30 and the worker performs the third step of removing the empty stock load 42 from the picking location 40B. It is said that. However, each process of these 1st processes and 3rd processes is not limited to the form performed by a worker. These steps may be automatically performed by a transfer robot, a moving mechanism, or the like. Further, the second process is not limited to a form automatically performed by a moving mechanism or the like, but may be a form manually performed by an operator.

  On the other hand, the object position recognition apparatus 10 according to the present embodiment includes an estimation apparatus 20. The estimation apparatus 20 according to the present embodiment estimates the positions of the individual holes 44A of the pan 44 accommodated in the arrival load 42 provided in the picking place 40B. Then, the object position recognition device 10 according to the present embodiment recognizes each position of the pan 44 in the arrival load 42 based on the position of each hole 44A of the pan 44 estimated by the estimation device 20. .

  As shown in FIG. 1, a picking robot 30 and a camera 32 are connected to the object position recognition device 10. The camera 32 according to the present embodiment is provided above the picking place 40B, the photographing direction is directed downward, and the distance image in the entire arrangement region of the arrival load 42 provided in the picking place 40B. Get the data. Here, the distance image data is data in which the value of each pixel of the image data indicates the distance to the subject for each pixel within the field angle of view taken by the camera 32. The Kinect (Kinect (registered trademark)) is exemplified.

  Thus, in this embodiment, although the case where a distance image camera is applied as the camera 32 is described, it is not limited to this. For example, a commercially available camera that captures a color image or a monochrome image may be applied as the camera 32.

  The estimation apparatus 20 according to the present embodiment sets the pixel indicating the distance in the range from the upper end to the center of the pan 44 in the distance image data obtained by the camera 32 as “0” (white), By setting other pixels to “1” (black), binary image data is generated. By binarizing the distance image data, the image area of the pan 44 and the other image areas can be divided by different values. In the present embodiment, as described above, the image area of the pan 44 is set to “0” and the other image areas are set to “1”. However, the present invention is not limited to this, and the image area of the pan 44 is set to “1”. The other image area may be “0”.

  In addition, the estimation device 20 estimates the position of the hole 44A of the pan 44 accommodated in the arrival load 42 provided in the picking place 40B using the generated binary image data.

  On the other hand, the object position recognizing device 10 according to the present embodiment, as described above, is based on the position of the hole 44A of the pan 44 estimated by the estimating device 20, and the individual positions of the pan 44 in the incoming load 42. Recognize Then, the object position recognition device 10 transmits position information indicating the position of the recognized pan 44 to the picking robot 30.

  When the picking robot 30 receives the position information from the object position recognition device 10, the picking robot 30 takes out the pan 44 from the receiving load 42 provided in the picking place 40B based on the position information. The picking robot 30 accommodates the necessary number of pans 44 taken out from the arrival load 42 in the shipment load 48 positioned at the object accommodation position 46A by the belt conveyor 46. When the picking robot 30 has received the required number of pans 44 in the shipping weight 48, the belt conveyor 46 moves and transports the shipping weight 48 to a predetermined location such as a shipping port in the delivery center.

  In addition, the arrival load 42 according to the present embodiment has a predetermined arrangement direction (an example in the present embodiment) that intersects the direction of the plan view in such a direction that the pan 44 can be seen through the hole 44A of the pan 44 in the plan view. As shown in FIG. 1, a plurality of storages can be accommodated in two directions (a row direction and a column direction). As shown in FIG. 1, the arrival load 42 according to the present embodiment includes a total of 12 pans 44 (up to 4 (row direction) × 3 (column method)), and holes 44 </ b> A of each pan 44. Needless to say, the maximum number is only an example, although it is supposed to be laid flat in a visually recognizable orientation.

  FIG. 2 shows an object position recognition apparatus 10 according to the present embodiment and an estimation apparatus 20 included in the object position recognition apparatus 10. As illustrated in FIG. 2, the estimation device 20 according to the present embodiment includes an acquisition unit 11, a detection unit 12, a derivation unit 13, a reception unit 14, and an estimation unit 15. The object position recognition device 10 according to the present embodiment includes an estimation device 20, a superimposition unit 17, and a recognition unit 18.

  The acquisition unit 11 uses the camera 32 to capture an imaging region including an object group region including all the pans 44 of the loading weight 42 in which the upper limit number of breads 44 that can be accommodated are accommodated in a planar shape and set to a normal weight. The distance image data of the shooting area is acquired by shooting. And the acquisition part 11 produces | generates the binary image data which shows the image containing the said object group area | region by binarizing the acquired distance image data as mentioned above.

  In addition, the detection unit 12 detects the positions of both ends in the arrangement direction of the pan 44 in the object group region in the binary image indicated by the binary image data generated by the acquisition unit 11. The deriving unit 13 derives a plurality of divided regions by equally dividing the upper end number between both ends detected by the detecting unit 12. Further, the estimating unit 15 estimates the position of the central portion in the plurality of divided regions derived by the deriving unit 13 as the position of the hole 44 </ b> A in each of the pans 44. The accepting unit 14 is assumed to accept the input of the upper limit number, and the deriving unit 13 derives the divided area using the upper limit number accepted by the accepting unit 14.

  By the way, in the object distribution system 90 according to the present embodiment, a plurality of arrival weights 42 can be accommodated in each of two directions, ie, a row direction and a column direction, as the arrangement direction of the pans 44. For this reason, the detection unit 12 according to the present embodiment detects the positions of the both end portions in the object group region for each of the row direction and the column direction. In addition, the deriving unit 13 derives a plurality of divided regions by equally dividing the both end portions detected by the detecting unit 12 by the corresponding upper limit number for each of the row direction and the column direction. Then, the estimating unit 15 estimates that the position of the central portion in the plurality of divided regions derived by the deriving unit 13 is the position of the hole 44A along each direction of the pan 44 for each of the row direction and the column direction. To do.

  In particular, the detection unit 12 according to the present embodiment includes, for each of a plurality of divided region groups along one of the row direction and the column direction among the plurality of divided regions derived by the derivation unit 13. The position of both ends is further detected. In addition, the detection unit 12 determines the positions of both ends of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction among the plurality of divided regions derived by the derivation unit 13. Further detect. In addition, the deriving unit 13 has a corresponding upper limit number between the respective ends of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction detected by the detecting unit 12. The plurality of divided areas are derived again by equally dividing. Then, the estimation unit 15 estimates the position of the central portion in the plurality of divided regions derived again by the deriving unit 13 as the position of the hole 44 </ b> A in each of the pans 44 for each of the row direction and the column direction.

  Moreover, the estimation part 15 which concerns on this embodiment is the hole 44A estimated when the number of bread | pans 44 accommodated in the arrival weight 42 is less than an upper limit number, when the arrival weight 42 is a positive weight. Is the provisional position of the hole 44 </ b> A of the pan 44 accommodated in the arrival load 42. Then, the estimation unit 15 estimates a position in which the ratio surrounded by images other than the image of the pan 44 among the temporary positions is less than a predetermined value as the position of the hole 44 </ b> A in each of the pans 44.

  In the object sorting system 90 according to the present embodiment, the distance image data is acquired by the camera 32. For this reason, the estimation unit 15 and the like according to the present embodiment uses binary image data obtained by binarizing the distance image data acquired by the camera 32 as described above. However, the present invention is not limited to this. For example, when a camera that captures a color image, a camera that captures a monochrome image, or the like is applied as the camera 32, image data of a plurality of gradations acquired by the camera 32 is binarized with a predetermined threshold value 2 Value image data may be used.

  On the other hand, the superimposing unit 17 superimposes mask data for masking the hole 44A on the position estimated by the estimating unit 15 in the binary image data, and the recognizing unit 18 receives the mask data from the superimposing unit 17. The arrangement position of the pan 44 is recognized using the superimposed binary image data. The recognition result of the arrangement position of the pan 44 by the recognition unit 18 is transmitted to the picking robot 30, and the recognition result received from the recognition unit 18 is used by the picking robot 30, and the individual accommodated in the arrival load 42. The picking of the bread 44 is performed.

  The acquisition unit 11 in the estimation device 20 is an example of an acquisition unit according to the present invention, the detection unit 12 is an example of a detection unit according to the present invention, and the derivation unit 13 is an example of a derivation unit according to the present invention. Moreover, the reception part 14 in the estimation apparatus 20 is an example of the reception part which concerns on this invention, and the estimation part 15 is an example of the estimation part which concerns on this invention. The recognition unit 18 of the object position recognition device 10 is an example of a recognition unit according to the present invention.

  The object position recognition device 10 and the estimation device 20 can be realized by a computer 50 shown in FIG. 3, for example. The computer 50 includes a central processing unit (CPU) 51, a memory 52, a storage unit 53, an input unit 54, a display unit 55, and an interface (I / F) unit 56. The CPU 51, memory 52, storage unit 53, input unit 54, display unit 55, and I / F unit 56 are connected to each other via a bus 59. The above-described picking robot 30 and camera 32 are connected to the I / F unit 56. In the present embodiment, the picking robot 30 and the camera 32 are connected to the computer 50 by wire. However, the present invention is not limited to this, and may be connected wirelessly.

  The storage unit 53 can be realized by an HDD (Hard Disk Drive), a flash memory, or the like. The storage unit 53 as a recording medium stores an object position recognition program 53A for causing the computer 50 to function as the object position recognition device 10 and the estimation device 20. The CPU 51 reads the object position recognition program 53A from the storage unit 53, expands it in the memory 52, and sequentially executes the processes included in the object position recognition program 53A.

  The object position recognition program 53A includes an acquisition process 53A1, a detection process 53A2, a derivation process 53A3, a reception process 53A4, an estimation process 53A5, a superimposition process 53A6, and a recognition process 53A7. The CPU 51 operates as the acquisition unit 11 illustrated in FIG. 2 by executing the acquisition process 53A1, and operates as the detection unit 12 illustrated in FIG. 2 and executes the derivation process 53A3 by executing the detection process 53A2. 2 operates as the derivation unit 13 shown in FIG. Further, the CPU 51 operates as the reception unit 14 illustrated in FIG. 2 by executing the reception process 53A4, and operates as the estimation unit 15 illustrated in FIG. 2 by executing the estimation process 53A5. Furthermore, the CPU 51 operates as the superimposing unit 17 illustrated in FIG. 2 by executing the superimposing process 53A6, and operates as the recognizing unit 18 illustrated in FIG. 2 by executing the recognition process 53A7. As a result, the computer 50 that has executed the object position recognition program 53A functions as the object position recognition device 10 and the estimation device 20. The object position recognition program 53A includes an example of the estimation program of the present invention.

  On the other hand, the storage unit 53 is provided with an image data storage area 53B. The CPU 51 develops the distance image data stored in the image data storage area 53B in the memory 52, so that the distance image data is created.

  The object position recognition device 10 may be a personal computer or a smart terminal that is a portable device equipped with a PDA (Personal Digital Assistant) function. Further, the object position recognition apparatus 10 can be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

  Next, the operation of this embodiment will be described. In the present embodiment, a plurality of arrival weights 42 to be processed by workers in the distribution center are stacked on the stacking place 40A of the table 40 as described above. Thereafter, the worker causes the computer 50 to execute the object position recognition program 53A, whereby the object position recognition process shown in FIG. 4 is performed. In order to avoid complications, here, it is assumed that distance image data obtained continuously by photographing with the camera 32 is stored in the image data storage area 53B of the storage unit 53 in real time.

  In step 100 of the object position recognition process, the receiving unit 14 controls the display unit 55 so as to display a predetermined initial information input screen as a screen for inputting various initial information, and in the next step 102, The reception unit 14 waits for input of predetermined information.

  FIG. 5 shows an example of the initial information input screen according to the present embodiment. As shown in FIG. 5, in the initial information input screen according to the present embodiment, the input area 55A for inputting the maximum number of breads 44 to be received in the arrival load 42, and the processing target in the current object position recognition process An input area 55B for inputting the number of arrival load 42 is displayed. Here, the maximum accommodation number corresponds to the upper limit number. In the object sorting system 90 according to the present embodiment, as described above, an object that can accommodate the pan 44 in a two-dimensional shape in the horizontal direction and the vertical direction with respect to one incoming load 42 is a processing target. For this reason, in the initial information input screen according to the present embodiment, as the input area 55A for inputting the maximum accommodation number, the input area 55A1 for inputting the maximum accommodation number in the row direction and the maximum accommodation number in the column direction are set. An input area 55A2 for input is displayed.

  When the initial information input screen shown in FIG. 5 is displayed on the display unit 55, the operator uses the input unit 54 to set the maximum accommodation number in the row direction of the bread 44 to the arrival load 42 in the input area 55A1. Similarly, the maximum accommodation number in the column direction is input to the input area 55A2. In addition, the worker uses the input unit 54 to input the number of arrival load 42 to be processed in the current object position recognition process into the input area 55B.

  When the above input is completed, the worker designates an “input end” button 55C displayed on the initial information input screen by the input unit 54. When the “input end” button 55C is designated, step 102 is affirmative and the routine proceeds to step 104.

  In step 104, the acquisition unit 11 determines whether or not the arrival load 42 containing the pan 44 is disposed in the picking place 40B, and the arrival load 42 containing the pan 44 is set to the picking place 40B. If it is placed at, an affirmative determination is made and the routine proceeds to step 106. Note that when executing the process of step 104, the acquisition unit 11 reads the latest distance image data stored in the image data storage area 53B. Further, the acquisition unit 11 generates binary image data by binarizing the read distance image data as described above. Then, the acquisition unit 11 determines whether or not the object group region is included in an image indicated by the generated binary image data (hereinafter referred to as “processing target image”), thereby performing the determination in step 104. Do.

  In step 106, the detection unit 12 uses the arrival weight 42 (hereinafter referred to as “processing target weight”) arranged in the picking place 40 </ b> B as a target in the current object position recognition processing. It is determined whether or not it is the last arrival load 42 of 42. Then, if this determination is a negative determination, that is, if the processing target weight is not the last arrival weight 42, the routine proceeds to step 108. It should be noted that whether or not the processing target weight in step 106 is the last arrival weight 42 is determined, for example, by the number of movements of the arrival weight 42 from the stacking location 40A to the picking location 40B. This can be done by determining whether or not the number of incoming loads 42 entered via the screen has been reached. Further, the determination in step 106 may be performed based on whether or not the arrival weight 42 has disappeared from the stacking place 40A of the table 40.

  In step 108, the detection unit 12 determines whether or not the processing target number is a positive number. If the determination is affirmative, the process proceeds to step 110. Note that the determination in step 108 is that the processing target weight is determined to be positive if the processing in step 110 (correct weight processing) described later has not been executed, and the processing target weight is correct. This is performed by determining that the number is not positive when the numbering process is executed.

  In step 110, the CPU 51 performs the positive number processing shown in FIG.

  In step 200 of the normal weight processing, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, binarizes the distance image data as described above, and generates binary image data. In the next step 202, the detection unit 12 extracts a boundary line between the area of the pan 44 and another area from the generated binary image data.

  FIG. 7 is a diagram showing an image in plan view of the arrival load 42 that is an ideal positive weight at the left end, and the binary indicated by the binary image data acquired by the processing of step 200. Three types of images (processing target images) are shown.

  As shown in the leftmost diagram of FIG. 7, in this embodiment, the receiving load 42 can accommodate a total of twelve donut-shaped pans 44 of 4 (in the row direction) × 3 (in the column direction). Has been. However, the actual processing target image is shown in FIG. 7 as a second example from the left of FIG. It may be in a state of touching. In addition, as an example, the processing target image may be in a state of being biased to one place as shown in the second diagram from the right in FIG. Further, the processing target image may be in a state where the individual pans 44 are separated as shown in the rightmost diagram of FIG. 7 as an example. In the following, the processing target image shown in the second diagram from the left in FIG. 7 is referred to as a “first image”, and the processing target image shown in the second diagram from the right in FIG. 7 is referred to as a “second image”. The processing target image shown in the rightmost diagram of FIG. 7 is referred to as a “third image”.

  For such an image, when the boundary line is extracted in step 202, as shown in the leftmost diagram of FIG. 8 as an example, the pixel values from the four sides around the processing target image toward the inside of the image are “ Points that change from “1” to “0” are detected as the points constituting the boundary line. When detecting each point, if the noise image is included outside the image area of the pan 44 in the processing target image, an accurate boundary line cannot be obtained. For this reason, the detection unit 12 according to the present embodiment performs a noise removal process on the binary image data before extracting the boundary line. In addition, as a noise removal process applied here, conventionally well-known techniques, such as the method by a median filter, the method by a moving average filter, are illustrated.

  By the process of extracting the boundary line, for example, the boundary line shown in the second diagram from the left in FIG. 8 is obtained for the first image. Similarly, the boundary line shown in the second diagram from the right in FIG. 8 is obtained for the second image, and the boundary line shown in the rightmost diagram of FIG. 8 is obtained for the third image.

  In the next step 204, the detection unit 12 specifies a circumscribed rectangular frame for the boundary line obtained by the processing in step 202. By this processing, for example, a circumscribed rectangular frame shown in the leftmost diagram of FIG. 9 is obtained for the first image. Similarly, the circumscribed rectangular frame shown in the center diagram of FIG. 9 is obtained for the second image, and the circumscribed rectangular frame shown in the rightmost diagram of FIG. 9 is obtained for the third image.

  In the next step 206, the detection unit 12 detects the positions of both end portions in each of the row direction and the column direction from the circumscribed rectangular frame obtained by the process of step 204. As described above, in this embodiment, since the pan 44 can be accommodated in the two-dimensional shape in the row direction and the column direction in the arrival load 42, in step 206, both ends in each direction in the row direction and the column direction. However, the present invention is not limited to this. For example, when the arrival weight 42 can accommodate only one row of breads 44, in step 206, the positions of both end portions only in the row direction (the arrangement direction of the pans 44) may be detected.

  In the next step 208, the derivation unit 13 divides the both ends in the row direction detected by the processing in step 206 equally by the maximum accommodation number in the row direction corresponding to the upper limit number, so that the row direction A plurality of divided regions for is derived. Further, the derivation unit 13 divides the both ends in the column direction detected by the processing of step 206 equally by the maximum accommodation number in the column direction corresponding to the upper limit number, thereby dividing the plurality of divisions in the column direction. Deriving a region. By this processing, for example, the divided area shown in the leftmost diagram of FIG. 9 is obtained for the first image. Similarly, for the second image, the divided region shown in the center diagram of FIG. 9 is obtained, and for the third image, the divided region shown in the rightmost diagram of FIG. 9 is obtained.

  By the way, for example, when the outline of the boundary line is substantially rectangular like the first image or the second image, each of the pans 44 is arranged in the divided area obtained by the above processing. It can be roughly estimated that this is a region. However, as in the third image, when the boundary line has a non-rectangular shape (for example, a trapezoidal shape or a parallelogram shape), the divided regions obtained by the above processing are: It cannot be estimated that each of the pans 44 is an area where the pan 44 is disposed.

  Therefore, the detection unit 12 according to the present embodiment includes a plurality of regions along one of the row direction and the column direction (the row direction in the present embodiment) among the plurality of divided regions derived by the deriving unit 13. The positions of both ends of each of the divided region groups are further detected. The detection unit 12 also includes a plurality of divided region groups along the other direction (the column direction in the present embodiment) of the row direction and the column direction among the plurality of divided regions derived by the derivation unit 13. The positions of both ends of each are further detected.

  And the derivation | leading-out part 13 which concerns on this embodiment is between the both ends of each of the some division area group along the row direction, and the some division area group along the row direction detected by the detection part 12. By dividing equally by the corresponding upper limit number (maximum accommodation number), the plurality of divided regions are derived again.

  Among the processes of the detection unit 12 and the derivation unit 13 described above, by performing the process for the divided region group along the row direction as the first stage, as an example, the third image is shown in the row direction as shown in FIG. The divided area for is adjusted. Moreover, the process with respect to the division area group along a column direction is performed as a 2nd step among the processes of the above detection part 12 and derivation | leading-out part 13 with respect to the division area which passed the process of the 1st step. As a result, as shown in FIG. 11, in addition to the divided areas in the row direction, the divided areas in the column direction are adjusted.

  The divided area obtained by the above processing is a divided area that also considers the deviation of the arrangement position due to each of the plurality of pans 44. Therefore, if the divided area is an area where each of the pans 44 is arranged, the divided area is high. It can be estimated with accuracy.

  In the next step 210, the estimating unit 15 determines the position of the central portion in the plurality of divided areas derived again by the deriving unit 13 for each of the row direction and the column direction as the position of the hole 44 </ b> A in each of the pans 44. Estimated.

  In the next step 212, the estimation unit 15 stores position information indicating the position of the hole 44 </ b> A in each estimated pan 44 in the memory 52. In the next step 214, the superimposing unit 17 superimposes mask data for masking the hole 44A on the position estimated by the estimating unit 15 in the binary image data indicating the processing target image, The normal number weight processing ends. When the normal weight processing is completed, the routine proceeds to step 116 of the object position recognition processing shown in FIG. In this embodiment, a circular image having a predetermined size (diameter) is used as the mask data with the same value as the pan 44 image in the binary image data (in this embodiment, “0”). The data to be formed is used.

  Note that the pan 44 corresponding to the small-sized divided region has a high degree of compression, and the diameter of the hole 44A may also be smaller. Considering this, the size of mask data (diameter of the circle) may be changed according to the size of each divided region.

  FIG. 12 shows the positions masked by the process of step 214 together with the boundary lines and the divided areas when the processing target images are the first image to the third image. Further, in the upper part of FIG. 13, the positions masked by the process of step 214 when the processing target images are the first to third images are shown together with the processing target images. Furthermore, the lower part of FIG. 13 shows the processing target image after the mask data is superimposed on the corresponding processing target image.

  As shown in FIGS. 12 and 13, the mask data overlap in step 214 shows a small mask leak for some pan 44 images, but the image corresponding to the hole 44 </ b> A for most pan 44 images. It can be seen that is missing (masked). Here, “mask leakage” means that a part of the region to be masked (the region of the hole 44A) cannot be masked. Therefore, by performing object position recognition for recognizing the position of the pan 44 on the processing target image after the mask data is superimposed, the influence of the image of the hole 44A is reduced. Recognition can be performed. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  On the other hand, when a negative determination is made in step 108 of the object position recognition shown in FIG. 4, the processing target weight is regarded as a fraction weight, and the process proceeds to step 112, where the CPU 51 performs the fraction number processing shown in FIG. I do.

  In step 300 of the fraction number processing, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, binarizes the distance image data as described above, and generates binary image data. In the next step 302, the detection unit 12 reads the position information stored by the processing of step 212 of the positive number processing executed most recently.

  In the next step 304, the superimposing unit 17 superimposes the mask data on the binary image data generated by the process of step 300 at the position indicated by the position information read by the process of step 302.

  FIG. 15 shows three types of binary images (processing target images) indicated by the binary image data acquired by the processing in step 300.

  The leftmost diagram of FIG. 15 shows an image in a state where two pans 44 in the right half of the first stage are picked with respect to the processing target number which is the first image as the processing target image in the normal number state. (Hereinafter referred to as “fourth image”). Further, the center diagram of FIG. 15 shows that the processing target image in the normal weight state is all four in the first stage and two in the central part of the second stage with respect to the processing target number of the second image. A total of six pans 44 are picked (hereinafter referred to as “fifth image”). Further, the rightmost diagram in FIG. 15 shows the three pans 44 on the right side of the first stage and the second stage of the second stage in relation to the processing target number in which the processing target image in the positive number state is the third image. An image (hereinafter, referred to as a “sixth image”) in which a total of five pans 44 in the left half are picked is shown. FIG. 16 shows the positions masked by the process of step 304 together with the process target image when the process target images are the fourth image to the sixth image.

  As shown in FIG. 16, in the process of step 304, since the mask data is actually superimposed up to the area where the pan 44 does not exist, the area where the pan 44 does not exist and the area where the mask data is superimposed remain as it is. There is a high risk of erroneous determination that the pan 44 is located. Therefore, in the next step 306, the superimposing unit 17 determines whether the arrangement of the mask data performed by the process of step 304 is appropriate for each arranged mask data.

  In the present embodiment, the determination as to whether the arrangement of the mask data is appropriate is performed as follows. That is, out of the circumference of the mask area based on the arranged mask data, the pixel (in this embodiment, “1” in this embodiment) other than the image of the pan 44 in the processing target image (in this embodiment, When the ratio surrounded by black pixels) is less than a predetermined value, it is determined to be appropriate. In the present embodiment, 70% is applied as the predetermined value. Therefore, for example, as shown in FIG. 17, when the ratio is 100% and 80%, it is determined that the arrangement of the mask data is not valid, and the ratio is 60%, 20%, and 0% ( In the case of no contact with black pixels), it is determined that the arrangement of the mask data is appropriate.

  In the next step 308, the superimposing unit 17 removes the mask data determined to be invalid in the arrangement in step 306 from the binary image data, and thereafter ends the fraction numbering process. When the fraction number processing ends, the process proceeds to step 116 shown in FIG.

  FIG. 18 shows the positions to be finally masked together with the processing target image when the processing target images are the fourth to sixth images. FIG. 19 shows the processing target image after mask data is superimposed on the processing target images of the fourth image to the sixth image. In FIG. 18, the position of the mask based on the mask data removed by the processing in step 308 is indicated by a white broken line.

  As shown in FIGS. 18 and 19, the fourth to sixth images are removed from the mask data in the area where the pan 44 image does not exist by the processing of step 304 to step 308, and the pan 44 image is removed. The mask data is superimposed on only the area. As a result, an area without the pan 44 is not masked, and a small mask leak is seen in some pan 44 images, but there is no image corresponding to the hole 44A in most pan 44 images ( You can see that it is masked. Accordingly, by performing object position recognition for recognizing the position of the pan 44 for these processing target images, recognition is performed without being affected by the image of the hole 44A or with the influence of the image of the hole 44A being reduced. Can do. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  On the other hand, when an affirmative determination is made in step 106 of the object position recognition shown in FIG. 4, the processing object weight is the last arrival weight of the arrival weight 42 that is the target in the current object position recognition process. 42 (hereinafter referred to as “final weight”), the process proceeds to step 114.

  In step 114, the CPU 51 performs a final weight process shown in FIG. Hereinafter, the final number processing will be described. In order to avoid complications, the present embodiment will describe a case where an ideal arrangement position of the pan 44 in the final weight (hereinafter referred to as “final weight arrangement position”) is set in advance.

  In step 400 of the final weight processing, the acquisition unit 11 reads the latest distance image data from the image data storage area 53B, binarizes the distance image data as described above, and generates binary image data. In the next step 402, the detection unit 12 reads the position information stored by the processing of step 212 of the positive number processing executed most recently.

  In the next step 404, the superimposing unit 17 superimposes the mask data on the binary image data generated by the process of step 400 at the position indicated by the position information read by the process of step 402.

  In the next step 406, the superimposing unit 17 removes the mask data corresponding to the position different from the final number arrangement position from the mask data arranged by the process in step 404, and then ends the final number process. To do. When the final number process is completed, the process proceeds to step 116 of the object position recognition process shown in FIG.

  FIG. 21 is a diagram for explaining the final weight processing when the processing target image in the positive weight processing executed most recently is the third image. In this case, the processing target image in the most recent positive weight processing executed is the image shown at the left end of FIG. 21, and the mask data is moved to the position shown in the second diagram from the left in FIG. Superimposed.

  In this case, the processing target image indicated by the binary image data acquired by the processing of step 400 of the final weight processing is, for example, the second image from the right in FIG. Note that the second diagram from the right in FIG. 21 shows a case where there is no first-stage pan 44 in the final weight, and a total of eight pans 44 in the second to third stages are accommodated. ing. In this case, in the process of step 406 of the final weight processing, as shown in the rightmost diagram of FIG. 21, all the mask data superimposed on the first stage is removed and arranged on the second to third stages. Mask data remains. As a result, the area without the pan 44 is not masked, and a small mask leak occurs in some pan 44 images, but the image corresponding to the hole 44A is almost absent in the pan 44 images. Therefore, by performing object position recognition for recognizing the position of the pan 44 for this processing target image, recognition can be performed without being affected by the image of the hole 44A or by reducing the effect of the image of the hole 44A. it can. As a result, the individual positions of the pan 44 can be recognized with higher accuracy.

  In step 116 of the object position recognition process shown in FIG. 4, the recognition unit 18 performs binary image data (hereinafter referred to as “masked”) on which the mask data is superimposed by any one of the above steps 110, 112, and 114. Object position recognition that recognizes the position of the pan 44 accommodated in the processing target weight is performed using “image data”. In the present embodiment, as the object position recognition, conventionally known object position recognition for recognizing the position of a pan without a hole 44A or a recess at the center is performed. In the object position recognition according to the present embodiment, a histogram indicating the accumulated value of the number of white pixels in each of the horizontal direction and the vertical direction of the processing target image indicated by the masked image data is created for each direction. Then, the position of each pan 44 is recognized as a boundary region between adjacent pans 44 where the valley portions of the histograms in the horizontal and vertical directions are adjacent. However, the object position recognition is not limited to this method, and any method may be applied as long as the position of each of a plurality of objects is recognized using binary image data.

  By the way, in the object position recognition, the position of the pan 44 may be erroneously recognized due to various causes. Hereinafter, a specific example of the cause of erroneous recognition in object position recognition will be described.

  For example, as shown in the left diagram of FIG. 22 as an example, when the area of the mask indicated by the mask data is smaller than the area of the image of the hole 44A of the pan 44, the object position may be erroneously recognized. . This is because an area between the image of the pan 44 and the image based on the mask data may be erroneously recognized as a space between the adjacent pans 44. As an example, as shown in the diagram on the right side of FIG. 22, when the area of the mask indicated by the mask data is larger than the area of the image of the pan 44, the object position may be erroneously recognized. This is because the space between adjacent pans 44 may not be identified.

  In order to solve this problem, the size of the mask indicated by the mask data may be designated by the operator. As an example of this form, it is assumed that the size of the mask is input by the operator via the input unit 54, and the size of the mask input by the worker via the input unit 54 is received by the receiving unit 14. Is exemplified. In this example, mask data corresponding to the size of the mask received by the receiving unit 14 is generated by the superimposing unit 17 and applied in the positive number processing, fraction number processing, and final number processing.

  Further, as a specific example of the cause of erroneous recognition in the object position recognition, as shown in FIG. 23 as an example, only a part of the pans 44 in the horizontal direction or the vertical direction due to the movement of the arrival weight 42 Even when the position is greatly displaced, the object position may be recognized incorrectly. Further, as shown in FIG. 24 as an example, even when only a part of the pans 44 are displaced due to the immediately preceding picking, the number of mask data may be larger than the number of pans 44 in the above-described fraction number processing. In this case as well, there is a case where the object position is recognized incorrectly.

  In order to deal with such misrecognition to some extent, in the object position recognition process according to the present embodiment, when an erroneous recognition occurs in the object position recognition in step 116, unlike the object position recognition, the distance image data Object position recognition is performed directly using.

  That is, in the next step 118, the recognizing unit 18 determines whether or not a misrecognition has occurred in the object position recognition in step 116, and proceeds to step 120 if a negative determination is made. Note that the recognition unit 18 according to the present embodiment determines whether or not the erroneous recognition in step 118 has occurred, whether or not the number of recognized pan 44 positions is inconsistent with the assumed number of pans 44. This is done by judging. Here, the assumed number of pans 44 is the number of pans 44 when the processing target weight is a positive weight, and the number of pans picked by the picking robot 30 from the processing target weight so far. It is obtained by subtracting the number of 44.

  In step 120, the recognition unit 18 transmits position information indicating the position of the pan 44 recognized by the object position recognition in step 116 to the picking robot 30, and then proceeds to step 132. When receiving the position information from the object position recognizing device 10, the picking robot 30 extracts one pan 44 from the processing target weight based on the position of the pan 44 indicated by the position information and sets it to the shipping weight 48. Accommodate.

  On the other hand, if an affirmative determination is made in step 118, it is assumed that an erroneous recognition has occurred in the object position recognition in step 116, and the process proceeds to step 122. Is read. Further, in step 122, the detection unit 12 reads the position information stored by the processing of step 212 of the positive number processing executed most recently. In step 122, the superimposing unit 17 superimposes mask data on the read distance image data as follows, as shown in FIG. 25 as an example. That is, with the center of the position indicated by the position information of the distance image indicated by the distance image data as a center, the upper surface and the lower surface of a predetermined size (including the area and height when viewed in plan) have different cylindrical shapes ( Distance image data indicating a shape obtained by cutting and removing the upper side of the cone horizontally is superimposed as mask data.

  Note that, in the mask data used for the distance image data here, similarly to the mask data used for the binary image data, erroneous recognition occurs in the object position recognition depending on the size of the pan hole to be recognized. May end up. For this reason, the size of the mask data used for the distance image data may be specified by the operator.

  In the next step 124, the recognition unit 18 performs object position recognition (hereinafter referred to as “second object position recognition”) on the distance image data on which the mask data is superimposed by the above processing. In the second object position recognition according to the present embodiment, as shown in FIG. 26 as an example, a horizontal histogram and a vertical histogram defined below are created, and a pan 44 in which valley portions of the histograms are adjacent to each other is created. The position of each pan 44 is recognized as a boundary region between the two. Here, the horizontal histogram is obtained by averaging or integrating each distance information in the corresponding vertical position in the horizontal direction. The vertical histogram is obtained by averaging or integrating each distance information in the corresponding horizontal position in the vertical direction. Here, as the distance information, the distance from the bottom surface is applied on the basis of the bottom surface (the surface in contact with the lower surface of the pan 44) of the arrival weight 42. However, the second object position recognition is not limited to this method, and any method may be applied as long as the position of each of the plurality of objects is recognized using the distance image data.

  In the next step 126, the recognizing unit 18 determines whether or not a misrecognition has occurred in the second object position recognition in step 124. If the determination is negative, the process proceeds to step 128. It should be noted that the recognition unit 18 according to the present embodiment also determines whether or not the erroneous recognition in step 126 has occurred, whether or not the number of recognized pan 44 positions is inconsistent with the assumed number of pans 44. This is done by judging.

  In step 128, the recognizing unit 18 transmits position information indicating the position of the pan 44 recognized by the second object position recognition in step 124 to the picking robot 30, and then proceeds to step 132. In response to this, the picking robot 30 performs the picking operation of one pan 44 as described in the description of step 120.

  In step 132, the recognizing unit 18 determines whether or not the picking has been completed for all the pans 44 of the incoming load 42 stacked in the stacking place 40A. If the determination is negative, the process returns to step 104. . If the determination is affirmative, the object position recognition process ends.

  On the other hand, if an affirmative determination is made in step 126, it is assumed that erroneous recognition has occurred in the second object position recognition, the process proceeds to step 130, and the recognition unit 18 performs a predetermined error process, The object position recognition process ends. In the object position recognition process according to the present embodiment, a process of displaying information indicating that a misrecognition has occurred on the display unit 55 is applied as the error process of step 130, but is not limited thereto. is not. For example, you may apply the other form which alert | reports that misrecognition generate | occur | produced, such as sounding a buzzer.

  As described above, in the present embodiment, an upper limit number of objects in the arrangement direction is provided in a container (stock load 42) that can accommodate a plurality of objects (pans 44) having holes in the center in a predetermined arrangement direction. Image data of an image obtained by photographing an object group area including all objects in the accommodated state is acquired. In the present embodiment, the positions of both ends in the arrangement direction of the object group region indicated by the acquired image data are detected. Further, in the present embodiment, a plurality of divided regions are derived by equally dividing the detected both ends with the above upper limit number. Furthermore, in the present embodiment, the position of the central portion in the arrangement direction in the derived plurality of divided regions is estimated as the position of the hole in each of the objects. Therefore, according to the present embodiment, the position of the object can be recognized in consideration of the unexpected bias or spread of the object, and as a result, the position of each of the plurality of objects having a hole in the center is recognized with high accuracy. can do.

  In particular, in this embodiment, a plurality of containers can be accommodated in each of the two directions of the row direction and the column direction with the object as the arrangement direction. In the present embodiment, the positions of both end portions of the object group region are detected for each of the row direction and the column direction. Further, in the present embodiment, a plurality of divided regions are derived by equally dividing the detected both ends by the corresponding upper limit number for each of the row direction and the column direction. In the present embodiment, the positions of the center in the row direction and the center in the column direction in the derived plurality of divided regions are estimated as the positions of the holes in each of the objects for each of the row direction and the column direction. Therefore, according to this embodiment, the present invention can be applied to a form in which an object can be accommodated in a two-dimensional plan view with respect to the container.

  In particular, in the present embodiment, the positions of the both end portions of each of the plurality of divided region groups along either the row direction or the column direction among the plurality of derived divided regions are further detected. Yes. In the present embodiment, the positions of the both end portions of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction are further detected among the derived plurality of divided regions. Yes. Further, in the present embodiment, the detected end portions of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction are equally divided by the number of corresponding objects. Thus, a plurality of divided regions are derived again. In the present embodiment, the position of the central portion in the row direction and the central portion in the column direction in the plurality of divided areas derived again is estimated as the position of the hole in each of the objects for each of the row direction and the column direction. Therefore, according to the present embodiment, even when the positions of some objects are shifted with respect to at least one of the row direction and the column direction, the positions of the plurality of objects are recognized with high accuracy. can do.

  Further, in this embodiment, when the number of objects accommodated in the container is less than the upper limit number, out of the positions of the holes estimated when the object is accommodated in the container above the upper limit number other than the object image The position where the ratio surrounded by the images is less than a predetermined value is estimated as the position of each hole of the object. Therefore, according to the present embodiment, the position of each of the plurality of objects can be recognized with high accuracy even when the number of objects accommodated in the container is less than the upper limit number.

  In the present embodiment, distance image data is generated by photographing, and based on the distance image data, binary image data that distinguishes an object region from other regions is acquired as the image data. Therefore, according to the present embodiment, the position of each of a plurality of objects can be recognized with high accuracy using binary image data based on distance image data.

  In the present embodiment, the upper limit number is input, and the divided area is derived using the received upper limit number. Therefore, according to this embodiment, a plurality of types of containers having different upper limit numbers can be targeted.

  Furthermore, in this embodiment, an annular object is applied as the object. Therefore, according to the present embodiment, the position of each of a plurality of objects can be recognized with high accuracy for an annular object.

  In the above embodiment, it is assumed that the weight is in the positive weight state (the state in which the upper limit number of objects are accommodated) when executing the positive weight processing. However, the present invention is not limited to this. . Even if the weight is less than the positive weight, if the objects in the weight are offset in the horizontal and vertical directions and the number of objects is known, the positive weight processing can be executed. is there.

  Moreover, although the said embodiment demonstrated the case where the pan 44 which has the hole 44A in the center as an object of this invention was applied, it is not limited to this. For example, a tire, a nut, a washer, an O-ring, or the like may be applied as the object of the present invention in addition to a food such as a donut or a bread with a depressed central portion. Also in this case, the same effect as the above embodiment can be obtained.

  Moreover, although the said embodiment demonstrated the case where the bread | pan 44 was accommodated in the receiving load 42 in two dimensions, it is not limited to this. For example, the pan 44 may be accommodated in a one-dimensional shape, that is, only for one row in the arrival weight 42.

  Further, in the above-described embodiment, the case has been described in which the divided areas shown in FIGS. 10 and 11 are adjusted after the divided areas shown in FIG. However, the present invention is not limited to this. For example, when the outline of the boundary line is substantially rectangular like the first image and the second image, the division area may not be adjusted.

  Moreover, although the said embodiment demonstrated the case where the camera which acquires distance image data was applied as the camera 32, it is not limited to this. For example, a commercially available camera that captures color images or monochrome images may be applied as the camera 32. In this case, since data of a plurality of gradations is acquired as image data by the camera 32, data obtained by binarizing the data of the plurality of gradations with a predetermined threshold is used as the image data. In this case, since a camera that captures a relatively inexpensive color image or monochrome image can be used as the camera 32, an object distribution system can be constructed at a low cost.

  Moreover, in the said embodiment, after superimposing the mask data on all the positions estimated by the estimation part 15 by the superimposition part 17, it is determined whether the mask arrangement | positioning by this mask data is appropriate. Although described, the present invention is not limited to this. For example, the superimposing unit 17 determines in advance whether or not the mask layout is valid for each of the positions estimated by the estimating unit 15 and superimposes the mask data only on the positions determined to be valid. May be.

  In the above-described embodiment, the case has been described in which the camera 32 is provided above the picking place 40B and the pan 44 is picked by the picking robot 30 from the arrival load 42 provided in the picking place 40B. It is not something. For example, the camera 32 may be provided above the stacking location 40A and the pan 44 may be picked by the picking robot 30 from the receiving load 42 provided in the stacking location 40A. Further, the camera 32 is provided above both the stacking place 40A and the picking place 40B, and the picking robot 30 picks the pan 44 selectively from the incoming load 42 provided in either the stacking place 40A or the picking place 40B. You may make it do.

  Moreover, although the said embodiment demonstrated the aspect which memorize | stores beforehand the object position recognition program 53A in the memory | storage part 53 (installation), it is not limited to this. For example, the object position recognition program 53A is read from a recording medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc ROM) into the storage unit 53 of the computer 50 via a medium read / write device. Alternatively, it may be downloaded from an external device to the storage unit 53 via a network.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. An acquisition unit that acquires image data of an image obtained by photographing an object group region including all the objects contained in the container in the state in which the predetermined number is accommodated in the arrangement direction, by an imaging unit;
A detection unit for detecting positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A derivation unit for deriving a plurality of divided regions by equally dividing the both ends detected by the detection unit by the predetermined number;
An estimation unit that estimates the position of the central portion in the arrangement direction in the plurality of divided regions derived by the deriving unit as the position of the hole or recess in each of the objects;
Including the estimation device.

(Appendix 2)
The container is capable of accommodating a plurality of the objects in two directions of a row direction and a column direction as the predetermined arrangement direction,
The detection unit detects the positions of the both ends of the object group region separately for each of the row direction and the column direction,
The derivation unit derives a plurality of divided regions by equally dividing the corresponding end portion between the both ends detected by the detection unit for each of the row direction and the column direction,
The estimating unit determines the positions of the center in the row direction and the center in the column direction in the plurality of divided regions derived by the deriving unit for each of the row direction and the column direction in the holes in each of the objects. Or estimate the position of the recess,
The estimation apparatus according to attachment 1.

(Appendix 3)
The detection unit includes positions of the both end portions for each of a plurality of divided region groups along either the row direction or the column direction among the plurality of divided regions derived by the derivation unit. Further, among the plurality of divided regions derived by the deriving unit, the positions of the both end portions of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction are further detected. And
The derivation unit corresponds to the object between the both ends of each of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction detected by the detection unit. The plurality of divided regions are derived again by dividing equally by the number of
The estimating unit determines the position of the central part in the row direction and the central part in the column direction in each of the objects in the plurality of divided regions derived again by the deriving unit for each of the row direction and the column direction. Estimate the position of the hole or recess,
The estimation apparatus according to attachment 2.

(Appendix 4)
When the number of the objects accommodated in the container is less than the predetermined number, the estimation unit determines the position of the hole or the recess estimated when the predetermined number of the objects are accommodated in the container. Of these, the position surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects,
The estimation apparatus according to any one of supplementary notes 1 to 3.

(Appendix 5)
The photographing unit generates distance image data by photographing,
The acquisition unit acquires, as the image data, binary image data that distinguishes the region of the object from other regions based on the distance image data.
The estimation apparatus according to any one of supplementary notes 1 to 4.

(Appendix 6)
A reception unit for receiving the predetermined number of inputs;
The deriving unit derives the divided region using a predetermined number received by the receiving unit.
The estimation apparatus according to any one of supplementary notes 1 to 5.

(Appendix 7)
The object is annular.
The estimation apparatus according to any one of supplementary notes 1 to 6.

(Appendix 8)
An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. An acquisition unit that acquires image data of an image obtained by photographing an object group region including all the objects contained in the container in the state in which the predetermined number is accommodated in the arrangement direction, by an imaging unit;
A detection unit for detecting positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A derivation unit for deriving a plurality of divided regions by equally dividing the both ends detected by the detection unit by the predetermined number;
An estimation unit that estimates the position of the central portion in the arrangement direction in the plurality of divided regions derived by the deriving unit as the position of the hole or recess in each of the objects;
A recognition unit that recognizes an arrangement position of the object using the image data in which the hole or the recess is masked based on the position estimated by the estimation unit in the image data;
An object position recognition apparatus including:

(Appendix 9)
The container is capable of accommodating a plurality of the objects in two directions of a row direction and a column direction as the predetermined arrangement direction,
The detection unit detects the positions of the both ends of the object group region separately for each of the row direction and the column direction,
The derivation unit derives a plurality of divided regions by equally dividing the corresponding end portion between the both ends detected by the detection unit for each of the row direction and the column direction,
The estimating unit determines the positions of the center in the row direction and the center in the column direction in the plurality of divided regions derived by the deriving unit for each of the row direction and the column direction in the holes in each of the objects. Or estimate the position of the recess,
The object position recognition apparatus according to appendix 8.

(Appendix 10)
The detection unit includes positions of the both end portions for each of a plurality of divided region groups along either the row direction or the column direction among the plurality of divided regions derived by the derivation unit. Further, among the plurality of divided regions derived by the deriving unit, the positions of the both end portions of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction are further detected. And
The derivation unit corresponds to the object between the both ends of each of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction detected by the detection unit. The plurality of divided regions are derived again by dividing equally by the number of
The estimating unit determines the position of the central part in the row direction and the central part in the column direction in each of the objects in the plurality of divided regions derived again by the deriving unit for each of the row direction and the column direction. Estimate the position of the hole or recess,
The object position recognition apparatus according to appendix 9.

(Appendix 11)
When the number of the objects accommodated in the container is less than the predetermined number, the estimation unit determines the position of the hole or the recess estimated when the predetermined number of the objects are accommodated in the container. Of these, the position surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects,
The object position recognition apparatus according to any one of appendix 8 to appendix 10.

(Appendix 12)
The photographing unit generates distance image data by photographing,
The acquisition unit acquires, as the image data, binary image data that distinguishes the region of the object from other regions based on the distance image data.
The object position recognition device according to any one of appendix 8 to appendix 11.

(Appendix 13)
A reception unit for receiving the predetermined number of inputs;
The deriving unit derives the divided region using a predetermined number received by the receiving unit.
The object position recognition device according to any one of appendix 8 to appendix 12.

(Appendix 14)
The object is annular.
The object position recognition device according to any one of appendix 8 to appendix 13.

(Appendix 15)
An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. In a state where a predetermined number is accommodated in the arrangement direction, obtain image data of an image obtained by photographing an object group region including all the objects accommodated in the accommodating body by an imaging unit,
Detecting the positions of both ends in the arrangement direction of the object group region indicated by the acquired image data,
Deriving a plurality of divided areas by equally dividing the detected both ends by the predetermined number,
Estimating the position of the central portion in the arrangement direction in the derived plurality of divided regions as the position of the hole or recess in each of the objects,
An estimation program for causing a computer to execute a process including the above.

(Appendix 16)
The container is capable of accommodating a plurality of the objects in two directions of a row direction and a column direction as the predetermined arrangement direction,
Detecting the positions of the both ends of the object group region separately for each of the row direction and the column direction,
In each of the row direction and the column direction, a plurality of divided areas are derived by equally dividing the detected end portions by the corresponding predetermined number,
For each of the row direction and the column direction, the position of the center in the row direction and the center in the column direction in the derived plurality of divided regions is estimated as the position of the hole or recess in each of the objects.
The estimation program according to attachment 15.

(Appendix 17)
Among the derived plurality of divided regions, the positions of the both end portions of each of the plurality of divided region groups along either the row direction or the column direction, and among the derived plurality of divided regions , Further detecting the positions of the both ends of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction,
By dividing equally between the both ends of each of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction detected by the number of the corresponding objects. Deriving multiple divided regions again,
For each of the row direction and the column direction, the position of the center in the row direction and the center in the column direction in the plurality of divided regions derived again is estimated as the position of the hole or recess in each of the objects.
The estimation program according to attachment 16.

(Appendix 18)
When the number of the objects accommodated in the container is less than the predetermined number, of the positions of the holes or recesses estimated when the predetermined number of the objects are accommodated in the container Estimating a position surrounded by an image other than an image to be less than a predetermined value as a position of the hole or recess in each of the objects,
The estimation program according to any one of appendix 15 to appendix 17.

(Appendix 19)
The photographing unit generates distance image data by photographing,
Based on the distance image data, binary image data that distinguishes the region of the object from other regions is acquired as the image data.
The estimation program according to any one of appendix 15 to appendix 18.

(Appendix 20)
Accept the predetermined number of inputs,
Deriving the divided region using the received predetermined number,
The estimation program according to any one of appendix 15 to appendix 19.

(Appendix 21)
The object is annular.
The estimation program according to any one of appendix 15 to appendix 20.

DESCRIPTION OF SYMBOLS 10 Object position recognition apparatus 11 Acquisition part 12 Detection part 13 Derivation part 14 Reception part 15 Estimation part 17 Superimposition part 18 Recognition part 20 Estimation apparatus 30 Picking robot 32 Camera 42 Stock weight 44 Pan 44A Hole 48 Ship weight 50 Computer 51 CPU
52 Memory 53 Storage Unit 53A Object Position Recognition Program 53A1 Acquisition Process 53A2 Detection Process 53A3 Derivation Process 53A4 Reception Process 53A5 Estimation Process 53A6 Superimposition Process 53A7 Recognition Process 53B Image Data Storage Area 54 Input Unit 55 Display Unit 90 Object Distribution System

Claims (8)

An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. An acquisition unit that acquires image data of an image obtained by photographing an object group region including all the objects contained in the container in the state in which the predetermined number is accommodated in the arrangement direction, by an imaging unit;
A detection unit for detecting positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A derivation unit for deriving a plurality of divided regions by equally dividing the both ends detected by the detection unit by the predetermined number;
An estimation unit that estimates the position of the central portion in the arrangement direction in the plurality of divided regions derived by the deriving unit as the position of the hole or recess in each of the objects;
Including the estimation device. The container is capable of accommodating a plurality of the objects in two directions of a row direction and a column direction as the predetermined arrangement direction,
The detection unit detects the positions of the both ends of the object group region separately for each of the row direction and the column direction,
The derivation unit derives a plurality of divided regions by equally dividing the corresponding end portion between the both ends detected by the detection unit for each of the row direction and the column direction,
The estimating unit determines the positions of the center in the row direction and the center in the column direction in the plurality of divided regions derived by the deriving unit for each of the row direction and the column direction in the holes in each of the objects. Or estimate the position of the recess,
The estimation apparatus according to claim 1. The detection unit includes positions of the both end portions for each of a plurality of divided region groups along either the row direction or the column direction among the plurality of divided regions derived by the derivation unit. Further, among the plurality of divided regions derived by the deriving unit, the positions of the both end portions of each of the plurality of divided region groups along the other arrangement direction of the row direction and the column direction are further detected. And
The derivation unit corresponds to the object between the both ends of each of the plurality of divided region groups along the row direction and the plurality of divided region groups along the column direction detected by the detection unit. The plurality of divided regions are derived again by dividing equally by the number of
The estimating unit determines the position of the central part in the row direction and the central part in the column direction in each of the objects in the plurality of divided regions derived again by the deriving unit for each of the row direction and the column direction. Estimate the position of the hole or recess,
The estimation apparatus according to claim 2. When the number of the objects accommodated in the container is less than the predetermined number, the estimation unit determines the position of the hole or the recess estimated when the predetermined number of the objects are accommodated in the container. Of these, the position surrounded by an image other than the image of the object is less than a predetermined value is estimated as the position of the hole or recess in each of the objects,
The estimation apparatus according to any one of claims 1 to 3. The photographing unit generates distance image data by photographing,
The acquisition unit acquires, as the image data, binary image data that distinguishes the region of the object from other regions based on the distance image data.
The estimation apparatus of any one of Claims 1-4. An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. An acquisition unit that acquires image data of an image obtained by photographing an object group region including all the objects contained in the container in the state in which the predetermined number is accommodated in the arrangement direction, by an imaging unit;
A detection unit for detecting positions of both ends in the arrangement direction of the object group region indicated by the image data acquired by the acquisition unit;
A derivation unit for deriving a plurality of divided regions by equally dividing the both ends detected by the detection unit by the predetermined number;
An estimation unit that estimates the position of the central portion in the arrangement direction in the plurality of divided regions derived by the deriving unit as the position of the hole or recess in each of the objects;
A recognition unit that recognizes an arrangement position of the object using the image data in which the hole or the recess is masked based on the position estimated by the estimation unit in the image data;
An object position recognition apparatus including: And further includes a superimposing unit that masks with a predetermined mask data at a position estimated by the estimating unit and determined to be appropriate for masking,
The recognizing unit recognizes an arrangement position of the object using the image data in which the hole or the concave portion is masked by the superimposing unit;
The object position recognition apparatus according to claim 6. An object having a hole or a recess provided in a central portion in a container that can store a plurality of objects in a predetermined arrangement direction intersecting a direction in plan view in a direction in which the hole or recess can be viewed in plan view. In a state where a predetermined number is accommodated in the arrangement direction, obtain image data of an image obtained by photographing an object group region including all the objects accommodated in the accommodating body by an imaging unit,
Detecting the positions of both ends in the arrangement direction of the object group region indicated by the acquired image data,
Deriving a plurality of divided areas by equally dividing the detected both ends by the predetermined number,
Estimating the position of the central portion in the arrangement direction in the derived plurality of divided regions as the position of the hole or recess in each of the objects,
An estimation program for causing a computer to execute a process including the above.