JP6052103B2 - Article recognition apparatus and article recognition equipment - Google Patents

Description

  The present invention recognizes a rectangular parallelepiped-shaped article housed in a container having an open top surface, and recognizes the position of the article top surface and the top surface of the article from a basic image that is a captured image from above the container housing the article. The present invention relates to an article recognition apparatus that recognizes a planar view posture.

  As a technique for recognizing the position of an article, for example, in Patent Document 1 below, a rectangular model (template) indicating an outline of the upper surface of an article on an image is generated from the shape data of the article, and the article is obtained from a basic image (image). A binary edge image that emphasizes the shape of the image is created, and a line segment search is performed from the candidate article data created by vector correlation between the rectangular model and the edge image, and the line segment data that forms the edge created from the edge image. A technique for determining an accurate position and posture of an article is disclosed.

  Further, as a position recognition technique for an object, a method of detecting a region matching a template image registered in advance from captured images is also conceivable. For example, Patent Document 2 below discloses a technique for registering a single or a plurality of template images such as a person's face in advance and detecting a matching region that matches the predetermined template image from the captured image. ing. In this way, in the method for detecting a region that matches the template image from the captured image, not only the outline of the template image but also the pattern that is included in the template image is detected. High detection accuracy. Further, since a template image registered in advance is used, the time required for the matching process is relatively short. For these reasons, the method of detecting a region matching the template image from the captured image is effective as a technique for recognizing the position of the article.

  By the way, the above-described object position recognition technology is also used when picking one article from a container in which a plurality of articles are stored by a picking device. Specifically, it is used when the inside of the container (the upper surface of the article) is imaged from above by the imaging device and an area corresponding to the upper surface of one article is detected from the captured image. However, in such a situation, there is a possibility that articles in various storage postures are mixed because the articles roll over in the container when the container is transported to the picking position. For example, when the article has a rectangular parallelepiped shape, there may be a mixture of articles having a storage posture with the short side along the height direction and articles with a storage posture having the long side along the height direction. That is, in such a scene, all six surfaces constituting the article can be the upper surface of the article. Therefore, in the picking device, when performing the matching process using the template image as described above, all the surfaces of the article that can be the article upper surface can be detected so that the region of the article upper surface can be detected in any storage posture. It is necessary to register a template image for. For this reason, when picking an article with a picking device in an equipment that handles various kinds of articles, if the position of the article is to be recognized by the matching process using the template image as described above, the various kinds of articles are used. Therefore, it is necessary to register all the template images of each surface for each of the above, and there is a problem that the registration work takes time. In addition, the amount of data of the registered template image is undesirably large. On the other hand, if the number of template images to be registered is reduced, if the surface corresponding to the unregistered template image is the top surface, the position of the article cannot be recognized, and the recognition rate of the article decreases. There is a fear.

JP 2007-140729 A JP 2010-021817 A

  Therefore, it is desired to realize an article recognition apparatus that simplifies the template image registration operation and suppresses a reduction in the article recognition rate.

  According to the present invention, a rectangular parallelepiped shaped article accommodated in a container having an open top surface is a recognition target, and the position of the article top surface and the article are determined from a basic image that is a captured image from above the container containing the article. The feature configuration of the article recognition apparatus that recognizes the planar view posture of the upper surface includes a template image corresponding to an image of a part of the six rectangular faces of the article and three sides extending from one vertex of the article. From the information acquisition part which acquires dimension information about beforehand, the article search processing part which performs pattern matching processing and edge detection processing, the detection information of the article search processing part, and the dimension information, the article is A storage state for determining a storage posture of which rectangular surface is stored as an upper surface and calculating a height of the upper surface of the article with respect to a reference plane to which the bottom surface of the article belongs Based on detection information of the separate part and the article search processing part, a plane position for specifying a plane position and a planar view posture of the article upper surface from a position in the basic image of the article existence area and an inclination with respect to the basic image The pattern matching process is a process for detecting an area included in the basic image and having a matching degree with the template image that is equal to or greater than a predetermined threshold as an article presence area, The edge detection process generates an edge image generated by extracting an edge component in the basic image, and an area in which a relationship with a rectangular model generated based on the outer shape of each rectangular surface matches a predetermined condition Is detected as the article presence region, and the article search processing unit executes the pattern matching process when an article search command is issued, In the pattern matching process in terms of executing the edge detection processing if it can not detect the article existence region.

  According to this characteristic configuration, the planar position of the article upper surface to be recognized and the arrangement attitude of the article upper surface in plan view from the article presence area detected by the pattern matching process or edge detection process executed by the article search processing unit. It is possible to specify a planar view posture. Further, the storage posture of the article to be recognized can be determined from the article presence area and the dimension information of the article, and the height of the article upper surface from the reference plane can be calculated. That is, it is possible to recognize the three-dimensional position with respect to the reference surface for the upper surface of the article to be recognized. Further, by giving the height of the reference surface from the container bottom surface in advance (including the case where the height of the reference surface is zero), it is possible to recognize the three-dimensional position in the container of the upper surface of the article to be recognized. Become.

  Moreover, according to the said structure, a pattern matching process is performed using the template image corresponding to the image of the one part rectangular surface of a rectangular parallelepiped article. That is, when a pattern or pattern is applied to the rectangular surface of the article, the template image also includes the pattern or pattern. Therefore, in the pattern matching process, not only the outer shape of the rectangular surface but also the region corresponding to the template image including a pattern, a pattern, etc. is detected in an area corresponding to the upper surface of the article. The existence area can be detected with high accuracy. Further, since the template image acquired in advance is used in the pattern matching process, the time required for the matching process can be shortened.

  Moreover, according to the said structure, the template image corresponding to the image of a part (5 or less) rectangular surface of the six rectangular surfaces of an article | item is acquired, and a pattern matching process is performed. Therefore, since it is not necessary to acquire template images for all the rectangular surfaces (six rectangular surfaces) that can be the upper surface of the article, it is not necessary to prepare all template images for the six rectangular surfaces in advance. Therefore, it is possible to simplify the operation of registering an image of the rectangular surface of the article as a template image in advance, and to suppress the data amount of the registered template image.

  On the other hand, if the number of template images to be acquired is reduced, if the template image corresponding to the image of the article upper surface that is the recognition target is not acquired, the pattern matching process performs the article existence region corresponding to the article upper surface of the recognition target. Cannot be detected. Therefore, the recognition rate on the upper surface of the article may be reduced. However, according to the above configuration, since the edge detection process is executed when the article existence area cannot be detected in the pattern matching process, there is a possibility that the article existence area corresponding to the upper surface of the article can be detected. Therefore, even when the number of template images acquired in advance is reduced, it is possible to suppress a decrease in the detection rate of the article presence area, and the recognition rate for the position of the article upper surface corresponding to the article presence area. Can also be suppressed. As a result, it is possible to realize an article recognition apparatus that simplifies the template image registration operation and suppresses a reduction in the recognition rate of the article.

  Here, the storage posture of the article in the container is determined in advance, and the template image is an image corresponding to the pattern of the rectangular surface that is the upper surface in the predetermined storage posture. Is preferred.

  According to this configuration, a template image corresponding to an image of a rectangular surface that is likely to be the upper surface of the article is acquired, and the pattern matching process is executed. Since the article basically has a predetermined storage posture, there is a high possibility that the upper surface of the article is a specific one of the six rectangular faces. Therefore, if a template image for a rectangular surface that is the upper surface in a predetermined storage posture is registered, the possibility that an article presence area can be detected in the pattern matching process increases, and as a result, the pattern matching process is improved. be able to.

  In addition, for each pixel in the basic image, the height information indicating the height from the container bottom surface of the upper surface of the article at the position corresponding to each pixel is acquired, and the most pixels having the same height information are included. The storage orientation when it is assumed that there is a virtual reference plane setting unit having a virtual reference plane as a plane formed by a pixel group having a height and a base upper article that is an article to which the bottom belongs to the virtual reference plane For each of the three different assumed storage postures assumed to be, a pixel having the same height information as the height from the bottom surface of the container in the assumed storage posture is extracted from the basic image to obtain a single height. A single-height image generating unit that generates a height image, wherein the article search processing unit includes the template included in the single-height image in the pattern matching processing. A region where the degree of coincidence with the image is equal to or greater than a predetermined threshold is detected as the article presence region, and in the edge detection process, an edge component included in the single height image is extracted and generated An edge image is generated, and an area where the relationship with the rectangular model matches a predetermined condition is detected as the article existence area, and the storage state determination unit includes the article in the single height image. It is preferable that the assumed storage posture corresponding to the single height image in which the article existence area is detected by the search processing unit is determined as the storage posture and the virtual reference surface is determined as the reference surface. is there.

  According to this configuration, the height from the bottom surface of the container where the upper surface of the article can exist is estimated from the height information of the virtual reference plane and the dimension information of the article, and the pixel having the estimated height information is single-ended. A height image is generated. The article search processing unit narrows the search range to the generated single height image, and searches for the article presence area from the single height image. As a result, the search range during pattern matching processing and edge detection processing can be narrowed down from the entire basic image to single-height images that have a relatively high possibility of detecting the article presence area, so that the article recognition rate It is possible to reduce the processing load at the time of pattern matching processing and edge detection processing and to shorten the processing time while suppressing the decrease in the processing time.

  In addition, when the article presence area is detected in the single height image, the article upper surface exists at the height indicated by the height information of the pixels constituting the single height image. The bottom of the article belongs to the virtual reference plane, and the height information of the pixels constituting the single height image can be set as the height from the bottom of the container on the top of the article to be recognized. The virtual reference plane can be the reference plane to which the bottom surface of the recognition target article actually belongs. Then, the assumed storage posture corresponding to the single height image in which the article presence area is detected can be the actual storage posture of the article. Thereby, without giving the height of the reference surface from the container bottom in advance, the three-dimensional position and the storage posture in the container on the upper surface of the article to be recognized can be recognized, and the article to be recognized can be recognized. The height of the placement surface from the bottom of the container and the storage posture of the article on the placement surface can also be recognized.

  Further, the container accommodates an article layer formed by a plurality of articles having the same shape arranged so that the upper surface of the article belongs to the same height, stacked in the height direction, and the virtual reference plane The setting unit further sets the surface to which the bottom surface of the base stage article, which is an article whose upper surface belongs to the virtual reference plane, belongs as the virtual reference lower surface, and the single height image generation unit For each of the three new assumed storage postures assumed as the storage posture of the base article, pixels having the same height information as the height from the container bottom surface of the upper surface in the new assumed storage posture Are extracted from the basic image to generate three new single-height images, and the storage state determination unit adds one of the three new single-height images to the article search processing unit. The article existence area When it is detected, the assumed storage posture corresponding to the single height image in which the article presence area is detected is determined as the storage posture, and the virtual reference lower surface is determined as the reference surface. Is preferred.

  According to this configuration, not only the upper surface of the article of the base upper article whose bottom surface belongs to the virtual reference plane, but also the upper surface of the article (base article) belonging to the article layer one level lower than the article layer formed of the upper article of the base. A three-dimensional position in the container can also be recognized. As a result, when there is no base upper article but there is a base article, the virtual reference plane is set again, and a new process is not executed again from the beginning. The dimension position can be recognized. Therefore, it is possible to reduce the time taken from the start of the article upper surface position recognition process to the recognition of the article upper surface position.

  In addition, the article search processing unit, in order from the single height image having the largest height among the single height images according to the assumed storage posture generated by the single height image generation unit, the pattern It is preferable to perform a matching process and the edge detection process.

  According to this configuration, the article presence area detected by the pattern matching process and the edge detection process corresponds to the position of the upper surface of the article that is the uppermost surface in the container, and the article recognition device is the article that is the uppermost surface in the container. The position of the upper surface and the planar view posture of the upper surface of the article can be recognized. Therefore, for example, when the article in the container is taken out from above by the picking apparatus or the like using the recognition result of the article recognition device, it can be taken out in order from the uppermost article.

  Further, the storage state determination unit calculates the height from the container bottom surface of the article upper surface from the height from the container bottom surface of the virtual reference plane or the virtual reference lower surface and the height of the article upper surface with respect to the reference plane, A bottom surface for determining whether the height from the container bottom surface of the article upper surface calculated by the storage state determination unit is smaller than the shortest side length among the three side lengths included in the dimension information. It is preferable that the apparatus further includes a check unit, and if the bottom surface check unit determines that the height of the article upper surface is smaller than the length of the shortest side of the article, an error output is performed.

  According to this configuration, when the height from the container bottom surface of the article upper surface calculated by the storage state determination unit is smaller than the length of the shortest side among the three sides of the article, that is, the article bottom surface is If the top surface of the first article belonging to the bottom of the container is recognized, but the calculation result is not possible, such as the bottom of the article being below the bottom of the container, the bottom check unit will output an error output. Do. Thereby, it can suppress that an article recognition device recognizes erroneously a position where the article upper surface cannot actually exist as a position of the article upper surface in the height direction.

  The storage state determination unit includes a pair of similar sides in which the difference in length of each side is smaller than a predetermined determination limit value in the three sides of the article based on the dimension information. And when the posture of the shorter side of the pair of similar sides along the height direction is determined as the storage posture, the storage posture as a result of the determination is It is preferable that the longer side of the pair of similar sides is updated to a posture along the height direction.

  By the way, when the picking device is equipped with a plurality of nozzles and the combination of the suction nozzles used for sucking the upper surface of the article is changed according to the area of the upper surface of the article, it corresponds to the upper surface of the article having a large area. If the upper surface of the article having a small area is to be sucked with the combination of the suction nozzles, the suction may not be possible. On the other hand, if the upper surface of an article having a large area is to be adsorbed by a combination of suction nozzles corresponding to the upper surface of the article having a small area, there is a high possibility that it can be adsorbed. Even when the recognition result of the article recognition apparatus according to the present invention is used for such a picking apparatus or the like, according to the above configuration, the common side and the three different rectangular surfaces of the rectangular parallelepiped article When a set of similar rectangular surfaces constituted by a pair of similar sides is included, the storage state determination unit is configured such that the smaller surface of the similar rectangular surfaces is the article upper surface. Is the discrimination result. Therefore, even in the picking apparatus using these recognition results, picking is performed with a combination of suction nozzles corresponding to the smaller surface of the similar rectangular surfaces. Therefore, in reality, although the article is stored in the container in a posture with the surface with the smaller area as the upper surface, it is erroneously determined as the posture with the surface with the larger area as the upper surface. Even in such a misleading posture, the final discrimination result is updated to the posture with the smaller surface as the upper surface, so that the picking device can appropriately perform the picking process.

  Further, a rectangular area corresponding to the upper surface of the article in the basic image determined by the planar position and planar view posture of the upper surface of the article specified by the planar position specifying unit and the dimensional information is an area of the container in the basic image. Further comprising an article wall edge check unit for determining whether or not included in the presence area, and when the article wall edge check part determines that the rectangular area is not included in the container existence area, A configuration that performs error output is preferable.

  According to this configuration, when the rectangular area corresponding to the upper surface of the article in the basic image is not included within the container existing area in the basic image, that is, a part of the rectangular area corresponding to the upper surface of the article protrudes outside the container. When a recognition result that is not possible in practice is obtained, the article wall check unit outputs an error. Thereby, it can suppress that an article recognition device recognizes the position where an article upper surface cannot actually exist erroneously as a position where an article upper surface exists.

  In addition, the article recognition facility according to the present invention may be configured such that the article recognition apparatus, the moving unit that moves the container in the horizontal direction, and the article search processing unit cannot detect the article existence area. Is preferably provided with a retry control unit that controls the moving unit so as to be horizontally moved from the imaging position of the basic image by a preset amount.

  According to this configuration, when the article presence area cannot be detected in the basic image, the container is horizontally moved by a preset amount from the imaging position of the basic image. As a result, when the basic image includes noise due to light reflection or the like, the influence of light reflection may be reduced by horizontally moving the container and changing the imaging position. Therefore, there is a possibility that the article presence area can be detected in the basic image taken again after the container is moved horizontally. Therefore, the recognition rate of the article by the article recognition apparatus can be improved.

Schematic configuration diagram of an article transport facility according to an embodiment of the present invention Control block diagram of picking apparatus according to an embodiment of the present invention Schematic which shows the mode of the picking of the picking apparatus which concerns on embodiment of this invention The perspective view of the container and article which concern on embodiment of this invention The figure which shows the basic image which concerns on embodiment of this invention The conceptual diagram for demonstrating the single height image generation process which concerns on embodiment of this invention Sectional drawing of the container which concerns on embodiment of this invention Histogram used when generating a single height image according to an embodiment of the present invention The figure which shows the container presence part in the basic image which concerns on embodiment of this invention The figure explaining the pattern matching process which concerns on embodiment of this invention The figure explaining the article recognition process which concerns on embodiment of this invention Flowchart of article recognition processing according to an embodiment of the present invention Flowchart of single height image generation processing according to an embodiment of the present invention

  An embodiment of an article recognition apparatus according to the present invention will be described by taking as an example a case where the present invention is applied to a picking apparatus 1 in an article conveyance facility 100. In the following embodiment, the surface on which the container 51 is placed when the picking conveyor 6 transports the container 51 is a transport surface P, the direction along the transport direction on the transport surface P is the transport direction WX, and the transport In the surface P, a direction orthogonal to the conveyance direction WX is defined as a width direction WY, and a direction orthogonal to the conveyance surface P is defined as a height direction WZ. A coordinate system having these three directions as orthogonal coordinate axes and having an origin set on the conveyance surface P of the picking conveyor 6 is defined as a “reference coordinate system”, and a position coordinate in the reference coordinate system is defined as a reference coordinate. Further, as will be described later, the container 51 is formed in a rectangular shape in plan view, but the direction along which the first side of the container 51 extends in a state where the container 51 is placed and supported on the transport surface P of the picking conveyor 6. In the container transport direction X, the direction along the second side orthogonal to the container transport direction X is the container inner width direction Y, the container transport direction X and the direction orthogonal to the container width direction Y are the container height direction. Let it be Z. A coordinate system having these three directions as orthogonal coordinate axes and having an origin set on the bottom surface of the container 51 is referred to as an “in-container coordinate system”, and position coordinates in the in-container coordinate system are referred to as in-container coordinates. In the present embodiment, the height direction WZ and the in-container height direction Z coincide with each other, and hence are collectively referred to as the height direction Z. This corresponds to the height direction of the present invention.

1. Article Conveying Equipment As shown in FIG. 1, the article conveying equipment 100 includes a picking device 1, a picking conveyor 6, a barcode reader 7, an image pickup device 8, an exit conveyor 9, an entrance conveyor 10, and an article transport conveyor. 11 and an automatic warehouse 13. In the article transport facility 100, first, based on an order such as how many goods (articles) are to be delivered, the container 51 storing the goods (articles) corresponding to the order is taken out from the automatic warehouse 13. Then, the article conveying facility 100 conveys the taken-out container 51 to the picking location 12 via the delivery conveyor 9 and the picking conveyor 6, and the picking device 1 supplies the number of articles 50 corresponding to the order from the container 51. The article 50 taken out is transferred to the article conveyor 11. Then, the container 51 after the number of articles 50 corresponding to the order is taken out is returned to the automatic warehouse 13 via the warehousing conveyor 10.

  The barcode reader 7 reads the barcode 52 (see FIG. 4) attached to the outer surface of the container 51 in a non-contact manner (for example, optical reading). In the present embodiment, the barcode reader 7 is provided at the picking location 12. The barcode 52 includes unique ID information of the container 51, and the barcode reader 7 outputs the unique ID information of the container 51 included in the read barcode 52 to the host controller 2 described later.

  The imaging device 8 is a device (for example, a stereo camera) that can record information on the distance from the captured object to the imaging device 8. The imaging device 8 images the container 51 that has been transported to the picking location 12 from above. In the present embodiment, the imaging device 8 includes a pair of cameras 8A and 8B as shown in FIG. The camera 8A is provided immediately above the center position of the picking location 12 and at a position where the height from the conveyance surface P is known, and images the entire picking location 12 in a planar manner. The other camera 8B is provided above a position slightly deviated from the center position of the picking location 12, and images the entire picking location 12 from obliquely above. In this example, the camera 8B is provided above a position shifted from the center position to the upstream side in the transport direction WX. Then, the imaging device 8 calculates distance information from the camera 8A in each pixel in the image captured by the camera 8A based on the parallax of the camera 8A and the camera 8B, and each pixel calculates distance information from the camera 8A. A captured image is generated. Note that the captured image corresponds to a basic image of the present invention.

  The delivery conveyor 9 conveys the containers 51 that have been conveyed from a storage shelf (not shown) of the automatic warehouse 13 via the stacker crane 13 </ b> B to the picking conveyor 6.

  The picking conveyor 6 transports the containers 51 transported from the delivery conveyor 9 to the picking location 12 and transports the containers 51 to the storage conveyor 10 when the picking process is completed. Further, in the picking conveyor 6, a picking location 12 is set across the entire width direction WY in a partial region (in the vicinity of the center in this example) in the transport direction WX. The picking location 12 is a location where the article 50 is taken out from the container 51 by the picking device 1. The article recognition apparatus 3 according to the present invention is used for recognizing the position of the article 50 to be taken out when the article 50 is taken out from the container 51. In the present embodiment, when the container 51 is transported to the picking location 12, the picking conveyor 6 transports the center location of the picking location 12 and the center location of the bottom surface of the container 51. The position where the center position of the bottom surface of the container 51 is matched with the center position of the picking location 12 is the imaging position of the basic image. Further, the picking conveyor 6 operates the transport surface P so as to horizontally move the container 51 present at the picking location 12 along the transport direction WX based on a retry operation command from the retry control unit 23 described later. In addition, the picking conveyor 6 in this embodiment is corresponded to the moving part of this invention.

  The warehousing conveyor 10 conveys the containers 51 from which the articles 50 that have been conveyed from the picking conveyor 6 have been taken out to the automatic warehouse 13. The container 51 transported to the automatic warehouse 13 is re-stocked into the storage shelf via a stacker crane or the like. The automatic warehouse 13 includes a rack 13A having a plurality of storage shelves that can store the containers 51, and a stacker crane 13B. The article conveyance conveyor 11 conveys the article 50 taken out from the container 51 by the picking device 1 to a collection conveyor (not shown). The article 50 conveyed to the collection conveyor is shipped.

  Here, the container 51 is configured in a rectangular parallelepiped shape whose upper surface is opened so that the inside of the container can be imaged by the imaging device 8. For example, the container 51 is a cardboard box or the like from which a tray or an upper surface is removed. Further, as shown in FIG. 4, the container 51 can store a plurality of articles 50 stacked in each direction of the in-container transport direction X, the in-container width direction Y, and the height direction Z. Specifically, as shown in FIGS. 5 and 7, the container 51 has a height of an article layer formed by a plurality of articles 50 having the same shape and arranged such that the upper surface of the article belongs to the same height. It is housed in a stacked state in the direction Z. In this embodiment, as shown in FIGS. 5 and 7, as the containers 51 that have been transported to the picking location 12, four containers 51 in the container transport direction X and three containers 51 in the container width direction Y are arranged. A description will be given by using an example in which two article layers composed of the individual articles 50 are laminated in the height direction Z, and three articles 50 are laminated in an irregular state. In this embodiment, a rectangular parallelepiped container 51 is used, but a cubic container 51 may be used.

  All the articles 50 accommodated in one container 51 are configured in the same rectangular parallelepiped shape. In addition, all the articles | goods 50 may be comprised by the cube shape of the same shape. In the present embodiment, the dimensions of the three sides of the article 50 are H1, W1, and L1, as shown in FIG. 4, and these dimension information is stored in advance in the article information database 4 (described later). . The relationship between the lengths of the sides of the article 50 is H1 <W1 <L1.

2. Picking Device The picking device 1 includes a host controller 2, an article recognition device 3, and a picking robot 5, and the host controller 2 and the article recognition device 3 are connected so as to be able to transmit information. Further, the host controller 2 and the picking robot 5 are connected so as to be able to transmit information. In the picking apparatus 1, first, when the host controller 2 detects the presence of the container 51 at the picking location 12 using a sensor or the like (not shown), it outputs an article search command to the article recognition apparatus 3. In response to the article search command, the article recognition apparatus 3 executes processing for recognizing the three-dimensional position and planar view posture of the article upper surface at the picking location 12 of the article 50 and outputs the recognition result information to the host controller 2. . The host controller 2 outputs the recognition result information and the container information acquired from the article information database 4 described later to the picking robot 5 together with the operation command. Note that the incidental information output together with the operation command is referred to as operation command information. That is, the operation command information includes recognition result information and container information.

  As shown in FIG. 3, the picking robot 5 includes a hand unit 14, a suction nozzle 15, and a control unit (not shown). The hand unit 14 is provided at the tip of an arm extending from a main body (not shown) of the picking robot 5 so as to be rotatable with respect to the arm. A plurality of suction nozzles 15 that can expand and contract in the lower space of the hand portion 14 are provided on the lower surface of the hand portion 14. The suction nozzle 15 extends to the upper surface of the article recognized by the article recognition apparatus 3, and takes out the article 50 from the container 51 by contracting upward while the upper surface of the article is sucked. The control unit receives the operation command and the operation command information (recognition result information and container information) from the robot control unit 24 of the host controller 2, and the container position information and the article included in the recognition result information acquired from the robot control unit 24 From the information of the three-dimensional position and the planar view posture of the upper surface, the three-dimensional position and the planar view posture of the article upper surface in the reference coordinate system are calculated. Based on this, the control unit determines the entry angle of the hand unit 14 into the container 51 and the suction nozzle 15 to be used. The picking robot 5 operates based on the position coordinates of the reference coordinate system. Then, the control unit sucks the article 50 to be recognized by the suction nozzle 15 thus determined, takes out the article 50, moves the arm, and transfers the taken-out article 50 to the article transport conveyor 11. Below, the detailed description of each structure of the high-order controller 2 and the goods recognition apparatus 3 is given.

2-1. Host Controller The host controller 2 includes an article information acquisition unit 21, a container information acquisition unit 22, a retry control unit 23, and a robot control unit 24. The host controller 2 is connected to an article information database 4, a picking conveyor 6, and a bar code reader 7 provided separately from each other in addition to the picking robot 5 and the article recognition apparatus 3.

  Here, the article information database 4 associates the dimension information of the three sides of each container 51 master-registered by another device (not shown) with the ID information assigned to each container 51, and thereby master container Stored as information. Note that the master container information includes ID information of the articles 50 accommodated in the containers 50 and information on the number of stocks of the articles 50 in each container 51 in association with the container ID information.

  Further, the article information database 4 associates each dimension information and template image of a plurality of types of articles 50 master-registered by another device (not shown) with the ID information of each article 50 as master article information. Storing. The dimension information of the article 50 is information indicating the dimensions of the three sides extending from one vertex of the article 50. The template image is an image that serves as a template for detecting the article presence region M by pattern matching processing described later. The template image included in the master article information in the present embodiment corresponds to an image of a part of the six rectangular faces of the article 50. Specifically, as shown by a region surrounded by a broken line in FIG. 10, two of the six rectangular surfaces of the article 50 are imaged by another imaging device (not shown), respectively. A rectangular image based on the captured image is registered in advance as template images P1 and P2. At that time, the size and resolution of each template image are also registered. The number registered as the template image may be five or less, and template images corresponding to images of one, three, or four rectangular surfaces of the six rectangular surfaces of the article 50 are included. Master registration may be performed in advance. Further, in the present embodiment, the planned storage posture of whether or not the rectangular shape of the article 50 is stored in the container 51 in a posture with the upper surface as the upper surface is determined in advance for each container 51 and is master-registered. The information includes the scheduled storage posture registered as the master in association with the ID information of the container 51. In the present embodiment, as shown in FIG. 4, a posture in which the side of the length H <b> 1 is along the height direction and the surface on which the product name “XXX” is displayed is the top storage posture. It is registered as. In addition, one of the template images included in the master article information is an image corresponding to a rectangular pattern that is an upper surface in the planned storage posture of the article 50. In this embodiment, as shown in FIG. This is the template image P2 on which the name “XXX” is displayed.

  The container information acquisition unit 22 is a functional unit that acquires unique container information of the container 51 existing in the picking location 12 from the master container information of the article information database 4. Specifically, the container information acquisition unit 22 acquires the ID information of the container 51 from the barcode reader 7 when detecting that the container 51 exists at the picking location 12. And the container information acquisition part 22 acquires the container information (dimension information of the container 51) of the container 51 matched with the said ID information from the master container information stored in the article information database 4. Then, the container information acquisition unit 22 outputs the acquired container information to the robot control unit 24 as operation command information.

  The article information acquisition unit 21 is a functional unit that acquires the article information of the article 50 accommodated in the container 51 existing in the picking location 12 from the master article information in the article information database 4. Specifically, the article information acquiring unit 21 is associated with the ID information of the container 51 from the master article information stored in the article information database 4 when the container information acquiring unit 22 acquires the container information. The article information (the dimension information and the template image of the article 50) of the article 50 being acquired is acquired. Then, the article information acquisition unit 21 outputs the acquired article information to the article recognition device 3.

  The retry control unit 23 is a functional unit that controls the picking conveyor 6 so as to horizontally move the container 51 by a predetermined amount from the imaging position of the basic image when the article presence region M cannot be detected. In the present embodiment, the retry control unit 23 controls the picking conveyor 6 based on the output result from the article recognition device 3. Specifically, when the retry control unit 23 receives information from the single height image generation unit 36 (described later) indicating that no article search area exists in any single height image, When information indicating that the article presence area M could not be detected in any single height image set as the article search range is received from the detection processing unit 39 (described later), an error output is output from the check processing unit 45. It works when it is done. In addition, when the retry control unit 23 in the present embodiment receives the output result from the article recognition device 3, first, the retry control unit 23 outputs the number of times the retry operation command is output to the picking conveyor 6 after the container 51 is transported to the picking location 12. calculate. When the retry control unit 23 determines that the number of output of the retry operation command is 0, the first retry set amount that is set in advance from the imaging position of the basic image toward the downstream side in the transport direction WX. A retry operation command is output to the picking conveyor 6 so as to move horizontally (in this example, 75 mm). When the retry control unit 23 acquires information indicating that the horizontal movement is completed from the picking conveyor 6, the retry control unit 23 outputs an article search command to the article recognition apparatus 3, and reprocessing is performed in the article recognition apparatus 3. In addition, when the retry control unit 23 determines that the number of output of the retry operation command is one, the container 51 has already been horizontally moved from the imaging position to the downstream side in the transport direction WX by a preset amount (75 mm). State. Therefore, the retry control unit 23 performs the first retry for the first retry on the upstream side in the transport direction WX with respect to the imaging position from the position shifted by the first retry setting amount (75 mm) on the downstream side in the transport direction WX with respect to the imaging position. The picking conveyor 6 is controlled so that the container 51 is horizontally moved to a position shifted by a set amount (75 mm). That is, the retry control unit 23 sets the second retry setting amount preset on the upstream side in the transport direction WX from the position shifted from the imaging position to the downstream side in the transport direction WX by the first retry setting amount (75 mm). A retry operation command is output to the picking conveyor 6 so as to move horizontally (in this example, 150 mm). When the retry control unit 23 acquires information indicating that the horizontal movement is completed from the picking conveyor 6, the retry control unit 23 outputs an article search command to the article recognition apparatus 3 via the robot control unit 24. The process is executed again. In addition, when the retry control unit 23 determines that the number of output of the retry operation command is two times, the retry control unit 23 outputs a reset instruction to the picking conveyor 6, and the picking conveyor 6 horizontally places the container 51 at the original imaging position. Move. Note that the article recognition apparatus 3, the host controller 2 (retry control unit 23), and the picking conveyor 6 according to the present embodiment correspond to the article recognition equipment of the present invention.

  The robot control unit 24 is a functional unit that controls the picking robot 5. In the present embodiment, when the robot control unit 24 detects that the container 51 has been transported to the picking location 12 by a sensor or the like, the robot control unit 24 outputs an article search command to the article recognition device 3, and the container information acquisition unit 22. The container information acquired in step 1 and the item information acquired by the item information acquisition unit 21 are output to the item recognition apparatus 3. In addition, when the robot control unit 24 acquires the recognition result information output from the article recognition device 3, the robot control unit 24 outputs an operation command to the control unit of the picking robot 5 and also outputs the operation command information. The recognition result information is container position information output from the container posture / position determining unit 33 and information on the three-dimensional position and planar view posture of the article upper surface output from the article position specifying unit 41.

2-2. Article Recognition Apparatus The article recognition apparatus 3 is an apparatus that recognizes a three-dimensional position of an article upper surface and a planar view posture that is an arrangement state of the article upper surface in plan view. The article recognition apparatus 3 includes an image acquisition unit 31, a container outer shape detection unit 32, a container posture / position determination unit 33, an article search range setting unit 34, an article search processing unit 37, a storage unit 40, and an article position. The identification unit 41, the information acquisition unit 44, and the check processing unit 45 are provided. And the article recognition device 3 of the present embodiment predicts the height of the surface that can be the uppermost surface in the container 51, generates a single height image only from pixels having the height information of the height, This is based on the technical idea that if a region matching the rectangular surface of the article 50 can be detected in the single height image, it can be recognized that there is an upper surface of the article that can be the uppermost surface at the height. The article recognition apparatus 3 includes an arithmetic processing unit such as a CPU as a core member, and is implemented by hardware and / or software (program) or both as a functional unit for performing various processes on input data. It is configured.

  The image acquisition unit 31 is a functional unit that acquires a captured image from the imaging device 8. In the present embodiment, when receiving an article search command from the host controller 2, the image acquisition unit 31 outputs an imaging instruction to the imaging device 8, and takes a captured image (basic image) captured based on the imaging instruction. Obtained from the imaging device 8. Further, since the basic image includes distance information from the imaging device 8 (camera 8A) to the upper surface of the article imaged, the image acquisition unit 31 includes the distance information and the known imaging device 8 (camera 8A). From the height from the conveyance surface P (the container bottom surface), for each pixel in the basic image, obtain height information indicating the height from the container bottom surface of the article upper surface at a position corresponding to each pixel, A basic image is generated in which the pixels have height information from the bottom of the container. Then, the image acquisition unit 31 causes the storage unit 40 to store a basic image in which each of the generated pixels has height information from the container bottom surface. In addition, the image acquisition unit 31 instructs the container outer shape detection unit 32 to execute a container detection process. An example of the basic image is shown in FIG.

  The information acquisition unit 44 is a functional unit that acquires the container information of the container 51 existing in the picking location 12 and the article information of the article 50 stored in the container 51, which are output from the host controller 2. In the present embodiment, the information acquisition unit 44 stores the acquired container information and article information in the storage unit 40. Since the article information includes the template image of the article 50 and the dimension information of the article 50, the information acquisition unit 44 corresponds to the information acquisition unit of the present invention.

  The container outer shape detection unit 32 is a functional unit that detects whether or not the outer shape of the container 51 is included in the basic image. Specifically, the container outline detector 32 first extracts the container outline from the contrast difference between the color of the transport surface P of the picking location 12 and the color of the container 51 in the basic image. Further, the container outer shape detection unit 32 receives the dimension information of the container 51 included in the container information from the storage unit 40, particularly the dimension in the direction along the transport direction WX and the width direction WY in a state where the container 51 is placed on the transport surface P. Dimension information in the direction along When the container outer shape detection unit 32 compares the extracted container outer shape with the acquired dimension information of the container 51 and determines that they match within a predetermined allowable range, the container outer shape detection unit 32 Recognize that there is. Then, the container outer shape detection unit 32 instructs the container posture / position determination unit 33 to specify the planar view posture and the planar position of the container 51, and sets the article search range to the article search range setting unit 34. Command to set. Note that the container outer shape detection unit 32 outputs an error report of a container detection error to the host controller 2 when the extracted container outer shape does not match the dimensional information of the container 51 within a predetermined allowable range. The host controller 2 that has received the error report outputs a reset instruction to the article recognition device 3. As a result, the captured image acquired by the image acquisition unit 31 stored in the storage unit 40 and the container information and article information acquired by the information acquisition unit 44 are deleted.

  The container posture / position determining unit 33 is a functional unit that determines the planar view posture of the container 51 in the basic image and specifies the planar position. Here, the basic image corresponds to the area of the picking location 12 as shown in FIG. In the present embodiment, the corner on the article transport conveyor 11 side in the width direction WY on the downstream side in the transport direction WX of the picking location 12 corresponds to the lower left corner of the basic image (the left side in FIG. 5). Then, with this position as the origin WO, the WX axis is taken in the direction along the conveyance direction WX direction of the basic image, and the WY axis is taken in the direction along the width direction WY, and the coordinate system is taken as the image coordinate system. In this embodiment, the image coordinate system matches the direction of each coordinate axis with the reference coordinate system used in the control of the picking robot 5. In this embodiment, when the container posture / position determination unit 33 receives a command from the container outer shape detection unit 32, as shown in FIG. 5, the four corners (51A, 51B, 51C, 51D) of the container 51 in the basic image are displayed. Image coordinates (WX, WY) are respectively calculated. Further, the container posture / position determining unit 33 calculates a container rotation angle indicating the inclination of the container 51 with respect to the basic image. Then, the container posture / position determination unit 33 outputs the calculated image coordinates (WX, WY) of the container 51 and the container rotation angle to the robot control unit 24 of the host controller 2 as container position information.

  The article search range setting unit 34 is a functional unit that sets an article search range that is a range in which the search process is executed when searching for the upper surface of the article 50 that is the recognition target in the basic image. In the present embodiment, the article search range setting unit 34 includes a virtual reference plane setting unit 35 and a single height image generation unit 36. Then, the article search range setting unit 34 determines a surface that can be the uppermost surface in the container 51 (a surface on which the upper surface of the article that is the uppermost surface can exist), and a pixel having height information of the surface that can be the uppermost surface. A single height image is generated, and the article search range to be searched by the article search processing unit 37 is narrowed down from the entire basic image to the height image. Thereby, an article search range can be narrowed and the processing burden of the article recognition apparatus 3 can be reduced. In addition, the picking robot 5 picks up the upper surface of the article 50 by sucking the suction nozzle 15 from above. At that time, if the article 50 on the bottom surface side in the container 51 is to be picked first, the article 50 on the top surface side may be damaged by the hand portion 14 or the like. In order to suppress this, in the present embodiment, the article recognition process is preferentially executed and picked from the article 50 to which the article upper surface belongs to the uppermost surface in the container 51.

  The virtual reference plane setting unit 35 uses a virtual reference plane as a plane on which the article 50 having the article upper surface is placed on a plane that can be the uppermost face in the container 51, that is, a plane to which the bottom face of the article 50 belongs. It is a function part set as a surface and a virtual reference lower surface. In the present embodiment, the virtual reference plane setting unit 35, as shown in FIGS. 5 and 7, when the container 51 is viewed from above, the height to which the widest plane in the container 51 belongs in a plan view, that is, A reference height H is a height at which most pixels having the same height information are included in the basic image, and a plane formed by the pixel group having the height information of the reference height H is a virtual reference plane. Set as S. Specifically, the virtual reference plane setting unit 35 acquires a basic image from the storage unit 40 based on a command from the container outer shape detection unit 32. Based on the basic image, the virtual reference plane setting unit 35 takes the pixel appearance rate (number of pixels) on the vertical axis and the pixel value (height of each pixel) on the horizontal axis, as shown in FIG. Generate a histogram. Then, the virtual reference plane setting unit 35 determines the mountain having the largest area when the histogram is divided by valleys (the mountain on the left side of the paper in FIG. 8), and the pixel value having the highest appearance rate among the peaks. Is the reference height H (the pixel value of the third histogram from the left side of FIG. 8). The histogram is created with a step size on the horizontal axis of 1 mm. Therefore, the pixel group having the height information of the reference height H includes not only the pixels having the height information of the reference height H but also within a predetermined allowable range from the reference height H (for example, about ± 1 mm). A configuration including pixels having height information (deviation height), that is, histograms on both sides of the histogram (second and fourth on the left side of FIG. 8) where the appearance rate peaks in the mountain on the left side of FIG. The pixel value may also be a pixel value having a reference height H. In this embodiment, as shown in FIG. 5 and FIG. 7, the plane formed from the upper surface of the article 50 of the second article 50 laminated in the height direction Z has the widest planar area, The plane corresponds to the virtual reference plane S in the basic image. Then, the upper surface of the article 50 (the third-layer article) whose bottom surface belongs to the virtual reference plane S is the uppermost surface in the container 51. Note that the article 50 (the third-layer article) whose bottom surface belongs to the virtual reference plane S corresponds to the base upper article of the present invention.

  By the way, as a method of calculating the height of the article upper surface, a method of directly calculating by the method using the above-described histogram is also conceivable. The height of the upper surface of the article calculated by the histogram corresponds to a mountain indicated on the right side of the sheet of FIG. 8 at a position higher than the virtual reference plane. However, since the number of pixels (appearance rate) serving as an index is small and the pixel value (height) has a width (3 mm in the example of FIG. 8) in the mountain shown on the right side of FIG. There is a high possibility that an error (2 mm in the example of FIG. 8) will occur between the height of and the height calculated by the histogram. Therefore, in the present application, a virtual reference plane is set on a plane where the number of pixels serving as an index is the largest, that is, the appearance rate is at a peak and the error from the actual height is small, and an article is placed on the virtual reference plane. Is estimated from the reference height H of the virtual reference surface and the dimensional information on the three sides of the known article, and the height of the article upper surface in each of three different assumed storage postures. If it can be detected that the article existence area corresponding to the article upper surface exists in the height, a method of setting the actual article upper surface height is used. Thereby, a more accurate height of the upper surface of the article can be calculated.

  Further, in the present embodiment, as illustrated in FIGS. 5 and 7, an example in which three articles 50 (third article) are stacked on the virtual reference plane S in an irregular manner will be described. However, the article 50 (the article on the left side in FIG. 6) on the virtual reference plane S may not exist. For example, in the conceptual diagram of FIG. 6, when there is no article 50 on the virtual reference plane S shown on the left side of the page, as shown on the right side of the page, The upper surface can be the uppermost surface in the container 51. Specifically, all the second-layer articles 50 are stored in a storage posture (upright posture) in which the sides of the length H1 are along the height direction, and the third-layer article 50 is placed on the second-layer article 50. When the article 50 is not placed, the top surface is the virtual reference plane S. Further, when the second-layer article 50 group includes articles 50 having a storage posture (rolling long posture) in which the side of the length L1 is along the height direction by rollover or the like, H1 <L1. Therefore, the upper surface of the article 50 in the rollover long posture is the uppermost surface. Further, in the case where the second-layer article 50 group includes the article 50 with the side of the length W1 along the height direction (the rollover short attitude) and does not include the article 50 with the rollover long attitude. Since H1 <W1, the upper surface of the article 50 in the rollover short posture is the uppermost surface. Therefore, the virtual reference plane setting unit 35 further sets the plane to which the bottom surface of the base article (the second article in this example) whose article upper surface belongs to the virtual reference plane S belongs as the virtual reference lower surface SS. Specifically, the virtual reference plane setting unit 35 subtracts the height H1 of the article 50 that is an upright posture included in the article information acquired from the storage unit 40 from the reference height H of the virtual reference plane S. A plane formed by the pixel group having the height information of “H−H1” is set as the virtual reference lower surface SS. Further, the virtual reference plane setting unit 35 causes the storage unit 40 to store information on the set virtual reference plane S and virtual reference lower surface SS.

  When it is assumed that the single-height image generation unit 36 has a base upper article (third article) that is the article 50 to which the bottom belongs to the virtual reference plane S set by the virtual reference plane setting unit 35, For each of the three different assumed storage postures assumed as the storage posture of the article 50, the logical height from the container bottom surface of the upper surface in the assumed storage posture is calculated, and the same height information as the logical height is calculated. Is extracted from the basic image to generate a single height image. In the present embodiment, as the assumed storage posture of the article 50, as shown on the left side of FIG. Therefore, the single height image generation unit 36, based on the dimension information of the article 50 acquired from the storage unit 40 and the reference height H of the virtual reference surface S, the container bottom surface that is the top surface in each assumed storage posture. Calculate the logical height from. Specifically, as shown on the left side of FIG. 6, the single height image generation unit 36 “H + H1” in the upright posture SC, “H + L1” in the rollover long posture SA, and rollover. In the case of the short posture SB, “H + W1” is calculated. Then, the virtual reference plane setting unit 35 extracts, from the basic image, pixels existing in the inner region of the container 51 from among the pixels having the height information of the calculated logical heights, and for each height. Generate a height image.

  The single-height image generation unit 36 further relates to each of the three new assumed storage postures assumed as the storage postures of the base stage article (second-layer article) on the virtual reference lower surface SS. Calculates the logical height of the upper surface in the assumed storage posture from the bottom surface of the container, and extracts three pixels having the same height information as the logical height from the basic image to generate three new single height images. To do. In the present embodiment, as shown on the right side of FIG. 6, as the three new assumed storage postures, an upright posture SSC, a rollover long posture SSA, and a rollover short SSB posture on the virtual reference lower surface SS can be considered. The single-height image generation unit 36 is a container on the upper surface in each assumed storage posture based on the dimension information of the article 50 acquired from the storage unit 40 and the height “H-H1” of the virtual reference lower surface SS. Calculate the logical height from the bottom. Specifically, as shown on the right side of FIG. 6, the single height image generation unit 36 is “H” in the upright posture SSC, “H−H1 + L1” in the rollover long posture SSA, In the case of the rollover short posture SSB, “H−H1 + W1” is calculated. Then, the virtual reference plane setting unit 35 extracts pixels existing in the inner region of the container 51 from the pixels having the calculated height information of each height from the basic image, and calculates the height of each height. Generate an image.

  Then, the single height image generation unit 36 performs binarization processing on each of the generated single height images corresponding to the upper surfaces of the six assumed storage postures, and generates a region corresponding to the upper surface of the article 50. It is determined whether or not (article search area) exists. For example, when there is no base upper article, that is, when the article 50 does not exist on the virtual reference plane S, the upright posture SC, the rollover long posture SA, and the rollover short on these virtual reference surfaces S. There is no region corresponding to the upper surface of the article 50 in the single height image corresponding to the three assumed storage postures of the posture SB. Therefore, the single height image generation unit 36 determines that there is no article search region in these single height images. On the other hand, for example, when there is a base step article in the rollover long posture SSA on the virtual reference lower surface SS, a single height image corresponding to the rollover long posture SSA on the virtual reference lower surface SS is displayed on the single height image. There is a region corresponding to the upper surface. Therefore, the single height image generation unit 36 determines that the article search region exists in the single height image corresponding to the rollover long posture SSA on the virtual reference lower surface SS. Here, the existence of the article search area means that there is a continuous area that is equal to or larger than a predetermined range, and in this example, a single height corresponding to each storage posture of the article 50. It is assumed that an article search region exists when an image includes a continuous region that is equal to or larger than the surface area of the rectangular surface that is the upper surface in the corresponding assumed storage posture. The article search area may exist when the area is a continuous area equal to or larger than the area of the rectangular surface × a predetermined threshold% (for example, 90%).

  In the present embodiment, the single height image generator 36 converts the single height image determined to have the article search region into a color image, assuming that the single height image is an effective single height image. Then, it is stored in the storage unit 40 as an article search range. Then, the single height image generation unit 36 performs a determination process for determining whether or not a similar article search region exists for all the single height images corresponding to the six assumed storage postures, and performs the determination. When the process ends, if an effective single height image is found, the article search processing unit 37 is instructed to execute the article search process. On the other hand, when the single height image generation unit 36 determines that the article search region does not exist in any of the six single height images corresponding to the six assumed storage postures, the single height image generation unit 36 notifies the host controller 2 to that effect. The information of is output. Then, the retry control unit 23 of the host controller 2 executes a retry process. In the example of FIG. 7, since there is an upright base upper article and an upright base article, the single-height image generation unit 36 has a single height of “H” and “H1 + H”. It is determined that an article search area exists in the one-height image, and a single height image having the heights “H” and “H1 + H” is set as an article search range and stored in the storage unit 40.

  The article search processing unit 37 performs a pattern matching process on the basic image using a template image, and executes an edge detection process when an article presence region M (described later) cannot be detected in the pattern matching process. It is. The article search processing unit 37 includes a pattern matching processing unit 38 and an edge detection processing unit 39.

  The pattern matching processing unit 38 selects an area in the single height image set as the article search range by the article search range setting unit 34 and having a matching degree with a template image equal to or greater than a predetermined threshold. This is a functional unit that is detected as the existence region M. Moreover, in this embodiment, the pattern matching process part 38 is an effective single height image according to the assumption stowed attitude | position produced | generated by the single height image generation part 36 in an order from a high height in order. A pattern matching process and an edge detection process are executed. Specifically, the pattern matching processing unit 38 first sets the assumed storage posture (the rollover long posture in this example) having the highest upper surface among the three assumed storage postures of the base upper article as the SA posture, The assumed stowed posture with the second highest surface (in this example, the rollover short posture) is the SB posture, and the assumed stowed posture with the lowest upper surface (in this example, the upright posture) is the SC posture. The three assumed storage postures of the corrugated article are also stored in the storage unit 40 as the SSA posture, the SSB posture, and the SSC posture in ascending order of the upper surface of the article in association with these assumed storage postures and the corresponding single height images. . At that time, the pattern matching processing unit 38 also stores the dimension information and the aspect ratio of the rectangular surface that is the upper surface of the article 50 in each posture in association with each posture. And the pattern matching process part 38 performs a pattern matching process in order of the high height single height image corresponding to SA posture, SB posture, SC posture, SSA posture, SSB posture, and SSC posture. In the example of FIG. 7, since the single height images corresponding to the SA posture, SB posture, SSA posture, and SSB posture are not stored in the storage unit 40, single height images associated with these postures. Does not exist and no single height image is associated. On the other hand, for the SC posture and the SSC posture, corresponding single height images (single height images of “H” and “H1 + H”) are stored in the storage unit 40. Is stored again in association with each posture (SC posture and SSC posture), dimension information (L1 × W1) of the rectangular surface that is the upper surface of the article 50 in each posture, aspect ratio (W1: L1), and the like. Stored in the unit 40. Then, the pattern matching processing unit 38 executes the pattern matching processing first from the single height image (“H1 + H” single height image) of the SC posture having a large height. Note that the pattern matching processing unit 38 stores, in the storage unit 40, a single corresponding to the assumed storage posture for the assumed storage posture (SA posture, SB posture, SSA posture, SSB posture) for which the corresponding single height image is not valid. Since the one-height image is not stored, the pattern matching process is not executed.

  When the pattern matching processing unit 38 receives an execution command for the article search process from the article search range setting unit 34, the pattern matching processing unit 38 acquires one template image included in the article information from the storage unit 40 as a search template, and as an article search range. The single-height image information having the highest height among the set single-height images is acquired. Then, the pattern matching processing unit 38 first determines the size of the search template and the size of the rectangular surface (hereinafter simply referred to as the upper surface of the assumed storage posture) as the upper surface in the assumed storage posture corresponding to the acquired single height image. Detect whether they match and are similar. If the dimensions of the upper surface of the assumed storage posture match and are similar, the pattern matching processing unit 38 matches the search template with the color image of the single height image. When the resolution of the color image and the resolution of the search template are different, the color image is enlarged at a magnification calculated from these resolutions, and the search template is matched with the enlarged color image. Then, when the pattern matching processing unit 38 detects an area in the color image where the degree of coincidence with the search template image is equal to or higher than a predetermined threshold (for example, 80% or more of the number of pixels constituting the search template). The detected area is stored in the storage unit 40 as the article presence area M, and an operation command is output to the article position specifying unit 41. At this time, the fact that the pattern matching method has been detected, the information on the assumed storage posture corresponding to the single height image that is the article search range at the time of detection, and the stage to which the article 50 in the assumed storage posture belongs ( Base stage or base upper stage (hereinafter, detection stage) is also stored. On the other hand, the pattern matching processing unit 38 is a region where the dimensions of the search template and the upper surface of the assumed storage posture do not match or are not similar, or the degree of match with the image of the search template is equal to or greater than a predetermined threshold. Is not detected from the storage unit 40 as a search template, and the same processing as described above is executed again. If the article presence region M cannot be detected even when the matching process is executed using all the template images included in the article information, an operation command is output to the edge detection processing unit 39. Further, the dimensions of the search template can be calculated from the image size and resolution at the time of master registration. Note that the template image of the present embodiment is a color image and includes a pattern formed on each rectangular surface.

  Further, when there is an operation command to execute the pattern matching process again from the edge detection processing unit 39, the pattern matching processing unit 38 sets the single article search range stored in the storage unit 40 as a single item. Among the height images, single-height image information having the second highest height is acquired, and pattern matching processing similar to the above-described processing is executed for the newly acquired single-height image.

  7-10, the single height image T1 corresponding to the upper surface of the article in the upright posture (SC posture) of the base upper article is a single height stored in the storage unit 40. It is a single height image (height image of “H + H1”) having the highest height among the images. Therefore, the pattern matching processing unit 38 acquires the information of the single height image T1 and the template image P1 that is one of the template images as a search template from the storage unit 40. However, the template image P1 and the rectangle on the upper surface of the article in the SC posture are not similar and have different dimensions. Therefore, the pattern matching processing unit 38 acquires another template image P2 from the storage unit 40 as a search template, assuming that there is no region matching the template image P1 in the single height image T1, that is, the article presence region M. The dimensions of the rectangular surface of the upper surface of the article in the template image P2 and the SC posture match and have a similar relationship. Therefore, the pattern matching processing unit 38 matches the template image P2 with the color image of the single height image T1. Then, as shown in the lower left area of the single height image T1 in FIG. 10, the pattern matching processing unit 38 has an article presence area M whose degree of coincidence with the image of the template image P2 is greater than or equal to a predetermined threshold value. Is detected. Then, the pattern matching processing unit 38 stores information on the SC posture, which is the assumed storage posture corresponding to the single height image T1, information on the article existence region M, information on the base upper article, which is the detection stage, and the like. Remember me.

  When receiving an operation command from the pattern matching processing unit 38, the edge detection processing unit 39 extracts an edge component included in the single height image used in the pattern matching processing unit 38 and generates an edge image. Then, the edge detection processing unit 39 sets an area in which the relationship between the rectangular model generated based on the outline of each rectangular surface of the article 50 included in the generated edge image matches a predetermined condition. This is a functional unit that is detected as a region M. Specifically, the edge detection processing unit 39 first obtains dimensional information and aspect ratio information on the upper surface of the assumed storage posture corresponding to the single height image used in the pattern matching processing unit 38 from the storage unit 40. To do. And the edge detection process part 39 produces | generates the rectangular model which is a template of the external shape of the said rectangular surface based on such information. On the other hand, the edge detection processing unit 39 extracts edge components included in the single height image used in the pattern matching processing unit 38 to generate an edge image. Then, using the generated edge image and the rectangular model, the edge detection processing unit 39 searches for an area in the edge image where the relationship with the rectangular model matches a predetermined condition. Then, the edge detection processing unit 39 satisfies a predetermined relationship with the rectangular model (for example, an area surrounded by three edges that match three of the four sides of the rectangular model). When a matching area is detected, the area is stored in the storage unit 40 as the article presence area M, and an operation command is output to the article position specifying unit 41. At that time, the edge detection processing unit 39 also includes information indicating that detection has been performed by edge detection, information on the assumed storage posture corresponding to the single height image set in the article search range at the time of detection, and a detection stage. Also memorize it.

  On the other hand, if the edge detection processing unit 39 fails to detect an area in the edge image that matches the predetermined condition with the rectangular model, the edge detection processing unit 39 performs the pattern matching processing again on the pattern matching processing unit 38. An operation command to execute is output. Note that the edge detection processing unit 39 counts the number of outputs of the operation command to execute the pattern matching process again, and when the operation command is output, the number of output of the operation command is set as “article search range”. If the number of single height images is −1 ”, the article presence region M can be detected in any single height image set as the article search range without outputting the operation command. If not, information indicating that is output to the host controller 2. Then, the retry process is executed by the retry control unit 23 of the host controller 2.

  The article position specifying unit 41 is a functional unit that specifies a three-dimensional position and a planar view posture of the article upper surface in the container 51 of the article 50 to be recognized. The article position specifying unit 41 includes a storage state determining unit 42 and a planar position specifying unit 43. In the present embodiment, upon receiving an operation command from the article search processing unit 37 (the pattern matching processing unit 38 or the edge detection processing unit 39), the article position specifying unit 41 acquires information on the article presence region M from the storage unit 40. To do. Then, the storage state determination unit 42 determines the storage posture of the article 50 with respect to the reference plane, specifies the position in the height direction Z in the container 51 of the upper surface of the article 50 to be recognized, and the plane position specification unit 43 specifies the planar position and planar view posture of the article 50 in the container 51 from the article existence region M. Then, the article position specifying unit 41 outputs the information on the three-dimensional position and the planar view posture of the article upper surface in the specified container 51 to the check processing unit 45. When the article position specifying unit 41 receives a response indicating that the article is normal from the check processing unit 45, the article position specifying unit 41 determines the three-dimensional position and the planar view posture of the article upper surface in the specified container 51, and stores these information in the host controller. 2 is output as recognition result information to the robot controller 24.

  The storage state discriminating unit 42 discriminates the storage posture of the rectangular article from which the article 50 is stored from the detection information of the article search processing unit 37 and the dimension information of the article 50, and the article. This is a functional unit that calculates the height of the upper surface of the article relative to the reference surface to which the bottom surface of 50 belongs. In the present embodiment, the storage state determination unit 42 is a single height image in which the article presence region M is detected by the article search processing unit 37 among the single height images stored in the storage unit 40. Is one of the assumed storage postures when it is assumed that the base upper article is present, the assumed storage posture corresponding to the single height image from which the article presence region M is detected is The virtual reference plane S is determined to be the reference plane to which the bottom surface of the article 50 actually belongs while being determined as being in the actual storage posture. Further, the storage state determination unit 42 is a hypothetical storage corresponding to a single height image in which the article presence region M is detected by the article search processing unit 37 among the single height images stored in the storage unit 40. When the posture is one of three new assumed storage postures when it is assumed that the base stage article exists, the article search processing unit 37 among the single height images stored in the storage unit 40. In FIG. 5, the assumed storage posture corresponding to the single height image in which the article presence area M is detected is determined as the actual storage posture, and the virtual reference lower surface SS is determined as the reference surface.

  Specifically, the storage state determination unit 42 has information on the article presence area M as detection information of the article search processing unit 37, and an assumed storage posture corresponding to the single height image in which the article presence area M is detected. Information and a detection stage are acquired from the storage unit 40. Then, the storage state determination unit 42 determines the assumed storage posture corresponding to the single height image in which the article presence area M is detected as the storage posture of the article 50. 7 to 10, when the storage state determination unit 42 acquires information on the article presence area M from the storage unit 40, the single height image in which the article presence area M is detected. Information on the SC posture and the detection stage corresponding to T1 are also acquired. Then, the storage state determination unit 42 determines that the storage posture is an upright posture of the base upper article, and determines the virtual reference surface S as a reference surface. The storage state determination unit 42 calculates the height of the article upper surface from the container bottom surface from the height of the virtual reference surface S or the virtual reference lower surface SS from the container bottom surface and the height of the article upper surface with respect to the reference surface. 7 to 10, the storage state determination unit 42 determines the height “H” of the virtual reference surface S determined as the reference surface and the height of the upper surface of the article relative to the reference surface, that is, the determined storage posture ( From the length “H1” of the side along the height direction in the upright posture), the height “H + H1” of the upper surface of the article from the bottom surface of the container is calculated. Thereby, the position of the upper surface of the article in the container 51 in the height direction Z can be specified. Note that the height of the single height image T in which the article presence region M is detected may be the height from the bottom of the container on the top of the article.

  The storage state determination unit 42 also has a determination limit in which a difference in length of each side is predetermined among the three sides of the article 50 based on the dimension information of the article 50 stored in the storage unit 40. A pair of similar sides smaller than the value is included, and when the posture in which the shorter side of the pair of similar sides is along the height direction is determined as the storage posture, The storage posture as a result is updated to a posture in which the longer side of the pair of similar sides is along the height direction. 7 to 10, the difference between W1 and H1 among the dimensions (H1, W1, L1) of the three sides of the article 50 is smaller than the determination limit value (for example, 2 cm), and has almost the same length ( W1 is slightly larger than H1), even when the storage state determination unit 42 determines that the storage posture is the SC posture in which the side of H1 is along the height direction, It is determined that the side is an SB posture along the height direction, and the determination result is updated.

  The plane position specifying unit 43 is a functional unit that specifies the plane position and the planar view posture of the article upper surface from the position in the basic image of the article existence region M and the inclination with respect to the basic image based on the detection information of the article search processing unit 37. It is. In the present embodiment, the plane position specifying unit 43 specifies the center position M1 of the article presence area M in the container 51 as the plane position of the article upper surface, and sets the rotation angle (tilt) of the article existence area M relative to the container 51 to the article. It is specified as a planar view posture of the upper surface. Specifically, the plane position specifying unit 43 sets the reference point at the corner of the container 51 on the downstream side in the transport direction WX of the container 51 and the article transport conveyor 11 in the width direction WY in the basic image as the origin O, and from the origin O to the container. 51, the X axis in the direction along the in-container transport direction X and the Y axis in the direction along the in-container width direction Y (that is, the in-container coordinate system is set), and the article presence region M from the storage unit 40. Get information about. Then, as shown in FIG. 11, the planar position specifying unit 43 calculates the in-container coordinates (X, Y) of the center position M1 of the article presence region M, and the in-container coordinates (X, Y) of the center position M1. ) Is specified as the planar position of the upper surface of the article. Further, the plane position specifying unit 43 uses a rectangular area corresponding to the article existence area M from a reference angle (0 °) where the rectangular area corresponding to the article existence area M is arranged to be similar to the rectangular area of the container 51. Calculates a rotation angle rotated around the center position M1 of the article presence area M, and specifies the rotation angle as a planar view posture. The origin O of the container 51 in the image coordinate system can be converted into coordinates in the reference coordinate system using a conversion coefficient determined from the installation position of the imaging device 8 and the relationship between the length and the actual distance in the reference image. Thereby, the plane position specifying unit 43 converts the in-container coordinates (X, Y) of the center position M1 in the in-container coordinate system into an image coordinate system, further converts into the reference coordinate system, and performs robot control as recognition result information. To the unit 24. Thus, the recognition result information includes the world coordinates (image coordinates) of the three-dimensional position of the center position M1.

  The check processing unit 45 is a functional unit that checks whether the value of the three-dimensional position and planar view posture of the article upper surface specified by the article recognition device 3 (article position specifying unit 41) is inappropriate. The check processing unit 45 includes a bottom surface check unit 46 and an article wall edge check unit 47. When the check processing unit 45 acquires the information about the three-dimensional position and the planar view posture of the article upper surface in the container 51 from the article position specifying unit 41, the acquired three-dimensional position and the planar view posture value of the article upper surface are inappropriate. If it is normal, information indicating that it is normal is output to the article position specifying unit 41, and if it is not normal, an error is output to the retry control unit 23 of the host controller 2.

  The bottom surface check unit 46 is a functional unit that checks whether the position of the upper surface of the article from the container bottom surface calculated by the storage state determination unit 42 is a position that cannot actually exist. For example, the bottom surface check unit 46 has the article upper surface of the article 50 placed on the container bottom surface of the container in which the position in the height direction Z of the article upper surface specified by the article position specifying unit 41 is embedded in the floor. Check whether it corresponds to the position. Specifically, when the bottom surface check unit 46 acquires information on the three-dimensional position of the top surface of the article from the article position specifying unit 41, the height from the bottom surface of the container calculated by the storage state determination unit 42 is It is determined whether or not it is smaller than the length of the shortest side among the lengths of the three sides included in the 50 dimension information. When the bottom surface check unit 46 determines that the height of the upper surface of the article from the container bottom surface is smaller than the length of the shortest side of the article 50, the reality is that the article 50 is recessed into the container bottom surface. Since there is no result, it is determined that the recognition of the upper surface of the article has failed, and an error is output. In the present embodiment, since the length of the shortest side of the article 50 is H1, the bottom face check unit 46 determines whether the height of the article upper surface from the container bottom calculated by the storage state determination unit 42 is less than H1. Determine. When the bottom surface check unit 46 determines that the height of the upper surface of the article from the container bottom surface calculated by the storage state determination unit 42 is less than H1, an error output is output to the retry control unit 23 of the host controller 2. Do. Note that when the bottom surface check unit 46 determines that the height of the top surface of the article from the container bottom calculated by the storage state determination unit 42 is equal to or higher than H1, the information indicating that the article is normal is indicated by the article position specifying unit 41. Output to.

  The article wall edge check unit 47 corresponds to the article upper surface in the basic image determined by the planar position and planar view posture of the article upper surface specified by the planar position specifying unit 43 and the dimension information of the article 50 stored in the storage unit 40. This is a functional unit that checks whether a rectangular area to be present does not exist at a position where it cannot actually exist. For example, the article wall edge check unit 47 checks whether or not the rectangular area corresponding to the upper surface of the article 50 is in a state of being recessed into the side wall of the container 51. Specifically, the article wall edge check unit 47 is a basic image determined by the planar position and planar view posture of the article upper surface specified by the planar position specifying unit 43 and the dimension information of the article 50 stored in the storage unit 40. It is determined whether or not the rectangular region corresponding to the upper surface of the article is included in the presence region of the container 51 in the basic image. In the present embodiment, as shown in FIG. 11, the article wall edge check unit 47 forms a rectangular area on the upper surface of the article from the center position M1 of the article presence area M, the planar view posture (rotation angle), and the dimension information of the article 50. The in-container coordinates (X, Y) at each of the four corners 50A, 50B, 50C, and 50D are calculated. On the other hand, the article wall check unit 47 acquires the dimension information of the container 51 from the storage unit 40, and the coordinates in the containers of 51A, 51B, 51C, 51D that are the four corners constituting the rectangular area corresponding to the existence area of the container 51 are obtained. Each point (X, Y) coordinate in the system is calculated. A point at one corner of the container 51 (51A in this example) is the origin O. Then, the article wall edge check unit 47 has the container coordinates (X, Y) at the four corner points of the rectangular areas 50A, 50B, 50C, 50D corresponding to the upper surface of the article 51A, 51B, 51C, 51D of the container 51. When it is determined that it is included in the rectangular area generated by connecting the four corner points, check result information indicating that the item is normal is output to the article position specifying unit 41. On the other hand, the bottom surface check unit 46 is a rectangular area generated by connecting the four corner points of the container 51 among the in-container coordinates (X, Y) of the four corner points of the rectangular area corresponding to the upper surface of the article. If it is determined that it is located outside, an error is output to the retry controller 23 of the host controller 2.

4). Processing procedure of article recognition processing The processing procedure of the article recognition processing by the host controller 2, the article recognition device 3, and the picking conveyor 6, that is, the article recognition equipment according to the present embodiment will be described with reference to FIGS. The procedure of the article recognition process described below is executed by hardware or software (program) or both constituting the functional units of the host controller 2, the article recognition apparatus 3, and the picking conveyor 6. When each of the above functional units is configured by a program, arithmetic processing devices (not shown) included in each of the host controller 2, the article recognition device 3, and the picking conveyor 6 configure each functional unit included in each. It may operate as a computer that executes a program, or the arithmetic processing unit of the host controller 2 may operate as a computer that executes a program that constitutes all the functional units of the article recognition facility.

4-1. About the article recognition process In the article recognition process according to the present embodiment, first, the information acquisition unit 44 acquires the container information and the article information output together with the article search command from the host controller 2 (step # 01). And the image acquisition part 31 acquires a captured image from the imaging device 8 based on the said article search instruction | command in step # 01 (step # 02). Next, the container outer shape detection unit 32 detects the outer shape of the container 51 from the captured image acquired in step # 02 (step # 03). If the container outline cannot be detected in step # 03 (step # 04: No), the container outline detector 32 outputs an error report to the host controller 2 and receives a reset instruction from the host controller 2 ( Step # 20). Then, the data acquired in step # 01 and step # 02 and stored in the storage unit 40 is erased accordingly. On the other hand, when the container outer shape is detected in step # 04 (step # 04: Yes), the article search range setting unit 34 executes the article search range setting process (step # 05).

  In the article search range setting process, as shown in FIG. 13, first, the article search range setting unit 34 acquires the dimension information of the three sides of the article 50 from the storage unit 40 (step # 51). Next, the virtual reference plane setting unit 35 calculates the reference height H from the basic image, sets the virtual reference plane S using the dimension information in Step # 51 and the reference height H, and the virtual reference bottom surface. SS is further set (step # 52). Then, based on the dimension information acquired in step # 51 and the virtual reference surface S and the virtual reference lower surface SS set in step # 52, the single height image generation unit 36 performs a base upper step on the virtual reference surface S. The logical height from the bottom of the container on the upper surface of the article is calculated for each of the six assumed storage attitudes of the three assumed storage attitudes of the article and the three assumed storage attitudes of the base stage article on the virtual reference lower surface SS. Then, the single height image generation unit 36 extracts pixels each having the same height information as the logical height, and generates a single height image corresponding to the upper surface of the article in the assumed storage posture. (Step # 53). The single height image generator 36 binarizes each single height image generated in step # 53 (step # 54), and determines whether or not an article search area exists (step #). 55). If the single height image generation unit 36 determines that the article search area exists (step # 56: Yes), the single height image generation unit 36 converts the color image into a color image as a valid single height image (step # 57). The article search range is set (step # 58) and stored in the storage unit 40 (step # 59). Then, when the process of determining whether or not the article search area exists for all six single height images is completed (step # 60: Yes), the single height image generation unit 36 searches for articles. The range setting process ends. When the single height image generation unit 36 has not completed the process of determining the presence or absence of the article search region for all six single height images (step # 60: No), Until the process of determining the presence / absence of the article search region for all the single height images is completed, the process returns to step # 56, and the determination process is executed for the remaining other single height images.

  On the other hand, when the single height image generating unit 36 determines in step # 56 that the article search area does not exist (step # 56: No), the article search area is also used for all other single height images. If it is determined that there is no item search area, that is, if it is determined that there is no article search area in all single height images (step # 61: Yes), it is determined that there is no valid single height image (step # 62). , Step # 06 in FIG. 12: Yes) The article search range setting process is terminated, and the process returns to FIG. 12, and the retry control unit 23 outputs a retry instruction to the picking conveyor 6 (Step # 17). In step # 61, if the single height image generation unit 36 does not determine that there are no article search regions in all six single height images (step # 61: No), for example, the article search region has already been obtained. If there is a single height image determined to be present, the process returns to step # 56 until the determination processing for all six single height images is completed (step # 60). : No). In step # 61, when the single height image generation unit 36 does not determine that there are no article search regions in all of the six single height images (step # 61: No), all the six single height images are generated. When the discrimination process for the height image is completed (step # 60: Yes), the article search range setting process is terminated.

  Returning to the processing of FIG. 12, when the single height image generation unit 36 determines in step # 06 that there is a valid single height image (step # 06: No), the pattern matching processing unit No. 38 performs pattern matching processing on the single height image having the highest height among the effective single height images set in the article search range (step # 07). When the article presence area M is detected in the pattern matching process (step # 08: Yes), the article position specifying unit 41 specifies the three-dimensional position and planar view posture of the article upper surface based on the article existence area M ( Step # 11). On the other hand, when the article presence region M is not detected in the pattern matching process (step # 08: No), the edge detection processing unit 39 performs the edge detection process on the effective single height image used in step # 07. Execute (Step # 09). When the article presence area M is detected in the edge detection process (step # 09: Yes), the article position specifying unit 41 specifies the three-dimensional position and planar view posture of the article upper surface based on the article existence area M ( Step # 11).

  On the other hand, if the article presence region M is not detected in the edge detection process (step # 10: No), and there is another effective single height image determined in step # 06 (step # 16: Yes), step Returning to # 07, the pattern matching processing unit 38 performs pattern matching processing on another valid single height image. In the present embodiment, the pattern matching processing unit 38 executes the pattern matching processing in the order of the single height image having the highest height. Further, when the article presence region M is not detected in the edge detection process (step # 10: No), and there is no other valid single height image determined in step # 06 (step # 16: No), the edge The detection processing unit 39 outputs information to that effect to the host controller 2, and the retry control unit 23 outputs a retry instruction to the picking conveyor 6 (step # 17). If the retry instruction output number is less than 3 (step # 18: No), the retry control unit 23 causes the picking conveyor 6 to execute a retry process (step # 19). If the number of times that the retry instruction is output is 3 times or more (step # 18: Yes), the retry control unit 23 issues a reset instruction (step # 20) and ends the article recognition process. When the article position specifying unit 41 specifies the three-dimensional position and the planar view posture of the article upper surface based on the article existence area M (step # 11), various checks are executed by the check processing unit 45 (step # 11). # 12). If the results of various checks in the check processing unit 45 are normal (step # 13: Yes), the article position specifying unit 41 determines the three-dimensional position and planar view posture of the article upper surface (step # 13). 14) The information on the three-dimensional position and the planar view posture of the article upper surface is output to the host controller 2 (step # 15), and the article recognition process is terminated. If the results of various checks in the check processing unit 45 are not normal (step # 13: No), the check processing unit 45 outputs information to that effect to the host controller 2, and the retry control unit 23 picks up. A retry instruction is output to the conveyor 6 (step # 17).

5. Other Embodiments Finally, other embodiments of the article conveyance facility 100 according to the present invention will be described. Note that the configurations disclosed in the following embodiments can be applied in combination with the configurations disclosed in other embodiments as long as no contradiction arises.

(1) In the above embodiment, using the imaging device 8 having a pair of cameras 8A and 8B such as a stereo camera, the distance information from the camera 8A in each pixel in the image captured by the camera 8A The configuration is such that each pixel generates a captured image having distance information from the camera 8A, calculated based on the parallax of the cameras 8A and 8B. However, the embodiment of the present invention is not limited to this, and a configuration using a distance image sensor that can acquire a captured image and distance information together as the imaging device 8 may be used.

(2) In the above embodiment, the retry control unit 23 is provided in the host controller 2, but may be provided in the article recognition device 3. In that case, the retry control unit 23 may be configured to directly control the picking conveyor 6 or may be configured to indirectly control the picking conveyor 6 via the host controller 2.

(3) In the above-described embodiment, the article recognition apparatus 3 further includes a last article determination unit that determines whether or not the recognition target article 50 is the last one stored in the container 51. The structure provided may be sufficient. In this case, the picking conveyor 6 is arranged so that the host controller 2 replaces the container 51 existing at the picking location 12 with another container 51 after taking out the last one article 50 accommodated in the container 51. It is preferable to have a configuration including a container replacement control unit (not shown) for controlling the operation. If the last article determination unit determines that the article 50 to be recognized is the last one contained in the container 51, it outputs information indicating that it is the last one to the host controller 2. When the container replacement control unit receives the information indicating that it is the last one, the picking process of the last article 50 by the picking robot 5 based on the recognition information result of the last article 50 acquired from the article recognition apparatus 3 is performed. When finished, the transport surface P of the picking conveyor 6 is moved from the upstream side to the downstream side in the transport direction WX. Thereby, the empty container 51 is conveyed to the warehousing conveyor 10 side, and the new container 51 in which the article 50 is accommodated is conveyed to the picking location 12. In the last article determination unit, for example, the host controller 2 obtains the inventory quantity of the articles 50 included in each container 51 acquired from the master container information via the container information acquisition unit 22 and the recognition result information from the article recognition device 3. The last one can be determined by comparing the number of times output to (equivalent to the number of articles 50 taken out by the picking robot).

(4) In the above embodiment, the host controller 2 is configured to automatically output an article search command to the article recognition device 3 when the presence of the container 51 at the picking location 12 is detected by a sensor or the like. However, the embodiment is not limited to this. The host controller 2 may be configured to output an article search command to the article recognition device 3 when an operator inputs a manual article search operation input such as pressing an article search processing start button.

(5) In the above-described embodiment, the virtual reference plane setting unit 35 sets the height at which the most pixels having the same height information are included in the basic image as the reference height H, and the height of the reference height H The plane formed by the pixel group having the depth information is set as the virtual reference plane S. However, the embodiment of the present invention is not limited to this, and may have a configuration in which the reference height H is given in advance.

(6) In the above embodiment, the virtual reference plane setting unit 35 determines the peak having the largest area when the histogram is divided by valleys as the reference height H of the virtual reference plane S, The height of the pixel value having the highest appearance rate, that is, the height including the most pixels having the same height information is set as the reference height H. However, the embodiment of the present invention is not limited to this. For example, the average value of pixel values (height) is calculated from the number of pixels and pixel values (height of each pixel) that make up the mountain with the largest area when the histogram is divided by valleys, and the average value is used as a reference It may be configured to have a height H.

(7) In the above embodiment, the article search range setting unit 34 is provided, a single height image is generated, and an effective single height image among the generated single height images is set as the article search range. The article search processing unit 37 is configured to execute the article search process for the article search range. The storage state determination unit 42 is configured to determine the assumed storage posture associated with the single height image in which the article presence area M is detected as the storage posture. However, the embodiment of the present invention is not limited to this. For example, when the article recognition apparatus 3 is configured not to include the article search range setting unit 34, the article search processing unit 37 does not narrow down to a single height image that is effective as the article search range, but the entire basic image. The structure which performs an article search process with respect to it may be sufficient. In this case, when the article presence area M is detected by the article search processing unit 37, the storage state determination unit 42 determines the article 50 from the rectangular area of the article existence area M corresponding to the upper surface of the article and the dimension information of the article 50. Recognizes which surface is the upper surface, and determines the storage posture. The storage state determination unit 42 further calculates the height of the article upper surface with respect to the reference plane to which the bottom surface of the article 50 belongs, based on the dimension information of the article 50 after determining the storage posture.

2: Host controller (article recognition device)
3: Article recognition device (article recognition device)
6: Picking conveyor (moving part) (article recognition equipment)
37: Article search processing unit 42: Storage state determination unit 43: Plane position specifying unit 44: Information acquisition unit 35: Virtual reference plane setting unit 36: Single height image generation unit 46: Bottom surface check unit 47: Article wall edge check unit 50: Article 51: Container M: Article existence area P1, P2: Template image S: Virtual reference plane SS: Virtual reference bottom surface T1: Single height image

Claims (9)

An object having a rectangular parallelepiped shape housed in a container having an open top surface is a recognition target, and a position of the top surface of the article and a planar view posture of the top surface of the article from a basic image that is a captured image from above the container housing the article. An article recognition device for recognizing
An information acquisition unit that acquires in advance dimensional information about a template image corresponding to an image of some of the six rectangular surfaces of the article and three sides extending from one vertex of the article;
An article search processing unit that performs pattern matching processing and edge detection processing;
From the detection information of the article search processing unit and the dimension information, it is possible to determine a storage posture of which of the rectangular surfaces the upper surface of the article is stored, and a reference to which the bottom surface of the article belongs A storage state determination unit that calculates a height of the upper surface of the article with respect to a surface;
Based on the detection information of the article search processing unit, a plane position specifying unit that specifies a plane position and a planar view posture of the article upper surface from a position in the basic image of the article existence region and an inclination with respect to the basic image; With
The pattern matching process detects, as an article presence area, an area that is included in the basic image and has a degree of matching with the template image that is equal to or greater than a predetermined threshold value.
The edge detection process generates an edge image generated by extracting an edge component in the basic image, and a relationship with a rectangular model generated based on the outer shape of each rectangular surface matches a predetermined condition. Detecting an area as the article presence area,
The article recognition processing unit executes the pattern matching process when an article search command is issued, and executes the edge detection process when the article presence area cannot be detected in the pattern matching process. The storage posture of the article in the container is predetermined,
The article recognition apparatus according to claim 1, wherein the template image is an image corresponding to a pattern of the rectangular surface that is an upper surface in the predetermined storage posture. For each pixel in the basic image, the height information indicating the height from the container bottom surface of the upper surface of the article at a position corresponding to each pixel is acquired, and the height including the most pixels having the same height information is included. A virtual reference plane setting unit that uses a plane formed by the pixel group that is the virtual reference plane;
When it is assumed that there is a base upper article that is an article to which the bottom belongs to the virtual reference plane, for each of three different assumed storage postures assumed as the storage posture, the surface of the upper surface in the assumed storage posture A single height image generating unit that generates a single height image by extracting pixels having the same height information as the height from the bottom of the container from the basic image, and
In the pattern matching process, the article search processing unit detects, as the article presence area, an area included in the single height image and having a degree of coincidence with the template image equal to or greater than a predetermined threshold. In the edge detection process, an edge image generated by extracting an edge component included in the single height image is generated, and a region whose relationship with the rectangular model matches a predetermined condition is generated. Detect as an article presence area,
The storage state determination unit determines, as the storage posture, the assumed storage posture corresponding to the single height image in which the article presence area is detected by the article search processing unit in the single height image. The article recognition apparatus according to claim 1, wherein the virtual reference plane is determined as the reference plane. In the container, an article layer formed by a plurality of articles having the same shape arranged so that the article upper surface belongs to the same height is accommodated in a state of being stacked in the height direction,
The virtual reference plane setting unit further sets, as a virtual reference lower surface, a surface to which a bottom surface of a base stage article that is an article whose article upper surface belongs to the virtual reference plane,
The single height image generation unit is an upper surface in each of the three new assumed storage postures assumed as the storage posture of the base stage article on the virtual reference lower surface in the new assumed storage posture. Extracting from the basic image pixels having height information equal to the height of the surface from the bottom of the container to generate three new single height images;
When the article search processing unit detects the article presence region in any of the three new single height images, the storage state determination unit detects the article presence region. The article recognition apparatus according to claim 3, wherein the hypothetical storage posture corresponding to the one-height image is determined as the storage posture, and the virtual reference lower surface is determined as the reference surface.   The article search processing unit is configured to perform the pattern matching process in order from a single height image having a higher height among single height images corresponding to the assumed storage posture generated by the single height image generation unit. The article recognition apparatus according to claim 3 or 4, wherein the edge detection process is executed. The storage state determination unit calculates the height from the container bottom surface of the article upper surface from the height from the container bottom surface of the virtual reference plane or the virtual reference lower surface, and the height of the article upper surface with respect to the reference plane,
A bottom surface for determining whether the height from the container bottom surface of the article upper surface calculated by the storage state determination unit is smaller than the shortest side length among the three side lengths included in the dimension information. A check section,
The article recognition according to any one of claims 3 to 5, wherein when the bottom face check unit determines that the height of the article top face is smaller than the length of the shortest side of the article, an error output is performed. apparatus.   The storage state determination unit determines, based on the dimension information, that the three sides of the article include a pair of similar sides in which a difference in length between the sides is smaller than a predetermined determination limit value. In this case, when the posture in which the shorter side of the pair of similar sides is along the height direction is determined as the storage posture, the storage posture as a result of the determination is determined as the storage posture. The article recognition apparatus according to any one of claims 1 to 6, wherein a longer one of the similar sides is updated to a posture along the height direction. A rectangular region corresponding to the upper surface of the article in the basic image determined by the planar position and planar view posture of the upper surface of the article specified by the planar position specifying unit and the dimensional information is an existence area of the container in the basic image. Further comprising an article wall edge check unit for determining whether or not it is included in the interior of
The article recognition apparatus according to any one of claims 1 to 7, wherein when the article wall check unit determines that the rectangular area is not included in the container existing area, an error output is performed. The article recognition apparatus according to any one of claims 1 to 8,
A moving unit for moving the container in a horizontal direction;
A retry control unit that controls the moving unit to horizontally move the container by a preset amount from the imaging position of the basic image when the article presence processing unit cannot detect the article presence region; An article recognition facility comprising:

Images