JP7155896B2 - Image recognition system - Google Patents

Description

The present invention relates to an image recognition system including an imaging device that acquires image information and an image recognition unit that performs image recognition processing on the image information.

The background art will be described below. In the following description, the symbols or names in parentheses are the symbols or names in the prior art documents. A conventional example of such an image recognition system is described in Japanese Patent Application Laid-Open No. 2014-091608 (Patent Document 1). The image recognition system of Patent Literature 1 is provided in picking equipment. In this picking facility, an operator takes out an article from the supply case 12 accommodated in the fluidized shelf 40 and stores the article in the article collection case 11 placed on the article collection conveyor 45. A movement operation is performed to move an article from a position (supply case 12) to a destination position (collection case 11).

The image recognition system of Patent Document 1 described above includes a source imaging device (supply-side camera 30a) that captures one source position and a destination imaging device that captures one destination position (collection and a side camera 30b). Then, the image recognition unit compares the image of the target article before and after it is taken out, which are included in the image information acquired over time by the imaging device for the movement source, and determines the image of the target article that has been taken out. In addition to determining the quantity, the stored target article is compared with the image before storing the target article and the image after storing the target article included in the image information acquired over time by the imaging device for the destination. Determine the quantity of Also, the image recognition unit determines whether or not these determination results are correct in view of the picking order information.

JP 2014-091608 A

In the image recognition system described above, an increase or decrease in the number of items is determined at the movement source position and the movement destination position, and whether or not it is correct in view of the picking order information is determined. When an object is moved from a movement source position to a specific movement destination position, or when an object object is moved from a specific movement destination position to a movement destination position other than the movement destination position, the object It does not determine whether the movement of the goods is correct. Therefore, for example, even if the increase in the number of articles at a specific destination position is correct in view of the picking order information, it is necessary to determine whether or not the articles have been picked up from the correct source position. was not done.

Therefore, it is desired to realize an image recognition system capable of determining whether or not the target article has been correctly moved from the movement source position to the movement destination position.

In view of the above, the image recognition system is characterized by an imaging device that acquires image information, an image recognition unit that performs image recognition processing on the image information, and a worker who moves an article from a movement source position to a movement destination position. a control device having a reference acquisition unit that acquires reference information for judging whether the moving work of the target article is correct or not; wherein the imaging device captures an image of a work area including a source area having a plurality of source positions and a destination area having a plurality of destination positions; The position of the target article and the position of the hand of the worker who moves the target article, which are included in the acquired image information, are image-recognized to determine the movement source position and the movement destination position of the target article. and determines whether or not the determination result is correct in view of the determination reference information. The image information captured by the imaging device is stored in the storage device in a state in which a symbol image indicating that correct extraction has been performed is attached to a position corresponding to the position of the operator's hand in the image information, and When it is determined that the determination result of the movement source position is incorrect in the movement work determination, the image information imaged by the imaging device is incorrectly extracted at a position corresponding to the position of the worker's hand . The image information is stored in the storage device with a symbol image indicating that the image is attached, and if it is determined that the determination result of the destination position is correct in the moving work determination, the image captured by the imaging device The image information is stored in the storage device with a symbol image indicating that the correct storage is performed at a position corresponding to the position of the worker's hand in the information, and the destination position is determined in the movement work determination. If the determination result is determined to be incorrect, a symbol image indicating that incorrect accommodation has been performed is attached to the position corresponding to the position of the worker's hand in the image information captured by the imaging device. The image information is stored in the storage device in a state.

According to this characteristic configuration, the image recognition unit performs image recognition of at least one of the position of the target article and the position of the hand of the worker who moves the target article, which is included in the image information acquired over time. Both the source position and the destination position of the moving target item can be recognized. Then, it is possible to determine whether or not both the movement source position and the movement destination position recognized in this way are correct in view of the judgment reference information. That is, it is possible to determine whether or not the combination of the movement source position and the movement destination position for each target article is correct. Therefore, according to this characteristic configuration, it is possible to realize an image recognition system capable of determining whether or not the target article has been correctly moved from the movement source position to the movement destination position.

Top view of picking equipment Side view of picking equipment Control block diagram Flowchart of lighting/extinguishing control Judgment control flow chart The figure which shows the display apparatus which displays an example of a moving image when the taking out of the target article from a target storage case is correct. The figure which shows the display apparatus which displays an example of the moving image in case a worker is moving a target article. FIG. 11 is a diagram showing a display device displaying an example of a moving image when the target article is correctly accommodated in the article collection case; The figure which shows the display apparatus which displays an example of a moving image when taking out of the target article from a target storage case is not correct. The figure which shows the display apparatus which displays an example of the moving image when accommodation of the target article to an article collection case is incorrect. Side view of picking equipment in another embodiment

1. Embodiment An embodiment of a picking facility equipped with an image recognition system will be described based on the drawings.
As shown in FIGS. 1 and 2, the picking equipment includes a fluidized shelf 1 that stores supply cases C1, a first transport device 2 that transports product collection cases C2, and an empty supply case C1. and a second conveying device 3 . A picking work location P is formed between the fluidized shelf 1 and the first transport device 2 . A direction perpendicular to the first direction X when viewed in the up-down direction Z is defined as a second direction Y in the following description. Also, one side in the first direction X will be described as a first direction first side X1, and the opposite side will be described as a first direction second side X2. Also, one side in the second direction Y will be described as the second direction first side Y1, and the opposite side will be described as the second direction second side Y2.

As shown in FIG. 1, a plurality of picking work locations P are formed so as to be aligned in the first direction X. As shown in FIG. At the picking work location P, the worker M picks up the following based on a plurality of pieces of article processing information (hereinafter referred to as order information OD (see FIG. 3)) generated in a form specifying the type of article W and the quantity of article W. Carry out picking work. At the picking work location P, as a picking work (moving work), the work of taking out the articles W of the type and quantity designated by the order information OD from the supply case C1 and storing them in the collection case C2 is performed. Such picking work is performed once or multiple times for one piece of order information OD.

As shown in FIG. 1, the picking facility is divided into a plurality of work areas A corresponding to a plurality of picking work locations P. As shown in FIG. In one work area A, one picking work place P is set, and part of the fluidized shelf 1, part of the first transport device 2, and part of the second transport device 3 are included. .

As shown in FIGS. 1 and 2, the supply case C1 and the product collection case C2 are each formed in a box shape that opens upward. In this embodiment, the shape of the supply case C1 and the product collection case C2 when viewed in the up-down direction Z is formed in a rectangular shape. As shown in FIG. 1, the item collection case C2 is conveyed by the first conveying device 2 in a posture in which the longitudinal direction of the item collection case C2 is along the first direction X. As shown in FIG. The opening of the product collection case C2 is referred to as a case opening 5. As shown in FIG.

The fluidized rack 1 is installed at a position adjacent to the picking work location P on the first side Y1 in the second direction. The flow shelf boards 6 are arranged in a plurality of stages in the vertical direction Z as shown in FIG. 2 and arranged in a plurality of rows in the first direction X as shown in FIG. As shown in FIG. 2 , a frontage 7 is formed above the tip portion (the end portion on the second side Y2 in the second direction) of each flow shelf plate 6 . In this embodiment, the fluidized shelves 1 having three fluidized shelves 6 arranged in the vertical direction Z are arranged in four rows in the first direction X for each work area A. As shown in FIG.

Each of the floating shelf boards 6 is arranged in a posture along the second direction Y when viewed in the up-down direction Z. As shown in FIG. A plurality of flow shelf plates 6 are arranged in the first direction X and parallel to each other. Each of the flow shelf boards 6 is arranged in a posture inclined downward toward the second side Y2 in the second direction. The supply case C1 placed on the flow shelf plate 6 moves along the second direction Y toward the second side Y2 in the second direction due to its own weight.

The fluidized shelf 1 is configured to be able to accommodate a plurality of supply cases C1 in a row in the second direction Y on the fluidized shelf plate 6 . In this embodiment, one type of article W is accommodated in one supply case C1. Of the one or more supply cases C1 on the flow shelf board 6, the supply case C1 closest to the second side Y2 in the second direction is designated as the target supply case CT1, and the article W is taken out from the target supply case CT1 in the picking operation. be The empty supply case C<b>1 is taken out from the fluidized shelf 1 by the operator M and placed on the second transport device 3 .

As shown in FIGS. 1 and 2, a first indicator 11 is provided at the tip of each flow shelf 6 (the end on the second side Y2 in the second direction). As shown in FIG. 1, the first indicator 11 includes a first display section 12 that displays the quantity of articles W to be taken out, a first lighting section 13 that can be turned on and off, and a first switch section 14 that can be pushed. and have In this embodiment, the first lighting section 13 is configured to be push-operable, and the first lighting section 13 also functions as the first switch section 14 . Further, in the present embodiment, the first lighting section 13 is configured to light in a color different from that of the first lighting section 13 installed in another work area A adjacent in the first direction X. For example, the colors that the first lighting unit 13 lights are alternately set to different colors (for example, red and green) for each picking work location P arranged in the first direction X.

As shown in FIGS. 1 and 2, the first transport device 2 is installed at a position adjacent to the picking work location P on the second side Y2 in the second direction. The first conveying device 2 conveys the plurality of supply cases C1 along the first direction X toward the first side X1 in the first direction while being aligned in the first direction X in a row. The item collection case C2 transported to the work area A does not contain the item W, or another work area A located on the upstream side of the first transport device 2 in the transport direction (second side X2 in the first direction) One or a plurality of articles W are stored by the picking operation performed in . The first conveying device 2 has a second switch section 21 . The second switch section 21 is installed in each work area A. As shown in FIG. In this embodiment, the second switch section 21 is composed of a tape switch.

As shown in FIG. 1, the first conveying device 2 has a second display 16 at a position corresponding to the article collection case C2 to be conveyed. The second display 16 moves along the first direction X toward the first side X1 in the first direction together with the article collection case C2 conveyed by the first conveying device 2 . In the present embodiment, the second display 16 is positioned within the width of the product collection case C2 in the first direction X, and is positioned on the second side Y2 in the second direction with respect to the product collection case C2, It is fixed to the collection case C2.

The second display 16 includes a second lighting section 17 that can be turned on and off, a second display section 18 that displays order ID information that is identification information for identifying the order information OD, and a control section 19 (see FIG. 3). and a communication unit 20 (see FIG. 3). In this embodiment, the second lighting section 17 is configured to be able to light a color corresponding to the work area A in which the second indicator 16 is present. For example, the second lighting unit 17 is controlled by the control unit 19 so as to alternately light different colors (for example, red and green) for each work area A arranged in the first direction X. Also, the second display 16 is configured to be able to wirelessly communicate with a control device H, which will be described later, by means of the communication unit 20 . The first lighting section 13 and the second lighting section 17 existing in the same work area A are designed to be lit in the same color.

The second transport device 3 is installed at a position adjacent to the first transport device 2 on the second side Y2 in the second direction. The second conveying device 3 conveys the supply case C1 placed by the worker M along the first direction X toward the first side X1 in the first direction.

In this embodiment, the flow shelf 1 corresponds to the storage shelf, the supply case C1 corresponds to the storage container stored in the storage shelf, and the target supply case CT1 corresponds to the movement source position. Further, in the present embodiment, the first conveying device 2 corresponds to the conveying device, and the product collection case C2 corresponds to the conveying container conveyed by the conveying device and to the destination position. The worker M performs a moving work (picking work) to move the articles W from the movement source position (supply case C1) to the movement destination position (item collection case C2). Also, the first lighting section 13 corresponds to the movement source display section, and the second lighting section 17 corresponds to the movement destination display section. The article W to be moved is hereinafter referred to as a target article WT.

As shown in FIGS. 1 and 2, the picking equipment is equipped with an imaging device 23 for acquiring image information. The imaging device 23 is installed to capture an image of a work area A including a movement source area A1 having a plurality of target supply cases CT1 and a movement destination area A2 having a plurality of collection cases C2. The first indicator 11 and the second indicator 16 are installed in the work area A. As shown in FIG. In this embodiment, the first display 11 is installed in the movement source area A1, and the second display 16 is installed in the movement destination area A2.

In the present embodiment, in addition to the movement source area A1 and the movement destination area A2, the work area A includes the target article WT when moving the target article WT from the target supply case CT1 to the collection case C2. A moving area A3 is also included, and the imaging device 23 also captures an image of this moving area A3. In the present embodiment, one imaging device 23 is installed for one work area A, and one imaging device 23 captures the moving source area A1, the moving destination area A2, and the moving area A2 of the working area A. A moving image of the moving area A3 is captured. The imaging device 23 is installed above the fluidized shelf 1, the first transport device 2, and the worker M, and on the first side X1 in the first direction with respect to the work area A to be imaged. is imaged from the upper side and obliquely downward from the first side X1 in the first direction. The image capturing device 23 transmits captured image information to the control device H, which will be described later.

In the movement source area A1, there are a plurality of target supply cases CT1, a frontage 7 for taking out the target articles WT from the plurality of target supply cases CT1, and a plurality of first indicators 11. In addition, there are a plurality of product collection cases C2 and a plurality of second indicators 16 in the destination area A2. In addition, one worker M basically exists in the movement area A3. The imaging device 23 images the plurality of target supply cases CT1, the plurality of frontages 7, the plurality of first indicators 11, the plurality of product collection cases C2, the plurality of second indicators 16, and the worker M. In the picking work, the worker M holds the target article WT in his/her hand and moves it. In this embodiment, the imaging device 23 also images the hand of the worker M who is performing the picking work at the picking work location P. FIG.

〔Control device〕
As shown in FIG. 3, the picking facility is equipped with a control device H that controls the fluidized shelf 1 and the first transfer device 2 . The control device H is composed of, for example, a computer having a CPU and a storage device. The control device H is connected to a management device K that manages the order information OD. The management device K manages a database that manages order information OD created in association with the sorting destination in a form that defines the types and quantities of articles W to be picked up based on orders from the sorting destination, and each It is configured with a scheduler or the like that manages in what timing or order the order information OD is to be processed. The order information OD includes order ID information for identifying the order information OD, sorting destination information indicating the sorting destination, type information indicating the type of the article W to be shipped to the sorting destination, and information for each type of the article W. Quantity information indicating quantity and . A sorting destination is set for each collection case C<b>2 conveyed to the first conveying device 2 . The control device H sets a target supply case CT1 from which the target article WT is taken out according to the type information included in the order information OD, and sets the target supply case CT1 from which the target article WT is put in according to the sorting destination information included in the order information OD. Set product case C2.

The control device H executes first lighting control, second lighting control, first extinguishing control, and second extinguishing control based on the order information OD to and controls the second display 16 provided on the first conveying device 2 .

In the first lighting control, based on the order information OD from the management device K, the first lighting of the first indicator 11 installed on the flow shelf board 6 corresponding to the type of article W indicated in the order information OD is performed. The first display unit 11 is controlled so that the unit 13 is turned on and the number of the target articles WT to be taken out from the target supply case CT1 is displayed numerically on the first display unit 12 of the first display unit 11 . In the second lighting control, based on the order information OD from the management device K, the second lighting section 17 of the second display 16 corresponding to the product collection case C2 set for the order information OD is lit. , controls the second display 16 so as to display the order ID information on the second display 16 of the second display 16 .

In the first extinguishing control, when the first switch portion 14 is pressed, the first lighting portion 13 provided in the same first display 11 as the first switch portion 14 is extinguished and the first light extinguishing control is performed. The first display 11 is controlled so as to stop the display on the display unit 12 . In the second extinguishing control, the second lighting section 17 of the second indicator 16 located in the same work area A as the second switch section 21 is extinguished and the second lighting section 17 is extinguished as the second switch section 21 is pushed. The second display 16 is controlled so as to stop the display on the display section 18 .

As shown in the flowchart of the lighting/extinguishing control shown in FIG. 4, the control device H executes the first lighting control and the second lighting control based on the order information OD (S1). Then, when the worker M who has finished taking out the target article WT from the target supply case CT1 presses the first lighting section 13 (the first switch section 14), the first light-off control is executed (S2, S3), and when the second switch section 21 is pushed by the operator M who has finished the work of storing the target article WT in the article collection case C2, the second extinguishing control is executed (S4, S5).

[Image recognition system]
As shown in FIG. 3, the picking facility is equipped with an image recognition system S. As shown in FIG. Next, the image recognition system S will be explained. The image recognition system S includes an imaging device 23 that acquires image information, an image recognition unit 24 that performs image recognition processing on the image information, and reference information for judging whether the picking work of the target article WT is correct or not. and a reference acquisition unit 25 that acquires the The image recognition system S also includes a first lighting unit 13 that indicates the source position of the target article WT and a second lighting unit 17 that indicates the destination position of the target article WT. In addition, the image recognition system S further includes a display device 26 for displaying imaging information. The image recognition section 24 and the reference acquisition section 25 are provided in the control device H in the form of programs.

The control device H (image recognition unit 24) performs image recognition of the display of the first lighting unit 13 and the display of the second lighting unit 17 included in the image information, and identifies the target supply case CT1 and the product collection case C2 of the target article WT. judge. In this embodiment, the display of the first lighting unit 13 in the lighting state included in the image information is image-recognized, and the target supply case CT1 corresponding to the first lighting unit 13 in the lighting state is identified as the target article WT to be taken out. It is determined as supply case CT1. Further, the display of the second lighting unit 17 in the lighting state included in the image information is image-recognized, and the product collection case C2 corresponding to the second lighting unit 17 in the lighting state is selected as the product collection case housing the target article WT. Determined as C2.

The control device H (reference acquisition unit 25) determines the target supply case CT1 corresponding to the first lighting unit 13 and the product collection case C2 corresponding to the second lighting unit 17, which are shown in the image recognition result of the image recognition unit 24, Acquired as criteria information. The control device H (image recognition unit 24) performs image recognition of the position of the hand of the worker M who moves the target article WT, which is included in the image information acquired over time, and becomes the movement source of the target article WT. The target supply case CT1 and the product collection case C2 that is the destination are determined. Then, it is determined whether or not the determination result is correct in view of the determination criterion information.

More specifically, the control device H performs image recognition of the position of the hand of the worker M based on the imaging information, and in a state where the first lighting unit 13 is lit, the object corresponding to the first lighting unit 13 is detected. When it is determined that the hand of the worker M has moved so as to take out the article W from the supply case CT1, it is determined that the target article WT has been taken out from the target supply case CT1 to be taken out, and correct removal is determined. On the other hand, the control device H performs image recognition of the position of the hand of the worker M based on the imaging information, and in a state where the first lighting unit 13 is lit, the object corresponding to the first lighting unit 13 is detected. When it is determined that the hand of the worker M has moved to take out the article W from the target supply case CT1 different from the supply case CT1, the target article WT is taken out from the target supply case CT1 other than the target supply case CT1 to be taken out. Therefore, it is determined that the extraction is not correct. Further, when the control device H cannot determine whether or not the article W is correctly taken out while the first lighting unit 13 is lit, the control device H determines that the take-out is unknown.

Further, the control device H performs image recognition of the position of the hand of the worker M based on the imaging information, and in a state where the second lighting unit 17 is lit, the product collection case C2 corresponding to the second lighting unit 17 is displayed. If it is determined that the hand of the worker M has moved so as to store the article W in the case C2, it is determined that the target article WT has been accommodated in the article collection case C2 to be accommodated, and that it is correctly accommodated. On the other hand, the control device H performs image recognition of the position of the hand of the worker M based on the imaging information, and in a state where the second lighting section 17 is lit, the group corresponding to the second lighting section 17 is displayed. When it is determined that the hand of the worker M has moved to accommodate the article W in the article collection case C2 different from the article case C2, the target article WT is placed in the article collection case C2 other than the article collection case C2 to be accommodated. It is judged as incorrect accommodation as being accommodated. Further, when the control device H cannot determine whether or not the storage of the article W is correct while the second lighting unit 17 is lit, the control device H determines that the storage is unknown.

Also, the control device H (reference acquisition unit 25) acquires type information indicating the type of the target article WT. In this embodiment, the control device H acquires the type information from the order information OD. The control device H (image recognition unit 24) recognizes the type of the target article WT included in the image information, determines the type of the target article WT, and determines whether the determination result is correct in view of the type information. do. In this embodiment, the control device H moves the operator M's hand so that the article W is stored in the article collection case C2 corresponding to the second lighting unit 17 while the second lighting unit 17 is lit. When it is determined that the object has moved, the type of the target article WT stored in the item collection case C2 is determined by image recognition. When the determined type and the type indicated by the type information match, the type is determined to be correct, and when the determined type and the type indicated by the type information do not match, the type is determined to be incorrect. Further, when the type of the target article WT housed in the article collection case C2 cannot be determined by image recognition, the type is determined to be unknown.

Furthermore, the control device H (reference acquisition unit 25) acquires quantity information indicating the quantity of target articles WT to be moved to the same product collection case C2 by the movement work. In this embodiment, the quantity information is acquired from the order information OD. The control device H (image recognition unit 24) performs image recognition of the number of target articles WT moved to the same collection case C2, which is included in the image information, determines the number of target articles WT, and displays the determination result. is correct in view of the quantity information. In this embodiment, the control device H moves the operator M's hand so that the article W is stored in the article collection case C2 corresponding to the second lighting unit 17 while the second lighting unit 17 is lit. When it is determined that the object has moved, the number of target articles WT stored in the item collection case C2 is determined by image recognition. When the determined quantity and the quantity indicated by the quantity information match, the quantity is determined to be correct, and when the determined quantity and the quantity indicated by the quantity information do not match, the quantity is determined to be incorrect. Also, if the quantity of the target articles WT housed in the product collection case C2 cannot be determined by image recognition, the quantity is determined to be unknown.

A specific example will be described based on the flowchart shown in FIG. 5 and the diagrams showing the display of the display device 26 in FIGS. The control device H can display imaging information captured by the imaging device 23 on the display device 26 in real time, and designates the imaging time by using the third display area D3 and the fourth display area D4 of the display device 26. By doing so, it is possible to display the imaging information captured by the imaging device 23 at the specified time on the display device 26 .

When the control device H recognizes the lighting of the first lighting unit 13 based on the imaging information, the control device H determines the target supply case CT1 from which the target article WT is taken out (S11, S12). At this time, as shown in FIG. 6, a first symbol B1 indicating that the lit first lighting unit 13 has been recognized, and a second symbol B2 indicating the frontage 7 corresponding to the first lighting unit 13 that has been recognized to be lit. , the image information captured by the imaging device 23 is stored. Further, information indicating that the target supply case CT1 from which the target article WT is taken out is stored in association with the imaging information. When the state in which the lighting of the first lighting unit 13 is recognized is displayed on the display device 26, as shown in FIG. Then, the first symbol B1 and the second symbol B2 are displayed, and information indicating that the target supply case CT1 from which the target article WT is to be taken out is displayed in the second display area D2 by a character string.

When the control device H recognizes the hand movement of the worker M in the movement source region A1 in the state where the first lighting unit 13 is lit (S13: Yes), it is determined that the worker M should take it out. If it is determined that the target article WT has been removed from the target supply case CT1 (S14: Yes), it is determined that the removal is correct (S15). At this time, as shown in FIG. 6, the imaging device 23 is attached with a third symbol B3 indicating the position of the hand of the worker M and a fourth symbol B4 indicating that correct extraction has been performed. It stores the information of the captured image. Also, information indicating that correct removal has been determined is stored in association with this imaging information. When the display device 26 is to display the state of correct removal, as shown in FIG. B3 and the fourth symbol B4 are displayed, and information indicating that the correct removal has been determined is displayed in the second display area D2 as a character string. Note that, in the illustrated example, the fourth symbol B4 indicating that the correct extraction has been performed is a symbol image including the character string "OK".

When the control device H recognizes the hand movement of the worker M in the movement source region A1 in the state where the first lighting unit 13 is lit (S13: Yes), it is determined that the worker M should take it out. If it is determined that the target article WT has been taken out from the target supply case CT1 different from the target supply case CT1 (S14: No), it is determined that the target article WT has been taken out incorrectly (S16). At this time, as shown in FIG. 9, the imaging device 23 is attached with a third symbol B3 indicating the position of the hand of the worker M and an eighth symbol B8 indicating that an incorrect extraction has been performed. stores image information captured by . In addition, in a state associated with this image pickup information, information indicating that an incorrect take-out has been determined is stored. If the display device 26 is to display the state of improper removal, as shown in FIG. 9, in the first display area D1, the third The symbol B3 and the eighth symbol B8 are displayed, and information indicating that the incorrect removal is determined is displayed in a character string in the second display area D2. Note that, in the illustrated example, the eighth symbol B8 indicating that an incorrect extraction has been performed is a symbol image including the character string "NG".

When the control device H recognizes the lighting of the second lighting unit 17 based on the imaging information, it determines the product collection case C2 containing the target product WT (S18, S19). At this time, as shown in FIG. 8, a fifth symbol B5 indicating that the lit second lighting unit 17 has been recognized, and the case opening 5 of the product collection case C2 corresponding to the second lighting unit 17 that has been recognized to be lit. The image information imaged by the imaging device 23 is stored with the sixth symbol B6 shown. Further, information indicating that the product collection case C2 containing the target product WT has been determined is stored in association with the imaging information. When displaying the state in which the lighting of the second lighting unit 17 is recognized on the display device 26, as shown in FIG. Then, the fifth symbol B5 and the sixth symbol B6 are displayed, and information indicating that the collection case C2 containing the target article WT has been determined is displayed in a character string in the second display area D2.

When the control device H recognizes the movement of the hand of the worker M in the destination area A2 while the second lighting unit 17 is lit (S20: Yes), it determines that the worker M should be accommodated. When it is determined that the target article WT is stored in the product collection case C2 (S21: Yes), it is determined that the target article WT is correctly taken out (S22), and the type and quantity of the target article WT stored in the product collection case C2 are determined. judge. At this time, as shown in FIG. 8, the imaging device 23 is attached with a third symbol B3 indicating the position of the hand of the worker M and a seventh symbol B7 indicating that the correct accommodation has been performed. It stores the information of the captured image. In addition, in a state associated with this imaging information, information indicating that correct storage has been determined and information indicating whether the type and number of target articles WT are correct or unknown are stored. When the display device 26 is caused to display the state of correct accommodation, as shown in FIG. B3 and the seventh symbol B7 are displayed, and at the same time, information indicating that correct removal has been determined and information indicating whether the type and number of the target article WT are correct or unknown are displayed in the second display area D2. , to display as a string. Note that, in the illustrated example, the seventh symbol B7 indicating that the correct extraction has been performed is a symbol image including a character string of "OK".

When the control device H recognizes the movement of the hand of the worker M in the destination area A2 while the second lighting unit 17 is lit (S20: Yes), it determines that the worker M should be accommodated. If it is determined that the target article WT is stored in a product collection case C2 different from the product collection case C2 (S21: No), it is determined to be incorrect storage (S24). At this time, as shown in FIG. 10, the imaging device 23 is attached with a third symbol B3 indicating the position of the hand of the worker M and a ninth symbol B9 indicating that an incorrect accommodation has been performed. stores image information captured by . In addition, in a state associated with this imaging information, information indicating that incorrect containment has been determined is stored. When the display device 26 displays the state determined to be incorrect accommodation, as shown in FIG. The symbol B3 and the ninth symbol B9 are displayed, and information indicating that the incorrect containment is determined is displayed in the second display area D2 as a character string. Note that, in the illustrated example, the ninth symbol B9 indicating that an incorrect extraction has been performed is a symbol image including the character string "NG".

Note that the control device H does not recognize the hand movement of the worker M in both the movement source area A1 and the movement destination area A2, but recognizes the hand movement of the worker M in the movement area A3. stores image information captured by the imaging device 23 with a third symbol B3 indicating the position of the hand of the worker M attached thereto, as shown in FIG. In the illustrated example, the control device H recognizes the lighting of the second lighting unit 17 based on the imaging information. Therefore, the control device H displays a fifth symbol B5 indicating that the lit second lighting unit 17 has been recognized, and a sixth symbol B5 indicating the case opening 5 of the product collection case C2 corresponding to the second lighting unit 17 that has been recognized to be lit. The image information captured by the imaging device 23 is stored with the symbol B6 added.

2. Other Embodiments Next, other embodiments of the image recognition system will be described.

(1) In the above embodiment, the reference acquisition unit 25 acquires the information indicating the movement source position and the movement destination position determined by the image recognition unit 24 as the determination reference information. not. For example, the reference acquisition unit 25 may acquire instruction information indicating an instruction to the worker M to move the target article WT as the determination reference information. In this case, for example, the order information OD acquired from the management device K may be used as the instruction information (criteria information). In this case, the reference acquisition unit 25 acquires order information OD (instruction information) from the management device K, as shown in FIG. Then, the reference acquisition unit 25 acquires the type information included in the order information OD as movement source information indicating the movement source position of the target article WT, and the sorting destination information included in the order information OD as the movement source position of the target article WT. Acquired as destination information indicating the destination position. More specifically, based on the type information included in the order information OD, the reference acquisition unit 25 can set the target supply case CT1 or the like containing the article W of the type as the movement source position. Further, based on the sorting destination information included in the order information OD, the reference acquisition unit 25 can set the pick-up case C2 or the like corresponding to the sorting destination as the movement destination position.

Further, when a work instruction indicating the content of the movement work is given to the worker M by voice, the work instruction may be used as the instruction information (criterion information). In this case, the picking equipment includes a microphone that collects the voice of the work instruction, and a voice recognition unit that recognizes the voice collected by the microphone and acquires information indicating the movement source position and the movement destination position of the target article WT. , should be provided. Then, the reference acquisition unit 25 preferably acquires information indicating the movement source position and the movement destination position acquired by the voice recognition unit as the determination reference information.

(2) In the above embodiment, the first lighting unit 13 is used as the movement source display unit, and the second lighting unit 17 is used as the movement destination display unit. However, the configuration is not limited to this. For example, the movement source display unit and the movement destination display unit may be display units that display the movement source position or the movement destination position by projecting an image by projection mapping onto the projection surface, and the projection surface is irradiated with laser light. By doing so, the display unit may display the movement source position or the movement destination position. In these cases, the projection plane is the target supply case CT1, which is the movement source position, the side of the storage shelf on which the target supply case CT1 is placed, or the like, facing the worker side, or the movement destination position, which is the product collection case C2. or the surface facing the operator side of the first conveying device 2 that conveys the product collection case C2.

(3) In the above embodiment, the configuration in which one imaging device 23 is provided for one work area A has been described as an example, but the present invention is not limited to this. For example, one work area A may be provided with a plurality of imaging devices 23 . In this case, for example, as shown in FIG. 11, a plurality of imaging devices 23 may be installed in one work area A so that the imaging directions are different from each other. In this case, the installation locations of the plurality of imaging devices 23 are arranged such that at least one imaging device 23 capable of capturing an image without being in the shadow of the worker M exists for each of the plurality of locations that need to be imaged by the imaging device 23 . is preferably set as the imaging direction.

(4) In the above embodiment, the imaging range of one imaging device 23 is set to correspond to one work area A, but the configuration is not limited to this. For example, the imaging device 23 may be installed so that a plurality of work areas A are included in the imaging range of one imaging device 23 . In this case, for example, one imaging device 23 may be installed for a plurality of work areas A. FIG. Alternatively, a plurality of work areas A adjacent to the imaging range of one imaging device 23 are included, and the imaging ranges of the adjacently arranged imaging devices 23 are installed so as to have areas that overlap each other. It may be configured as According to these configurations, even if the worker M leaves his own work area A and performs work in another work area A, the image recognition is performed following the movement of the worker M. It is possible to do

(5) In the above embodiment, the image recognition unit 24 determines the movement source position, the movement destination position, the type, and the quantity of the target article WT, and determines whether or not these determination results are correct. Illustrated, but not limited to. The image recognition unit 24 may be configured to determine at least the movement source position and the movement destination position, and to determine whether or not these determination results are correct. For example, the image recognition unit 24 may be configured not to determine one or both of the type and quantity of the target article WT. Further, the image recognition unit 24 may determine items other than the above items in addition to or instead of any of the above items. For example, the image recognition unit 24 may be configured to determine whether or not the target article WT is damaged.

(6) In the above embodiment, the second display 16 is fixed to the product collection case C2, so that the second display 16 moves integrally with the product collection case C2. However, the configuration for moving the second display 16 integrally with the item collection case C2 may be changed as appropriate. For example, by supporting the second display 16 on the first transport device 2, or by providing a transport unit separate from the transport unit that transports the product collection case C2, the second display 16 and the product collection case C2 It is good also as a structure moved integrally.

(7) In the above embodiment, the control device H stores the imaging information captured by the imaging device 23 and the information determined by the image recognition unit 24, and later stores the imaging information captured by the imaging device 23 and the image recognition. Although the configuration in which the information determined by the unit 24 can be confirmed has been described as an example, the present invention is not limited to this. For example, instead of storing imaging information or determined information in the control device H, emergency control may be executed based on information determined by the image recognition section 24 . As this emergency control, for example, it is preferable to operate an emergency device such as turning on an emergency lamp or sounding an emergency buzzer when an incorrect removal or storage is performed. It should be noted that the control device H may store imaging information and determined information even when emergency control is executed in this way.

(8) In the above embodiment, the image recognition system S is provided for the picking facility that performs the picking operation of collecting the articles W stored in the supply case C1 of the storage shelf into the article collection case C2 supported by the transport device. Although the configuration described above has been described as an example, it is not limited to this. For example, the image recognition system S performs a picking operation in which an operator M moves a carriage that supports an article collection case C2, and collects articles W stored in the supply case C1 on the storage rack into the article collection case C2 on the carriage. may be provided in a picking facility that performs Further, for example, the image recognition system S may be provided in an assorting facility that performs an assortment operation of collecting the articles W stored in the supply case C1 supported by the transport device into the article collection case C2 of the storage shelf. good.

(9) In the above embodiment, the target supply case CT1 (supply case C1) is the source position, and the collection case C2 is the destination position. However, the source position and the destination position may be changed as appropriate. For example, a storage position (such as a place where no container exists) set in a storage shelf or a transport device, or a transport container transported by a transport device may be used as the movement source position. Further, for example, a storage position (such as a place where no container exists) set in a storage shelf or a transport device, or a storage container or the like stored in a storage shelf may be set as the destination position.

(10) The configurations disclosed in the respective embodiments described above can be applied in combination with the configurations disclosed in other embodiments as long as there is no contradiction. Regarding other configurations, the embodiments disclosed in this specification are merely examples in all respects. Therefore, various modifications can be made as appropriate without departing from the scope of the present disclosure.

3. Outline of the above embodiment An outline of the image recognition system described above will be described below.

The image recognition system includes an imaging device that acquires image information, an image recognition unit that performs image recognition processing on the image information, and an object to be moved by a worker from a movement source position to a movement destination position. and a reference acquisition unit that acquires reference information for judging the correctness of the movement work of the target article, wherein the imaging device includes a movement source region having a plurality of the movement source positions and a plurality of the movement destination positions. and a work area including a destination area, and the image recognition unit recognizes the position of the target article and the hand of the worker who moves the target article, which are included in the image information acquired over time. image recognition of at least one of the positions of (1) to determine the movement source position and the movement destination position of the target article, and determine whether or not the determination result is correct in view of the determination reference information.

According to this configuration, the image recognition unit performs image recognition of at least one of the position of the target article and the position of the hand of the worker who moves the target article, which is included in the image information acquired over time. It is possible to recognize both the source position and the destination position of the target article to be moved. Then, it is possible to determine whether or not both the movement source position and the movement destination position recognized in this way are correct in view of the judgment reference information. That is, it is possible to determine whether or not the combination of the movement source position and the movement destination position for each target article is correct. Therefore, according to this characteristic configuration, it is possible to realize an image recognition system capable of determining whether or not the target article has been correctly moved from the movement source position to the movement destination position.

and a movement source display unit indicating the movement source position of the target article, and a movement destination display unit indicating the movement destination position of the target article, wherein the movement source display section and the movement destination display section , the image recognition unit is installed in the work area, and the image recognition unit performs image recognition of the display of the movement source display unit and the display of the movement destination display unit included in the image information, and recognizes the movement source position and the movement destination display unit of the target article. Preferably, the destination position is determined, and the reference acquisition unit acquires information indicating the source position and the destination position determined by the image recognition unit as the determination reference information.

According to this configuration, the reference acquisition unit acquires, as the determination reference information, information indicating the movement source position and the movement destination position determined based on the display included in the image information recognized by the image recognition unit. Therefore, based on the determination result of the movement source position and the movement destination position of the target article, it is possible to appropriately determine whether or not the worker has correctly moved the target article according to the display of the movement source display section and the display of the movement destination display section. can be determined.

Here, the reference acquisition unit acquires instruction information indicating an instruction to the worker to move the target article as the determination reference information, and the instruction information is movement source information indicating the movement source position of the target article. and destination information indicating the destination position of the target article.

According to this configuration, since the reference acquisition unit acquires the instruction information as the determination reference information, it is possible to appropriately determine whether or not the determination results of the movement source position and the movement destination position of the target article are correct in view of the instruction information. can judge.

The reference acquisition unit acquires type information indicating the type of the target product, and the image recognition unit performs image recognition on the type of the target product included in the image information to determine the type of the target product. It is preferable to determine whether the determination result is correct in view of the type information.

According to this configuration, it is possible to determine whether or not not only the movement source position and the movement destination position of the target article but also the type of the target article that has been moved are correct. Therefore, it is possible to more appropriately determine whether or not the worker has correctly performed the work of moving the target article.

In addition, the reference acquisition unit acquires quantity information indicating the quantity of the target articles to be moved to the same destination position by the movement work, and the image recognition unit acquires the same movement destination, which is included in the image information. It is preferable to image-recognize the quantity of the target articles moved to the position, determine the quantity of the target articles, and further determine whether or not the determination result is correct in view of the quantity information.

According to this configuration, it is possible to determine whether or not not only the movement source position and the movement destination position of the target article but also the quantity of the target article that has been moved are correct. Therefore, it is possible to more appropriately determine whether or not the worker has correctly performed the work of moving the target article.

Further, the movement source position is any one of a storage position set in a storage shelf or a transport device, a storage container stored in the storage shelf, or a transport container transported by the transport device, and It is preferable that the destination position is any one of the accommodation position, the accommodation container, and the transfer container.

According to this configuration, when the movement source position and the movement destination position have various configurations, it is possible to appropriately determine whether or not the worker has correctly performed the work of moving the target article in each configuration.

Moreover, it is preferable that the imaging range of one imaging device includes a plurality of the work areas.

According to this configuration, it is possible to reduce the number of imaging devices compared to the case where an imaging device is provided for each work area, so the manufacturing cost of the image recognition system can be suppressed. In addition, even when workers leave their respective work areas to perform work, it is possible to perform image recognition following the movement of such workers.

Further, it is preferable that a plurality of imaging devices having different imaging directions are installed in one work area.

According to this configuration, by imaging the work area from different imaging directions, it is possible to improve the accuracy of image recognition of the position of the target article and the position of the worker's hand that moves the target article.

The technology according to the present disclosure can be used in an image recognition system that includes an imaging device that acquires image information and an image recognition unit that performs image recognition processing on image information.

1: Fluidized shelf (storage shelf)
2: First conveying device (conveying device)
13: First lighting section (moving source display section)
17: Second lighting section (destination display section)
23: Imaging device 24: Image recognition unit 25: Reference acquisition unit A: Work area A1: Source area A2: Destination area CT1: Target supply case (source position, container)
C2: Item collection case (destination position, transport container)
M: Worker OD: Order information (instruction information)
W: Goods WT: Target goods

Claims (8)

an imaging device that acquires image information;
an image recognition unit that performs image recognition processing on the image information;
a reference acquisition unit that acquires, with respect to a target article of a movement work in which a worker moves an article from a movement source position to a movement destination position, judgment reference information on whether the movement work of the target article is correct or not;
a control device having a storage device that stores the image information including information on the imaging time,
The imaging device captures an image of a work area including a source area having a plurality of source positions and a destination area having a plurality of destination positions;
The image recognition unit
Image recognition of the position of the target article and the position of the hand of the worker who moves the target article included in the image information acquired over time, and performing image recognition of the original position of the target article and the movement of the target article. determining the destination position, determining whether or not the determination result is correct in view of the determination reference information, and determining whether the movement work is correct;
The control device is
When it is determined in the movement work determination that the determination result of the movement source position is correct, the image information imaged by the imaging device is correctly taken out at a position corresponding to the position of the worker's hand. storing the image information in the storage device with a symbol image indicating
When it is determined that the determination result of the movement source position is incorrect in the movement work determination, the image information captured by the imaging device is incorrectly extracted at a position corresponding to the position of the worker's hand. storing the image information in the storage device with a symbol image indicating that
When it is determined in the movement work determination that the determination result of the destination position is correct , the correct accommodation has been performed at the position corresponding to the position of the worker's hand in the image information captured by the imaging device. storing the image information in the storage device with a symbol image indicating
When it is determined that the determination result of the destination position is not correct in the movement work determination, an incorrect accommodation is performed at a position corresponding to the position of the worker's hand in the image information captured by the imaging device. an image recognition system, wherein the image information is stored in the storage device with a symbol image indicating that the image information has been attached. further comprising a movement source display unit that indicates the movement source position of the target article, and a movement destination display unit that indicates the movement destination position of the target article,
The movement source display unit and the movement destination display unit are installed in the work area,
The image recognition unit performs image recognition of the display of the movement source display unit and the display of the movement destination display unit included in the image information, and determines the movement source position and the movement destination position of the target article,
2. The image recognition system according to claim 1 , wherein said reference acquisition unit acquires information indicating said movement source position and said movement destination position determined by said image recognition unit as said determination reference information. The reference acquisition unit acquires, as the determination reference information, instruction information indicating an instruction to the worker to move the target article,
2. The image recognition system according to claim 1 , wherein said instruction information includes source information indicating said source position of said target article and destination information indicating said destination position of said target article. The reference acquisition unit acquires type information indicating the type of the target article,
The image recognition unit performs image recognition of the type of the target article included in the image information, determines the type of the target article, and determines whether or not the determination result is correct in view of the type information. Item 4. The image recognition system according to any one of Items 1 to 3 . The reference acquisition unit acquires quantity information indicating the quantity of the target article to be moved to the same destination position by the movement work,
The image recognition unit performs image recognition of the quantity of the target articles moved to the same destination position, which is included in the image information, determines the quantity of the target articles, and the determination result is included in the quantity information. 5. The image recognition system according to any one of claims 1 to 4 , further determining whether or not it is correct in view of the above. The movement source position is any one of a storage position set in a storage shelf or a transport device, a storage container stored in the storage shelf, and a transport container transported by the transport device,
6. The image recognition system according to any one of claims 1 to 5 , wherein the destination position is any one of the accommodation position, the accommodation container, and the transfer container. 7. The image recognition system according to any one of claims 1 to 6 , wherein an imaging range of one imaging device includes a plurality of said work areas. 8. The image recognition system according to any one of claims 1 to 7 , wherein a plurality of said imaging devices with mutually different imaging directions are installed for said one work area.

Images