JP2018097622A - Checking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the time required for checking including image matching.SOLUTION: A checking device includes: an imaging unit which images the contents of a container to acquire captured image which represent appearance of commodity boxes contained in the container; and an image retrieval unit which retrieves a determination image to be used for matching with each of the captured images, from a database, for checking whether a commodity box is contained in the container, so as to conform to shipment information associated with the container. The commodity boxes are classified into groups in advance on the basis of shipping frequency. The groups include at least a first group and a second group with shipping frequency lower than that of the first group. The image retrieval unit determines whether the determination image representing appearance of a commodity box belonging to the first group matches the captured image before determining whether the determination image representing appearance of a commodity box belonging to the second group matches the captured image.SELECTED DRAWING: Figure 8

Description

  The present invention relates to inspection.

  Patent Document 1 discloses an apparatus having a product box master table (database) and a character information acquisition unit. The product box master table associates the entire image information (determination image) with the partial feature information. The overall image information represents the overall appearance of each product box. The partial feature information includes position information of a part of the appearance of each product box and attribute information of each product box included in the part. The character information acquisition unit acquires character information and position information in an image obtained by photographing the matching target product box.

  This device is intended for a product box in which the character information and position information acquired by the character information acquisition unit matches the character information indicating the content of the attribute information included in the partial feature information of the product box master table and the position information. Extracted as product box candidates, and the entire image information of the extracted plurality of target product box candidates and the captured images are collated.

JP 2014-44481 A

  Compared with collation of character information, image collation performed in the above prior art generally takes time. For this reason, when the number of images for determination included in the database increases, it takes time for collation, and the efficiency of inspection work decreases. Based on the above, the present invention has a solution to shorten the time required for inspection including image matching.

  SUMMARY An advantage of some aspects of the invention is to solve the above-described problems, and the invention can be implemented as the following forms.

  One aspect of the present invention is an inspection device configured to be able to search a database in which an image for determination representing the appearance of a product box in which products are packaged is stored; by imaging the contents of a transport container, An imaging unit that acquires a captured image representing an appearance of each of the product boxes stored in the transport container; and the product box is stored in the transport container so as to conform to shipping information associated with the transport container. An image search unit that searches the database for the image for determination that matches the captured image for each of the captured images. Grouped in advance based on frequency, and the group includes at least a first group and a second group that is less frequently shipped than the first group. The image search unit represents the appearance of the product box belonging to the second group, as to whether or not the determination image representing the appearance of the product box belonging to the first group matches the captured image. The inspection apparatus is characterized in that it is performed before the determination as to whether or not the determination image matches the captured image.

  According to this embodiment, the time required for inspection including image matching is shortened. It is determined whether or not a certain captured image is matched with a determination image corresponding to a product box belonging to the first group, and if a result of matching is obtained, a product box belonging to the second group is obtained. Since it is not necessary to determine whether or not to match the corresponding determination image, it can be omitted. The product boxes belonging to the first group are more likely to be matched because the shipping frequency is higher than the product boxes belonging to the second group. Therefore, according to this form, compared with the case where the frequency of shipment is not taken into consideration, there is a higher possibility that the matching determination of the product boxes belonging to the second group can be omitted, and as a result, the above effect can be obtained.

  The said form WHEREIN: You may provide the shipping information acquisition part which acquires the said shipping information using the reading result of the information displayed on the said conveyance container. According to this aspect, for example, the shipping information associated with each transport container is more reliable than the method of acquiring the shipping information based on the imaging result of the imaging unit or the method of specifying the shipping information by the order of the transport containers. Can be obtained.

  In the above embodiment, the shipping information acquisition unit is configured to be communicable with an external device that manages the shipping information; the shipping information acquisition unit transmits the reading result to the external device, and corresponds to the reading result from the external device. Shipping information may be acquired. According to this form, the inspection device does not need to store the shipping information.

  In the above embodiment, the image search unit determines whether or not the captured image matches the image for determination representing the appearance of the first product box, and is shipped less frequently than the first product box. You may implement before the determination whether the said captured image matches the said image for determination showing the external appearance of a 2nd goods box. According to this embodiment, the time required for inspection including image matching is further reduced. It is determined whether or not a certain captured image is matched with the image for determination corresponding to the first product box. If a result of matching is obtained, the image for determination corresponding to the second product box is obtained. Since it is not necessary to determine whether or not to match, it can be omitted. The first product box is more likely to be matched because the shipping frequency is higher than that of the second product box. Therefore, according to this form, compared with the case where the frequency of shipment is not taken into consideration, there is a higher possibility that the matching determination of the second product box can be omitted, and as a result, the above effect can be obtained.

  In the above embodiment, the specific surface of the product box has a symbol that can specify character information associated with the product packaged in the product box at a predetermined position; An image in which the symbol is excluded from the specific surface; the database includes position information indicating the position of the symbol in the determination image, the character information, and the determination image; A symbol acquisition unit that acquires the symbol from the captured image using the position information associated with the image for determination as a matching result by the image search unit; and the symbol acquisition A specifying unit that specifies the product box based on the character information specified by the symbol acquired by the unit. According to this aspect, since the product box is specified by the character information specified by the symbol, the product box can be reliably specified as compared with the case where only image matching is used.

  In the above embodiment, the specifying unit may specify the product box by a combination of the determination image and the character information. According to this aspect, even when the product box cannot be specified only by the character information, the product box can be specified by the combination of the determination image and the character information.

  The present invention can be realized in various forms other than the above. For example, it can be realized in the form of an inspection method, a program for realizing the method, a non-temporary storage medium storing the program, and the like.

The schematic block diagram of an inspection system. The top view of the conveyance container which accommodated the several goods box. The side view of the conveyance container which accommodated the goods box. The schematic block diagram of an inspection apparatus. The sequence diagram which shows an inspection process. The display screen of shipping information is shown. The flowchart which shows collation processing. A table that shows the data structure of the database. An example of an image for determination. An example of an image for determination. An image for notifying products with excess or deficiency.

  FIG. 1 is a schematic configuration diagram of an inspection system 10. The inspection system 10 is a system for inspecting products shipped from a factory or a distribution warehouse. The inspection system 10 includes a server 20 and an inspection device 100. The inspection device 100 is configured to be able to communicate with the server 20 as an external device. The inspection device 100 inspects the product box as the contents of the carried container 30 when the transport container 30 containing the product box is carried into the inspection device 100 by the user.

  FIG. 2 is a top view of the transport container 30 containing a plurality of product boxes. FIG. 3 is a side view of the transport container 30 containing the product box. As shown in FIGS. 2 and 3, the vertical direction is defined as the Z direction, and the horizontal plane is defined as the XY plane.

  The transport container 30 shown in FIGS. 2 and 3 accommodates product boxes A to I. All the product boxes A to I have a substantially rectangular parallelepiped shape. For this reason, in FIG. 2, the product boxes A to I are illustrated as rectangles. In each of the product boxes A to I, as shown in FIG. 2, symbols are written on the surface visible from the upper surface. The symbol here is a bar code or a character string. This symbol is information for specifying the item code, and is hereinafter referred to as an item symbol. The item code is character information for specifying the type of product. When the item symbol is a character string, the item symbol is the item code itself.

  As described above, the item symbol of any product box can be visually recognized from the upper surface. The result of the user manually determining the posture of the product box so that the item symbol can be acquired by imaging from the upper surface (described later). It is. Hereinafter, the surface on which the item symbol is written in the product box is referred to as a specific surface.

  It is assumed that each of the product boxes A to I wraps a product specified by an item symbol written on each product box. Furthermore, it is assumed that a correct number of products are accommodated in each of the product boxes A to I. The correct number of products is uniquely determined by information obtained from the appearance of each of the product boxes A to I. Specifically, the information obtained from the appearance includes a design of a specific surface, an area of the specific surface, a character string (for example, “× 4” indicating four) written on the outside of the product box, and an item symbol. Or at least one of them.

  As shown in FIG. 3, a bar code 33 is displayed on the transport container 30. The bar code 33 is information for uniquely identifying information (hereinafter referred to as shipping information) for specifying the item code of each product to be accommodated in the transport container 30 and the number of shipments of each product. is there.

  FIG. 4 is a schematic configuration diagram of the inspection apparatus 100. The inspection device 100 includes a PC 110, an illumination power source 120, a camera 140, illumination devices 151 and 152, and a barcode scanner 160.

  FIG. 4 also shows the transport container 30 carried into the inspection apparatus 100. However, in FIG. 4, the shape of the transport container 30 is simplified and shown as a rectangular parallelepiped. When the user carries the transport container 30 into the inspection apparatus 100, the user places the transport container 30 at a predetermined position. By disposing the transport container 30 at a predetermined position, each part of the inspection apparatus 100 functions correctly as described later.

  The camera 140 functions as an imaging unit that images the transport container 30 from above and inputs the captured image to the PC 110. As shown in FIG. 3, the camera 140 has a deep depth of focus so that specific surfaces with different heights can be captured at a time. The camera 140 can image the entire contents of the transport container 30 as long as the transport container 30 is disposed at a predetermined position. The illumination power source 120 supplies power to the illumination devices 151 and 152. The illumination devices 151 and 152 illuminate the transport container 30 from above when the camera 140 captures an image.

  The barcode scanner 160 reads the barcode 33 and inputs the read result to the PC 110. The PC 110 is a general-purpose personal computer and includes an arithmetic device, an input / output interface, and the like. The PC 110 communicates with the server 20 via a wired line.

  FIG. 5 is a sequence diagram showing the inspection process. First, the user carries the transport container 30 into the inspection apparatus 100 (S300). Subsequently, the user operates the input interface (mouse or keyboard) of the PC 110 to start the operation of the inspection apparatus 100 (S310).

  When the PC 110 starts operating in response to the above operation, the barcode scanner 160 is used to acquire the reading result of the barcode 33 (S321). Subsequently, the PC 110 requests a return of shipping information by transmitting the reading result of the barcode 33 to the server 20 (S331). The execution of S331 realizes a shipping information acquisition unit.

  When the server 20 receives the reading result of the barcode 33, the server 20 returns shipping information corresponding to the received content to the inspection apparatus 100 (S342). Upon receiving the shipping information, the inspection apparatus 100 displays the shipping information on the screen (S351), and executes a verification process (S400).

  FIG. 6 shows a display screen for shipping information. This display screen is displayed on a display included in the PC 110. The screen displays a list of item codes and the number of products shipped. In the present embodiment, the number of product boxes is not displayed, but the number of product boxes may be displayed.

  FIG. 7 is a flowchart showing the matching process. First, imaging is executed (S410). In S410, the camera 140 and the illumination devices 151 and 152 described above are used.

  Next, an image indicating a specific surface of each product box is extracted from the captured image (S420). That is, a rectangular area indicating the boundary of the product box is divided for each product box, and each area is extracted as an image representing a specific surface of each product box. The image extracted in this way is hereinafter referred to as an extracted image.

  Subsequently, image matching is executed (S430). Specifically, a determination image that matches each extracted image is searched from a database stored in the inspection apparatus 100 itself. The execution of S430 realizes an image search unit.

  FIG. 8 is a table showing the data structure of the database. In the database, information on the shipping frequency group, the determination image, the item code, and the number of packages is associated.

  The determination image is an image of a specific surface of each product box. The item code indicates the type of product as described above. The number of packages indicates the number of products packaged in the product box.

  9 and 10 show examples of determination images. The item symbol itself is excluded from the determination image. The determination image includes position information indicating an area where the item code is written instead of the item symbol itself. The position information is indicated by a broken line as a region SB1 in FIG. 9 and a region SB2 in FIG.

  In the present embodiment, as shown in FIG. 8, there are a determination image in which only one item code is associated, and a determination image in which a plurality of item codes are associated. That is, in the present embodiment, it is not always possible to specify a product simply by specifying a determination image. Similarly, in the present embodiment, there may be only one determination image corresponding to a certain item code, or there may be a plurality of determination images. The reason why a plurality of determination images correspond to a certain item code is because product boxes having different numbers of packages are prepared for the products of the item code. For this reason, there are cases where the number of products shipped cannot be specified simply by specifying the item code. Therefore, in the present embodiment, the product box is specified by a combination of the determination image and the item code. If the product box is specified, the product and the number of packages are specified.

  The shipping frequency group is a grouping of shipping frequency of product boxes in three levels (high, medium, low). The shipping manager determines which shipping frequency group each product box belongs to. The shipping manager periodically reviews the shipping frequency of each product box based on the shipping results and predictions, and stores them in the PC 110. Note that product boxes belonging to the same shipping frequency group are not necessarily sorted in order of shipping frequency. In the present embodiment, the number of determination images stored in the database is enormous (for example, 1000 types or more), and sorting all the determination images accurately in order of shipping frequency requires a great effort. In the first place, the shipping frequency can change from moment to moment depending on the order status, so it is difficult to define the exact shipping frequency. Therefore, the shipping manager creates the database by sorting into the groups of high, medium, and low, and sorting in the order in which the shipping frequency is estimated to be high.

  The matching determination in S430 is performed sequentially for each extracted image. Searching for a determination image that matches a certain extracted image is realized by determining match / mismatch in order from the determination image stored in the higher rank. For this reason, the image is generally subjected to matching determination from a determination image having a high shipping frequency. That is, the determination as to whether the determination image belonging to the high group matches the extracted image is performed prior to the determination as to whether the determination image belonging to the middle group matches the extraction image. Further, the determination as to whether or not the determination image belonging to the middle group matches the extracted image is performed before the determination as to whether or not the determination image belonging to the low group matches the extracted image.

  If it is determined that they match, the matching determination for the extracted image ends at that time. For this reason, matching with the determination image stored lower than the matched determination image is not performed. For example, when the shipping frequency matches the determination image belonging to the high group, the determination with the determination image belonging to the middle group and the low group is not performed.

  When image matching is completed for all the extracted images, the item symbol in each extracted image is read (S440). Since the image matching is completed, the area where the item symbol is written can be specified as described above, and reading is performed on this area. If the item symbol is a bar code, it is read according to the standard, and if the item symbol is a character string, it is read using optical character recognition. The execution of S440 implements a symbol acquisition unit.

  Subsequently, a product box is specified (S445). The execution of S445 realizes the specifying unit. S445 is executed based on the item code specified by the read item symbol. More specifically, it is determined by the combination of the determination image and the item code specified by the read item symbol.

  Next, the number of products having the same item code is summed (S450). If a product box is specified, the number of products packaged in each product box can be specified using a database. Using this, the number of products is totaled for each item code.

  Subsequently, it is determined whether the inspection is passed (S460). That is, it is determined whether or not the result obtained in S450 matches the shipping information. If it is rejected (S460, NO), the user is notified of which product box has excess or deficiency (S470).

  FIG. 11 illustrates an example of the above notification. As shown in FIG. 11, the cells storing the two item codes from the top and the number of commodities shipped are shaded. This shading indicates that the cell is colored in yellow or the like on an actual screen. This coloring indicates that the product box for packaging the product specified by the colored item code is correctly accommodated in the transport container 30.

  On the other hand, the cell storing the third item code from the top and the number of shipments is not shaded. That is, this cell is not colored on the screen. Uncolored cells indicate products that have excessive or insufficient shipments. If a product that is not included in the shipping information is included, “Not eligible for shipping” is displayed in the number of shipments.

  When the user visually recognizes that some of the cells are not colored, the user recognizes that there is an excess or deficiency in the number of product boxes that contain the products specified by the item codes stored in the uncolored cells. Thereafter, the user performs an operation so as to eliminate the excess and deficiency of the product boxes, and then inputs a re-imaging instruction to the PC 110.

  The inspection apparatus 100 waits until an imaging instruction is input (S480), and after the instruction, executes S410 to S460 again. On the other hand, when the result obtained in S450 matches the shipping information (S460, YES), the collation process is finished.

  When the collation process ends, as shown in FIG. 5, the PC 110 notifies the user that the inspection has passed (S501) and transmits it to the server 20 (S511), and the inspection process ends. The inspection pass means that the product box is accommodated in the transport container 30 so as to conform to the shipping information associated with the transport container 30.

  The notification in S501 is realized by coloring all the cells in the list display shown in FIG. Furthermore, characters such as “passed inspection” may be displayed on the display, or a sound such as “passed inspection” may be output.

  According to the embodiment described above, matching is performed by performing image matching using a database sorted based on shipping frequency, compared to a case where image matching is executed without considering shipping frequency. Time is shortened.

  When creating a database based on shipping frequency as described above, avoiding sorting strictly in the order of shipping frequency and using rough grouping reduces the labor required to create the database and the time required for matching judgment To balance with the profits obtained by

  Further, the product box can be specified accurately by performing the image matching and the recognition of the item symbol in stages. Furthermore, since the PC 110 automatically acquires the inspection result, the user does not need to input the inspection result to the PC 110.

  The present invention is not limited to the embodiments, examples, and modifications of the present specification, and can be implemented with various configurations without departing from the spirit of the present invention. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in the embodiments described in the summary section of the invention are to solve some or all of the above-described problems, or In order to achieve part or all of the effects described above, replacement or combination can be performed as appropriate. If the technical feature is not described as essential in this specification, it can be deleted as appropriate. For example, the following are exemplified.

Unlike the embodiment, the number of shipping frequency groups may not be three, but may be two or four or more.
The shipping frequency group information may not be included in the database. That is, it is only necessary that the sorting order of the determination images is determined based on the shipping frequency group.

The inspection apparatus may acquire the shipping information without using the reading of the barcode displayed on the side surface of the transport container. For example, a barcode may be displayed on the transport container so as to be read from the upper surface, and the barcode may be read using a camera.
Alternatively, the inspection apparatus does not have to acquire shipping information from the server. For example, if the inspection apparatus stores in advance the relationship between the order of the transport containers and the shipping information, the shipping information can be determined by counting the order of the transport containers.

In the database, the determination images may be accurately sorted in order of shipping frequency. In this way, the time required for image matching is further shortened.
The database may not be stored in the PC 110, and may be stored in, for example, a server.

  An item symbol may be included in the determination image. By doing so, the item symbol becomes a part of the image, and all the determination images are different from each other, so that the product box can be specified only by the determination image.

  When the product box can be specified only by the item code, the product box may be specified only by the item symbol as the specification of the product box in S445. In this case, the image matching is executed to specify the area of the item symbol.

  When it is determined that the product is rejected in S460, the user may perform an operation so as to pass the inspection by a method other than the method of causing the inspection device to pick up an image again. For example, when the item symbol of the product box displayed as having an excess or deficiency is a barcode, the barcode of the product box may be read with a barcode scanner. The barcode scanner used here may be the barcode scanner 160 for reading the barcode of the transport container, or another barcode scanner.

  In the embodiment, after performing the matching determination in S430 one by one for each of the extracted images, and performing the matching determination for all the extracted images, the item symbol in each extracted image is read in S440, After that, the product box was specified in S450. However, for example, “matching determination in S430 is performed on one extracted image among a plurality of extracted images, an item symbol in the extracted image is read in S440, and then a product box is specified in S450”. It may be a method of performing matching determination for all the extracted images by repeating.

DESCRIPTION OF SYMBOLS 10 ... Inspection system 20 ... Server 30 ... Transport container 33 ... Bar code 100 ... Inspection apparatus 110 ... PC
DESCRIPTION OF SYMBOLS 120 ... Illumination power supply 140 ... Camera 151 ... Illumination device 152 ... Illumination device 160 ... Barcode scanner

Claims (6)

It is an inspection device configured to be able to search a database in which a judgment image representing the appearance of a product box in which products are packaged is stored,
An imaging unit that acquires captured images representing the appearance of each of the product boxes accommodated in the transport container by capturing the contents of the transport container;
In order to inspect whether the product box is accommodated in the transport container so as to conform to the shipping information associated with the transport container, for each captured image, the determination image that matches the captured image is An image search unit for searching from the database;
An inspection device comprising:
The product boxes are grouped in advance based on the frequency of shipment,
The group includes at least a first group and a second group that is less frequently shipped than the first group,
The image search unit represents the appearance of the product box belonging to the second group, as to whether or not the determination image representing the appearance of the product box belonging to the first group matches the captured image. An inspection apparatus, which is performed prior to determining whether or not the determination image matches the captured image. The inspection apparatus according to claim 1, further comprising a shipping information acquisition unit that acquires the shipping information using a reading result of the information displayed on the transport container. It is configured to be able to communicate with an external device that manages the shipping information,
The inspection device according to claim 2, wherein the shipping information acquisition unit transmits the reading result to the external device, and acquires the shipping information corresponding to the reading result from the external device. The image search unit determines whether or not the captured image matches the image for determination representing the appearance of the first product box. The second product is less frequently shipped than the first product box. The inspection according to any one of claims 1 to 3, wherein the inspection is performed before the determination as to whether or not the captured image matches the determination image representing the appearance of the box. apparatus. On the specific surface of the product box, a symbol that can specify character information associated with the product packaged in the product box is written at a predetermined position,
The determination image is an image in which the symbol is excluded from the specific surface,
In the database, the determination image, the character information, and position information indicating which position of the determination image is written are associated with each other,
A symbol acquisition unit that acquires the symbol from the captured image using the position information associated with the image for determination as a matching result by the image search unit;
The specific part which specifies the said product box based on the said character information specified by the said symbol acquired by the said symbol acquisition part is provided. The any one of Claim 1 to 4 characterized by the above-mentioned. Inspection equipment described in 1. The inspection device according to claim 5, wherein the specifying unit specifies the product box by a combination of the determination image and the character information.

Images