JP2016185845A - Inspection processing apparatus, inspection processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an inspection processing apparatus that increases inspection efficiency of a commodity.SOLUTION: An inspection processing apparatus is provided with: a collation section which determines whether or not the collation section can specify a commodity on an image obtained by imaging a target commodity using an image for collation recorded in a database for collation; and a commodity image registration section which registers identification information of the target commodity and the image in the database for collation while associating the identification information with the image, when the collation section has determined that the collation section cannot specify the commodity.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an inspection processing apparatus, an inspection processing method, and a program.

  In the process of taking out and shipping products handled by logistics companies such as warehousing, 3PL, and wholesale, from the product storage space such as shelves, before shipping the product, the product to be shipped is received and received compared to the entry / exit details. There is an inspection work to confirm whether the product is described in the specification. In this inspection work, there is a technology to save labor in inspection work by comparing the image of the product with images of various products registered in the database for verification in advance and specifying the product to be inspected. To do.

  For example, in Patent Document 1, a product is specified by a feature amount of an image obtained by capturing an image of a product placed on a pallet with a camera, and the product is loaded on the pallet determined in association with the specified product. There is disclosed an inspection apparatus that can inspect a product placed on a pallet by specifying the attaching method and calculating the quantity of the product from the stacking method.

JP 2013-67499 A

  In the method of taking an image to be inspected and specifying the product by the feature amount of the image, if the product to be inspected is a thin item such as paper, the matching image of the product to be registered in the matching database is also displayed on the surface. However, if the product to be inspected is a three-dimensional object, it is necessary to register a matching image for all the surfaces of the product. For example, in the case of a product having a cubic box shape, registration for at least six sides is required. In addition, considering the possibility that the angle at which the product appears on the camera can be 360 ° from any direction, even if an image matching process that can cope with a change in angle to some extent is used, registration from several tens of directions is possible. This is necessary, and the registration amount of verification images tends to be enlarged. Since the amount of calculation in the image matching process increases in proportion to the number of registered matching images, the processing time increases when the image matching process is performed using an enlarged matching database. In addition, the number of types of products handled in logistics operations often exceeds 10,000, and as the number of types of products becomes enormous, the number of images for verification increases accordingly, and verification processing takes more time. As a result, there is a waiting time until a user who performs the inspection work or a transfer device such as a conveyor system linked to the inspection system receives a notification of the inspection result. As a result, the user feels waiting stress, and the conveyor system results in a time-out of reception of the result, which causes the inspection work to be inefficient.

  Therefore, an object of the present invention is to provide an inspection processing apparatus, an inspection processing method, and a program for solving the above-described problems.

  The present invention relates to a collation unit that determines whether or not a product shown in an image obtained by capturing a target product can be identified using a collation image recorded in a collation database, and the collation unit determines that the product cannot be identified In this case, the inspection processing apparatus includes: a product image registration unit that registers the identification information of the target product and the image in association with each other in the matching database.

  Further, the present invention determines whether or not it is possible to specify a product that appears in an image obtained by capturing the target product using the verification image recorded in the verification database, and when it is determined that the product cannot be specified in the determination. The inspection processing method is characterized in that the identification information of the target product and the image are associated with each other and registered in the verification database.

  Further, the present invention provides a means for determining whether or not the computer of the inspection processing apparatus can identify a product that appears in an image obtained by capturing a target product using a matching image recorded in a matching database, When it is determined that a product cannot be specified, the program causes the identification information of the target product and the image to be associated with each other and registered in the matching database.

  According to the present invention, the effect of increasing the efficiency of inspection work can be obtained.

It is a block diagram which shows the minimum structure of the inspection processing apparatus by 1st embodiment of this invention. It is a block diagram which shows an example of a structure of the inspection processing apparatus by 1st embodiment of this invention. It is a figure which shows an example of the inspection processing system by 1st embodiment of this invention. It is a flowchart of the inspection process using the inspection processing apparatus by 1st embodiment of this invention. It is a figure which shows an example of the inspection confirmation screen by 1st embodiment of this invention. It is a figure which shows an example of the information which the database used with the inspection processing apparatus by 1st embodiment of this invention memorize | stores. It is a block diagram which shows an example of the hardware constitutions of the inspection processing apparatus by 1st embodiment of this invention.

<First embodiment>
Hereinafter, an inspection processing apparatus according to an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing the minimum configuration of the inspection processing apparatus according to the first embodiment of the present invention.
In this figure, reference numeral 100 represents an inspection processing apparatus. As shown in FIG. 1, the inspection processing apparatus 100 includes at least a verification unit 109 and a product image registration unit 111. The inspection processing apparatus 100 is, for example, a computer including a CPU (Central Processing Unit).
The matching unit 109 determines whether or not a product shown in an image obtained by capturing the target product can be specified using a matching image recorded in a matching database 110 described later.
The product image registration unit 111 associates the identification information of the target product with the image captured of the target product in the verification database 110 in a case where an image of the target product is not recorded in the verification database 110. sign up.

FIG. 2 is a block diagram showing an example of the configuration of the inspection processing apparatus according to the first embodiment of the present invention.
As shown in FIG. 2, the inspection processing apparatus 100 includes a control determination unit 101, a weight acquisition unit 102, a product image acquisition unit 103, an image buffer unit 104, an input reception unit 105, a display information output unit 106, A feature amount calculation unit 107, a product quantity / weight calculation unit 108, a verification unit 109, a verification database 110, a product image registration unit 111, and a product image deletion unit 112 are provided. The product image registration unit 111 is as described above.

The control determination unit 101 instructs information transmission and processing to other functional units.
The weight acquisition unit 102 acquires the weight of the inspection target product measured by the weight measuring instrument from the weight measuring instrument.
The product image acquisition unit 103 acquires an image including the inspection target product imaged by the imaging device from the imaging device.
The image buffer unit 104 is an area for storing an image captured by the product image acquisition unit 103.
The input reception unit 105 acquires information input by the user and outputs the information to the control determination unit 101. The information input by the user is, for example, the quantity of the inspection target product.
The display information output unit 106 generates various information necessary for the inspection work and outputs the information to a display device connected to the inspection processing apparatus 100.
The feature amount calculation unit 107 calculates the feature amount of the image captured by the product image acquisition unit 103 by a general feature amount calculation method such as calculating a graphic feature amount.
The product quantity weight calculation unit 108 divides the weight acquired from the weight acquisition unit 102 by the quantity of the inspection target product received by the input reception unit 105 to calculate the unit weight of the target product. Further, the product quantity weight calculation unit 108 calculates the quantity of the product by dividing the acquired weight by the unit weight.

The matching database 110 is a database that stores product identification information (product code), product matching images, product unit weight, and the like for each product.
The product image deletion unit 112 deletes the image of the target product stored in the matching database 110 based on a predetermined rule.
Note that at least a part of the control determination unit 101 to the product image deletion unit 112 is a function provided when the CPU of the inspection processing apparatus 100 reads and executes a program from a storage unit such as a hard disk.

FIG. 3 is a diagram showing an example of an inspection processing system according to the first embodiment of the present invention.
Hereinafter, the inspection processing system of this embodiment will be described with reference to FIG.
In this figure, reference numeral 1 represents an inspection processing system. FIG. 3 is a view of the inspection processing system 1 as viewed from above. As shown in FIG. 3, the inspection processing system 1 includes an inspection processing device 100, an imaging device 200, a weight measuring device 300, a conveyor 301, a display device 400, and an input device 500. Reference numeral 600 indicates a product to be inspected, and is conveyed in the direction of the arrow 302 by the conveyor 301.

  The imaging device 200 is, for example, a camera that images an inspection target product. The weight measuring device 300 is a device that measures the weight of a product to be inspected. In FIG. 3, the imaging device 200 is attached to the column 303 with the lens facing the ground so that the inspection target product 600 that moves on the conveyor 301 can be imaged from above. The column 303 is fixed to the ground or a conveyor. The attachment position of the imaging device 200 is not limited to the example of this figure. For example, the imaging device 200 may be attached at a position where the side surface of the inspection target product 600 flowing on the conveyor 301 is imaged. The inspection processing apparatus 100 is connected to the imaging apparatus 200 and acquires an image of the inspection target product 600 captured by the imaging apparatus 200. The inspection processing apparatus 100 is connected to the weight measuring device 300 and acquires the weight of the inspection target product measured by the weight measuring device 300.

  The display device 400 is a display, for example. The display device 400 is connected to the inspection processing device 100 and displays information necessary for the inspection work generated by the inspection processing device 100. The input device 500 is a keyboard, a mouse, a touch panel, or the like. The worker operates the inspection processing apparatus 100 using the input device 500 while referring to information displayed on the display device 400 to perform inspection work.

  Next, a rough flow of the inspection work will be described with reference to FIG. First, the worker takes out the inspection target product 600 from a shelf and places it on the conveyor 301. Then, the inspection target product 600 moves in the direction of the arrow 302 by the conveyor 301. In addition, the worker or the conveyor system inputs an imaging instruction signal to the imaging apparatus 200 when the inspection target product 600 comes to a predetermined position, triggered by the worker or from a passage sensor or the like. When receiving the imaging instruction signal, the imaging apparatus 200 captures the inspection target product 600 and transmits the captured image to the inspection processing apparatus 100. The inspection processing apparatus 100 uses the acquired image to read a database for collation provided in its own apparatus, and searches for products. When the inspection target product 600 uniquely matches the registered image for the product A, the inspection processing apparatus 100 identifies the inspection target product 600 as the product A.

In the meantime, the inspection target product 600 moves to the position of the weight measuring device 300. Therefore, the worker or the conveyor system inputs a weight measurement instruction signal so as to measure the total weight of the inspection target product 600. The weight measuring device 300 measures the total weight of the inspection target product 600 and transmits the measured total weight data to the inspection processing apparatus 100.
Next, the inspection processing apparatus 100 calculates the quantity of the inspection target product 600 placed on the conveyor 301 by dividing the total weight data by the unit weight of the specified product A. The inspection processing apparatus 100 displays the identification information of the identified product A and the calculated quantity of the product A on the display device 400. The worker compares the information displayed on the display device 400 with the product and the product quantity described in the entry / exit description to check whether the contents are correct.

  When the product cannot be identified from the image captured by the imaging device 200, the inspection processing apparatus 100 displays the captured image of the inspection target product 600 on the display device 400. The worker visually checks the product image displayed on the display device 400, inputs the product identification information indicated by the product image using the input device 500, and corresponds to the product image displayed on the display device 400. Attach it as an image for verification. As a result, when the imaging apparatus 200 captures the same image as this time when the same inspection target product 600 is inspected next time, which inspection target product 600 is checked by collating with the newly registered verification image. It becomes possible to specify whether it is a product.

  The inspection processing apparatus 100 calculates the excess or deficiency of the inspection target product 600 for the product and its quantity specified through such a process, and the worker carries the inspection target product 600 of the excess and shortage quantity on the conveyor 301. Adjust the quantity by increasing, decreasing, or decreasing. As a result of the above work and processing, when the quantity of the product A displayed on the display device 400 by the worker is in accordance with the entry / exit details, the inspection work for the product is completed.

  By the way, when the inspection target product 600 is a three-dimensional object, there are various methods for mounting the product to be inspected on the conveyor 301. For example, if the product to be inspected 600 is a product B stored in a cubic box, there are six methods for placing the product B on the conveyor 301. Accordingly, it is necessary to register at least six types of matching images for the product B in the matching database 110. Also, whether the worker placed the product B near the end of the conveyor 301, placed the product B near the center, or placed the product B straightly so that the side of the product B and the moving direction of the conveyor 301 are parallel. Even if the same surface of the same product B is imaged from above depending on whether it is placed obliquely or to which position the product B has moved, etc., the imaging device 200 is the product to be inspected. There is a change in the angle at which 600 is imaged. For this reason, even if, for example, a matching image captured from directly above is registered for each of the six surfaces of the product B in the matching database 110, the inspection processing apparatus 100 does not necessarily match the product with the matching images. It is not always possible to specify B. On the other hand, if the image of the product B photographed from various angles is registered as a matching image, the possibility that the product B can be specified increases accordingly. However, as the number of registered matching images increases, the matching process is performed. It takes time and inspection work becomes inefficient.

  Therefore, in the present embodiment, by suppressing the number of verification images registered in the verification database 110 as much as possible, the time required for the product verification process is reduced and the inspection work can be performed efficiently. Next, the inspection process and the registration / deletion process of the verification image in this embodiment will be described with reference to FIG.

FIG. 4 is a flowchart of inspection processing using the inspection processing apparatus according to the first embodiment of the present invention.
First, the worker takes out the inspection target products 600 to be inspected from the product storage space such as a shelf as many as the inspection target products 600 and places them on the conveyor 301 (step S10). Next, when the inspection target product 600 moves to a predetermined position by the conveyor 301, the worker or the conveyor system inputs an imaging instruction signal to the imaging device 200. The imaging device 200 images the inspection target product 600 (step S11) and transmits the captured image to the inspection processing device 100. In the inspection processing apparatus 100, the product image acquisition unit 103 acquires the image (target image) (step S12) and stores it in the image buffer unit 104.

Here, for example, when an operator inputs an inspection start instruction signal from an inspection confirmation screen, which will be described later, the control determination unit 101 instructs the collation unit 109 to specify the product shown in the target image. The matching unit 109 compares each target image with each of the matching images obtained by capturing various products registered in the matching database 110 from various directions, and executes image matching (step S13). Specifically, the matching unit 109 instructs the feature amount calculation unit 107 to calculate the feature amount of the inspection target product 600 included in the target image. The feature amount calculation unit 107 reads out the target image from the image buffer unit 104 and extracts an image of the inspection target product 600 that appears in the target image by extracting, for example, an image other than the image in which the background conveyor 301 is captured from the target image. Further, the feature amount calculation unit 107 calculates a feature amount with respect to the extracted image of the inspection target product 600 by a predetermined known feature calculation method, and outputs the calculation result to the control determination unit 101. The control determination unit 101 records the feature amount in the memory. For example, the collation unit 109 instructs the feature amount calculation unit 107 to calculate a feature amount for each of the collation images registered in the collation database 110 for all products. The feature amount calculation unit 107 calculates a feature amount for each of the matching images registered in the matching database 110 and outputs the feature amount to the matching unit 109. The matching unit 109 compares the feature amount of the target image recorded in the memory with the feature amount calculated for one matching image of the product for which the feature amount calculation unit 107 is present. When the feature quantity of the target image matches the feature quantity of the collation image, or when the difference between the feature quantities is within a predetermined range, the collation unit 109 is a merchandise corresponding to the collation image. Is identified. The matching unit 109 repeatedly executes image matching until a product can be identified.
Note that the feature quantity of the matching image is registered in advance in the matching database 110 in association with the matching image, and the matching unit 109 reads the feature quantity from the matching database 110 and records it in the memory. You may compare with the feature-value of the target image. Moreover, the identification information of the product to be inspected described in the entry / exit specification may be input to the inspection processing apparatus 100 in advance, and the collation unit 109 may perform the collation processing only for these products.

  Next, the collation unit 109 determines whether or not a product has been identified (step S14). When the product can be identified (step S14; Yes), the collation unit 109 outputs the identified product identification information and unit weight to the control determination unit 101. The control determination unit 101 records the identified product identification information and unit weight in the memory. Next, the weight measuring device 300 measures the total weight of the inspection target product 600 using the inspection target product 600 placed on the weight measuring device 300 as a trigger. The weight acquisition unit 102 acquires information on the total weight of the inspection target product 600 measured by the weight measuring device 300 (step S15). The weight acquisition unit 102 outputs the acquired total weight information to the control determination unit 101. The control determination unit 101 records information on the total weight in the memory.

  Next, the control determination unit 101 issues a calculation instruction to the product quantity weight calculation unit 108. The product quantity weight calculation unit 108 reads the total weight information and the unit weight of the specified product from the memory, and calculates the product quantity by dividing the total weight information by the unit weight (step S16). The product quantity / weight calculation unit 108 outputs the calculated quantity to the control determination unit 101, and the control determination unit 101 records the quantity of the product in the memory. The control determination unit 101 reads out the target image and the identification information and quantity of the identified product from the memory and outputs them to the display information output unit 106. The display information output unit 106 generates an image of a confirmation screen (inspection confirmation screen) used by the worker for inspection work and displays the image on the display device 400.

Here, the inspection confirmation screen will be described with reference to FIG. FIG. 5 is a diagram showing an example of an inspection confirmation screen according to the first embodiment of the present invention. Reference numeral 40 denotes an area for displaying the target image. Reference numeral 41 is an image of the inspection target product 600 shown in the target image. Reference numeral 42 is a line for designating the contour of the inspection target product 600 designated by the worker. Reference numeral 43 is an inspection start instruction button for the operator to input the inspection start instruction signal described in step S13. When the operator presses the button 43 or the conveyor system detects the passage of the product, the processing from step S13 described above is executed. In the product code column 44 and the quantity column 45, the inspection result is displayed. As a result of the image matching process, when the matching unit 109 specifies the product and the product quantity / weight calculation unit 108 calculates the quantity of the product placed on the weight measuring device 300, the display information output unit 106 acquires from the control determination unit 101. The identification information of the specified product is displayed in the product code column 44, and the quantity of the product acquired from the control determination unit 101 is displayed in the quantity column 45. When the inspection is successful and the product identification information and quantity are displayed, the display information output unit 106 changes the display of “inspection start” on the button 43 to “inspection completed”, for example. When the worker presses the button 43 displayed as “inspection complete”, an inspection completion instruction signal is input to the inspection processing apparatus 100. Then, the verification unit 109 performs verification result recording processing described later, the display information output unit 106 erases the images and values displayed in the respective columns of the inspection confirmation screen, and the inspection confirmation screen displays the inspection of the next product. It will be ready to.
The product code input field 46 and the quantity input field 47 are input fields for the worker to input the identification information and the quantity of the product to be registered when the inspection process fails and a new product image is registered. The registration button 48 is a registration instruction button used when an operator registers a new product image in the matching database 110 together with information input in the product code input field 46 and the quantity input field 47. The product code input field 46, the quantity input field 47, and the registration button 48 will be described after step S20.

The worker checks whether the product identification information (product code) displayed in the product code column 44 on the inspection confirmation screen and the quantity displayed in the quantity column 45 are in accordance with the receipt / shipment specification (step S17). When it is not according to the receipt / shipment specification (step S17; No), the worker again selects the inspection target product from the shelf and places it on the conveyor 301. Then, the operation or processing from step S10 is repeated.
In the case of receipt / shipment details (step S17; Yes), the collation unit 109 performs collation record recording processing (step S18). The verification result recording process is to record the date and time of successful verification, the number of successful verifications in a predetermined period, etc., for the product image stored in the verification database 110.

  Here, the verification record recording process for the verification database 110 will be described with reference to FIG. FIG. 6 is a diagram showing an example of information stored in a database (collation database 110) used in the inspection processing apparatus according to the first embodiment of the present invention. As shown in FIG. 6, the collation database 110 stores information such as identification information for each product, unit weight, collation image, collation success last date and time, and the number of collation successes. The number of successful collations includes, for example, the total number of times that the product has been successfully identified using the images for collation for each period such as monthly. In the case of FIG. 5, for the product with the identification information “001”, three matching images “XXX1”, “XXX2”, and “XXX3” are registered. The image “XXX1” indicates that the image is a collation image that was used 10 times in the past “period 1” and 8 times in the “period 2” when the product was successfully identified in the product collation process. . Similarly, the image “XXX2” indicates that the image is a verification image used for specifying the product once in the past “period 1” and 0 times in the “period 2”. The same applies to the image “XXX3”. In such a case, for example, the product image deletion unit 112 deletes the matching image related to the image “XXX2” by regarding it as data unnecessary for specifying the product. In this way, by deleting an image that is not used for specifying a product, the amount of product verification processing can be reduced, and the inspection work can be made more efficient. “Final collation date” and “successful collation” in the collation database 110 are items for storing values that serve as a guide for which collation images to delete. In step S18, when the product can be identified, the collation unit 109 stores the current date and time in the “final collation date and time” of the collation image for the collation image used when the product has been successfully identified. The collation unit 109 increments the number of successful collations stored in the corresponding period of “Number of successful collations” by one. These processes are called collation record recording processes.

  When the verification result recording process is completed, the product image deletion unit 112 performs an image deletion process (step S19). The image deletion process is a process of deleting an unnecessary image that is not used for specifying a product from the matching database 110.

  For example, the product image deletion unit 112 deletes a matching image that has not been used for specifying a product for a predetermined period. Specifically, the product image deletion unit 112 has not been used for a long time for specifying a product for a value of “final verification date” in the verification database 110 that has been in the past for a predetermined period of time. Therefore, the frequency with which the product is imaged at an angle or the like at which the image for comparison is imaged is low, and is regarded as unnecessary data, and the data is deleted.

  In addition, for example, when the total number of times that the matching images are used for specifying the products in a predetermined period is equal to or less than a predetermined number, the product image deleting unit 112 deletes the matching images. Specifically, the product image deletion unit 112 has a total value of the number of successful collations in a predetermined period (for example, for the latest period X and period X-1) among the “number of successful collations” in the collation database 110. Those data below a predetermined value are deleted. Thereby, for example, it is possible to delete a verification image that was previously used for specifying a product but is no longer used recently, thereby reducing the registration amount of the verification image.

  In addition, for example, the product image deletion unit 112 deletes a matching image with a small feature amount. Specifically, the product image deletion unit 112 uses the feature amount of the matching image registered in the matching database 110 when the number of feature amounts calculated by the feature amount calculation unit 107 is equal to or less than a predetermined threshold. If there is a small amount, there is a possibility that accurate collation cannot be performed in the image collation process. In addition, about a predetermined threshold value, you may differ for every goods.

  Further, for example, when the area of the target product shown in the verification image is not included in the area range defined for each target product, the product image deletion unit 112 deletes the verification image. Specifically, for the image registered in the database for matching 110, the product image deletion unit 112, when the product has a cubic shape, only a part of the surface of the surface of the product is reflected, If a part of the area is not included in the specified area range, it may not contain enough features of the product and the product may be identified by mistake. To do. On the other hand, when a matching image obtained by capturing a part of a certain surface is more characteristic than the image obtained by capturing the entire surface and is effective for the matching process, The product image deletion unit 112 sets the size to be included, determines that a matching image obtained by capturing the entire surface is not included in the determined area range, and deletes the matching image. . In addition, an image obtained by imaging a plastic bottle or the like from above is not suitable for product matching unless there is a feature on the upper surface of the lid of the plastic bottle. In addition, it is considered that it is not often that the plastic bottle is placed on the conveyor with the lid side facing down and the product is imaged from above. In such a case, if the area of the image registered in the database for matching 110 is the area of the lid of the plastic bottle or the area of the bottom of the plastic bottle, it is determined that it is not included in the area range determined for each target product. Then, the product image deletion unit 112 deletes the image. In addition, not only the area but also the shape of the object that appears in the image is determined for each product, and if the shape of the product that appears in the image registered in the matching database 110 is not the determined shape, the product image deletion unit 112 The image may be deleted.

  In addition, for example, the product image deletion unit 112 deletes other images while leaving one when the degree of similarity is high among a plurality of currently recorded verification images. For example, the product image deletion unit 112 compares similar feature amounts calculated by the feature amount calculation unit 107 with respect to an image of a product registered in the matching database 110 to obtain a similar image. For example, the product image deletion unit 112 deletes other verification images while leaving only the verification images that have the highest number of successful identifications of products among similar images.

The product image deletion unit 112 deletes images based on the predetermined rule exemplified above, but the minimum number of images for verification currently recorded in the verification database is determined for each target product. When the number is reached, the product image deletion unit 112 may not perform this image deletion process. In this way, the minimum number of images is provided because, for example, if there are 6 images for which only 5 images for matching are registered, the product cannot be specified when an unregistered surface is captured. This is to prevent such a situation.
Note that the timing of executing this image deletion process may not be after the verification result recording process. Regardless of the inspection process in FIG. 4, for example, it may be performed periodically once a week. Alternatively, this image deletion processing may be executed only for the product only when the processing from step S13 to step S14 takes a predetermined time or more.

  By the processing in step S19, a collation image that is no longer necessary and has no record of successful product identification for a certain period or longer, for example, one year or longer, is automatically deleted. As a result, the enlargement of the data amount is suppressed and only the effective information is reduced, thereby reducing the calculation amount of the image matching and obtaining an effective inspection result at high speed. As a result, it is possible to reduce the waiting time stress of the user or reduce the result reception timeout by the conveyor system.

  Next, returning to step S14, the case where the product cannot be specified will be described. When the product cannot be specified (step S14; No), the collation unit 109 outputs information indicating that the product cannot be specified to the control determination unit 101. The control determination unit 101 notifies the display information output unit 106 of the product identification failure. The display information output unit 106 generates an image of the inspection confirmation screen illustrated in FIG. 5 and displays it on the display device 400. The display information output unit 106 displays “Unknown” in the product code column 44 and the quantity column 45 (see FIG. 5).

  The worker inputs identification information (product code) of the inspection target product 600 in the product code input field 46 (step S20). For example, the worker confirms the identification information of the inspection target product 600 by looking at the inspection target product 600 placed on the weight measuring device 300. Next, the worker inputs the outline 42 using the input device 500 (step S21). The reason why the contour line 42 is input is to separate the image of the foreground inspection target product 600 from the background conveyor or the like shown in the image. Next, the worker inputs the quantity of the inspection target product 600 placed on the weight measuring device 300 in the quantity input field 47 (step S22). As the quantity of the inspection target product 600, for example, the worker inputs a value obtained by counting the quantity of the inspection target product 600 placed on the weight measuring device 300. Next, the worker presses the registration button 48 using the input device 500 (step S23). Then, the product quantity weight calculation unit 108 divides the total weight of the inspection target product 600 measured by the weight measuring device 300 by the value input in the quantity input field 47 to calculate the unit weight of the inspection target product 600. . The product quantity weight calculation unit 108 outputs the calculated unit weight to the product image registration unit 111. The product image registration unit 111 includes the product identification information received by the input reception unit 105, the unit weight acquired from the product quantity weight calculation unit 108, and the contour line 42 among the product images stored in the image buffer unit 104. The specified portion of the image is registered in the verification database 110. As described above, in the present embodiment, images of products are not registered in advance, and images of products that could not be collated are registered at a timing when the products are inspected. Suppress registration of similar matching images.

Further, the product image registration unit 111 may not perform the current registration when data having the same identification information is already registered in the verification database 110 by a predetermined number or more. Alternatively, when a predetermined number or more of matching images are registered in the matching database 110 for a product that already has the same identification information, the product image registration unit 111 deletes the matching image in the product image deletion unit 112. According to the instruction illustrated in step S19, at least one of the already registered data may be deleted, and the image captured this time may be registered as a verification image instead. Thereby, the enlargement of the database 110 for collation can be prevented.
When registering a matching image in the matching database 110, the product image registration unit 111 calculates the feature amount of the matching image registered in the feature amount calculation unit 107 and associates it with the matching image. You may make it register to the database 110 for collation.

  When a new image for verification is registered, the processing after step S13 is repeated for the inspection target product 600. This processing flow is completed.

  In the above description, an example has been described in which the image captured by the image capturing apparatus 200 is registered in the database for matching 110 when the matching unit 109 fails to specify a product. However, the following may be performed. . For example, the inspection processing apparatus 100 is connected to a storage device (not shown). This storage device stores a master database. The master database stores images taken from a plurality of angles for all products that can be inspected. For example, several tens of types of images are stored for each product. When the verification unit 109 fails to specify the product, the worker inputs the identification information of the product 600 to be inspected into the inspection processing apparatus 100, and inputs a verification instruction signal using the master database. Then, the collation unit 109 performs collation by using the product image registered in the master database based on the product identification information designated by the worker. When the collation is successful and the product can be identified, the collation unit 109 acquires an image from which the product has been successfully identified from the master database. The product image registration unit 111 registers the image acquired by the verification unit 109 from the master database as a verification image in the verification database 110 in association with the identification information of the product.

FIG. 9 is a block diagram showing an example of a hardware configuration of the inspection processing apparatus according to the first embodiment of the present invention.
As shown in FIG. 9, the inspection processing apparatus 100 is realized by a computer apparatus including a CPU 901, a memory 902, a storage device 903 such as a hard disk, and a communication IF 904 (communication interface) for network connection. However, the configuration of the inspection processing apparatus 100 is not limited to the computer apparatus shown in FIG. For example, the inspection processing apparatus 100 may transmit the image or weight data to the outside via the communication IF 904, collate the product to be inspected outside, and receive the result.

  Note that the block diagram used in the present embodiment shows functional unit blocks, not hardware unit configurations. These functional blocks are realized by any combination of hardware and software. Further, the inspection processing apparatus 100 may be realized by one physically coupled apparatus, or may be realized by connecting two or more physically separated apparatuses by wire or wirelessly and by these plural apparatuses. Also good.

  The inspection processing apparatus 100 described above has a computer inside. Each process of the inspection processing apparatus 100 described above is stored in a computer-readable recording medium in the form of a program, and the above process is performed by the computer reading and executing the program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.
The inspection confirmation screen is an example of a registration request screen. The display information output unit 106 is an example of a registration request screen output unit.

DESCRIPTION OF SYMBOLS 1 ... Inspection processing system 100 ... Inspection processing apparatus 101 ... Control determination part 102 ... Weight acquisition part 103 ... Product image acquisition part 104 ... Image buffer part 105 ... Input reception part 106 ... Display information output unit 107 ... Feature quantity calculation unit 108 ... Product quantity weight calculation unit 109 ... Verification unit 110 ... Verification database 111 ... Product image registration unit 112 ... Product image deletion unit 200 ... imaging device 300 ... weight measuring device 400 ... display device 500 ... input device 600 ... inspection product 901 ... CPU
902 ... Memory 903 ... Storage device 904 ... Communication IF

Claims (12)

A collation unit that determines whether or not a product shown in an image obtained by capturing the target product using a collation image recorded in the collation database can be specified;
A product image registration unit that associates the identification information of the target product with the image and registers it in the database for verification when the verification unit determines that the product cannot be specified;
An inspection processing apparatus comprising: The verification database stores images acquired from a master database that stores a plurality of verification images for the target product.
The inspection processing apparatus according to claim 1. The inspection processing apparatus according to claim 1, further comprising: a product image deletion unit that deletes an image stored in the database for matching based on a predetermined rule. The product image deletion unit deletes a matching image that has not been used for specifying a product for a predetermined period of time.
The inspection processing apparatus according to claim 3. The product image deletion unit deletes the matching image when the total number of times used for specifying the product in a predetermined period is equal to or less than a predetermined number of times,
The inspection processing apparatus according to any one of claims 3 to 4. The product image deletion unit deletes a matching image with a small feature amount.
The inspection processing apparatus according to any one of claims 3 to 5. The product image deletion unit, when the area of the target product in the verification image is not included in the area range determined for each target product, delete the verification image,
The inspection processing apparatus according to any one of claims 3 to 6. The product image deletion unit deletes other verification images leaving one when the similarity is high among a plurality of verification images currently recorded,
The inspection processing apparatus according to any one of claims 3 to 7. The product image deletion unit does not delete the verification image when the number of verification images currently stored reaches the minimum number of images determined for each target product.
The inspection processing apparatus according to any one of claims 3 to 8. A product image acquisition unit that acquires an image of the product to be inspected;
A registration request screen output unit for outputting a registration request screen for registering the image in the verification database;
Further comprising
The collation unit determines whether or not the product captured in the product can be specified for the image acquired by the product image acquisition unit,
The product image registration unit registers the identification information of the target product input on the registration request screen in association with the image acquired by the product image acquisition unit.
The inspection processing apparatus according to any one of claims 1 to 9. Determine whether or not it is possible to identify the product that appears in the image of the target product using the image for verification recorded in the database for verification,
When it is determined that the product cannot be specified in the determination, the identification information of the target product and the image are associated and registered in the matching database.
The inspection processing method characterized by the above-mentioned. Computer of inspection processing equipment,
Means for determining whether or not a product shown in an image obtained by capturing a target product can be specified using an image for verification recorded in a database for verification;
Means for registering the identification information of the target product and the image in the matching database in association with each other when it is determined that the product cannot be specified in the determination;
Program to function as.

Images