JP2022113972A - Inspection system using image recognition, inspection method, and program for inspection system - Google Patents

Abstract

To provide an inspection system capable of performing an automatic inspection of at least three objects and a combination thereof accurately and at high speed.SOLUTION: An inspection system of the present invention that performs inspection of at least three of objects includes: a photographing unit that photographs an image including at least three of the objects; a preprocessing unit that extracts an image portion for each of the objects from the photographed image; an image recognition unit that performs image recognition of the objects included in the image portion for the extracted image portion; and a determination unit that determines an inspection result based on a recognition result of the image recognition unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to an inspection system that supports shipping inspections (inspections), and more specifically, inspections using image recognition of multiple products included in a shipment to determine whether the combination of multiple products included in the shipment is appropriate. The present invention relates to an inspection system capable of confirming whether or not

Conventionally, various techniques are known for supporting shipping inspections (inspections) of products. For example, in Patent Document 1, identification symbols such as barcodes attached to products are read by a reading device such as a barcode reader, and identification of target products, excess, shortage, mistakes, etc. are confirmed to support shipping inspection. A shipping inspection device is disclosed.

Japanese Patent Laid-Open No. 2002-200003 discloses automating product collation work by collating a photographed product image with a pre-registered image and automatically confirming whether they match or not. .

Patent Literature 3 discloses an inspection system that collectively reads identification symbols such as barcodes attached to a plurality of products and inspects the plurality of products.

JP-A-6-4552 JP 2016-121019 A Japanese Patent No. 6557525

Depending on the nature of the products/services provided by the business, there are cases where multiple types of products are packaged and shipped in one shipment. For example, in an Internet connection service, a communication device main body such as a modem, a power adapter, and a base for installing the communication device main body are packaged as devices (products) necessary for using the service. The power adapter and base must be packed in accordance with the standards of the main body of the communication equipment. When multiple types of products are packed and shipped in this way, not only inspection of each product to be packed but also inspection of whether or not the combination of multiple types of products to be packed is appropriate is required. .

In the inspection system of Patent Document 3, although a plurality of identification symbols such as barcodes are read collectively, in order to inspect a plurality of types of products, the plurality of read identification symbols are grouped and reference data for each group is used. This requires complicated processing processes such as preparation of products, and a lot of time is required for inspection of each product. In addition, if there is a product that cannot be affixed with a barcode due to the nature of the product, it is necessary to conduct a separate inspection for that product as well as an inspection to determine whether the combination of multiple products is appropriate.

Due to these problems, visual inspection is still the main method of inspection when multiple types of products are packed in a shipment such as Internet connection service. However, there are more than 100 types of modems, more than 150 types of adapters, and more than 20 types of bases, and there are multiple manufacturers that manufacture modems, and multiple manufacturers that manufacture adapters and bases. be. Therefore, the size and font of characters that can identify these types, the type of barcode, their position, and the shape of the object differ for each manufacturer, so for visual inspection, the manufacturing business It is required to memorize in advance such information, which differs from person to person. In addition, since modems, adapters and bases are rental items, not only new items but also rental items may be reused. Therefore, the amount of information to be memorized in advance increases. Thus, the visual inspection requires skill of the operator, such as knowledge of the products to be packed. Moreover, it is difficult for even an expert to keep inspection quality constant, and there is a problem of variation in inspection quality due to differences in judgment criteria among operators.

Conventionally, attempts have been made to automate such inspections, but as described above, there are various types of inspection objects, character sizes and fonts, barcode types and their positions, and object shapes. . Therefore, it is not possible to adopt the method commonly used by manufacturers to specify the position of the information on the object as the position to be read, and accurate and high-speed automation of inspection has not been achieved. .

SUMMARY OF THE INVENTION It is an object of the present invention to provide an inspection system capable of accurately and simultaneously automatically inspecting at least three objects and their combinations. and

An inspection system of the present invention for solving the above problems is an inspection system for inspecting at least three objects, comprising: an imaging unit for imaging an image including at least three objects; a preprocessing unit that extracts each image portion; an image recognition unit that performs image recognition of an object included in each extracted image portion; and an inspection based on the recognition result of the image recognition unit. and a determination unit.

In the inspection system of the present invention, the preprocessing section preferably extracts an image portion for each object from the photographed image based on the coordinate information.

In the inspection system of the present invention, it is preferable that the preprocessing section divides the photographed image into extracted image portions.

Preferably, the image recognition section performs at least three different image recognitions.

In the inspection system of the present invention, the image recognition unit includes a first image recognition unit that performs image recognition based on a character code acquired from the first image portion by optical character recognition, and a barcode from the second image portion. at least a second image recognition unit that performs image recognition based on the bar code acquired by recognition, and a third image recognition unit that performs image recognition based on template matching of the target object included in the third image portion; It is preferable to have

In the inspection system of the present invention, it is preferable that the first image recognition section performs image recognition based on whether or not a plurality of keywords are included in the acquired character code.

In the inspection system of the present invention, the second image recognition section preferably performs image recognition based on an identification master including the type of barcode symbol and data identification key for each article data of the object.

Preferably, the inspection system of the present invention further includes a storage section for storing the image captured by the imaging section.

An inspection method of the present invention for solving the above-mentioned problems is an inspection method for inspecting at least three objects, comprising the steps of capturing an image including at least three objects; a step of recognizing an object included in each extracted image portion; a step of determining an inspection result based on the recognition result of the image recognition unit; characterized by comprising

An inspection system program of the present invention for solving the above problems is an inspection system program for inspecting at least three objects, wherein an image portion for each object is extracted from an image containing at least three objects. an image recognition unit that performs image recognition of an object included in each extracted image portion; a determination unit that determines an inspection result based on the recognition result of the image recognition unit; is realized in a computer.

According to the present invention, an image portion for each object is extracted from a photographed image, and image recognition of the object included in the extracted image portion is performed for each extracted image portion. All objects in the image can be uniquely identified by recognizing the identifying information. As such, automatic inspection of at least three objects and their combinations can be performed accurately and simultaneously.

1 is a configuration diagram schematically showing the network configuration of an inspection system according to an embodiment of the present invention; FIG. FIG. 2 is a block diagram schematically showing the functions of an inspection system; FIG. 1 is a schematic diagram of an example of an imaging device that constitutes an imaging unit; FIG. 4 is a diagram showing an example of an image captured by a capturing unit; 4 is a conceptual diagram of a configuration of information stored in a storage unit; FIG. FIG. 4 is a diagram showing an example of the relationship between a photographed image and image portions; The figure which shows an example of a structure of an adapter. The figure which shows an example of a structure of a modem. The figure which shows an example of a structure of a base. 4 is a flowchart showing main processing of the inspection system; 4 is a flowchart showing details of processing of an image recognition unit of the inspection system;

Embodiments of the present invention will be exemplified in detail below with reference to the drawings. In this specification, common terms are used for common configurations, and the same reference numerals are used as necessary to omit detailed descriptions by showing substantially the same configuration in each drawing.

The inspection system of the present embodiment is an inspection system that inspects at least three objects, and includes a photographing unit that photographs an image containing at least three objects, and an image portion for each object that is extracted from the photographed image. an image recognition unit that performs image recognition of an object included in each extracted image portion; a determination unit that determines an inspection result based on the recognition result of the image recognition unit; is provided. An embodiment of the present invention in which the objects to be inspected are a modem M, a modem power adapter A (hereinafter also simply referred to as an "adapter"), and a base B will be described below. Note that the number of objects can be four or more, and in this case, two or more objects may be inspected by image recognition technology, or all objects may be inspected by different image recognition technologies.

FIG. 1 is a configuration diagram schematically showing the configuration of a network including user terminals 2 used in an inspection system 1 according to one embodiment of the present invention. As illustrated, the user terminal 2 is communicably connected to a photographing device 4 and an external terminal and/or an external server 5 via a communication network 3 such as the Internet. Although only one external terminal and/or external server 5 is illustrated in FIG. 1, the user terminal 2 may be communicatively connected to a plurality of other external terminals and/or external servers 5. . The user terminal 2 is an example of a device implementing part or all of the inspection system 1 of the present invention. The imaging device 4 and the external terminal and/or external server 5 are configured to be able to cooperate with the user terminal 2, and are examples of devices that can implement a part of the inspection system 1 of the present invention. Note that FIG. 1 does not show other external devices that do not constitute the inspection system 1 of the present invention.

The user terminal 2 is configured as a general computer, and as shown in FIG. 10, and these constituent elements are electrically connected via a bus or the like (not shown).

The computer processor 5 is configured as a CPU, GPU, or the like, reads various programs stored in the storage 10 or the like into the main memory 7, and executes various instructions included in the programs.

The input/output I/F 8 includes various input/output devices for exchanging information with a user or the like. The input/output I/F 8 includes, for example, information input devices such as keyboards and pointing devices (for example, mouse, touch panel, etc.), voice input devices such as microphones, and image input devices such as cameras. The input/output I/F 8 includes an image output device such as a display and an audio output device such as a speaker.

The communication I/F 9 is implemented as hardware such as a network adapter, various types of communication software, or a combination thereof, and is configured to enable wired or wireless communication via the communication network 3 or the like.

The storage 10 is configured by, for example, a magnetic disk, flash memory, or the like. The storage 10 stores various programs including an operating system, various data, and the like.

In this embodiment, the inspection system 1 can be configured using a plurality of computers each having the hardware configuration described above. For example, the inspection system 1 can be configured using one or more user terminals 2 . In particular, the inspection system 1 of this embodiment uses a plurality of user terminals 2 .

In this embodiment, the user terminal 2 can be configured as a smart phone, a tablet terminal, and a personal computer.

The external terminal and the external server 5 are configured as a general computer like the user terminal 2, and, like the user terminal 2, have a computer processor, a main memory, an input/output I/F, and a communication I/F. , and a storage (storage device), and these constituent elements are electrically connected via a bus or the like (not shown). Each component of the external terminal and the external server 5 may have the same configuration as the user terminal 2 . The external server 5 includes, for example, a database server and/or a cloud service providing server.

Next, the functions of the inspection system 1 of this embodiment will be described. FIG. 2 is a block diagram schematically showing the functions of the inspection system 1. As shown in FIG. As illustrated, the inspection system 1 of the present embodiment includes an input unit 11, an imaging unit 12, a storage unit 13, a preprocessing unit 14, an image recognition unit 15, a determination unit 16, and a display unit 17. , has These functions are realized by the cooperative operation of hardware such as the computer processor 5 and the main memory 6, and various programs and data stored in the storage 9 and the like. It is implemented by the computer processor 5 executing instructions included in the loaded program. Moreover, these functions of the inspection system 1 can be realized by the user terminal 2, the imaging device 4, the external terminal and/or the external server 5, or their cooperation. These functions of the inspection system 1 are communicably connected to each other by wire or wirelessly.

The input unit 11 receives an operation from an operator and outputs input information. In this embodiment, at the start of the inspection work, the operator operates the input unit to input the article code of the combination of the target articles to be inspected. The input unit 11 outputs the input product code information to the storage unit 13 , the determination unit 16 and the display unit 17 . In this embodiment, the input unit 11 is configured in the user terminal 2 .

The photographing unit 12 photographs images including images of at least three inspection objects. FIG. 3 is a schematic diagram of an example of the photographing device 4 that constitutes the photographing section 12. As shown in FIG. The photographing device 4 is connected to the user terminal 2 and outputs image data of the photographed image to the user terminal 2 . The photographing device 4 constituting the photographing unit 12 of the present embodiment includes a camera 41, a rack 42 for fixing the camera, a mounting table 43 for placing an object, and a photographing button (Fig. not shown). Although the imaging unit 12 is configured as a separate device from the user terminal 2 in this embodiment, it may be configured as a part of the user terminal 2 .

In this embodiment, the camera 41 is configured to photograph the entire mounting table 43 with 3000×4096 pixels. The camera 41 of this embodiment is provided at the tip of the rack 42 so as to be positioned 40 cm above the mounting table 43 . Therefore, since the cardboard or the like in which the object to be inspected is packed can be positioned between the camera 41 and the mounting table 43, a space for moving and storing the cardboard or the like can be secured, and the packing work after the inspection can be facilitated. I am able to In addition, in this embodiment, a space is provided between the mounting table 43 and the mounting surface of the mounting table 43 so that a cardboard or the like in which the object to be inspected is packed can be positioned.

Further, in the present embodiment, there are no obstacles around the mounting table 43 that hinder the movement of the object in the direction parallel to the mounting surface of the mounting table 43 . Therefore, movement of the object onto the mounting table 43 along the mounting surface of the mounting table 43, photographing of the object, movement of the object from the mounting table 43 along the mounting surface of the mounting table 43, and packing can be done smoothly.

In this embodiment, a mounting table 43 made of blue polyethylene is used. By making the mounting table 43 made of polyethylene, it is possible to prevent the object from being damaged by the mounting table 43 . Also, the blue color of the mount 43 is not a color commonly used for modems, adapters and bases. By making the color of the mounting table 43 different from the color of the object in this way, it is possible to easily separate the background from the photographed image.

Since the adapter, modem, and base, which are objects to be inspected in this embodiment, have different height dimensions, the focal length of each object to be inspected may differ when photographing from above the mounting table 43. The photographing unit 4 of this embodiment can acquire an image in which the adapter, modem, and base are all in focus at the same time, and the combination of the adapter, modem, and base to be inspected is changed. The depth of field is set so that the adapter, modem, and base can all be in focus at the same time without changing the camera position. there is

In this embodiment, the photographing button for photographing operation of the camera is configured to be movable. By making it movable, the photographing button can be arranged near the mounting table 43, and the photographing can be performed smoothly after the object is arranged. Note that the shooting operation of the camera may be performed by pressing a button of the camera, or may be performed based on the input to the input unit 11 .

FIG. 4 is a diagram showing an example of an image taken by the imaging unit 12. The image shown in FIG. 4 includes images of the modem M, the adapter A, and the base B, which are objects. The photographing unit 12 outputs the photographed image to the storage unit 13 for storage.

In this embodiment, the storage unit 13 is configured in each of the user terminal 2 and the external terminal and/or external server 5 . The storage unit 13 of the external terminal and/or the external server 5 stores various kinds of master information used for inspection. Note that these pieces of master information may be stored in the storage unit 13 of the user terminal 2 . The storage unit 13 outputs information for inspecting the modem M, the adapter A, and the base B to be inspected to the determination unit 16 based on the article code information output by the input unit 11 .

FIG. 5 is a conceptual diagram of the configuration of information stored in the storage unit 13 of this embodiment. The storage unit 13 of the present embodiment stores set combination master information I1, modem master information I2, barcode identification master information I3, adapter conversion master information I4, base master information I5, and individual identification number master information as master information. I6 is stored. Note that FIG. 5 shows only part of the master information. Actually, the set combination master information I1 is the number of combinations of the modem M, the adapter A and the base B, and the modem master information I2 is the modem number. The number of types, the barcode identification master information I3 is the number of identification keys, the adapter conversion master information I4 is the number of adapter types, the base master information I5 is the number of base types, and the individual identification number master information I6 is the modem type. Each number contains a corresponding number of data. Data with the same names in FIG. 5 indicate the same data.

The set combination master information I1 includes a product code that can specify the combination of modem M, adapter A and base B, a modem product code that can specify modem M, an adapter product code that can specify adapter A, and base B. and an identifiable base item code. The modem item code, adapter item code and base item code are each associated with the item code so as to identify the modem M, adapter A and base B included in the combination identified by the item code.

The modem master information I2 includes a modem article code, data identification key 1 and data identification key 2. FIG. Data Identification Key 1 and Data Identification Key 2 are each associated with a modem article code. The data identification key 1 and the data identification key 2 externally refer to the data identification key of the barcode identification master information I3.

The barcode identification master information I3 includes a data identification key, a barcode symbol type, and a regular expression of barcode content. The barcode symbol type and barcode content regular expression are each associated with an identification key. An example of the barcode content format is shown next to the barcode content regular expression for reference.

The adapter conversion master information I4 includes an adapter article code, keyword 1, keyword 2, keyword 3, and keyword 4. Keyword 1, keyword 2, keyword 3 and keyword 4 are each associated with an adapter product code.

Base master information I5 includes a base product code and base master image data. Base master image data is associated with a base item code. Note that in FIG. 5, names of base image data are shown instead of actual base images.

The individual identification number master information I6 includes a modem article code and an individual identification number. A unique identification number is associated with the modem item code.

In this embodiment, the storage unit 13 is configured in each of the user terminal 2 and the external terminal and/or external server 5 . In this embodiment, the storage unit 13 of the external terminal and/or the external server 5 stores various kinds of master information. Prior to the start of inspection work, the storage unit 13 of the external server 5 outputs master information to the storage units 13 respectively implemented in the plurality of user terminals 2, and the user terminals 2 operate based on the output master information. Carry out inspection work. By adopting such a configuration, even when a new combination of target products is generated, the new combination can be inspected on a plurality of user terminals 2 simply by updating the master information in the storage unit 13 of the external server 5. It allows you to do it immediately.

The storage unit 13 of the present embodiment is configured to store inspection results in addition to images captured by the imaging unit 12 .

The preprocessing unit 14 extracts image portions including the target objects, that is, the adapter A, the modem M, and the base B, from the photographed image, and outputs information of the extracted image portions to the image recognition unit 15 . FIG. 6 is a diagram showing an example of the relationship between captured images and image portions. In this embodiment, extraction of an image for each object is performed based on coordinate information. Specifically, for example, as shown in FIG. 6, the preprocessing unit sets the X-axis so that the maximum value is 100 along the horizontal sides of the captured image, with the upper left corner of the captured image as a reference. , the Y-axis is set so that the maximum value is 75 along the vertical sides.

In this embodiment, the coordinate information set in this way is used to specify the range of the image portion to be extracted. Specifically, in this embodiment, the ranges of the first to third image portions are specified so as to be the areas specified by P1 to P3, respectively. As shown in FIG. 6, the first image portion P1 contains the image of the adapter A, the second image portion P2 contains the image of the modem M, and the third image portion P3 contains the image of the base B. Contains images.

Especially in this embodiment, the preprocessing unit 14 divides the captured image into the extracted first to third image portions. The preprocessing unit 14 of this embodiment further deletes the background image. Specifically, the portions displayed in the blue background color are deleted from each of the divided first to third image portions. Therefore, the preprocessing unit 14 of this embodiment generates an image obtained by removing the background image from the divided first to third image portions, that is, an image substantially including only the adapter A, the modem M, and the base B, Output to the image recognition unit 15 . Therefore, in this embodiment, it is possible to reduce the amount of image data to be output and contribute to improving the processing speed of the subsequent image processing unit 15 and communication processing speed with an external server.

In particular, the preprocessing unit 14 of this embodiment converts the divided first image portion P1 into data in a file format suitable for OCR. In this embodiment, the first image recognition section 15A converts the first image portion P1 into JPEG (jpg) format data. Such file format conversion can be appropriately changed according to the OCR method. Such file format change may be performed by the first image recognition section 15A.

In this embodiment, since the first image portion P1 is converted into JPEG (jpg) format data, the data size is reduced within a range in which the accuracy of OCR does not deteriorate. Therefore, it is possible to reduce the amount of communication transmission and improve the response speed in each subsequent process while maintaining the OCR accuracy.

The preprocessing unit 14 of the present embodiment further binarizes the first image portion P1 whose file format has been converted. By this binarization processing, the contrast between the character image and the background image included in the first image portion P1 becomes clear, so character recognition by OCR becomes easy, and higher accuracy OCR is realized. Such binarization processing may be performed by the first image recognition section 15A.

The preprocessing unit 14 of the present embodiment further converts the divided second image portion P2 into data in a file format suitable for barcode recognition. In this embodiment, the preprocessing unit 14 converts the second image portion P2 into bitmap (bmp) format data. Such file format conversion can be appropriately changed according to the barcode recognition method. Such file format change may be performed by the second image recognition section 15B.

The preprocessing unit 14 of this embodiment further converts the divided third image portion P3 into data in a file format suitable for template matching. In this embodiment, the third image recognition unit 15C converts the third image portion P3 into bitmap (bmp) format data. Such file format conversion can be appropriately changed according to the template matching method and the type of base B master image stored in the storage unit 13 . Note that such file format change may be performed by the third image recognition section 15C.

The preprocessing unit 14 of this embodiment further performs rotation correction of the data of the third image portion P3 whose file format has been converted. For example, if the base has a rectangular shape, the rotation correction can be performed so that the long sides are parallel to a predetermined reference direction. Further, if the base has a round shape, for example, the rotation correction can be performed so that the longest straight portion included in the round shape is parallel to a predetermined reference direction. Such rotation correction may be performed by the third image recognition section 15C.

The image recognition unit 15 performs image recognition of an object included in each image portion preprocessed by the preprocessing unit 14 . The image recognition section 15 of this embodiment includes a first image recognition section 15A, a second image recognition section 15B and a third image recognition section 15C. The first image recognition section 15A, the second image recognition section 15B, and the third image recognition section 15C each perform different image recognition.

The first image recognition unit 15A performs image recognition based on character codes acquired from the first image portion P1 by optical character recognition (OCR). Specifically, first, the first image recognition unit 15A obtains text information of the character image included in the first image portion P1 whose file format has been converted by OCR. In this embodiment, text information of all character images included in the first image portion is obtained. Specifically, the text information of the character image included in the adapter A in the first image portion P1 is acquired.

FIG. 7 is a diagram showing an example of the configuration of the adapter A. As shown in FIG. As shown in FIG. 7, the main body of the adapter A is marked with a character string that can identify the adapter A. As shown in FIG.

The OCR of this embodiment is performed by an AI-OCR service (external OCR) provided by a cloud provider. Specifically, the first image recognition unit 15A transmits the data of the first image portion P1 whose file format has been converted to an external server of an AI-OCR service (external OCR) provided by a cloud provider via the Internet. Output to 5. Next, the first image recognition unit 15A acquires the result of OCR by external OCR via the Internet. Also, the OCR may be performed by the first image recognition section 15A.

The first image recognition unit 15A recognizes keyword 1 and keyword 2 as search targets based on the adapter product code specified by the input to the input unit 11 at the start of inspection work and output by the storage unit 13 based on the product code. , keyword 3 and keyword 4 are identified from the adapter conversion master information I4 stored in the storage unit 13 in advance. Then, the first image recognition unit 15A searches all text information of the obtained OCR result for each adapter A to determine whether or not the specified keyword 1, keyword 2, keyword 3, and keyword 4 are included. Identify the object (adapter A) included in the first image portion P1. Specifically, the first image recognition unit 15A recognizes the adapter A from which all keywords 1, 2, 3 and 4 corresponding to the adapter product code corresponding to the product code output by the input unit 11 have been extracted. is identified as the object (adapter A) contained in the first image portion. In the present embodiment, such identification eliminates the need to fix the range of coordinate positions to be subjected to OCR processing in the first image portion P1. Therefore, even with the adapter A in which the position of the character image, which is the keyword, differs from model to model, the keyword can be reliably extracted. Further, delicate adjustment work for arranging the position of the adapter A at the time of photographing at a determined fixed coordinate position becomes unnecessary. Therefore, the character image of the adapter A only needs to be included in the photographed image regardless of the position in the photographed image. Therefore, the arrangement of the adapter A at the photographing position by the operator can be simplified, and the inspection can be smoothly performed. becomes possible.

The accuracy of OCR is not 100%, and if the target product is a reused product, the printing of the characters to be recognized may become faint due to the use of the previous user, or if small scratches are included, OCR may not be possible. Mistranslation may occur. Therefore, in this embodiment, in addition to the master keyword, the characters included in the master keyword that are likely to be erroneously converted are stored in advance in the adapter conversion master information I4 of the storage unit 13, and are used in searches using the master keyword. In addition, a search is performed using a character string obtained by erroneously converting some characters of the master keyword as a keyword. By additionally performing such a search, the accuracy of image recognition by the first image recognition section 15A is improved.

The first image recognition unit 15A outputs information on the adapter A included in the identified first image portion P1 to the determination unit 16 as a recognition result. When the adapter A included in the first image portion P1 cannot be identified, the first image recognition section 15A outputs that effect to the determination section 16. FIG.

The second image recognition unit 15B performs image recognition based on the barcode acquired from the second image portion P2 by barcode recognition. Specifically, first, the second image recognition unit 15B converts the second image portion P2 divided by the preprocessing unit 14 into data in a file format suitable for barcode recognition. In this embodiment, the second image recognition section 15B converts the second image portion P2 into bitmap (bmp) format data. Such file format conversion can be appropriately changed according to the barcode recognition method. The second image recognition unit 15B acquires the barcode included in the second image portion P2 whose file format has been converted by barcode recognition, and performs text conversion processing on the acquired barcode. Text conversion processing can be performed with known bar code reading software. In barcode recognition, in addition to text conversion processing of the obtained barcode, specification of the barcode symbol type is performed. In the present embodiment, when a plurality of barcodes are included in the second image portion P2, text conversion processing and identification of barcode symbol types are simultaneously performed for all the included barcodes of different types. be able to. By performing such processing in the present embodiment, it becomes unnecessary to fix the coordinate position range, which is the target of the text conversion processing in the second image portion P2. This eliminates the need for delicate adjustment work for arranging the position of the modem B at the time of photographing at a determined fixed coordinate position. Therefore, the modem B can be included in the photographed image regardless of its position in the photographed image. Therefore, the arrangement of the modem B at the photographing position by the operator can be simplified, and the inspection can be performed smoothly. Become.

FIG. 8 is a diagram showing an example of the configuration of modem M. As shown in FIG. As shown in FIG. 8, the main body of the modem M is provided with a plurality of barcodes that can identify the adapter M. As shown in FIG. Modem M is assigned an individual identification number such as a unique MAC address and serial number. It is possible to extract from the resulting textual information and the specified barcode symbology type.

Specifically, the second image recognition unit 15B identifies the modem product code output by the storage unit 13 based on the product code specified by the input to the input unit 11 at the start of the inspection work, and further specifies the modem product code. Data identification key 1 and data identification key 2 are specified from the obtained modem article code from the modem master information I2 stored in the storage unit 13 in advance. Next, the second image recognition unit 15B refers to the data identification key of the barcode identification master information I3 stored in advance in the storage unit 13 based on the specified data identification key 1 and data identification key 2, At least two regular expressions of barcode symbol types and barcode contents are specified as a combination of data identification key 1 and data identification key 2 . If the modem to be inspected is not a new item but is a reused rental item, the use of the rental item before inspection by the user may make the printing of the barcode to be recognized faint, or cause the barcode to be recognized to become faint. Barcodes may not be converted to text correctly due to small scratches. In this regard, in this embodiment, the same individual identification number information is assigned to two different barcodes. Since it is possible to obtain the individual identification number by converting the information from the bar code into text, the probability of being able to read the reused product can be improved.

Next, the second image recognition unit 15B performs text conversion processing and identification of barcode symbol types for all the barcodes included in the second image portion P2. The second image recognition unit 15B determines whether or not the regular expression of the barcode content and the barcode symbol type match for each barcode with respect to the result of the text conversion process and the identification of the barcode symbol type.
If they do not match, the second image recognition unit 15B treats as if the individual identification number of the modem to be inspected is not included in the text conversion result of the bar code. If they match, the second image recognition unit 15B extracts the individual identification number of the modem to be inspected from the results of text conversion of the barcode based on the regular expression of the contents of the barcode. Based on the individual identification number, the modem article code is identified from the individual identification number master information I6, and the object (modem M) included in the second image portion P2 is identified.

In the present embodiment, by performing such identification, even if the second image portion P2 includes a plurality of bar codes, it is possible to extract the individual identification number of the modem included in the second image portion P2. It is possible to easily identify the type of bar code symbol and data identification key required.

The second image recognition unit 15B outputs the information of the modem article code of the specified modem included in the second image portion P2 to the determination unit 16 as a recognition result. If the second image recognition section 15B cannot identify the modem product code of the modem included in the second image portion P2, the second image recognition section 15B outputs that effect to the determination section 16. FIG.

FIG. 9 is a diagram showing an example of the configuration of the base B. As shown in FIG. The third image recognition unit 15C performs image recognition based on template matching of the base B, which is the object included in the third image portion P3. Specifically, the third image recognition unit 15C generates an image by template matching processing that compares the rotation-corrected data of the third image portion P3 with all the base master images registered in the base master information I5. Recognition is performed to identify the base article code of base B and to identify the objects contained in the third image portion P3.

The third image recognition unit 15C outputs the identified base article code of the base B included in the third image portion P3 to the determination unit 16 as a recognition result. When the third image recognition unit 15C cannot specify the base B included in the third image portion P3, it outputs that fact to the determination unit 16. FIG.

The determination unit 16 performs inspection based on the recognition result of the image recognition unit 15 . The determination unit 16 of the present embodiment determines the set combination master information I1 stored in the storage unit 13 based on the recognition results of the first image recognition unit 15A, the second image recognition unit 15B, and the third image recognition unit 15C. , it is determined whether or not the combination of the adapter A, modem M and base B is correct, and the determination result is output to the display unit 17 to inform the operator.

Next, a main operation example of the inspection system 1 of this embodiment will be described with reference to the flowchart shown in FIG. Prior to the operation of the inspection system, the operator places the adapter A, modem M and base B at predetermined positions on the mounting table 33 .

In step ST1, the code of the article to be inspected is specified by the operator's input to the input unit 11, and in step ST2, the user terminal 2 acquires master information from the external terminal and/or the external server 5, Stored in the storage unit 13 . At step ST3, the operator takes an image including the adapter A, the modem M and the base B, which are inspection objects. In step ST4, the captured image is stored in the storage section 13. FIG.

Next, in step ST5, the preprocessing unit 14 preprocesses the photographed image, extracts image portions for each object from the photographed image, and outputs the extracted image to the image recognition unit 15. FIG. In step ST6, the image recognition unit 15 performs image recognition of the image portion, particularly the object included in the image portion.

At step ST7, the determination section 16 inspects whether or not the combination of the adapter A, modem M and base B is correct based on the recognition result of the image recognition section 15. FIG. If the result of the correct combination is not obtained, for example, the process returns to step ST3, the image to be recognized is photographed again, and image recognition is performed again. If the result of the correct combination cannot be obtained by re-imaging, it may be determined that the combination of adapter A, modem M and base B is incorrect (step ST8). Further, when the combination is determined to be correct as a result of the inspection (step ST8), the inspection result is output to the display section 18 and stored in the storage section 13 in step ST9.

Next, a specific operation example of the preprocessing unit and the image recognition unit in the inspection system 1 of this embodiment will be described with reference to the flowchart shown in FIG.

First, in step ST51, the preprocessing unit 14 extracts image portions P1 to P3 for each object from the photographed image.

For the first image portion P1, the file format is changed by the preprocessing unit 14 (step STA52), and the data of the first image portion P1 whose file format has been converted by the first image recognition unit 15A is sent to an external OCR service. (step STA61), OCR processing of the first image portion P1 by an external OCR service (step STA62), acquisition of OCR results from the external OCR service (step STA63), identification of adapter A by master keyword search for OCR results The processing of (step STA64) is sequentially performed, and information on the specified adapter A is output to the determination unit 16 (step STA65).

For the second image portion P2, in parallel with the processing for the first image portion P1, the file format is changed by the preprocessing section 14 (step STB52), and the file format is converted by the second image recognition section 15B. Acquisition of the barcode contained in the second image portion P2 (step STB61), text conversion processing of the acquired barcode (step STB62), individual identification number obtained based on the text conversion result, individual identification number of this result (step STB63) and identification of the modem M based on the comparison result (step STB64) are sequentially performed, and information on the identified modem M is output to the determination unit 16 (step STB63). STBs 65).

For the third image portion P3, in parallel with the processing of the first image portion P1 and the processing of the second image portion P2, the preprocessing unit 14 changed the file format (step STC52), and the file format was converted. A template matching process (step STC61) by comparing the base B included in the third image portion P3 with a master image of a plurality of bases, and a process of specifying the base B based on the template matching result (step STC62) are sequentially performed, Information on the specified base B is output to the determination unit 16 (step STC63).

The inspection system of the present embodiment can be understood as an inspection method, for example, an inspection method for inspecting at least three objects, in which an image including images of at least three objects is captured. a step of extracting an image portion of each object from the photographed image; a step of recognizing the image of the object included in the image portion for each extracted image portion; and a step of determining the inspection result.

The inspection system of this embodiment can be understood as an inspection system program. a preprocessing unit that extracts an image portion for each object from the preprocessing unit, an image recognition unit that performs image recognition of the object included in the extracted image portion for each extracted image portion, and based on the recognition result of the image recognition unit A judgment unit for judging the inspection result can be grasped as an inspection system program realized in a computer.

According to the inspection system of the above embodiment, an image portion for each object is extracted from an image taken in one shot, and image recognition of the object included in the image portion is performed for each extracted image portion. Therefore, image recognition for each object can be performed in parallel independently of image recognition for other objects. In addition, since other objects are not included in the image portion, image recognition errors due to other objects do not occur. Therefore, automatic inspection for each object can be performed accurately and at high speed, and automatic inspection of objects and their combinations can be performed accurately and at high speed.

According to the inspection system of the above-described embodiment, each object can be automatically recognized only by setting the object and photographing it. In particular, despite the large number of product types, barcodes are not attached, and there are many parts that are very similar in terms of notation and shape. It is possible to prevent the problem of variation in inspection quality caused by the skill of the operator, the work environment, the work time, etc., by inspecting the adapters, bases, and their combinations that could not be obtained.

In the inspection system of the above embodiment, image recognition to be executed according to the type of object included in the image portion is determined in advance. Therefore, image recognition of each object can be optimized, and automatic inspection can be performed accurately and at high speed.

In the inspection system of the above embodiment, since the photographed images are stored in the storage unit, it is possible to confirm whether or not the products were packed in the correct combination after shipment. can be proven with high evidence.

The present invention is not limited to the embodiments described above, and many modifications are possible within the technical idea of the present invention by those skilled in the art.

For example, in the above embodiment, the file formats of the first to third image portions are changed by the preprocessing section, but it is also possible for the first to third image recognition sections to change them. In the above embodiment, OCR processing is performed using an external server, but may be processed inside the user terminal. Further, in the above embodiment, template matching processing and barcode processing are performed inside the user terminal, but they may be processed outside the user terminal. In the above embodiment, the objects to be inspected are a combination of the adapter, modem, and base, but combinations other than these combinations may be used. Furthermore, as image recognition, we perform OCR processing, barcode processing, and template matching processing. good. In addition, although three objects are inspected in the above embodiment, the number of objects to be inspected may be any number as long as the number of objects is plural.

According to the present invention, an image portion for each object is extracted from a photographed image, and image recognition of the object included in the extracted image portion is performed for each extracted image portion. All objects in the image can be uniquely identified by recognizing the identifying information. As such, automatic inspection of at least three objects and their combinations can be performed accurately and simultaneously.

1: inspection system 2: user terminal 3: communication network 4: imaging device 5: external terminal and/or external server 6: computer processor 7: main memory 7 8: input/output I/F
9: communication I/F 10: storage 11: input unit 12: imaging unit 13: storage unit 14: preprocessing unit 15: image recognition unit 16: determination unit 17: display unit 41: camera 42: rack 43: mounting table A : Adapter M: Modem B: Base P1-P3: Image part

An inspection system of the present invention for solving the above problems is an inspection system for inspecting at least three objects, comprising: an imaging unit for imaging an image including at least three objects; a preprocessing unit that extracts each image portion; an image recognition unit that performs image recognition of an object included in each extracted image portion; and an inspection based on the recognition result of the image recognition unit. a determination unit, wherein the image recognition unit includes at least a first image recognition unit that performs image recognition based on character codes obtained from the first image portion by optical character recognition, and the first image recognition unit includes , image recognition is performed based on whether or not a plurality of keywords are included in the acquired character code, and in addition to the master keyword, the first image recognition unit recognizes characters obtained by erroneously converting part of the characters of the master keyword. It is characterized by searching columns .

An inspection method of the present invention for solving the above-mentioned problems is an inspection method for inspecting at least three objects, comprising the steps of capturing an image including at least three objects; a step of recognizing an object included in each extracted image portion; a step of determining an inspection result based on the recognition result of the image recognition unit; The step of performing image recognition performs at least first image recognition for performing image recognition based on character codes obtained from the first image portion by optical character recognition, and the first image recognition is obtained by Image recognition is performed based on whether or not a plurality of keywords are included in the character code, and in the first image recognition, in addition to the master keyword, a character string obtained by erroneously converting part of the characters of the master keyword is retrieved. It is characterized by

An inspection system program of the present invention for solving the above problems is an inspection system program for inspecting at least three objects, wherein an image portion for each object is extracted from an image containing at least three objects. an image recognition unit that performs image recognition of a target object included in each extracted image portion; a determination unit that determines an inspection result based on the recognition result of the image recognition unit; is implemented in a computer, the image recognition unit includes at least a first image recognition unit that performs image recognition based on character codes obtained from the first image portion by optical character recognition, and the first image recognition unit is , image recognition is performed based on whether or not a plurality of keywords are included in the acquired character code, and in addition to the master keyword, the first image recognition unit recognizes characters obtained by erroneously converting part of the characters of the master keyword. It is characterized by searching columns .

Claims (10)

An inspection system for inspecting at least three objects,
an imaging unit that captures an image containing at least three objects;
a preprocessing unit that extracts an image portion for each object from the captured image;
an image recognition unit that performs image recognition of an object included in each extracted image portion;
and a determination unit that performs inspection based on the recognition result of the image recognition unit. 2. The inspection system according to claim 1, wherein the preprocessing section extracts an image portion for each object from the photographed image based on the coordinate information. 3. The inspection system according to claim 1, wherein the preprocessing unit divides the photographed image into extracted image portions. The inspection system according to any one of claims 1 to 3, wherein the image recognition section performs at least three different image recognitions. The image recognition part
a first image recognition unit that performs image recognition based on character codes obtained by optical character recognition from the first image portion;
a second image recognition unit that performs image recognition based on the barcode obtained by barcode recognition from the second image portion;
a third image recognition unit that performs image recognition based on template matching of an object included in the third image portion;
The inspection system according to any one of claims 1 to 4, comprising at least: 6. The inspection system according to claim 5, wherein the first image recognition unit performs image recognition based on whether or not a plurality of keywords are included in the acquired character code. 7. The second image recognition unit according to claim 5 or 6, wherein the second image recognition unit performs image recognition based on an identification master including a barcode symbol type, a regular expression of barcode content, and a data identification key for each article data of an object. inspection system. 8. The inspection system according to any one of claims 1 to 7, further comprising a storage section for storing the image captured by the imaging section. An inspection method for inspecting at least three objects,
capturing an image including at least three objects;
extracting an image portion for each object from the captured image;
performing image recognition of an object included in each extracted image portion;
and determining an inspection result based on the recognition result of the image recognition unit. A program for an inspection system that inspects at least three objects,
a preprocessing unit for extracting an image portion for each object from an image containing at least three objects;
an image recognition unit that performs image recognition of an object included in each extracted image portion;
A program for an inspection system that realizes in a computer a determination unit that determines inspection results based on recognition results of an image recognition unit.

Images