JP2024004458A - Inspection equipment and PTP packaging machine - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide inspection equipment that can enhance inspection accuracy while applying past additional learning.

SOLUTION: An additional learning unit 52h generates a trained AI model 200 by causing a neural network of an AI model 200 to additionally learn image data related to conforming products as additional learning data. When a defective product is incorrectly determined to be a conforming product by quality determination using an active AI model 200z used for inspection, the additional learning unit 52h generates a new trained AI model 200 by causing the neural network of the AI model 200 at a stage before obtaining the active AI model 200z to learn data other than additional learning data identified as factors of erroneous determination among the additional learning data used to generate the active AI model 200z from the AI model 200.

SELECTED DRAWING: Figure 6

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an inspection device for inspecting a predetermined inspection object and a PTP packaging machine for producing a PTP sheet in which tablets are accommodated in pockets.

PTP (Press Through Pack) sheets are known as blister sheets that are generally used in the fields of pharmaceuticals, foodstuffs, and the like. The PTP sheet includes a container film having a pocket portion in which a tablet is accommodated, and a cover film attached to the container film so as to seal the opening side of the pocket portion.

Further, a predetermined inspection device may perform an appearance inspection of an inspection target such as a PTP sheet or a tablet. In recent years, inspection devices have been proposed that perform visual inspections of inspection objects using an AI model generated by training a neural network with only image data related to non-defective inspection objects as learning data (e.g. , Patent Documents 1, 2, etc.).

JP 2022-29241 Publication Patent No. 7046150

By the way, an inspection apparatus using an AI model may make an erroneous determination, such as determining a good product to be inspected as a defective product. In this case, by causing the neural network of the AI model to additionally learn the image data related to the erroneous determination, it is possible to reduce the occurrence of erroneous determinations.

However, as additional learning is repeated, overlearning occurs in the neural network, and as a result, a situation such as erroneously determining that a defective inspection object is a non-defective item (erroneous determination due to overlearning) may occur. When this kind of situation occurs, the conventional approach is to return the AI model to its initial state (for example, to factory shipment or no learning state), making it impossible to make any use of past additional learning. There is a risk.

The present invention has been made in view of the above circumstances, and its purpose is to provide an inspection device and the like that can improve inspection accuracy while making use of past additional learning.

In the following, each means suitable for solving the above object will be explained in terms of sections. Note that effects specific to the corresponding means will be added as necessary.

Means 1. An inspection device for inspecting a predetermined inspection object,
an image data acquisition means capable of acquiring image data related to the inspection target;
It is possible to output image data according to the input image data by using a neural network that has an encoding unit that extracts feature quantities from input image data and a decoding unit that reconstructs image data from the feature quantities. a trained identification means, which is the learned identification means, which is generated by causing the neural network of the configured identification means to learn only image data related to a non-defective inspection object as learning data;
additional learning means for generating a new learned identification means by causing the neural network of the identification means to additionally learn image data related to a non-defective inspection object as additional learning data;
Reconstructed image data acquisition means capable of inputting original image data, which is image data obtained by the image data acquisition means, into the learned identification means and acquiring reconstructed image data as reconstructed image data;
a determining unit capable of comparing the original image data and the reconstructed image data obtained by inputting the original image data to the learned identifying unit, and determining the quality of the inspection target based on the comparison result;
identification means storage means for storing the old identification means, which is the identification means at a stage where some or all of the additional learning by the additional learning means has not been performed;
Specific learning data that stores all of the specific learning data that is the additional learning data that was used to generate the current identification means that is the learned identification means that is used for inspecting the inspection object from the old identification means. and storage means;
The additional learning means inputs the original image data into the current identification means by the reconstructed image data acquisition means, and performs a quality determination by the determination means based on the reconstructed image data, thereby inspecting defective products. When the target object is erroneously determined to be a good product, the neural network of the old identification means stored in the identification means storage means receives the erroneous judgment which is the additional learning data identified as the cause of the erroneous judgment. An inspection device characterized in that it is configured to be able to generate a new learned identification means by learning the specific learning data excluding factor data.

In addition, the "image data related to non-defective inspection objects" used as "learning data" in the above means 1 may include image data related to non-defective inspection objects accumulated from previous inspections, For example, image data related to inspection objects of non-defective products selected by the method, virtual non-defective image data generated using these image data, etc. can be mentioned.

In addition, the "trained identification means" in the above means 1 is generated by learning only the image data related to non-defective inspection objects. The reconstructed image data generated when input to the identification means almost matches the original image data from which noise portions (defective portions) have been removed. That is, when there is a defective part in the object to be inspected, virtual image data regarding the object to be inspected, assuming that there is no defective part, is generated as reconstructed image data.

Furthermore, the above-mentioned "neural network" includes, for example, a convolutional neural network having a plurality of convolutional layers. The above "learning" includes, for example, deep learning. The above-mentioned "identification means (generation model)" includes, for example, an autoencoder (self-encoder), a convolutional autoencoder (convolutional self-encoder), etc. (the same applies to the means 6).

Furthermore, the "old identification means" includes not only identification means for which neural network learning has been performed, but also identification means for which the neural network has not been trained (the same applies to means 6).

According to the above means 1, the original image data is compared with the reconstructed image data that has been reconstructed by inputting the original image data to the learned identification means, and based on the comparison result, the quality of the inspection object is determined. can be determined. Therefore, both image data to be compared relate to the same inspection object. Therefore, since the shape and appearance of the inspection target are almost the same in both image data to be compared, there is no need to set relatively loose judgment conditions to prevent false detection due to differences in shape and appearance. Strict judgment conditions can be set. Furthermore, the conditions for obtaining the image data (for example, the placement position and placement angle of the inspection object with respect to the camera, the brightness and darkness conditions, the viewing angle of the camera, etc.) can be matched in both image data to be compared. As a result, pass/fail judgments can be made with high accuracy.

Further, according to the above means 1, when the defective inspection object is erroneously determined to be a non-defective product by the pass/fail judgment based on the reconstructed image data that is reconstructed by inputting the original image data to the current identification means. In other words, if it is thought that a misjudgment has occurred due to overlearning, the additional learning means is a discrimination means at a stage where some or all of the additional learning has not been performed (a discrimination means at a stage before the current discrimination means). Learning is performed on the neural network of the old identification means (for example, an initial AI model) to generate a new learned identification means. At this time, the additional learning means informs the neural network of the old identification means of the specific learning data that is the additional learning data used to generate the current identification means from the old identification means. Train data other than the additional training data. Therefore, in the newly generated learned identification means, the learning state can be corrected so as to more reliably suppress the occurrence of erroneous determinations due to overlearning while making use of past additional learning data. As a result, it is possible to more reliably prevent a defective inspection object from being erroneously determined to be a non-defective item, and it is possible to improve inspection accuracy.

In addition, the additional learning means performs learning on the neural network of the old identification means stored in the identification means storage means, rather than on the neural network of the current identification means used in the test. Therefore, processing related to learning of a neural network and processing related to extraction (identification) of data causing false judgments, which can be a hugely repetitive process, can be performed without stopping the test. Thereby, inspection of a large number of inspection objects can be performed quickly and efficiently.

Means 2. The identification means storage means stores, separately from the current identification means, a duplicate current identification means which is the identification means having the same content as the current identification means,
The additional learning means inputs the original image data into the current identification means by the reconstructed image data acquisition means, and determines whether the inspection object is a non-defective product based on the quality judgment of the judgment means based on the reconstructed image data. When an object is erroneously determined to be defective, the neural network of the duplicate current identification means stored in the identification means storage means is additionally trained with the original image data that caused the erroneous determination. , the inspection device according to means 1, characterized in that it is configured to be able to generate a new learned identification means.

According to the above means 2, when a non-defective inspection object is erroneously determined to be a defective item, the neural network of the duplicate currently used identification means stored in the identification means storage means is asked to identify the source that caused the erroneous judgment. By additionally learning image data, it is possible to generate a new learned identification means. Therefore, it is possible to generate a learned identification means that more reliably prevents a good inspection object from being erroneously determined to be a defective product, and it is possible to further improve inspection accuracy.

Further, according to the above-mentioned means 2, since the duplicate current identification means stored separately from the current identification means used in the test is subjected to additional learning, there is no need to stop the test for additional learning. . Therefore, the speed and efficiency of testing can be further improved.

Means 3. a defective image data storage means for storing defective image data that is the original image data related to the erroneous determination when the defective inspection object is erroneously determined to be a good product by the quality determination of the determination means;
2. The inspection device according to means 1, further comprising factor data specifying means for specifying the erroneous determination factor data from the specific learning data based on the degree of similarity to the defective image data.

Note that the factor data specifying means includes, for example, similarity between specific learning data and defective image data (e.g., similarity between both images based on the feature amount extracted from the defective image data and the feature amount extracted from the specific learning data). It has a publicly known similarity determination means (for example, an AI generated by training a neural network with at least defective image data) that can calculate a score indicating the degree of similarity of data), and the calculated similarity and a predetermined An example of this method is one that identifies data that causes erroneous determination based on a comparison result with a threshold value.

According to the above means 3, based on the degree of similarity to the defective image data that is the original image data related to the misjudgment (the original image data that caused the misjudgment), the misjudgment factor data (the misjudgment cause data) is selected from the specific learning data. Additional learning data that may be the cause can be identified. Therefore, the erroneous determination factor data can be specified more accurately, and in turn, the accuracy of the test using the newly generated learned identification means can be more reliably increased.

Means 4. Based on the reconstructed image data that was reconstructed by inputting the original image data related to the erroneous judgment that triggered the generation of the learned discrimination means to the newly generated learned discrimination means, The inspection apparatus according to the means 1, further comprising a confirmation means for image output AI that confirms the learning state of the newly generated learned identification means by making the judgment means perform a quality judgment.

According to the above-mentioned means 4, whether or not an erroneous judgment based on the original image data related to the erroneous judgment that triggered the generation of the learned identification means occurs in the newly generated learned identification means, that is, It is possible to easily confirm whether better inspection accuracy has been achieved. In addition, if it is confirmed that a similar misjudgment occurs even in a test using a newly generated trained discriminator, we will take measures to deal with such misjudgment [re-identifying the misjudgment factor data using another method]. At the same time, re-learning after removing the re-identified false judgment factor data, measures other than learning (for example, reviewing the threshold used in the stage of comparing the original image data and reconstructed image data), etc. ) can be easily performed.

Means 5. a display means capable of displaying information related to learning processing by the additional learning means and confirmation results by the image output AI confirmation means;
4. The inspection device according to means 4, further comprising: approval input receiving means capable of accepting input related to approval for replacing the currently used identification means with the newly generated learned identification means.

According to the above means 5, it is possible to more reliably prevent unintentional replacement of the currently used identification means used in the test with the newly generated learned identification means. This makes it possible to more reliably prevent problems caused by unintended replacement.

Means 6. An inspection device for inspecting a predetermined inspection object,
an image data acquisition means capable of acquiring image data related to the inspection target;
The neural network of the identification means, which is configured to classify input image data according to predetermined items and output the classification results, is trained by using image data related to inspected objects for good and defective products as learning data. a learned identification means that is the learned identification means that is generated;
additional learning means for generating a new learned identification means by causing the neural network of the identification means to additionally learn image data related to the inspection object as additional learning data;
The old identification means, which is the identification means at a stage where some or all of the additional learning by the additional learning means has not been performed, and the current identification means, which is the learned identification means, which is used for the inspection of the inspection object. Separately, an identification means storage means for storing a duplicate current identification means which is the identification means having the same content as the current identification means;
specific learning data storage means for storing all specific learning data that is the additional learning data used in generating the current identification means from the old identification means;
The additional learning means is configured to control the neural network of the duplicate current identification means stored in the identification means storage means when an erroneous determination occurs in the classification result output by inputting the original image data to the current identification means. , it is possible to generate a new learned identification means by additionally learning the original image data that caused the misjudgment,
Based on the classification result output by inputting the original image data related to the erroneous judgment that triggered the generation of the learned identification means to the newly generated learned identification means, If the classification output AI confirmation means for confirming the learning state of the learned identification means determines that the learning state is appropriate, the currently used identification means is replaced with the newly generated learned identification means. While it is determined that it can be replaced with
An erroneous judgment occurs in the classification result outputted by inputting the original image data to the newly generated trained identification means, and the learning state is determined to be inappropriate by the classification output AI confirmation means. If the neural network of the old identification means stored in the identification means storage means is data related to the inspection object with the same classification result as the incorrectly determined classification result, and An inspection device capable of generating a new learned identification means by learning the specific learning data excluding erroneous judgment factor data that is data specified as a factor of erroneous judgment.

Note that the "image data related to the inspection object" used as "learning data" in the above means 6 includes not only image data related to the inspection object which is a good product, but also image data related to a defective inspection object. It will be done. In addition, "image data related to the object to be inspected" may include image data obtained during inspection, virtual image data generated based on the image data, and the like.

Furthermore, the learned identification means may classify the inspection object according to its quality, or may classify according to the type of defect in the inspection object in addition to the quality of the inspection object.

According to the above means 6, when an erroneous judgment regarding the classification result of the inspection object occurs, the original image that caused the erroneous judgment is sent to the neural network of the duplicate current identification means stored in the identification means storage means. By additionally learning data, it is possible to generate a new learned identification means. Therefore, it is possible to obtain a learned identification means with better inspection accuracy.

Further, during additional learning, learning is performed not on the neural network of the current identification means used in the test, but on the neural network of the duplicate current identification means stored in the identification means storage means. Thereby, additional learning can be performed without stopping the test, and the speed and efficiency of the test can be further improved.

In addition, according to the above means 6, the confirmation means for the classification output AI determines whether or not a second erroneous judgment based on the original image data that caused the erroneous judgment will occur in the newly generated learned identification means. In other words, it can be easily confirmed whether or not better inspection accuracy is achieved.

Furthermore, the classification output AI cannot be used for inspection of the inspection object until it has been confirmed that it is in a state where no erroneous judgments will occur (a state in which better inspection accuracy is achieved) using the confirmation means for the classification output AI. It is determined that the currently used identification means can be replaced with the newly generated learned identification means. Therefore, it is possible to more reliably prevent inappropriate learned identification means that may cause erroneous judgments from being used in tests.

On the other hand, if it is confirmed by the classification output AI confirmation means that an erroneous judgment occurs in a test using a newly generated learned discriminator, that is, if it is thought that an erroneous judgment has occurred due to overlearning, The additional learning means performs learning on the neural network of the old classification means (for example, the initial AI model), which is a classification means at a stage where additional learning has not been performed partially or completely (classification means at a stage before the current classification means). and generate a new learned identification means. At this time, among the specific learning data that is additional learning data used to generate the current identification means from the old identification means, data regarding the inspection object with the same classification result as the incorrect classification result is selected. Based on this, data that is considered to be the cause of the misjudgment is identified as misjudgment factor data.

For example, suppose that it is confirmed that a non-defective item is incorrectly classified as a defective item during an inspection using a newly generated learned identification means. In this case, the misjudgment factor data is specified from among the "data related to the defective inspection object" of the specific learning data. Further, for example, suppose that in an inspection using a newly generated learned identification means, it is confirmed that the type of defect occurring in the object to be inspected is incorrectly determined to be type Y when it should be determined to be type X. In this case, the misjudgment factor data is specified from among the specific learning data, "data related to the inspection object in which a defect of type Y has occurred." This means that when a misjudgment occurs in a classification result, among the data used for additional learning (specific learning data), image data related to the same classification result as the misjudged classification result will cause the misjudgment. This is because it is thought to be a factor.

Then, the additional learning means generates a new learned identification means by causing the neural network of the old identification means to learn data obtained by excluding the false determination factor data from the specific learning data. Therefore, when the learning state is determined to be inappropriate by the confirmation means for classification output AI, it is possible to more reliably suppress the occurrence of false judgments due to overfitting while making use of past additional learning data. You can modify the learning state. As a result, it is possible to more reliably prevent erroneous determination regarding the classification of the object to be inspected, and improve inspection accuracy.

In addition, if the learning state is determined to be inappropriate by the classification output AI confirmation means, the old identification stored in the identification means storage means is used instead of the neural network of the current identification means used for testing. Learning is performed on the neural network of the means. Therefore, processing related to learning of a neural network and processing related to extraction (identification) of data causing false judgments, which can be a hugely repetitive process, can be performed without stopping the test. This allows inspection of a large number of inspection objects to be performed more quickly and more efficiently.

Means 7. An erroneous judgment occurred in the classification result output by inputting the original image data to the newly generated trained identification means, and the learning state was determined to be inappropriate by the classification output AI confirmation means. a misjudgment occurrence image data storage means for storing misjudgment occurrence image data that is the original image data related to the misjudgment in the case;
7. The inspection device according to means 6, further comprising factor data specifying means for specifying the misjudgment factor data from the specific learning data based on the degree of similarity to the image data in which misjudgment has occurred.

Note that the factor data specifying means may be based on, for example, the degree of similarity of the specific learning data to the image data where misjudgment has occurred (for example, the similarity between the feature amount extracted from the image data where misjudgment has occurred and the feature amount extracted from the specific learning data). It has a publicly known similarity determination means (for example, an AI generated by having a neural network learn at least the image data in which misjudgment has occurred) capable of calculating a score indicating the degree of similarity between both image data based on the image data. An example of this method is one in which erroneous determination factor data is specified based on a comparison result between the degree of similarity determined and a predetermined threshold value.

According to the means 7, when it is confirmed by the classification output AI confirmation means that an erroneous judgment occurs in the learned identification means, the original image data related to the erroneous judgment (original image data that caused the erroneous judgment ), misjudgment factor data (additional learning data that is considered to be a cause of misjudgment) can be specified from among the specific learning data. Therefore, the erroneous determination factor data can be specified more accurately, and in turn, the accuracy of the test using the newly generated learned identification means can be more reliably increased.

Means 8. a display means capable of displaying information related to learning processing by the additional learning means and confirmation results by the classification output AI confirmation means;
7. The inspection device according to means 6, further comprising: approval input receiving means capable of accepting input related to approval for replacing the currently used identification means with the newly generated learned identification means.

According to the above means 8, the same effect as the above means 5 is achieved.

Means 9. A PTP packaging machine equipped with the inspection device according to means 1 or 6, which inspects a tablet or a PTP sheet as an object to be inspected.

According to the above-mentioned means 9, the same effects as the above-mentioned means 1 or 6 can be achieved.

Note that the technical matters related to the above means may be combined as appropriate. For example, the technical matter related to the above-mentioned means 2 may be combined with at least one of the technical matters related to the above-mentioned means 3 to 5. Furthermore, the technical matters concerning means 7 above may be combined with the technical matters concerning means 8 above.

It is a perspective view of a PTP sheet. It is a partially enlarged sectional view of a PTP sheet. It is a perspective view of a PTP film. It is a schematic block diagram of the inspection device in a 1st embodiment. It is a schematic block diagram of the PTP packaging machine in a 1st embodiment. It is a block diagram showing the functional composition of the inspection device in a 1st embodiment. FIG. 2 is a partially cutaway front view showing a schematic configuration of a pocket forming device and a heating device. It is a flowchart which shows the flow of a pocket part molding process. FIG. 2 is a schematic diagram for explaining the structure of a neural network in the first embodiment. It is a flowchart which shows the flow of learning processing of a neural network in a 1st embodiment. It is a flowchart which shows the flow of quality determination processing. It is a flowchart which shows the flow of additional learning processing in a 1st embodiment. It is a flowchart which shows the flow of confirmation processing in a 1st embodiment. It is a flowchart which shows the flow of additional learning modification processing in a 1st embodiment. 5 is a flowchart showing the flow of various processes in the first embodiment. FIG. 7 is a diagram showing a shading pattern etc. that occurs in a pocket portion without molding defects. 17 is a graph showing the luminance value of each pixel along line AA in FIG. 16. FIG. 7 is a diagram showing shading patterns etc. that occur in a pocket portion with molding defects. 19 is a graph showing the luminance value of each pixel along line BB in FIG. 18. FIG. 7 is a diagram illustrating sequentially generated AI models, additional learning data used in additional learning processing, and the like in the first embodiment. It is a schematic block diagram of the PTP packaging machine in 2nd Embodiment. It is a block diagram showing the functional composition of the inspection device in a 2nd embodiment. It is a flowchart which shows the flow of learning processing of a neural network in a 2nd embodiment. It is a flowchart which shows the flow of additional learning processing in a 2nd embodiment. It is a flowchart which shows the flow of confirmation processing in a 2nd embodiment. It is a flowchart which shows the flow of additional learning modification processing in a 2nd embodiment. 7 is a flowchart showing the flow of various processes in the second embodiment. FIG. 7 is a diagram illustrating sequentially generated AI models, additional learning data used in additional learning processing, and the like in the second embodiment. FIG. 7 is a diagram illustrating an AI model stored in an AI storage unit, additional learning data stored in an image data storage unit, and the like in another embodiment. FIG. 3 is a perspective view of a blister pack according to another embodiment.

Embodiments will be described below with reference to the drawings.
[First embodiment]
First, the PTP sheet 1 as the "inspection object" will be explained. As shown in FIGS. 1 and 2, the PTP sheet 1 includes a container film 3 having a plurality of pockets 2, and a cover film 4 attached to the container film 3 so as to close the pockets 2. ing.

The container film 3 is made of a colorless and transparent thermoplastic resin material such as PP (polypropylene) or PVC (polyvinyl chloride), and has translucency. On the other hand, the cover film 4 is made of an opaque material (for example, aluminum foil, etc.) on the surface of which a sealant made of, for example, polypropylene resin or the like is provided.

The PTP sheet 1 is formed into a substantially rectangular shape when viewed from above. In the PTP sheet 1, two rows of pockets are formed in the transverse direction, each consisting of five pockets 2 arranged along the longitudinal direction of the PTP sheet 1. That is, a total of ten pocket portions 2 are formed. Each pocket portion 2 accommodates one tablet 5 as a content.

The pocket portion 2 includes a bottom portion 2a having a substantially circular shape in plan view and arranged to face the cover film 4, and a film flat portion (non-pocket molded portion) 3b connected to the periphery of the bottom portion 2a. and a substantially cylindrical side portion 2b connecting the two.

Although the bottom portion 2a in this embodiment is formed to have a gently curved approximately arc-shaped cross section, the present invention is not limited to this, and the bottom portion 2a may be formed into a flat shape. Alternatively, a configuration may be adopted in which the corner portion 2c where the bottom portion 2a and the side portion 2b intersect is not clearly formed, and the cross section is formed into an arcuate shape with a larger curvature.

The PTP sheet 1 is manufactured by punching out a strip-shaped PTP film 6 (see FIG. 3) formed from a strip-shaped container film 3 and a strip-shaped cover film 4.

Next, a schematic configuration of the PTP packaging machine 11 that manufactures the PTP sheet 1 will be described with reference to FIG. 5.

On the most upstream side of the PTP packaging machine 11, the original fabric of the strip-shaped container film 3 is wound into a roll. The drawn-out end side of the container film 3 wound into a roll is guided by a guide roll 13. The container film 3 is hung on an intermittent feed roll 14 on the downstream side of the guide roll 13. The intermittent feed roll 14 is connected to a motor that rotates intermittently, and conveys the container film 3 intermittently.

Between the guide roll 13 and the intermittent feed roll 14, a heating device 15 and a pocket forming device 16 are arranged in order along the conveyance path of the container film 3. In a state where the container film 3 is heated by the heating device 15 and becomes relatively flexible, a plurality of pocket portions 2 are formed at predetermined positions on the container film 3 at once by the pocket portion forming device 16. Note that the formation of the pocket portion 2 is performed during an interval between the conveyance operations of the container film 3 by the intermittent feed roll 14. The configurations of the heating device 15 and the pocket forming device 16 will be described in detail later.

Further, an inspection device 21 is disposed between the guide roll 13 and the intermittent feed roll 14 and downstream of the pocket forming device 16. The inspection device 21 is for inspecting the PTP sheet 1 including the pocket portions 2 by inspecting the molding state of the pocket portions 2 formed by the pocket portion forming device 16. The configuration of the inspection device 21 will be detailed later.

The container film 3 sent out from the intermittent feed roll 14 is hung over a tension roll 18, a guide roll 19, and a film receiving roll 20 in this order.

Since the film receiving roll 20 is connected to a motor that rotates at a constant rate, it conveys the container film 3 continuously at a constant speed. The tension roll 18 is in a state in which the container film 3 is pulled to the tension side by elastic force, and prevents the container film 3 from loosening due to the difference in conveying operation between the intermittent feed roll 14 and the film receiving roll 20, and tightens the container film 3. The film 3 is kept under tension at all times.

A tablet filling device 22 is disposed between the guide roll 19 and the film receiving roll 20 along the conveyance path of the container film 3.

The tablet filling device 22 has a function as a filling means for automatically filling the pocket portion 2 with the tablet 5. The tablet filling device 22 drops the tablets 5 by opening a shutter at predetermined intervals in synchronization with the conveyance operation of the container film 3 by the film receiving roll 20. 2 is filled with tablets 5. The operation of the tablet filling device 22 is controlled by a filling control device 82, which will be described later.

On the other hand, the original fabric of the cover film 4 formed into a band shape is wound into a roll shape on the most upstream side. The drawn-out end of the cover film 4 wound into a roll is guided toward a heating roll 25 by a guide roll 24 . The heating roll 25 can be brought into pressure contact with the film receiving roll 20, so that the container film 3 and the cover film 4 are fed between both rolls 20 and 25. Then, the container film 3 and the cover film 4 pass between the rolls 20 and 25 in a heated and pressurized state, so that the cover film 4 is stuck to the container film 3 and the pocket part 2 is closed with the cover film 4. . As a result, a PTP film 6 in which a tablet 5 is housed in each pocket portion 2 is manufactured.

The PTP film 6 sent out from the film receiving roll 20 is hung on a tension roll 27 and an intermittent feed roll 28 in this order.

Since the intermittent feed roll 28 is connected to a motor that rotates intermittently, the PTP film 6 is intermittently conveyed. The tension roll 27 is in a state in which the PTP film 6 is pulled to the tension side by elastic force, and prevents the PTP film 6 from loosening due to the difference in conveyance operation between the film receiving roll 20 and the intermittent feed roll 28, and thereby The film 6 is kept under tension at all times.

The PTP film 6 sent out from the intermittent feed roll 28 is hung on a tension roll 31 and an intermittent feed roll 32 in this order.

Since the intermittent feed roll 32 is connected to a motor that rotates intermittently, the PTP film 6 is intermittently conveyed. The tension roll 31 is in a state where the PTP film 6 is pulled toward the tension side by elastic force, and prevents the PTP film 6 from loosening between the intermittent feed rolls 28 and 32.

Between the intermittent feed roll 28 and the tension roll 31, a slit forming device 33 and a marking device 34 are arranged in this order along the conveyance path of the PTP film 6. The slit forming device 33 has a function of forming a separation slit at a predetermined position of the PTP film 6. The marking device 34 has a function of marking a predetermined position (for example, a tag portion) of the PTP film 6. Note that in FIG. 1 and the like, illustrations of the separation slit and the markings are omitted.

The PTP film 6 sent out from the intermittent feed roll 32 is hung on a tension roll 35 and a continuous feed roll 36 in that order on the downstream side thereof.

A sheet punching device 37 is disposed between the intermittent feed roll 32 and the tension roll 35 along the conveyance path of the PTP film 6. The sheet punching device 37 has a function as a sheet punching means (separation means) for punching out the outer edge of the PTP film 6 in units of PTP sheets.

The PTP sheet 1 punched by the sheet punching device 37 is conveyed by a conveyor 38 and temporarily stored in a hopper 39 for finished products. However, when a defective product signal is input from a filling control device 82 (described later) to a defective sheet discharging mechanism 40 that can selectively discharge PTP sheets 1, the defective PTP sheet 1 is separately ejected by the defective sheet discharging mechanism 40. It is discharged and transferred to a defective product hopper (not shown).

A cutting device 41 is disposed downstream of the continuous feed roll 36. The scrap portion 42 remaining in a band shape after punching by the sheet punching device 37 is guided by the tension roll 35 and the continuous feed roll 36, and then guided to the cutting device 41. The cutting device 41 has a function of cutting the scrap portion 42 into predetermined dimensions. The cut scrap portion 42 is stored in a scrap hopper 43 and then disposed of separately.

Next, the configurations of the heating device 15 and the pocket forming device 16 will be explained with reference to FIG.

The heating device 15 includes an upper heater plate 15a and a lower heater plate 15b. Both heater plates 15a and 15b are configured to be heated by a heater (not shown). Both heater plates 15a and 15b are provided so as to sandwich the conveyance path of the container film 3, and are movable in the direction toward or away from the container film 3, respectively.

Each heater plate 15a, 15b is provided with a plurality of protrusions 15c, 15d at positions corresponding to the portions 3a of the pocket portion 2 to be formed in the container film 3.

The container film 3 that is intermittently conveyed is partially (spotly) heated by being sandwiched between the protrusions 15c and 15d as the heater plates 15a and 15b move toward each other during the temporary stop. The affected part becomes softened. In this embodiment, the contact portions of the protrusions 15c and 15d with the container film 3 are made one size smaller than the planar shape of the pocket portion 2.

The pocket molding device 16 includes a lower mold 61 and an upper mold 71. The lower die 61 is provided with a plurality of insertion holes 64 at positions corresponding to the positions of the pocket portions 2. Further, the lower mold 61 is fixed to a fixed support base 63 via a cylindrical lower mold chamber 62. A plurality of through holes are formed in the support base 63, and a rod-shaped slider 65 is inserted through the through holes via a bearing mechanism. The slider 65 can be moved up and down by a cam mechanism (not shown).

A pocket molding die 66 is fixed to the upper part of the slider 65, and the pocket molding die 66 includes a plurality of rod-shaped plugs 66a that can be inserted into the insertion holes 64. The tip of the plug 66a has a shape corresponding to the inner surface of the pocket portion 2. The pocket mold 66 moves up and down in accordance with the up and down movement of the slider 65 driven by the cam mechanism. Note that the lower mold 61, the pocket molding mold 66, etc. can be replaced as appropriate depending on the type of PTP sheet 1 to be produced.

Further, a circulation path 67 for circulating cooling water (or hot water) is formed inside each of the slider 65 and the pocket mold 66. This suppresses variations in surface temperature of each plug 66a.

During molding of the pocket portion 2, the plug 66a is arranged in this order to an initial position, an intermediate stop position, and a protruding position, and finally returns to the initial position. The initial position is the position where the plug 66a is placed at the start of the molding process of the pocket portion 2, and the plug 66a placed at this position is located below the insertion hole 64 and outside the insertion hole 64. Become. The intermediate stop position is a position where the plug 66a is placed at an intermediate stage of the molding process of the pocket portion 2, and the plug 66a placed at this position is placed in the insertion hole 64 and is inserted between the container film 3 and the plug 66a. A predetermined gap is formed. The protruding position is the position where the plug 66a is placed at the final stage of the molding process of the pocket part 2, and the tip surface of the plug 66a placed at this position is lowered by an amount corresponding to the depth of the pocket part 2. It will be in a state where it protrudes from the mold 61. Incidentally, such operation of the plug 66a is controlled by a molding control device 81, which will be described later.

On the other hand, the upper mold 71 is fixed to an upper plate 73 which is movable up and down via a plate 72, and is movable along the direction toward or away from the lower mold 61. The upper mold 71 is provided with a gas supply hole 74 at a position facing the insertion hole 64 of the lower mold 61.

Furthermore, a gas supply path 75 communicating with the gas supply hole 74 is formed inside the plate 72 and the upper plate 73, and a gas supply device 76 configured with a compressor or the like is connected to the gas supply path 75. , a predetermined high-pressure gas (inert gas, air in this embodiment) is supplied.

In this embodiment, a total of ten pockets 2 corresponding to one PTP sheet 1 are simultaneously molded by one operation of the pocket molding device 16. That is, five pockets 2 are formed in the film width direction (Y direction) of the container film 3 and two pockets 2 are formed in the film transport direction (X direction) at the same time.

Next, returning to FIG. 5, the forming control device 81 for controlling the forming of the pocket portion 2 by the heating device 15 and the pocket portion forming device 16 will be described. The molding control device 81 is configured by a computer system having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The forming control device 81 stores various information regarding the initial position, intermediate stop position, and protrusion position of the plug 66a, and controls the operation of the plug 66a based on this information. Note that the information regarding the initial position, intermediate stop position, and protrusion position of the plug 66a is changed as appropriate depending on the depth of the pocket portion 2 in the PTP sheet 1 to be manufactured.

Here, the pocket portion molding process executed under the control of the molding control device 81 will be described with reference to FIG. 8.

In the pocket portion forming process, first, an intermediate stop position arranging process of step S101 is performed. In the intermediate stop position arrangement step, the pocket molding die 66 is moved upward by the movement of the slider 65, thereby moving the plug 66a placed at the initial position upward.

When the plug 66a reaches the set intermediate stop position, the movement of the slider 65 is stopped, and the plug 66a is placed at the intermediate stop position. At this time, the tip end surface of the plug 66a is separated from the container film 3 by a predetermined distance. This predetermined distance is usually smaller than the depth of the pocket portion 2.

Next, in the clamping step of step S102, the upper mold 71 is moved downward, so that the container film 3 is sandwiched between the lower mold 61 and the upper mold 71, which are in a fixed state. At this time, the annular portion of the container film 3 located around the portion to be formed 3a (see FIG. 7), which will become the pocket portion 2, is held between the molds 61 and 71. Note that the intermediate stop position arranging step and the clamping step may be performed at the same time, or the clamping step may be performed before the intermediate stop position arranging step.

In the subsequent expansion step of step S103, by supplying gas from the gas supply device 76 to the gas supply hole 74 via the gas supply path 75, the molding portion 3a of the pocket portion 2 in the container film 3 is expanded. Blow compressed air from the front side (upper side in Figure 7). By supplying the gas, the portion to be formed 3a bulges to the side (lower side in FIG. 7) opposite to the protruding side of the pocket portion 2 (upper side in FIG. 7), and is stretched and becomes thinner.

Then, the portion to be formed 3a bulges out until it is supported by the tip end surface of the plug 66a. When the portion 3a to be formed is expanded by supplying gas, the thickness of the portion 3a to be formed after the expansion is approximately the same overall.

Note that depending on the intermediate stop position of the plug 66a, the amount of stretching of the container film 3 changes, and the thickness of the portion to be formed 3a also changes. When the intermediate stop position of the plug 66a is relatively high, the amount of stretching of the container film 3 is relatively small, so that the portion to be formed 3a becomes thick as a whole. On the other hand, when the intermediate stop position of the plug 66a is relatively low, the amount of stretching of the container film 3 becomes relatively large, so that the portion to be formed 3a becomes thin as a whole.

In the final molding process of step S104, the plug 66a moves upward and is placed in the protruding position. As a result, the direction of the bulge in the portion to be formed 3a is reversed, and the pocket portion 2 having a predetermined depth is formed.

Note that when the container film 3 is deformed by pressing, the portion of the portion to be formed 3a corresponding to the bottom portion 2a comes into contact with the plug 66a and is cooled, so that the portion corresponding to the bottom portion 2a is hardly stretched. Therefore, if the portion 3a to be formed is made thick overall by setting the intermediate stop position relatively high, the portion corresponding to the bottom portion 2a will be kept thick when pressed by the plug 66a, and as a result, In addition, the side portion 2b of the pocket portion 2 to be molded is relatively thin.

On the other hand, if the portion to be formed 3a is made thin as a whole by setting the intermediate stop position relatively low, the portion corresponding to the bottom portion 2a will be kept thin when pressed by the plug 66a. In addition, the side portion 2b of the pocket portion 2 to be molded is relatively thick.

In this way, by adjusting the intermediate stop position of the plug 66a and adjusting the thickness of the portion 3a to be formed, the balance between the thicknesses of the bottom portion 2a and side portions 2b of the pocket portion 2 to be finally molded can be adjusted. It becomes possible to do so.

After the final molding step, the plug 66a is placed in the initial position and the container film 3 is released from being held between the molds 61 and 71, thereby completing the pocket molding step.

Next, the filling control device 82 for controlling the filling of tablets 5 by the tablet filling device 22 will be described. The filling control device 82 is configured by a computer system including a CPU, RAM, and the like.

The filling control device 82 can switch and control whether or not to fill a predetermined pocket portion 2 with a tablet 5 based on the inspection result by the inspection device 21 . Specifically, the filling control device 82 receives the quality determination result regarding the predetermined PTP sheet 1 (the molded state of the 10 pocket parts 2) from the inspection device 21, and the quality determination result is a "good product" determination. In this case, the tablet filling device 22 is controlled to fill all ten pockets 2 included in the PTP sheet 1 with tablets 5.

On the other hand, when the quality determination result regarding the predetermined PTP sheet 1 is a "defective product" determination, the filling control device 82 does not fill the tablets 5 into all ten pockets 2 included in the PTP sheet 1. The tablet filling device 22 is controlled as follows. Furthermore, the filling control device 82 outputs a defective product signal to the defective sheet discharge mechanism 40. As a result, the defective sheet discharge mechanism 40 discharges the PTP sheet 1 (defective sheet) associated with the defective product signal.

Next, the configuration of the inspection device 21 will be explained in detail. As shown in FIGS. 4 to 6, the inspection device 21 includes a lighting device 50, a camera 51, and an inspection control device 52.

The illumination device 50 irradiates a predetermined electromagnetic wave onto a predetermined range of the container film 3 from the protruding side of the pocket portion 2 (lower side in FIG. 4). The illumination device 50 includes an electromagnetic wave irradiation device 50a and a diffusion plate 50b that covers the electromagnetic wave irradiation device 50a, and is configured to be capable of surface emission. The lighting device 50 in this embodiment irradiates the container film 3 with electromagnetic waves containing ultraviolet light.

The camera 51 is sensitive to the wavelength range of electromagnetic waves irradiated from the illumination device 50. The camera 51 is provided on the opening side of the pocket portion 2 of the container film 3 (upper side in FIG. 4), and the optical axis OL of its lens is along the vertical direction (Z direction) perpendicular to the film flat portion 3b of the container film 3. It is arranged like this.

Further, a bandpass filter 51a is provided corresponding to the lens of the camera 51. By providing the bandpass filter 51a, only the ultraviolet light that has passed through the container film 3 out of the electromagnetic waves emitted from the illumination device 50 is two-dimensionally imaged by the camera 51. Further, the transmission image data acquired by the camera 51 in this manner becomes luminance image data in which each pixel (each coordinate position) has different luminance based on the difference in the transmittance of ultraviolet light in the container film 3.

Particularly in this embodiment, the bandpass filter 51a is a filter that passes only ultraviolet light having a wavelength of 253±20 nm, for example, at which the transmittance of the container film 3 is approximately 30±10%. This means that even if the transmittance of electromagnetic waves passing through the container film 3 is too high or too low, there is a risk that a difference in light transmittance between the thin and thick parts of the bottom part 2a of the pocket part 2 will hardly occur. It's for a reason.

The imaging range of the camera 51 in this embodiment is at least a total of 10 pockets 2 corresponding to one PTP sheet 1 formed into the container film 3 in one operation of the pocket forming device 16. It is set to image the included range at once.

The inspection control device 52 includes a CPU that executes predetermined arithmetic processing, a ROM that stores various programs and fixed value data, a RAM that temporarily stores various data when executing various arithmetic processing, and peripheral circuits thereof. It consists of a computer system that includes:

The inspection control device 52 has a main control section 52a, a lighting control section 52b, a camera control section 52c, an image capture section 52d, an image processing section 52e, and a reconstructed image data acquisition section, which will be described later, by the CPU operating according to various programs. 52f, a determination section 52g, an additional learning section 52h, and a storage processing execution section 52n. In this embodiment, the reconstructed image data acquisition section 52f constitutes a "reconstructed image data acquisition means," the determination section 52g constitutes a "judgment means," and the additional learning section 52h constitutes an "additional learning means." .

However, the various functional units described above are realized by the cooperation of various hardware such as the CPU, ROM, and RAM, and there is no need to clearly distinguish between the functions realized by hardware or software. , some or all of these functions may be realized by a hardware circuit such as an IC.

Furthermore, the inspection control device 52 is provided with an input section 521 consisting of a keyboard, a mouse, a touch panel, etc., a display section 522 having a display screen such as a liquid crystal display, a communication section 528 capable of transmitting and receiving various data to and from the outside, and the like. ing. In this embodiment, the display unit 522 constitutes a "display means".

The inspection control device 52 also includes an inspection information storage section 523, a used AI storage section 524, an unused AI storage section 525, and an image data storage section, each of which is composed of an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. 526 and an additional learning information storage section 527. In this embodiment, the unused AI storage section 525 constitutes an "identification means storage means".

Prior to the various functional units that constitute the inspection control device 52, the input unit 521, the display unit 522, each of the storage units 523 to 527, and the communication unit 528 will be explained first.

The input unit 521 is an input means for inputting information to the inspection control device 52. In this embodiment, predetermined additional learning commands and additional learning modification commands can be input via the input unit 521.

The additional learning command is input by an operator or the like when, for example, a good PTP sheet 1 is erroneously determined to be a defective product by a quality determination by the determination unit 52g, which will be described later. On the other hand, the additional learning correction command is input by an operator or the like when, for example, the defective PTP sheet 1 is erroneously determined to be a non-defective product by the quality determination by the determination unit 52g. Of course, a configuration may also be adopted in which an additional learning instruction or an additional learning correction instruction is automatically input to the inspection control device 52 when it is determined that an erroneous determination has occurred by an inspection device provided separately other than the inspection device 21. .

It should be noted that an additional learning process, which will be described later, is performed by inputting an additional learning command, and an additional learning modification process, which will be described later, is performed by inputting an additional learning correction command. Further, various setting information stored in the examination information storage section 523 can be changed using the input section 521.

The display section 522 is configured to be able to display various information stored in each of the storage sections 523 to 527, for example. Therefore, the display unit 522 displays the image data (transparent image data) obtained by the camera 51, additional learning data 301 and defective image data 302, which will be described later, information used for inspection (for example, threshold values, etc.), and information on the PTP sheet 1. It is possible to display the pass/fail judgment results.

The inspection information storage unit 523 is for storing various setting information used in the inspection, pass/fail determination results, and the like. In this embodiment, various setting information includes, for example, the shapes and dimensions of the PTP sheet 1, the pocket part 2, and the tablet 5, the shape and dimensions of the pocket frame W for defining the area of the pocket part 2, and the relative position with respect to the camera 51. Positional relationships, threshold values used for quality determination and similarity determination of image data, etc. are stored.

The used AI storage unit 524 stores the AI model 200 as an "identification means". The AI model 200 has a function of outputting image data according to input image data using a deep neural network 190 (hereinafter simply referred to as "neural network 190"; see FIG. 9).

Here, the structure of the neural network 190 will be explained with reference to FIG. 9. FIG. 9 is a schematic diagram conceptually showing the structure of the neural network 190. The neural network 190 includes an encoder section 191 as an "encoding section" that extracts feature quantities (latent variables) TA from input image data (shade pattern data) GA, and an encoder section 191 that extracts feature quantities (latent variables) TA from input image data (shade pattern data) GA, and extracts image data (shade pattern data ) It has a convolutional auto-encoder (CAE) structure including a decoder section 192 as a "decoding section" that reconstructs GB.

Since the structure of the convolutional autoencoder is well-known, a detailed explanation will be omitted, but the encoder section 191 has a plurality of convolution layers 193, and each convolution layer 193 applies a plurality of filters to input data. The result of the convolution operation using (kernel) 194 is output as input data for the next layer. Similarly, the decoder unit 192 has a plurality of deconvolution layers 195, and in each deconvolution layer 195, a result of deconvolution operation using a plurality of filters (kernels) 196 is performed on input data. is output as the input data of the next layer. Then, in learning processing, additional learning processing, etc., which will be described later, the weights (parameters) of each filter 194 and 196 will be updated.

Returning to FIG. 6, the used AI storage unit 524 stores, as the AI model 200, the current AI model 200z currently used for inspecting the PTP sheet 1. The current AI model 200z will be explained in more detail later.

The unused AI storage unit 525 stores AI models 200 that are not used for testing. The AI model 200 stored in the unused AI storage unit 525 includes an AI model 200z that is stored separately from the current AI model 200z used for inspecting the PTP sheet 1 (stored in the used AI storage unit 524) A duplicate current AI model 200z1, which is an AI model 200 with the same content as the current AI model 200z, and an initial AI model 200a are included.

The duplicate current AI model 200z1 is obtained by copying the current AI model 200z, and even if various processes such as update processing are performed on itself, it will not affect the current AI model 200z in any way. be. In this embodiment, the current AI model 200z, the duplicate current AI model 200z1, and the initial AI model 200a each constitute a "trained identification means", and the initial AI model 200a constitutes an "old identification means". Further, the current AI model 200z constitutes a "current identification means", and the duplicate current AI model 200z1 constitutes a "duplicate current identification means".

The current AI model 200z is an AI model 200 that is generated by additional learning processing and additional learning correction processing that will be described later by the additional learning unit 52h, and is adopted in the approval processing that will be described later. The inspection of the PTP sheet 1 by the inspection device 21 is performed using the current AI model 200z. On the other hand, the duplicate current AI model 200z1 is not used for inspecting the PTP sheet 1.

The current AI model 200z and the duplicate current AI model 200z1 can be sequentially updated by additional learning processing and additional learning correction processing by the additional learning section 52h. In this embodiment, “AI(n)” (see FIG. 20) is stored in the used AI storage unit 524 and the unused AI storage unit 525 as the current AI model 200z and the duplicate current AI model 200z1. Note that n represents a natural number.

The initial AI model 200a is an AI model 200 that has not undergone any additional learning processing by the additional learning section 52h. In this embodiment, the unused AI storage unit 525 stores "AI(0)" (see FIG. 20), which is the factory-shipped AI model 200, as the initial AI model 200a.

Note that, at the time of factory shipment, the initial AI model 200a is stored in each of the AI storage units 524 and 525, respectively. The initial AI model 200a stored in the used AI storage unit 524 is used to inspect the PTP sheet 1 until it is updated to the current AI model 200z by additional learning processing.

The trained AI models 200, such as the current AI model 200z, the duplicate current AI model 200z1, and the initial AI model 200a, are trained by a neural network using only image data related to a good PTP sheet 1 with no molding defects in the pocket portion 2 as learning data. This is a generative model constructed by performing deep learning on 190, and has the structure of a so-called autoencoder. In this embodiment, the initial AI model 200a has been subjected to a learning process described later on its neural network 190, and the current AI model 200z and the duplicate current AI model 200z1 are the same as the initial AI model 200a. This is achieved by performing additional learning processing on the neural network 190 multiple times.

Here, the learning process (initial learning process) performed when generating the initial AI model 200a will be described. In this embodiment, the learning process is performed in advance at a location different from the installation location of the inspection device 21 (for example, a manufacturing factory of the inspection device 21, etc.) based on execution of a predetermined learning program. Note that, prior to the learning process, a large amount of image data regarding a good PTP sheet 1 with no molding defects in the pocket portion 2 is prepared as learning data. This learning data has the same format as image data (shaded pattern data) obtained by the image data obtaining section 52x, which will be described later.

As shown in FIG. 10, in the learning process, first, in step S201, an untrained neural network 190 is prepared. For example, the neural network 190 stored in advance in a predetermined storage device or the like is read out. Alternatively, the neural network 190 is constructed based on network configuration information (for example, the number of layers of the neural network, the number of nodes in each layer, etc.) stored in the storage device or the like.

Next, in step S202, reconstructed image data is acquired. That is, the learning data prepared in advance is applied to the input layer of the neural network 190 as input data. Then, reconstructed image data output from the output layer of the neural network 190 is obtained.

In the subsequent step S203, the learning data is compared with the reconstructed image data (reconstructed shading pattern data) output by the neural network 190 in step S202, and whether the error is sufficiently small (below a predetermined threshold) is compared. whether there is one or not).

Here, if the error is sufficiently small, it is determined in step S205 whether or not the end condition of the learning process is satisfied. For example, if an affirmative determination is made in step S203 a predetermined number of times without going through the process of step S204, which will be described later, or if learning using all of the prepared learning data is repeated a predetermined number of times. If so, it is determined that the termination condition is satisfied. If the termination conditions are met, the neural network 190 and its learning information (updated parameters, etc., which will be described later) are stored in each AI storage unit 524, 525 as the initial AI model 200a, and the learning process is terminated.

On the other hand, if the termination condition is not satisfied in step S205, the process returns to step S202 and the learning of the neural network 190 is performed again.

If the error is not sufficiently small in step S203, network updating processing (learning of the neural network 190) is performed in step S204, and then the process returns to step S202 to repeat the series of processes described above.

In the network update process in step S204, the neural network 190 uses a known learning algorithm such as error backpropagation to minimize the loss function representing the difference between the learning data and the reconstructed image data. The weights (parameters) of each of the filters 194 and 196 are updated to more appropriate ones. Note that as the loss function, for example, BCE (Binary Cross-entropy) can be used.

By repeating the processes of steps S202 to S204 many times, the error between the learning data and the reconstructed image data is minimized in the neural network 190, and more accurate reconstructed image data is output.

The initial AI model 200a that is finally obtained is image data (shade pattern data, sometimes referred to as "original image data") obtained by an image data acquisition unit 52x, which will be described later, and is a non-defective PTP. When image data related to sheet 1 is input, reconstructed image data that substantially matches the input image data is generated. In addition, the initial AI model 200a is image data (original image data) obtained by an image data acquisition unit 52x, which will be described later. This generates reconstructed image data that substantially matches the input image data, from which the portion corresponding to the input image data has been removed. That is, when the PTP sheet 1 is defective, virtual image data regarding the PTP sheet 1 assuming that there is no defective portion is generated as reconstructed image data regarding the PTP sheet 1.

Note that it is not necessary to perform the above learning process each time the inspection device 21 is manufactured; the neural network 190 and its learning information (updated parameters, etc.) are acquired in advance, and these are stored in each AI storage section of the inspection device 21. 524 and 525 may be stored as the initial AI model 200a.

The image data storage section 526 stores various image data such as image data (shaded pattern data) obtained by an image data acquisition section 52x, which will be described later. The image data (shade pattern data) obtained by the image data acquisition unit 52x corresponds to one PTP sheet 1, and includes a total of 10 shading patterns on the bottom 2a of the pocket section 2, etc. It represents. In this embodiment, the image data storage section 526 constitutes a "specific learning data storage means" and a "defective image data storage means."

The image data stored in the image data storage section 526 includes additional learning data 301 used for additional learning processing by the additional learning section 52h. The additional learning data 301 is image data related to a good PTP sheet 1 among a plurality of image data (shaded pattern data) obtained by the image data acquisition unit 52x, and which satisfies a predetermined condition (for example, an operator, etc.). (selected by).

Further, the additional learning data 301 includes image data that causes an erroneous determination when a non-defective PTP sheet 1 is determined to be a defective product by the determination unit 52g, which will be described later. That is, the additional learning data includes image data related to the PTP sheet 1 that was determined to be a defective product by the determination unit 52g but was actually a non-defective product.

In this embodiment, “IM(1)” to “IM(n)” (see FIG. 20) are stored as additional learning data 301. For example, "IM(1)" is image data used in additional learning processing for the neural network 190 of the initial AI model 200a, and the AI model 200 generated by this additional learning processing is "AI(1)". be.

Further, the additional learning data 301 includes specific learning data 301a. The specific learning data 301a is additional learning data 301 used in generating the current AI model 200z from the initial AI model 200a. Therefore, in this embodiment, all the additional learning data 301 corresponds to the specific learning data 301a.

Furthermore, the image data stored in the image data storage unit 526 includes defective image data 302. The defective image data 302 is shading pattern data (original image data) related to the erroneous determination when the defective PTP sheet 1 is erroneously determined to be a good product by the quality determination process by the determination unit 52g, which will be described later. be. That is, the defective image data 302 is different from the additional learning data 301 and is related to the defective PTP sheet 1.

In the present embodiment, it is determined whether or not the defective PTP sheet 1 is erroneously determined to be a good product by the quality determination by the determination unit 52g, or whether the non-defective PTP sheet 1 is determined to be a defective product by the quality determination by the determination unit 52g. Whether or not it has been erroneously determined that there is a problem is determined by visual inspection by an operator or the like. Of course, an inspection device other than the inspection device 21 may be provided separately, and the inspection device may determine whether an erroneous determination has occurred.

Then, by inputting information using the input section 521 by an operator or the like, it is determined that the defective PTP sheet 1 is a good product among the image data (shaded pattern data) stored in the image data storage section 526. When an erroneous determination is made, the image data related to the erroneous determination is identified as defective image data 302.

The additional learning information storage unit 527 stores information related to learning of the AI model 200, such as various information related to additional learning processing and additional learning correction processing (that is, learning processing by the additional learning unit 52h).

The information stored in the additional learning information storage unit 527 includes, for example, all the learned (learning processing, additional learning processing, additional learning correction processing) generated up to the present time, including the current AI model 200z and the initial AI model 200a. This includes the AI model 200 (in which training was performed) and additional learning data 301 used for learning the trained AI model 200. In this embodiment, the trained AI model 200 constitutes a "trained identification means" like the initial AI model 200a and the current AI model 200z.

In addition, the information stored in the additional learning information storage unit 527 includes, for example, the input section performed in various processes (for example, additional learning processing, additional learning correction processing, confirmation processing and approval processing, respectively, which will be described later). 521, the date and time when an input related to approval was received in the approval process described later, etc.

Note that all trained AI models 200 generated up to this point include trained AI models 200 that were rejected in the approval process described later. The trained AI model 200 that was rejected is given information indicating that it was rejected, and what is the trained AI model 200 that was adopted (that is, the current or past working AI model 200z)? It is possible to distinguish. Further, the additional learning data 301 includes data that was used to generate the trained AI model 200 that was rejected. Note that the additional learning information storage section 527 may store information related to all operations performed on the input section 521 described above as an audit trail.

The communication unit 528 includes, for example, a wireless communication interface conforming to a communication standard such as a wired LAN (Local Area Network) or a wireless LAN, and is configured to be able to transmit and receive various data to and from the outside. For example, the results of the quality determination performed by the determination unit 52g are output to the outside (filling control device 82, etc.) via the communication unit 528.

Next, the various functional units that constitute the inspection control device 52 will be explained in detail.

The main control section 52a is a functional section that controls the entire inspection apparatus 21, and is configured to be able to transmit and receive various signals with other functional sections such as the illumination control section 52b and the camera control section 52c.

The lighting control unit 52b is a functional unit that drives and controls the lighting device 50, and controls lighting timing and the like based on a command signal from the main control unit 52a.

The camera control section 52c is a functional section that drives and controls the camera 51, and controls the imaging timing and the like based on a command signal from the main control section 52a. The timing of the illumination and imaging is controlled by the main control unit 52a based on a signal from an encoder (not shown) provided in the PTP packaging machine 11.

The illumination control section 52b and camera control section 52c irradiate the container film 3 with electromagnetic waves from the illumination device 50 at every interval when the conveyance of the container film 3 in which the pocket section 2 has been formed is stopped. A process is executed in which the electromagnetic waves (ultraviolet light) transmitted through the container film 3 are imaged by the camera 51. Transmission image data captured and generated by the camera 51 is converted into a digital signal (image signal) inside the camera 51, and then transferred to the inspection control device 52 (image capture unit 52d) in the form of a digital signal. be done.

The image capture unit 52d is a functional unit for capturing image data captured and acquired by the camera 51. The captured image data (transparent image data) is stored in the image data storage section 526.

The image processing unit 52e is a functional unit that performs predetermined image processing on the image data captured by the image capture unit 52d to obtain grayscale pattern data (original image data). In this embodiment, the image processing section 52e generates shading pattern data by extracting a portion related to the pocket section 2 from the image data (transparent image data) captured by the image capturing section 52d.

To explain in more detail, first, mask processing is performed on the image data (transparent image data) captured by the image capturing section 52d. Specifically, pocket frames W (see FIG. 16) are set respectively according to the positions of the ten pocket parts 2 in the image data (transparent image data) captured by the image capture unit 52d, and the pocket frames are A process is performed in which a mask M is applied to an area other than the pocket area specified by W, that is, an area corresponding to the film flat portion 3b.

Next, the image within each pocket frame W in the masked image data (masking image data) is extracted to generate shading pattern data. The generated shading pattern data corresponds to one PTP sheet 1, and represents a total of 10 shading patterns K (see FIGS. 16 and 18) on the bottom 2a of the pocket portion 2. The shading pattern data is stored in the image data storage section 526. In this embodiment, some of the acquired grayscale pattern data becomes the additional learning data 301 and the defective image data 302. Note that a total of 10 pieces of shading pattern data each representing the shading pattern K related to one pocket portion 2 may be obtained from the transparent image data.

Here, FIG. 16 is a diagram showing a shading pattern K that occurs on the bottom part 2a of the pocket part 2 without molding defects in the shading pattern data, and FIG. It is a graph showing the brightness value concerning each pixel along.

Further, FIG. 18 is a diagram showing a shading pattern K that occurs on the bottom part 2a of the pocket part 2 with a molding defect, and FIG. It is a graph showing brightness values.

As shown in FIGS. 16 to 19, the shading pattern K is two-dimensional image information that has luminance information (for example, any value among 256 gradations from 0 to 255) for each pixel, and the pocket area From the relationship between the difference in wall thickness (thickness distribution) at each position (coordinate position) of the bottom part 2a, etc. of the pocket part 2, and the transmittance of electromagnetic waves passing therethrough, it is possible to This corresponds to an image showing a dimensional distribution (an intensity distribution image of transmitted electromagnetic waves).

In this embodiment, since the pocket frame W is set to match the opening periphery of the pocket portion 2 (the connecting portion between the side portion 2b and the film flat portion 3b), the shading pattern K does not include the pocket portion. The pattern includes not only the bottom part 2a of the pocket part 2, but also the side part 2b of the pocket part 2, and the corner part 2c of the pocket part 2 where the bottom part 2a and the side part 2b intersect.

Further, in the range corresponding to the bottom portion 2a of the shading pattern K, a shading pattern having a shading distribution (luminance distribution) approximately corresponding to the wall thickness distribution of the bottom portion 2a is obtained. On the other hand, regarding the range corresponding to the side part 2b and the corner part 2c, the brightness information does not correspond to the electromagnetic waves transmitted along the thickness direction (X direction and Y direction) of the side part 2b etc. Since it corresponds to electromagnetic waves transmitted along the stretching direction (Z direction) during molding, it has little relationship with the wall thickness of the side portion 2b, etc.

In this embodiment, an image data acquisition section 52x is configured by the camera 51, an image capture section 52d, and an image processing section 52e. ) is obtained. In this embodiment, the image data acquisition section 52x constitutes an "image data acquisition means."

The reconstructed image data acquisition unit 52f inputs the gradation pattern data (original image data) obtained by the image data acquisition unit 52x to a trained AI model 200 such as the current AI model 200z, and obtains the reconstructed image data. Acquire as reconstructed image data. More specifically, the reconstructed image data acquisition section 52f supplies the grayscale pattern data (original image data) obtained by the image data acquisition section 52x to the input layer of the neural network 190 as input data. Then, the reconstructed image data acquisition unit 52f obtains the reconstructed shading pattern data output from the output layer of the neural network 190 as reconstructed image data. Note that when inspecting the PTP sheet 1, the reconstructed image data acquisition unit 52f inputs the original image data to the current AI model 200z stored in the used AI storage unit 524. On the other hand, during confirmation processing to be described later, the reconstructed image data acquisition unit 52f inputs the original image data to a trained AI model 200 that is newly generated by additional learning processing and is different from the current AI model 200z.

The determination unit 52g is a functional unit that performs quality determination processing regarding the molding state of the pocket portion 2. The determination unit 52g compares the shading pattern data (original image data) obtained by the image data acquisition unit 52x with reconstructed image data that was reconstructed by inputting the shading pattern data into the learned AI model 200. Then, based on the comparison result, the quality of the pocket section 2 and, in turn, the PTP sheet 1 is determined.

With reference to FIG. 11, the quality determination process by the determination unit 52g will be described in more detail. In the quality determination process, first, in step S301, the reconstructed image data acquisition unit 52f inputs the grayscale pattern data (original image data) acquired by the image data acquisition unit 52x into the trained AI model 200. , obtain reconstructed image data. Note that the input target of the original image data is the current AI model 200z when inspecting the PTP sheet 1, and the newly generated trained AI model 200 during the confirmation process described later.

Next, in step S302, the grayscale pattern data (original image data) obtained by the image data acquisition unit 52x is compared with the reconstructed image data related to the grayscale pattern data obtained by the reconstructed image data acquisition unit 52f. Then, the difference in luminance for each pixel of both data is calculated. Subsequently, pixels for which the difference is not within a predetermined tolerance range are identified as defective pixels.

Next, in step S303, the area of the connected component of the defective pixel (defective area) is calculated.

Next, in step S304, it is determined whether the maximum area among the calculated areas is less than or equal to a predetermined threshold value set in advance. In other words, it is determined whether the defective area is within the allowable range. If the maximum area is equal to or larger than the threshold value, it is determined that a defect has occurred in at least one pocket portion 2, and in step S305, the PTP sheet 1 corresponding to the shading pattern data (original image data) is ”.

On the other hand, if the maximum area is less than the threshold value, the PTP sheet 1 corresponding to the shading pattern data (original image data) is determined to be "good" in step S306. The pass/fail determination result is stored in the inspection information storage section 523. Further, when inspecting the PTP sheet 1, the pass/fail determination result is output to the filling control device 82 or the like.

The above-described quality determination method is just one example, and the quality determination may be performed using other methods, such as a method of determining the degree of dispersion (distribution status) of connected components of defective pixels. Of course, if there is even one defective pixel, the product may be determined to be "defective" regardless of its size.

The additional learning unit 52h sequentially generates new trained AI models 200 by performing additional learning of the neural network 190 using the additional learning data 301. The additional learning unit 52h performs additional learning processing when the additional learning command is input by an operator or the like.

The additional learning process will be described in more detail with reference to FIGS. 12 and 20. In the additional learning process, first, in step S401, reconstructed image data is acquired. For example, if an additional learning command is input in a state where “IM (1)” is newly stored as unlearned additional learning data 301, the additional learning unit 52h stores the data in the unused AI storage unit 525. For the input layer of the neural network 190 of "AI(0)" which is the replicated current AI model 200z1 (however, the replicated current AI model 200z1 at this stage is the initial AI model 200a), the additional learning data 301 " IM(1)'' is given as input data. Then, reconstructed image data output from the output layer of the neural network 190 is obtained.

In the subsequent step S402, the additional learning data 301 is compared with the reconstructed image data output by the neural network 190 in step S401, and it is determined whether the error is sufficiently small (whether it is below a predetermined threshold). judge.

Here, if the error is sufficiently small, it is determined in step S404 whether or not the additional learning termination condition is satisfied. For example, if additional learning using the stored additional learning data 301 is repeated a predetermined number of times, it is determined that the termination condition is satisfied. If the termination conditions are met, the neural network 190 and its learning information (updated parameters, etc.) are stored in the unused AI storage unit 525 as a new trained AI model 200, and the additional learning process is terminated. .

On the other hand, in step S402, if the error is not small enough, in step S403, network update processing (learning of the neural network 190) is performed, and then the process returns to step S401 to repeat the above series of processes. The network update process is similar to the process in step S204 in the learning process described above. By repeating the processing of steps S401 to S403, the error between the additional learning data 301 and the reconstructed image data is minimized in the neural network 190, and more accurate reconstructed image data is output.

Note that the trained AI model 200 newly generated by the additional learning process is the shading pattern data (original image data) obtained by the image data acquisition unit 52x, like the current AI model 200z and the initial AI model 200a. When image data related to a non-defective PTP sheet 1 is input, reconstructed image data that substantially matches the input image data is generated. In addition, the newly generated trained AI model 200, like the current AI model 200z and the initial AI model 200a, is gray pattern data (original image data) obtained by the image data acquisition unit 52x, and is defective. When image data related to the PTP sheet 1 is input, reconstructed image data that substantially matches the input image data from which noise portions (portions corresponding to defective portions) have been removed is generated.

Further, the additional learning section 52h includes a factor data specifying section 52i and an image output AI confirmation section 52j as functional sections. In this embodiment, the factor data specifying section 52i constitutes a "factor data specifying means", and the confirmation section 52j for image output AI constitutes a "confirmation means for image output AI".

The factor data specifying unit 52i particularly functions in additional learning correction processing. When the defective PTP sheet 1 is erroneously determined to be a good product by the quality determination process of the determination unit 52g, the factor data identification unit 52i performs specific learning based on the degree of similarity to the defective image data 302 related to the erroneous determination. The additional learning data 301 that caused the misjudgment is identified from the data 301a as misjudgment factor data. In this embodiment, the factor data identification unit 52i determines the degree of similarity between the specific learning data 301a and the defective image data 302 (for example, the feature amount extracted from the defective image data 302 and the feature amount extracted from the specific learning data 301a). (e.g., an AI generated by making a neural network learn at least the defective image data 302, etc.) capable of calculating a score indicating the degree of similarity between both data 301a and 302 based on It is. Then, the factor data specifying unit 52i specifies misjudgment factor data based on the comparison result between the degree of similarity calculated by the similarity determining means and a preset threshold value. The calculated degree of similarity is, for example, from 0 to 100, and one or more pieces of specific learning data 301a whose degree of similarity is equal to or greater than a threshold value are specified as data that causes a false determination.

The image output AI confirmation unit 52j performs a confirmation process to confirm the learning state of the trained AI model 200 newly generated by the additional learning process or the additional learning correction process. The confirmation process is performed when a new trained AI model 200 is generated by the additional learning process or the additional learning correction process. The confirmation process is performed as follows.

As shown in FIG. 13, first, in step S501, it is determined whether the learning state of the newly generated AI model 200 is appropriate.

Specifically, the reconstructed image data acquisition unit 52f inputs additional learning data 301 or defective image data 302 to the newly generated trained AI model 200, and obtains reconstructed image data. Here, if the new trained AI model 200 is generated by additional learning processing, the additional learning data 301 (original image data related to the misjudgment that triggered the additional learning processing) is input to model 200. On the other hand, if the new trained AI model 200 is generated by the additional learning correction process, the defective image data 302 (original image data related to the misjudgment that triggered the additional learning correction process) Input to the AI model 200.

Next, the determination unit 52g obtains a pass/fail determination result based on comparing the additional learning data 301 or the defective image data 302 with the reconstructed image data.

Then, it is determined whether the obtained quality determination result is correct or not. Here, since the additional learning data 301 is related to the non-defective PTP sheet 1, when the additional learning data 301 is input to the new trained AI model 200, if the determination result is "non-defective", It can be said that the pass/fail judgment result is correct. In addition, since the defective image data 302 is related to the defective PTP sheet 1, when the defective image data 302 is input to the new trained AI model 200, if the determination result is "defective". , it can be said that the pass/fail judgment result is correct.

Then, if the pass/fail determination result is correct, that is, if the learning state is appropriate, it is determined in step S502 that the current AI model 200z can be replaced with the newly generated trained AI model 200, Finish the confirmation process. On the other hand, if the pass/fail judgment result is incorrect, that is, if the learning state is inappropriate, it is determined in step S503 that it is impossible to replace the current AI model 200z with the newly generated trained AI model 200. Make a judgment and end the confirmation process.

In addition, in the confirmation process, additional learning data 301 other than the original image data related to misjudgment, which triggered the additional learning process or additional learning correction process, is input to the newly generated trained AI model 200. Then, it may be determined whether the obtained quality determination result is correct or not.

Further, the additional learning section 52h cooperates with the factor data identification section 52i, the image output AI confirmation section 52j, and the like to execute additional learning and modification processing in response to the input of the additional learning and modification command.

In the additional learning correction process, as shown in FIG. 14, first, in step S551, the factor data specifying unit 52i specifies one or more erroneous determination factor data. Thereby, misjudgment factor data is extracted from the specific learning data 301a.

Next, in step S552, relearning processing is executed. In the relearning process, the neural network 190 of the initial AI model 200a stored in the unused AI storage unit 525 is made to learn the specific learning data 301a excluding the erroneous judgment factor data identified in step S551, and a new A trained AI model 200 is generated. For example, in step S551, "IM(3)" is selected as the misjudgment factor data from "IM(0)" to "IM(n)" which are the specific learning data 301a stored in the image data storage unit 526. Suppose that is specified. In this case, in step S552, specific learning data 301a other than "IM(3)" such as "IM(0)" to "IM(2)" and "IM(4)" are added to the initial AI model 200a. ” to “IM(n)”, the above additional learning process is performed. As a result, a new trained AI model 200 is generated that has learned the specific learning data 301a other than "IM(3)" which is the misjudgment factor data.

Next, in step S553, it is determined whether the learning state of the new trained AI model 200 is appropriate. This process is similar to the process in step S501 in the confirmation process. If the learning state is appropriate, it is assumed that the defective image data 302 that caused the erroneous determination has been corrected so that an appropriate pass/fail determination is performed, and the additional learning correction process is terminated.

On the other hand, if the learning state is inappropriate (step S553: NO), the process returns to step S551 and the erroneous determination factor data is identified again. At this time, the factor data specifying unit 52i performs appropriate adjustments such as changing the threshold value for comparison with the similarity score or specifying only one piece of misjudgment factor data with the highest degree of similarity. Respecify the determination factor data. Note that if it is determined in step S553 that the learning state is inappropriate, processing for changing the threshold value used in the pass/fail determination process may also be performed.

Then, by repeating the above process until the learning state becomes correct, a trained AI model 200 that can respond appropriately to the defective image data 302 that caused the erroneous determination is finally obtained.

Further, the additional learning section 52h includes an approval processing execution section 52k, an approval input reception section 52l, and a replacement section 52m as functional sections. In this embodiment, the approval input receiving unit 52l constitutes an "approval input receiving means".

The approval processing execution unit 52k generates a new trained AI model 200 by executing additional learning processing and additional learning correction processing, and can replace the current AI model 200z with the learned AI model 200 through confirmation processing. If it is determined that this is the case, the approval process is executed. By executing the approval process, at least various information related to the learning process (additional learning process and additional learning correction process) by the additional learning unit 52h, which is stored in the additional learning information storage unit 527, and the confirmation result by the confirmation process. is displayed on the display section 522. For example, an image showing additional learning data 301 used to generate a new trained AI model 200 is displayed.

In addition, by executing the approval process, the currently used AI model 200z used for inspecting the PTP sheet 1 (stored in the used AI storage unit 524) is displayed on the display unit 522, and the newly generated AI model When approving replacement with 200, an approval button is displayed that is selected by an operator or the like. Note that the display section 522 also displays a non-approval button that is selected by the operator or the like when the replacement is not approved.

When the approval button is selected using the input unit 521, the approval input receiving unit 52l inputs the current AI model 200z (stored in the used AI storage unit 524) used for inspecting the PTP sheet 1. , accepts input regarding approval for replacement with the newly generated AI model 200. When the approval input receiving unit 52l receives input related to approval, it outputs a predetermined replacement execution command to the replacement unit 52m, and ends the approval process.

On the other hand, when the non-approval button is selected using the input unit 521, the approval input receiving unit 52l ends the approval process without outputting the replacement execution command.

Note that when the approval process is executed, a confirmation process execution button is displayed, and when the operator or the like selects this button, the confirmation process is executed again for the newly generated trained AI model 200. It may be configured as follows. At this time, the additional learning data 301 used in the confirmation process may be selected manually or automatically.

When the replacement execution command is input, the replacement unit 52m replaces the currently used AI model 200z (stored in the used AI storage unit 524) used for inspecting the PTP sheet 1 with a newly generated learning model. A replacement process for replacing the AI model 200 with the completed AI model 200 is executed.

In the replacement process, first, the production of PTP sheets 1 by the PTP packaging machine 11 is temporarily stopped. Then, the current AI model 200z stored in the used AI storage unit 524 is replaced with a new trained AI model 200 stored in the unused AI storage unit 525. After replacing the current AI model 200z, production of the PTP sheet 1 is restarted. As a result, inspection of the PTP sheet 1 using the new current AI model 200z will begin. Note that through the replacement process, the new learned AI model 200 stored in the unused AI storage unit 525 becomes the duplicate current AI model 200z1.

The storage processing execution unit 52n stores information related to learning of the AI model 200, such as various information related to additional learning processing and additional learning correction processing (that is, learning processing by the additional learning unit 52h), in the additional learning information storage unit 527. let The storage processing execution unit 52n stores all trained AI models 200 (rejected in the approval process) generated up to the present time, including the current AI model 200z and the initial AI model 200a, in the additional learning information storage unit 527. (including the AI model 200 that has changed), additional learning data 301 used to generate the trained AI model 200, and various processes (for example, additional learning processing, additional learning correction processing, confirmation processing, approval processing, etc.) The contents of all the operations performed on the input unit 521, the date and time when the input related to approval was received in the approval process, the date and time when the replacement of the current AI model 200z was executed, etc. are stored.

Briefly speaking, the flow of various processes (additional learning processing, additional learning correction processing, etc.) by the additional learning section 52h described in detail above is as shown in FIG. 15.

That is, if the defective PTP sheet 1 is erroneously determined to be a good product in the inspection (step S11: YES), an additional learning correction command is input by an operator, etc., and the additional learning correction processing (step S12-A) is performed. ) is executed. That is, a new trained AI model 200 is generated by causing the neural network 190 of the initial AI model 200a to learn the specific learning data 301a excluding the data that causes erroneous determination.

On the other hand, if the good PTP sheet 1 is erroneously determined to be a defective product during the inspection (step S11: NO), an additional learning command is input by the operator, etc., and the additional learning process (step S12-B) is performed. executed. That is, a new trained AI model 200 is generated by additionally learning the image data related to the erroneous determination with respect to the duplicate current AI model 200z1.

Then, when a new trained AI model 200 is generated, a confirmation process in step S14 is performed. In the confirmation process, first, in step S14-1, the learning state of the generated AI model 200 is confirmed. Here, if the learning state is appropriate (step S14-1: YES), it is determined that the current AI model 200z can be replaced with a new trained AI model 200 (step S14-2-A). On the other hand, if the learning state is inappropriate (step S14-1: NO), it is determined that the current AI model 200z cannot be replaced with a new trained AI model 200 (step S14-2-B).

If it is determined that replacement is possible, an approval process (step S15) is executed, and information required by an operator or the like to perform the replacement is displayed on the display unit 522. When the approval button is selected by the operator or the like (step S16: YES), the PTP packaging machine 11 is temporarily stopped, and the current AI model 200z is replaced with a new trained AI model 200 (step S16: YES). S17). On the other hand, if the operator or the like selects the disapproval button (step S16: No), no replacement is performed and the current AI model 200z is used as is.

In addition, if a new erroneous judgment occurs before the confirmation process or approval process is completed, the new additional learning process or additional learning correction process will be executed until the replacement is approved or disapproved. It may be put on hold.

As described in detail above, according to the present embodiment, the original image data (shaded pattern data) acquired by the image data acquisition unit 52x and the original image data are input to the current AI model 200z and reconstructed. The PTP sheet 1 is compared with the reconstructed image data, and the quality of the PTP sheet 1 is determined based on the comparison result. Therefore, both image data to be compared are related to the same PTP sheet 1. Therefore, since the shape and appearance of PTP sheet 1 are almost the same in both image data to be compared, there is no need to set relatively loose judgment conditions to prevent false detection due to differences in shape and appearance. Strict judgment conditions can be set. Furthermore, the conditions for obtaining the image data (for example, the arrangement position and arrangement angle of the PTP sheet 1 with respect to the camera 51, the brightness and darkness state, the viewing angle of the camera 51, etc.) can be matched in both image data to be compared. As a result, pass/fail judgments can be made with high accuracy.

In addition, if the defective PTP sheet 1 is erroneously determined to be a good product by the quality judgment based on the reconstructed image data that was reconstructed by inputting the original image data to the current AI model 200z, in other words, due to overlearning. If it is thought that a misjudgment has occurred, the additional learning unit 52h uses the neural structure of the initial AI model 200a, which is the AI model 200 at a stage where additional learning has not been performed (the AI model 200 at a stage before the current AI model 200z). Learning is performed on the network 190 to generate a new trained AI model 200. At this time, the additional learning unit 52h causes the neural network 190 of the initial AI model 200a to learn data other than the erroneous determination factor data out of the specific learning data 301a. Therefore, in the newly generated AI model 200, while making use of the past additional learning data 301 (specific learning data 301a), the learning state is modified to more reliably suppress the occurrence of false judgments due to overfitting. It can be carried out. As a result, it is possible to more reliably prevent a defective PTP sheet 1 from being erroneously determined to be a good product, and it is possible to improve inspection accuracy.

In addition, the additional learning unit 52h performs learning on the neural network 190 of the replicated current AI model 200z1 instead of the neural network 190 of the current AI model 200z used for inspection. Therefore, the process related to the learning of the neural network 190 and the process related to the extraction (identification) of erroneous determination factor data can be performed without stopping the test, which can be a hugely repetitive process. Thereby, a large number of PTP sheets 1 can be inspected quickly and efficiently.

Furthermore, in the present embodiment, when a good PTP sheet 1 is erroneously determined to be a defective product, the neural network 190 of the replica current AI model 200z1 is made to additionally learn the original image data that caused the erroneous determination. By doing so, it is possible to generate a new trained AI model 200. Therefore, it is possible to more reliably prevent a good PTP sheet 1 from being erroneously determined to be a defective product, and the inspection accuracy can be further improved.

In addition, since this additional learning targets the replicated current AI model 200z1, there is no need to stop testing for additional learning. Therefore, it becomes possible to inspect a large number of PTP sheets 1 more quickly and more efficiently.

Furthermore, the factor data specifying unit 52i selects misjudgment factor data from the specific learning data 301a based on the degree of similarity to the defective image data 302, which is the original image data related to the misjudgment (original image data that caused the misjudgment). (Additional learning data 301 that is considered to be the cause of the misjudgment) can be specified. Therefore, the misjudgment factor data can be specified more accurately, and in turn, the accuracy of the test using the newly generated AI model 200 can be more reliably increased.

In addition, the image output AI confirmation unit 52j determines whether or not an erroneous judgment based on the original image data related to the erroneous judgment that triggered the generation of the AI model 200 occurs in the newly generated AI model 200. In other words, it is possible to easily confirm whether or not better inspection accuracy is achieved. In addition, if it is confirmed that a similar misjudgment occurs in a test using the newly generated AI model 200, we will take measures to deal with such misjudgment (for example, by re-identifying the data that caused the misjudgment and , re-learning after removing the re-identified erroneous determination factor data, changing the threshold value used in the step of comparing the original image data and the reconstructed image data, etc.).

In addition, the current AI model 200z is replaced with the newly generated trained AI model 200 only after approval by the operator or the like. Therefore, it is possible to more reliably prevent unintentional replacement of the current AI model 200z with the newly generated trained AI model 200. This makes it possible to more reliably prevent problems caused by unintended replacement.
[Second embodiment]
Next, the second embodiment will be described with reference to FIG. 21 and the like, focusing on the differences from the first embodiment. The main differences between the first embodiment and the second embodiment regarding the AI model and its learning are as follows.
(Above first embodiment)
- Learned AI model 200: This model is generated by learning only the image data related to the good PTP sheet 1, and outputs the image data when the image data is input.
-Additional learning process...Executed when a good PTP sheet 1 is erroneously determined to be a defective product.
・Additional learning correction processing...Executed when the defective PTP sheet 1 is incorrectly determined to be a good product.
(Second embodiment)
- Learned AI model 400: This model is generated by learning image data related to good and defective PTP sheets 1, and outputs a classification result when the image data is input.
・Additional learning process: This process is executed not only when a good PTP sheet 1 is incorrectly judged as a defective product, but also when a defective PTP sheet 1 is incorrectly judged as a good product, or when an incorrect classification result occurs. Ru.
・Additional learning correction processing: Executed when an erroneous determination occurs in the confirmation processing performed following the additional learning processing.

The second embodiment will be described in detail below.

In the first embodiment, the inspection device 21 is disposed downstream of the pocket forming device 16, and determines the quality of the PTP sheet 1 by inspecting the forming state of the pocket portion 2. On the other hand, as shown in FIG. 21, the inspection device 23 in the second embodiment is disposed downstream of the tablet filling device 22, and inspects the state of the tablets 5 filled in the pocket portion 2 and the condition of the pocket portion. The quality of the PTP sheet 1 is determined by inspecting the PTP sheet 2 for the presence or absence of foreign matter. In addition, the inspection device 23 determines whether the PTP sheet 1 is good or bad and determines the type of defect in the PTP sheet 1. In the second embodiment, the defective sheet discharge mechanism 40 discharges the defective PTP sheet 1 when a defective product signal is input from the inspection device 23.

The inspection device 23 is a transmitted light type inspection device that inspects the container film 3 from the pocket portion 2 protrusion side before sealing. The inspection device 23 includes a lighting device 53, a camera 54, and an inspection control device 55.

The illumination device 53 is disposed downstream of the tablet filling device 22 and on the opening side of the pocket portion 2 of the container film 3, and irradiates a predetermined range of the container film 3 with predetermined light (near infrared light, etc.). .

The camera 54 is provided on the protrusion side of the pocket portion 2 of the container film 3 and is sensitive to the wavelength region of the light irradiated from the illumination device 53. The camera 54 images the light that has passed through the container film 3 out of the light (near infrared light) irradiated from the illumination device 53 . The camera 54 images an area including a total of ten pocket sections 2 corresponding to at least one PTP sheet 1 at a time.

The inspection control device 55 is configured by a computer system including a CPU and the like, similar to the inspection control device 52 in the first embodiment. As shown in FIG. 22, the inspection control device 55 includes a main control section 55a, a lighting control section 55b, a camera control section 55c, an image capturing section 55d, an image processing section 55e, and a judgment section by operating the CPU according to various programs. It functions as various functional units such as a processing unit 55g, an additional learning unit 55h, and a storage processing execution unit 55n. In the second embodiment, the additional learning section 55h constitutes an "additional learning means".

Note that the main control section 55a, lighting control section 55b, camera control section 55c, image capturing section 55d, and image processing section 55e are the main control section 52a, lighting control section 52b, camera control section 52c, It functions similarly to the image capture section 52d and the image processing section 52e.

Furthermore, the inspection control device 55 is provided with an input section 551, a display section 552, and a communication section (not shown) capable of transmitting and receiving various data to and from the outside. The communication unit functions similarly to the communication unit 528 in the first embodiment. In the second embodiment, the display section 552 constitutes a "display means".

The examination control device 55 also has an examination information storage section 553, a used AI storage section 554, an unused AI storage section 555, an image data storage section 556, and an additional learning information storage section 557, each of which is composed of a storage device such as an HDD. We are prepared. In this embodiment, the unused AI storage section 555 constitutes an "identification means storage means."

The input unit 551 is an input device used to input information to the inspection control device 55, change the information, etc., and can input a predetermined additional learning command via the input unit 551. The additional learning command is input by an operator or the like when, for example, there is a misjudgment in the classification result by the trained AI model 400, which will be described later. Of course, a configuration may also be adopted in which an additional learning command is automatically input to the inspection control device 55 when it is determined that an erroneous determination has occurred by an inspection device provided separately other than the inspection device 23. Note that additional learning processing, which will be described later, is performed by inputting the additional learning command.

The display unit 552 is a display device that can display various information stored in each of the storage units 553 to 557.

The inspection information storage section 553 is for storing various setting information used in the inspection, pass/fail determination results, and the like. In the second embodiment, thresholds for quality determination and classification thresholds are stored in advance as various setting information. These threshold values are used for classification by the trained AI model 400.

The used AI storage unit 554 stores the AI model 400 as an "identification means". The AI model 400 uses a deep neural network (hereinafter simply referred to as "neural network") to classify input original image data according to predetermined items, and outputs the classification results. In other words, the AI model 400 in the second embodiment is a classification type AI model.

The used AI storage unit 554 stores, as the AI model 400, a current AI model 400z used for inspecting the PTP sheet 1. The current AI model 400z will be explained in more detail later.

The unused AI storage unit 555 stores AI models 400 that are not used for testing. The AI model 400 stored in the unused AI storage unit 555 includes a duplicate current AI model with the same content as the current AI model 400z, which is stored separately from the current AI model 400z used for inspecting the PTP sheet 1. 400z1 and an initial AI model 400a.

In the second embodiment, the trained AI models 400 such as the current AI model 400z, the duplicate current AI model 400z1, and the initial AI model 400a constitute a "trained identification means", and the initial AI model 400a constitutes an "old identification means". "means" constitutes "means". Further, the current AI model 400z constitutes a "current identification means", and the duplicate current AI model 400z1 constitutes a "duplicate current identification means".

The current AI model 400z is an AI model 400 that is generated by additional learning processing and additional learning correction processing by the additional learning unit 55h, and is adopted in the approval processing described later. The inspection of the PTP sheet 1 by the inspection device 23 is performed using the current AI model 400z.

The current AI model 400z and the duplicate current AI model 400z1 can be sequentially updated by additional learning processing and additional learning correction processing by the additional learning section 55h. In this embodiment, each AI storage unit 554, 555 stores "AI2(n)" (see FIG. 28) as the current AI model 400z or the duplicate current AI model 400z1. Note that n represents a natural number.

The initial AI model 400a is an AI model 400 at a stage where learning processing to be described later has been performed, but no additional learning processing by the additional learning section 55h has been performed. In this embodiment, the AI storage unit 554 stores the factory-shipped AI model 400 "AI2(0)" (see FIG. 28) as the initial AI model 400a.

The trained AI models 400, such as the current AI model 400z and the initial AI model 400a, use not only the image data related to the non-defective PTP sheet 1 but also the image data related to the defective PTP sheet 1 as learning data, and use the neural network as training data. This is a generative model built using deep learning. In other words, the trained AI model 400 is generated by learning image data related to non-defective and defective PTP sheets 1. In the second embodiment, the initial AI model 400a is a neural network that has undergone a learning process described later, and the current AI model 400z and the duplicate current AI model 400z1 are the initial AI model 400a. Additional learning processing by the additional learning unit 55h is performed multiple times on the neural network.

When original image data is input, the trained AI model 400 calculates the probability that the PTP sheet 1 related to the original image data is a "good product", the probability that the PTP sheet 1 is a "defective product", and the probability that the PTP sheet 1 is a "defective product". Calculate the probability regarding the type. Then, the trained AI model 400 compares the calculated probability of being a "good product" or "defective product" with the threshold value for quality determination stored in the inspection information storage unit 553. The quality of the PTP sheet 1 based on the original image data is determined, and the quality determination result is output.

In addition, the trained AI model 400 compares the probability regarding the calculated defect type with the classification threshold stored in the inspection information storage unit 553 to determine whether the PTP sheet 1 related to the input original image data Determine the type of defect and output the determination result. The types of defects in the second embodiment are "missing tablets," "contamination with foreign matter," and "other defects."

To explain the determination in more detail, the trained AI model 400 calculates a score related to "missing tablets" and a score related to "foreign substance contamination", and if the calculated score exceeds the classification threshold, the trained AI model 400 The type of defect (“missing tablet” or “contamination with foreign matter”) related to the above is output as the determination result. On the other hand, when the trained AI model 400 determines the PTP sheet 1 as defective, and each calculated score is below the classification threshold, it outputs "other defects" as the determination result. .

Note that "tablet chipping" means that the tablet 5 is chipped, and "foreign object contamination" means that a foreign object other than the tablet 5, such as a fragment of the tablet 5, is present in the pocket part 2. "Other defects" means defects other than "missing tablets" and "contamination of foreign matter". Of course, other defect types other than "missing tablet" and "contamination with foreign matter" may also be used. Other types of defects include, for example, "tablet cracking" (cracking occurs in the tablet 5), "missing tablet" (tablet 5 is not filled in the pocket 2), and "tablet shape... Examples include "difference in size" (the shape and size of the tablet 5 do not match the manufactured product) and "tablet surface peeling" (the surface layer of the tablet 5 has peeled off).

Here, the learning process (initial learning process) performed when generating the initial AI model 400a will be described. Note that, prior to this learning process, a large number of image data related to the non-defective PTP sheet 1 and image data related to the defective PTP sheet 1 are prepared as learning data. In the second embodiment, the image data related to the defective PTP sheet 1 includes data of different types of defects. Further, each learning data is associated with information about the correct answer of the classification result. Therefore, for example, learning data related to a defective product is associated with information about the defective product and information about the type of defect.

As shown in FIG. 23, in the learning process, first, in step S601, an untrained neural network is prepared. Next, in step S602, classification results are obtained. That is, learning data prepared in advance is applied to the input layer of the neural network as input data. Then, the classification results output from the output layer of the neural network are obtained.

In the following step S603, the output classification result is compared with the correct classification result associated with the learning data, and it is determined whether the output classification result is correct.

Here, if the output classification result is correct, it is determined in step S605 whether the end condition of the learning process is satisfied. For example, if an affirmative determination is made in step S603 without going through the process of step S604, which will be described later, a predetermined number of times in a row, or if learning using all of the prepared learning data is repeated a predetermined number of times. If so, it is determined that the termination condition is satisfied. If the termination conditions are met, the neural network and its learning information (updated parameters, etc. to be described later) are stored as the initial AI model 400a in each of the used AI storage units 554 and 555, and the learning process is terminated. Note that the initial AI model 400a stored in the used AI storage unit 554 is updated to the current AI model 400z with additional learning processing and additional learning correction processing.

On the other hand, if the termination condition is not satisfied in step S605, the process returns to step S602 and the learning of the neural network is performed again.

If the output classification result is incorrect in step S603, network updating processing (neural network learning) is performed in step S604, and then the process returns to step S602 to repeat the series of processes described above.

In the network update process in step S604, a known learning algorithm is used to update the weights (parameters) in the neural network to more appropriate ones so that the output classification results are correct. By repeating the processing in steps S602 to S604 many times, the neural network can output more accurate classification results.

The image data storage section 556 stores various image data such as image data (shade pattern data) obtained by an image data acquisition section 55x described later. The image data (shade pattern data) obtained by the image data acquisition unit 55x corresponds to one PTP sheet 1, and represents a total of 10 shading patterns related to the pocket part 2 and the tablet 5. It is something. In the second embodiment, the image data storage section 556 constitutes "specific learning data storage means" and "misjudgment occurrence image data storage means."

The image data stored in the image data storage section 556 includes additional learning data 501 used for additional learning processing by the additional learning section 55h. Additional learning data 501 is image data that has caused a misjudgment in the classification result by the current AI model 400z among a plurality of image data (shaded pattern data) obtained by the image data acquisition unit 55x, and is based on predetermined conditions. (for example, selected by an operator, etc.). Therefore, the additional learning data 501 includes not only image data related to the non-defective PTP sheet 1 but also image data related to the defective PTP sheet 1. Note that the additional learning data 501 is associated with information regarding the correct answer of the classification result.

In this embodiment, “IM2(1)” to “IM2(n)” (see FIG. 28) are stored as additional learning data 501. For example, "IM2(1)" is image data used in additional learning processing for the neural network of the initial AI model 400a, and the AI model 400 generated by this additional learning processing is "AI2(1)". .

Further, the additional learning data 501 includes specific learning data 501a. The specific learning data 501a is additional learning data 501 used in generating the current AI model 400z from the initial AI model 400a. Therefore, in the second embodiment, all additional learning data 501 corresponds to specific learning data 501a.

Further, the image data stored in the image data storage unit 556 includes image data 502 in which a false determination has occurred. The erroneous determination image data 502 is grayscale pattern data (original image data) related to an erroneous determination when a classification result is erroneously determined in a confirmation process to be described later.

In the second embodiment, whether or not a misjudgment has occurred in the classification result is determined by visual inspection by an operator or the like. Of course, it is also possible to determine whether an erroneous determination has occurred using an inspection device provided separately other than the inspection device 23.

Then, by inputting information using the input unit 551 by an operator or the like, the image data 502 in which a misjudgment has occurred is identified among the image data (shaded pattern data) stored in the image data storage unit 556.

The additional learning information storage unit 557 stores information related to learning of the AI model 400, such as various information related to additional learning processing and additional learning correction processing (that is, learning processing by the additional learning unit 55h). In the second embodiment, in addition to the information listed in the first embodiment, for example, in association with the trained AI model 400, the threshold value for pass/fail judgment used by the AI model 400, the threshold value for the classification threshold values, etc. are stored. Further, in the second embodiment, a heat map (data obtained by emphasizing pixels that are the basis for determination in image data according to their importance) is stored for each trained AI model 400.

Next, the various functional units described above that constitute the inspection control device 55 will be explained. However, since the control units 55a to 55c, the image capture unit 55d, and the image processing unit 55e function in the same manner as the control units 52a to 52c, etc. in the first embodiment, a description thereof will be omitted. In the second embodiment, the camera 54, the image capture section 55d, and the image processing section 55e constitute an image data acquisition section 55x, and the image data acquisition section 55x acquires the shading pattern data regarding the PTP sheet 1. (original image data) is acquired. In the second embodiment, the image data acquisition section 55x constitutes "image data acquisition means."

The acquired shading pattern data (original image data) corresponds to one PTP sheet 1 and represents a total of 10 areas related to the pocket portion 2 and the tablet 5. The shading pattern data is stored in the image data storage section 556. In the second embodiment, some of the acquired grayscale pattern data becomes the additional learning data 501 and the erroneous determination image data 502.

The determination processing unit 55g obtains the classification result output from the current AI model 400z by inputting the grayscale pattern data (original image data) related to the PTP sheet 1 to be inspected into the current AI model 400z. The acquired classification results are the inspection results of the PTP sheet 1 and indicate whether the PTP sheet 1 is good or bad and the type of defect.

The additional learning unit 55h sequentially generates trained AI models 400 by performing additional learning processing of the neural network using the additional learning data 501. The additional learning unit 55h performs additional learning processing when the additional learning command is input.

In the additional learning process, as shown in FIG. 24, classification results are acquired in step S701. That is, for example, "IM2(1)" which is the newly stored unlearned additional learning data 501 is transferred to the duplicate current AI model 400z1 stored in the unused AI storage unit 555 (however, the duplicate current AI model 400z1 at this stage The model 400z1 is applied to the input layer of the neural network of "AI2(0)", which is the initial AI model 400a). Then, the classification results output from the output layer of the neural network are obtained. Note that the additional learning data 501 such as "IM2(1)" may be image data related to a good PTP sheet 1, or may be image data related to a defective PTP sheet 1. Further, when the additional learning data 501 is image data related to the defective PTP sheet 1, the additional learning data 501 is related to at least one type of defect.

In the following step S702, it is determined whether the classification result is correct. Here, if the classification result is correct, it is determined in step S704 whether the conditions for ending the additional learning process are satisfied. For example, if additional learning using the stored additional learning data 501 is repeated a predetermined number of times, it is determined that the termination condition is satisfied. If the termination condition is met, the neural network and its learning information (updated parameters, etc.) are stored in the unused AI storage unit 555 as a new trained AI model 400.

On the other hand, if the classification result is incorrect in step S702, a network update process (neural network learning) is performed in step S704, and then the process returns to step S701 to repeat the above series of processes. The network update process is similar to the process in step S604 in the learning process described above. By repeating the processing in steps S701 to S704, the neural network can output more accurate classification results. When the additional learning termination condition is satisfied, the additional learning process is terminated.

Further, the additional learning section 55h includes a factor data specifying section 55i and a confirmation section 55j for classification output AI as functional sections. In the second embodiment, the factor data specifying section 55i constitutes a "factor data specifying means", and the confirmation section 55j for classification output AI constitutes a "confirmation means for classification output AI".

When it is confirmed that a misjudgment has occurred in the classification result through the confirmation process described later, the factor data specifying unit 55i specifies the misjudgment occurrence image data 502 related to the misjudgment (that caused the misjudgment). Based on the degree of similarity, the additional learning data 501 that caused the misjudgment is identified from the specific learning data 501a as misjudgment factor data. In the second embodiment, the factor data specifying unit 55i specifies misjudgment factor data from the specific learning data 501a related to the PTP sheet 1 with the same classification result as the misjudged classification result. For example, if an erroneous determination of "foreign object contamination" occurs in the confirmation process, the erroneous determination factor data is specified from the specific learning data 501a related to the PTP sheet 1 of "foreign object contamination." Further, for example, if an erroneous determination of "good product" occurs in the confirmation process, the erroneous determination factor data is specified from the specific learning data 501a related to the "good product" PTP sheet 1. This means that when an erroneous classification result occurs, among the data used for additional learning (specific learning data 501a), image data related to the same classification result as the erroneous classification result This is because it is thought to be a factor causing this. Note that the identification of the erroneous determination factor data can be performed using the same method as in the first embodiment.

The classification output AI confirmation unit 55j executes confirmation processing to confirm the learning state of the trained AI model 400 newly generated by additional learning processing or the like.

The confirmation process is performed as follows. That is, as shown in FIG. 25, first, in step S801, when a new trained AI model 400 is generated by additional learning processing, it is checked whether the learning state of the AI model 400 is appropriate. do.

Specifically, the classification result is obtained by inputting the original image data related to the erroneous determination that triggered the generation of the AI model 400 to the newly generated trained AI model 400.

Next, it is determined whether the classification result is correct based on a comparison between the obtained classification result and the correct classification result associated with the input original image data.

If the classification result is correct (step S801: YES), it is determined in step S802 that the current AI model 400z can be replaced with the newly generated trained AI model 400, and the confirmation process ends. On the other hand, if the classification result is incorrect (step S801: No), an additional learning correction process to be described later is executed, and the confirmation process is ended.

In addition, in the confirmation process, additional learning data 501 other than the original image data related to the misjudgment, which triggered the additional learning process, was input to the newly generated trained AI model 400. It may also be determined whether the classification result is correct.

Further, if the classification result is determined to be incorrect in the confirmation process (step S801: No), the additional learning unit 55h makes an erroneous determination using the trained AI model 400 newly generated by the additional learning process. If this occurs, additional learning correction processing is executed in cooperation with the factor data identification unit 55i, the image output AI confirmation unit 55j, and the like.

In the additional learning correction process, as shown in FIG. 26, first, in step S851, the factor data specifying unit 55i specifies one or more erroneous determination factor data.

Next, in step S852, relearning processing is executed. In the relearning process, the neural network of the initial AI model 400a stored in the unused AI storage unit 555 is trained with the specific learning data 501a excluding the erroneous judgment factor data identified in step S851, and a new A trained AI model 400 is generated. For example, in step S851, "IM2(3)" is selected as the misjudgment factor data from "IM2(0)" to "IM2(n)" which are the specific learning data 501a stored in the image data storage unit 556. Suppose that is specified. In this case, in step S852, specific learning data 501a other than "IM2(3)" such as "IM2(0)" to "IM2(2)" and "IM2(4)" are added to the initial AI model 400a. ” to “IM2(n)”, the above additional learning process is performed. As a result, a new trained AI model 400 that has learned the specific learning data 501a other than "IM2(3)" which is the erroneous determination factor data is generated.

Next, in step S853, it is determined whether the learning state of the new trained AI model 400 is appropriate using the erroneous determination image data 502. This process is similar to the process in step S801 in the confirmation process. If the learning state is appropriate, the new learned AI model 400 is corrected so that an appropriate pass/fail judgment is made even for the image data 502 that caused the misjudgment. is stored in the unused AI storage unit 555, and then the additional learning correction process ends.

On the other hand, if the learning state is inappropriate, the process returns to step S851 and the erroneous determination factor data is identified again. Note that if it is determined in step S853 that the learning state is inappropriate, processing for changing the threshold used for classification may also be performed.

Then, by repeating the above process until the learning state becomes correct, a trained AI model 400 that can respond appropriately to the erroneous judgment image data 502 that caused the erroneous judgment is finally generated. Ru. The generated trained AI model 400 is stored in the unused AI storage unit 555.

Further, the additional learning section 55h includes an approval processing execution section 55k, an approval input reception section 55l, and a replacement section 55m as functional sections. In the second embodiment, the approval input receiving unit 55l constitutes an "approval input receiving means".

The approval process execution unit 55k, the approval input reception unit 55l, and the replacement unit 55m function similarly to the approval process execution unit 52k, the approval input reception unit 52l, and the replacement unit 52m in the first embodiment. Therefore, by the replacement unit 55m executing the replacement process, the current AI model 400z is replaced with the newly generated trained AI model 400, and the PTP sheet 1 is replaced using the new current AI model 400z. The inspection will begin.

The storage processing execution unit 55n basically functions in the same manner as the storage processing execution unit 52n in the first embodiment. Therefore, the storage processing execution unit 55n stores information related to learning of the AI model 400, such as various information related to additional learning processing and additional learning correction processing (that is, learning processing by the additional learning unit 55h), to the additional learning information storage unit 557. to remember. However, the storage process execution unit 55n in the second embodiment stores the learned AI models that were adopted in the approval process or rejected in the same process, in the additional learning information storage unit 557. 400, the threshold value for pass/fail determination and the threshold value for classification are stored in association with these learned AI models 400. Furthermore, the storage processing execution unit 55n stores a heat map for each trained AI model 400.

Briefly speaking, the flow of various processes (additional learning processing, etc.) by the additional learning section 55h described in detail above is as shown in FIG. 27.

That is, if an erroneous determination occurs in the classification result output by the inspection device 23 (current AI model 400z) (step S21: YES), an additional learning command is input by an operator or the like, and the additional learning process (step S22) is performed. ) is executed. That is, a new trained AI model 400 is generated by making the duplicate current AI model 400z1 stored in the unused AI storage unit 555 learn the image data of a good product or a defective product related to the misjudgment. .

Then, in a confirmation process (step S23), the learning state of the newly generated trained AI model 400 is confirmed. If it is determined that the learning state is appropriate (step S23-1: YES), it is determined in step S23-2-A that the current AI model 400z can be replaced with a new trained AI model 400. Ru.

On the other hand, if it is determined that the learning state is inappropriate (step S23-1: NO), additional learning correction processing is performed in step S23-2-B. Then, the additional learning correction process (step S23-2-B) is repeated until the classification result in the process of step S23-1 is correct, and the erroneous judgment that caused the erroneous judgment finally occurs. A trained AI model 400 that can appropriately respond to the image data 502 is obtained. In the additional learning correction process that is repeatedly performed, the process of re-identifying the erroneous determination factor data and learning the specific learning data 501a excluding the re-identified erroneous determination factor data is repeatedly performed.

Then, in step S23-2-A, if it is determined that the new trained AI model 400 can be replaced, an approval process (step S24) is executed, so that an operator etc. can perform the replacement. The required information is displayed on the display section 551. When the approval button is selected by the operator or the like (step S25: YES), the PTP packaging machine 11 is temporarily stopped, and the current AI model 400z is replaced with a new trained AI model 400 (step S25: YES). S26). On the other hand, if the operator or the like selects the disapproval button (step S25: No), no replacement is performed and the current AI model 400z is used as is.

As described in detail above, according to the second embodiment, when a misjudgment regarding the classification result of the PTP sheet 1 occurs, the neural network of the replica current AI model 400z1 is configured to cause the misjudgment to occur. By additionally learning the original image data, it is possible to generate a new trained AI model 400. Therefore, a trained AI model 400 with better inspection accuracy can be obtained.

Further, during additional learning, learning is performed not on the neural network of the current AI model 400z used for testing, but on the neural network of the duplicate current AI model 400z1 stored in the unused AI storage unit 555. Thereby, additional learning can be performed without stopping the test, and the speed and efficiency of the test can be further improved.

Furthermore, the confirmation unit 55j for classification output AI checks whether or not another misjudgment based on the original image data that caused the misjudgment will occur in the newly generated trained AI model 400, that is, whether or not another misjudgment will occur based on the original image data that caused the misjudgment. It is possible to easily check whether good inspection accuracy is achieved.

In addition, the classification output AI confirmation unit 55j confirms that the classification output AI is in a state where no erroneous judgment will occur (a state in which better test accuracy is achieved) before it is used for testing. It is determined that the currently used AI model 400z can be replaced with a newly generated trained AI model 400. Therefore, it is possible to more reliably prevent an inappropriate AI model 400 that may cause erroneous determination from being erroneously used in an inspection.

On the other hand, if the classification output AI confirmation unit 55j confirms that an erroneous judgment occurs in the test using the newly generated trained AI model 400, in other words, it is considered that an erroneous judgment has occurred due to overfitting. If so, the additional learning unit 55h performs learning on the neural network of the initial AI model 400a, which is the AI model 400 that has not undergone additional learning, and generates a new trained AI model 400. At this time, among the specific learning data 501a which is the additional learning data 501 used to generate the current AI model 400z from the initial AI model 400a, the PTP sheet 1 with the same classification result as the misjudged classification result is Among such data, data that is considered to be the cause of the misjudgment is specified as misjudgment factor data.

The additional learning unit 55h then generates a new trained AI model 400 by causing the neural network of the initial AI model 400a to learn data obtained by excluding the false determination factor data from the specific learning data 501a. Therefore, when the learning state is determined to be inappropriate by the classification output AI confirmation unit 55j, while making use of the past additional learning data 501 (specific learning data 501a), it is possible to further reduce the occurrence of erroneous judgments due to overfitting. The learning state can be modified to ensure suppression. As a result, it is possible to more reliably prevent erroneous determination regarding the classification of the PTP sheet 1, and improve inspection accuracy.

In addition, if the classification output AI confirmation unit 55j determines that the learning state is inappropriate, the information is stored in the unused AI storage unit 555 instead of in the neural network of the current AI model 400z used for inspection. Learning is performed on the neural network of the stored initial AI model 400a. Therefore, processing related to learning of a neural network and processing related to extraction (identification) of data causing false judgments, which can be a hugely repetitive process, can be performed without stopping the test. Thereby, a large number of PTP sheets 1 can be inspected more quickly and more efficiently.

In addition, when it is confirmed by the classification output AI confirmation unit 55j that a misjudgment has occurred, the misjudgment generated image data 502, which is the original image data related to the misjudgment (original image data that caused the misjudgment), is Misjudgment factor data can be specified from the specific learning data 501a based on the degree of similarity. Therefore, misjudgment factor data can be specified more accurately, and as a result, the accuracy of the test using the newly generated trained AI model 400 can be more reliably improved.

Further, the current AI model 400z is replaced with the newly generated trained AI model 400 only after approval by an operator or the like. Therefore, it is possible to more reliably prevent unintentional replacement of the current AI model 400z with the newly generated trained AI model 400. This makes it possible to more reliably prevent problems caused by unintended replacement.

Note that the present invention is not limited to the content described in the above embodiment, and may be implemented as follows, for example. Of course, other applications and modifications not exemplified below are also possible.

(a) In the above embodiment, the initial AI models 200a and 400a constitute the "old identification means". On the other hand, AI models 200,400 that have not undergone some additional learning processing, such as "AI(1)" to "AI(n-1)" and "AI2(1)" to "AI2(n -1)", the "old identification means" may be configured. Therefore, for example, "AI(99)" may constitute the "old identification means" (see FIG. 29). In addition, in such a configuration, data related to "AI(0)" to "AI(98)" and additional learning of these AI models 200 ("AI(0)" to "AI(98)") If the used additional learning data 301 (“IM(1)” to “IM(99)”) is not stored in the image data storage unit 526, it is possible to determine whether the defective PTP sheet 1 is a good product, etc. When a misjudgment is made, for example, the misjudgment factor data is identified from the additional learning data 301 after "IM (100)", and the additional learning data after "IM (100)" excluding this misjudgment factor data is generated. 301 may be used to perform relearning processing of "AI(99)".

In addition, AI models 200,400 that have not undergone some or all additional learning processing, such as "AI(0)" to "AI(n-1)" and "AI2(0)" to "AI2(n -1)" may respectively constitute the "old identification means", and the unused AI storage units 525, 555 may store all of these AI models 200, 400. Further, in this case, the relearning process may be performed on the AI model 200 immediately before additionally learning the error determination factor data. For example, if the misjudgment factor data is identified as "IM(3)", relearning processing is performed on "AI(2)", which is the AI model 200 immediately before additionally learning this misjudgment factor data. You can do it like this.

Furthermore, the "old identification means" may be changed as appropriate in relation to the current AI model 200z. For example, with the current AI model 200z as a reference, the AI model 200 at a stage before additional learning processing has been performed the most recent predetermined number of times (for example, 10 times, etc.) may be used as the "old identification means". In this case, as the current AI model 200z is updated, the AI model 200 serving as the "old identification means" is changed.

(b) In the above embodiment, the initial AI models 200a and 400a are assumed to have been subjected to learning processing in advance, but the initial AI models 200a and 400a may be those that have not undergone learning processing good. In this case, the learned AI models 200, 400 may be generated by performing the learning process described in the above embodiment on the initial AI models 200a, 400a by the additional learning units 52h, 55h. Therefore, the additional learning units 52h and 55h may be capable of executing both the learning process and the additional learning process.

(c) In the embodiment described above, the misjudgment factor data is specified based on the degree of similarity with the defective image data 302 and the misjudgment occurrence image data 502, but the identification method may be changed as appropriate. For example, the additional learning data 301, 501 used in the most recent additional learning process may be specified as the erroneous determination factor data.

(d) In the above embodiment, the inspection devices 21 and 23 are used to inspect the PTP sheet 1, but the object to be inspected by the inspection devices 21 and 23 is not limited to the PTP sheet 1. Therefore, for example, the tablet 5 may be inspected by the inspection devices 21 and 23.

Note that the type, shape, etc. of the tablets are not limited to the above embodiments. For example, tablets include not only medicines but also tablets used for consumption. In addition, tablets include not only plain tablets, but also sugar-coated tablets, film-coated tablets, orally disintegrating tablets, enteric-coated tablets, gelatin-coated tablets, and various other capsules such as hard capsules and soft capsules. This also includes tablets.

Furthermore, for example, a blister pack 800 other than a PTP sheet as shown in FIG. 30 can be used as the object to be inspected. Blister pack 800 has a pocket portion 801 for accommodating predetermined contents.

(e) The configuration of the AI models 200, 400 as "identification means" and the learning method thereof are not limited to the above embodiments. For example, when performing neural network learning processing, reconstructed image data acquisition processing, etc., it may be configured to perform processing such as normalization on various data as necessary. Furthermore, the structure of the neural network 190 is not limited to that shown in FIG. 9, and may have a configuration in which a pooling layer is provided after the convolutional layer 193, for example. Of course, the number of layers of the neural network 190, the number of nodes in each layer, the connection structure of each node, etc. may be different.

Further, in the above embodiment, the AI model 200 (neural network 190) is a generative model having the structure of a convolutional autoencoder (CAE), but is not limited to this, for example, a variational autoencoder (VAE) It is also possible to use a generative model with the structure of a different type of autoencoder, such as (Variational Autoencoder).

Further, in the above embodiment, the neural network 190 is configured to learn using the error backpropagation method, but the configuration is not limited to this, and a configuration may be adopted in which learning is performed using various other learning algorithms.

In addition, the neural network may be configured by a circuit dedicated to AI processing, such as a so-called AI chip. In that case, only learning information such as parameters is stored in the AI storage units 524, 525 (554, 555), and the AI processing dedicated circuit reads this information and sets it in the neural network to configure the AI model 200, 400. It is also possible to do so.

(f) In the second embodiment, the trained AI model 400 is capable of classifying the input original image data into non-defective products, defective products, and various types of defects. It is sufficient if it includes good/defective products. Therefore, the trained AI model 400 may classify products into non-defective products or defective products, but may not classify products based on the type of defect.

1...PTP sheet (inspection object), 11...PTP packaging machine, 21, 23...inspection device, 52f...reconstructed image data acquisition section (reconstructed image data acquisition means), 52g...judgment section (judgment means), 52h , 55h... Additional learning unit (additional learning means), 52i, 55i... Factor data specifying unit (factor data specifying unit), 52j... Confirmation unit for image output AI (confirmation unit for image output AI), 52x, 55x... Image data acquisition unit (image data acquisition means), 55j... Confirmation unit for classification output AI (confirmation unit for classification output AI), 190... Neural network, 191... Encoder unit (encoding unit), 192... Decoder unit ( decoding unit), 200, 400...AI model (identification means), 200a, 400a...initial AI model (trained identification means, old identification means), 200z, 400z...current AI model (trained identification means, current identification means) ), 200z1, 400z1...Replicated current AI model (trained identification means, replicated current identification means), 525,555...Unused AI model storage section (identification means storage means), 526,556...Image data storage section (specific learning data storage means, defective image data storage means).

Claims (9)

An inspection device for inspecting a predetermined inspection object,
an image data acquisition means capable of acquiring image data related to the inspection target;
It is possible to output image data according to the input image data by using a neural network that has an encoding unit that extracts feature quantities from input image data and a decoding unit that reconstructs image data from the feature quantities. a trained identification means, which is the learned identification means, which is generated by causing the neural network of the configured identification means to learn only image data related to a non-defective inspection object as learning data;
additional learning means for generating a new learned identification means by causing the neural network of the identification means to additionally learn image data related to a non-defective inspection object as additional learning data;
Reconstructed image data acquisition means capable of inputting original image data, which is image data obtained by the image data acquisition means, into the learned identification means and acquiring reconstructed image data as reconstructed image data;
a determining unit capable of comparing the original image data and the reconstructed image data obtained by inputting the original image data to the learned identifying unit, and determining the quality of the inspection target based on the comparison result;
identification means storage means for storing the old identification means, which is the identification means at a stage where some or all of the additional learning by the additional learning means has not been performed;
Specific learning data that stores all of the specific learning data that is the additional learning data that was used to generate the current identification means that is the learned identification means that is used for inspecting the inspection object from the old identification means. and storage means;
The additional learning means inputs the original image data into the current identification means by the reconstructed image data acquisition means, and performs a quality determination by the determination means based on the reconstructed image data, thereby inspecting defective products. When the target object is erroneously determined to be a good product, the neural network of the old identification means stored in the identification means storage means receives the erroneous judgment which is the additional learning data identified as the cause of the erroneous judgment. An inspection device characterized in that it is configured to be able to generate a new learned identification means by learning the specific learning data excluding factor data. The identification means storage means stores, separately from the current identification means, a duplicate current identification means which is the identification means having the same content as the current identification means,
The additional learning means inputs the original image data into the current identification means by the reconstructed image data acquisition means, and determines whether the inspection object is a non-defective product based on the quality judgment of the judgment means based on the reconstructed image data. When an object is erroneously determined to be defective, the neural network of the duplicate current identification means stored in the identification means storage means is additionally trained with the original image data that caused the erroneous determination. 2. The inspection device according to claim 1, wherein the inspection device is configured to be able to generate a new learned identification means. a defective image data storage means for storing defective image data that is the original image data related to the erroneous determination when the defective inspection object is erroneously determined to be a good product by the quality determination of the determination means;
2. The inspection apparatus according to claim 1, further comprising factor data specifying means for specifying the misjudgment factor data from the specific learning data based on the degree of similarity to the defective image data. Based on the reconstructed image data that was reconstructed by inputting the original image data related to the erroneous judgment that triggered the generation of the learned discrimination means to the newly generated learned discrimination means, 2. The inspection apparatus according to claim 1, further comprising a confirmation unit for image output AI that confirms the learning state of the newly generated learned identification unit by making the judgment unit perform a quality judgment. a display means capable of displaying information related to learning processing by the additional learning means and confirmation results by the image output AI confirmation means;
5. The inspection apparatus according to claim 4, further comprising: approval input receiving means capable of accepting input regarding approval for replacing the currently used identification means with the newly generated learned identification means. An inspection device for inspecting a predetermined inspection object,
an image data acquisition means capable of acquiring image data related to the inspection target;
The neural network of the identification means, which is configured to classify input image data according to predetermined items and output the classification results, is trained by using image data related to inspected objects for good and defective products as learning data. a learned identification means that is the learned identification means that is generated;
additional learning means for generating a new learned identification means by causing the neural network of the identification means to additionally learn image data related to the inspection object as additional learning data;
The old identification means, which is the identification means at a stage where some or all of the additional learning by the additional learning means has not been performed, and the current identification means, which is the learned identification means, which is used for the inspection of the inspection object. Separately, an identification means storage means for storing a duplicate current identification means which is the identification means having the same content as the current identification means;
specific learning data storage means for storing all specific learning data that is the additional learning data used in generating the current identification means from the old identification means;
The additional learning means is configured to control the neural network of the duplicate current identification means stored in the identification means storage means when an erroneous determination occurs in the classification result output by inputting the original image data to the current identification means. , it is possible to generate a new learned identification means by additionally learning the original image data that caused the misjudgment,
Based on the classification result output by inputting the original image data related to the erroneous judgment that triggered the generation of the learned identification means to the newly generated learned identification means, If the classification output AI confirmation means for confirming the learning state of the learned identification means determines that the learning state is appropriate, the currently used identification means is replaced with the newly generated learned identification means. While it is determined that it can be replaced with
An erroneous judgment occurs in the classification result outputted by inputting the original image data to the newly generated trained identification means, and the learning state is determined to be inappropriate by the classification output AI confirmation means. If the neural network of the old identification means stored in the identification means storage means is data related to the inspection object with the same classification result as the incorrectly determined classification result, and An inspection device capable of generating a new learned identification means by learning the specific learning data excluding erroneous judgment factor data that is data specified as a factor of erroneous judgment. An erroneous judgment occurred in the classification result output by inputting the original image data to the newly generated trained identification means, and the learning state was determined to be inappropriate by the classification output AI confirmation means. a misjudgment occurrence image data storage means for storing misjudgment occurrence image data that is the original image data related to the misjudgment in the case;
7. The inspection apparatus according to claim 6, further comprising factor data specifying means for specifying the misjudgment factor data from the specific learning data based on the degree of similarity to the image data in which misjudgment has occurred. a display means capable of displaying information related to learning processing by the additional learning means and confirmation results by the classification output AI confirmation means;
7. The inspection apparatus according to claim 6, further comprising: approval input receiving means capable of accepting an input regarding approval for replacing the currently used identification means with the newly generated learned identification means. A PTP packaging machine equipped with the inspection device according to claim 1 or 6, which inspects a tablet or a PTP sheet as an object to be inspected.

Images