JP2021026450A - Computer program, information processing device, information processing method and generation method of learned model - Google Patents

Abstract

To provide a computer program which can realize the proper authenticity determination of a card regardless of a printer used in the fabrication, and provide an information processing device, information processing method and generation method of a learned model which realizes the proper authenticity determination.SOLUTION: A processing element of a control unit of a determination device executes authenticity determination processing by executing a computer program. In the authenticity determination processing, the control unit acquires image data of a card, extracts partial image data being a portion of the acquired image data, inputs the extracted partial image data to an authenticity level estimation model, and calculates the authenticity level of the card as the output of the authenticity level estimation model. The authenticity level estimation model is a learned model that has learned a relation between the partial image data being a portion of the image data of the card and the authenticity level indicating such a possibility that the card is authentic.SELECTED DRAWING: Figure 8

Description

The present invention relates to a computer program, an information processing apparatus, an information processing method, and a method for generating a trained model.

Based on the image data, a card (for example, see Patent Document 1) for determining whether or not the card, for example, the identification card shown in the image of the image data is genuine has been proposed. When a counterfeit card is created by printing using an area gradation printer, color unevenness occurs in the part of the counterfeit card corresponding to the part filled with one color in the real card. .. When the image of the image data has color unevenness, the determination device described in Patent Document 1 determines that the card shown in the image of the image data is a counterfeit card.

JP-A-2017-215739

However, in the determination device described in Patent Document 1, it is possible to determine whether or not a counterfeit card created by using a printer different from the area gradation type printer is genuine, that is, to determine the authenticity. It may not be possible.

The present invention has been made in view of such circumstances, and an object of the present invention is a computer program, an information processing device, and an information processing method capable of appropriately determining the authenticity of a card regardless of the printer used for counterfeiting. And to provide a method of generating a trained model that realizes an appropriate authenticity judgment.

The computer program according to one aspect of the present invention acquires the image data of the card, extracts the partial image data which is a part of the acquired image data, and uses the extracted partial image data as a part of the image data of the card. The relationship between the image data and the authenticity of the card is input to the learned model, and the computer is made to execute the process of calculating the authenticity.

The information processing apparatus according to one aspect of the present invention includes an acquisition unit that acquires image data of a card, an extraction unit that extracts partial image data that is a part of the image data acquired by the acquisition unit, and an extraction unit that extracts the partial image data. It includes an input unit for inputting the partial image data to the trained model in which the relationship between the partial data which is a part of the image data of the card and the authenticity of the card is learned, and a calculation unit for calculating the authenticity.

In the information processing method according to one aspect of the present invention, the image data of the card is acquired, the partial image data which is a part of the acquired image data is extracted, and the extracted partial image data is a part of the image data of the card. The computer executes a process of inputting the relationship between the partial data and the authenticity of the card into the trained model that has learned the relationship and calculating the authenticity.

In the trained model according to one aspect of the present invention, the teacher data in which the partial image data which is a part of the image data of the card is associated with the correct answer label indicating whether or not the card is genuine is acquired and acquired. Based on the teacher data, the computer executes a process to generate a trained model that learns the relationship between the partial image data and the authenticity of the card.

According to the above-mentioned computer program, information processing device, and information processing method, the authenticity of the card can be appropriately determined regardless of the printer used for counterfeiting.
According to the above-mentioned method for generating a trained model, it is possible to generate a trained model that realizes an appropriate authenticity determination.

It is explanatory drawing of the outline of the determination apparatus in Embodiment 1. FIG. It is explanatory drawing of a partial image data. It is a schematic diagram of an authenticity estimation model. It is a block diagram which shows the main part structure of a model generator. It is a flowchart which shows the procedure of the authenticity estimation model generation processing. It is a flowchart which shows the procedure of the authenticity estimation model generation processing. It is a block diagram which shows the main part structure of the determination device. It is a flowchart which shows the procedure of the authenticity determination processing. It is explanatory drawing of the authenticity determination. It is explanatory drawing of another example of a partial image data. It is explanatory drawing of another example of a partial image data. It is a block diagram which shows the main part structure of the determination device in Embodiment 2. It is explanatory drawing of the 1st partial image data and the 2nd partial image data. It is a flowchart which shows the procedure of the authenticity determination processing. It is a flowchart which shows the procedure of the authenticity determination processing. It is a block diagram which shows the main part structure of the determination apparatus in Embodiment 3. It is a figure which shows the model selection table. It is a flowchart which shows the procedure of the authenticity determination processing. It is a block diagram which shows the main part structure of the determination device in Embodiment 4. It is explanatory drawing of the photograph of a card. It is a block diagram which shows the main part structure of the re-learning system in Embodiment 5. It is a flowchart which shows the procedure of the process performed by the determination apparatus and the model generation apparatus. It is explanatory drawing of the outline of the authentication apparatus in Embodiment 6. It is an external view of the authentication device. It is an example diagram of the target card. It is a schematic diagram of a type identification model. It is explanatory drawing of the reduced image data. It is a block diagram which shows the main part structure of a model generator. It is explanatory drawing of the reduced image data included in the 2nd teacher data. It is a flowchart which shows the procedure of the type specific model generation processing. It is a flowchart which shows the procedure of the type specific model generation processing. It is a block diagram which shows the main part structure of an authentication apparatus. It is explanatory drawing of the data stored in the storage part. It is a figure which shows the table for authentication. It is a chart which shows the contents of a read rule database. It is a flowchart which shows the procedure of a specific process. It is a flowchart which shows the procedure of a specific process. It is a flowchart which shows the procedure of an authentication process. It is explanatory drawing of the top and bottom of the card in Embodiment 7. It is a schematic diagram of a type identification model. It is the schematic of the relearning system in Embodiment 8. It is a block diagram which shows the main part structure of an authentication apparatus. It is a block diagram which shows the main part structure of a model generator. It is a flowchart which shows the procedure of the process performed by the authentication machine and the model generator. It is a block diagram which shows the main part structure of the authentication system in Embodiment 9.

Hereinafter, the present invention will be described in detail with reference to the drawings showing the embodiments thereof.
(Embodiment 1)
FIG. 1 is an explanatory diagram of an outline of the determination device 1 according to the first embodiment. The image data of the image in which the card is shown is input to the determination device 1. Examples of the card include a driver's license, an Individual Number Card, an Individual Number Notification Card, a Basic Resident Register Card, a Special Permanent Resident Certificate, a Residence Card, a Passport Card, or a Social Security Card. Further, the authenticity estimation model R is stored in the determination device 1. The authenticity estimation model R is a learned model that has learned the relationship between the partial image data that is a part of the image data of the card and the authenticity that indicates the probability that the card is genuine.

When the image data of the card is input, the determination device 1 extracts a plurality of partial image data from the input image data. The determination device 1 inputs the extracted partial image data into the authenticity estimation model R, and calculates the authenticity as the output of the authenticity estimation model R. The determination device 1 inputs the extracted plurality of partial image data into the authenticity estimation model R separately, and calculates a plurality of authenticity. The determination device 1 determines whether or not the card shown in the input image data is genuine based on the calculated authenticity. When the determination device 1 determines that the card is genuine, for example, it reads the personal information written on the card from the input image data, authenticates the user based on the read personal information, or reads the individual. Register information, etc. The model generation device 2 generates an authenticity estimation model R. The determination device 1 functions as an information processing device.

FIG. 2 is an explanatory diagram of partial image data. FIG. 2 shows an image of the card. Here, a driver's license is shown as an example of a card. Vertical line hatches indicate parts filled with one color, and diagonal line hatches indicate parts with diagonal lines. The color used for filling is different from white. The background color of the part other than the hatching of the vertical line is white. Vertical hatching and diagonal hatching are similarly defined in drawings other than FIG.

In FIG. 2, four partial images B10, B11, B12, and B13 corresponding to the four partial image data are shown. Each of the four partial images B10, B11, B12, and B13 is surrounded by a broken line. Each of the partial images B10, B12, and B13 is an image of a curved portion on the card. The partial image B11 is an image of a straight line portion on the card.
The number of partial image data is not limited to 4, and may be 2, 3, or 5 or more. Further, the image of the partial image data is not limited to the image of the straight line portion or the curved portion.

A card of a common type, for example, image data of a driver's license is input to the determination device 1. The determination device 1 stores coordinate data indicating a plurality of coordinates in the image of the image data. The plurality of coordinates indicated by the coordinate data indicate the locations of the plurality of partial images corresponding to the plurality of partial image data in the image of the card. When the partial image has a rectangular shape, the location of the partial image is indicated by, for example, the coordinates of the four corners of the partial image. When the partial image has a rectangular shape and the vertical and horizontal lengths of the partial image are fixed, the coordinates of the coordinate data are, for example, the positions of the upper left, lower left, upper right, or lower right corners of the partial image. Shown. In this case, one coordinate indicates the location of one partial image.

FIG. 3 is a schematic diagram of the authenticity estimation model R. The authenticity estimation model R is a trained model of machine learning including deep learning, and is, for example, a CNN (Convolutional Neural Network). The authenticity estimation model R has an input layer, a plurality of intermediate layers, and an output layer. FIG. 3 shows an example in which the number of intermediate layers is 2. The number of intermediate layers may be 3 or more.

Each of the input layer, middle layer and output layer has one or more neurons. The neurons in each layer are unidirectionally connected to the neurons in the previous and next layers with the desired weight and bias. A vector having the same number of components as the number of neurons in the input layer is input to the authenticity estimation model R as input data. For example, a plurality of pixel values constituting partial image data are input to the input layer of the authenticity estimation model R.

In addition, a layer for convolution may be included in a plurality of intermediate layers. In this layer, an appropriate filter including a convolution filter is applied to the input data, and a plurality of data obtained after applying the filter is output to the next layer, for example, the pouring layer.

The input values input to each neuron in the input layer are input to the first intermediate layer. In this intermediate layer, the output is calculated using an activation function that includes weights and biases. The calculated value is input to the next intermediate layer. In the same manner below, the output of the output layer is transmitted to the subsequent layers one after another until the output is calculated. Parameters such as weights and biases that connect neurons are learned using the backpropagation method.

The model generation device 2 performs machine learning using the teacher data in which the partial image data is associated with the correct answer label indicating whether or not the card is genuine, and generates the authenticity estimation model R. The correct label is one piece of information. In machine learning, the model generator 2 optimizes the parameters used for the calculation in the intermediate layer.

The output layer of the authenticity estimation model R outputs the probability that the card shown in the image of the partial image data input to the input layer is genuine, that is, the authenticity and the probability that this card is fake. The probability is represented by a real number greater than or equal to zero and less than or equal to 1. In the output layer, a softmax function is used and adjusted so that the sum of the two probabilities is 1, i.e. 100%. By subtracting the probability that the card is fake from 1, the probability that the card is genuine is calculated, so the probability that the card is fake also indicates the authenticity.

FIG. 4 is a block diagram showing a main configuration of the model generator 2. The model generation device 2 includes an input unit 20, an operation unit 21, a temporary storage unit 22, a storage unit 23, and a control unit 24. These are connected to the internal bus 25.

A plurality of teacher data and first evaluation data and second evaluation data for evaluating the authenticity estimation model R are input to the input unit 20. The control unit 24 acquires the teacher data, the first evaluation data, and the second evaluation data input to the input unit 20. Each of the first evaluation data and the second evaluation data includes the partial image data and the correct answer label indicating whether or not the card shown in the image of the partial image data is genuine, like the teacher data.

The partial image of the partial image data included in the teacher data, the first evaluation data, and the second evaluation data is one of a plurality of locations indicated by the coordinate data stored in the determination device 1 in the image of the card having the same type. It is a partial image corresponding to one.

The operation unit 21 has an input interface such as a keyboard or a mouse. The user operates the input interface. The operation unit 21 notifies the control unit 24 of the operation performed in the input interface.
The temporary storage unit 22 is, for example, a RAM (Random Access Memory). Data is temporarily stored in the temporary storage unit 22. When the power supply to the temporary storage unit 22 is stopped, the data stored in the temporary storage unit 22 is erased.

The storage unit 23 is a non-volatile memory. The authenticity estimation model R and the computer program P2 are stored in the storage unit 23. The control unit 24 has a processing element such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tenser Processing Unit), and also serves as a calculation unit for performing calculations. The processing element (computer) of the control unit 24 executes the computer program P2 to store the teacher data, the first evaluation data, and the second evaluation data in the storage unit 23, and the authenticity estimation model R. Executes the authenticity estimation model generation process that generates.

The computer program P2 may be provided to the model generation device 2 by using the non-temporary storage medium A2 readablely stored by the processing element included in the control unit 24. In this case, the computer program P2 read from the storage medium A2 by a reading device (not shown) is stored in the storage unit 23. The storage medium A2 is an optical disk, a flexible disk, a magnetic disk, a magnetic optical disk, a semiconductor memory, or the like. The optical disk is a CD (Compact Disc) -ROM (Read Only Memory), a DVD (Digital Versatile Disc) -ROM, or a BD (Blu-ray (registered trademark) Disc). The magnetic disk is, for example, a hard disk. Further, the computer program P2 may be downloaded from an external device (not shown) connected to a communication network (not shown), and the downloaded computer program P2 may be stored in the storage unit 23.

The number of processing elements included in the control unit 24 may be two or more. In this case, a plurality of processing elements may jointly execute the storage process and the authenticity estimation model generation process according to the computer program P2.

A plurality of teacher data, first evaluation data, and second evaluation data are input to the input unit 20 from a device (not shown). When these data are input to the input unit 20, the control unit 24 executes a storage process. In the storage process, the control unit 24 acquires a plurality of teacher data, first evaluation data, and second evaluation data input to the input unit 20, and stores the acquired data in the storage unit 23. After that, the control unit 24 ends the storage process.

5 and 6 are flowcharts showing the procedure of the authenticity estimation model generation process. The user of the model generation device 2 performs an execution operation instructing the operation unit 21 to execute the authenticity estimation model generation process. When the execution operation is performed, the control unit 24 executes the model generation process when a plurality of teacher data, the first evaluation data, and the second evaluation data are stored in the storage unit 23. The storage unit 23 stores the values of the first variable and the second variable used in the authenticity estimation model generation process. The control unit 24 sets these values separately.

In the authenticity estimation model generation process, the control unit 24 first forms a path of the authenticity estimation model R (step S1). Specifically, the control unit 24 has a pathway between a plurality of neurons regarding the input layer and an intermediate layer adjacent to the input layer, a pathway between a plurality of neurons regarding two adjacent intermediate layers, and an intermediate layer and an intermediate layer. It forms a pathway between multiple neurons with respect to the output layer adjacent to the layer. Parameters such as weight and bias between neurons are set as appropriate.

After executing step S1, the control unit 24 sets the values of the first variable and the second variable to zero (step S2), and the plurality of teacher data, the first evaluation data, and the data for the first evaluation stored in the storage unit 23. The second evaluation data is loaded into the temporary storage unit 22 (step S3). Next, the control unit 24 selects one teacher data from the temporary storage unit 22 (step S4), and inputs the partial image data included in the selected teacher data into the authenticity estimation model R (step S5). The input of data to the authenticity estimation model R means the input to the input layer of the authenticity estimation model R. Next, the control unit 24 calculates the authenticity as the output of the authenticity estimation model R (step S6). As described above, as the output of the authenticity estimation model R, in addition to the authenticity, the probability that the card is fake is also output.

Next, the control unit 24 increments the value of the first variable by 1 (step S7), and determines whether or not the value of the first variable is the first reference value (step S8). The first reference value is stored in the storage unit 23, is a preset constant value, and is an integer of 2 or more. When the control unit 24 determines that the value of the first variable is not the first reference value (S8: NO), the control unit 24 executes step S4 again. In step S4 executed again, the control unit 24 selects one teacher data that has not been selected from the plurality of teacher data loaded in step S3.

The control unit 24 repeatedly executes steps S4 to S7 until the value of the first variable becomes the first reference value, and continues to calculate the authenticity. The first reference value is the so-called batch size. When the control unit 24 determines that the value of the first variable is the first reference value (S8: YES), the plurality of authenticities obtained by repeatedly executing step S6 and the plurality of authenticities selected in step S4. The parameters of the authenticity estimation model R are updated based on the contents indicated by the correct label of the teacher data of (step S9). Next, the control unit 24 determines whether or not all the teacher data stored in the temporary storage unit 22 has been selected in step S4 (step S10).

When it is determined that all the teacher data has not been selected (S10: NO), the control unit 24 sets the value of the first variable to zero (step S11), and executes step S4 again. In step S4 executed again, the control unit 24 selects one teacher data that has not been selected from the plurality of teacher data loaded in step S3. The parameters are repeatedly updated based on the plurality of authenticities obtained by repeatedly executing step S6 until all the teacher data are selected.

When it is determined that all the teacher data has been selected (S10: YES), the control unit 24 evaluates the authenticity estimation model R using the first evaluation data (step S12). In step S12, the control unit 24 inputs the partial image data included in the first evaluation data into the authenticity estimation model R, and calculates the authenticity. The control unit 24 confirms that an appropriate authenticity is calculated as an output of the authenticity estimation model R based on the authenticity and the content indicated by the correct answer label included in the first evaluation data. If an appropriate authenticity has not been calculated, for example, the authenticity estimation model generation process may be terminated. In this case, for example, the number of teacher data is increased, and the authenticity estimation model generation process is executed again.

After executing step S12, the control unit 24 increments the value of the second variable by 1 (step S13), and determines whether or not the value of the second variable is the second reference value (step S14). The second reference value is stored in the storage unit 23, is a preset constant value, and is an integer of 2 or more. When the control unit 24 determines that the value of the second variable is not the second reference value (S14: NO), the control unit 24 sets the value of the first variable to zero (step S15), and executes step S4 again. At this time, it is assumed that there is no selected teacher data among the plurality of teacher data loaded in step S3, and step S4 is executed. The parameters of the authenticity estimation model R are updated using the plurality of teacher data loaded in step S3 until the value of the second variable becomes the second reference value. The number of times each teacher data is used is the same as the number of second reference values. The second reference value is the so-called epoch number.

When the control unit 24 determines that the value of the second variable is the second reference value (S14: YES), the control unit 24 evaluates the authenticity estimation model R using the second evaluation data (step S16). In step S16, the control unit 24 inputs the partial image data included in the second evaluation data into the authenticity estimation model R, and calculates the authenticity as the output of the authenticity estimation model R. The control unit 24 confirms that the appropriate authenticity is calculated as the output of the authenticity estimation model R based on the calculated authenticity and the content indicated by the correct answer label included in the second evaluation data. .. If an appropriate authenticity has not been calculated, for example, the authenticity estimation model generation process may be terminated. In this case, for example, the number of teacher data is increased, and the authenticity estimation model generation process is executed again.

After executing step S16, the control unit 24 stores the authenticity estimation model R whose parameters have been updated in the storage unit 23 (step S17), and the teacher data used for updating the authenticity estimation model R, the first evaluation. Data and the second evaluation data are deleted from the temporary storage unit 22 and the storage unit 23 (step S18). After executing step S18, the control unit 24 ends the authenticity estimation model generation process.

As described above, in the authenticity estimation model generation process, the control unit 24 updates the parameters of the authenticity estimation model R based on the plurality of teacher data stored in the storage unit 23. By updating the parameters, the authenticity estimation model R is generated.
When the model generation device 2 is connected to the determination device 1 via a network, the model generation device 2 transmits the authenticity estimation model R generated by the control unit 24 to the determination device 1. The determination device 1 updates the authenticity estimation model R stored in itself to the authenticity estimation model R received from the model generation device 2. The determination device 1 is installed at a place where authentication based on personal information or registration of personal information is performed.

As described above, in the teacher data, the partial image data which is a part of the image data of the card is associated with a correct answer label indicating whether or not the card is genuine. Therefore, in the authenticity estimation model generation process, the control unit 12 generates an authenticity estimation model R that realizes an appropriate authenticity determination.

The types of partial image data included in all the teacher data used to generate the authenticity estimation model R are common. The type of partial image data differs depending on the part of the card shown in the image of the partial image data. As one type of partial image data, as shown in FIG. 2, partial image data of a straight line portion or a curved portion on a card can be mentioned.

When a card, for example, a driver's license, is forged using a printer, the forged card has a wider line width and is more likely to have uneven line color than a real card. Therefore, it is effective to use the partial image data of the straight line portion or the curved line portion on the card as one type of partial image data. Further, as the partial image data of the teacher data, the partial image data of the counterfeit card created by using various printers is used. As a result, an appropriate authenticity is calculated as the output of the authenticity estimation model R regardless of the printer used for forging the card.

Next, the determination device 1 will be described in detail. FIG. 7 is a block diagram showing a main configuration of the determination device 1. The determination device 1 includes an input unit 10, a storage unit 11, and a control unit 12. These are connected to the internal bus 13.
The image data of the card is input to the input unit 10. The control unit 12 acquires the image data input to the input unit 10. The control unit 12 functions as an acquisition unit.

The storage unit 11 is a non-volatile memory. The storage unit 11 stores the authenticity estimation model R generated by the model generation device 2, the coordinate data Z indicating a plurality of coordinates, and the computer program P1. As described in the description of the partial image data with reference to FIG. 2, the plurality of coordinates of the coordinate data Z indicate the locations of the plurality of partial images in the image of the card. The control unit 12 has a processing element such as a CPU, GPU, or TPU. By executing the computer program P1, the processing element (computer) of the control unit 12 executes a determination of whether or not the card shown in the image of the image data is genuine, that is, an authenticity determination process for performing an authenticity determination. To do.

The computer program P1 may be provided to the determination device 1 by using the non-temporary storage medium A1 readablely stored by the processing element included in the control unit 12. In this case, the computer program P1 read from the storage medium A1 by a reading device (not shown) is stored in the storage unit 11. The storage medium A1 is an optical disk, a flexible disk, a magnetic disk, a magnetic optical disk, a semiconductor memory, or the like. Further, the computer program P1 may be downloaded from an external device (not shown) connected to a communication network (not shown), and the downloaded computer program P1 may be stored in the storage unit 11.

The number of processing elements included in the control unit 12 may be 2 or more. In this case, a plurality of processing elements may jointly execute the authenticity determination process according to the computer program P1.

FIG. 8 is a flowchart showing the procedure of the authenticity determination process. The control unit 12 executes the authenticity determination process when the image data of the card is acquired from the input unit 10. In the authenticity determination process, the control unit 12 first extracts a plurality of partial image data from the image data acquired from the input unit 10 (step S21). In step S21, the control unit 12 extracts the plurality of partial image data corresponding to the plurality of partial images at the plurality of locations indicated by the coordinate data Z in the image of the image data acquired from the input unit 10. For example, as shown in FIG. 2, the control unit 12 extracts a plurality of partial image data corresponding to the line portion on the card. The control unit 12 also functions as an extraction unit.

Here, the control unit 12 extracts a plurality of partial image data having a common type. Further, the type of the partial image data extracted in step S21 is the same as the type of the partial image data used for generating the authenticity estimation model R.

The method of extracting the partial image data is not limited to the method using the coordinate data Z. For example, partial image data may be extracted using a trained model that has learned the relationship between the image data of the card and the location of the partial image in the image of the image data, for example, R-CNN (Region-CNN). In this case, the control unit 12 inputs the image data to the trained model and determines the location, for example, the coordinates of the plurality of partial images based on the output of the trained model. The control unit 12 extracts a plurality of partial image data corresponding to a plurality of partial images at a plurality of locations indicated by the determined coordinates from the image data.

Next, the control unit 12 selects one partial image data from the plurality of partial image data extracted in step S21 (step S22), and inputs the selected partial image data into the authenticity estimation model R (step). S23). The control unit 12 also functions as an input unit. Next, the control unit 12 calculates the authenticity as the output of the authenticity estimation model R (step S24). The control unit 12 also functions as a calculation unit.

After executing step S24, the control unit 12 determines whether or not all the partial image data extracted in step S21 is selected in step S22 (step S25). When the control unit 12 determines that all the partial image data has not been selected (S25: NO), the control unit 12 executes step S22 again. In step S22 executed again, partial image data that has not yet been selected is selected. After that, the authenticity is calculated again in step S24. The control unit 12 continues to calculate the authenticity until all the partial image data extracted in step S21 is selected. As a result, the control unit 12 calculates the authenticity of all the partial image data extracted in step S12.

When the control unit 12 determines that all the partial image data has been selected (S25: YES), the image data acquired from the input unit 10 is based on the plurality of authenticities obtained by repeatedly executing step S24. It is determined whether or not the card shown in the image of is genuine (step S26).

FIG. 9 is an explanatory diagram of authenticity determination. As a first example of authenticity determination, the control unit 12 determines that the card is genuine when all the authenticity values calculated by repeatedly executing step S24 are equal to or greater than the authenticity threshold value. The authenticity threshold value is stored in the storage unit 11 and is a preset constant value. The control unit 12 determines that the card is not genuine when at least one of the plurality of authenticities calculated by repeatedly executing step S24 is less than the authenticity threshold.

As a second example of authenticity determination, the control unit 12 determines that the card is genuine when the average value of all the authenticity values calculated by repeatedly executing step S24 is equal to or greater than the authenticity threshold value. The control unit 12 determines that the card is not genuine when the above-mentioned average value is less than the authenticity threshold.

As a third example of authenticity determination, the control unit 12 performs a card when the number of authenticity that is less than the authenticity threshold is less than the reference number among a plurality of authenticity calculated by repeatedly executing step S24. Is determined to be genuine. The reference number is a constant natural number stored in the storage unit 11 and set in advance. The control unit 12 determines that the card is not genuine when the number of authenticity that is less than the authenticity threshold is equal to or greater than the reference number among the plurality of authenticity calculated by repeatedly executing step S24.

When the control unit 12 determines that the card is not genuine (S26: NO), the control unit 12 deletes the image data acquired from the input unit 10 from the storage unit 11 (step S27). When the control unit 12 determines that the card is genuine (S26: YES), or after executing step S27, the control unit 12 ends the authenticity determination process.

The image data of the card determined to be genuine by the authenticity determination is used, for example, for reading the personal information written on the card. The read personal information is used for registration or authentication of the cardholder.
In addition, the control unit 12 may execute step S27 and instruct an output unit (not shown) to output information indicating that the card is forged. As a result, the forgery of the card is notified. After executing this process, the control unit 12 ends the authenticity determination process.

As described above, the authenticity estimation model R is learned using a plurality of partial image data corresponding to a plurality of counterfeit cards created by using various printers. Therefore, in the authenticity determination process, the control unit 12 can calculate an appropriate authenticity regardless of the printer used for counterfeiting, and can appropriately determine the authenticity of the card. Further, in the authenticity determination process, a plurality of partial image data are extracted from one image data, a plurality of authenticity is calculated based on the extracted plurality of partial image data, and the authenticity is determined based on the calculated plurality of authenticity. I do. Therefore, the authenticity determination is accurate.

Further, since the type of partial image data extracted from the image data by the control unit 12 in the authenticity determination process is the same as the type of partial image data used for learning the authenticity estimation model R, the authenticity determination is more accurate. As described above, when a card is forged using a printer, the difference between the real card and the forged card tends to occur in the line portion. Therefore, when the partial image data extracted from the image data by the control unit 12 in the authenticity determination process and the partial image data of the line portion on the card are used as the partial image data used for learning the authenticity estimation model R, the authenticity is used. The judgment is more accurate.

As described in the description of the partial image data using FIG. 2, the partial image data used in the authenticity determination process and the authenticity estimation model generation process is not limited to the partial image data of the line portion on the card.
FIG. 10 is an explanatory diagram of another example of partial image data. In FIG. 10, partial images B20, B21, and B22 are shown as partial images of the partial image data. Each of the partial images B20 and B21 is an image of a portion of the card that is entirely filled with two colors including white. The partial image B22 is an image of a portion of the card that is entirely filled with a color other than white.

In the example of FIG. 10, a driver's license is shown. In a driver's license, a color other than white, for example, green, blue, or gold is used as the background color of the characters on which the expiration date is written. The region of the partial image B20 is a region including the boundary line between the background color and white. In addition, the driver's license is printed with a photo of the owner of the driver's license. In the face photograph, a color other than white, for example, blue is used as the background color. The region of the partial image B21 is a region including the boundary line between the background color and white. The area of the partial image B22 is the area of the part where the owner is not shown in the face photograph.

As a plurality of partial image data of the same type used in the authenticity determination process and the authenticity estimation model generation process, the partial image data of the part of the card that is entirely filled with two colors including white, and the card. In the above, a plurality of partial image data including both or one of the partial image data of the portion entirely filled with one color other than white may be used.

FIG. 11 is an explanatory diagram of another example of the partial image data. In FIG. 11, partial images B30, B31, and B32 are shown as partial images of the partial image data. Each of the partial images B30, B31, and B32 is an image of the shaded pattern portion on the card. The driver's license is also shown in the example of FIG. The driver's license has a column that indicates the type of vehicle that is permitted to drive. In this column, a red diagonal line pattern is used as the background. Each of the partial images B30, B31, and B32 is an image of a shaded pattern portion.
As the plurality of partial image data used in the authenticity determination process and the authenticity estimation model generation process and having a common type, a plurality of partial image data corresponding to the shaded pattern portion on the card may be used.

Similar to the line part on the card, when the card is forged using a printer, the difference between the real card and the fake card is that the whole part is filled with two colors and the whole part is filled with one color. It tends to occur in the part where there is a pattern and the part with a diagonal line. When comparing a counterfeit card with a real card, there is a high possibility that the borderline of the part that is entirely filled with two colors will be blurred, and the part that is entirely filled with one color may cause color unevenness. It is high, and there is a high possibility that the diagonal line will be distorted in the pattern part of the diagonal line. Therefore, as a plurality of partial image data of the same type used in the authenticity determination process and the authenticity estimation model generation process, instead of the plurality of partial image data corresponding to the line portion on the card, FIG. 10 or FIG. Even when the plurality of shown partial image data are used, the accuracy of authenticity determination is improved.

Further other examples of the partial image data will be described. Images of a plurality of image data having a common card type may contain common characters. For example, the images of the driver's license shown in FIGS. 2, 10 and 11 show common characters such as "name", "address" and "delivery". As the partial image of the partial image data, for example, three partial images showing "name", "address", and "delivery" may be used. Therefore, as a plurality of partial image data of the same type used in the authenticity determination process and the authenticity estimation model generation process, a plurality of partial image data corresponding to a plurality of common characters always written on the card are used. May be good.

The input of the plurality of partial image data to the authenticity estimation model R may be realized by inputting the concatenated image data obtained by concatenating the plurality of partial image data into the authenticity estimation model R. .. In this case, the authenticity estimation model R is generated using the linked image data.

(Embodiment 2)
In the authenticity determination process of the first embodiment, the control unit 12 of the determination device 1 uses one type of partial image data. However, in the authenticity determination process, the control unit 12 may use a plurality of types of partial image data.
Hereinafter, the difference between the second embodiment and the first embodiment will be described. Other configurations other than the configurations described later are common to the first embodiment. Therefore, the same reference reference numerals as those in the first embodiment are assigned to the components common to the first embodiment, and the description thereof will be omitted.

FIG. 12 is a block diagram showing a main configuration of the determination device 1 according to the second embodiment. When the second embodiment is compared with the first embodiment, the storage unit 11 of the determination device 1 stores the first authenticity estimation model R1 and the second authenticity estimation model R2 instead of the authenticity estimation model R. Has been done. Each of the first authenticity estimation model R1 and the second authenticity estimation model R2 is a trained model similar to the authenticity estimation model R. Further, the storage unit 11 stores the first coordinate data Z1 and the second coordinate data Z2 instead of the coordinate data Z.

The control unit 24 of the model generation device 2 according to the second embodiment executes the two authenticity estimation model generation processes in the same manner as the authenticity estimation model generation process according to the first embodiment. In the first authenticity estimation model generation process, the first authenticity estimation model R1 is generated using the teacher data in which the correct answer label is associated with the first partial image data which is a kind of partial image data. In the second authenticity estimation model generation process, the second authenticity estimation model R2 is generated using the teacher data in which the correct answer label is associated with the second partial image data which is a kind of partial image data.

FIG. 13 is an explanatory diagram of the first partial image data and the second partial image data. FIG. 13 shows images of cards similar to those in FIGS. 2, 10 and 11. As the first partial image related to the first partial image data, the first partial images E10, E11, E12, and E13 are shown. The second partial images E20, E21, and E22 are shown as the second partial image related to the second partial image data. The first partial images E10, E11, E12, E13 and the second partial images E20, E21, E22 are each surrounded by a broken line.

In the example of FIG. 13, each of the first partial images E10, E11, E12, and E13 is the same as the partial images B10, B11, B12, and B13 shown in FIG. 2, and is an image of a straight line portion or a curved portion on the card. .. The second partial images E20, E21, and E22 are the same as the partial images B20, B21, and B22 shown in FIG. 10, and the image of the portion of the card that is entirely filled with two colors including white, or It is an image of a part where the whole is filled with one color other than white.
As described above, the type of the first partial image data is different from the type of the second partial image data.

The plurality of coordinates indicated by the first coordinate data Z1 indicate the locations of the plurality of first partial images in the image of the card, similarly to the plurality of coordinates indicated by the coordinate data Z. The plurality of coordinates indicated by the second coordinate data Z2, like the plurality of coordinates indicated by the coordinate data Z, indicate the locations of the plurality of second partial images in the image of the card.

14 and 15 are flowcharts showing the procedure of the authenticity determination process. Similar to the first embodiment, the control unit 12 of the determination device 1 executes the authenticity determination process when the image data of the card is acquired from the input unit 10. In the authenticity determination process according to the second embodiment, the control unit 12 executes steps S31 to S44. Steps S31 to S37 are the same as steps S21 to S27 of the authenticity determination process in the first embodiment. In the description of steps S21 to S27, steps S31 to S37 can be described by replacing the partial image data and the coordinate data Z with the first partial image data and the first coordinate data Z1, respectively. Steps S38 to S44 are the same as steps S21 to S27 of the authenticity determination process in the first embodiment. In the description of steps S21 to S27, steps S38 to S44 can be described by replacing the partial image data and the coordinate data Z with the second partial image data and the second coordinate data Z2, respectively. Therefore, detailed description of steps S31 to S44 will be omitted.

In the authenticity determination process according to the second embodiment, the control unit 12 first executes step S31. In steps S31 and S38, the control unit 12 extracts a plurality of first partial image data and a plurality of second partial image data from the common image data acquired from the input unit 10. Here, the plurality of partial images corresponding to the plurality of first partial image data are a plurality of partial images at a plurality of locations indicated by the first coordinate data Z1. The plurality of partial images corresponding to the plurality of second partial image data are a plurality of partial images at a plurality of locations indicated by the second coordinate data Z2.
As in the first embodiment, the control unit 12 uses a trained model, for example, R-CNN, instead of the first coordinate data Z1 or the second coordinate data Z2, to use the first partial image or the second partial image. You may decide where to extract.

The location of the first partial image of the first partial image data used for generating the first authenticity estimation model R1 is the location of the plurality of first partial images corresponding to the plurality of first partial image data extracted in step S31. For example, it is one of a plurality of locations indicated by the first coordinate data Z1. Similarly, the location of the second partial image of the second partial image data used for generating the second authenticity estimation model R2 is a plurality of second partial images corresponding to the plurality of second partial image data extracted in step S38. , For example, one of a plurality of locations indicated by the second coordinate data Z2.

In step S36, the control unit 12 determines whether or not the card is genuine based on the plurality of authenticities calculated by repeatedly executing step S34. When the control unit 12 determines that the card is not genuine (S36: NO), the control unit 12 executes step S37 and deletes the image data acquired from the input unit 10 from the storage unit 11. After executing step S37, the control unit 12 ends the authenticity determination process.

When the control unit 12 determines that the card is genuine (S36: YES), the control unit 12 executes step S38. In step S42, the control unit 12 determines whether or not the card is genuine based on the plurality of authenticities calculated by repeatedly executing step S41. When the control unit 12 determines that the card is not genuine (S43: NO), the control unit 12 executes step S44 and deletes the image data acquired from the input unit 10 from the storage unit 11. When the control unit 12 determines that the card is genuine (S43: YES), or after executing step S44, the control unit 12 ends the authenticity determination process.

As described above, in the authenticity determination process in the second embodiment, it is determined that the card is genuine in both the authenticity determination using the first partial image data and the authenticity determination using the second partial image data. In this case, the image data acquired by the control unit 12 from the input unit 10 remains in the storage unit 11. The image data remaining in the storage unit 11 is used, for example, for reading personal information written on the card, as in the first embodiment.

As in the first embodiment, when the control unit 12 executes step S37 or step S44, the control unit 12 may instruct an output unit (not shown) to output information indicating that the card is forged. As a result, the forgery of the card is notified. After executing this process, the control unit 12 ends the authenticity determination process.

As described above, in the second embodiment, two types of trained models (first authenticity estimation model R1 and first authenticity estimation model R1) corresponding to two types of partial image data and learning the relationship between the partial image data and the authenticity are learned. 2 The authenticity estimation model R2) is prepared in the storage unit 11 of the determination device 1. In the authenticity determination process according to the second embodiment, the control unit 12 extracts two types of partial image data from the image data acquired from the input unit 10, and each of the extracted two types of partial image data corresponds to the type. Fill in the trained model. The control unit 12 calculates the authenticity of the card separately as the output of the two types of trained models, and determines whether or not the card is genuine based on the calculated authenticity.

Therefore, in the authenticity determination process of the second embodiment, the control unit 12 of the determination device 1 performs the authenticity determination using two types of partial image data, so that a more accurate authenticity determination result can be obtained. The determination device 1 in the second embodiment similarly exhibits the effect of the determination device 1 in the first embodiment.

(Embodiment 3)
In the second embodiment, the image data of one type of card is input to the input unit 10 of the determination device 1. However, image data of a plurality of types of cards may be input to the input unit 10.
Hereinafter, the differences between the third embodiment and the second embodiment will be described. Other configurations other than the configurations described later are common to the second embodiment. Therefore, the same reference reference numerals as those in the second embodiment are assigned to the components common to the second embodiment, and the description thereof will be omitted.

FIG. 16 is a block diagram showing a main configuration of the determination device 1 according to the third embodiment. When the third embodiment is compared with the second embodiment, the storage unit 11 of the determination device 1 has a plurality of first authenticity estimates instead of the first authenticity estimation model R1 and the second authenticity estimation model R2. Models R10, R11, ... And a plurality of second authenticity estimation models R20, R21, ... Are stored. A plurality of first authenticity estimation models R10, R11, ... And a plurality of second authenticity estimation models R20, R21, ... Each are trained models similar to the authenticity estimation model R in the first embodiment. It is learning the relationship between partial image data and authenticity. The first partial image data is input to the plurality of first authenticity estimation models R10, R11, .... Second partial image data is input to the plurality of second authenticity estimation models R20, R21, ....

Further, in the storage unit 11, instead of the first coordinate data Z1 and the second coordinate data Z2, a plurality of first coordinate data Z10, Z11, ..., And a plurality of second coordinate data Z20, Z21, ... Is remembered. The first coordinate data Z10, Z11, ... Each indicates the location of a plurality of first partial images, as in the second embodiment. The second coordinate data Z20, Z21, ... Each indicates the location of a plurality of second partial images, as in the second embodiment. The first coordinate data Z10, Z11, ... And the second coordinate data Z20, Z21, ... Are the first authenticity estimation models R10, R11, ... And the second authenticity estimation models R20, R21, respectively. Corresponds to ...

Further, refer to the case where the storage unit 11 selects one from a plurality of first authenticity estimation models R10, R11, ... And a plurality of second authenticity estimation models R20, R21, .... The model selection table H to be created is stored.

FIG. 17 is a chart showing the model selection table H. In the third embodiment, a plurality of image data corresponding to a plurality of types of cards are input to the input unit 10 of the determination device 1. The plurality of first authenticity estimation models R10, R11, ... Correspond to a plurality of types of cards. The plurality of second authenticity estimation models R20, R21, ... Also correspond to a plurality of types of cards.

In the example of FIG. 17, as the types of cards, a driver's license, an Individual Number card, a residence card, and the like are listed. Each type of card is associated with one of a plurality of first authenticity estimation models R10, R11, ..., And one of a plurality of second authenticity estimation models R20, R21, .... There is. In the example of FIG. 17, the driver's license is associated with the first authenticity estimation model R10 and the second authenticity estimation model R20.

FIG. 18 is a flowchart showing the procedure of the authenticity determination process. Similar to the second embodiment, the control unit 12 of the determination device 1 executes the authenticity determination process when the image data of the card is acquired from the input unit 10. A part of the authenticity determination process in the third embodiment is common to the authenticity determination process in the second embodiment. Therefore, in the authenticity determination process in the third embodiment, the part common to the authenticity determination process in the second embodiment, that is, the detailed description of steps S31 to S44 will be omitted.

In the authenticity determination process according to the third embodiment, the control unit 12 first executes a specific process of specifying the type of the card shown in the image of the image data based on the image data acquired from the input unit 10 ((). Step S51). In the specific process, the control unit 12 may specify the card type by using the trained model that has learned the relationship between the image data and the card type. Further, the control unit 12 may specify the type of the card based on the pattern of the border of the card shown in the image of the image data.

Next, the control unit 12 selects the first authenticity estimation models R10, R11, ... And the plurality of second authenticity estimation models R20, R21, ... Based on the model selection table H. , The first authenticity estimation model and the second authenticity estimation model corresponding to the type of the card specified in the identification process of step S51 are selected (step S52). In step S52, for example, when the control unit 12 specifies the card type as a driver's license in the identification process of step S51, the first authenticity estimation model R10 and the first authenticity estimation model R10 corresponding to the driver's license are displayed in the model selection table H. 2 Select the authenticity estimation model R20. The control unit 12 executes step S31 after executing step S52.

In steps S31 and S38, similarly to the second embodiment, the control unit 12 extracts a plurality of first partial image data and a plurality of second partial image data from the common image data acquired from the input unit 10. Here, in the image of the image data, the location of the plurality of first partial images corresponding to the plurality of first partial image data is the location indicated by the first coordinate data corresponding to the first authenticity estimation model selected in step S52. Is. Similarly, in the image of the image data, the location of the plurality of second partial images corresponding to the plurality of second partial image data is the location indicated by the second coordinate data corresponding to the second authenticity estimation model selected in step S52. Is. The control unit 12 extracts a plurality of first partial image data and a plurality of second partial image data corresponding to the card type specified in the specific process of step S51.

Note that the control unit 12 replaces the first coordinate data Z10, Z11, ..., Or the second coordinate data Z20, Z21, ..., As in the second embodiment, with a trained model, for example, R-CNN. May be used to determine where to extract the first or second partial image.

The locations of the first partial image of the first partial image data used for generating the first authenticity estimation models R10, R11, ... Are, for example, a plurality of locations indicated by the first coordinate data Z10, Z11, ... It is one of. Similarly, the location of the second partial image of the second partial image data used to generate the second authenticity estimation models R20, R21, ... Is indicated by, for example, the second coordinate data Z20, Z21, ... It is one of several places. When the types of cards are common, the location of the partial image corresponding to the partial image data for generation is one of the locations of the plurality of partial images corresponding to the plurality of extracted partial image data.

In the example of FIG. 17, the first authenticity estimation model R10 and the second authenticity estimation model R20 correspond to a driver's license. Therefore, the first partial image of the first partial image data used for generating the first authenticity estimation model R10 is, for example, one of a plurality of first partial images E10, E11, E12, and E13 shown in FIG. Similarly, the second partial image of the second partial image data used for generating the second authenticity estimation model R20 is, for example, one of a plurality of second partial images E20, E21, and E22 shown in FIG.

In step S33, the control unit 12 inputs the first partial image data selected in step S32, that is, one first partial image data extracted in step S31, into the first authenticity estimation model selected in step S52. .. Similarly, in step S40, the control unit 12 selects the second partial image data selected in step S39, that is, one second partial image data extracted in step S38, in the second authenticity estimation model in step S52. Enter in. The input to each of the first authenticity estimation model and the second authenticity estimation model means the input to the input layer of the first authenticity estimation model and the second authenticity estimation model.

In the authenticity determination process in the third embodiment, the card is genuine in both the authenticity determination using the first partial image data and the authenticity determination using the second partial image data, as in the second embodiment. When it is determined, the image data acquired by the control unit 12 from the input unit 10 remains in the storage unit 11.

In the authenticity determination process according to the third embodiment, the control unit 12 executes an appropriate authenticity determination according to the type of the card shown in the image of the image data acquired from the input unit 10. The determination device 1 in the third embodiment similarly exhibits the effect of the determination device 1 in the second embodiment.

In the authenticity determination processing in the second and third embodiments, the control unit 12 executes the authenticity determination based on a plurality of authenticity calculated as the output of the first authenticity estimation model. Here, when the control unit 12 determines that the card is genuine, the control unit 12 executes the authenticity determination based on a plurality of authenticity calculated as the output of the second authenticity estimation model. The method of authenticity determination is not limited to this method. For example, the authenticity determination may be executed only once based on a plurality of authenticity calculated as outputs of the first authenticity estimation model and the second authenticity estimation model.

Further, in the second and third embodiments, the number of each type of the partial image data used for generating the authenticity estimation model and the partial image data extracted from the image data is not limited to 2, but is 3 or more. You may. When the number of types of partial image data is 3, the first partial image data, the second partial image data, and the third partial image data are used for extraction and generation, and in the authenticity determination process, the first authenticity estimation model is used. , The second authenticity estimation model and the third authenticity estimation model are used. In this case, the first partial image is, for example, an image of a line portion. The second partial image is, for example, the partial image data of the portion of the card that is entirely filled with two colors including white, and the portion of the card that is entirely filled with one color other than white. A plurality of partial image data including both or one of the partial image data. The third partial image is, for example, an image of a pattern portion of diagonal lines.
Further, in the third embodiment, the number of types of partial image data may be one. In this case, the number of authenticity estimation models matches the number of card types.

Further, in the second and third embodiments, the input of the plurality of first partial image data to the first authenticity estimation model is the first connected image data obtained by concatenating the plurality of first partial image data. It may be realized by inputting to the first authenticity estimation model. In this case, the first authenticity estimation model is generated using the first connected image data. Similarly, when inputting a plurality of second partial image data into the second authenticity estimation model, the second connected image data obtained by concatenating the plurality of second partial image data is used as the second authenticity estimation model. It may be realized by inputting. In this case, the second authenticity estimation model is generated using the second connected image data.

(Embodiment 4)
Suppose the card is embossed, hologramed or watermarked. The embossing is a convex portion provided on the surface of the card. When the hologram is illuminated with light, a picture appears. The picture also appears when the watermark is illuminated with light. Therefore, when a card that has been embossed, holographic, or watermarked is irradiated with light, an embossed shadow, a hologram picture, or a watermark picture appears.

When a processed card is counterfeited using a printer, it is unlikely that an embossed shadow, a hologram picture, or a watermark picture will appear even if the counterfeit card is irradiated with light. Authenticity may be determined using this phenomenon.
Hereinafter, the fourth embodiment will be described as different from the first embodiment. Other configurations other than the configurations described later are common to the first embodiment. Therefore, the same reference reference numerals as those in the first embodiment are assigned to the components common to the first embodiment, and the description thereof will be omitted.

FIG. 19 is a block diagram showing a main configuration of the determination device according to the fourth embodiment. When the fourth embodiment is compared with the first embodiment, the input unit 10 of the determination device 1 is connected to the image pickup device 3 in addition to the internal bus 13. The image pickup device 3 captures an image of the card and outputs the image data of the captured image to the input unit 10 of the determination device 1.

FIG. 20 is an explanatory diagram of shooting of the card C. The image pickup device 3 includes a mounting table 30, an irradiator 31, and an imager 32. In the image pickup apparatus 3, the card C is placed on the mounting table 30, and the irradiator 31 irradiates the card C with light from diagonally above the upper surface of the card C. The imager 32 captures an image of the card C while the irradiator 31 irradiates the card C with light. The image pickup device 3 generates image data of an image taken by the image pickup device 32, and outputs the generated image data to the input unit 10 of the determination device 1. In the determination device 1, when the image data is input to the input unit 10, the control unit 12 acquires the image data input to the input unit 10 and executes the authenticity determination process, as in the first embodiment.

Card C is embossed, hologramed or watermarked. Therefore, when the card C is genuine, the image of the image data output by the image pickup apparatus 3 includes an embossed shadow, a hologram picture, or a watermark picture. In the image of the card C, the plurality of partial images at the plurality of locations indicated by the coordinate data Z are images of a portion on the card C in which an embossed shadow, a hologram picture, or a watermark picture appears.

Therefore, the partial image of the partial image data used to generate the authenticity estimation model R and the partial image of the partial image data extracted from the image data are embossed shadows, hologram pictures, or watermarks on the card C. It is an image of the part where the picture of is reflected. Further, in step S21 of the authenticity determination process, the control unit 12 is a portion on the card C where an embossed shadow, a hologram picture, or a watermark picture is reflected from the partial image data acquired from the input unit 10. Extract image data.
The determination device 1 according to the fourth embodiment configured as described above has the same effect as the determination device 1 according to the first embodiment.

In the fourth embodiment, the number of irradiators 31 included in the image pickup apparatus 3 may be two or more. In this case, the installation locations of the plurality of irradiators 31 are different from each other. In the image pickup apparatus 3, the card C is placed on the mounting table 30, and a plurality of irradiators 31 sequentially irradiate the card C with light from diagonally above the upper surface of the card C. Each time the irradiator 31 that irradiates the card C with light is changed, the imager 32 captures an image of the card C that is illuminated by the irradiator 31. The image pickup apparatus 3 generates a plurality of image data corresponding to the plurality of images captured by the imager 32, and outputs the generated plurality of image data to the input unit 10 of the determination device 1. In the determination device 1, when a plurality of image data are input to the input unit 10, the control unit 12 acquires a plurality of image data input to the input unit 10 and executes an authenticity determination process for each acquired image data. To do.

In this case, a plurality of authenticity estimation models R corresponding to a plurality of image data are prepared. For example, when the card C is determined to be genuine in all the authenticity determination processes, the personal information written on the card is read using one of the plurality of image data input to the input unit 10. Etc. are performed.

The configuration of the fourth embodiment may be added to each of the configurations of the second and third embodiments. In this case, in each of the second and third embodiments, the imaging device 3 is connected to the input unit 10 as in the fourth embodiment. When the number of the irradiators 31 is 1, the image pickup device 3 irradiates the first image data of the image taken by the photographer 32 with the irradiator 31 not irradiating the light and the irradiator 31 with the light. The second image data of the image taken by the camera 32 in this state is output to the input unit 10.

When the number of the irradiators 31 is 2 or more, the image pickup apparatus 3 sequentially changes the irradiators 31 that irradiate the card C with light. Each time the irradiator 31 that irradiates the card C with light is changed, the imager 32 captures an image of the card C that is illuminated by the irradiator 31. The image pickup apparatus 3 generates a plurality of second image data corresponding to the plurality of images captured by the imager 32. The image pickup apparatus 3 outputs the first image data and the generated plurality of second image data to the input unit 10 of the determination apparatus 1.

The control unit 12 of the determination device 1 executes the authenticity determination process of the second embodiment or the third embodiment using the first image data, and one or more of the second image data in the fourth embodiment. Executes the authenticity determination process of. For example, when the card C is determined to be genuine in the authenticity determination process of the second embodiment or the third embodiment, the control unit 12 executes one or more authenticity determination processes in the fourth embodiment. ..
Further, in the second and third embodiments, the partial image data in the fourth embodiment may be used as one type of partial image data.

(Embodiment 5)
The model generation device 2 in the first embodiment may relearn the authenticity estimation model R based on the result of the determination performed by the determination device 1.
Hereinafter, the differences between the fifth embodiment and the first embodiment will be described. Other configurations other than the configurations described later are common to the first embodiment. Therefore, the same reference reference numerals as those in the first embodiment are assigned to the components common to the first embodiment, and the description thereof will be omitted.

FIG. 21 is a block diagram showing a main configuration of the re-learning system according to the fifth embodiment. The re-learning system includes a plurality of determination devices 1, 1, ..., And a model generation device 2. These are connected to network N. In the fifth embodiment, each determination device 1 transmits the authenticity estimation model R and the teacher data to the model generation device 2. The model generation device 2 relearns the authenticity estimation model R.

The determination device 1 according to the fifth embodiment includes a communication unit 14 in addition to the constituent units of the determination device 1 according to the first embodiment. The communication unit 14 is also connected to the internal bus 13. The communication unit 14 transmits an update file including the authenticity estimation model R, a plurality of teacher data, the first evaluation data, and the second evaluation data to the model generation device 2 according to the instruction of the control unit 12. The communication unit 14 receives the authenticity estimation model R updated by the re-learning from the model generation device 2. The control unit 12 acquires the authenticity estimation model R received by the communication unit 14 from the communication unit 14.

When the authenticity determination process is completed, the control unit 12 of the determination device 1 generates a plurality of teacher data in which the partial image data input to the authenticity estimation model R is associated with a correct answer label indicating the result of the authenticity determination. Then, the generated plurality of teacher data are stored in the storage unit 11. The first evaluation data and the second evaluation data are stored in the storage unit 11 in advance, for example.
In the determination device 1, the processing element of the control unit 12 executes an update process for updating the authenticity estimation model R by executing the computer program P1.

When the fifth embodiment is compared with the first embodiment, the model generation device 2 includes a communication unit 26 instead of the input unit 20. The communication unit 26 is connected to the internal bus 25 and the network N. The communication unit 26 receives the update file from the communication unit 14 of the determination device 1. The update file received by the communication unit 26 of the model generation device 2 is acquired by the control unit 24. The communication unit 26 transmits the update file to the communication unit 14 of the determination device 1 according to the instruction of the control unit 24. In the model generation device 2, the processing element of the control unit 24 executes a re-learning process for re-learning the authenticity estimation model R by executing the computer program P2.

FIG. 22 is a flowchart showing a procedure of processing performed by the determination device 1 and the model generation device 2. FIG. 22 shows an update process executed by the control unit 12 of the determination device 1 and a relearning process executed by the control unit 24 of the model generation device 2. Each of the control units 12 and 24 periodically executes the update process and the relearning process.

In the update process, the control unit 12 of the determination device 1 determines whether or not to update the authenticity estimation model R (step S61). For example, when the number of teacher data stored in the storage unit 11 is equal to or greater than a predetermined number, the control unit 12 determines that the authenticity estimation model R is updated, and determines that the teacher data stored in the storage unit 11 is updated. When the number is less than a predetermined number, it is determined that the authenticity estimation model R is not updated.

When the control unit 12 determines to update the authenticity estimation model R (S61: YES), the control unit 12 instructs the communication unit 14, the authenticity estimation model R stored in the storage unit 11, a plurality of teacher data, and the first An update file including the 1st evaluation data and the 2nd evaluation data is transmitted to the communication unit 26 of the model generator 2 via the network N (step S62). The control unit 24 of the model generation device 2 determines whether or not the communication unit 26 has received the update file from the communication unit 14 of the determination device 1 (step S71).

When the control unit 24 of the model generation device 2 determines that the communication unit 26 has received the update file (S71: YES), the control unit 24 stores the update file received by the communication unit 26 in the storage unit 23 (step S72). The authenticity estimation model generation process shown in 5 and 6 is executed (step S73). As a result, the authenticity estimation model R received by the communication unit 26 is relearned.
In the authenticity estimation model generation process executed in step S73, since the route has already been formed, the process is started from step S2 without executing step S1.

Next, the control unit 24 instructs the communication unit 26 to transmit the relearned authenticity estimation model R to the communication unit 14 of the determination device 1 (step S74). When the communication unit 26 determines that the update file has not been received (S71: NO), or after executing step S74, the control unit 24 ends the re-learning process.

After executing step S62, the control unit 12 of the determination device 1 determines whether or not the communication unit 14 has received the authenticity estimation model R from the communication unit 26 of the model generation device 2 (step S63). When the control unit 12 determines that the communication unit 14 has not received the authenticity estimation model R (S63: NO), the control unit 12 executes step S63 again and waits until the communication unit 14 receives the authenticity estimation model R. To do. When the control unit 12 determines that the communication unit 14 has received the authenticity estimation model R (S63: YES), the control unit 12 receives the authenticity estimation model R stored in the storage unit 11 by the communication unit 14. Update to the estimation model R (step S64).
When the control unit 12 determines that the authenticity estimation model R is not updated (S61: NO), the control unit 12 ends the update process after executing step S64.

The determination device 1 in the fifth embodiment similarly exhibits the effect of the determination device 1 in the first embodiment. Further, the authenticity estimation model R stored in the determination device 1 in the fifth embodiment is periodically improved by performing re-learning.

In the re-learning of the authenticity estimation model R in the fifth embodiment, the control unit 12 of the determination device 1 instructs the communication unit 14 to use the parameters of the authenticity estimation model R instead of the authenticity estimation model R. The indicated information may be transmitted to the communication unit 26 of the model generation device 2. In this case, the control unit 24 of the model generation device 2 generates the authenticity estimation model R based on the parameters received by the communication unit 26, and relearns the generated authenticity estimation model R. Further, the control unit 24 may instruct the communication unit 26 to transmit information indicating the parameters of the authenticity estimation model R to the communication unit 14 of the determination device 1 instead of the authenticity estimation model R. In this case, the control unit 12 of the determination device 1 updates the parameters of the authenticity estimation model R stored in the storage unit 11 based on the received information.

Further, in the second to fourth embodiments, the control unit 12 of the determination device 1 and the control unit 24 of the model generation device 2 may execute the update process and the relearning process, respectively, as in the fifth embodiment. In the second and third embodiments, a plurality of authenticity estimation models are stored in the storage unit 11 of the determination device 1. Therefore, for each authenticity estimation model, the control unit 12 of the determination device 1 and the control unit 24 of the model generation device 2 each execute an update process and a relearning process.

Further, in the first to fifth embodiments, the image data input to the input unit 10 of the determination device 1 may be the one in which characters are described, that is, the image data of the document, and thus the image data of the card. Not limited. Examples of the image data input to the input unit 10 include image data such as a booklet or a contract. The booklet is a passport, a bank passbook, a student certificate, a pension certificate, a disability certificate, a maternal and child health certificate, and the like.

(Embodiment 6)
In step S51 of the authenticity determination process of the third embodiment, the control unit 12 of the determination device 1 executes a specific process of specifying the type of the card shown in the image of the image data. The control unit 12 may execute the specific process using the second trained model that has learned the relationship between the image data of the card and the type of the card.
Hereinafter, the difference between the sixth embodiment and the third embodiment will be described. Other configurations other than the configurations described later are common to the third embodiment. Therefore, the same reference reference numerals as those in the third embodiment are assigned to the components common to the third embodiment, and the description thereof will be omitted.

FIG. 23 is an explanatory diagram of an outline of the authentication device 5 according to the sixth embodiment. The user inserts a plurality of types of cards including a driver's license into the authentication device 5. The card has a rectangular plate shape. The authentication device 5 reads the personal information displayed on the inserted card and authenticates the owner of the card based on the read personal information. The authentication device 5 includes a determination device 1 and an image pickup device 40. The image pickup device 40 is connected to the determination device 1.

The plurality of types of cards include a target card that is publicly issued to prove the identity and should be inserted into the authentication device 5, and a non-target card that should not be inserted into the authentication device 5. .. In addition to the driver's license, the types of target cards include My Number Card, My Number Notification Card, Basic Resident Register Card, Special Permanent Resident Certificate, Residence Card, Passport Card, Social Security Card, and the like. A plurality of types of target cards are set in advance. Examples of non-applicable cards include point cards and electronic money cards. One or more types of non-applicable cards are also preset.

The image pickup device 40 captures images of the first surface and the second surface of the card inserted by the user, generates two image data corresponding to the two captured images, and determines the generated two image data. Output to. In the following, the image of the first surface will be referred to as the first surface image, and the image of the second surface will be referred to as the second surface image. The image data of the first surface image is described as the first surface image data, and the image data of the second surface image is described as the second surface image data. The first surface image and the second surface image have a rectangular shape.

In the storage unit 11 of the determination device 1, in addition to the plurality of first authenticity estimation models R10, R11, ..., The plurality of second authenticity estimation models R20, R21, ..., The type identification model G is provided. It is remembered. The type-specific model G is a second trained model in which the relationship between the reduced image data obtained by reducing the image data of the card, the type of the card, and the front and back sides is learned. Here, the front and back of the card is one of the orientations of the card.
As described in the description of the third embodiment, the number of types of partial image data is not limited to 2, and may be 1 or 3 or more. In the sixth embodiment, the configuration will be described assuming that the number of types of partial image data is two.

When the first surface image data and the second surface image data are input from the image pickup device 40, the control unit 12 of the determination device 1 generates reduced image data of the input first surface image data, and the generated reduced image. The data is input to the type-specific model G. The control unit 12 calculates the probability of indicating the certainty of the type of the card and the classification of the front and back sides as the output of the type identification model G. The control unit 12 identifies the type and front and back of the card shown in the image of the first surface image data based on the calculated probability.

When the control unit 12 cannot specify the type and front and back of the card shown in the image of the first surface image data based on the first surface image data, the control unit 12 reduces the input second surface image data. Image data is generated, and the generated reduced image data is input to the type identification model G. The control unit 12 calculates the probability of indicating the certainty of the type of the card and the classification of the front and back sides as the output of the type identification model G. The control unit 12 identifies the type and front and back of the card shown in the image of the second side image data based on the calculated probability.

After specifying the type and front and back of the card, the control unit 12 of the determination device 1 executes the authenticity determination process using the image data on the front or back side of the card. When the control unit 12 determines that the card shown in the image of the image data is genuine, the control unit 12 reads and reads the personal information displayed on the card from both or one of the image data on the front side and the back side of the card. Authenticate the cardholder based on personal information.

FIG. 24 is an external view of the authentication device 5. The authentication device 5 includes a storage box 5a in which the determination device 1 and the image pickup device 40 are housed. The outer surface of the storage box 5a is provided with a rectangular opening 5b penetrating in the inward and outward directions. The user of the authentication device 5 inserts the card C into the opening 5b. The long side of the opening 5b is slightly longer than the short side of the card C. The short side of the opening 5b is slightly longer than the thickness of the card C. Therefore, the size of the opening 5b is substantially the same as the size of the end face on the short side side of the card C. The user of the authentication device 5 inserts the card C into the opening 5b from the short side of the card C.

FIG. 25 is an example diagram of the target card. Personal information such as name, date of birth, address, or face photograph is displayed as personal information on at least one of the front side and the back side of the target card included in the plurality of types of cards C. In the example of FIG. 25, personal information such as name, date of birth, address, and face photograph is displayed on the front side of the target card, and personal information is not displayed on the back side of the target card. Personal information is written by hand on the back side of the target card shown in FIG. 25, for example. Regarding the target card, the display position where personal information is displayed differs depending on the type. The front side and the back side of each of the plurality of types of cards C are appropriately defined. For example, the side on which the name is displayed may be defined as the front side, and the side on which the name is not displayed may be defined as the back side.

When the card C is inserted into the opening 5b in the authentication device 5 shown in FIG. 24, the image pickup device 40 of the authentication device 5 is the first surface corresponding to the first surface and the second surface of the card C inserted into the opening 5b, respectively. The surface image and the second surface image are photographed, and the first surface image data and the second surface image data corresponding to the photographed first surface image and the second surface image are generated. The first surface image and the second surface image have a rectangular shape. As described above, the determination device 1 identifies the type and front and back of the card C by using both or one of the first side image data and the second side image data. The meaning of "specific" includes meanings such as "discrimination", "identification", and "judgment".

FIG. 26 is a schematic view of the type identification model G. The type-specific model G is also a trained model of machine learning including deep learning, for example, CNN. The type-specific model G has an input layer, a plurality of intermediate layers, and an output layer. FIG. 26 shows an example in which the number of intermediate layers is 2. The number of intermediate layers may be 3 or more.

Each of the input layer, middle layer and output layer has one or more neurons. The neurons in each layer are unidirectionally connected to the neurons in the previous and next layers with the desired weight and bias. A vector having the same number of components as the number of neurons in the input layer is input as input data of the type identification model G.

In addition, a layer for convolution may be included in a plurality of intermediate layers. In this layer, an appropriate filter including a convolution filter is applied to the input data, and a plurality of data obtained after applying the filter is output to the next layer, for example, the pouring layer.

The input values input to each neuron in the input layer are input to the first intermediate layer. In this intermediate layer, the output is calculated using an activation function that includes weights and biases. The calculated value is input to the next intermediate layer. In the same manner below, the output of the output layer is transmitted to the subsequent layers one after another until the output is obtained. Various parameters such as weights and biases that connect neurons are learned by using, for example, an error backpropagation method.
The input data to be input to the input layer of the type-specific model G is the reduced image data obtained by reducing the first surface image data or the second surface image data. For example, the pixel value of the reduced image data is input to the input layer of the type-specific model G as input data.

FIG. 27 is an explanatory diagram of reduced image data. The first side image or the second side image is shown on the upper side of FIG. 27, and the reduced image corresponding to the reduced image data is shown on the lower side of FIG. 27. The number of vertical and horizontal pixels for each of the first surface image data and the second surface image data is represented by M (M: natural number) and N (N: natural number).

By reducing the number of vertical and horizontal pixels for each of the first-side image data and the second-side image data, reduced image data in which the personal information displayed on the card cannot be read is generated. For example, reduced image data is generated by changing the number of vertical and horizontal pixels to (M / 10) and (N / 10). M and N are, for example, 380 and 600, respectively. When the first surface image data or the second surface image data is reduced, the first surface image or the second surface image data is also reduced.

In the type identification model G, the reduced image data is used by using the reduced image data of the front side or the back side of the card, the type of the card, and the second teacher data including the second correct answer label indicating the front side or the back side of the card. The type of card and the relationship between the front and back are learned. The second correct label is one piece of information. As the reduced image data of the second teacher data, the reduced image data on the front and back of a plurality of types of cards including a driver's license are used.
In the middle layer, the relationship between the reduced image data, the type of card, and the front and back is learned based on the second teacher data.

The output layer outputs a probability indicating the type of card shown in the reduced image corresponding to the input reduced image data and the certainty of the classification of the front and back sides. Specifically, for each card type and front / back combination, the probability that the card type and front / back combination related to the input reduced image data correspond is shown. In FIG. 26, the type of card and the combination of front and back are shown by (type, front or back). For each combination of (driver's license, front), (driver's license, back), (my number card, front) and (my number card, back), the type of card and the combination of front and back related to the input reduced image data. Indicates the probability of being applicable.
The reduced image data is reduced until it becomes impossible to acquire personal information. Therefore, the probability is not output based on the personal information displayed on the card.

FIG. 28 is a block diagram showing a main configuration of the model generator 2. When the model generation device 2 in the sixth embodiment is compared with the model generation device 2 in the third embodiment, there is a difference in the contents stored in the storage unit 23. In the sixth embodiment, in addition to the plurality of first authenticity estimation models R10, R11, ..., the plurality of second authenticity estimation models R20, R21, ..., And the computer program P2, the storage unit 23 is stored. , Type-specific model G is stored.

In the input unit 20, in addition to the plurality of teacher data, the first evaluation data and the second evaluation data, the plurality of second teacher data, the third evaluation data for evaluating the type identification model G, and the third evaluation data The fourth evaluation data is input. The control unit 24 acquires the second teacher data, the third evaluation data, and the fourth evaluation data input to the input unit 20. Similar to the second teacher data, the third evaluation data and the fourth evaluation data show the reduced image data, the type of the card shown in the reduced image of the reduced image data, and the front side or the back side of the card. Includes a second correct label to indicate.

By executing the computer program P2, the control unit 24 stores the second teacher data, the third evaluation data, and the fourth evaluation data in the storage unit 23 in addition to the storage process and the authenticity estimation model generation process. The second storage process for generating the type-specific model G and the type-specific model generation process for generating the type-specific model G are executed.
When the number of processing elements included in the control unit 24 is 2 or more, the plurality of processing elements perform not only the storage processing and the authenticity estimation model generation processing but also the second storage processing and the type identification model according to the computer program P2. The generation process may be executed jointly.

A plurality of second teacher data, third evaluation data, and fourth evaluation data are input to the input unit 20 from a device (not shown). When these data are input to the input unit 20, the control unit 24 executes a second storage process. In the second storage process, the control unit 24 acquires a plurality of second teacher data, third evaluation data, and fourth evaluation data input to the input unit 20, and stores these acquired data. Store in 23. After that, the control unit 24 ends the second storage process.

In the type-specific model generation process, the control unit 24 inputs the reduced image data included in the second teacher data into the type-specific model G, and outputs the type-specific model G as the probability of card type and front / back classification. Calculate the probability of indicating. The control unit 24 updates parameters such as weights and biases between nodes used in the intermediate layer of the type identification model G based on the calculated probabilities. As a result, the control unit 24 generates the type-specific model G.

FIG. 29 is an explanatory diagram of reduced image data included in the second teacher data. The reduced image data of the second teacher data is not limited to the simple reduced image data of a plurality of types of cards. As shown on the upper side of FIG. 29, noise is added to the first side image data or the second side image data of the card, and the added first side image data or the second side image data is reduced. As a result, reduced image data in which noise is added to the simple reduced image data of the card is generated. The control unit 24 also stores the reduced image data to which noise is added in the storage unit 23 as the reduced image data of the second teacher data.

The noise shown in FIG. 29 is noise corresponding to dirt on the card. The noise added to the reduced image data is not limited to the noise corresponding to the stain, but also includes the noise corresponding to the color change caused by sunburn, aging deterioration, or the like. The control unit 24 also acquires the reduced image data to which the noise is added, and stores the acquired reduced image data in the storage unit 23. One of the reduced images of the reduced image data to which noise corresponding to the color change is added is a reduced image of a yellowed card.

As shown in the lower part of FIG. 29, the first side image data or the first side image data obtained by deleting or changing the personal information contained in the first side image data or the second side image data of the card, and deleting or changing the card. Reduce the two-sided image data. As a result, the reduced image data in which the personal information contained in the simple reduced image data of the card is deleted or changed is generated. The control unit 24 also stores the reduced image data obtained by deleting or changing the personal information in the storage unit 23 as the reduced image data of the second teacher data. The change of personal information is, for example, a change of name, date of birth or address. The deletion of personal information is, for example, the deletion of a part of a name or an address.

30 and 31 are flowcharts showing the procedure of the type identification model generation process. The user of the model generation device 2 performs a second execution operation instructing the execution of the type-specific model generation process in the operation unit 21. When the second execution operation is performed and a plurality of second teacher data, third evaluation data, and fourth evaluation data are stored in the storage unit 23, the control unit 24 is a type identification model. Execute the generation process. The storage unit 23 stores the values of the third variable and the fourth variable used in the type identification model generation process. The control unit 24 sets these values separately.

In the type-specific model generation process, the control unit 24 first forms a path for the type-specific model G (step S81). Specifically, the control unit 24 has a pathway between a plurality of neurons regarding the input layer and an intermediate layer adjacent to the input layer, a pathway between a plurality of neurons regarding two adjacent intermediate layers, and an intermediate layer and an intermediate layer. It forms a pathway between multiple neurons with respect to the output layer adjacent to the layer. Parameters such as weight and bias between neurons are set as appropriate.

After executing step S81, the control unit 24 sets the values of the third variable and the fourth variable to zero (step S82), and the plurality of second teacher data stored in the storage unit 23, the third evaluation. Data and the fourth evaluation data are loaded into the temporary storage unit 22 (step S83). Next, the control unit 24 selects one second teacher data from the temporary storage unit 22 (step S84), and inputs the reduced image data included in the selected second teacher data into the type identification model G (step S84). Step S85). The input of data to the type-specific model G means the input of data to the input layer of the type-specific model G. Next, the control unit 24 calculates the probability of indicating the type of the card and the certainty of the classification of the front and back sides as the output of the type identification model G (step S86).

Next, the control unit 24 increments the value of the third variable by 1 (step S87), and determines whether or not the value of the third variable is the third reference value (step S88). The third reference value is stored in the storage unit 23, is a preset constant value, and is an integer of 2 or more. When the control unit 24 determines that the value of the third variable is not the third reference value (S88: NO), the control unit 24 executes step S84 again. In step S84 executed again, the control unit 24 selects one second teacher data that has not been selected from the plurality of second teacher data loaded in step S83.

The control unit 24 repeatedly executes steps S84 to S87 until the value of the third variable becomes the third reference value, and continues to calculate the probability indicating the certainty of the card type and the front and back classification. The first reference value is the so-called batch size. When the control unit 24 determines that the value of the third variable is the third reference value (S88: YES), the control unit 24 includes a plurality of calculation results obtained by repeatedly executing step S86, and a plurality of data selected in step S84. The parameters of the type-specific model G are updated based on the contents indicated by the second correct answer label of the second teacher data of (step S89). Next, the control unit 24 determines whether or not all the second teacher data stored in the temporary storage unit 22 is selected in step S84 (step S90).

When the control unit 24 determines that all the second teacher data has not been selected (S90: NO), the control unit 24 sets the value of the third variable to zero (step S91), and executes step S84 again. In step S84 executed again, the control unit 24 selects one second teacher data that has not been selected from the plurality of second teacher data loaded in step S83. The parameters are repeatedly updated based on the plurality of calculation results obtained by repeatedly executing step S86 until all the second teacher data are selected.

When the control unit 24 determines that all the second teacher data has been selected (S90: YES), the control unit 24 evaluates the type identification model G using the third evaluation data (step S92). In step S92, the control unit 24 inputs the reduced image data included in the third evaluation data into the type identification model G, and calculates the probability of indicating the certainty of the card type and the front and back classification. The control unit 24 confirms that an appropriate calculation result is calculated as the output of the type identification model G based on the calculation result and the content indicated by the second correct answer label included in the third evaluation data. To do. If an appropriate calculation result has not been calculated, for example, the type-specific model generation process may be terminated. In this case, for example, the number of the second teacher data is increased, and the type identification model generation process is executed again.

After executing step S92, the control unit 24 increments the value of the fourth variable by 1 (step S93), and determines whether or not the value of the fourth variable is the fourth reference value (step S94). The fourth reference value is stored in the storage unit 23, is a preset constant value, and is an integer of 2 or more. When the control unit 24 determines that the value of the fourth variable is not the fourth reference value (S94: NO), the control unit 24 sets the value of the third variable to zero (step S95), and executes step S84 again. At this time, it is assumed that there is no second teacher data selected among the plurality of second teacher data loaded in step S83, and step S84 is executed. The parameters of the type-specific model G are updated using the plurality of second teacher data loaded in step S83 until the value of the fourth variable becomes the fourth reference value. The number of times each second teacher data is used is the same as the number of fourth reference values. The fourth reference value is the so-called epoch number.

When the control unit 24 determines that the value of the fourth variable is the fourth reference value (S94: YES), the control unit 24 evaluates the type identification model G using the fourth evaluation data (step S96). In step S96, the control unit 24 inputs the reduced image data included in the fourth evaluation data into the type-specific model G, and outputs the type-specific model G as a probability of indicating the certainty of the card type and the front and back classification. Is calculated. It is confirmed that the control unit 24 calculates an appropriate calculation result as the output of the type identification model G based on the calculation result and the content indicated by the second correct answer label included in the fourth evaluation data. To do. If an appropriate calculation result has not been calculated, for example, the type-specific model generation process may be terminated. In this case, for example, the number of the second teacher data is increased, and the type identification model generation process is executed again.

After executing step S96, the control unit 24 stores the type-specific model G whose parameters have been updated in the storage unit 23 (step S97), and the second teacher data, the third, used for updating the type-specific model G. The evaluation data and the fourth evaluation data are deleted from the temporary storage unit 22 and the storage unit 23 (step S98). After executing step S98, the control unit 24 ends the type identification model generation process.

As described above, in the type-specific model generation process, the control unit 24 updates the parameters of the type-specific model G that inputs the reduced image data based on the plurality of second teacher data stored in the storage unit 23. To do. By updating the parameters, the type-specific model G is generated.

As described above, the type identification model G is used in the process of identifying the type and front and back of the card from the reduced image data on one side of the card. In the type-specific model generation process, the control unit 24 generates an appropriate type-specific model G by updating the parameters based on the second teacher data acquired from the input unit 20 in the second storage process.

The plurality of reduced image data included in the plurality of second teacher data used in the type identification model generation process is a plurality of reduced image data obtained by reducing the image data on the front and back sides of the plurality of types of cards. The plurality of reduced image data used in the type-specific model generation process includes reduced image data on the front and back sides of all types of cards.

As described above, the control unit 12 of the determination device 1 uses the type identification model G generated by the control unit 24 of the model generation device 2 in the type identification model generation process. The control unit 12 of the determination device 1 identifies the type and front and back of the card inserted in the opening 5b based on the calculation result calculated as the output of the type identification model G.

In the type identification model generation process, as described above, the control unit 24 acquires a plurality of reduced image data obtained by reducing the image data on the front and back sides of the plurality of types of cards from the input unit 20, and obtains a plurality of acquired image data. The reduced image data is input as the reduced image data of the plurality of second teacher data. As a result, the type-specific model G is generated. By inputting the reduced image data that has been reduced until it becomes impossible to acquire personal information, the probability is output based on information different from the personal information displayed on the card inserted in the opening 5b. It is possible to generate a type-specific model G to be used.

Further, in the type identification model generation process, as described above, the control unit 24 acquires the reduced image data obtained by adding noise to at least one of the reduced image data of the plurality of types of cards from the input unit 20. , The acquired reduced image data is included in the input of the type identification model G as the reduced image data of the second teacher data. Therefore, in the authentication performed by the determination device 1, the reduced image data input to the type identification model G, that is, the reduced image data on the first surface or the second surface of the card inserted in the opening 5b is stained. Even if it is the reduced image data of one side of the card or the card whose color has changed, appropriate identification is performed using the type identification model G. As mentioned above, the color of the card changes due to sunburn or aging.

Further, in the type identification model generation process, the control unit 24 acquires the reduced image data obtained by deleting or changing the personal information contained in at least one of the reduced image data of the plurality of types of cards from the input unit 20. , The acquired reduced image data is included in the input of the type identification model G as the reduced image data of the second teacher data. Input to the type-specific model G is not limited to simple reduced image data of multiple types of cards, but also reduced image data obtained by adding noise and reduced images obtained by deleting or changing personal information. Contains data and. Therefore, since a large amount of reduced image data is used in the type-specific model generation process, the calculation result calculated as the output of the type-specific model G is more appropriate.

For simple reduced image data, reduced image data with added noise, reduced image data with partially changed personal information, or reduced image data with partially deleted personal information, a model generator. The control unit 24 of 2 may be generated. In this case, data including an image data showing the front side or the back side of the card and a second correct label corresponding to the image data is input to the input unit 20. Based on the image data input to the input unit 20, the control unit 24 partially deletes the reduced image data to which noise is added, the reduced image data in which the personal information is partially changed, or the personal information is partially deleted. Generate reduced image data. As a result, the second teacher data including the reduced image data and the second correct label is generated. The generated second teacher data is stored in the storage unit 23.

FIG. 32 is a block diagram showing a main configuration of the authentication device 5. The authentication device 5 includes a camera 41 and a display 42 in addition to the determination device 1 and the image pickup device 40. As shown in FIG. 24, the camera 41 and the display 42 are installed on the outer surface of the storage box 5a. The determination device 1 in the sixth embodiment includes an input / output unit 15 and an output unit 16 in addition to the components included in the determination device 1 in the third embodiment.

The image pickup device 40 is connected to the input unit 10 of the determination device 1. The camera 41 is connected to the input / output unit 15 of the determination device 1. The display 42 is connected to the output unit 16 of the determination device 1. The input / output unit 15 and the output unit 16 are connected to the internal bus 13.

When the card is inserted into the opening 5b, the image pickup apparatus 40 captures the first side image and the second side image of the inserted card, and the first side image corresponding to the photographed first side image and the second side image. The image data and the second surface image data are generated, and the generated first surface image data and the second surface image data are output to the input unit 10 of the determination device 1.

As described above, the size of the opening 5b is substantially the same as the size of the end face on the short side of the card. Therefore, the shapes of the frames of the first surface image and the second surface image substantially match the shape of the surface of the card. The first surface image data and the second surface image data input from the image pickup apparatus 40 to the input unit 10 are acquired from the input unit 10 by the control unit 12.

The input / output unit 15 outputs a shooting signal instructing the camera 41 to shoot a face image showing the user's face in accordance with the instruction of the control unit 12. When the shooting signal is input to the camera 41, the camera 41 shoots a face image and outputs the face image data corresponding to the shot face image to the input / output unit 15. The face image data input to the input / output unit 15 is acquired from the input / output unit 15 by the control unit 12.

The output unit 16 includes unspecified information indicating that the card could not be specified according to the instruction of the control unit 12, and non-target information indicating that the specified card type is a non-target card type. , The unreadable information indicating that the card could not be read although the card was successfully identified is output to the display 42. When non-specified information, non-target information, or unreadable information is input to the display 42, the display 42 displays the content indicated by the input information. As a result, the user is notified that the card could not be identified, that the card type is a non-target card type, or that the reading could not be performed.

The output unit 16 further outputs result information indicating success or failure of authentication to the display 42 according to the instruction of the control unit 12. When the result information is input to the display 42, the display 42 displays the content indicated by the result information. As a result, the user is notified of the success or failure of the authentication.

FIG. 33 is an explanatory diagram of data stored in the storage unit 11. As described above, the storage unit 11 contains the type-specific model G, a plurality of first authenticity estimation models R10, R11, ..., a plurality of second authenticity estimation models R20, R21, ..., And a computer program. P1 is stored. The storage unit 11 further stores a plurality of first coordinate data Z10, Z11, ..., A plurality of second coordinate data Z20, Z21, ..., An authentication table 60, and a read rule database 61. .. The type-specific model G stored in the storage unit 11 is a trained model generated by the control unit 24 of the model generation device 2.

Similar to the third embodiment, the processing element of the control unit 12 executes the authenticity determination process by executing the computer program P1. In step S71 of the authenticity determination process, the control unit 12 executes a specific process of specifying the type and front and back of the card.
The processing element of the control unit 12 further executes an authentication process for authenticating the cardholder by executing the computer program P1. When the number of processes possessed by the control unit 12 is 2 or more, a plurality of processing elements may jointly execute not only the authenticity determination process but also the authentication process according to the computer program P1.

In the specifying process, the control unit 12 specifies the type and front and back of the card inserted in the opening 5b. In the authenticity determination process, the control unit 12 determines whether or not the card is genuine as a card belonging to the type specified in the specific process. When the control unit 12 determines that the card is genuine, the control unit 12 executes an authentication process. In the authentication process, the control unit 12 reads personal information from at least one of the first side image data and the second side image data based on the type and front and back specified in the specific process, and based on the read personal information, the card. Authenticate the owner of.

FIG. 34 is a chart showing an authentication table 60. In the example of FIG. 34, the authentication table 60 includes a name field, an address field, and a date of birth field. In the name field, a plurality of names corresponding to a plurality of users who use the authentication device 5 are stored. In the address field, a plurality of addresses corresponding to each of the plurality of names stored in the name field are stored. In the date of birth field, a plurality of dates of birth corresponding to each of the plurality of names stored in the name field are stored. The name, address and date of birth are stored in advance in the authentication table 60. The authentication table is used to authenticate the cardholder.

FIG. 35 is a chart showing the contents of the read rule database 61. The read rule database 61 includes a type field and a read rule field. The type of the target card is stored in the type field. The types of non-applicable cards, point cards, electronic money cards, etc. are not stored in the type field. As described above, the types of target cards include a driver's license, an Individual Number Card, an Individual Number Notification Card, a Basic Resident Register Card, a Special Permanent Resident Certificate, a Residence Card, a Passport Card, a Social Security Card, and the like. In the reading rule field, a reading rule for reading a plurality of personal information corresponding to the type stored in the type field is stored. Specifically, each reading rule indicates a display position of personal information. The display position of one personal information is defined by the orientation (front side or back side) of the card on which the personal information is displayed and the display location of the personal information on the front side or the back side of the card. The type and read rule are stored in advance in the read rule database 61.

36 and 37 are flowcharts showing the procedure of the specific processing. The control unit 12 executes the authenticity determination process when the first screen image data and the second surface image data of the card are acquired from the input unit 10. The control unit 12 executes a specific process in step S71 of the authenticity determination process.

In the specific process, the control unit 12 first generates reduced image data of the first surface image data acquired from the input unit 10 (step S101). The control unit 12 reduces the number of vertical and horizontal pixels of the first surface image data, thereby reducing the first surface image data, and generates reduced image data of the first surface image data.

Next, the control unit 12 inputs the reduced image data generated in step S101 into the type identification model G (step S102). In the type-specific model G, for example, the pixel value data of each pixel constituting the reduced image data is input to each neuron in the input layer. The data input to each neuron in the input layer is output to the neurons in the adjacent intermediate layer. In the intermediate layer, based on the reduced image data input to the input layer, the control unit 12 performs an operation using an activation function including weights and biases between neurons, and the operation result performed in the intermediate layer is output. Output to the layer. In this way, in the type identification model G, the probability indicating the certainty of the card type and the classification of the front and back sides is output from the output layer.

After executing step S102, the control unit 12 calculates the probability of indicating the certainty of the card type and the front and back classification as the output of the type identification model G (step S103). Next, the control unit 12 determines whether or not it is possible to specify the type of card and the front and back sides based on the probability calculated in step S103 (step S104).

Specifically, the control unit 12 calculates the variance value of the probability distribution calculated as the output of the type identification model G. The variance value is large when only one probability is extremely high among the probabilities related to all combinations of card types and front and back sides, and when the probabilities related to all combinations of card types and front and back sides are substantially the same. ,small. In step S104, the control unit 12 determines that it is impossible to specify the calculated variance value if it is less than the predetermined value, and specifies it if the calculated variance value is equal to or more than the predetermined value. Is determined to be possible. A variance value of greater than or equal to a predetermined value corresponds to a predetermined standard. The control unit 12 determines that it is impossible to specify the probability distribution calculated as the output of the type identification model G if the distribution does not satisfy the predetermined criteria.

The predetermined standard is not limited to the dispersion value being equal to or higher than the predetermined value. For example, the difference value between the first and second highest probabilities among all combinations of card types and front and back sides. May be greater than or equal to the second predetermined value. Further, the predetermined criterion may be that the highest probability among all combinations of card types and front and back sides is a third predetermined value, for example, 40% or more.

When the control unit 12 determines that it is impossible to specify (S104: NO), the control unit 12 generates reduced image data of the second surface image data (step S105) in the same manner as in step S102. The control unit 12 reduces the second surface image data by reducing the number of vertical and horizontal pixels of the second surface image data, and generates reduced image data of the second surface image data.

Next, the control unit 12 inputs the reduced image data generated in step S106 into the type identification model G (step S106), and indicates the certainty of the card type and the front and back classification as the output of the type identification model G. The probability is calculated (step S107). As described above, when the control unit 12 determines in step S105 that it is impossible to specify the first surface image data, the control unit 12 obtains the first surface image data and the second surface image data from the input unit 10. Both are input to the type-specific model G.

The control unit 12 may generate reduced image data of both the first surface image data and the second surface image data in step S101. In this case, if the control unit 12 determines in step S104 that it is impossible to perform the identification, the control unit 12 executes step S106. In step S106, the control unit 12 inputs the reduced image data of the second surface image data generated in step S101 into the type identification model G.

Next, the control unit 12 determines whether or not it is possible to specify the type of the card and the front and back sides based on the probability calculated in step S107, as in step S104 (step S108).

When the control unit 12 determines that it is impossible to specify the second surface image data (S108: NO), the control unit 12 instructs the output unit 16 to output the non-identifiable information to the display 42 (step S109). ). As a result, the display 42 displays the content indicated by the non-identifiable information, and the user is notified that the card cannot be specified.
After executing step S109, the control unit 12 ends the specific process and also ends the authenticity determination process.

When the control unit 12 determines that it is possible to specify the first surface image data (S104: YES), or when it determines that it is possible to specify the second surface image data (S104: YES). S108: YES), the type and front and back of the card inserted in the opening 5b are specified based on the calculation result calculated in step S103 or step S107 (step S110). The front and back identification is to specify whether the first side and the second side of the card are the front side or the back side, respectively. When one surface of the first surface and the second surface is specified, the other surface is automatically specified.
As described above, the control unit 12 of the determination device 1 accurately identifies the card shown in the first surface image or the second surface image by using the type identification model G.

In step S110, the control unit 12 identifies, for example, the type of card to the type of combination having the highest probability in the calculation result. Further, for example, when the highest combination in the calculation result indicates the front side, the control unit 12 determines that the reduced image data input to the type identification model G is the reduced image data of the first surface image data of the card. It is specified that the first surface and the second surface are the front side and the back side, respectively. In the same case, when the reduced image data input to the type identification model G is the reduced image data of the second side image data, the control unit 12 has the first side and the second side of the card on the back side and the front side, respectively. To identify.

When the control unit 12 determines that the identification can be performed in step S104, in step S111, the reduced image data of the first surface image data is input to the type identification model G, so that the type identification model G can be specified. Specify based on the calculation result calculated as the output. When the control unit 12 determines that the identification can be performed in step S108, in step S111, the reduced image data of the second surface image data is input to the type identification model G, so that the type identification model G can be specified. Specify based on the calculation result calculated as the output.

Next, the control unit 12 determines whether or not the type specified in step S110 is different from the plurality of types related to the target card (step S111). When the control unit 12 determines that the type specified in step S110 is different from the plurality of types related to the target card (S111: YES), the control unit 12 instructs the output unit 16 to output the non-target information to the display 42 (S). Step S112). As a result, the display 42 displays the content indicated by the non-target information, and the user is notified that the type of the card specified in step S110 is the non-target type.
After executing step S112, the control unit 12 ends the specific process and also ends the authenticity determination process.

When the control unit 12 determines that the type specified in step S110 matches one of the plurality of types related to the target card (S111: NO), the control unit 12 ends the identification process, that is, step S51 of the authenticity determination process, and authenticates the card. Step S52 of the determination process is executed. In step S52 and subsequent steps, the control unit 12 performs an authenticity determination using the image data corresponding to the front side or the back side of the card among the first screen image data and the second screen image data. That is, the first partial image data and the second partial image data are extracted from this image data. Among the image data on the front side and the back side of the card, the image data used for the authenticity determination is preset according to the type of the card. For example, when the card is a driver's license, it is preset that the image data on the front side of the card is used for authenticity determination.

The control unit 12 may perform an authenticity determination using two image data corresponding to the front side and the back side of the card. In this case, in the authenticity determination process, when the control unit 12 determines, for example, that the card shown in the image data on the front side is genuine and the card shown in the image data on the back side is genuine, the card Is determined to be genuine. When, for example, the control unit 12 determines that the card shown in the image data on the front side or the back side is not genuine, the control unit 12 determines that the card is not genuine.

Similar to the third embodiment, the control unit 12 ends the authenticity determination process when it is determined that the card is genuine (S45: YES) in the authenticity determination process. In this case, the control unit 12 executes the authentication process after completing the authenticity determination process.
FIG. 38 is a flowchart showing the procedure of the authentication process. In the authentication process, the control unit 12 first reads the card reading rule corresponding to the type specified in step S110 of the specific process from the read rule database 61 (step S121). Next, the control unit 12 obtains the personal information displayed on one or both of the front side and the back side of the card from the input unit 10 according to the reading rule read in step S121, and the first side image data and the second side. Read from the image data (step S122).

Next, the control unit 12 determines whether or not the reading of the personal information is successful (step S123). When the control unit 12 determines that the reading has failed (S123: NO), the control unit 12 instructs the output unit 16 to output the unreadable information to the display 42 (step S124). As a result, the display 42 displays the content indicated by the unreadable information, and the user is notified that the card cannot be read.

Although the authenticity is determined, there is a possibility that the reading of personal information may fail because the type specified in step S110 of the specific process is different from the actual card type.
The control unit 12 ends the authentication process after executing step S124.

When the control unit 12 determines that the reading is successful (S123: YES), the control unit 12 authenticates the owner of the card inserted in the opening 5b (step S125).
For example, when the personal information read in step S122 includes a face image showing a person's face, the control unit 12 instructs the input / output unit 15 to output a shooting signal to the camera 41. As described above, when the shooting signal is input to the camera 41, the camera 41 shoots a face image and outputs the face image data corresponding to the shot face image to the input / output unit 15. The face image data input to the input / output unit 15 is acquired from the input / output unit 15 by the control unit 12.

The control unit 12 authenticates based on whether or not the person in the face image read in step S122 is the same as the person in the face image corresponding to the face image data acquired from the input / output unit 15. If the person in the face image read in step S122 is the same as the person in the face image corresponding to the face image data acquired from the input / output unit 15, the control unit 12 determines that the authentication is successful. If the person in the face image read in step S122 is different from the person in the face image corresponding to the face image data acquired from the input / output unit 15, the control unit 12 determines that the authentication has failed.

For example, when the personal information read in step S122 includes a name, an address, and a date of birth, the control unit 12 authenticates using the authentication table 60 (see FIG. 34) in step S125. For example, when the name read in step S122 is stored in the authentication table 60, the control unit 12 has the address and date of birth read in step S122 for authentication corresponding to the name read in step S122. When it matches the address and date of birth in Table 60, it is determined that the cardholder has been successfully authenticated.

If the name read in step S122 is not stored in the authentication table 60, the control unit 12 determines that the authentication of the cardholder has failed. When the name read in step S122 is stored in the authentication table 60, the control unit 12 has the address and date of birth read in step S122 corresponding to the name read in step S126. If the address and date of birth do not match, it is determined that the authentication of the cardholder has failed.

After executing step S125, the control unit 12 instructs the output unit 16 to output the authentication result, that is, the result information indicating the success or failure of the authentication to the display 42 (step S126). As a result, the authentication result of step S125 is displayed on the display 42, and the authentication result of step S125 is notified to the user.
After executing step S126, the control unit 12 ends the authentication process.

As described above, in the determination device 1 of the authentication device 5, the control unit 12 inputs the first surface image data or the second surface image data to the type identification model G, and outputs the type of the card as the output of the type identification model G. And the probability of showing the certainty of the classification of the front and back is output. The control unit 12 specifies the type of card and the front and back sides based on this probability. If the type and front and back of the card cannot be specified by inputting the first side image data into the type identification model G, the card can be input by inputting the second side image data into the type identification model G. Identify the type and front and back of the.

Further, since the control unit 12 inputs the reduced image data to the type-specific model G, it is possible to prevent the type-specific model G from outputting the probability based on the personal information displayed on the card. When the type of card and the front and back sides are specified, the control unit 12 reads personal information from at least one of the first side image data and the second side image data acquired from the input unit 10 based on the specific result.
Further, the determination device 1 in the sixth embodiment similarly exerts the effect of the determination device 1 in the third embodiment.

In the sixth embodiment, the authentication device 5 may function as a registration device for newly registering personal information. In the authentication process, for example, when the personal information read in step S122 includes a face image showing a person's face, the control unit 12 uses the face image data taken by the camera 41 as described above. Use to authenticate. When the authentication is successful, the control unit 12 stores the personal information read in step S125, for example, the name, date of birth, address, etc. in a database (not shown) of the storage unit 11.

Further, as described in the notes of the fourth embodiment, the configuration of the fourth embodiment may be added to the configuration of the sixth embodiment, and the portion of the fourth embodiment may be added as one kind of partial image data. Image data may be used. In this case, the image pickup device 40 is configured in the same manner as the image pickup device 3 in the fourth embodiment, for example.
Further, the plurality of authenticity estimation models stored in the storage unit 11 of the determination device 1 may be relearned in the same manner as in the fifth embodiment.

(Embodiment 7)
In the sixth embodiment, the control unit 12 of the determination device 1 specifies the type and front and back of the card in the opening 5b. However, the control unit 12 may specify the type and front and back sides of the card by using the type identification model G, and may specify whether one side of the card is upward or downward.
Hereinafter, the differences between the seventh embodiment and the sixth embodiment will be described. Other configurations other than the configurations described later are common to the sixth embodiment. Therefore, the same reference reference numerals as those in the sixth embodiment are assigned to the components common to the sixth embodiment, and the description thereof will be omitted.

FIG. 39 is an explanatory view of the top and bottom of the card according to the seventh embodiment. When one long side of the opening 5b of the authentication device 5 is the front side of the card, the first side image of the first side image data when the card is inserted from one short side is from the other short side. It is different from the first side image of the first side image data when the card is inserted. The fact that one long side of the opening 5b of the authentication device 5 is the front side of the card means that, for example, in FIG. 24, the upper surface of the card is the front side. In the example of FIG. 39, the orientation of the card in which the character orientation is normal is defined as upward, and the orientation of the card in which the character orientation is inverted is defined as downward.

FIG. 40 is a schematic view of the type identification model G. In the type identification model G, the second teacher data including the reduced image data of the front side or the back side of the card, the type of the card, the front side or the back side of the card, and the second correct answer label indicating the upward or downward direction of the card is provided. By using it, the relationship between the reduced image data and the type of card, front and back, and top and bottom is learned.
In the intermediate layer, the relationship between the reduced image data, the card type, the front and back, and the top and bottom is learned based on the second teacher data.

The output layer outputs the probability indicating the type of the card shown in the reduced image of the reduced image data input to the input layer, and the certainty of the classification of the front and back and the top and bottom. Specifically, for each card type, front / back and top / bottom combination, the probability that the card type, front / back and top / bottom combinations related to the reduced image data correspond is shown. In FIG. 40, the types of cards, front and back, and top and bottom combinations are shown by (type, front or back, top or bottom). Reduced image data for each combination of (driver's license, front, top), (driver's license, front, bottom), (driver's license, back, top) and (driver's license, back, bottom) Indicates the probability that the card type, front and back, and top and bottom combinations are applicable.

The second teacher data acquired by the control unit 24 of the model generator 2 is the reduced image data of the front side or the back side of the card, the type of the card, the front side or the back side of the card, and the second upward or downward direction of the card. Includes the correct label of. The control unit 24 executes the type identification model generation process as in the sixth embodiment.

Similar to the sixth embodiment, the control unit 12 of the determination device 1 uses the type identification model G generated by the control unit 24 of the model generation device 2 in the type identification model generation process. The control unit 12 of the determination device 1 identifies the type and front and back of the card inserted in the opening 5b based on the calculation result calculated as the output of the type identification model G, and either upward or downward on one side of the card. Identify if. The model generator 2 according to the seventh embodiment similarly exhibits the effect of the model generator 2 according to the sixth embodiment.

The control unit 12 of the determination device 1 executes a specific process in the authenticity determination process as in the sixth embodiment. In the seventh embodiment, the control unit 12 calculates the probability of indicating the type of the card, the front and back, and the certainty of the upper and lower classification as the output of the type identification model G. In each of the steps S104 and S108 of the specifying process, the control unit 12 determines whether or not it is possible to specify the type, front and back, and top and bottom of the card. In step S110, the control unit 12 identifies the type and front and back of the card inserted in the opening 5b based on the calculation result calculated in step S103 or step S107, and faces upward and on the first or second surface. Identify which is downward.

As described above, in the determination device 1, the control unit 12 inputs the first surface image data or the second surface image data to the input layer of the type identification model G, and outputs the type of the card as the output of the type identification model G. Calculate the probability of showing the certainty of the front and back and top and bottom classification. Based on this probability, the control unit 12 specifies the type and front and back of the card, and also specifies whether one side of the card is upward or downward. Further, when the control unit 12 cannot specify the type, front and back, and top and bottom of the card by inputting the first surface image data into the input layer of the type identification model G, the control unit 12 inputs the second surface image data. By inputting to the input layer of the type-specific model G, the type and front and back of the card are specified, and whether one side of the card is upward or downward is specified.

The determination device 1 in the seventh embodiment similarly exhibits the effect of the determination device 1 in the sixth embodiment. In the determination device 1 according to the seventh embodiment, it is specified whether the first surface or the second surface on which the first surface image or the second surface image is captured is upward or downward, so that personal information can be read. It's easy.

(Embodiment 8)
FIG. 41 is a schematic diagram of the re-learning system according to the eighth embodiment.
Hereinafter, the differences between the eighth embodiment and the sixth embodiment will be described. Other configurations other than the configurations described later are common to the sixth embodiment. Therefore, the same reference reference numerals as those in the sixth embodiment are assigned to the components common to the sixth embodiment, and the description thereof will be omitted.

The re-learning system includes a plurality of authentication devices 5 and a model generation device 2. These are connected to network N. As will be described later, a plurality of determination devices 1 included in the plurality of authentication devices 5 are connected to the model generation device 2 via the network N. The plurality of determination devices 1 and the model generation device 2 are connected in the same manner as in the fifth embodiment. In the eighth embodiment, the determination device 1 included in each authentication device 5 transmits the type identification model G, the second teacher data, the third evaluation data, and the fourth evaluation data to the model generation device 2. The model generation device 2 relearns the type-specific model G.

FIG. 42 is a block diagram showing a main configuration of the authentication device 5. The determination device 1 of the authentication device 5 has a communication unit 14 in addition to the components included in the determination device 1 according to the sixth embodiment. The communication unit 14 is connected to the internal bus 13 and the network N as in the fifth embodiment. Each determination device 1 operates in the same manner as in the fifth embodiment. Therefore, the control unit 12 of the determination device 1 performs an update process for updating the authenticity estimation model. The first authenticity estimation model and the second authenticity estimation model are each one of the authenticity estimation models.

The communication unit 14 transmits a second update file including the type identification model G, a plurality of second teacher data, the third evaluation data, and the fourth evaluation data to the model generation device 2 according to the instruction of the control unit 12. To do. The communication unit 14 receives the type-specific model G updated by re-learning from the model generation device 2. The control unit 12 acquires the type identification model G received by the communication unit 14 from the communication unit 14. The second teacher data includes the reduced image data input to the type identification model G, the type of the card, and the second correct label indicating the front side or the back side of the card. The content indicated by the second correct answer label is based on the specific result specified by the control unit 12 in the specific process.

When the control unit 12 of the determination device 1 specifies the type and front and back of the card in step S110 in the identification process, the control unit 12 is reflected in the reduced image data input to the type identification model G and the reduced image corresponding to the reduced image data. A second teacher data including a second correct answer label indicating the type and orientation of the card is generated, and the generated second teacher data is stored in the storage unit 11. The orientation of the card indicated by the second correct label is either the front side or the back side. In the identification process, the control unit 12 inputs the reduced image data of the first side image data into the type identification model G, identifies the type of the card as a driver's license, and specifies the first side of the card as the front side. Assume. At this time, the control unit 12 generates the second teacher data including the reduced image data of the first surface image data and the driver's license and the second correct answer label indicating the front side.

The third evaluation data and the fourth evaluation data are stored in the storage unit 11 in advance, for example.
In the determination device 1, the processing element of the control unit 12 also executes a second update process for updating the type identification model G by executing the computer program P1.

FIG. 43 is a block diagram showing a main configuration of the model generation device 2. When the eighth embodiment is compared with the sixth embodiment, the model generation device 2 in the eighth embodiment includes a communication unit 26 instead of the input unit 20. The communication unit 26 is connected to the internal bus 13 and the network N as in the fifth embodiment. The model generator 2 operates in the same manner as in the fifth embodiment. Therefore, the control unit 24 of the model generation device 2 performs a re-learning process for re-learning the authenticity estimation model. As described above, each of the first authenticity estimation model and the second authenticity estimation model is one of the authenticity estimation models.

The communication unit 26 receives the second update file from the communication unit 14 of the determination device 1. The second update file received by the communication unit 26 of the model generation device 2 is acquired by the control unit 24. The communication unit 26 transmits the type identification model G to the communication unit 14 of the determination device 1 according to the instruction of the control unit 24. In the model generation device 2, the processing element of the control unit 24 executes a second re-learning process for re-learning the type-specific model G by executing the computer program P2.

FIG. 44 is a flowchart showing a procedure of processing performed by the determination device 1 and the model generation device 2. FIG. 44 shows a second update process executed by the control unit 12 of the determination device 1 and a second relearning process executed by the control unit 24 of the model generation device 2. The control unit 12 periodically executes the second update process. The control unit 24 also periodically executes the second relearning process. The second update process and the second relearning process are the same as the update process and the relearning process described in the description of the fifth embodiment.

In the second update process, the control unit 12 of the determination device 1 determines whether or not to update the type identification model G (step S131). For example, when the number of the second teacher data stored in the storage unit 11 is equal to or greater than a predetermined number, the control unit 12 determines that the type identification model G is to be updated, and the control unit 12 determines that the type identification model G is to be updated and stores it in the storage unit 11. When the number of teacher data of 2 is less than a predetermined number, it is determined that the type-specific model G is not updated.

When the control unit 12 determines that the type identification model G is to be updated (S131: YES), the control unit 12 instructs the communication unit 14, the type identification model G stored in the storage unit 11, and the plurality of second teacher data. A second update file including the third evaluation data and the fourth evaluation data is transmitted to the communication unit 26 of the model generation device 2 via the network N (step S132). The control unit 24 of the model generation device 2 determines whether or not the communication unit 26 has received the second update file from the communication unit 14 of the determination device 1 (step S141).

When the control unit 24 of the model generation device 2 determines that the communication unit 26 has received the type identification model G and the second teacher data (S141: YES), the type identification model G and the second type identification model G received by the communication unit 26 The teacher data of the above is stored in the storage unit 23 (step S142), and the type identification model generation process shown in FIGS. 30 and 31 is executed (step S143). As a result, the type-specific model G received by the communication unit 26 is relearned. The second teacher data used in the re-learning of the type-specific model G is the reduced image data input to the type-specific model G by the control unit 12 of the determination device 1 and the card specified by the control unit 12 in step S110 of the specific process. Includes a second teacher data indicating the orientation (front or back) of.
In the type identification model generation process executed in step S143, since the route has already been formed, the process is started from step S82 without executing step S81.

Next, the control unit 24 instructs the communication unit 26 to transmit the relearned type identification model G to the communication unit 14 of the determination device 1 (step S144). The control unit 24 ends the second re-learning process when it determines that the communication unit 26 has not received the second update file (S141: NO) or after executing step S144.

After executing step S132, the control unit 12 of the determination device 1 determines whether or not the communication unit 14 has received the type identification model G from the communication unit 26 of the model generation device 2 (step S133). When the control unit 12 determines that the type-specific model G has not been received (S133: NO), the control unit 12 executes step S133 again and waits until the communication unit 14 receives the type-specific model G. When the control unit 12 determines that the communication unit 14 has received the type identification model G (S133: YES), the control unit 12 receives the type identification model G stored in the storage unit 11 by the communication unit 14. Update to (step S134).
If the control unit 12 determines that the type-specific model G is not updated (S131: NO), the control unit 12 ends the second update process after executing step S134.

The determination device 1 in the eighth embodiment similarly exhibits the effect of the determination device 1 in the sixth embodiment. Further, the type identification model G stored in the determination device 1 in the eighth embodiment is periodically improved by performing re-learning.

The control unit 24 of the model generation device 2 may execute the type-specific model generation process as in the seventh embodiment. Also in this case, in the type identification model generation process executed in step S143 of the second relearning process, step S81 is omitted, and the control unit 12 executes from step S82. Further, the type-specific model G in the eighth embodiment may be the same as the type-specific model G in the seventh embodiment. In this case, the orientation of the card indicated by the second correct label is one of the front side and the back side, and one of the upper side and the lower side. In the specific process, the control unit 12 inputs the reduced image data of the first side image data into the type identification model G, identifies the type of the card as a driver's license, and sets the orientation of the card with respect to the first side to the front side and the upper side. Suppose that you have identified. At this time, the control unit 12 generates the second teacher data including the reduced image data of the first surface image data, the driver's license, and the second correct answer label indicating the front side and the upper side.

In the re-learning of the type-specific model G in the eighth embodiment, the control unit 12 of the determination device 1 instructs the communication unit 14 to provide information indicating the parameters of the type-specific model G instead of the type-specific model G. It may be transmitted to the communication unit 26 of the model generator 2. In this case, the control unit 24 of the model generation device 2 generates the type-specific model G based on the parameters received by the communication unit 26, and relearns the generated type-specific model G. Further, the communication unit 26 may transmit information indicating the parameters of the type identification model G to the communication unit 14 instead of the type identification model G. In this case, the control unit 12 of the determination device 1 updates the parameters of the type identification model G based on the received information.

In the authentication device 5 in the sixth to eighth embodiments, the image pickup device 40 is a scanner, a camera, or the like. Examples of the camera include an industrial camera, a home camera, a mobile phone camera, a smartphone camera, and the like.

(Embodiment 9)
FIG. 45 is a block diagram showing a main configuration of the authentication system according to the ninth embodiment. The authentication system includes a determination device 1 and a user terminal 6. The determination device 1 and the user terminal 6 are connected to the network N. The user terminal 6 is operated by the user. The user terminal 6 transmits the first side image data and the second side image data of the card to the determination device 1 via the network N. Based on the first surface image data and the second surface image data received from the user terminal 6, the determination device 1 executes the authenticity determination process and the authentication process as in the determination device 1 in the sixth embodiment, and executes the authenticity determination process and the authentication process of the card. Authenticate the owner.

Hereinafter, the differences between the ninth embodiment and the sixth embodiment will be described. Other configurations other than the configurations described later are common to the sixth embodiment. Therefore, the same reference reference numerals as those in the sixth embodiment are assigned to the components common to the sixth embodiment, and the description thereof will be omitted.

The determination device 1 includes a storage unit 11, a control unit 12, and a communication unit 17. These are connected to the internal bus 13.
The communication unit 17 receives the image data of the first side and the second side of the card from the user terminal 6. The two received image data are acquired by the control unit 12. Further, the communication unit 17 transmits the non-specified information, the non-target information, the unreadable information, and the result information to the user terminal 6 according to the instruction of the control unit 12.

Each of the authenticity determination process and the authentication process in the ninth embodiment is the same as the authenticity determination process and the authentication process in the sixth embodiment. When the communication unit 17 receives the image data of the first surface and the second surface from the user terminal 6, the control unit 12 of the determination device 1 sequentially executes the authenticity determination process and the authentication process. In step S125 of the authentication process according to the ninth embodiment, the control unit 12 authenticates based on personal information different from the face image, for example, personal information such as name, address or date of birth.

In step S126 of the authentication process in the ninth embodiment, the communication unit 17 is instructed to transmit the result information to the user terminal 6. As a result, for example, the authentication result is displayed on a display (not shown) included in the user terminal 6. In steps S109 and S112 of the specific processing and step S124 of the authentication processing in the ninth embodiment, the communication unit 17 is instructed to transmit the non-identifiable information, the non-target information, or the unreadable information to the user terminal 6. As a result, for example, the card could not be specified on the display (not shown) of the user terminal 6, the card type was a non-target card type, or the card could not be read. Is displayed.

The determination device 1 according to the ninth embodiment configured as described above has the same effect as that of the determination device 1 according to the sixth embodiment.
The type-specific model G is generated by the model generation device 2 as in the sixth embodiment.

The type-specific model G in the ninth embodiment may be the type-specific model G in the seventh embodiment. In this case, the control unit 12 of the determination device 1 specifies the type and front and back of the card, and also specifies whether one side of the card is the upper side or the lower side. Further, in the ninth embodiment, as in the eighth embodiment, the model generation device 2 is connected to the network N, and the model generation device 2 is stored in the storage unit 11 of the determination device 1. The authenticity estimation model and type. The specific model G may be retrained.

In the sixth to ninth embodiments, the type-specific model G may be a second trained model in which the output layer outputs the combination having the highest probability among the combinations of the card type and the front and back sides. .. Further, the type identification model G uses the image data corresponding to one image showing the front and back sides of the card and the second teacher data including the second correct answer label indicating the type of the card to obtain the image data. It may be a second trained model that has learned the relationship with the card type. In this case, the calculation result calculated as the type identification model G shows a plurality of probabilities related to the type, and the control unit 12 of the determination device 1 specifies the type of the card in the identification process. As described above, as the reduced image data of the second teacher data, the reduced image data on the front and back of each of the plurality of types of cards including the driver's license is used.

Further, the type-specific model G is a second learning in which the relationship between the reduced image data and the document type is learned by using the second teacher data including the second correct answer label indicating the document type instead of the card. It may be a finished model. Documents include passports, bank passbooks, student notebooks, pension notebooks, handicapped person notebooks or maternal and child health notebooks, and contracts. In this case, the calculation result calculated as the output of the type identification model G indicates a plurality of probabilities related to the type, and the control unit 12 of the determination device 1 determines the document type based on the image data of the document in the identification process. Identify. When the control unit 12 determines that the document is genuine in the authenticity determination process, the control unit 12 reads the personal information from the image data of the document according to the reading rule corresponding to the specified type in the authentication process. The image data of the document is generated by, for example, the image pickup apparatus 40 and is acquired from the image pickup apparatus 40.

Further, the image data included in the second teacher data and the image data input to the type-specific model G are not limited to the reduced image data, and may be the image data before reduction. Even in this case, the determination device 1 and the model generation device 2 in the sixth to ninth embodiments similarly perform the other effects other than the effect obtained by the reduction among the above-mentioned effects. In this configuration, for example, the image data from which the personal information has been deleted may be used as the image data of the second teacher data. Further, in the image data before reduction, the image data in which personal information is changed or deleted, or the image data in which noise is added to the image data before reduction may be used as the image data of the second teacher data. ..
Further, the determination device 1 in the first to ninth embodiments may be provided with a function of generating an authenticity estimation model performed by the model generation device 2. The determination device 1 in the sixth to ninth embodiments may be provided with a function of generating the type-specific model G performed by the model generation device 2.

The specific device according to the present disclosure includes a second acquisition unit that acquires image data of the card, image data of a plurality of types of cards including a driver's license, and a second correct answer label indicating the type and orientation of the card. The second trained model in which the relationship between the image data and the type and orientation of the card is learned using the teacher data of the above, and the image data acquired by the acquisition unit are input to the second trained model, and the above It includes a specific part that identifies the type and orientation of the card based on the output from the second trained model.
The determination device 1 functions as a specific device. The control unit 12 functions as a specific unit and an acquisition unit.

In the specific device according to the present disclosure, the orientation of the card is front and back, front and back, and up and down.

In the specific device according to the present disclosure, the acquisition unit acquires two image data of two cards, and the specific unit inputs the two image data acquired by the acquisition unit into the second trained model. To do.

The specific device according to the present disclosure includes a reduction unit that reduces the image data acquired by the acquisition unit, and the specific unit inputs the image data reduced by the reduction unit into the second learned model.
The control unit 12 also functions as a reduction unit.

The specific device according to the present disclosure includes a reading unit that reads personal information on the card in accordance with a card reading rule corresponding to the type of card specified by the specific unit.
The control unit 12 functions as a reading unit.

When the specific device according to the present disclosure cannot read the personal information on the card by the reading unit, the first information indicating that the card has been successfully identified but the card cannot be read is output. It has an information output unit.
The unreadable information corresponds to the first information. The output unit 16 or the communication unit 17 functions as the first information output unit.

In the specific device according to the present disclosure, the second trained model outputs a probability indicating the certainty of classification of card type and orientation. The specific device according to the present disclosure includes a second information output unit that outputs second information indicating that the card could not be specified when the output distribution of the probabilities does not satisfy a predetermined criterion.
The unspecified information corresponds to the second information. The output unit 16 or the communication unit 17 functions as a second information output unit.

In the specific device according to the present disclosure, when the type of the card specified by the specific unit is different from the plurality of preset types, the type of the card specified by the specific unit is not the target type. It is provided with a third information output unit that outputs the third information indicating that.
The non-target information corresponds to the third information. The output unit 16 or the communication unit 17 functions as a third information output unit.

In the generation method according to the present disclosure, the image data of a plurality of types of cards including a driver's license and the second teacher data including the second correct answer label indicating the type and orientation of the cards are acquired and acquired second. A second trained model used in the process of specifying the type and orientation of the card is generated from the image data of the card based on the teacher data of.

In the generation method according to the present disclosure, the orientation of the card is front and back, or front and back and up and down.

In the generation method according to the present disclosure, the reduced image data of the plurality of types of cards is acquired, and the acquired reduced image data is input to the second trained model as the second teacher data.

In the generation method according to the present disclosure, the deleted modified image data in which the personal information included in the image data of the second teacher data is deleted or changed is acquired, and the acquired deleted modified image data is used as the second teacher data. Include in the input of the trained model of 2.

In the generation method according to the present disclosure, additional image data in which noise is added to the image data of the card of the second teacher data is acquired, and the acquired additional image data is used as the second teacher data in the second trained model. Include in the input of.

In the computer program according to the present disclosure, the computer acquires and acquires the image data of a plurality of types of cards including a driver's license and the second teacher data including the second correct answer label indicating the type and orientation of the card. Based on the second teacher data, a process of generating a second trained model used for a process of specifying the type and orientation of the card is executed from the image data of the card.

The second learned model according to the present disclosure is an input layer in which image data of a plurality of types of cards including a driver's license are input, image data of the plurality of types of cards, and a second indicating the type and orientation of the cards. Based on the second teacher data including the correct answer label of 2, the intermediate layer that learned the relationship between the image data and the card type and orientation, and the certainty of the classification of the card type and orientation related to the image data are shown. It is provided with an output layer that outputs a probability, performs calculations in the intermediate layer based on the image data of the card input to the input layer, and classifies the type and orientation of the card related to the image data input to the input layer. The computer is made to function so as to output the probability indicating the certainty of the above from the output layer.

The technical features (constituent requirements) described in the first to ninth embodiments can be combined with each other, and by combining them, a new technical feature can be formed.
The disclosed embodiments 1-9 should be considered as exemplary in all respects and not restrictive. The scope of the present invention is indicated by the scope of claims, not the above-mentioned meaning, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Judgment device (information processing device)
3 Imaging device 12 Control unit (acquisition unit, extraction unit, input unit, calculation unit)
31 Irradiator C card G type specific model (second trained model)
P1 Computer program R Authenticity estimation model (learned model)
R1, R10, R11 1st authenticity estimation model (learned model)
R2, R20, R21 2nd authenticity estimation model (learned model)

Claims (12)

Get the image data of the card,
Extract the partial image data that is a part of the acquired image data,
The extracted partial image data is input to the trained model that has learned the relationship between the partial image data that is a part of the image data of the card and the authenticity of the card.
A computer program that causes a computer to execute the process of calculating the authenticity. Extract multiple partial image data from the acquired image data and
A plurality of extracted partial image data are input to the trained model separately,
The computer program according to claim 1, wherein the computer executes a process of determining whether or not the card is genuine based on a plurality of calculated authenticities. From the acquired image data, multiple partial image data of the same type are extracted, and
The extracted plurality of partial image data is input separately to the trained model in which the relationship is learned using the plurality of partial image data of the same type and type of the extracted partial image data.
The computer program according to claim 1 or 2, wherein a computer is made to execute a process of determining whether or not the card is genuine based on a plurality of calculated authenticities. According to claim 3, a computer is made to execute a process of extracting a plurality of partial image data corresponding to a line portion on a card or a plurality of partial image data corresponding to a pattern portion of a diagonal line on a card from the acquired image data. The listed computer program. From the acquired image data, both or one of the partial image data of the portion entirely filled with two colors on the card and the partial image data of the portion entirely filled with one color on the card can be obtained. The computer program according to claim 3, wherein a computer executes a process of extracting a plurality of partial image data including the data. There are multiple types of trained models that support multiple types of partial image data and have learned the above relationships.
Extract multiple types of partial image data from the acquired image data
Input each of the extracted multiple types of partial image data into the trained model corresponding to the type,
The computer program according to any one of claims 1 to 5, which causes a computer to execute a process of determining whether or not a card is genuine based on a plurality of calculated authenticities. Acquire the image data generated by taking a picture of the card illuminated by the irradiator,
Any of claims 1 to 6 that causes a computer to execute a process of extracting partial image data of a portion where an embossed shadow, a hologram picture, or a watermark picture appears on the card from the acquired image data. The computer program described in one. Identify the type of card based on the acquired image data,
A trained model corresponding to a specific type is selected from a plurality of trained models that correspond to a plurality of types of cards and have learned the above relationship.
The computer program according to any one of claims 1 to 7, wherein a computer executes a process of inputting the extracted partial image data into the selected trained model. The acquired image data is input to the second trained model in which the relationship between the card image data and the card type is learned.
The computer program according to claim 8, wherein the computer executes a process of specifying the type of the card based on the output of the second trained model. The acquisition unit that acquires the image data of the card,
An extraction unit that extracts partial image data that is a part of the image data acquired by the acquisition unit, and an extraction unit.
An input unit that inputs the partial image data extracted by the extraction unit into the trained model that has learned the relationship between the partial data that is a part of the image data of the card and the authenticity of the card.
An information processing device including a calculation unit for calculating the authenticity. Get the image data of the card,
Extract the partial image data that is a part of the acquired image data,
The extracted partial image data is input to the trained model that has learned the relationship between the partial data that is a part of the image data of the card and the authenticity of the card.
An information processing method in which a computer executes the process of calculating the authenticity. Acquire the teacher data in which the partial image data, which is a part of the image data of the card, is associated with the correct answer label indicating whether the card is genuine or not.
A method of generating a trained model in which a computer executes a process of generating a trained model that learns the relationship between partial image data and card authenticity based on the acquired teacher data.

Images