TWM583989U - Serial number detection system - Google Patents

Abstract

A serial number detecting system includes an input module, a serial number divided area detecting module, and a serial number identifying module. Wherein, the input module accepts a picture, and the serial number segmentation area detecting module determines whether there is at least one serial type text in the picture by using a first type of neural network model, and if so, the serial number type The text is framed to form a serial number segmentation area. The serial number identification module identifies the text of the serial number in the serial number segment by a second type of neural network model to obtain a serial number.

Description

Serial number detection system

The present invention refers to a serial number detection system, and more particularly to a serial number detection system that uses a neural network for identification.

In order to distinguish or classify individual products or documents, individual products or documents are numbered. This number is generally called a serial number. In order to facilitate the recording and management, people will input these serial numbers into the computer. However, manual manual input is quite time consuming and error prone, and devices have been introduced on the market that utilize machine scanning and automatically determine the serial number. However, the relevant machines on the market for product serial number testing must be carried out under strict restrictions. When sampling the product serial number, it is often necessary to move the machine to a specified range and angle to be successfully identified, but this will also increase the time of the operation. In addition, if the serial number is displayed on a screen or a paper with a watermark printed on it, it will be more difficult to distinguish.

Therefore, how to make the machine scan the serial number without considering the specified range and angle is a subject worthy of consideration by those who have common knowledge in the field.

The purpose of the present invention is to provide a serial number detection system that does not need to define a specified range and angle when scanning a serial number. The serial number detection system includes an input module, a serial number segmentation detection module, and a serial number identification module. Wherein, the input module accepts a picture, and the serial number segmentation area detecting module determines whether there is at least one serial type in the picture by using a first type of neural network model. The word, if any, is framed by the text of the serial number to form a serial number segmentation area. The serial number identification module identifies the text of the serial number in the serial number segment by a second type of neural network model to obtain a serial number.

As described above, the first type of neural network model includes a first convolutional neural network model and a target detection neural network model. The first convolutional neural network model extracts features from the image to output a feature vector, and the target detection neural network model frames the text of the serial number according to the input of the feature vector to form the sequence segmentation region. The first convolutional neural network model is a VGG model, a ResNet model, or a DenseNet model, and the target detection neural network model is a YOLO model, a CTPN model, or an EAST model.

As described above, the second type of neural network model includes a second convolutional neural network model and a recursive neural network model. The second convolutional neural network model processes the image in the sequence segmentation region to output a sequence of characters, the recursive neural network model outputting a sequence number based on the input of the sequence of characters. Among them, the recursive neural network model implements the Connectionist Temporal Classification algorithm.

As with the serial number detection system described above, the second type of neural network model is the Seq2Seq model.

The serial number detection system as described above further includes a comparison module. The comparison module is electrically connected between the serial number identification module and a serial number database, and the comparison module compares the serial number obtained by the serial number identification module to store the serial number database. .

The image detection system includes an image pre-processing module, and the image pre-processing module is communicatively coupled between the input module and the serial number segment detection module.

The above described features and advantages will be more apparent from the following description.

10‧‧‧ images

12‧‧‧Segmentation area

12a‧‧‧ Picture sequence

30‧‧‧ Serial Number Database

40‧‧‧Image input device

100‧‧‧Sequence detection system

110‧‧‧Input module

115‧‧‧Image pre-processing module

120‧‧‧Sequence segmentation area detection module

122‧‧‧First type of neural network model

1221‧‧‧First Convolutional Neural Network Model

1223‧‧‧ Target detection neural network model

130‧‧‧Serial Identification Module

132‧‧‧Second type neural network model

1321‧‧‧Second convolutional neural network model

1323‧‧‧Recursive neural network model

140‧‧‧ alignment module

The various embodiments are described below with reference to the accompanying drawings, in which FIG. FIG. 1 illustrates an embodiment of a file information extraction and archiving system of the present invention.

FIG. 2A illustrates an image taken by the image input device.

FIG. 2B illustrates a segmentation area on the image.

FIG. 3 is a block diagram of a first type of neural network model.

Figure 4 is a block diagram of a second type of neural network model.

FIG. 5 is a schematic diagram of disassembling a sequence number division area into a plurality of picture sequences.

The present invention is best understood by reference to the detailed description and the accompanying drawings set forth herein. Various embodiments are discussed below with reference to the drawings. However, those skilled in the art will readily appreciate that the detailed description of the drawings herein is for the purpose of explanation and description For example, the teachings presented and the needs of a particular application may result in a variety of alternative and suitable methods for implementing the functionality of any of the details described herein. Thus, any method may extend beyond the specific implementation selections in the following embodiments described and illustrated.

Certain terms are used in the specification and subsequent patent applications to refer to specific elements. Pieces. Those of ordinary skill in the art should understand that different vendors may refer to the same component by different nouns. The scope of this specification and the subsequent patent application do not use the difference of the names as the means for distinguishing the elements, but the difference in function of the elements as the criterion for distinguishing. The term "including" or "including" as used throughout the specification and subsequent claims is an open term and should be interpreted as "including but not limited to". In addition, the term "coupled" or "connected" is used herein to include any direct and indirect electrical connection means. Therefore, if a first device is coupled to a second device, the first device can be directly electrically connected to the second device, or can be electrically connected to the second device through other devices or connection means. Device.

Please refer to FIG. 1. FIG. 1 illustrates an embodiment of a file information extraction and archiving system of the present invention. The serial number detecting system 100 includes an input module 110, an image pre-processing module 115, and a serial segmentation area detection. The measurement module 120, a serial number identification module 130, and a comparison module 140, wherein the comparison module 140 is connected to a serial number database 30. In the present embodiment, the serial number database 30 is, for example, a database that describes the serial number of the mobile phone. In addition, the input module 110 is electrically connected to an image input device 40. In this embodiment, the image input device 40 is a smart phone having a camera function, but can also be a digital camera. By the image input device 40 and the input module 110, the captured image 10 (for example, the photo shown in FIG. 2A) can be imported into the image pre-processing module 115. The image pre-processing module 115 can perform image pre-processing on the image, for example, direction correction, surface correction, image denoising, binarization, etc., so that the image has high contrast characteristics to facilitate subsequent processing. In this embodiment, the input module 110, the image pre-processing module 115, the serial number segmentation detection module 120, the serial number identification module 130, and the comparison module 140 are disposed on the servo end, and the servo end is, for example, One or more servers.

The pre-processed image 10 is transmitted to the serial number segmentation area detecting module 120. The serial number segmentation area detecting module 120 includes a first type of neural network model 122. The first type of neural network model 122 can image the image. The text of the serial number type is selected in the frame 10 to form at least one serial number division area 12 (a plurality of shown in FIG. 2B). It should be noted that the serial number in the serial number division area 12 exists as an image, that is, the serial number in the serial number division area 12 cannot be edited at this stage. In order to convert these serial numbers into editable serial numbers, the serial number recognition module 130 can be used. Hereinafter, a more detailed operation mechanism of the serial number divided area detecting module 120 and the serial number identifying module 130 will be described.

Referring to FIG. 3 simultaneously, the first type of neural network model 122 includes a first convolutional neural network model 1221 and a target detection neural network model 1223. The first convolutional neural network model 1221 belongs to convolution. Convolutional neural network, including convolutional layer and sampling layer (both convolutional layer and sampling layer are not drawn in the figure), wherein the convolutional layer is mainly used for feature extraction, and sampling The layer is used to reduce the parameters required by the first convolutional neural network model 1221 to avoid overfitting. The first convolutional neural network model 1221 generates a feature vector based on the input image 10, and then the feature vector is re-entered into the target detection neural network model 1223. In this embodiment, the first convolutional neural network model 1221 may be a VGG model, a ResNet model, or a DenseNet model. In addition, the target detection neural network model 1223 can be a YOLO model. Preferably, it is a CTPN model or an EAST model. After the calculation of the target detection neural network model 1223, the sequence numbers in the image 10 are framed to form the above-described sequence division area 12 (as shown in Fig. 2B).

After the serial number in the image 10 is framed to form the serial number segmentation area 12, the serial number identification module 130 identifies the serial number in the serial number division area 12 by a second type of neural network model 132. Referring to FIG. 4 simultaneously, the second type of neural network model 132 includes a second convolutional neural network model 1321 and a recursive neural network model 1323. The second convolutional neural network model 1321 and the first The convolutional neural network model 1221 is the same as the convolutional neural network, and the second convolutional neural network model 1321 can pre-judge the sequence number in the sequence division area 12. Although the second convolutional neural network model 1321 can initially determine the sequence number in the sequence number segmentation area 12, it is preferable to add the recursive neural network model 1323 to the second convolutional neural network model 1321. The serial number in the serial number division area 12 is better recognized, and the detailed detailed mechanism will be described later.

When the second convolutional neural network model 1321 recognizes the sequence number in the sequence number division area 12, the sequence number division area 12 is first split into a plurality of picture sequences 12a (see FIG. 5). For example, if the number division area 12 includes the character "S", the picture sequence 12a may be the left part of the "S" or the right part of the "S", so that the second convolutional nerve It is possible for the network model 1321 to recognize the character "S" as the two "S" characters. Or, conversely, it is also possible to recognize multiple characters into one, such as the string "llc.", and the second convolutional neural network model 1321 may treat two of the l ("ll") as one. ("l"). The recursive neural network model 1323 is a Recurrent Neural Network (RNN). Since the recursive neural network refers to the previous input, that is, it has the function of short-term memory, it can be used for the second convolution. The possible output errors of the neural network model 1321 are corrected, and the sequence numbers in the sequence division area 12 are correctly recognized.

In the present embodiment, the recursive neural network model 1323 uses, for example, the Connectionist Temporal Classification algorithm (hereinafter referred to as the CTC algorithm). At present, the CTC algorithm is mainly used in speech recognition. For detailed operation, please refer to the following page: “Sequence Modeling With CTC” (https://distill.pub/2017/ctc/)

The creator of the case found that the CTC algorithm can also be applied to the serial number identification of this case and has good effect. The main reason is that the situation of speech recognition has some in common with the situation of the serial number identification of this case. Some of the more common situations in speech recognition are: some people speak faster, some speak slower, or some people get longer in certain phonemes; and the CTC algorithm is officially developed for these situations. In the serial number identification of the case, the spacing between the characters and the characters in some serial numbers will be relatively open (corresponding to some people in the speech recognition, the speech is relatively slow), and the spacing between the characters and the characters in some serial numbers will be pulled. It is relatively compact (corresponding to some people in speech recognition speaking faster), and most of the images in this case are obtained by taking pictures, which is more likely to produce characters and words in the file because of the angle or distance of the photographer. The spacing between the elements has changed. Therefore, the creators of this case used the CTC algorithm to solve such problems and achieved good results.

Additionally, the second type of neural network model 132 can also be the Seq2Seq model. The Seq2Seq model generally includes an encoder (Encoder) and a decoder (Decoder), wherein the encoder can be a convolutional neural network, which also first splits the sequence number segmentation region 12 into a plurality of image sequences 12a (as shown in the figure). 5), and converting the picture sequence 12a into a context vector, and then inputting the context vector to the decoder, which then converts the context vector into an editable string.

It is worth noting that since capturing images (as shown in Figure 2A) involves taking pictures, different situations occur for different people, so the first type of neural network model 122 and the second type of neural network model 132 During training, images of different angles and various light environments can be input. Images under different angles and various light environments can be directly taken or obtained by computer simulation. Since the first type of neural network model 122 and the second type of neural network model 132 have images input in different situations during the training phase, even if there is a Moire pattern on the image, the first type of neural network model 122 The second type of neural network model 132 can still be correctly identified. It should be noted that in FIG. 2A, although the serial number displayed on the mobile phone screen is taken as an example, those skilled in the art should be aware that the serial number detecting system 100 of the present invention can also be applied to reading other media. The serial number displayed, for example, the serial number printed on the watermark.

After the editable serial number is obtained by the serial number identification module 130, the comparison module 140 can be used to compare the serial number with the serial number stored in the serial number database 30. In a preferred embodiment, if the serial number database 30 does not have the serial number, the serial number database 30 can register the serial number in the serial number database 30. In a preferred embodiment, if the serial number is not present in the database 30, the serial number detecting system 100 can issue a notification message to the user to allow the user to take the next action.

In summary, compared with the prior art, the serial number detection system of the present invention does not need to define a specified range and angle when scanning the serial number.

Although the present invention has been disclosed in the above preferred embodiments, it is not intended to limit the present invention, and any person skilled in the art can make some changes and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of this creation is subject to the definition of the scope of the patent application attached.

Claims (9)

A serial number detecting system comprises: a server end, which is composed of at least one server, the server end comprises: an input module, accepting a picture; a serial number segmentation area detecting module, and a first type of neural network The road model determines whether there is at least one serial type text in the picture, and if so, the text of the serial number type is framed to form a serial number divided area; and a serial number identification module is used as a second type The neural network model identifies the text of the serial number in the segmentation area to obtain a serial number; an image input device is electrically connected to the input module. The serial number detection system according to claim 1, wherein the first type of neural network model comprises a first convolutional neural network model and a target detection neural network model, the first convolutional neural network The network model extracts the feature to output a feature vector, and the target detection neural network model frames the text of the serial number according to the input of the feature vector to form the serial segmentation region. The serial number detection system according to claim 2, wherein the first convolutional neural network model is a VGG model, a ResNet model, or a DenseNet model. The serial number detection system described in claim 2 or 3, wherein the target detection neural network model is a YOLO model, a CTPN model, or an EAST model. The serial number detection system of claim 1, wherein the second type of neural network model comprises a second convolutional neural network model and a recursive neural network model, the second convolutional neural network The network model processes the image in the sequence segmentation region to output a sequence of characters, and the recursive neural network model outputs the sequence number based on the input of the sequence of characters. The serial number detection system described in claim 5, wherein the recursive neural network model implements a Connectionist Temporal Classification algorithm. The serial number detection system of claim 1, wherein the second type of neural network model is a Seq2Seq model. The serial number detecting system described in claim 1 further includes a matching module electrically connected between the serial number identifying module and a serial number database, and the comparing module is compared. Whether the serial number obtained by the serial number identification module is stored in the serial number database. The number detection system of claim 1 further includes an image pre-processing module, wherein the image pre-processing module is communicatively coupled between the input module and the serial number segment detection module.