CN111192106A

CN111192106A - Information acquisition method and device based on picture identification and computer equipment

Info

Publication number: CN111192106A
Application number: CN201911242032.1A
Authority: CN
Inventors: 陈昌伟
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2019-12-06
Filing date: 2019-12-06
Publication date: 2020-05-22
Anticipated expiration: 2039-12-06
Also published as: CN111192106B

Abstract

The invention discloses an information acquisition method and device based on picture identification and computer equipment. The method comprises the following steps: the method comprises the steps of sending corresponding certificate uploading prompt information to a user terminal according to a bill output request input by the user terminal, receiving a certificate picture fed back by the user terminal, identifying according to a preset picture classification model and a preset information identification template to obtain certificate content information, sending the certificate content information to the user terminal for confirmation if a verification result of verifying the certificate content information according to a preset verification rule is in line, and feeding back successful prompt information to the user terminal if the user terminal feeds back confirmation information. The invention is based on the image classification recognition technology, and the information in the certificate picture is recognized based on the picture information recognition model, so that the efficiency of acquiring corresponding information from the certificate picture can be greatly improved.

Description

Information acquisition method and device based on picture identification and computer equipment

Technical Field

The invention relates to the technical field of computers, in particular to an information acquisition method and device based on picture recognition and computer equipment.

Background

During the process of order taking and the like of an enterprise, the enterprise needs to acquire personal information input by a user, generate order taking information matched with the personal information according to the personal information and finish order taking of the order. However, it takes time for the user to input the personal information and upload the personal information to the server, and the personal information of the user includes complex information including many characters, such as an identification number and a driver license number, so that the user is prone to have an information input error during the process of inputting the personal information, which greatly affects the efficiency of acquiring the user information by the enterprise. Therefore, the existing information acquisition method has the problem of low acquisition efficiency.

Disclosure of Invention

The embodiment of the invention provides an information acquisition method and device based on picture identification, computer equipment and a storage medium, and aims to solve the problem that the information acquisition efficiency of an information acquisition method in the prior art is low.

In a first aspect, an embodiment of the present invention provides an information acquisition method based on picture identification, including:

if a bill output request input by a user terminal is received, generating certificate uploading prompt information matched with the bill output request according to preset certificate demand information, and sending the certificate uploading prompt information to the user terminal to prompt a user of the type of the certificate required to be uploaded;

if a certificate picture fed back by the user terminal according to the certificate uploading prompt information is received, determining the classification category of the certificate picture as a target classification category according to a preset picture classification model;

acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain certificate content information from the certificate picture, wherein the certificate content information comprises image information and character information;

verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result;

if the verification result is in line, the certificate content information is sent to the user terminal for the user to confirm;

and if receiving confirmation information fed back by the user terminal according to the certificate content information, feeding back prompt information of information input success to the user terminal.

In a second aspect, an embodiment of the present invention provides an information acquiring apparatus based on picture recognition, including:

the certificate uploading prompting unit is used for generating certificate uploading prompting information matched with a bill output request according to preset certificate demand information and sending the certificate uploading prompting information to the user terminal to prompt the user of the type of the certificate required to be uploaded if the bill output request input by the user terminal is received;

the target classification type acquisition unit is used for determining the classification type of the certificate picture as a target classification type according to a preset picture classification model if the certificate picture fed back by the user terminal according to the certificate uploading prompt information is received;

the certificate content information identification unit is used for acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain certificate content information from the certificate picture, wherein the certificate content information comprises image information and character information;

the certificate content information verification unit is used for verifying whether the certificate content information accords with a preset verification rule according to the certificate type to obtain a verification result;

the certificate content information sending unit is used for sending the certificate content information to the user terminal for the user to confirm if the verification result is in line;

and the prompt information sending unit is used for feeding back prompt information of successful information input to the user terminal if receiving the confirmation information fed back by the user terminal according to the certificate content information.

In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor, when executing the computer program, implements the information acquiring method based on picture recognition according to the first aspect.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the information acquiring method based on picture identification according to the first aspect.

The embodiment of the invention provides an information acquisition method and device based on picture identification, computer equipment and a storage medium. The method comprises the steps of sending corresponding certificate uploading prompt information to a user terminal according to an input order output request, receiving a certificate picture fed back by the user terminal, identifying according to a preset picture classification model and a preset information identification template to obtain certificate content information, sending the certificate content information to the user terminal for confirmation if a verification result of verifying the certificate content information according to a preset verification rule is in accordance, and generating corresponding order output information according to a preset order output information generation rule if the user terminal feeds back confirmation information. By the method, the information in the certificate picture is identified based on the picture information identification model, and the efficiency of acquiring corresponding information from the certificate picture can be greatly improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of an information acquisition method based on picture recognition according to an embodiment of the present invention;

fig. 2 is a schematic view of an application scenario of an information acquisition method based on picture recognition according to an embodiment of the present invention;

fig. 3 is a schematic sub-flow chart of an information acquisition method based on picture recognition according to an embodiment of the present invention;

fig. 4 is another schematic flowchart of an information obtaining method based on picture recognition according to an embodiment of the present invention;

fig. 5 is a schematic view of another sub-flow of an information obtaining method based on picture recognition according to an embodiment of the present invention;

fig. 6 is a schematic view of another sub-flow of an information obtaining method based on picture recognition according to an embodiment of the present invention;

fig. 7 is a schematic block diagram of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention;

FIG. 8 is a schematic block diagram of sub-units of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention;

FIG. 9 is another schematic block diagram of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention;

FIG. 10 is a schematic block diagram of another sub-unit of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention;

FIG. 11 is a schematic block diagram of another sub-unit of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention;

FIG. 12 is a schematic block diagram of a computer device provided by an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an information obtaining method based on picture recognition according to an embodiment of the present invention; fig. 2 is a schematic view of an application scenario of the information acquisition method based on picture recognition according to the embodiment of the present invention. The information acquisition method based on the picture recognition is applied to the management server 10, the method is executed through application software installed in the management server 10, and the user terminal 20 realizes the transmission of data information through establishing network connection with the management server 10. The management server 10 is an enterprise terminal for executing an information acquisition method based on picture recognition to complete order placing, and the user terminal 20 is a terminal device for sending data information to the management server 10, such as a desktop computer, a notebook computer, a tablet computer, or a mobile phone. Fig. 2 shows only one user terminal 20 transmitting information to the management server 10, but in practical applications, the management server 10 may transmit information to a plurality of user terminals 20 at the same time.

As shown in fig. 1, the method includes steps S110 to S160.

S110, if a bill output request input by a user terminal is received, generating certificate uploading prompt information matched with the bill output request according to preset certificate demand information, and sending the certificate uploading prompt information to the user terminal to prompt a user of the type of the certificate required to be uploaded.

And if a bill output request input by the user terminal is received, generating certificate uploading prompt information matched with the bill output request according to preset certificate demand information, and sending the certificate uploading prompt information to the user terminal to prompt the user of the type of the certificate required to be uploaded. When a user purchases a product of a certain enterprise, due to the difference in price between different types of products, the price of the product corresponding to the user's demand needs to be acquired to be sent to the user. A user inputs a bill output request through a user terminal, namely the requirement information of a product required to be purchased by the user, and the bill output request only contains one product required to be purchased by the user; the management server further needs to acquire certificate information of the user after acquiring the order request to complete order output, products corresponding to the order output request are different, and the certificate required information includes certificate information required to be provided by corresponding to each product, so that certificate upload prompt information matched with the order output request can be generated according to the certificate required information and sent to the user terminal, wherein the certificate upload prompt information includes one or more certificate types.

And S120, if the certificate picture fed back by the user terminal according to the certificate uploading prompt information is received, determining the classification category of the certificate picture as a target classification category according to a preset picture classification model.

And if the certificate picture fed back by the user terminal according to the certificate uploading prompt information is received, determining the classification category of the certificate picture as a target classification category according to a preset picture classification model. After receiving the certificate uploading prompt information, the user can feed back the corresponding certificate picture to the management server according to the certificate type in the certificate uploading prompt information, and the user can shoot through a camera device (such as a camera of a mobile phone and a camera of a computer) preset on the user terminal to obtain the corresponding certificate picture and feed back the certificate picture to the management server, or directly select the certificate picture stored in the user terminal and feed back the certificate picture to the management server. The certificate pictures can contain a certificate picture or a plurality of certificate pictures, and after the management server receives the certificate pictures fed back by the user, the information in the certificate pictures can be identified to obtain the certificate content information containing the text information and the image information. The image classification model is a model for classifying the certificate images, and the certificate images can be classified according to the corresponding certificate types so as to determine the target classification categories of the certificate images.

The certificate content information contains text information and image information, the text information can be information such as name, identification card number, native place, license plate number and the like, and the image information can be information such as a non-crown photo, a vehicle appearance photo and the like.

In an embodiment, as shown in fig. 3, step S120 includes substeps S121 and S122.

And S121, calculating the matching rate between the certificate picture and the target characteristic vector of each classification category in the picture classification model according to the picture classification model.

And calculating the matching rate between the certificate picture and the target characteristic vector of each classification category in the picture classification model according to the picture classification model. The image classification model comprises a plurality of classification categories, each classification category corresponds to one target feature vector, and the specific calculation process comprises the steps of converting the certificate image into the corresponding feature vectors based on the convolutional neural network in the image classification model, and calculating the matching rate between the feature vectors and the target feature vectors corresponding to each classification category, so that the matching rate between the certificate image and each classification category can be obtained. The target feature vector is a calculation result obtained by calculating the certificate pictures of the corresponding classification classes through the picture classification model, a plurality of feature vectors corresponding to a plurality of certificate pictures included in one classification class can be calculated through the picture classification model, and the target feature vector corresponding to the classification class can be obtained by calculating the average value of the plurality of feature vectors.

For example, the resolution of the certificate picture is 600 × 600, and according to a calculation formula in a first convolution kernel of a convolution neural network, a convolution operation is performed by taking resolution 16 × 16 as a window and step length 1 to obtain a vector matrix with size 585 × 585, namely a shallow feature of the picture; according to a pooling calculation formula, performing down-sampling by taking the resolution of 13 multiplied by 13 as a window and the step length of 13 to obtain a vector matrix with the size of 45 multiplied by 45, namely the deep level characteristics of the picture; convolution operations are performed with a resolution of 5 × 5 as a window and a step size of 5 according to a calculation formula in 5 second convolution kernels to obtain 5 vector matrices with a size of 9 × 9. Calculating the obtained 5 9 x 9 vector matrixes by a first full-connection calculation formula, wherein the first full-connection calculation formula comprises five nodes in total, and each node is connected with 1 node and 9 nodesThe x 9 vector matrix is associated, that is, the values of five nodes associated with 5 9 x 9 vector matrices are calculated by five calculation formulas respectively, and the first calculation formula can be expressed as C₁＝w₁×X₁+b₁Wherein, C₁Is calculated for the first node, X₁Is the value in the first vector matrix corresponding to the picture, w₁And b₁Values of five nodes associated with the corresponding vector matrix direction can be calculated through five calculation formulas for preset parameter values in a first calculation formula associated with the first node and the first vector matrix; calculating the values of the five nodes through a second full-connection calculation formula to obtain the final feature vector of the certificate picture, wherein the calculation formula is F₁＝a₁×C₁+a₂×C₂+a₃×C₃+a₄×C₄+a₅×C₅(ii) a Wherein C is₁、C₂、C₃、C₄、C₅For the values of the five nodes associated with the vector matrix of the picture, a₁、a₂、a₃、a₄、a₅For the preset parameter values from five nodes to the final output node, since the 9 × 9 vector matrix contains 81 values, the feature vector of the picture finally obtained is a 1 × 81-dimensional vector matrix, which can be F_x＝(f₁，f₂……f₈₁) To indicate. The target feature vector is also a 1 × 81-dimensional vector matrix calculated by the above method, and the matching rate between the feature vector and the target feature vector can be calculated by a calculation formula. Specifically, the matching rate can be represented by P ═ 1- ((f)₁-g₁)²+(f₂-g₂)²+…+(f₈₁-g₈₁)²)/(g₁ ²+g₂ ²+…+g₈₁ ²) Wherein the feature vector of the certificate picture is F_x＝(f₁，f₂……f₈₁) The target feature vector is G ═ G₁，g₂……g₈₁)。

And S122, determining a classification category with the highest matching rate as a target classification category of the certificate picture according to the matching rate of the certificate picture and each classification category.

And determining a classification category with the highest matching rate as a target classification category of the certificate picture according to the matching rate of the certificate picture corresponding to each classification category. And after the matching probability corresponding to the certificate picture and each classification category is obtained through calculation, determining the classification category with the highest matching probability as the target classification category of the certificate picture, and processing the certificate picture through a picture identification method corresponding to the classification category by determining the target classification category of the certificate picture.

In an embodiment, as shown in fig. 4, step S210 is further included before step S120.

S210, training an initial image classification model according to a preset data set and preset model training rules, and taking the initial image classification model after training as the image classification model.

Training an initial image classification model according to a preset data set and preset model training rules, and taking the trained initial image classification model as the image classification model. The model training rules comprise data splitting information and parameter value adjusting rules, the preset data set comprises a plurality of pictures of the certificate corresponding to each classification type, one picture corresponds to one piece of data, the preset data set also comprises a target type corresponding to each picture, and the target type is the certificate type corresponding to the picture obtained through artificial recognition. Before the certificate picture is identified by using the picture classification model, a data set and a preset model training rule are required to be preset to train the picture classification model in the picture classification model, so that the accuracy of the picture classification model in classifying and identifying the certificate picture is improved.

In an embodiment, as shown in fig. 5, step S210 includes sub-steps S211, S212, S213, and S214.

S211, splitting the data set according to the data splitting information to obtain a training data set and a testing data set.

And splitting the data set according to the data splitting information to obtain a training data set and a testing data set. The splitting information comprises a splitting ratio for splitting the data contained in the data set, the data in the data set can be split into a training data set and a testing data set corresponding to the splitting ratio according to the splitting ratio, and the training data set and the testing data set both comprise a plurality of pictures of the certificate corresponding to each classification type.

S212, calculating a test feature vector corresponding to the test data set according to the convolutional neural network in the initial image classification model.

Calculating a test feature vector corresponding to the test data set according to a convolutional neural network in the image classification model, calculating a feature vector corresponding to each piece of data in one or more test data sets according to the convolutional neural network, acquiring a plurality of feature vectors corresponding to each certificate in the test data sets, wherein each certificate corresponds to one classification category, and calculating an average value of the plurality of feature vectors corresponding to each certificate to obtain the test feature vector.

And S213, performing iterative training on the convolutional neural network according to the parameter value adjusting rule, the test feature vector set and the training data set so as to adjust the parameter values in the convolutional neural network.

And performing iterative training on the convolutional neural network according to the parameter value adjusting rule, the test feature vector set and the training data set so as to adjust the parameter values in the convolutional neural network. Specifically, a piece of data in a training data set is obtained, a training feature vector of the piece of data is calculated according to a convolutional neural network, the matching rate between the training feature vector and the feature vector corresponding to the data target class in the test feature vector is calculated, and the parameter value of a formula in the convolutional neural network is adjusted according to the matching rate and a parameter adjustment rule, so that one-time training of the convolutional neural network is completed. And sequentially acquiring all data contained in one training data set to carry out iterative training on the convolutional neural network, so as to obtain a preliminary convolutional neural network corresponding to the training data set, and acquiring the preliminary convolutional neural network corresponding to each training data set according to the method.

The parameter adjustment rule comprises adjustment amplitude mapping information, wherein the adjustment amplitude mapping information is an adjustment value used for obtaining a formula parameter value in a convolutional neural network required by training at this time, the adjustment value corresponding to the matching rate in the adjustment amplitude mapping information is obtained according to the matching rate between a certain piece of data obtained by calculation in the training at this time and a corresponding feature vector of a data target class in a test feature vector, the parameter value in the formula can be adjusted according to the adjustment value, and if the adjustment value is positive, the parameter value in the formula is adjusted in a forward direction; and if the adjustment value is negative, performing negative adjustment on the parameter value in the formula.

The method comprises the steps of calculating the accuracy of a primary convolutional neural network corresponding to each training data set according to each training data set, specifically, calculating each piece of data in the corresponding training data set according to the primary convolutional neural network to obtain a training category corresponding to each piece of data in the training data set, comparing whether the training category of each piece of data is the same as the target category of each piece of data, calculating the percentage of the number of the data with the same category to the total data, obtaining the accuracy of the primary convolutional neural network corresponding to the training data set, and taking the primary convolutional neural network with the highest accuracy as the convolutional neural network after parameter value adjustment.

S214, calculating the test data set according to the convolutional neural network after parameter value adjustment to obtain a target feature vector corresponding to each classification category in the initial image classification model, and thus obtaining the trained initial image classification model as the image classification model.

And calculating the test data set according to the convolutional neural network after parameter value adjustment so as to obtain a target feature vector corresponding to each classification category in the initial image classification model, thereby obtaining the trained initial image classification model as the image classification model. Calculating a feature vector corresponding to each piece of data in one or more test data sets according to the convolutional neural network after parameter value adjustment, acquiring a plurality of feature vectors corresponding to each certificate in the test data sets, wherein each certificate corresponds to one classification category, and calculating an average value of the plurality of feature vectors corresponding to each certificate to acquire a target feature vector corresponding to each classification category, namely completing training of a picture classification model.

S130, acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain certificate content information from the certificate picture.

And acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain the certificate content information from the certificate picture. The certificate content information is information contained in the certificate extracted from the certificate picture, wherein the certificate content information comprises image information and character information. The information identification template comprises identification templates corresponding to all certificate types, the certificate pictures can be identified according to the identification templates corresponding to the certificate types after being classified into the corresponding certificate types, specifically, the identification template corresponding to one classification type can identify the certificate pictures corresponding to the classification type, and each identification template comprises a character identification area and an image identification area which are matched with the certificate corresponding to the identification template. After the target classification type is determined, the identification template matched with the target classification type in the information identification template can be used, specifically, the identification template firstly positions the edge of the certificate to be identified in the certificate picture, obtains the edge of the certificate to be identified, obtains a character area in the certificate picture through a character identification area in the identification template and identifies the character area to obtain character information, and obtains image information in the certificate picture through an image identification area in the identification template.

For example, the certificate content information may include image information such as a photograph of the user's crown or a photograph of the vehicle's exterior, and text information such as the user's name, age, identification number, driver license number, or license plate number.

The specific process of identifying the text information contained in the text area is that after the text area of a certain certificate picture is obtained, black pixels in the text area are compared with pixels of characters in a preset character information base to obtain characters corresponding to the black pixels, and then the text information contained in the text area can be identified.

S140, verifying whether the certificate content information accords with a preset verification rule according to the certificate type to obtain a verification result.

And verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result. The situation that all persons corresponding to a plurality of certificate pictures are inconsistent or the certificate pictures uploaded by the user cannot correspond to the certificate types in the certificate uploading prompt information one by one exists in the certificate pictures uploaded by the user, so that the certificate content information identified according to the certificate pictures needs to be verified, and the certificate content information can be verified through preset verification rules to obtain a verification result. Specifically, the certificate verification rule is rule information used for verifying whether the certificate content information corresponds to the certificate types in the certificate uploading prompt information one by one, and verifying whether all persons of all certificate pictures corresponding to the certificate content information are consistent.

In an embodiment, as shown in fig. 6, step S140 includes sub-steps S141, S142 and S143.

S141, verifying whether the certificate content information corresponds to the certificate type one by one according to the verification rule to obtain a first verification result.

And verifying whether the certificate content information corresponds to the certificate type one by one according to the verification rule to obtain a first verification result. Specifically, certificate types corresponding to each piece of information in certificate content information are obtained, whether the number of the certificate types corresponding to the certificate content information is the same as the number of the certificate types or not is judged according to a check rule, and if the number of the certificate types is not the same, a first verification result is that the certificate types do not pass; if the certificate types are the same, whether the certificate types corresponding to each piece of information in the certificate content information correspond to the certificate types or not is judged, if not, the first verification result is failed, and if so, the verification result is passed.

And S142, verifying whether all the persons of all the certificate content information are consistent or not to obtain a second verification result.

And verifying whether all the persons of all the certificate content information are consistent or not to obtain a second verification result. And acquiring the owner of each piece of information in the certificate content information, wherein each piece of information has only one owner, and verifying whether the owners of each piece of information are consistent to obtain a second verification result, wherein if the owners are consistent, the second verification result is passed, otherwise, the second verification result is not passed.

S143, obtaining a verification result whether the certificate content information accords with the verification rule according to the first verification result and the second verification result.

And acquiring a verification result whether the certificate content information conforms to the verification rule or not according to the first verification result and the second verification result. If the first verification result and the second verification result both pass, obtaining a verification result that the certificate content information accords with the verification rule; and if the first verification result or the second verification result is not passed, obtaining a verification result that the certificate content information does not accord with the verification rule.

And S150, if the verification result is in line, sending the certificate content information to the user terminal for the user to confirm.

And if the verification result is in line, the certificate content information is sent to the user terminal for the user to confirm. If the verification result is that the certificate content information is in accordance with the requirement, the certificate content information needs to be sent to the user terminal, the user receives the certificate content information obtained after the certificate picture uploaded by the user terminal is identified through the user terminal and confirms the certificate content information, if the user does not find errors in the certificate content information, the user can feed back the confirmation information to the management server, and if the user finds errors in the certificate content information, the user can feed back modification request information to the management server to modify the errors in the certificate content information.

And S160, if the confirmation information fed back by the user terminal according to the certificate content information is received, feeding back the prompt information of successful information input to the user terminal.

And if receiving confirmation information fed back by the user terminal according to the certificate content information, feeding back prompt information of information input success to the user terminal. The user can be prompted to input information successfully through the prompt information, after the management server obtains corresponding limited information, bill output information corresponding to the certificate content information can be generated according to the bill output information generation rule, wherein the bill output information generation rule comprises bill output rules corresponding to multiple products, each bill output rule comprises product quotations corresponding to various certificate information, a bill output request comprises one product required to be purchased by the user, the bill output rule corresponding to the bill output request in the bill output information generation rule is obtained, the certificate content information is matched with the certificate information in the bill output rule to obtain the product quotation matched with the certificate content information, and the bill output information is generated according to the product quotation and the certificate content information.

In the information acquisition method based on picture recognition provided by the embodiment of the invention, the corresponding certificate uploading prompt information is sent to the user terminal according to the input order output request, the certificate picture fed back by the user terminal is received, the certificate content information is obtained according to the preset picture classification model and the preset information recognition template, if the verification result of verifying the certificate content information according to the preset verification rule is in accordance, the certificate content information is sent to the user terminal for confirmation, and if the user terminal feeds back the confirmation information, the corresponding order output information is generated according to the preset order output information generation rule. By the method, the information in the certificate picture is identified based on the picture information identification model, the problem that the information input speed is low due to the fact that a user manually inputs personal information is avoided, and the efficiency of acquiring corresponding information from the certificate picture can be greatly improved.

The embodiment of the invention also provides an information acquisition device based on picture identification, which is used for executing any embodiment of the information acquisition method based on picture identification. Specifically, referring to fig. 7, fig. 7 is a schematic block diagram of an information acquisition apparatus based on picture recognition according to an embodiment of the present invention. The information acquisition apparatus based on picture recognition may be configured in the management server 10.

As shown in fig. 7, the information acquisition apparatus 100 based on picture recognition includes a certificate upload presentation unit 110, an object classification category acquisition unit 120, a certificate content information recognition unit 130, a certificate content information verification unit 140, a certificate content information transmission unit 150, and a presentation information transmission unit 160.

And the certificate uploading prompt unit 110 is used for generating certificate uploading prompt information matched with the order output request according to preset certificate demand information and sending the certificate uploading prompt information to the user terminal so as to prompt the user of the type of the certificate required to be uploaded if the order output request input by the user terminal is received.

And a target classification category obtaining unit 120, configured to determine, according to a preset image classification model, a classification category of the certificate image as a target classification category if the certificate image fed back by the user terminal according to the certificate uploading prompt information is received.

In another embodiment of the present invention, as shown in fig. 8, the target classification category obtaining unit 120 includes sub-units: a class matching ratio calculation unit 121 and a target classification class determination unit 122.

And the class matching rate calculating unit 121 is configured to calculate, according to the image classification model, a matching rate between the certificate image and a target feature vector of each classification class in the image classification model.

And the target classification category determining unit 122 is configured to determine, according to the matching rate between the certificate picture and each of the classification categories, one classification category with the highest matching rate as the target classification category of the certificate picture.

In another embodiment of the present invention, as shown in fig. 9, the information acquiring apparatus 100 based on picture identification further includes sub-units: a picture classification model training unit 210.

The image classification model training unit 210 is configured to train an initial image classification model according to a preset data set and preset model training rules, so as to use the trained initial image classification model as the image classification model.

In other embodiments of the present invention, as shown in fig. 10, the image classification model training unit 210 includes sub-units: a data set splitting unit 211, a test feature vector obtaining unit 212, a model iteration training unit 213, and a target feature vector obtaining unit 214.

A data set splitting unit 211, configured to split the data set according to the data splitting information to obtain a training data set and a test data set.

And a test feature vector obtaining unit 212, configured to calculate a test feature vector corresponding to the test data set according to the convolutional neural network in the initial image classification model.

And the model iterative training unit 213 is configured to perform iterative training on the convolutional neural network according to the parameter value adjustment rule, the test feature vector set, and the training data set, so as to adjust the parameter values in the convolutional neural network.

A target feature vector obtaining unit 214, configured to calculate the test data set according to the convolutional neural network after parameter value adjustment is performed, so as to obtain a target feature vector corresponding to each classification category in the initial image classification model, so as to obtain the trained initial image classification model as the image classification model.

And the certificate content information identification unit 130 is used for acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to obtain the certificate content information from the certificate picture.

And the certificate content information verification unit 140 is configured to verify whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result.

In another embodiment of the present invention, as shown in fig. 11, the certificate content information verification unit 140 includes sub-units: a first authentication unit 141, a second authentication unit 142, and an authentication result acquisition unit 143.

The first verification unit 141 is configured to verify whether the certificate content information corresponds to the certificate type one to one according to the verification rule to obtain a first verification result.

And the second verification unit 142 is configured to verify whether all the persons of all the certificate content information are consistent to obtain a second verification result.

A verification result obtaining unit 143, configured to obtain a verification result of whether the certificate content information conforms to the verification rule according to the first verification result and the second verification result.

And a certificate content information sending unit 150, configured to send the certificate content information to the user terminal for a user to confirm if the verification result is a match.

And the prompt information sending unit 160 is configured to, if receiving the confirmation information fed back by the user terminal according to the certificate content information, feed back the prompt information with successful information input to the user terminal.

The information acquisition device based on picture recognition provided by the embodiment of the invention is used for executing the information acquisition method based on picture recognition, corresponding certificate uploading prompt information is sent to a user terminal according to an input order output request, a certificate picture fed back by the user terminal is received, certificate content information is obtained according to a preset picture classification model and a preset information recognition template through recognition, if the verification result of verifying the certificate content information according to a preset verification rule is in accordance, the certificate content information is sent to the user terminal for confirmation, and if the user terminal feeds back the confirmation information, the corresponding order output information is generated according to a preset order output information generation rule. By the method, the information in the certificate picture is identified based on the picture information identification model, the problem that the information input speed is low due to the fact that a user manually inputs personal information is avoided, and the efficiency of acquiring corresponding information from the certificate picture can be greatly improved.

The above-mentioned information acquisition apparatus based on picture recognition may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 12.

Referring to fig. 12, fig. 12 is a schematic block diagram of a computer device according to an embodiment of the present invention.

Referring to fig. 12, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform an information acquisition method based on picture recognition.

The processor 502 is used to provide computing and control capabilities that support the operation of the overall computer device 500.

The internal memory 504 provides an environment for running the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be caused to execute an information acquisition method based on picture recognition.

The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 12 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 500 to which aspects of the present invention may be applied, and that a particular computing device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following functions: if a bill output request input by a user terminal is received, generating certificate uploading prompt information matched with the bill output request according to preset certificate demand information, and sending the certificate uploading prompt information to the user terminal to prompt a user of the type of the certificate required to be uploaded; if a certificate picture fed back by the user terminal according to the certificate uploading prompt information is received, determining the classification category of the certificate picture as a target classification category according to a preset picture classification model; acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain certificate content information from the certificate picture; verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result; if the verification result is in line, the certificate content information is sent to the user terminal for the user to confirm; and if receiving confirmation information fed back by the user terminal according to the certificate content information, feeding back prompt information of information input success to the user terminal.

In an embodiment, when executing the step of determining the classification category of the certificate picture as the target classification category according to a preset picture classification model if receiving the certificate picture fed back by the user terminal according to the certificate upload prompt information, the processor 502 executes the following operations: calculating the matching rate between the certificate picture and the target feature vector of each classification category in the picture classification model according to the picture classification model; and determining a classification category with the highest matching rate as a target classification category of the certificate picture according to the matching rate of the certificate picture corresponding to each classification category.

In an embodiment, before executing the step of determining, according to a preset image classification model, the classification category of the certificate image as a target classification category if the certificate image fed back by the user terminal according to the certificate upload prompt information is received, the processor 502 further executes the following operations: training an initial image classification model according to a preset data set and preset model training rules, and taking the trained initial image classification model as the image classification model.

In an embodiment, when the processor 502 performs the step of training an initial image classification model according to a preset data set and preset model training rules to use the trained initial image classification model as the image classification model, the following operations are performed: splitting the data set according to the data splitting information to obtain a training data set and a testing data set; calculating a test feature vector corresponding to the test data set according to the convolutional neural network in the initial picture classification model; performing iterative training on the convolutional neural network according to the parameter value adjusting rule, the test feature vector set and the training data set so as to adjust the parameter values in the convolutional neural network; and calculating the test data set according to the convolutional neural network after parameter value adjustment so as to obtain a target feature vector corresponding to each classification category in the initial image classification model, thereby obtaining the trained initial image classification model as the image classification model.

In one embodiment, the processor 502 performs the following operations when performing the step of verifying whether the certificate content information conforms to the preset verification rule according to the certificate type to obtain the verification result: verifying whether the certificate content information corresponds to the certificate types one by one according to the verification rule to obtain a first verification result; verifying whether all the persons of all the certificate content information are consistent or not to obtain a second verification result; and acquiring a verification result whether the certificate content information conforms to the verification rule or not according to the first verification result and the second verification result.

Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 12 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 12, and are not described herein again.

It should be understood that, in the embodiment of the present invention, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program when executed by a processor implements the steps of: if a bill output request input by a user terminal is received, generating certificate uploading prompt information matched with the bill output request according to preset certificate demand information, and sending the certificate uploading prompt information to the user terminal to prompt a user of the type of the certificate required to be uploaded; if a certificate picture fed back by the user terminal according to the certificate uploading prompt information is received, determining the classification category of the certificate picture as a target classification category according to a preset picture classification model; acquiring an information identification template matched with the target classification category from a preset information identification template library to identify the certificate picture so as to identify and obtain certificate content information from the certificate picture; verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result; if the verification result is in line, the certificate content information is sent to the user terminal for the user to confirm; and if receiving confirmation information fed back by the user terminal according to the certificate content information, feeding back prompt information of information input success to the user terminal.

In an embodiment, the step of determining, according to a preset image classification model, a classification category of the certificate image as a target classification category if the certificate image fed back by the user terminal according to the certificate upload prompt information is received includes: calculating the matching rate between the certificate picture and the target feature vector of each classification category in the picture classification model according to the picture classification model; and determining a classification category with the highest matching rate as a target classification category of the certificate picture according to the matching rate of the certificate picture corresponding to each classification category.

In an embodiment, before the step of determining, according to a preset image classification model, a classification category of the certificate image as a target classification category when the certificate image fed back by the user terminal according to the certificate upload prompt information is received, the method further includes: training an initial image classification model according to a preset data set and preset model training rules, and taking the trained initial image classification model as the image classification model.

In an embodiment, the step of training an initial image classification model according to a preset data set and preset model training rules to use the trained initial image classification model as the image classification model includes: splitting the data set according to the data splitting information to obtain a training data set and a testing data set; splitting the data set according to the data splitting information to obtain a training data set and a testing data set; calculating a test feature vector corresponding to the test data set according to the convolutional neural network in the initial picture classification model; performing iterative training on the convolutional neural network according to the parameter value adjusting rule, the test feature vector set and the training data set so as to adjust the parameter values in the convolutional neural network; and calculating the test data set according to the convolutional neural network after parameter value adjustment so as to obtain a target feature vector corresponding to each classification category in the initial image classification model, thereby obtaining the trained initial image classification model as the image classification model.

In an embodiment, the step of verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result includes: verifying whether the certificate content information corresponds to the certificate types one by one according to the verification rule to obtain a first verification result; verifying whether all the persons of all the certificate content information are consistent or not to obtain a second verification result; and acquiring a verification result whether the certificate content information conforms to the verification rule or not according to the first verification result and the second verification result.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a computer-readable storage medium, which includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. The computer-readable storage medium is a physical non-transitory storage medium, the computer-readable storage medium is a non-volatile storage medium, and the computer-readable storage medium may be an internal storage unit of the foregoing device, for example, a physical storage medium such as a hard disk or a memory of the device. The storage medium may also be an external storage device of the device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and other physical storage Media provided on the device.

While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An information acquisition method based on picture recognition is characterized by comprising the following steps:

2. The information acquisition method based on picture recognition according to claim 1, wherein the determining the classification category of the certificate picture as a target classification category according to a preset picture classification model comprises:

calculating the matching rate between the certificate picture and the target feature vector of each classification category in the picture classification model according to the picture classification model;

and determining a classification category with the highest matching rate as a target classification category of the certificate picture according to the matching rate of the certificate picture corresponding to each classification category.

3. The information acquisition method based on picture recognition according to claim 1, wherein before determining the classification category of the certificate picture as a target classification category according to a preset picture classification model, the method further comprises:

training an initial image classification model according to a preset data set and preset model training rules, and taking the trained initial image classification model as the image classification model.

4. The method according to claim 3, wherein the model training rules include data splitting information and parameter value adjustment rules, and the training of the initial image classification model according to the preset data set and the preset model training rules takes the initial image classification model after training as the image classification model includes:

splitting the data set according to the data splitting information to obtain a training data set and a testing data set;

calculating a test feature vector corresponding to the test data set according to the convolutional neural network in the initial picture classification model;

performing iterative training on the convolutional neural network according to the parameter value adjusting rule, the test feature vector set and the training data set so as to adjust the parameter values in the convolutional neural network;

and calculating the test data set according to the convolutional neural network after parameter value adjustment so as to obtain a target feature vector corresponding to each classification category in the initial image classification model, thereby obtaining the trained initial image classification model as the image classification model.

5. The information acquisition method based on picture recognition according to claim 1, wherein the verifying whether the certificate content information conforms to a preset verification rule according to the certificate type to obtain a verification result comprises:

verifying whether the certificate content information corresponds to the certificate types one by one according to the verification rule to obtain a first verification result;

verifying whether all the persons of all the certificate content information are consistent or not to obtain a second verification result;

and acquiring a verification result whether the certificate content information conforms to the verification rule or not according to the first verification result and the second verification result.

6. An information acquisition apparatus based on picture recognition, comprising:

7. The information acquisition apparatus based on picture recognition according to claim 6, wherein the target classification category acquisition unit includes:

the class matching rate calculation unit is used for calculating the matching rate between the certificate picture and the target feature vector of each class in the picture classification model according to the picture classification model;

and the target classification category determining unit is used for determining one classification category with the highest matching rate as the target classification category of the certificate picture according to the matching rate of the certificate picture corresponding to each classification category.

8. The information acquisition apparatus based on picture recognition according to claim 6, further comprising:

and the picture classification model training unit is used for training an initial picture classification model according to a preset data set and preset model training rules so as to take the initial picture classification model after training as the picture classification model.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the picture recognition-based information acquisition method according to any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to execute the information acquisition method based on picture recognition according to any one of claims 1 to 5.