TWM611233U - System for examining financial service application - Google Patents

Abstract

A system for financial service application using document content recognition technology is provided. The system uses a serving system to receive a document provided by a client. In the method, a first intelligent model is applied to an image of the document for positioning first class regions that cover objects in multiple fields according to well-defined fields of the document. A second intelligent model is applied to the image of the document for identifying hot zones so as to determine second class regions covering the texts or numbers in the document. Multiple recognition regions can be determined according to correlations between the first class regions and the second class regions. The texts or numbers in the recognition regions can be recognized. A financial capability regarded as one of financial statements can be obtained or calculated according to a recognition result. The system thereby reviews a financial service application submitted by a client.

Description

Financial service application review system

The manual proposes a technology for applying for financial services, in particular, it refers to a financial service application review system that uses intelligent methods to identify the content of a document as a certification document for applying for financial services.

When applying for various loans or credit cards to the bank, the bank will ask the applicant for financial proof documents, such as passbooks and withholding vouchers, in order to assess the applicant's financial strength and repayment ability, and determine a credit limit accordingly.

For example, in the prior art, when an applicant needs a loan, it needs to fill in various documents, especially when the above-mentioned certification documents are submitted, the bank staff will then review the application documents and the certification documents. This process often consumes manpower and time.

In addition to the counter application, the existing technology also provides applicants to upload relevant certification documents through the software interface, but the bank side is still a manual review method, and the efficiency improvement is limited.

The specification discloses a financial service application review system using document content identification technology, in which the method uses an intelligent model to identify the content in the document to achieve the purpose of automatically applying for a specific financial service.

According to an embodiment, the proposed financial service application review system includes a server system with a database in which documents provided by customers are stored, and the server system runs a financial service application review method, which uses the document content identification technology. In this document content recognition technology, the proposed server system receives the document provided by the customer through a user interface, and then executes an image recognition method on the image of the document, and locates each field in the document according to multiple defined fields Multiple first-class areas covered by the middle object. In addition, a content recognition method is performed on the image of the document to obtain the characters and the correlation between the characters in the document, and a plurality of second-type areas of characters or numbers can be determined. Then, according to the correlation between the adjacent first type area and the second type area in the image, multiple recognition areas of multiple text or digital information are determined, and areas that do not need to be recognized are excluded. After recognizing the text or numbers in each recognition area, When applied to a financial service application, a financial power is calculated based on the identification result, which serves as a financial power proof to review a financial service application submitted by a customer.

The document may be a certification document used to prove the financial strength of the client, and the certification document is an image file with multiple fields of time and amount information.

In order to further understand the features and technical content of the present invention, please refer to the following detailed descriptions and drawings about the present invention. However, the drawings provided are only for reference and explanation, and are not used to limit the present invention.

The following is a specific specific embodiment to illustrate the implementation of this creation, and those skilled in the art can understand the advantages and effects of this creation from the content disclosed in this specification. This creation can be implemented or applied through other different specific embodiments, and various details in this specification can also be based on different viewpoints and applications, and various modifications and changes can be made without departing from the concept of this creation. In addition, the drawings of this creation are merely schematic illustrations, and are not depicted in actual size, and are stated in advance. The following implementations will further describe the related technical content of this creation in detail, but the disclosed content is not intended to limit the protection scope of this creation.

It should be understood that although terms such as "first", "second", and "third" may be used herein to describe various elements or signals, these elements or signals should not be limited by these terms. These terms are mainly used to distinguish one element from another, or one signal from another signal. In addition, the term "or" used in this document may include any one or a combination of more of the associated listed items depending on the actual situation.

The disclosure reveals a financial service application review system that uses document content recognition technology. One of the main purposes of the proposed system is to enable systems (such as financial institutions) to quickly review customer-specific service requests with intelligent and automated procedures, such as Credit card application and loan application services provided by financial institutions.

Figure 1 shows an example of the structure of the financial service application review system. The financial service application review system uses the document content recognition technology proposed in the disclosure. The artificial intelligence technology realized by the software method is mainly used to identify the information in the document. The purpose of financial service application review.

The illustration shows that the financial service application review system is equipped with a server system 11, in which various system-side functions are realized by software means and hardware such as a computer host, such as the fuzzy judgment module 111 that performs image blur judgment. This is before the image recognition method. The pre-processing procedures of the company use image processing technology to determine whether the images of the received documents (application documents or certification documents) are qualified; it also includes the financial certificate classification module 112 that classifies customers to provide financial proof, mainly from documents provided by customers It also includes an image correction module 113 that calibrates the received image so that subsequent identification can be performed; an object detection module 114 that detects various object information in the image by an image processing method is also proposed, which is used Perform an image recognition method on the image of the document, locate the area covered by the object in each field in the document based on multiple defined fields, such as the first type of area proposed in the embodiment; then propose to locate the proposed document (such as The location correction module 115 of the position of the information (such as Chinese, English, etc.) in the financial proof document).

Furthermore, the system proposes a text and digital recognition module 116 for recognizing text and digital content. The content recognition method is adopted to recognize the content of the image of the document, and the characters in the document and the relationship between the characters can be obtained. The area that determines the text or number, such as the second type of area proposed in the embodiment, the content recognition method continues to determine the multiple identification areas of multiple text or digital information according to the correlation between the adjacent first type area and the second type area. At the same time, areas that do not need to be identified are excluded to identify characters or numbers in multiple identification areas in the document. The system also proposes a verification correction module 117 that can verify the correctness of the identified content according to the preset format of the document, and calculates financial resources for the identified amount (providing a financial service application review system to review financial service applications) The financial calculation module 118 and so on. The server system 11 is provided with a database 13 for storing documents provided by each customer, such as application documents and certification documents for specific financial services.

The system provides customers using various client devices 101, 103 to generate electronic files of application documents and certification documents via a user interface starting from web pages or specific software programs, and upload them to the server system 11 via the network 10. In addition, financial practitioners can scan related application documents and financial proof documents through the financial service device 105, send them to the server system 11 via the network 10, and then process the above-mentioned functional modules to obtain financial information.

According to an implementation example, this example is not intended to limit the proposed creation. The above functional modules implemented with software and hardware have specific means and methods. For example, the fuzzy judgment module 111 can use Laplacian variance; The financial certificate classification module 112 can use the conventional Resnet 50 model; the image correction module 113 can use the first intelligent model (for example: CRAFT) in the document content identification method proposed in the disclosure; the object detection module 114 applies The algorithm can be one of the first intelligent model and the second intelligent model used in the document content recognition method (for example, one of CRAFT and Yolo); the algorithm technology that can be used by the text and number recognition module 116 can be conventional knowledge One of crnn+ctc and attention models; the financial calculation module 118 can use one of field value mapping and financial value logical operations.

The server system 11 can implement a server in a financial institution, and use the server to run online banking or mobile banking to provide customers with remote applications for specific financial services (such as credit cards, loans) via web pages or mobile device applications (APP), so as to be automated Process related documents to complete the application process. The document content identification technology proposed in the disclosure is applicable to various document images. Application documents such as personal identification documents can be all kinds of financial proof documents.

Fig. 2 shows a flowchart of an embodiment of a financial service application review method running in a server system.

When the user operates the client device or self-service financial service device mentioned in the above embodiment on the terminal, the request for a certain financial service is sent to the server system through the network, that is, the server system receives the request for the financial service (step S201). At this time, the server system initiates a processing flow, including sending information or generating information from a software program to request the customer to submit relevant documents, such as an application document for a certain financial service and a certification document sufficient to prove the customer’s financial resources. The system can issue a credit limit (step S203). At this time, the client can use the client device to take photos, scan or fill in the application documents and generate the images of the certification documents, and the server system receives these files and images (step S205).

At this time, the servo system runs a software program through its processor to execute the document content recognition method (step S207), which covers some image processing algorithms. Refer to the embodiment flow described in FIG. 3, one of its main purposes is to be able to use software Method or use a specific intelligent method to identify the information in the document. For the purpose of applying for financial services, the purpose is to obtain financial information (step S209), extract the information in English numbers, and further calculate the customer's financial resources (step S211). According to the information therein, the financial service application submitted by the customer is reviewed (step S213). For example, by calculating the financial resources of the customer from the documents provided by the customer and evaluating the customer's repayment ability, a credit limit can be approved accordingly.

In order to achieve the above technical objectives, a method for identifying file content is adopted, as shown in the flowchart of the embodiment shown in FIG. 3.

In this process embodiment, at the beginning, the customer generates document images at the terminal, such as scanning passbooks, withholding vouchers, and other documents that can prove the customer’s financial strength. The documents can also include application documents, which are uploaded to the server system and passed through the server. After receiving the file image (step S301), image processing algorithms can be used to perform pre-processing procedures, such as sharpness recognition to exclude images that cannot be processed, and various types of files can be classified (step S303), and image correction can be performed (step S305) ).

For example, in the pre-processing procedure of image processing, image processing technology can be used to determine whether the image is qualified (whether it is blurred or skewed). One of the methods is based on the pixel value of the entire image (grayscale value, brightness). Value, etc.) to determine whether there is a clear boundary (the English numbers of general documents should have obvious boundaries), if the pixel value is too uniform, it is judged to be blurred, and the information to be re-uploaded is proposed. If it is confirmed that the image quality of the document is qualified, if necessary, the image correction process of step S305 may be executed, including skew correction, keystone correction, color temperature correction, and so on.

Then, in step S307, an image recognition method is performed on the image of the document, which can locate multiple first-type areas covered by the objects in each field in the certification document based on the multiple defined fields, and select the objects among them. Refer to the example shown in Figure 4 for the legend. According to an embodiment, the document is a proof of financial resources, and the method will use smart technology to locate the fields in the proof document, such as the date in the passbook, summary, balance, etc., which are known and defined fields . For example, one of the embodiments adopts the first intelligent model. After the machine learning algorithm is used to complete the training of the first intelligent model, it can be applied to the received document image to select Chinese characters or text in the document image. The first type of area covered by the numbers.

In step S309, a content recognition method is performed on the image of the received document. According to the recognized characters and their related field information, the characters in the document and the relationship between the characters can be determined, and the characters or numbers can be determined. Of multiple second-class areas. For example, one of the embodiments adopts the second smart model, which can be used to determine the text or number hotspot of the document, and then according to the characters and the correlation between the characters, the second intelligent model with text or numbers is selected in the image of the document. The second category area.

After that, the first and second smart models mentioned above can be combined to obtain the appropriate recognition area. The method will determine the number of characters or characters in the document image based on the correlation between the first type area and the second type area adjacent to the document image. The multiple identification areas of the digital information and the areas that do not need to be identified at the same time are excluded (step S311), the purpose of which is mainly to identify the characters or numbers in the multiple identification areas in the document. Referring to the example shown in FIG. 6A, the correlation between the adjacent first-type area and the second-type area may be the area of the adjacent first-type area and the second-type area.

Then, in another embodiment, some steps to facilitate subsequent software procedures can be performed, such as cropping the recognition area into an independent image to obtain an image of the relevant area that needs to be recognized (step S313), so as to facilitate image processing. The technology recognizes the content (step S315). In one embodiment, in order to ensure that the automatically recognized information is available, the content can also be verified. The concept is to verify the recognized text or number according to the rules of related format and logic (step S317), and then the content is confirmed (step S319). ) To perform subsequent applications. For example, a financial institution issues a credit line from the financial information in the file based on a customer’s request for financial services.

It is mentioned here that when verifying the identified content, logic can be applied to determine whether the result is reasonable (rule base), for example, the amount in the identified passbook should have a specific format and number of digits, and the relevant certificate number should also be Have a specific format. Moreover, since documents (such as financial proof documents) can be used to prove the customer’s financial resources, especially information with multiple fields, the relationship between the information should be reasonable, and the time and amount information should also be reasonable. These are the rules for verifying content. In this way, it can be verified whether the content meets the definition, and if it is unreasonable, it will be returned or replaced by manual processing.

The above uses the intelligent model to efficiently obtain the information in the document, including text, numbers, and symbols. However, after continuous correction, the intelligent model can also increase the accuracy and optimize the model to facilitate the operation of the system. The machine learning algorithms that can be used here are introduced as follows.

According to the above-mentioned first smart model embodiment, the adopted first smart model algorithm implements a real-time object detection technology, which divides the input image into multiple regions (grids), and locates and classifies objects in each region. And calculate the bounding box of the object in it and its corresponding probability, that is, firstly identify the category of the specific object in the image of each area and calculate the probability of having the specific object to predict whether there is a preset object. According to the embodiment of the document content recognition method proposed in the disclosure, the first intelligent model is used to obtain the position of each object in various fields in the document according to the type and probability of the object in the document. In this example, the position is displayed as a text or number. Frame the first type of area judged to be covered by meaningful text or numbers on the image of the document, and each text or number frame has an area boundary. In this way, taking documents such as the customer’s bankbook or withholding voucher as an example, the first intelligent model trained by machine learning algorithms is used to input a large number of passbooks or withholding voucher images to obtain the fields recorded in various documents Information, such as date and time fields, summary fields, withdrawal and deposit fields, and balance fields, allows the system to find the position of the text or number in the document image (the first type of area).

In the embodiment of the second smart model, the object recognition technology realized by the second smart model algorithm is used to consider that the text in various forms of documents may be irregular shapes, and it is not easy to obtain by traditional image text recognition methods. Trustworthy information, so a machine learning method that can obtain the association between characters and characters to identify the content is used to learn the characters in the document and predict the association between the characters to determine whether they are the same text or number. Through repeated training of a large amount of data to optimize the model, a second intelligent model for recognizing characters or numbers is obtained. In this example, two heat maps can be generated through the second intelligent model algorithm. The first heat map is the probability of the center point of each character, and the second heat map is the probability of the center point of the space between connected characters. Using these two heat maps to obtain the word borders, determine the second type of area with text or numbers, instead of directly identifying the content itself, so it can be judged for the irregularly shaped text or number areas in the image.

After the above intelligent model, the meaningful area in the file is obtained. Take the passbook as an example, the relevant fields such as time (date), summary, amount, etc., the content recorded in the meaningful area is the user's financial information, such as salary ( Such as the average monthly salary) and the relevant content of the deposit balance and other information. Afterwards, the actual content can be obtained through text or digital recognition technology.

Figure 4 shows an example schematic diagram of the implementation of the document content identification method. This example shows one of the financial proof documents that the customer can submit, that is, the inner page of the passbook. The figure shows the sample pattern of the inner page of the passbook, which covers a variety of Chinese and English field information After applying the above-mentioned smart model, multiple recognition areas (401-411) can be identified.

According to this legend, the date (401) and DATE (402) indicate the time (403) of a transaction, such as "1050129", "1050202", etc.; the summary (404) and MEMO (405) show the actions and actions of this transaction Related transaction codes, such as: "Inter-bank withdrawal", "Inter-bank transfer in"; Withdrawal (406) and WITHDRAWAL (407) display the withdrawal amount in the transaction, such as: "$20,005.00"; Deposit (408) and DEPOSIT (409) ) Column displays the deposit amount, such as: "$18,000.00"; finally, it displays a balance (410) and BALANCE (411). This example displays the balance after several transactions, such as: "$132,747.0", "$150,747.0", etc.

It is worth mentioning that, like the example shown in the figure, although there are clear fields in the passbook, the records in the passbook are often offset due to printing positioning problems, or because there are other information in it that appear to be generally text-recognizable. Technology (such as Optical Character Recognition, OCR) cannot accurately identify the information in it, especially for identifying financial documents that require more rigorous identification technology.

According to this legend, the boxes that are not in the original passbook are displayed. This is the smart model used in the document content recognition method proposed in the disclosure book. According to the description of the above embodiment, for example, the first smart described in the above embodiment can be applied. The first intelligent model trained by the model algorithm (not used to limit the method) can locate the text or number in each field according to the multiple fields defined in the passbook, and can select the recognized For the obtained recognition area, select the first type of area covered by the text or number in this example.

Moreover, for example, the second smart model trained by the second smart model algorithm described in the above embodiment (not used to limit the method) can be used to determine the text or digital hotspot area of the passbook, and then the characters can be logically determined The relationship between the meaning of and the characters allows the intelligent model to frame the second type of area with words or numbers from the image of the proof document in the passbook.

For an embodiment of the area correlation between adjacent areas of different types, refer to FIG. 6A, which shows two adjacent areas selected by the first and second smart models. In this example, the frame-selected recognition area 601 is displayed. With recognition area two 603, the union of the two recognition areas will eventually be used as the recognition area. FIG. 6B shows that two adjacent areas of recognition area three 605 and area four 607 selected by different intelligent models are adjacent to each other, but the overlapping area of these two adjacent areas is relatively small.

Since the calculation and learning processes of different smart models are different, different learning may be seen for the same information. Therefore, the document content recognition method proposed in the disclosure book also integrates the first, second (or more) smart models. As a result, the required recognition area is determined according to the correlation between the adjacent first-type area and the second-type area. For example, the first-type area frames the maximum range covered by the category, when the first-type area and the second-type area When the overlapping area is large (such as larger than a threshold proposed by the system), it is expressed as meaningful information; on the contrary, if the overlapping area is small (such as lower than the threshold proposed by the system), it means that this area does not have meaningful content and can be discarded.

Furthermore, after the adjacent areas selected by different smart models, as shown in Fig. 6C, if the recognition area 5 609 and the recognition area 6 611 of the second type of area are included in the recognition area 7 613 of the first type of area, then the first type of area The recognition area 5 609 and the recognition area 6 611 of the second type area are merged into one recognition area.

After that, when the identification area in the certificate document such as the passbook is obtained, the image can be cropped, for example, the schematic diagram of the implementation of the document content identification method shown in FIGS. 5A and 5B is implemented.

Figure 5A shows the cutting out of the information in the time (date) field in the passbook. Figure 5B shows the cutting out information of the balance of each transaction. The balance matches the date, allowing the system to determine the customer's financial information.

According to the document content recognition method embodiment, when performing text recognition, more than one recognition method (model) is introduced, and double confirmation can be performed to eliminate errors. Also because of the differences in object recognition and image positioning by different machine learning algorithms, If the two identification results are consistent, it is the final identification result.

In summary, according to the financial service application review system using the document content recognition technology described in the above embodiments, after receiving the file image file uploaded by the client, the intelligent model can be used to quickly identify multiple identification areas in the certification file After obtaining financial information, such as calculating the balance of the passbook, the annual income in the withholding voucher, or the financial data of other supporting documents, the text or numbers in the Chinese bank can quickly process customer loan, credit card and other application requirements.

The content disclosed above is only a preferred and feasible embodiment of the present model, and does not limit the scope of the patent application of the present model. Therefore, all equivalent technical changes made by using the description and schematic content of the present model are included in the application of the present model. Within the scope of the patent.

11: Servo system 13: database 10: Internet 101, 103: client device 105: Self-service financial service device 111: fuzzy judgment module 112: Financial Certificate Classification Module 113: Image correction module 114: Object detection module 115: positioning correction module 116: Character and Number Recognition Module 117: Verify and correct the module 118: Financial calculation module 401~411: Recognition area 601: Identification Area One 603: Recognition Area Two 605: Recognition Area Three 607: Identification Area Four 609: Identification Area Five 611: Identification Area Six 613: Identification Area Seven Steps S201～S213: Document content recognition process Steps S301～S319: Financial service application review process

Figure 1 shows an embodiment diagram of the financial service application review system architecture;

Figure 2 shows a flowchart of an embodiment of a financial service application review method applied in the system;

Fig. 3 shows a flowchart of an embodiment of a method for recognizing file content;

Figure 4 shows one of the exemplary schematic diagrams of executing the method of document content recognition;

5A and 5B show the second schematic diagram of an example of executing the method for identifying the content of a document; and

6A, 6B, and 6C show exemplary schematic diagrams of confirming the recognition area in the execution of the document content recognition method.

11: Servo system

13: database

10: Internet

101, 103: client device

105: Self-service financial service device

111: fuzzy judgment module

112: Financial Certificate Classification Module

113: Image correction module

114: Object detection module

115: positioning correction module

116: Character and Number Recognition Module

117: Verify and correct the module

118: Financial calculation module

Claims (10)

A financial service application review system includes: a server system, provided with a database, the database storing a document provided by each customer through a user interface, wherein the server system includes a server, which at least includes: an object detection module Group, used to perform an image recognition method on the image of the document, locate multiple first-type areas covered by objects in each field in the document according to multiple defined fields; A content recognition method is performed on the image of the document to obtain the characters and the association between the characters in the document, and determine the plurality of second-type areas of characters or numbers, and then according to the adjacent first-type area and the second-type area The correlation determines the multiple identification areas of multiple text or digital information, and at the same time excludes areas that do not need to be identified to identify the text or numbers in the multiple identification areas in the document; and a financial calculation module for The identification result generates financial information to provide the financial service application review system for reviewing a financial service application submitted by the customer. The financial service application review system according to claim 1, wherein the servo system further includes an image correction module for correcting the received image of the document, so that subsequent identification can be performed. The financial service application review system of claim 2, wherein the servo system further includes a fuzzy judgment module, which is a pre-processing procedure before the image recognition method, and uses image processing technology to judge whether the image of the document is qualified. The financial service application review system according to claim 1, wherein the server system further includes a financial certificate classification module for classifying the document to obtain one of the financial proof documents. The financial service application review system according to claim 4, wherein the server system It also includes a positioning correction module to locate the position of the characters and numbers in the financial proof document. The financial service application review system according to claim 5, wherein the server system further includes a verification and correction module that verifies the characters or numbers recognized in the document according to the format. The financial service application review system according to claim 1, wherein the server system implements a server in a financial institution, and uses the server to run an online bank or a mobile bank to provide the client with a remote webpage or The mobile device application applies for the financial service. The financial service application review system according to any one of claim items 1 to 7, wherein the object detection module in the server system applies a first intelligent model or a second intelligent model according to a defined number Position the text or number of each field in each field, so as to select the first type of area covered by the text or number in the image of the certification document. The financial service application review system according to claim 8, wherein the text and digital recognition module uses the second intelligent model to determine the text or digital hotspot of the certification document, and based on the character and the relationship between the characters, the Select the second type of area with files or numbers in the image of the certification document. The financial service application review system according to claim 9, wherein the correlation between the adjacent first-type area and the second-type area is the area of the adjacent first-type area and the second-type area, or It is the format of the recognized value in the adjacent area of the first type and the area of the second type.

Images