TWI759785B - System and method for recommending audit criteria based on integration of qualitative data and quantitative data - Google Patents

Abstract

A system for recommending audit criteria based on integration of qualitative data and quantitative data, which executes a method for recommending audit criteria based on integration of qualitative data and quantitative data, includes: a storage module for receiving an in-progress analysis data of a supplier audit and storing a historical analysis data; a topic model transformation module for analyzing an audit finding of the historical analysis data to obtain a topic model probability distribution so that a feature vector module can generate a feature vector set, according to a supplier business data relevant to the topic model probability distribution and the historical analysis data, as well as a feature vector value of the in-progress analysis data; a categorization module for determining a cluster to which the feature vector value belongs so that a recommendation module can, according to the determined cluster, offer recommended audit items which are pre-calculated by an audit criteria list.

Description

A system and method for integrating qualitative data and quantitative data to recommend audit criteria

The invention belongs to the field of natural language processing, and specifically relates to a recommendation system and method, in particular to a system and method for integrating qualitative data and quantitative data to recommend audit criteria.

In the past, there were financial audits such as US 8,050,988 B2 and US 2006/0106686 A1, which proposed a structured audit system for financial risks and opportunities and suggestions for financial audits from the risk perspective. Others such as US 7,885,841 B2, US 5,765,138, US 7,346,527 B2, US Patents such as 2008/019546 A1, US 8,504,412 B1 also cover automation such as audit planning and audit project generation.

Although recommendation systems using natural language processing have been published or announced in the past, such as US 2016/0148327 A1, US 2018/0165696 A1 and CN 107,807,962 B, they failed to consider the risks of suppliers and their background information such as scale, business performance and operation. Quantitative information such as time.

Therefore, in order to better recommend audit criteria, the present inventor proposes a system for recommending audit criteria by integrating qualitative data and quantitative data, including a storage module, a topic model conversion module, a feature vector module, and a normalization module. A class module and a recommendation module; the storage module is used for receiving an in-progress analysis data of the supplier audit and storing a historical analysis data of the supplier audit completed in the past, the in-progress analysis data and the history The analytical data consist of qualitative data of an audit finding and a Quantitative data of supplier operation data; the subject model conversion module is connected to the storage module for analyzing the audit findings of the historical analysis data, so as to establish and update a subject model and obtain the historical analysis data A subject model probability distribution of the audit finding, and converting the audit finding of the ongoing analysis data according to the subject model, and obtaining a subject model probability distribution of the audit finding of the ongoing analysis data ;The feature vector module is connected to the subject model conversion module and the storage module, and is used for the probability distribution of the subject model according to the historical analysis data and the supplier management data of the historical analysis data A corresponding set of feature vectors is generated, and the feature vector module is used to generate corresponding to the ongoing analysis data according to the subject model probability distribution of the ongoing analysis data and the supplier operation data of the ongoing analysis data An eigenvector value of the analysis data; the classification module is connected to the eigenvector module, and is used to perform cluster analysis on the eigenvector set into multiple clusters, and determine the eigenvector value in the A cluster that belongs to a plurality of clusters; the recommendation module is connected to the classification module and the topic model conversion module, and is used for receiving an audit criteria list used for supplier audit, and according to the characteristics The cluster to which the vector values belong, yields a corresponding one of the recommended audit criteria items on a related one of the topics.

The present invention is also a method for integrating qualitative data and quantitative data to recommend audit criteria, comprising: receiving, by a storage module, an ongoing analysis data of supplier audits and storing a historical analysis data of past supplier audits, the Both the ongoing analysis data and the historical analysis data include qualitative data of an audit finding and quantitative data of a supplier's operating data; the audit findings of the historical analysis data are analyzed by a subject model conversion module to establish and updating a topic model and obtaining a topic model probability distribution of the audit findings of the historical analysis data, and the topic model conversion module converting the audit findings of the ongoing analysis data according to the topic model, and obtaining a subject model probability distribution of the audit findings of the ongoing analysis data for a feature vector The module generates a corresponding feature vector set according to the subject model probability distribution and the supplier operation data of the historical analysis data, and the subject model for the feature vector module to use based on the ongoing analysis data The probability distribution and the supplier operation data of the ongoing analysis data generate a feature vector value corresponding to the ongoing analysis data; a classification module is used to perform cluster analysis on the feature vector set into multiple clusters. class, and determine a cluster to which the eigenvector value belongs among the plurality of clusters, so that a recommendation module can receive a list of audit criteria for supplier auditing, and according to which eigenvector value belongs to According to the clustering, a corresponding one of the recommended audit criteria items is generated on a related topic.

Wherein, the feature vector set may use K-means clustering to perform cluster analysis.

In the cluster analysis, the dimension of the feature vector for establishing the cluster analysis can be reduced by a weighted K-means feature selection algorithm.

Wherein, the classification module can calculate a distance value between the feature vector value and the centroid of each of the clusters, and the cluster with the smallest distance value is used as the cluster to which the feature vector value belongs. clustering.

Wherein, the topic model can be established by at least one of Latent Dirichlet Allocation (LDA) or Non-Negative Matrix Factorization (NMF), and the topic model conversion module converts the historical The audit findings of the analytical data are mapped to the subject model probability distributions of the audit findings of the historical analytical data according to the subject model, and the audit findings of the ongoing analytical data are mapped to the subject model The subject model probability distribution of the audit findings of the in-progress analysis data is obtained by mapping.

Wherein, the quantitative data of the supplier's operation data may include at least any one or a combination of supplier headcount data, turnover data, and operating time data.

According to the above technical features, the following effects can be achieved:

1. The recommendation of audit criteria takes into account the background information of the supplier's operation (such as quantitative information such as scale, business performance and operating time), and will provide more suitable audit criteria than only natural language processing recommendations.

2. Regarding the qualitative information and supplier-related quantitative information collected in past audits, regularly conduct cluster analysis and feature selection on suppliers through natural language processing and unsupervised learning, so as to objectively establish audit findings and business indicators correlation.

100: System

1: storage module

11: In-progress analysis data

111: In-progress audit findings

112: Ongoing Supplier Operational Data

12: Historical analysis data

121: Audit findings completed

122: Historical Supplier Operation Data

2: Theme model conversion module

3: Feature vector module

30: Feature vector collection

31: Eigenvector value

4: Classification module

40: Cluster Analysis

5: Recommended modules

50: Checklist of Auditing Standards

51: Recommended audit criteria items

S10: Modeling step

S100: Step 1 of the modeling steps

S101: Step 2 of the modeling steps

S102: Step 3 of the modeling steps

S103: Step 4 of the modeling steps

S104: Step 5 of the modeling steps

S105: Step 6 of the modeling steps

S106: Step 7 of the modeling steps

S107: Step 8 of the modeling steps

S108: Step 9 of the modeling steps

S109: Step ten of modeling steps

S110: Step Eleven of Modeling Steps

S111: Step 12 of the modeling steps

S112: Step thirteen of the modeling steps

S20: Recommended steps for auditing standards

S200: Step 1 of the recommended steps for auditing standards

S201: Step 2 of the recommended steps for auditing standards

S202: Step 3 of the recommended steps for auditing standards

S203: Step 4 of the recommended steps for auditing standards

[Figure 1] is a schematic block diagram of a system according to an embodiment of the present invention.

[Fig. 2] is a detailed flow chart including modeling steps and audit criteria recommendation steps in another embodiment of the present invention.

In view of the above technical features, the main functions of the system and method for integrating qualitative data and quantitative data to recommend audit criteria of the present invention will be clearly presented in the following embodiments.

Please refer to the first figure, which shows a system 100 for integrating qualitative data and quantitative data to recommend audit criteria according to an embodiment of the present invention, which can be implemented as a cloud system or a stand-alone device, and mainly includes a storage module 1 and a theme model A conversion module 2, a feature vector module 3, a classification module 4, and a recommendation module 5; the above-mentioned system 100 is used to implement a method for integrating qualitative data and quantitative data to recommend audit criteria according to another embodiment of the present invention ; In the following, the system 100 for integrating qualitative data and quantitative data to recommend audit criteria will be further specifically described: The storage module 1 is used to receive an ongoing analysis data 11 of supplier audits and store a historical analysis data 12 of past supplier audits; the in-progress analysis data 11 includes qualitative data, that is, an ongoing Audit findings 111, and a certain amount of information, namely an ongoing supplier operation data 112; the ongoing audit findings 111 are objective statements made by auditors on what the audited suppliers saw during the audit process. After completion, the status of the in-progress audit finding 111 is updated to a completed audit finding 121; the in-progress supplier operation data 112 is a numerical data set, which may include, but is not limited to, for example, a supplier number data, a turnover data, a business time data, etc.; the in-progress supplier business data 112 can be collected and obtained in advance, after the audit is completed, the status is updated to a historical supplier business data 122, and the historical analysis data 12 is the completed audit findings 121 and a general term for the historical supplier operating data 122 .

The topic model conversion module 2 is connected to the storage module 1, and regularly updates a topic model according to the completed audit findings 121, so as to obtain a topic model probability distribution. The topic model can be established by at least one of Latent Dirichlet Allocation (LDA) algorithm or Non-Negative Matrix Factorization (NMF). The topic model conversion module 2 uses the latest topic model to respectively perform the completed audit finding 121 stored in the storage module 1 and the in-progress audit finding 111 received by the storage module 1 . A linear combination of mapping transformations into the topic models produces the topic model probability distribution.

The feature vector module 3 is connected to the subject model conversion module 2 and the storage module 1, and reads the subject model probability distribution of the completed audit finding 121 and stores it with the storage module 1 The historical supplier operation data 122 is combined to generate a feature vector set 30. At the same time, the subject model probability distribution of the ongoing audit findings 111 is read in and combined with all the data in the storage module 1. The above-mentioned ongoing supplier operation data 112 is combined to generate a feature vector value 31 .

The classification module 4 is connected to the eigenvector module 3, and can determine an optimal number of clusters by using the least square sum algorithm within the group for the eigenvector set 30, and use the optimal number of clusters as follows. K-means clustering performs cluster analysis 40 on the feature vector set 30 and divides it into a plurality of clusters; when performing cluster analysis 40, the feature vector set 30 is based on the historical supplier The operation data 122 is combined with the subject model probability distribution of the completed audit finding 121, and each dimension of a feature vector has different contributions and influences to the results of the cluster analysis 40, so the classification module 4 can use Weighted K-means performs feature selection to reduce the dimension of the feature vector for establishing the cluster analysis 40; and determines a cluster to which the feature vector value 31 belongs to the plurality of clusters; specifically, the The classification module 4 calculates a distance value between the eigenvector value 31 and the centroid of each of the clusters, and can determine the cluster with the smallest distance value as the eigenvector value 31 the cluster to which it belongs.

Next, the recommendation module 5 connected to the categorization module 4 and the topic model conversion module 2 receives an audit criteria list 50 used for supplier audit, and determines according to the categorization module 4 The cluster to which the eigenvector value 31 belongs is obtained from the coordinates of the center of gravity of the cluster to obtain at least one topic with high correlation, and the topic model is used to calculate the term frequency-inverse document frequency (term frequency-inverse document frequency, tf-idf) to query the audit criteria list 50 and return a recommended audit criteria item 51 corresponding to each of the topics sorted by relevance.

The following embodiment and with reference to the second figure, will further describe the details of the method for integrating qualitative data and quantitative data to recommend audit criteria, which mainly includes a modeling step S10 and an audit criteria recommendation step S20. The modeling step S10 is mainly based on a completed audit finding in a storage module, a list of audit criteria and a historical supplier operation data (such as supplier number data, turnover data, operating time data, etc.) to perform aggregation. Class analysis, which can be performed only once or periodically or irregular updates. In the audit criterion recommendation step S20, a newly provided in-process audit finding and an in-progress supplier operation data can be classified to provide corresponding items of a recommended audit criterion.

The modeling step S10 includes:

Step 1 S100 of a modeling step: create an audit event, input an audit criteria list to a recommendation module, and output from the storage module an ID of all existing suppliers and the ID corresponding to the ID Completed audit findings (csv file).

Step 2 S101 of a modeling step: A topic model conversion module uses pandas tool to read the completed audit findings outputted in Step 1 S100 of the modeling step.

Step 3 S102 of a modeling step: The topic model conversion module uses the gensim tool to segment the completed audit findings in Step 2 S101 of the modeling step.

Step 4 S103 of a modeling step: The topic model conversion module uses spacy tools and NLTK (Natural Language Tool Kit) tools to stop the completed audit findings after word segmentation in Step 3 S102 of the modeling step. Pre-processing such as word removal and root extraction. It should be added that the above pandas, gensim, spacy, and NLTK are all natural language or data analysis and processing software tools of the Python programming language.

Step 5 S104 of a modeling step: The topic model conversion module converts the completed audit findings processed in Step 4 S103 of the modeling step into a term frequency space vector.

Step 6 S105 of a modeling step: The topic model conversion module uses a Latent Dirichlet Allocation (LDA) algorithm to perform the audit on the completed audit processed in Step 5 S104 of the modeling step Discovery builds a topic model and optimizes it.

Step 7 S106 of a modeling step: The topic model conversion module maps the completed audit findings into a topic model probability distribution of the topic model, that is, D=Σ φ T, where D is the The audit has been completed and found that T is the topic model and φ is the probability that T is in D.

Step 8 S107 of a modeling step: a feature vector module extracts φ and reads a certain amount of information from the storage module, that is, the historical supplier operation data V and performs a combined operation to generate a feature vector F= V +=φ, and a set of eigenvectors F ⁿ is formed by all the eigenvectors F .

Step 9 S108 of a modeling step: a classification module analyzes the feature vector set F ⁿ by using the K-means algorithm, and obtains a within-cluster sum of squares (WCSS) The optimal number of clusters k.

w _j =1，其中，w _j 為對應m個一特徵的權重集合。 Step 10 of a modeling step S109: arbitrarily specify w _j for the feature vector F of m dimension, but

w _j =1, where w _j is a weight set corresponding to m one feature.

c _il

d(x _ij ,z _lj )之(C,Z,w)，其中，C _il 為一正交矩陣，僅當i=l時為1，亦即是僅計算n個x與其所屬聚類重心Z_k的距離值。 Step 11 of a modeling step S110: Given β (β>1) and k, randomly assign a cluster center of gravity Z _k , and fix the two recursively to solve the minimum

c _il

(C, Z, w) of d ( x _ij , z _lj ), where C _il is an orthogonal matrix, which is 1 only when i=l, that is, only n x and the cluster centroid Z to which they belong are calculated The distance value of _k .

Step 12 of a modeling step S111: m=p+q, select p from m according to the size of w _j and q≧0, and r in p is the selected φ. More specifically, the modeling step Step 12 S111 is to select the first p features with a larger weight w from the m features, and the remaining q features are not selected, and among the p features, there are r features. The features are derived from the topic model probability distribution.

Step 13 S112 of a modeling step: query the audit criteria list by using tf-idf for r topics, and return an audit criteria item corresponding to each of the topics sorted by relevance.

The audit criterion recommendation step S20 includes:

Step 1 S200 of an audit criterion recommendation step: the storage module receives an in-process analysis data from a client (such as a smart phone, a notebook computer, a tablet computer, etc.), and the in-process analysis data includes the On-going supplier operating data and on-going audit findings described.

Step 2 of an audit criterion recommendation step S201 : the topic model conversion module maps the in-progress audit findings of the in-progress analysis data with the established topic model to obtain the in-progress audit findings. A subject model probability distribution, D _A =Σ φ _A T _A , where A represents the audit event. The feature vector module reads the probability distribution of the subject model found in the ongoing audit and performs a combined operation with the ongoing supplier business data in the storage module to generate a feature vector value.

Step 3 S202 of an auditing criterion recommendation step: the classification module calculates the minimum distance from each of the cluster centroids Z _k based on the p _A features of the eigenvector value, and determines a cluster C _A at that time .

Step 4 S203 of an audit criterion recommendation step: the recommendation module recommends the corresponding correspondence generated by Step 13 S112 of the modeling step in order from the center of gravity of the cluster _CA according to the degree of relevance to the topic The items of the audit criteria are sent to the client, that is, the items of the recommended audit criteria.

In this way, users can upload supplier operation data and audit findings in real time at the audit site. The audit findings are converted into topic distribution by topic model, and the supplier operation data can be integrated for unsupervised learning (such as K -means algorithm), after classifying the original clusters, and sorting the topics with higher probability in the clusters, the corresponding recommended audit criteria for each topic can be returned in sequence as a reference for audit opportunities.

Based on the descriptions of the above embodiments, one can fully understand the operation, use and effects of the present invention, but the above-mentioned embodiments are only preferred embodiments of the present invention, which should not limit the implementation of the present invention. Scope, that is, simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the description of the invention, all fall within the scope of the present invention.

S10: Modeling step

S100: Step 1 of the modeling steps

S101: Step 2 of the modeling steps

S102: Step 3 of the modeling steps

S103: Step 4 of the modeling steps

S104: Step 5 of the modeling steps

S105: Step 6 of the modeling steps

S106: Step 7 of the modeling steps

S107: Step 8 of the modeling steps

S108: Step 9 of the modeling steps

S109: Step ten of modeling steps

S110: Step Eleven of Modeling Steps

S111: Step 12 of the modeling steps

S112: Step thirteen of the modeling steps

S20: Recommended steps for auditing standards

S200: Step 1 of the recommended steps for auditing standards

S201: Step 2 of the recommended steps for auditing standards

S202: Step 3 of the recommended steps for auditing standards

S203: Step 4 of the recommended steps for auditing standards

Claims (10)

A system for integrating qualitative data and quantitative data to recommend audit criteria, comprising: a storage module for receiving an ongoing analysis data of supplier audits and storing a historical analysis data of supplier audits that have been completed in the past. Both the middle analysis data and the historical analysis data include a qualitative data of audit findings and a quantitative data of supplier operation data; a theme model conversion module is connected to the storage module for analyzing the historical analysis data. The audit finding is to establish a topic model or update the topic model, and obtain a topic model probability distribution of the audit finding of the historical analysis data, and the topic model conversion module converts according to the topic model The audit findings of the in-progress analysis data, and obtain a subject model probability distribution of the audit findings of the in-progress analysis data; a feature vector module, connecting the subject model conversion module and the storage A module for generating a corresponding feature vector set according to the subject model probability distribution of the historical analysis data and the supplier operation data of the historical analysis data, and the feature vector module is used to generate a corresponding feature vector set according to the The subject model probability distribution of the ongoing analysis data and the supplier operation data of the ongoing analysis data generate a feature vector value corresponding to the ongoing analysis data; a classification module is connected to the feature vector a module for performing cluster analysis on the feature vector set into multiple clusters, and determining a cluster to which the feature vector value belongs among the multiple clusters; and a recommendation module for connecting all clusters The classification module and the topic model conversion module are used to receive an audit criteria list used for supplier audit, and generate a corresponding recommendation on a related topic according to the cluster to which the feature vector value belongs audit criteria. The system for recommending audit criteria by integrating qualitative data and quantitative data according to claim 1, wherein the classification module calculates a distance value between the feature vector value and the centroid of each of the clusters, and uses the distance The cluster with the smallest value is taken as the cluster to which the eigenvector value belongs. According to the system for recommending auditing standards by integrating qualitative data and quantitative data in claim 1, the quantitative data of supplier operation data includes at least any one of supplier number data, turnover data, and operating time data, or combination. For a system for recommending auditing criteria by integrating qualitative data and quantitative data according to claim 1, the topic model is at least using Latent Dirichlet Allocation (LDA) or Non-Negative Matrix Factorization (Non-Negative Matrix Factorization). Once established, the topic model conversion module maps the audit findings of the historical analysis data to the topic model probability distribution of the audit findings of the historical analysis data according to the topic model, and converts the The subject model probability distribution of the audit findings of the ongoing analysis data is obtained by mapping the audit findings of the ongoing analysis data according to the subject model. A method for integrating qualitative data and quantitative data to recommend audit criteria, comprising: receiving, by a storage module, an in-process analysis data of supplier audits and storing a historical analysis data of supplier audits that have been completed in the past, the in-progress analysis data Both the data and the historical analysis data include qualitative data of an audit finding and quantitative data of a supplier's business data; the audit findings of the historical analysis data are analyzed by a subject model conversion module to establish a subject model or updating the topic model, and obtaining a topic model probability distribution of the audit findings of the historical analysis data, and the topic model conversion module converting the audit findings of the ongoing analysis data according to the topic model , and obtain a thematic model of the audit findings of the ongoing analysis data type probability distribution, for a feature vector module to generate a corresponding feature vector set according to the subject model probability distribution and the supplier operation data of the historical analysis data, and for the feature vector module to generate a corresponding feature vector set according to the The subject model probability distribution of the ongoing analysis data and the supplier operating data of the ongoing analysis data generate a feature vector value corresponding to the ongoing analysis data; and a classification module is used to classify the feature The vector set is divided into a plurality of clusters by cluster analysis, and a cluster to which the feature vector value belongs to the plurality of clusters is determined, so that a recommendation module can receive an audit criteria list for supplier auditing, And according to the cluster to which the feature vector value belongs, a corresponding recommended audit criterion item is generated for a related topic. According to the method for integrating qualitative data and quantitative data for recommendation of auditing criteria according to claim 5, wherein the feature vector set is clustered using K-means clustering. The method for recommending audit criteria by integrating qualitative data and quantitative data according to claim 5, wherein the classification module calculates a distance value between the feature vector value and the center of gravity of each of the clusters, and uses the distance The cluster with the smallest value is taken as the cluster to which the eigenvector value belongs. According to the method of integrating qualitative data and quantitative data for auditing criteria recommendation in claim 5, wherein the cluster analysis reduces the dimension of the feature vector for establishing the cluster analysis by weighted K-means feature selection algorithm. According to the method of integrating qualitative data and quantitative data for recommendation of auditing criteria according to claim 5, wherein the topic model is established by at least one of implicit Dirichlet distribution or non-negative matrix decomposition, and the topic model conversion module converts the mapping the audit findings of the historical analysis data to the topic model probability distributions of the audit findings of the historical analysis data according to the topic model, and The subject model probability distribution of the audit findings of the ongoing analysis data is obtained by mapping the audit findings of the ongoing analysis data according to the subject model. According to the method of integrating qualitative data and quantitative data for auditing standard recommendation in claim 5, the quantitative data of supplier operation data includes at least any one of supplier number data, turnover data, and operating time data, or combination.

Images