TWI700664B - Text processing method and system - Google Patents

Abstract

A text processing system includes a storage module and a processing module. The storage module stores multiple destination URLs and multiple keywords. For each destination URL, the processing module obtains a text to be analyzed corresponding to the destination URL according to the destination URL, and for each destination URL to be analyzed Text, the processing module uses a word segmentation algorithm to obtain multiple word breaks corresponding to the text to be analyzed. The processing module uses the word breaks and keywords corresponding to each text to be analyzed, from the In the text to be analyzed, a plurality of target texts and at least one target segmentation corresponding to them are obtained, and the processing module uses a clustering and grouping algorithm according to the at least one target segmentation corresponding to each target text. The target text is divided into at least one cluster.

Description

Text processing method and system

The invention relates to a text processing system related to natural language processing technology, in particular to a text processing system applied to the field of money laundering prevention and control.

Anti-Money Laundering/Combating the Financing of Terrorism (AML/CFT, Anti-Money Laundering/Combating the Financing of Terrorism) laws and regulations and operating standards are becoming more and more stringent. Customer identification and customer due diligence operations are becoming more and more complicated. Manpower input for AML operations Both also increased substantially.

If the current AML name check operation touches the negative news list, you need to review each news item by item, read the news verbatim as a true alert or false alert, and also determine whether the protagonist of the news event and the customer belong to the same person. Refer to the information scattered in different internal systems and websites to determine whether the customer identity is the same. It is time-consuming to search for transaction data of each system across systems, and labor to collect and aggregate information on customers and their related associates, resulting in slow name verification operations and high risk of manual operations, such as operational errors. With the rapid growth of various financial businesses, the continuous improvement and development of suspected money laundering or terrorist transactions, and the continuous development of the AML system warning mechanism, the number of name check cases that touched negative news has also increased significantly, resulting in an overload of operators.

Therefore, in order to reduce the burden of manpower allocation and reduce misjudgments, natural language analysis is used to analyze relevant AML texts to improve the efficiency of case review, strengthen the quality of review of negative news cases and work consistency, reduce labor requirements and reduce compliance costs.

Therefore, the purpose of the present invention is to provide a text processing method using natural language analysis.

Therefore, the text processing method of the present invention is implemented by an electronic device that stores multiple URLs for linking to multiple texts to be analyzed, and multiple keywords. The text processing method includes a step (A ), one step (B), one step (C), and one step (D).

Step (A) is to use the electronic device to obtain the text to be analyzed corresponding to the destination URL according to the destination URL for each destination URL.

In step (B), for each text to be analyzed, using the electronic device, according to the text to be analyzed, a segmentation algorithm is used to obtain multiple word segments corresponding to the text to be analyzed.

Step (C) is to use the electronic device to obtain multiple target texts and at least one corresponding one from the texts to be analyzed based on the word breaks and keywords corresponding to each text to be analyzed Target hyphenation.

Step (D) is to use the electronic device to divide the target text into at least one cluster according to the at least one target segmentation corresponding to each target text by using a clustering algorithm.

Another object of the present invention is to provide a text processing system using natural language analysis.

Therefore, the text processing system of the present invention includes a storage module and a processing module electrically connected to the storage module.

The storage module stores multiple URLs for linking to multiple texts to be analyzed, and multiple keywords.

Wherein, for each destination URL, the processing module obtains the text to be analyzed corresponding to the destination URL according to the destination URL, and for each text to be analyzed, the processing module uses a word segmentation according to the text to be analyzed Algorithm to obtain multiple word breaks corresponding to the text to be analyzed. The processing module obtains multiple targets from the text to be analyzed according to the word breaks and keywords corresponding to each text to be analyzed For the text and at least one target segmentation corresponding to the text, the processing module uses a clustering algorithm to divide the target text into at least one cluster according to the at least one target segmentation corresponding to each target text.

The effect of the present invention is to obtain a plurality of target texts and the corresponding at least one target segmentation from the texts to be analyzed by the processing module, and use the clustering algorithm to obtain the target texts. The texts are divided into at least one cluster. In this way, when reviewing one by one, you only need to review any target text in each cluster to achieve the same effect as the conventional practice, greatly improving the efficiency of case review , Strengthen the consistency of the review quality and operation of negative news cases, reduce the demand for operation manpower and reduce compliance costs.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are represented by the same numbers.

Referring to FIG. 1, the text processing system of the present invention is a text processing system applied to money laundering prevention, and its embodiment includes an electronic device 1. The electronic device 1 includes a storage module 11, a display module 12, and a processing module 13 electrically connected to the storage module 11 and the display module 12. In this embodiment, it is particularly used for money laundering prevention. system.

The storage module 11 stores multiple URLs for linking to multiple texts to be analyzed, and multiple keywords. In this embodiment, the keywords are multiple money laundering keywords related to the money laundering field.

In this embodiment, the implementation of the electronic device 1 is, for example, a personal computer, a server, or a cloud host, but it is not limited to this.

Referring to FIG. 2, the text processing system applied to money laundering prevention of the present invention executes a text processing method applied to money laundering prevention to illustrate the storage module 11, the display module 12, and the computer device 1 The operation details of the components of the processing module 13. The text processing method includes a step 51, a step 52, a step 53, and a step 54.

In step 51, for each destination URL, the processing module 13 obtains the text to be analyzed corresponding to the destination URL according to the destination URL.

In step 52, for each text to be analyzed, the processing module 13 uses a word segmentation algorithm to obtain multiple word segments corresponding to the text to be analyzed according to the text to be analyzed. It is worth noting that in this embodiment, the word segmentation algorithm is a conventional technique published by [Ma, Wei-Yun and Chen, Keh-Jiann] in 2003.

In step 53, the processing module 13 obtains a plurality of target texts and at least one corresponding one from the texts to be analyzed according to the word segmentation and the money laundering keywords corresponding to each text to be analyzed. Target hyphenation.

Referring to FIG. 3, it is worth noting that step 53 further includes a sub-step 531 and a sub-step 532.

In sub-step 531, for each text to be analyzed, the processing module 13 determines whether the word segmentation corresponding to the text to be analyzed contains at least one target that matches any one of the money laundering keywords. Hyphenation. When the processing module 13 determines that the text to be analyzed has the corresponding at least one target segmentation, proceed to step 532; when the processing module 13 determines that the text to be analyzed does not have the corresponding at least one target When the word is broken, the text processing method applied to money laundering prevention is ended.

In sub-step 532, for each text to be analyzed, the processing module 13 uses the text to be analyzed as the target text, and obtains the at least one target segmentation corresponding to it.

In step 54, the processing module 13 uses a clustering algorithm to divide the target text into at least one cluster according to the at least one target word segmentation corresponding to each target text and display it on the display module 12.

4, it is worth noting that step 54 further includes a sub-step 541 and a sub-step 542.

In sub-step 541, for each target text, the processing module 13 uses a text embedding model for converting the text into a numerical vector according to the at least one target segmentation corresponding to the target text to obtain the corresponding target A text vector group of text. Wherein, the processing module 13 obtains the text vector group corresponding to each target text by using the text embedding model according to the at least one target segmentation corresponding to each target text. It is worth noting that in this embodiment, the text embedding model is PV-DBOW (Paragraph Vector-Distributed Bag of Words) published by [Le and Mikolov] in 2014, but it is not limited to this.

In sub-step 542, according to the text vector group corresponding to each target text, the clustering algorithm is used to divide the target text into the at least one cluster and display it on the display module 12. Among them, each cluster system is a tree represented by a tree structure. It is worth noting that in this embodiment, the clustering algorithm is the Balanced Iterative Reducing and Clustering using Hierarchies (Balanced Iterative Reducing and Clustering using Hierarchies) published by [Zhang et al] in 1996. , But not limited to this.

5, it is worth noting that the sub-step 542 further includes a sub-step 542A, a sub-step 542B, a sub-step 542C, a sub-step 542D, a sub-step 542E, and a sub-step 542F.

In sub-step 542A, the processing module 13 classifies a text vector group to be grouped into a candidate cluster, and the text vector group to be grouped is one of the text vector groups.

In sub-step 542B, the processing module 13 determines whether the next text vector group to be grouped belongs to one of the currently existing candidate clusters, and the next text vector group to be grouped is one of the text vector groups that have not yet been classified. One of them. When the processing module 13 determines that the next text vector group to be grouped belongs to that one of the currently existing candidate clusters, proceed to sub-step 542C; when the processing module 13 determines that the next When the text vector group to be grouped does not belong to any candidate cluster, the process sub-step 542D is performed. In particular, the processing module 13 determines the overall distance of the current candidate cluster in the vector space after the next text vector group to be grouped is added to the other of the currently existing candidate clusters Whether the distance exceeds a preset threshold to determine whether the next text vector group to be grouped belongs to one of the currently existing candidate clusters.

In sub-step 542C, the processing module 13 classifies the next text vector group to be grouped as one of the currently existing candidate clusters.

In sub-step 542D, the processing module 13 classifies the next text vector group to be grouped as another new candidate cluster.

In sub-step 542E, the processing module 13 determines whether there are any text vector groups that have not yet been classified. When the processing module 13 determines that there are text vector groups that have not yet been classified, it returns to the process sub-step 542B; when the processing module 13 determines that there are no text vector groups that have not been classified, it proceeds to the process sub-step 542F .

In sub-step 542F, the processing module 13 uses the currently existing candidate cluster as the at least one cluster and displays it on the display module 12.

In summary, the present invention is applied to a text processing system for money laundering prevention. The processing module 13 filters out a plurality of target texts related to money laundering and the corresponding at least one text from the texts to be analyzed. Target segmentation, and then using the text embedding model to convert the at least one target segmentation corresponding to each target text into the text vector group corresponding to each target text, and then using the clustering and grouping algorithm, The target texts are divided into the at least one cluster represented by the tree structure. In this way, when reviewing one by one during the verification operation, only the target text represented by the root of each cluster needs to be reviewed, namely It can achieve the same effect as the conventional practice, greatly improve the efficiency of case review, strengthen the quality and consistency of work in the review of negative news cases, and reduce labor requirements and reduce compliance costs. Therefore, it can indeed achieve the purpose of the invention.

However, the above are only examples of the present invention. When the scope of implementation of the present invention cannot be limited by this, all simple equivalent changes and modifications made in accordance with the scope of the patent application of the present invention and the content of the patent specification still belong to Within the scope of the patent for the present invention.

1: Electronic device

11: Storage module

12: Display module

13: Processing module

51~54: steps

531~532: Sub-step

541~542: Sub-step

542A~542F: sub-step

The other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, in which: Figure 1 is a block diagram illustrating an embodiment of the text processing system of the present invention; Figure 2 is a flowchart illustrating A text processing method executed by this embodiment; Figure 3 is a flowchart illustrating how the text processing method obtains a target text and the detailed flow of at least one target segmentation corresponding to it; Figure 4 is a flowchart illustrating How the text processing method obtains the text vector group of all target texts and divides them into at least one cluster of detailed processes; and Figure 5 is a flowchart illustrating how the text processing method divides all target texts into at least one cluster Detailed process.

1: Electronic device

11: Storage module

12: Display module

13: Processing module

Claims (8)

A text processing method implemented by an electronic device storing a plurality of URLs for linking to a plurality of texts to be analyzed and a plurality of keywords. The text processing method includes the following steps: (A) For each destination URL, by the electronic device, according to the destination URL, obtain the text to be analyzed corresponding to the destination URL; (B) for each text to be analyzed, by the electronic device, according to the text to be analyzed, Use a word segmentation algorithm to obtain multiple word breaks corresponding to the text to be analyzed; (C) Using the electronic device, according to the word breaks and keywords corresponding to each text to be analyzed, To analyze the text, obtain a plurality of target texts and at least one target segmentation corresponding thereto, and at least one target segmentation of each target text matches any one of the keywords; and (D) by the The electronic device uses a clustering algorithm to divide the target text into at least one cluster according to the at least one target segmentation corresponding to each target text. Step (D) includes the following steps: (D- 1) For each target text, using the electronic device, according to the at least one target segmentation corresponding to the target text, use a text embedding model for converting the text into a numeric vector to obtain a corresponding target text The text vector group, and (D-2) by the electronic device, according to the text vector group corresponding to each target text, the clustering algorithm is used to divide the target text into the at least one cluster. The text processing method according to claim 1, wherein step (C) includes the following Steps: (C-1) For each text to be analyzed, use the electronic device to determine whether the word segmentation corresponding to the text to be analyzed exists in at least one that matches any of the keywords Target segmentation; and (C-2) For each text to be analyzed, when the electronic device determines that there is a corresponding at least one target segmentation in the text to be analyzed, use the electronic device to analyze the The text is used as the target text, and the at least one target segmentation corresponding to it is obtained. The text processing method according to claim 1, wherein (D-2-1) uses the electronic device to classify a text vector group to be grouped into a candidate cluster, and the text vector group to be grouped is the text One of the vector groups; (D-2-2) Using the electronic device to determine whether the next text vector group to be grouped belongs to one of the currently existing candidate clusters, and the next text vector group to be grouped is One of the text vector groups that have not yet been classified; (D-2-3) When it is determined that the next text vector group to be grouped belongs to that one of the currently existing candidate clusters, use the The electronic device classifies the next text vector group to be grouped as one of the currently existing candidate clusters; (D-2-4) When it is determined that the next text vector group to be grouped does not belong to any In the case of a candidate cluster, use the electronic device to classify the next text vector group to be grouped as another new candidate cluster; and (D-2-5) use the electronic device to repeat step (D-2) -2) straight When all the text vector groups that have not yet been classified are classified, the currently existing candidate cluster is the at least one cluster. The text processing method according to claim 1, wherein in step (D-2), the clustering algorithm is a balanced iterative simplification and hierarchical clustering method, and each cluster is a tree structure. A text processing system includes: a storage module, which stores a plurality of link URLs for linking to a plurality of texts to be analyzed, and a plurality of keywords; a processing module, which is electrically connected to the storage module; wherein, for each A destination URL, the processing module obtains the text to be analyzed corresponding to the destination URL according to the destination URL. For each text to be analyzed, the processing module uses a word segmentation algorithm to obtain Corresponding to multiple word breaks of the text to be analyzed, the processing module obtains multiple target texts and their corresponding texts from the text to be analyzed according to the word breaks and keywords corresponding to each text to be analyzed. Corresponding at least one target word segmentation, at least one target word segmentation of each target text matches any of the keywords, the processing module uses the at least one target word segmentation corresponding to each target text A clustering algorithm is used to divide the target text into at least one cluster. For each target text, the processing module uses a method for converting the text into at least one target segmentation corresponding to the target text. The text embedding model of the numerical vector obtains a text vector group corresponding to the target text. The processing module uses the clustering algorithm to divide the target text into the text vector group corresponding to each target text. The at least one cluster. The text processing system according to claim 5, wherein, for each text to be analyzed, the processing module determines whether the word breaks corresponding to the text to be analyzed exist in any one of the keywords At least one target segmentation that matches, for each text to be analyzed, when the processing module determines that the text to be analyzed has the corresponding at least one target segmentation, the processing module uses the text to be analyzed as the Target text, and obtain the at least one target segmentation corresponding to it. The text processing system according to claim 5, wherein the processing module classifies a text vector group to be grouped into a candidate cluster, the text vector group to be grouped is one of the text vector groups, and the processing The module determines whether the next text vector group to be grouped belongs to one of the currently existing candidate clusters. The next text vector group to be grouped is one of the text vector groups that have not yet been classified. When the processing module When determining that the next text vector group to be grouped belongs to one of the currently existing candidate clusters, the processing module classifies the next text vector group to be grouped into one of the currently existing candidate clusters For that, when the processing module determines that the next text vector group to be grouped does not belong to any candidate cluster, the processing module classifies the next text vector group to be grouped as another new candidate cluster , The processing module repeatedly determines and classifies one of the next text vector groups that have not yet been classified until the classification of all the text vector groups that have not yet been classified is completed, and the current candidate cluster is Is the at least one cluster. The text processing system according to claim 5, wherein the clustering and grouping algorithm is a balanced iterative simplification and hierarchical grouping method, and each cluster is a tree-like structure.

Images