CN116450753A - Customs service data structured management method and device, computer equipment group and storage medium - Google Patents

Abstract

The invention relates to the technical field of computer application, and discloses a customs service data structured management method, a customs service data structured management device, a computer equipment group and a storage medium. According to the method, the customs service data is processed by applying the edge computing thought for the first time, the redundancy elimination entry of the customs data to be inspected in batches can be realized by collecting and preprocessing the terminal data, the wind control type is automatically judged, the risk screening result is returned, the refined effective information is uploaded to the fusion result database, meanwhile, the service is synchronously maintained by heat deployment, so that the network bandwidth uploaded by each terminal data source is reduced, the processing pressure of a customs data center server is reduced, the management efficiency and the accuracy of a customs service data system are improved, the risk formed by high-risk cargos and illegal cargos is timely and accurately blocked, the labor cost of on-site inspection of customs ports is further reduced, and the efficient fusion of the customs service data is realized.

Description

Customs service data structured management method and device, computer equipment group and storage medium

Technical Field

The present invention relates to the field of computer applications, and in particular, to a method and apparatus for structured management of customs service data, a computer device group, and a storage medium.

Background

Customs data, which is data of life cycles of customs collection, storage, use, processing, transmission, provision, disclosure and the like in the process of performing responsibilities, is a product of development of customs technological productivity. Along with globalization of economy and rapid increase of international trade volume, aiming at the problems of large variety, large quantity, high complexity and the like of business data to be processed by a customs risk screening system, the traditional manual customs data processing method has the characteristics of complex procedure, slow response, low data multiplexing rate and inconvenient storage. Therefore, accelerating the development of digitization is a necessary choice for customs to raise the overall level.

At present, a plurality of customs service data structuring management methods exist, but the management of the life cycle of a single piece of data is emphasized, and the following defects exist in the whole: firstly, the customs clearance identification system is difficult to meet the requirements of quick clearance and risk management and control in terms of efficiency, accuracy and loading capacity, and the customs clearance identification system needs to formulate a comprehensive data processing rule and identify the capability of reading original customs data because of the problem of difficult identification caused by improper trade standard difference and unstructured customs data storage of various countries; secondly, the data storage redundancy degree of new and old systems of the customs countersignature and each provincial customs level is high, the system interlayer level is complex, the data is difficult to multiplex, and the information sharing and the collaboration efficiency of each party are low; thirdly, the risk knowledge and risk model method for using and processing data in the whole flow is single, the risk screening time is long, and the management system cannot accurately analyze the overall risk of the commodity transaction in real time in the face of various business data; and fourthly, the clusters need to be suspended for service during the updating period of the risk model system, the waiting time of users is long, or scripts need to be utilized for sequential hot deployment, the time complexity is high, the cargo port-stagnating time is prolonged, and the port cost of enterprises is increased.

Disclosure of Invention

In order to solve the technical problems, the invention aims to provide a structured management method, a structured management device, a structured management computer equipment set and a structured management storage medium for customs service data, which firstly propose to process customs service data by applying an edge computing idea, can realize redundancy elimination and input of batched customs data to be inspected, automatically judge wind control types, return risk discrimination results, upload refined effective information to a fusion result database, and synchronously maintain services by hot deployment, thereby reducing network bandwidth uploaded by each terminal data source, relieving the processing pressure of a customs data center server, improving the management efficiency and accuracy of a customs service data system, timely and accurately blocking the risk formed by high-risk goods and illegal goods, further reducing the inspection labor cost of customs port sites, and realizing efficient fusion of customs service data.

In a first aspect, the present invention provides a customs service data structured management method, which includes a service object reading stage, an accompanying document reading stage, a service object association stage, a scene discrimination stage, a parameter condition execution stage, a risk knowledge inquiry stage, a risk model calling stage, a knowledge map inquiry stage, a fusion result generation stage and a fusion result message pushing stage.

And the service object reading stage is used for filling in basic information when the service object is read, and creating a service object structure. The structure of the material is divided into two types: firstly, a common business object structure is not needed to be further configured in the business object association, a scene is not needed to be selected, and only the data object structure is added to the common business object structure; and secondly, the association relation type needs to be further configured and one-to-one scenes are selected. They need to configure the structure type and choose whether to host the table when adding the data object structure. The read result association comprises two functions, namely functions of creating, modifying, deleting and inquiring basic information of the service object, and configuring association relation fields of the service object, such as creating, modifying, deleting and inquiring.

The accompanying bill reading stage is used for extracting unstructured data information. The user can complete the tasks of the reading stage of the attached bill through the algorithms such as image preprocessing, text detection, detection correction, text recognition, information correction and the like of the unstructured bill according to the steps of importing/updating the template, setting the filtering condition of the attached bill, selecting the state of the template and the like. The read result association comprises two functions, namely, the functions of inquiring the new addition, modification, deletion, inquiry and extraction of the information field mapping table, such as the new addition, modification, deletion, inquiry and the like of the field mapping table.

The business object association stage is connected with the business object reading stage and is used for reading business association relation messages from the business object system message queue service and assembling the associated business object information into business object information according to the business object configuration rule. Considering the problems of inconsistent arrival time and uncertain time span of a data message required by an associated service queue, the information in the associated service queue needs to be stored in a database for persistent processing for a certain period of time. Meanwhile, in order to improve the processing efficiency of other subsequent data messages, other object IDs which do not arrive in the association relation message are required to be put into a waiting list, so that whether the data needed by other association service queues can be rapidly positioned when the other subsequent service data messages arrive is convenient. And initializing a service queue if the message of the association relation is received by monitoring the message, adding the message to the service queue if the message of the association relation is received, checking and assembling, and submitting the message to a scene discrimination service stage for processing. For other types of messages, only the message needs to be judged whether to be the information needed in a certain association relation message, if so, the message is spliced into the corresponding association relation message data to be processed in the next step (judging whether to finish splicing), otherwise, the message needs to be temporarily stored for convenient inquiry.

And the scene discrimination stage is connected with the service object association stage and is used for searching the corresponding scene ID. And searching the corresponding scene ID from the cache according to the data object ID, and if the direct return exists, searching the special service corresponding information table (taking the data object ID as a main key) if the direct return does not exist. If the database is successfully searched, the scene ID is cached and returned, and if the corresponding record does not exist in the database, the service object associated data cannot be fused.

The parameter related condition executing stage is connected with the scene judging stage and is used for checking the input document parameters, and the parameter related condition executing stage comprises the steps of judging whether the parameter related condition is empty in a checking parameter related condition list, whether the context data parameters required by executing the parameter related condition are empty or not and the like, and if the checking is abnormal, performing abnormal processing; if the parameter check passes, the preparation of the context data required for MVEL expression execution begins, i.e., the context data parameters are stuffed into the context variables of the MVEL execution.

The risk knowledge inquiring stage is connected with the scene judging stage and is used for inquiring and correlating the mapping relation between the risk knowledge special data object and the appointed data. Fuzzy query is performed by inputting query conditions, a result list is obtained, and then a specific structure is clicked and checked, wherein the structure is of two types: firstly, a common risk knowledge structure is not needed to be further configured; secondly, the association relation type of the special data object of the risk knowledge needs to be further established, the association of the special data object and the table structure to be associated is configured with the query field, and then specific information field extraction configuration is carried out on the established association. The query result comprises two parts of functions, namely the checking of the risk knowledge structure, the addition, modification, deletion and query of the risk knowledge special data object.

And the risk model calling stage is connected with the scene judging stage and is used for configuring the API address of the risk model, clicking the model to call the filtering condition filtering setting, selecting the scene and editing and storing the MVEL expression filtering condition. For the situation that the association relation type of the risk model needs to be set, the association setting method of the risk model is identical to the configuration method of the special data object of the risk knowledge. Because the risk model mainly realizes risk scoring through a deep learning algorithm, the time is generally long, and the occurrence of operation bottleneck of the system caused by too frequent model calling is prevented by setting a model calling filtering condition and a overtime discarding mechanism.

Optionally, the risk model is updated in real time in a synchronous manner of heat deployment, compared with other manners, the method does not need to interrupt service, the shortest time of using the new model is the shortest heat deployment time of the cluster server, the complete deployment time is the longest heat deployment time of the cluster server, the heat deployment efficiency of the cluster is improved, and the overall service effect is not seriously affected by abnormal downtime of a single server.

The knowledge graph inquiring stage is connected with the scene judging stage and is used for checking the structure of the return field of the knowledge graph; for the situation that the type of the association relation of the knowledge spectrum needs to be set, the knowledge spectrum association setting method is completely the same as the method for configuring the special data object of the risk knowledge, the knowledge spectrum association can inquire the data object of the knowledge spectrum, and meanwhile, the association fields can be subjected to functions of building, inquiring, modifying, deleting and the like.

The fusion result generation stage is connected with the accompanying bill reading stage, the scene discrimination stage, the parameter condition execution stage, the risk knowledge inquiry stage, the risk model calling stage and the knowledge spectrum inquiry stage and is used for inquiring, processing and result generation of data such as risk knowledge, knowledge spectrum, risk model, accompanying bill extraction results, parameter condition execution results and the like which are relied on in the data fusion processing process of the system. After the query interface is called, if the query fails to return the result immediately (such as the asynchronous return mode of the interface or the failure of the query), according to the implementation or program configuration of the query interface, additional result query interfaces can be called additionally or re-requested after waiting for a certain time, and the result is considered abnormal until the attempt reaches a certain condition or fails. If the result can be returned, the result data returned by each interface is generated in a fusion way and is delivered to a fusion result message pushing stage.

And the fused result message pushing stage is connected with the fused result generating stage and is used for storing the clearance document risk discrimination result. And generating a fusion demand ID according to a snowflake algorithm, analyzing fields such as a batch number, an application ID, a loading mode, a scene ID, a delay time and the like of the message, dividing the message into increment and full operation according to the loading mode, obtaining effective time according to the delay time, and storing all obtained basic information into a database.

The invention provides a customs service data management device, which comprises a data object structure maintenance module, a service object fusion setting module, an accompanying bill fusion setting module, a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge map fusion setting module, a parameter table management module and a scene data fusion requirement synchronization module.

The data object structure maintenance module is used for designing a data object structure and comprises the basic information of the data object and the functions of creating, editing, inquiring, deleting and the like of the common fields and the special fields contained in the basic information. Necessary functional support is provided for other modules such as business object structure maintenance, risk model fusion setting and the like.

The business object fusion setting module is used for business object structure maintenance and business object association. A business object is an important carrier of data objects, and a business object structure may include a plurality of data object structures, which are the minimum units for transmitting a section of data in a customs clearance operating system.

The attached document fusion setting module is used for providing functional services such as fusion setting management, issuing setting update, extracting setting update, template matching, OCR processing and the like, and providing technical support for attached document information fusion by adopting interaction modes such as message queues and the like.

The risk knowledge fusion setting module is used for generating related knowledge generated from the Internet through an intelligent algorithm by the risk knowledge base construction work, and mainly exists in a knowledge table form. The risk knowledge table takes public code table information such as enterprise codes, commodity codes and the like as a main key, so that each risk knowledge can be accurately associated with a whole bill or a specific commodity item through the public codes, and fusion success is ensured.

The risk model fusion setting module is used for risk model new construction/modification, risk model filtering condition setting and risk model association setting.

The knowledge spectrum fusion setting module is used for checking the available knowledge spectrum of the system, setting the return results of various spectrum calls and the association mode with the common data object.

The parameter table management module is used for providing service for the process of parameter fusion. The management of the parameter list mainly comprises two parts of user parameter list management and public parameter list management, wherein the user parameter list management mainly comprises three functions of adding, inquiring and deleting, and the public parameter list management comprises two functions of public parameter list inquiring and public parameter list structure checking.

The scene data fusion demand synchronization module is used for analyzing and storing the fusion demand message, inquiring the analyzed information in the database table to obtain the fusion demand information, and storing all the fusion demand information. The fusion demand analysis is mainly divided into two major categories, and if the fusion demand message loading mode is full, all rule information is analyzed and converted into fusion demand information for storage and loading. If the loading mode of the fusion demand message is full, the old version corresponding to the fusion demand message needs to be added, deleted and modified, and then the modified fusion demand message is analyzed and stored in an incremental mode.

In a third aspect, the present invention provides a computer device set, including a data terminal node set, a risk screening node set, a fusion result node set, and a memory and a processor communicatively coupled thereto.

The data terminal node group is used for data acquisition, OCR (optical character recognition), data preprocessing, service fusion setting and scene discrimination, and comprises a data object structure maintenance module, a service object structure maintenance module and an attached document fusion setting module of the customs service data management device according to the second aspect, and a service object reading stage, an attached document reading stage, a service object association stage and a scene discrimination stage of the customs service data structural management method according to the first aspect are executed. All data information is stored in the current data terminal nodes, all the data terminal nodes are not communicated with each other, and all the data terminal nodes are only communicated with the risk screening node group. All structured and unstructured data

The risk screening node group is used for risk management, control and prediction of customs clearance documents, and comprises a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge graph fusion setting module and a parameter table management module of the customs service data management device in the second aspect, and a parameter condition executing stage, a risk knowledge query stage, a risk model calling stage, a knowledge graph query stage and a fusion result generating stage of the customs service data structural management method in the first aspect. And according to the document risk screening requirement, the risk screening node informs the field parameters to be sent by the data terminal node, receives the data information of the data terminal node and completes the requirement of risk screening. All risk data information is stored in the current risk screening nodes, and all risk screening nodes communicate through a load balancing gateway to complete functions such as bidirectional verification of electronic documents, synchronization of risk model training sample sets and the like. In addition, according to the difference of the corresponding speed and result precision of each module in the group, the risk screening node group consists of a simple demand risk screening group and a complex demand risk screening group, and the simple demand risk screening group completes the setting method of risk knowledge and knowledge maps and the calling function of a simple risk model to carry out preliminary risk screening. And sending the document field parameters of the discrimination result falling into the confidence interval to the complex demand risk discrimination group to finish the training result with higher fine granularity so as to ensure the result precision of model training.

The fusion result node group is used for receiving the fusion requirement result of the risk discrimination node, responding to the request access, and executing the fusion result message pushing stage of the customs service data structured management method according to the first aspect. The fusion result node group realizes high-performance distributed index inquiry and storage through a master-slave strategy of library division, table division and read-write separation.

The memory is used for storing a computer program, and the processor is used for reading the computer program and executing the customs service data structure management method of the first aspect.

Based on the above-mentioned invention content, a novel customs wisdom data management process comprising document structured data processing flow, related parameter data content distribution to risk discrimination nodes and national business structured data clustering is also provided. The customs unstructured document data are processed in batches at the terminal nodes and stored in the relational database of the current node in the form of structured data, and the document data feature is extracted and stored, so that the storage redundancy of the database is effectively reduced, the parameter extraction time of multiple risk screening requests is shortened, and the data transmission efficiency of the data terminal nodes and the risk screening nodes is improved. The document data discrimination result precision is filtered step by step through the multistage risk discrimination group, the document data result with low result precision requirement or obviously exceeding the normal result interval is returned by the risk discrimination scheme with low complexity and high timeliness through the simple requirement risk discrimination group, and the data which cannot be filtered is judged by the risk discrimination scheme with high complexity and higher processing precision through the complex requirement risk discrimination group. Based on the method, the multi-stage load balancing risk screening system can deal with the unification of the real-time performance and the accuracy of risk management and control. In addition, the system adopts a multisource heterogeneous data fusion solution based on edge calculation, and designs the functions of the distributed nodes according to a high-cohesion low-coupling principle, so that the atomicity of each node and the usability of the system are enhanced.

In a fourth aspect, the present invention provides a storage medium having stored thereon instructions which, when executed on a computer system, perform the customs service data structured management method of the first aspect.

In a fifth aspect, the present invention provides a computer system program product comprising instructions which, when run on a computer system, cause the computer system to perform the customs service data structure management method of the first aspect.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a system architecture diagram of a customs service data structure management method provided by the invention.

Fig. 2 is a functional module interaction diagram of the customs service data structure management method provided by the invention.

Fig. 3 is a schematic flow chart of a business object association phase process provided by the present invention.

Fig. 4 is a schematic flow chart of parameter table lookup function execution provided by the present invention.

Fig. 5 is a schematic flow chart of a parameter-related conditional execution service provided by the invention.

Fig. 6 is a schematic flow chart of risk knowledge specific data object configuration provided by the present invention.

Fig. 7 is a schematic flow chart of parsing and managing a converged demand packet according to the present invention.

Fig. 8 is a schematic flow chart of updating an old converged demand message provided by the present invention.

Fig. 9 is a schematic flow chart of data fusion requirement completion provided by the invention.

Fig. 10 is a schematic flow chart of a fused data query provided by the present invention.

Fig. 11 is a schematic flow chart of application service hot deployment provided by the invention.

Fig. 12 is a schematic flow chart of a load balancing gateway risk model version update provided by the present invention.

Fig. 13 is a schematic structural diagram of a computer device set provided by the present invention.

Detailed Description

The invention will be further elucidated with reference to the drawings and to specific embodiments. The present invention is not limited to these examples, although they are described in order to assist understanding of the present invention. Specific structural and functional details disclosed herein are merely representative of example embodiments of the invention. This invention may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein.

It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments of the present invention.

It should be understood that for the term "and/or" that may appear herein, it is merely one association relationship that describes an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a alone, B alone, and both a and B; for the term "/and" that may appear herein, which is descriptive of another associative object relationship, it means that there may be two relationships, e.g., a/and B, it may be expressed that: a alone, a alone and B alone; in addition, for the character "/" that may appear herein, it is generally indicated that the context associated object is an "or" relationship.

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments of the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, and do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should be appreciated that in some alternative designs, the functions/acts noted may occur out of the order in which the figures appear. For example, two figures shown in succession may in fact be executed substantially concurrently or the figures may sometimes be executed in the reverse order, depending upon the functionality/acts involved.

It should be understood that specific details are provided in the following description to provide a thorough understanding of the example embodiments. However, it will be understood by those of ordinary skill in the art that the example embodiments may be practiced without these specific details. For example, a system may be shown in block diagrams in order to avoid obscuring the examples with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the example embodiments.

As shown in fig. 1 to 10, the method for structured management of customs service data provided in the first aspect of the present embodiment may be, but is not limited to, suitable for executing on a processing device having a certain computing resource. The customs service data structured management method comprises a service object reading stage, an accompanying bill reading stage, a service object association stage, a scene discrimination stage, a parameter condition execution stage, a risk knowledge inquiry stage, a risk model calling stage, a knowledge map inquiry stage, a fusion result generation stage and a fusion result message pushing stage.

The service object reading stage may include, but is not limited to, the following steps S101 to S104:

s101, in a data terminal node service object fusion setting module, basic information such as English names, service object descriptions and the like in service objects is collected, the type is selected from a common service object or an association relation service object and submitted to a corresponding database, and if the service object exists in the database, the step is skipped.

S102, in the service object selected in S101, information such as English names and field descriptions in the field of the service object is collected, the data type of the corresponding database is selected and submitted to the corresponding database, and if the required field information exists in the database, the step is skipped.

S103, basic information such as English names, data object descriptions and the like in the data objects is collected, and the type is selected from common data objects or special data objects and submitted to a corresponding database.

S104, collecting field information of the data object in the service object selected in the step S103, performing key value pair matching, and submitting the key value pair matching to a corresponding database.

The accompanying document reading stage may include, but is not limited to, the following steps S201 to S204:

s201, acquiring source files of unstructured data and semi-structured data including but not limited to images, documents and the like in a document fusion setting module attached to a data terminal node.

S202, extracting basic information of a business object and field information of the business object through algorithms such as image preprocessing, text detection, detection correction, character recognition and information correction of an unstructured bill, submitting the basic information and the field information of the business object to a corresponding database, and skipping the step if the business object exists in the database.

S203, extracting data object basic information and data object field information through algorithms such as image preprocessing, text detection, detection correction, text recognition, information correction and the like of the unstructured receipts, and submitting the data object basic information and the data object field information to a corresponding database.

The business object association stage may include, but is not limited to, the following steps S301 to S304:

s301, reading the service association relation message from the clearance service system message queue service, and storing the information in the association service queue to a database for lasting processing for a certain period of time.

S302, other object IDs which do not arrive in the association relation message are placed in a waiting list, so that whether data needed by other association service queues are needed can be rapidly positioned when other subsequent service data messages arrive.

S303, initializing a service queue if the message of the association relation is received by monitoring the message, adding the message to the service queue if the message is received, checking that the assembly is completed, and submitting the message to a scene discrimination service stage for processing.

S304, judging whether the message is the information needed in a certain association relation message or not for other types of messages, if so, assembling the message into the corresponding association relation message data for the next processing (judging whether assembling is completed or not), otherwise, only temporarily storing the message to facilitate inquiry.

The scene discrimination stage may include, but is not limited to, the following steps S401 to S402:

s401, searching corresponding scene IDs from the cache according to the data object IDs, and if the direct return exists, searching a special service corresponding information table (taking the data object IDs as a main key) if the direct return does not exist.

S402, if the database is successfully searched, the scene ID is cached and returned, and if no corresponding record exists in the database, the service object associated data cannot be fused.

The parameter related condition executing stage may include, but is not limited to, the following steps S501 to S502:

s501, checking the input document parameters, including judging whether the parameter-related conditions are empty in a parameter-related condition checking list, whether the context data parameters required by executing the parameter-related conditions are empty, and the like.

S502, if the check is abnormal, performing abnormal processing; if the parameter check passes, the preparation of the context data required for MVEL expression execution begins, i.e., the context data parameters are stuffed into the context variables of the MVEL execution.

The risk knowledge query stage may include, but is not limited to, the following steps S601 to S603:

s601, performing fuzzy query by inputting query conditions, and clicking to check a specific structure after a result list is obtained.

S602, if the risk knowledge structure is a common risk knowledge structure, no further configuration is needed.

S603, if the association relationship type of the data object special for risk knowledge is used, the mapping relationship between the required risk knowledge object and the data object to be associated needs to be further established, query fields are configured for the required risk knowledge object and the data object to be associated, and specific information field extraction configuration is performed for the established association relationship.

The risk model invoking stage may include, but is not limited to, the following steps S701 to S702:

s701, configuring an API address of a risk model, clicking the model to call filtering condition filtering setting, selecting a scene, editing and storing MVEL expression filtering conditions, and if the configuration information exists, skipping the step.

S702, establishing a mapping relation between a required risk model object and a data object to be associated, configuring query fields for the required risk model object and the data object to be associated, and extracting and configuring specific information fields for the established risk model object and the data object to be associated.

The knowledge graph query stage may include, but is not limited to, the following steps S801 to S802:

S801, fuzzy query is carried out by inputting query conditions, and a specific structure is clicked and checked after a result list is obtained.

S802, establishing a mapping relation between the required knowledge graph object and the data object to be associated, configuring query fields for the knowledge graph object and the data object to be associated, and extracting and configuring specific information fields for the established knowledge graph object and the data object to be associated.

The fusion result generation stage may include, but is not limited to, the following steps S901 to S902:

s901, after the query interface is called, if the query fails to return the result immediately (such as an asynchronous return mode of the interface or a query failure), according to the implementation or program configuration of the query interface, additional result query interfaces can be called or re-requested after waiting for a certain time, and the result is considered to be abnormal until the attempt reaches a certain condition or fails.

S902, if the result can be returned, fusing and generating result data returned by each interface, and delivering the result data to a fused result message pushing stage.

The step of pushing the fusion result message may include, but is not limited to, the following steps S1001 to S1002:

s1001, generating a fusion demand ID according to a snowflake algorithm, and analyzing fields such as a batch number, an application ID, a loading mode, a scene ID, delay time and the like of the message.

S1002, dividing the loading mode into increment and full operation, obtaining effective time according to delay time, and storing the obtained basic information into a database.

After the step S502 and before the step S601, the method further includes: aiming at the current risk knowledge base, if the required risk knowledge or risk knowledge field information is updated, the corresponding risk knowledge or risk knowledge field is updated first to obtain the latest risk knowledge base.

After the step S602 and before the step S701, the method may include, but is not limited to, the following steps S1101 to S1106:

s1101, if the required risk model information is updated for the current risk model library, updating corresponding risk model fields to obtain the latest risk model library, and re-deploying application service flows of corresponding risk models as shown in fig. 11-12.

S1102, remotely calling a risk model server to be updated, uploading a new risk model file, and immediately re-deploying the server after the server is successfully received.

S1103, once the server updating service is successful, the server informs the load balancing gateway of the version updating information of the model version, the load balancing gateway updates the current model version, and discards the returned messages of all the old models.

S1104, after the other servers finish hot deployment, updating the model version in the message.

S1105. to ensure availability, a cluster of message queues is added before balancing the loaders to ensure that messages are not congested.

S1106. if the first server crashes during this period, the new service consumption will be suspended, but as other servers are added after updating, the congestion situation will be better relieved.

After the step S702 and before the step S801, the method further includes: aiming at the current knowledge-graph library, if the required knowledge-graph or knowledge-graph field information is updated, the corresponding knowledge-graph or knowledge-graph field is updated first to obtain the latest knowledge-graph library.

As shown in fig. 2, a second aspect of the present embodiment provides a virtual device for implementing the customs service data structure management method according to the first aspect, where the virtual device includes a data object structure maintenance module, a service object fusion setting module, an accompanying document fusion setting module, a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge map fusion setting module, a parameter table management module, and a scene data fusion requirement synchronization module.

The data object structure maintenance module is used for designing a data object structure and comprises the basic information of the data object and the functions of creating, editing, inquiring, deleting and the like of the common fields and the special fields contained in the basic information. Necessary functional support is provided for other modules such as business object structure maintenance, risk model fusion setting and the like.

The business object fusion setting module is used for business object structure maintenance and business object association. A business object is an important carrier of data objects, and a business object structure may include a plurality of data object structures, which are the minimum units for transmitting a section of data in a customs clearance operating system.

The attached document fusion setting module is used for providing functional services such as fusion setting management, issuing setting update, extracting setting update, template matching, OCR processing and the like, and providing technical support for attached document information fusion by adopting interaction modes such as message queues and the like.

The risk knowledge fusion setting module is used for generating related knowledge generated from the Internet through an intelligent algorithm by the risk knowledge base construction work, and mainly exists in a knowledge table form. The risk knowledge table takes public code table information such as enterprise codes, commodity codes and the like as a main key, so that each risk knowledge can be accurately associated with a whole bill or a specific commodity item through the public codes, and fusion success is ensured.

The risk model fusion setting module is used for risk model new construction/modification, risk model filtering condition setting and risk model association setting.

The knowledge spectrum fusion setting module is used for checking the available knowledge spectrum of the system, setting the return results of various spectrum calls and the association mode with the common data object.

The parameter table management module is used for providing service for the process of parameter fusion. The management of the parameter list mainly comprises two parts of user parameter list management and public parameter list management, wherein the user parameter list management mainly comprises three functions of adding, inquiring and deleting, and the public parameter list management comprises two functions of public parameter list inquiring and public parameter list structure checking.

The scene data fusion demand synchronization module is used for analyzing and storing the fusion demand message, inquiring the analyzed information in the database table to obtain the fusion demand information, and storing all the fusion demand information. The fusion demand analysis is mainly divided into two major categories, and if the fusion demand message loading mode is full, all rule information is analyzed and converted into fusion demand information for storage and loading. If the loading mode of the fusion demand message is full, the old version corresponding to the fusion demand message needs to be added, deleted and modified, and then the modified fusion demand message is analyzed and stored in an incremental mode.

The working process, working details and technical effects of the foregoing device provided in the second aspect of the present embodiment may refer to the method for structured management of customs service data described in the first aspect, which are not described herein again.

As shown in fig. 13, a third aspect of the present embodiment provides a computer device set for implementing the method for structured management of customs service data according to the first aspect, where the computer device set includes a data terminal node set, a risk screening node set, a fusion result node set, and a memory and a processor that are communicatively connected to each other.

The data terminal node group is used for data acquisition, OCR (optical character recognition), data preprocessing, service fusion setting and scene discrimination, and comprises a data object structure maintenance module, a service object structure maintenance module and an attached document fusion setting module of the customs service data management device according to the second aspect, and a service object reading stage, an attached document reading stage, a service object association stage and a scene discrimination stage of the customs service data structural management method according to the first aspect are executed. All data information is stored in the current data terminal nodes, all the data terminal nodes are not communicated with each other, and all the data terminal nodes are only communicated with the risk screening node group.

The risk screening node group is used for risk management, control and prediction of customs clearance documents, and comprises a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge graph fusion setting module and a parameter table management module of the customs service data management device in the second aspect, and a parameter condition executing stage, a risk knowledge query stage, a risk model calling stage, a knowledge graph query stage and a fusion result generating stage of the customs service data structural management method in the first aspect. And notifying the field parameters to be sent by the data terminal node according to the document risk screening requirement, and receiving the data information of the data terminal node to fulfill the requirement of risk screening. All risk data information is stored in the current risk screening nodes, and all risk screening nodes communicate by using a content distribution network technology to complete functions such as bidirectional verification of electronic documents, synchronization of training sample sets of risk models and the like. In addition, according to the difference of the corresponding speeds and the result precision of the modules in the group, the risk screening node group consists of a simple demand risk screening group and a complex demand risk screening group, and the simple requirement risk screening group completes the risk knowledge, the setting method of the knowledge graph and the calling function of the simple risk model, and performs preliminary risk screening. And sending the document field parameters of the discrimination result falling into the confidence interval to the complex demand risk discrimination group to finish the training result with higher granularity so as to ensure the result precision of model training. Based on the method, the multi-stage load balancing risk screening system can deal with the unification of the real-time performance and the accuracy of risk management and control.

The fusion result node group is used for receiving the fusion requirement result of the risk discrimination node, responding to the request access, and executing the fusion result message pushing stage of the customs service data structured management method according to the first aspect. The fusion result node group realizes high-performance distributed index inquiry and storage through a master-slave strategy of library division, table division and read-write separation.

The memory is used for storing a computer program, and the processor is used for reading the computer program and executing the customs service data structure management method of the first aspect.

The working process, working details and technical effects of the foregoing computer device set provided in the third aspect of the present embodiment may refer to the customs service data structure management method described in the first aspect, which are not described herein again.

A fourth aspect of the present embodiment provides a storage medium storing instructions containing the customs service data structure management method according to the first aspect, i.e. the storage medium stores corresponding instructions thereon, which when executed on a computer, perform the customs service data structure management method according to the first aspect. The storage medium refers to a carrier for storing data, and may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash Memory, a flash disk, and/or a Memory Stick (Memory Stick), etc.

The working process, working details and technical effects of the foregoing storage medium provided in the fourth aspect of the present embodiment may refer to the method for structured management of customs service data in the first aspect, which are not described herein again.

A fifth aspect of the present embodiment provides a computer program product comprising instructions which, when run on a computer system, cause the computer system to perform the customs service data structure management method of the first aspect.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents. Such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Finally, it should be noted that the invention is not limited to the alternative embodiments described above, but can be used by anyone in various other forms of products in the light of the present invention. The above detailed description should not be construed as limiting the scope of the invention, which is defined in the claims and the description may be used to interpret the claims.

Claims (8)

1. A customs service data structured management method is characterized by comprising a service object reading stage, an accompanying bill reading stage, a service object association stage, a scene discrimination stage, a parameter related condition execution stage, a risk knowledge inquiry stage, a risk model calling stage, a knowledge map inquiry stage, a fusion result generation stage and a fusion result message pushing stage;

the service object reading stage may include, but is not limited to, the following steps S101 to S104:

s101, in a data terminal node service object fusion setting module, basic information such as English names, service object descriptions and the like in service objects is collected, the type is selected from a common service object or an association relation service object and submitted to a corresponding database, and if the service object exists in the database, the step is skipped;

s102, in the service object selected in the step S101, information such as English names, field descriptions and the like in the field of the service object is collected, key value pair matching is carried out, the key value pair matching is submitted to a corresponding database, and if required field information exists in the database, the step is skipped;

s103, collecting basic information such as English names, data object descriptions and the like in the data objects, selecting the type of the common data object or the special data object, and submitting the basic information to a corresponding database;

S104, collecting field information of the data object in the service object selected in the step S103, performing key value pair matching, and submitting the key value pair matching to a corresponding database;

the accompanying document reading stage may include, but is not limited to, the following steps S201 to S204:

s201, acquiring source files of unstructured data and semi-structured data including but not limited to images, documents and the like in a document fusion setting module attached to a data terminal node;

s202, extracting basic information of a service object and field information of the service object through algorithms such as image preprocessing, text detection, detection correction, text recognition, information correction and the like of an unstructured bill, submitting the basic information and the field information of the service object to a corresponding database, and skipping the step if the service object exists in the database;

s203, extracting data object basic information and data object field information through algorithms such as image preprocessing, text detection, detection correction, text recognition, information correction and the like of the unstructured receipts, and submitting the data object basic information and the data object field information to a corresponding database;

the business object association stage may include, but is not limited to, the following steps S301 to S304:

s301, reading a service association relation message from a clearance service system message queue service, and storing information in an association service queue to a database for lasting treatment for a certain period of time;

S302, other object IDs which do not arrive in the association relation message are put into a waiting list;

s303, initializing a service queue if the message of the association relation is received by monitoring the message, adding the message to the service queue if the message is received while waiting, checking that the assembly is completed, and submitting the message to a scene discrimination service stage for processing;

s304, judging whether the message is the information required by a certain association relation message or not for other types of messages, if so, assembling the message into the corresponding association relation message data for the next processing (judging whether assembling is completed or not), otherwise, only temporarily storing the message to facilitate inquiry;

the scene discrimination stage may include, but is not limited to, the following steps S401 to S402:

s401, searching a corresponding scene ID from the cache according to the data object ID, and if the direct return exists, searching a special service corresponding information table (taking the data object ID as a main key) if the direct return does not exist;

s402, if the database is successfully searched, the scene ID is cached and returned, and if no corresponding record exists in the database, the service object associated data cannot be fused;

the parameter condition execution stage may include, but is not limited to, the following steps S501 to S502:

S501, checking the input document parameters, including judging whether the parameter-related conditions are empty in a parameter-related condition checking list, whether the context data parameters required by executing the parameter-related conditions are empty, and the like;

s502, if the check is abnormal, performing abnormal processing; if the parameter check passes, starting to prepare the context data required by the MVEL expression execution, namely filling the context data parameters into the context variables of the MVEL execution;

the risk knowledge query stage may include, but is not limited to, the following steps S601 to S603:

s601, performing fuzzy query by inputting query conditions, and clicking to check a specific structure after a result list is obtained;

s602, if the risk knowledge structure is a common risk knowledge structure, no further configuration is needed;

s603, if the relationship type is the relationship type of the data object special for risk knowledge, the mapping relationship between the required risk knowledge object and the data object to be associated needs to be further established, query fields are configured for the required risk knowledge object and the data object to be associated, and specific information field extraction configuration is performed for the established relationship;

the risk model invoking stage may include, but is not limited to, the following steps S701 to S702:

s701, configuring an API address of a risk model, setting a model call filtering condition, selecting a scene, editing and storing MVEL expression filtering conditions, and if the configuration information exists, skipping the step;

S702, establishing a mapping relation between a required risk model object and a data object to be associated, configuring query fields for the required risk model object and the data object to be associated, and extracting and configuring specific information fields for the established risk model object and the data object to be associated;

the knowledge graph query stage may include, but is not limited to, the following steps S801 to S802:

s801, performing fuzzy query by inputting query conditions, and clicking to check a specific structure after a result list is obtained;

s802, establishing a mapping relation between a required knowledge graph object and a data object to be associated, configuring query fields for the required knowledge graph object and the data object to be associated, and extracting and configuring specific information fields for the established knowledge graph object and the data object to be associated;

the fusion result generation stage may include, but is not limited to, the following steps S901 to S902:

s901, after a query interface is called, if the query fails to return a result immediately (such as an asynchronous return mode of the interface or a query failure), according to the implementation or program configuration of the query interface, additional result query interfaces can be considered to be called or re-requested after waiting for a certain time, and the result is considered to be abnormal until the query fails or the query fails after trying to reach a certain condition;

s902, if the result can be returned, fusing and generating result data returned by each interface, and delivering the result data to a fused result message pushing stage;

The step of pushing the fusion result message may include, but is not limited to, the following steps S1001 to S1002:

s1001, generating a fusion demand ID according to a snowflake algorithm, and analyzing fields such as batch number, application ID, loading mode, scene ID, delay time and the like of the message;

s1002, dividing the loading mode into increment and full operation, obtaining effective time according to delay time, and storing the obtained basic information into a database.

2. The method for structured management of customs service data according to claim 1, wherein after said step S602 and before said step S701, said method may include, but is not limited to, the following steps S1101 to S1106:

s1101, aiming at the current risk model library, if the needed risk model information is updated, firstly updating the corresponding risk model field to obtain the latest risk model library, and further, redeploying an application service flow of the corresponding risk model;

s1102, remotely calling a risk model server to be updated, uploading a new risk model file, and immediately re-deploying the server after the server is successfully received;

s1103, once a server has successful updating service, the server informs a message of updating the version of the load balancing gateway model, the load balancing gateway updates the version of the current model, and discards the messages returned by all the old models;

S1104, after the other servers finish hot deployment, updating the model version in the message;

s1105, adding a message queue cluster before balancing the loader to ensure that the message is not jammed;

s1106. if the first server crashes during this period, the new service consumption will be suspended, but as other servers are added after updating, the congestion situation will be better relieved.

3. The customs service data structured management device is characterized by comprising a data object structure maintenance module, a service object fusion setting module, an attached bill fusion setting module, a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge map fusion setting module, a parameter table management module and a scene data fusion requirement synchronization module;

the data object structure maintenance module is used for designing a data object structure, and comprises the basic information of the data object and the functions of creating, editing, inquiring, deleting and the like of the common fields and the special fields contained in the basic information; providing necessary functional support for other modules such as service object structure maintenance, risk model fusion setting and the like;

the business object fusion setting module is used for maintaining a business object structure and associating business objects; the business object is an important carrier of the data object, and one business object structure can contain a plurality of data object structures, which are the minimum units for transmitting a section of data in the customs operation system;

The attached bill fusion setting module is used for providing functional services such as fusion setting management, issuing setting update, extracting setting update, template matching, OCR processing and the like, and providing technical support for attached bill information fusion by adopting interaction modes such as message queues and the like;

the risk knowledge fusion setting module is used for generating related knowledge generated from the Internet through an intelligent algorithm by the risk knowledge base construction work, and mainly exists in a knowledge table form; the risk knowledge table takes public code table information such as enterprise codes, commodity codes and the like as a main key, so that each risk knowledge can be accurately associated with a whole bill or a specific commodity item through the public codes, and fusion success is ensured;

the risk model fusion setting module is used for risk model creation/modification, risk model filtering condition setting and risk model association setting;

the knowledge spectrum fusion setting module is used for checking the available knowledge spectrum of the system, setting the return results of various spectrum calls and the association mode with the common data object;

the parameter table management module is used for providing service for the process of parameter fusion; the management of the parameter list mainly comprises two parts of user parameter list management and public parameter list management, wherein the user parameter list management mainly has three functions of adding, inquiring and deleting, the public parameter table management comprises two functions of public parameter table inquiry and public parameter table structure check;

The scene data fusion demand synchronization module is used for analyzing and storing the fusion demand message, inquiring the analyzed information in the database table to obtain fusion demand information, and storing all the fusion demand information; the fusion demand analysis is mainly divided into two major categories, and if the loading mode of the fusion demand message is full, all rule information is analyzed and converted into fusion demand information for storage and loading; if the loading mode of the fusion demand message is full, the old version corresponding to the fusion demand message needs to be added, deleted and modified, and then the modified fusion demand message is analyzed and stored in an incremental mode.

4. A customs service data structured management computer device group, comprising a data terminal node group, a risk screening node group, a fusion result node group, and a memory and a processor communicatively connected with each other, wherein the memory is used for storing a computer program, and the processor is used for reading the computer program and executing the customs service data structured management method according to any one of claims 1-2.

5. The data terminal node group in the customs service data structure management computer equipment group according to claim 4 is used for data acquisition, OCR recognition, data preprocessing, service fusion setting and scene discrimination, and comprises a data object structure maintenance module, a service object structure maintenance module and an accompanying document fusion setting module of the customs service data structure management device according to claim 3, and a service object reading stage, an accompanying document reading stage, a service object association stage and a scene discrimination stage of the customs service data structure management method according to any one of claims 1 to 2 are executed; all data information is stored in the current data terminal nodes, all the data terminal nodes are not communicated with each other, and all the data terminal nodes are only communicated with the risk screening node group.

6. The risk screening node group in the customs service data structured management computer equipment group according to claim 4 is used for risk management and prediction of clearance documents, and comprises a risk knowledge fusion setting module, a risk model fusion setting module, a knowledge graph fusion setting module and a parameter table management module of the customs service data structured management device according to claim 3, and a parameter condition executing stage, a risk knowledge query stage, a risk model calling stage, a knowledge graph query stage and a fusion result generating stage of the customs service data structured management method according to any one of claims 1 to 2 are executed;

the risk discrimination node group informs the data terminal nodes of field parameters to be sent according to the document risk discrimination requirement, receives the data information of the data terminal nodes and completes the requirement of risk discrimination; all risk data information is stored in the current risk screening nodes, and the risk screening nodes communicate by using a content distribution network technology to finish the functions of bidirectional verification of electronic documents, synchronization of training sample sets of risk models and the like; in addition, according to the difference of the corresponding speeds and the result precision of each module in the group, the risk discrimination node group consists of a simple demand risk discrimination group and a complex demand risk discrimination group, and the simple demand risk discrimination group completes the setting method of risk knowledge and knowledge maps and the calling function of a simple risk model to carry out preliminary risk discrimination; and sending the document field parameters of the discrimination result falling into the confidence interval to the complex demand risk discrimination group to finish the training result with higher granularity so as to ensure the result precision of model training.

7. The fused result node group in the customs service data structured management computer equipment group according to claim 4 is configured to receive the fused requirement result of the risk screening node, and respond to the request for access, and includes a scene data fused requirement synchronization module of the customs service data structured management apparatus according to claim 3, and execute the fused result message pushing stage of the customs service data structured management method according to any one of claims 1 to 2; the fusion result node group realizes high-performance distributed index inquiry and storage through a master-slave strategy of library division, table division and read-write separation.

8. A storage medium having stored thereon instructions which, when executed on a computer, perform the customs service data structured management method of any one of claims 1 to 2.