CN109684322B - Data processing system and method for automatic maritime affair auditing - Google Patents
Data processing system and method for automatic maritime affair auditing Download PDFInfo
- Publication number
- CN109684322B CN109684322B CN201811601257.7A CN201811601257A CN109684322B CN 109684322 B CN109684322 B CN 109684322B CN 201811601257 A CN201811601257 A CN 201811601257A CN 109684322 B CN109684322 B CN 109684322B
- Authority
- CN
- China
- Prior art keywords
- data
- manifest
- analysis
- audit
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Quality & Reliability (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a data processing system and a method for automatic maritime affair auditing, which comprises a data acquisition and cleaning subsystem, an auditing analysis subsystem and a data processing and conversion subsystem, wherein the data acquisition and cleaning subsystem is used for acquiring original manifest data, cleaning and converting the original manifest data and transmitting the cleaned and converted manifest data to the auditing analysis subsystem; the audit analysis subsystem is used for carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data; the automatic auditing subsystem is used for generating an auditing result according to the analysis data, judging whether the manifest data contains abnormal data or not, and identifying, tracking and disposing the abnormal data; the statistical analysis subsystem carries out real-time statistical analysis on the audit result and the abnormal data by adopting an OLAP multidimensional analysis method to generate an analysis report; the parameter configuration module is used for distributing user authority, recording an execution log of the audit task and a user use log; and the external interface module is used for realizing the butt joint with external data or a platform.
Description
Technical Field
The invention relates to the technical field of port charge management systems, in particular to a data processing system and a data processing method for automatic maritime affair auditing.
Background
At present, Chinese port construction charge collection methods mainly comprise two types: one is that the maritime administration directly collects the port construction fee, and the other is that the ship agent, the cargo carrier, the port operator and other collection units collect the port construction fee instead.
The current collection system is mainly a collection management system taking collection as a main part, and maritime workers need to check each link of collection of the royalties in various checking modes such as daily checking, special item checking, special case checking and the like to obtain the royalty data, so that two problems exist:
1. the work intensity of the maritime affair management mechanism is increased due to the fact that a large number of personnel are needed for checking, and the efficiency is low;
2. in the inspection process, the data obtained by the maritime personnel is not accurate enough.
Disclosure of Invention
The present invention is directed to solve the above problems in the prior art, and provides a data processing system and method for automatic maritime affairs audit, which can obtain accurate real-time audit data by analyzing the third-party import/export declaration information data and the reference data.
In order to achieve the purpose, the invention adopts the technical scheme that:
a data processing system for automatic maritime affairs audit comprises a data acquisition and cleaning subsystem, an audit analysis subsystem, an automatic audit subsystem, a statistical analysis subsystem, a parameter configuration module and an external interface module, wherein:
the data acquisition and cleaning subsystem is used for acquiring original manifest data, cleaning and converting the acquired original manifest data, and transmitting the cleaned and converted manifest data to the audit analysis subsystem;
the audit analysis subsystem carries out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database;
the automatic auditing subsystem is used for generating an auditing result according to the analysis data, judging whether the manifest data contains abnormal data or not, and identifying, tracking and disposing the abnormal data;
the statistical analysis subsystem carries out real-time statistical analysis on the audit result and the abnormal data by adopting an OLAP multi-dimensional analysis method and generates an analysis report;
the parameter configuration module is used for distributing user authority and recording an execution log of the audit task and a user use log;
the external interface module is used for realizing the butt joint with external data or a platform.
A data processing method for automatic maritime audit, comprising:
collecting, cleaning and converting original manifest data;
carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database;
generating an audit result according to the analysis data, judging whether the manifest data contains abnormal data, and identifying, tracking and disposing the abnormal data;
and carrying out real-time statistical analysis on the audit result and the abnormal data by utilizing an OLAP multi-dimensional analysis method, and generating an analysis report.
The invention obtains the following beneficial effects:
according to the invention, by arranging the data acquisition and cleaning subsystem, the audit analysis subsystem, the automatic audit subsystem, the statistical and analysis subsystem, the parameter configuration module and the external interface module, the acquired and cleaned data are compared with foreign trade data in the third-party comparison information database and daily service data and reference data of a collection unit in the maritime comparison information database, so that the accuracy of the data reported by the collection unit can be effectively checked; in addition, the statistical analysis is carried out on the manifest data by utilizing an OLAP multidimensional analysis method, and more labor cost is not needed, so that the supervision efficiency is improved, and the cost is also reduced.
Drawings
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a block diagram of an embodiment of a data processing system for automatic maritime auditing according to the present invention;
FIG. 2 is a schematic structural diagram of an embodiment of an external interface module according to the present invention;
FIG. 3 is a flow diagram illustrating one embodiment of a data processing method for automatic maritime auditing according to the present invention;
fig. 4A-4C are flow diagrams of one embodiment of a data collection, cleaning process of the present invention.
Detailed Description
The following description is presented to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.
It is understood that the terms "a" and "an" should be interpreted as meaning that a number of one element or element is one in one embodiment, while a number of other elements is one in another embodiment, and the terms "a" and "an" should not be interpreted as limiting the number.
The terminology used herein is for the purpose of describing various embodiments only and is not intended to be limiting. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, numbers, steps, operations, components, elements, or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, elements, or groups thereof.
An embodiment of a data processing system for maritime audit management according to the present invention will be described in detail with reference to the accompanying drawings.
Data processing system for automatic maritime affair audit
A data processing system for automatic maritime auditing, comprising one or more databases, each of which is referred to in one embodiment of the data processing system of the present invention, is described herein, the data processing system comprising:
a. the third party comparison information database is used for storing cleaned and converted ship import and export customs declaration information of a customs unit and export foreign trade data of a ship agency company;
b. the maritime affair comparison information database is used for storing daily business data of collection agencies from a port construction charge collection management system (namely a collection system) after being cleaned and converted;
c. the maritime affair audit database is used for storing the audit process and result of the maritime affair management worker and the data related to the abnormal handling process of the collection unit;
d. the collecting unit database is used for storing basic relevant information of collecting units, such as unit names, registered addresses, legal persons, unit qualification, contact calls and the like;
e. the parameter configuration database is used for storing the contents including mechanism information, user information, system roles, system resources, system menus and the like, and also comprises data such as system log management, a data dictionary, an attachment table and the like.
Fig. 1 is a schematic diagram of a module structure of an embodiment of a data processing system for automatic maritime affairs auditing according to the present invention, please refer to fig. 1, in this implementation, the data processing system includes a data collection and cleaning subsystem, an auditing analysis subsystem, an automatic auditing subsystem, a statistical analysis subsystem, a parameter configuration module, and an external interface module, wherein:
the data acquisition and cleaning subsystem is used for acquiring original manifest data, cleaning and converting the acquired original manifest data, and transmitting the cleaned and converted manifest data to the audit analysis subsystem, the third party comparison information database and the maritime affairs comparison information database; the original manifest data are from original manifest data of a collection unit database and a maritime affairs audit check database; specifically, incomplete, erroneous and repeated original manifest data can be cleaned, and converted through smoothing, summation, generalization, normalization and the like;
the audit analysis subsystem analyzes the data integrity and/or accuracy of the cleaned and converted manifest data according to the total reference data of the reference database;
the automatic auditing subsystem is used for generating an auditing result according to the analysis data, judging whether the manifest data contains abnormal data, identifying and tracking the abnormal data and sending the abnormal data to the statistical analysis subsystem for disposal;
the statistical analysis subsystem carries out real-time statistical analysis on the audit result and abnormal data by adopting an OLAP (on-line analytical processing) multidimensional analysis method and generates an analysis report;
the parameter configuration module is used for distributing user authority and recording an execution log of the audit task and a user use log;
the external interface module is used for realizing the butt joint with external data or a platform. Description of the respective subsystems
The structure and/or operation of each subsystem or module of the data processing system in the present embodiment will be described below.
1. Data acquisition and cleaning subsystem
The data acquisition and cleaning subsystem in this embodiment includes a source data acquisition module and a data cleaning conversion module, wherein:
the source data acquisition module is used for acquiring original manifest data and transmitting the original manifest data to the data cleaning and converting module; the original cabin bill data is derived from a collection unit database and a maritime check and inspection database, and the collection unit database stores maritime ship data of a port construction charge collection management system for carrying out collection work; the maritime affair audit database stores customs audited ship import and export customs declaration information and export foreign trade data (namely third party manifest data) declared by a ship agency company;
the data cleaning and converting module is used for cleaning and converting the original manifest data; and storing the cleaned and converted manifest data into an audit analysis subsystem, a third party comparison information database and a maritime affairs comparison information database.
Specifically, the source data acquisition module can periodically inquire data source ends of a third party manifest data interface module and a maritime port fee check management database (namely, a third party comparison information database and a maritime comparison information database) to acquire original manifest data;
in this embodiment, according to different database interfaces, the data collection and cleaning process can be divided into the following parts:
(1) acquiring original third party manifest data of the previous 7 days by calling a third party manifest data interface module at idle time every day by using a timer;
(2) the data cleaning and converting module cleans and converts the acquired third party original manifest data to obtain third party manifest data, transmits and stores the third party manifest data to the audit analysis subsystem and the maritime port construction fee audit database, and sets the disposal state of the manifest to be 0;
(3) reading all the manifest data with the processing state of 0 in the maritime port construction charge audit database;
(4) calling a marine vessel data interface module by transmitting vessel IMO numbers (namely vessel name codes), vessel voyage numbers, import and export marks and the like to acquire marine vessel data, storing the marine vessel data into a marine port construction fee audit management database, calling a manifest cargo and container data interface module by vessel codes to acquire manifest data and container data corresponding to a manifest, cleaning, converting and storing the manifest data and the container data into the marine port construction fee audit management database, and setting the handling state of the manifest as 1;
wherein, before acquireing marine vessel data and acquireing bill of lading data and container data, can judge earlier whether there is marine vessel data or bill of lading data and container data: if the data does not exist, whether the time interval between the current time and the electronic bill of lading creation time is larger than a preset time interval or not is further judged, if the time interval is larger than the preset time interval, the bill data is marked as abnormal data, the processing state of the bill is set to be 8, and if the time interval is smaller than or equal to the preset time interval, the bill data is represented as normal data, and the processing state of the bill is unchanged.
(5) Reading all the manifest data with the disposal state of '1' in the maritime port construction fee audit management database;
(6) calling a third party manifest cargo and container data interface module to judge whether third party cargo and container data exist or not;
if the third-party cargo and container data exist, the third-party cargo and container data are cleaned, converted and stored into a maritime port construction fee audit management database, and the handling state of the manifest is set to be 2;
if the third-party cargo and container data do not exist, judging whether the time interval between the current time and the time for creating the manifest is greater than a preset time interval, if so, indicating that the third-party container does not exist, marking the manifest data as abnormal data, and setting the manifest setting state to be 8; if the time interval is smaller than or equal to the time interval, the data of the manifest are normal, and the setting state of the manifest is unchanged.
In this embodiment, the preset interval may be set to 90 days, and it is understood that the preset interval is not limited thereto, and may also be set to other time periods.
2. Audit analysis subsystem
The audit analysis subsystem in this embodiment is configured to import the assessment data and perform consistency analysis on the assessment data, and specifically, the subsystem includes:
the data analysis module is used for carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database to obtain an analysis result; comparing and analyzing the cleaned and converted manifest data with foreign trade data (approved by customs units and third party manifest data of ship agents) in the third party comparison information database and daily business data of collection units in the maritime comparison information database;
in this embodiment, data integrity analysis may be performed on the manifest data, for example, it is determined whether each check item is included in both the maritime system and the customs system to determine whether the data is complete, where the check items may be the manifest number and the container number; data accuracy analysis can also be carried out on the manifest data, for example, unique containers can be locked by retrieving a ' voyage number ', ' ship IMO number (namely: ship code), ' container number ' and the like, and comparison by one to ensure that the information of each container is accurate and correct according to reference data;
and the analysis result viewing module is used for displaying the analysis result, and when the analysis result of the audit analysis subsystem is abnormal, a user can be reminded of viewing the abnormal result through the drifting display and/or the information prompt.
3. Automatic auditing subsystem
The automatic audit subsystem in this embodiment includes:
the auditing module is used for determining whether the manifest data contains abnormal data or not according to the analysis data, marking the abnormal data and generating an auditing result with abnormal marks; displaying the compared analysis result to a user, determining whether the compared analysis result is listed as an abnormal audit item according to the instruction of the user, marking the abnormal audit item, and generating a final audit result;
a handling module to track and handle the exception data; namely, the modification of the collection unit updates the audit abnormal item, and the updated result is fed back to the user, so that the user can complete the tracking of the abnormal item modification condition.
4. Statistical analysis subsystem
The statistical analysis subsystem in this embodiment is mainly used for performing statistics on audit data, and specifically, the subsystem includes:
the statistical module can count the total cabin lists in the jurisdiction, the total number of the total containers in the jurisdiction, the cabin lists which are subjected to audit inspection, the number of the containers which are subjected to audit inspection, the cabin lists which are found to be in audit inspection problems, the number of the containers which are found to be in audit inspection problems and/or the number of the containers which are found to be in audit inspection problems and the like according to the name of a representative unit, the service type, the time range and the like to obtain statistical results, and sorts the statistical results according to the frequency of the statistical results; specifically, a certain charging time range (for example, 3 years or 5 years) may be selected according to the error reasons, the service types, and other options of the abnormal items found in the auditing process, and the data marked as abnormal in the data processing system is subjected to summation calculation by year, quarter, or month, and is displayed in a sorted manner according to the occurrence frequency from large to small;
the analysis module can perform cluster analysis on the audit reasons according to the name of the representative unit, the service type, the charging time range and the like by adopting an OLAP multi-dimensional analysis method and generate an analysis report.
5. Parameter configuration module
The parameter configuration module in this embodiment includes:
a system configuration module for setting and storing a plurality of configuration items, the configuration items including: information of a maritime administration institution, information of a collection institution, user information, and the like; specifically, the system management module can be in butt joint with an external platform (such as a maritime 4A authentication system, namely a unified safety management platform), the external platform realizes unified management of organization, users and roles, unified users, unified authentication, unified permission, a unified workbench, a unified menu and a unified integrated management platform are provided, and the parameter configuration module can directly use the organization, users and role data uniformly deployed by the external platform;
the log management module is used for recording the execution time and the execution result of the audit task and the operation history of the user; specifically, the log management module is used for managing an execution log of the audit monitoring task and a use log of a user; the execution log of the audit monitoring task is mainly used for recording the execution time and the execution result of the audit task, wherein the execution time and the execution result are important data sources of the audit result; the user usage log is mainly used for recording the operation history of the data processing system for the automatic maritime affair auditing, and the operation history is an important basis for auditing the user behavior.
6. External interface module
The external interface module in this embodiment includes:
the service interface module is used for realizing wired or wireless interconnection of goods import and export data;
the portal interface module is used for being in butt joint with an external platform to realize integration and combination so as to realize user integration, role integration, mechanism integration, single sign-on, tasks to be handled and the like;
wherein the service interface module comprises one or more of the following modules:
the marine vessel data interface module is used for acquiring marine vessel data, including import vessel data and export vessel data;
the bill of cargo and container data interface module acquires bill of lading data and container data corresponding to the bill of lading through keywords of the bill of lading for example;
the third party manifest data interface module is used for acquiring customs entrance forecast manifest data and exit cleaning manifest data of the collection units;
the third party manifest cargo and container data interface module is used for acquiring data of the third party cargo and the container through keywords of the manifest for example;
the nuclear marketing remark interface module is used for acquiring abnormal data generated after the data of the manifest are analyzed;
the auditing abnormal data interface module is used for acquiring and inquiring the abnormal data, and particularly, the interface module is used for acquiring customs entrance forecast manifest data and exit cleaning manifest data of a collection unit.
Third, a data processing method for automatic maritime affairs audit
FIG. 3 is a flow chart illustrating an embodiment of a data processing method for automatic maritime auditing according to the present invention, and FIGS. 4A-4C are flow charts illustrating an embodiment of a data collection and cleaning process according to the present invention; referring to fig. 3 and fig. 4A-4C, the data processing method of the present embodiment includes the following steps:
step 100: collecting, cleaning and converting original manifest data;
step 200: carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database;
step 300: generating an audit result according to the analysis data, judging whether the manifest data contains abnormal data, and identifying, tracking and disposing the abnormal data;
step 400: and carrying out real-time statistical analysis on the audit result and the abnormal data by utilizing an OLAP multi-dimensional analysis method, and generating an analysis report. Wherein, step 100 comprises the following substeps:
step 110: calling a third party manifest data interface module to acquire third party original manifest data;
step 120: cleaning, converting and storing the third party original manifest data, and setting the processing state of the manifest to be 0;
step 130: reading all third party manifest data with the handling state of 0;
step 140: calling a marine vessel data interface module to judge whether marine vessel data exists or not;
step 141: if the data of the marine vessel exists, calling a bill cargo and container data interface module to obtain bill of lading data and container data corresponding to the bill of lading, cleaning, converting and storing the bill of lading data and the container data, and setting the handling state of the bill of lading to be 1;
step 142: if no marine vessel data exists, marking the manifest data as abnormal data and setting the manifest handling state to be 8%;
step 150: reading all the manifest data with the processing state of 1;
step 160: calling a third party manifest cargo and container data interface module to judge whether third party cargo and container data exist or not;
step 161: if the third-party cargo and container data exist, the third-party cargo and container data are cleaned, converted and stored, and the handling state of the manifest is set to be 2;
step 162: and if the data of the third-party goods and the containers does not exist, marking the data of the manifest as abnormal data, and setting the setting state of the manifest as 8.
The data processing method for automatic maritime affair auditing in this embodiment is substantially the same as the work flow of the data processing system, and the description thereof is omitted here.
In the aspect of software architecture, the invention is realized by adopting a three-layer structure based on a JavaEE platform and combining a large-scale enterprise application framework and components. The front page is an interface between an application program and a user, and is mainly used for displaying information to the user and providing an input and output control for user operation; the application layer is mainly responsible for the transmission and conversion of user data; the service logic layer is the main core content of the module and comprises the processing of various service logic processes; the data access layer is mainly responsible for storing data into a database and carrying out persistence operation on the data; in the aspect of data processing, a mode of combining OWB and SQL is adopted, the advantages of both are combined, an ETL data processing project is quickly established, and the development speed and efficiency of ETL are greatly improved by the aid of the flexibility of SQL.
The invention has the beneficial effects that:
according to the invention, by arranging the data acquisition and cleaning subsystem, the audit analysis subsystem, the automatic audit subsystem, the statistical and analysis subsystem, the parameter configuration module and the external interface module, the acquired and cleaned data are compared with foreign trade data in the third-party comparison information database and daily service data and reference data of a collection unit in the maritime comparison information database, so that the accuracy of the data reported by the collection unit can be effectively checked; in addition, the statistical analysis is carried out on the manifest data by utilizing an OLAP multidimensional analysis method, and more labor cost is not needed, so that the supervision efficiency is improved, and the cost is also reduced.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. It is to be noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in an acquisition machine-readable storage medium. Based on such understanding, the technical solution of the present invention or a part thereof, which essentially contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an obtaining machine (which may be a personal obtaining machine, a server, or a network machine) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (8)
1. A data processing system for automatic maritime affairs audit is characterized by comprising a data acquisition and cleaning subsystem, an audit analysis subsystem, an automatic audit subsystem, a statistical analysis subsystem, a parameter configuration module and an external interface module, wherein:
the data acquisition and cleaning subsystem is used for acquiring original manifest data, cleaning and converting the acquired original manifest data, and transmitting the cleaned and converted manifest data to the audit analysis subsystem;
the audit analysis subsystem carries out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database;
the automatic auditing subsystem is used for generating an auditing result according to the analysis data, judging whether the manifest data contains abnormal data or not, and identifying, tracking and disposing the abnormal data;
the statistical analysis subsystem carries out real-time statistical analysis on the audit result and the abnormal data by adopting an OLAP multi-dimensional analysis method and generates an analysis report;
the parameter configuration module is used for distributing user authority and recording an execution log of the audit task and a user use log;
the external interface module is used for realizing the butt joint with external data or a platform;
wherein, the automatic audit subsystem comprises:
the auditing module is used for determining whether the manifest data contains abnormal data according to the analysis data, marking the abnormal data and generating an auditing result with abnormal marks, wherein the auditing result comprises displaying the compared analysis result to a user, determining whether the compared analysis result is listed as an auditing abnormal item according to an instruction of the user and marking the auditing abnormal item to generate a final auditing result;
the processing module is used for tracking and processing the abnormal data, modifying and updating the abnormal data and feeding back an updated result to a user;
the data acquisition and cleaning subsystem is used for executing the following operations:
calling a third party manifest data interface module to acquire original manifest data;
cleaning, converting and storing the original manifest data, and setting the processing state of the manifest to be 0;
reading all the manifest data with the processing state of 0;
calling a marine vessel data interface module to judge whether marine vessel data exists or not; if the data of the marine vessel exists, calling a bill cargo and container data interface module to obtain bill of lading data and container data corresponding to the bill of lading, cleaning, converting and storing the bill of lading data and the container data, and setting the handling state of the bill of lading to be 1; if no marine vessel data exists, marking the manifest data as abnormal data and setting the manifest handling state to be 8%;
reading all the manifest data with the processing state of 1;
calling a third party manifest cargo and container data interface module to judge whether third party cargo and container data exist or not; if the third-party cargo and container data exist, the third-party cargo and container data are cleaned, converted and stored, and the handling state of the manifest is set to be 2; and if the data of the third-party goods and the containers does not exist, marking the data of the manifest as abnormal data, and setting the setting state of the manifest to be 8.
2. The data processing system of claim 1, wherein the data collection and cleansing subsystem comprises a source data acquisition module and a data cleansing conversion module, wherein:
the source data acquisition module is used for acquiring original manifest data and transmitting the original manifest data to the data cleaning and converting module;
and the data cleaning and converting module is used for cleaning and converting the original manifest data and transmitting and storing the cleaned and converted manifest data to the audit analysis subsystem.
3. The data processing system of claim 1, wherein the audit analysis subsystem comprises:
the data analysis module is used for carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database to obtain an analysis result;
and the analysis result viewing module is used for displaying the analysis result.
4. The data processing system of claim 1, wherein the statistical analysis subsystem comprises:
the statistical module is used for counting and sequencing the total cabin lists in the jurisdiction, the total number of the total containers in the jurisdiction, the cabin lists which are subjected to audit inspection, the number of the containers which are subjected to audit inspection, the cabin lists of discovered audit inspection problems and/or the number of the discovered audit inspection problem containers;
and the analysis module is used for performing cluster analysis on the audit result and the abnormal data by adopting an OLAP multi-dimensional analysis method and generating an analysis report.
5. The data processing system of claim 1, wherein the parameter configuration module comprises:
the system configuration module is used for setting and storing a plurality of configuration items, wherein the configuration items comprise maritime management organization information, collection organization information and user information;
and the log management module is used for recording the execution time and the execution result of the audit task and the operation history of the user.
6. The system of claim 1, wherein the external interface module comprises:
the service interface module is used for realizing wired or wireless interconnection with external service data;
and the portal interface module is used for being butted with an external platform to realize integration.
7. The system of claim 6, wherein the traffic interface module comprises one or more of:
the marine vessel data interface module is used for acquiring marine vessel data, including import vessel data and export vessel data;
the data interface module of the cargo and the container of the manifest is used for acquiring the bill of lading data and the container data corresponding to the manifest;
the third party manifest data interface module is used for acquiring customs entrance forecast manifest data and exit cleaning manifest data of the collection units;
the third party cargo and container data interface module is used for acquiring data of third party cargo and containers;
the nuclear marketing remark interface module is used for acquiring abnormal data generated after the data of the manifest are analyzed;
the audit abnormal data interface module is used for acquiring and inquiring the abnormal data;
and the reacquire manifest data interface module is used for reacquiring manifest data.
8. A data processing method for automatic maritime audit, comprising:
collecting, cleaning and converting original manifest data;
carrying out data integrity and/or accuracy analysis on the cleaned and converted manifest data according to the reference data in the reference database;
generating an audit result according to the analysis data, judging whether the manifest data contains abnormal data, and identifying, tracking and disposing the abnormal data;
carrying out real-time statistical analysis on the audit result and the abnormal data by utilizing an OLAP multi-dimensional analysis method, and generating an analysis report;
wherein the generating of the audit result according to the analysis data comprises: displaying the compared analysis result to a user, determining whether the compared analysis result is listed as an abnormal audit item according to the instruction of the user, marking the abnormal audit item, and generating a final audit result;
the identifying, tracking and handling of anomaly data comprises: identifying, modifying and updating the abnormal data, and feeding back an updated result to a user;
the collecting, cleaning and converting the original manifest data comprises:
calling a third party manifest data interface module to acquire original manifest data;
cleaning, converting and storing the original manifest data, and setting the processing state of the manifest to be 0;
reading all the manifest data with the processing state of 0;
calling a marine vessel data interface module to judge whether marine vessel data exists or not; if the data of the marine vessel exists, calling a bill cargo and container data interface module to obtain bill of lading data and container data corresponding to the bill of lading, cleaning, converting and storing the bill of lading data and the container data, and setting the handling state of the bill of lading to be 1; if no marine vessel data exists, marking the manifest data as abnormal data and setting the manifest handling state to be 8%;
reading all the manifest data with the processing state of 1;
calling a third party manifest cargo and container data interface module to judge whether third party cargo and container data exist or not; if the third-party cargo and container data exist, the third-party cargo and container data are cleaned, converted and stored, and the handling state of the manifest is set to be 2; and if the data of the third-party goods and the containers does not exist, marking the data of the manifest as abnormal data, and setting the setting state of the manifest to be 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811601257.7A CN109684322B (en) | 2018-12-26 | 2018-12-26 | Data processing system and method for automatic maritime affair auditing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811601257.7A CN109684322B (en) | 2018-12-26 | 2018-12-26 | Data processing system and method for automatic maritime affair auditing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109684322A CN109684322A (en) | 2019-04-26 |
| CN109684322B true CN109684322B (en) | 2021-01-22 |
Family
ID=66189645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811601257.7A Active CN109684322B (en) | 2018-12-26 | 2018-12-26 | Data processing system and method for automatic maritime affair auditing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109684322B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111061732A (en) * | 2019-12-05 | 2020-04-24 | 深圳迅策科技有限公司 | Report generation method based on big data processing |
| CN111309717A (en) * | 2020-01-23 | 2020-06-19 | 普信恒业科技发展（北京）有限公司 | Auditing method and system |
| CN111429068B (en) * | 2020-03-31 | 2021-04-23 | 天津市商务局(天津市人民政府口岸服务办公室) | Data supervision method, device and system |
| CN115658795B (en) * | 2022-12-12 | 2023-05-16 | 交通运输部水运科学研究所 | Maritime business data index management method and system supporting multi-scene evaluation |
| CN115982665B (en) * | 2023-03-21 | 2023-07-14 | 北京东华博泰科技有限公司 | Quality anomaly auditing method and system for water turbine measurement data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6094651A (en) * | 1997-08-22 | 2000-07-25 | International Business Machines Corporation | Discovery-driven exploration of OLAP data cubes |
| CN104915793A (en) * | 2015-06-30 | 2015-09-16 | 北京西塔网络科技股份有限公司 | Public information intelligent analysis platform based on big data analysis and mining |
| CN106056292A (en) * | 2016-06-03 | 2016-10-26 | 武汉铁锚焊接材料股份有限公司 | Business operation intelligent decision support system and method thereof |
| CN107256443A (en) * | 2017-04-18 | 2017-10-17 | 国网信通亿力科技有限责任公司 | Line loss real-time computing technique based on business and data integration |
| CN107423390A (en) * | 2017-07-21 | 2017-12-01 | 上海德拓信息技术股份有限公司 | A kind of real time data synchronization algorithm based on inside OLTP OLAP mixed relationship type Database Systems |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101110863A (en) * | 2007-08-16 | 2008-01-23 | 南京联创科技股份有限公司 | Analyzing method for value-added service general settlement |
| CN101894319A (en) * | 2010-06-28 | 2010-11-24 | 中国烟草总公司湖南省公司 | Tobacco enterprise data quality management system and method |
| US9801066B1 (en) * | 2016-06-02 | 2017-10-24 | Duo Security, Inc. | Method for automatic possession-factor authentication |
| CN108470228A (en) * | 2017-02-22 | 2018-08-31 | 国网能源研究院 | Financial data auditing method and audit system |
| CN107230017A (en) * | 2017-05-26 | 2017-10-03 | 安徽皖通科技股份有限公司 | A kind of Highway and Waterway construction and transport market Credit Information Services system |
| CN108564254B (en) * | 2018-03-15 | 2021-04-09 | 国网四川省电力公司绵阳供电公司 | Power distribution equipment state visualization platform based on big data |
-
2018
- 2018-12-26 CN CN201811601257.7A patent/CN109684322B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6094651A (en) * | 1997-08-22 | 2000-07-25 | International Business Machines Corporation | Discovery-driven exploration of OLAP data cubes |
| CN104915793A (en) * | 2015-06-30 | 2015-09-16 | 北京西塔网络科技股份有限公司 | Public information intelligent analysis platform based on big data analysis and mining |
| CN106056292A (en) * | 2016-06-03 | 2016-10-26 | 武汉铁锚焊接材料股份有限公司 | Business operation intelligent decision support system and method thereof |
| CN107256443A (en) * | 2017-04-18 | 2017-10-17 | 国网信通亿力科技有限责任公司 | Line loss real-time computing technique based on business and data integration |
| CN107423390A (en) * | 2017-07-21 | 2017-12-01 | 上海德拓信息技术股份有限公司 | A kind of real time data synchronization algorithm based on inside OLTP OLAP mixed relationship type Database Systems |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109684322A (en) | 2019-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109684322B (en) | Data processing system and method for automatic maritime affair auditing | |
| CA3185178C (en) | Data quality analysis | |
| CN107908550B (en) | Software defect statistical processing method and device | |
| CN105868373B (en) | Method and device for processing key data of power business information system | |
| US20050278296A1 (en) | Method and system for creating, sustaining and using a transactional bill of materials (T-BOM ™) | |
| CN106327068A (en) | Intelligent LIMS system based on monitoring of marine environment and fishery resources | |
| US20140025593A1 (en) | Compliance Analysis System | |
| Schuh et al. | The digital shadow of services: a reference model for comprehensive data collection in MRO services of machine manufacturers | |
| Hikmawati et al. | Improving Data Quality and Data Governance Using Master Data Management: A Review | |
| CN113762735A (en) | Data quality management system and method based on rule base | |
| CN113411253B (en) | Relation topology analysis method and device based on mail, terminal equipment and medium | |
| US20090055203A1 (en) | Method, program code, and system for business process analysis | |
| CN116701358B (en) | Data processing method and system | |
| CN113095647A (en) | Vehicle inspection system | |
| US8504412B1 (en) | Audit automation with survey and test plan | |
| CN108073624B (en) | Service data processing system and method | |
| CN112965793B (en) | Identification analysis data-oriented data warehouse task scheduling method and system | |
| Haindl et al. | An operational constraint language to evaluate feature-dependent non-functional requirements | |
| CN113987015A (en) | Equipment asset state management method, system, equipment and storage medium | |
| CN113052493A (en) | Efficiency monitoring system and working method | |
| CN109697602B (en) | Data processing system for checking fees | |
| US7103615B2 (en) | Process evaluation distributed system | |
| CN111222834A (en) | Enterprise field emergency work data analysis method and system | |
| CN113435701B (en) | Method and device for processing consumption quality information | |
| Schulte et al. | Active files as a measure of software maintainability |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |