CN116680459B - Foreign trade content data processing system based on AI technology - Google Patents

Foreign trade content data processing system based on AI technology Download PDF

Info

Publication number
CN116680459B
CN116680459B CN202310942859.3A CN202310942859A CN116680459B CN 116680459 B CN116680459 B CN 116680459B CN 202310942859 A CN202310942859 A CN 202310942859A CN 116680459 B CN116680459 B CN 116680459B
Authority
CN
China
Prior art keywords
data
layer
information
foreign trade
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310942859.3A
Other languages
Chinese (zh)
Other versions
CN116680459A (en
Inventor
蒋兰波
胡欣然
王复民
钟敏
杨超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changsha Purple Horn E Commerce Co ltd
Original Assignee
Changsha Purple Horn E Commerce Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changsha Purple Horn E Commerce Co ltd filed Critical Changsha Purple Horn E Commerce Co ltd
Priority to CN202310942859.3A priority Critical patent/CN116680459B/en
Publication of CN116680459A publication Critical patent/CN116680459A/en
Application granted granted Critical
Publication of CN116680459B publication Critical patent/CN116680459B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/906—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/211—Schema design and management
    • G06F16/212—Schema design and management with details for data modelling support
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/904—Browsing; Visualisation therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G06Q30/0202—Market predictions or forecasting for commercial activities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Artificial Intelligence (AREA)
  • Development Economics (AREA)
  • Software Systems (AREA)
  • Strategic Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Computer Security & Cryptography (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Technology Law (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Game Theory and Decision Science (AREA)

Abstract

The invention discloses a foreign trade content data processing system based on an AI technology, which relates to the field of data processing and comprises a data acquisition module, a data preprocessing module, a labeling classification module, a data analysis module, a visual monitoring center and a safety reinforcement module, wherein the output end of the data acquisition module is connected with the input end of the data preprocessing module, the output end of the data preprocessing module is connected with the input end of the labeling classification module, the output end of the labeling classification module is connected with the input end of the data analysis module, the data analysis module is in bidirectional connection with the visual monitoring center, the output end of the data acquisition module is connected with the input end of the visual monitoring center, and the safety reinforcement module works in the whole course; the invention can realize the automatic processing of foreign trade content; and the automation degree and the intelligent degree are high.

Description

Foreign trade content data processing system based on AI technology
Technical Field
The present invention relates to the field of data processing, and more particularly to a foreign trade content data processing system based on AI technology.
Background
Along with the rapid development of the foreign trade industry, huge amounts of foreign trade information bring great challenges to enterprises, how to quickly and accurately acquire required contents from the information becomes a difficult problem facing the enterprises, and at present, some foreign trade information processing software or platforms exist in the market, but the software or the platform usually needs manual intervention, and needs a great deal of time and effort, consumes high manpower and material costs and is easy to cause errors, so that trade delay is caused.
AI technology is one of technologies widely used in various industries in recent years, and mainly includes aspects of machine learning, speech recognition, natural language processing, computer vision, and the like. These techniques can simulate human cognition and decision processes through self-learning and adaptation, and can provide more intelligent and efficient data processing and analysis functions for enterprises. With the continuous update and development of AI technology, more and more enterprises begin to adopt foreign trade content data processing systems based on AI technology to improve content management and processing efficiency, but there are still problems of delay in processing speed, insufficient data privacy and security, and data visualization.
Accordingly, the present invention discloses a foreign trade content data processing system based on AI technology.
Disclosure of Invention
Aiming at the defects of the prior art, the invention discloses a foreign trade content data processing system based on an AI technology, which can realize automatic processing of foreign trade content; identifying keywords, entities and topics of the captured information through a deep multilingual extraction model, and carrying out labeled classification on the captured information by adopting a CSS label selector so as to improve the data analysis efficiency; deep mining is carried out on the captured information by adopting a self-adaptive strategy optimization algorithm so as to accurately grasp industry rules and potential customers; predicting a data change trend by adopting a history superposition prediction algorithm so as to accurately predict market trend and industry dynamics; real-time remote monitoring of foreign trade transaction flow information is realized through a high-speed wireless two-way communication network, and the foreign trade transaction flow information is visually displayed by adopting a visual insight platform QlikView; safety reinforcement is carried out through a double-layer application firewall so as to improve network safety and system stability; and the automation degree and the intelligent degree are high.
The invention adopts the following technical scheme:
a foreign trade content data processing system based on AI technology, the system comprising:
the data acquisition module is used for collecting and storing foreign trade content data in real time, and the data acquisition module is used for capturing foreign trade quotation, quotation and transaction flow information in parallel in real time through crawler tools PySpider and storing, managing and backing up the captured information by adopting a cloud storage server AWS 3;
The data preprocessing module is used for cleaning and converting the captured data, and the data preprocessing module adopts a data preprocessing tool Trifaca to rapidly clean, convert and sort large-scale captured information so as to improve the data quality;
the labeling classification module is used for translating and labeling the collected foreign trade content information, the labeling classification module identifies keywords, entities and topics of the captured information through the deep multi-language extraction model, and the CSS label selector is used for labeling the captured information so as to improve the data analysis efficiency;
the data analysis module is used for processing and analyzing the captured data, the data analysis module adopts a self-adaptive strategy optimization algorithm to deeply mine the captured information so as to accurately grasp industry rules and potential customers, and adopts a history superposition prediction algorithm to predict the data change trend so as to accurately predict market trend and industry dynamics;
the history superposition prediction algorithm sets the data sets of the history grabbing data and the real-time grabbing data as data setsT is the moment of capturing data, and the matrix expression for dividing the captured data sample into different characteristic data sets according to the parameter characteristics affecting market trend and industry dynamics is as follows:
（1）
In the formula (1), n is the number of parameter characteristics affecting market trend and industry dynamics, i is more than or equal to 1 and less than or equal to n, m is the number of data of each parameter characteristic, j is more than or equal to 1 and less than or equal to m, and the parameter characteristic data set affecting market trend and industry dynamics isThe ith market trend and industry dynamic parameter feature data set isThe ith output function formula affecting market trend and industry dynamic parameter characteristic data trend prediction is as follows:
（2）
in the formula (2) of the present invention,for the ith market trend and industry dynamic parameter feature data trend prediction result,predicting a weighting function for the ith market trend and industry dynamic parameter characteristic data trend, and +_>For auxiliary weighting parameters->For the jth data in the ith market trend and industry dynamic parameter feature dataset,the (j-1) th data in the (j) th data set for influencing market trend and industry dynamic parameter characteristic data set, and the data set for influencing market trend and industry dynamic parameter characteristic data trend prediction is + ->The trend prediction output function formula of the captured data at the time t+1 is as follows:
（3）
in the formula (3) of the present invention,for the trend of the data captured at time t+1, < >>For the capture data at time t, +.>For the capture data at time t-1, +. >Predicting a weighting function for capturing the trend of the data at time t+1, < >>For auxiliary weighting parameters->Is a maximum function;
the visual monitoring center is used for remotely monitoring foreign trade transaction flow information;
the security reinforcement module is used for protecting the security of foreign trade content data, the security of a system and the security of a kernel platform, and the security reinforcement module performs security reinforcement through a double-layer application firewall;
the output end of the data acquisition module is connected with the input end of the data preprocessing module, the output end of the data preprocessing module is connected with the input end of the labeling classification module, the output end of the labeling classification module is connected with the input end of the data analysis module, the data analysis module is in bidirectional connection with the visual monitoring center, the output end of the data acquisition module is connected with the input end of the visual monitoring center, and the safety reinforcement module works in the whole course.
As a further technical scheme of the present invention, the deep multi-language extraction model includes an input layer, a language recognition layer, a data preprocessing layer, a word segmentation layer, a word vectorization layer, a sentence representation layer, an attention mechanism layer, a neural network layer and an output layer, wherein the output end of the input layer is connected with the input end of the language recognition layer, the output end of the language recognition layer is connected with the input end of the data preprocessing layer, the output end of the data preprocessing layer is connected with the input end of the word segmentation layer, the output end of the word segmentation layer is connected with the input end of the word vectorization layer, the output end of the word vectorization layer is connected with the input end of the sentence representation layer, the output end of the sentence representation layer is connected with the input end of the attention mechanism layer, the output end of the attention mechanism layer is connected with the input end of the neural network layer, and the output end of the neural network layer is connected with the input end of the output layer.
As a further technical scheme of the invention, the working method of the deep multilingual extraction model comprises the following steps:
step 1, format conversion is carried out on the information of the captured foreign trade quotation, quotation and transaction flow, and the information is input into a deep multilingual clustering model through an input layer;
step 2, language recognition is carried out on input text data through a language recognition layer, the language recognition layer judges languages of input information through a convolutional neural network language recognition model and translates the languages, the convolutional neural network language recognition model represents an input information sequence into a vector with a fixed length, features are extracted through multi-layer convolution and pooling operation, on the last layer of full-connection layer, the input information sequence outputs fractional values corresponding to language categories, and a softmax activation function is adopted to convert the fractional values into probability values of the language categories;
step 3, removing stop words, part-of-speech tagging and named entity identification operations on the input information through a data preprocessing layer to obtain entities for capturing information;
step 4, the word segmentation layer carries out word segmentation processing on the input information by using a word segmentation tool NLPIR, and the word segmentation tool NLPIR splits the original text into word sequences;
Step 5, the Word vectorization layer adopts a deep learning Word2Vec Word vector model to calculate the weight of each Word of the input information so as to obtain the key Word of the input information, the Word2Vec Word vector model represents the Word in the text as a high-dimensional vector, and the relation of the words is calculated through the similarity of the high-dimensional vector so as to realize emotion analysis and text classification;
step 6, the sentence representation layer converts the input information after word segmentation into vectors with fixed dimensions by adopting a convolutional neural network CNN and a cyclic neural network RNN model, the convolutional neural network CNN captures local information by carrying out convolutional operation on word vectors and compresses the word vectors into vectors with fixed lengths by adopting a pooling layer so as to obtain the representation of the whole sentence, and the cyclic neural network RNN model transmits context information and outputs the representation of the whole sentence by carrying out recursive calculation on the word vectors;
step 7, focusing attention on the input information keywords by adopting an attention mechanism layer so as to improve the accuracy of the extraction model;
step 8, deeply processing the segmented input information through a feedforward neural network and a cyclic neural network model to extract abstract semantic features of the text;
And 9, outputting keywords, entities and topics of the captured information through an output layer.
As a further technical scheme of the invention, the depth mining of the grabbing information by the self-adaptive strategy optimization algorithm comprises the following steps:
step one, determining a target, determining grabbing information as an objective function, and setting the objective function as F (x), wherein x is a parameter to be optimized, and industry rules and potential customers areSetting an initial parameter vector x to be optimized according to parameters to be optimized 0 As an initial solution of the optimization process of the adaptive strategy optimization algorithm;
step two, implementing an optimization algorithm, estimating the dynamic situation and optional action of the current industry rule by adopting an adaptive strategy optimization algorithm according to the preprocessing result and the keywords, entities and subjects of the captured information, and carrying out iterative operation on the adaptive strategy optimization algorithm according to the scale and complexity of the captured information data set to realize parameter adjustment, wherein in each iteration, the adaptive strategy optimization algorithm is carried out according to the current parameter vector x n Calculate the objective function value F (x) n ) And according to F (x n ) For x n Decision making, generating a parameter vector x of the next round of optimization iteration n+1 Current parameter vector x n For the nth industry rule and potential customer, n is the number of parameter vectors, and the objective function value F (x n ) For the nth grab information, parameter vector x n+1 Is the n+1th industry rule and potential customer;
thirdly, data modeling and evaluation are carried out, wherein the data model of the self-adaptive strategy optimization algorithm is evaluated and verified in the process of data modeling and analysis through optimizing the quality change of the data model of the iterative self-adaptive strategy optimization algorithm, so that the accuracy and the effectiveness of the data model of the self-adaptive strategy optimization algorithm are ensured, the next layer analysis and mining are carried out on the result of the self-adaptive strategy optimization algorithm, a Gaussian optimization strategy is selected to carry out optimization iteration on a parameter vector, and the parameter vector x generated in the iteration is adopted n+1 Judging the effectiveness of the Gaussian optimization strategy, and optimizing the objective function F (x of iteration in the next round n+1 ) Is smaller than the objective function F (x n ) Then use the parameter vector x n+1 Continuing the optimization iteration, F (x n+1 ) Greater than F (x) n ) Then the parameter vector x is retained n Switching different strategies to perform optimization iteration;
step four, optimization and improvement, wherein in the depth mining process, the data model of the adaptive strategy optimization algorithm is optimized and improved by introducing priori knowledge, selecting proper initial solution, improving fitness function and design parameter control strategy so as to improveAccuracy and efficiency of data mining, stopping iteration when a predefined accuracy or time limit is reached, parameter vector x generated by the last iteration n As an optimal industry rule or potential customer output.
As a further technical scheme of the invention, the visual monitoring center realizes real-time remote monitoring of foreign trade transaction flow information through a high-speed wireless two-way communication network, and adopts a visual insight platform QlikView to visually display the foreign trade transaction flow information, wherein the foreign trade transaction flow information comprises contract creation information, payment settlement information and logistics tracking information.
As a further technical scheme of the invention, the high-speed wireless two-way communication network adopts an MQTT lightweight bottom layer protocol, a UDP transport layer protocol, an HTTP/2 secure transport protocol and a WebSocket two-way communication protocol to realize real-time data interaction between a client and a server so as to reduce network communication delay, and distributes data to a transmission node through server load balancing logic and message queue service so as to realize rapid retransmission of node faults.
As a further technical scheme of the invention, the visual insight platform QlikView acquires mass data source association data based on an association data model to realize multidimensional data association analysis, and adopts an interactive chart, a heat point diagram, a map and an instrument board to realize real-time monitoring of trend, relationship and change rule of data, and the visual insight platform QlikView adopts a Token user identity verification mechanism to verify the identity of an access user so as to improve the safety of information access.
As a further technical scheme of the invention, the SSL secure socket layer protocol acceleration card is adopted by the double-layer application firewall to improve the secure access speed and the device performance processing capability, the SSL secure socket layer protocol acceleration card shortens the user access time and lightens the load of a server in the firewall by accelerating the processing process of the connection of a secure socket layer and a transmission layer, and the double-layer application firewall classifies, monitors and protects sensitive data in an internal network by cooperating with anti-leak attack APT and data security service platform security devices, and recognizes SQL injection, cross-site script attack XSS and command injection attack behaviors by an intelligent security engine and recognizes unknown threat and unblemented Ding Loudong attack behaviors to improve network security and system stability.
Has the positive beneficial effects that:
the invention discloses a foreign trade content data processing system based on an AI technology, which can realize automatic processing of foreign trade content; identifying keywords, entities and topics of the captured information through a deep multilingual extraction model, and carrying out labeled classification on the captured information by adopting a CSS label selector so as to improve the data analysis efficiency; deep mining is carried out on the captured information by adopting a self-adaptive strategy optimization algorithm so as to accurately grasp industry rules and potential customers; predicting a data change trend by adopting a history superposition prediction algorithm so as to accurately predict market trend and industry dynamics; real-time remote monitoring of foreign trade transaction flow information is realized through a high-speed wireless two-way communication network, and the foreign trade transaction flow information is visually displayed by adopting a visual insight platform QlikView; safety reinforcement is carried out through a double-layer application firewall so as to improve network safety and system stability; and the automation degree and the intelligent degree are high.
Drawings
FIG. 1 is a schematic diagram of the overall architecture of a foreign trade content data processing system based on AI technology;
FIG. 2 is a schematic diagram of a high-speed wireless two-way communication network in a foreign trade content data processing system based on AI technology;
FIG. 3 is a schematic diagram of a model of an adaptive strategy optimization algorithm in a foreign trade content data processing system based on AI technology;
fig. 4 is a working circuit diagram of a visual monitoring center in a foreign trade content data processing system based on AI technology.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1-4, a foreign trade content data processing system based on AI technology, the system comprising:
the data acquisition module is used for collecting and storing foreign trade content data in real time, and the data acquisition module is used for capturing foreign trade quotation, quotation and transaction flow information in parallel in real time through crawler tools PySpider and storing, managing and backing up the captured information by adopting a cloud storage server AWS 3;
The data preprocessing module is used for cleaning and converting the captured data, and the data preprocessing module adopts a data preprocessing tool Trifaca to rapidly clean, convert and sort large-scale captured information so as to improve the data quality;
the labeling classification module is used for translating and labeling the collected foreign trade content information, the labeling classification module identifies keywords, entities and topics of the captured information through the deep multi-language extraction model, and the CSS label selector is used for labeling the captured information so as to improve the data analysis efficiency;
the data analysis module is used for processing and analyzing the captured data, the data analysis module adopts a self-adaptive strategy optimization algorithm to deeply mine the captured information so as to accurately grasp industry rules and potential customers, and adopts a history superposition prediction algorithm to predict the data change trend so as to accurately predict market trend and industry dynamics;
the history superposition prediction algorithm sets the data sets of the history grabbing data and the real-time grabbing data as data setsT is the moment of capturing data, and the matrix expression for dividing the captured data sample into different characteristic data sets according to the parameter characteristics affecting market trend and industry dynamics is as follows:
（1）
In the formula (1), n is the number of parameter characteristics affecting market trend and industry dynamics, i is more than or equal to 1 and less than or equal to n, m is the number of data of each parameter characteristic, j is more than or equal to 1 and less than or equal to m, and the parameter characteristic data set affecting market trend and industry dynamics isThe ith market trend and industry dynamic parameter feature data set isThe ith output function formula affecting market trend and industry dynamic parameter characteristic data trend prediction is as follows:
（2）
in the formula (2) of the present invention,for the ith market trend and industry dynamic parameter feature data trend prediction result,predicting a weighting function for the ith market trend and industry dynamic parameter characteristic data trend, and +_>For auxiliary weighting parameters->For the jth data in the ith market trend and industry dynamic parameter feature dataset,the (j-1) th data in the (j) th data set for influencing market trend and industry dynamic parameter characteristic data set, and the data set for influencing market trend and industry dynamic parameter characteristic data trend prediction is + ->The trend prediction output function formula of the captured data at the time t+1 is as follows:
（3）
in the formula (3) of the present invention,for the trend of the data captured at time t+1, < >>For the capture data at time t, +.>For the capture data at time t-1, +. >Predicting a weighting function for capturing the trend of the data at time t+1, < >>For auxiliary weighting parameters->Is a maximum function; the visual monitoring center is used for remotely monitoring foreign trade transaction flow information;
the security reinforcement module is used for protecting the security of foreign trade content data, the security of a system and the security of a kernel platform, and the security reinforcement module performs security reinforcement through a double-layer application firewall;
the output end of the data acquisition module is connected with the input end of the data preprocessing module, the output end of the data preprocessing module is connected with the input end of the labeling classification module, the output end of the labeling classification module is connected with the input end of the data analysis module, the data analysis module is in bidirectional connection with the visual monitoring center, the output end of the data acquisition module is connected with the input end of the visual monitoring center, and the safety reinforcement module works in the whole course.
In a specific embodiment, the system is based on an AI technology and is mainly used for helping enterprises to process massive foreign trade content data more efficiently, so that the working efficiency and accuracy are improved. The system utilizes web crawler technology to automatically collect various information about foreign trade on the Internet, including commodity price, supplier information, market trend, competition analysis and the like, and cleans, analyzes, classifies and presents the information on a user interface through natural language processing technology. Based on big data technology, the system can carry out deep analysis on market trend, competition condition and supply chain, provide key data and analysis report, and assist decision maker in making more scientific and effective business strategy. Based on the blockchain and intelligent contract technology, the system can assist the user to realize supply chain intelligent management, including contract signing, payment settlement, logistics tracking and the like, so that the cooperation efficiency is greatly improved and the trade safety is ensured.
In the above embodiment, the deep multiple language extraction model includes an input layer, a language recognition layer, a data preprocessing layer, a word segmentation layer, a word vectorization layer, a sentence representation layer, an attention mechanism layer, a neural network layer and an output layer, wherein the output end of the input layer is connected with the input end of the language recognition layer, the output end of the language recognition layer is connected with the input end of the data preprocessing layer, the output end of the data preprocessing layer is connected with the input end of the word segmentation layer, the output end of the word segmentation layer is connected with the input end of the word vectorization layer, the output end of the word vectorization layer is connected with the input end of the sentence representation layer, the output end of the sentence representation layer is connected with the input end of the attention mechanism layer, the output end of the attention mechanism layer is connected with the input end of the neural network layer, and the output end of the neural network layer is connected with the input end of the output layer.
In the above embodiment, the working method of the deep multilingual extraction model includes the following steps:
step 1, format conversion is carried out on the information of the captured foreign trade quotation, quotation and transaction flow, and the information is input into a deep multilingual clustering model through an input layer;
Step 2, language recognition is carried out on input text data through a language recognition layer, the language recognition layer judges languages of input information through a convolutional neural network language recognition model and translates the languages, the convolutional neural network language recognition model represents an input information sequence into a vector with a fixed length, features are extracted through multi-layer convolution and pooling operation, on the last layer of full-connection layer, the input information sequence outputs fractional values corresponding to language categories, and a softmax activation function is adopted to convert the fractional values into probability values of the language categories;
step 3, removing stop words, part-of-speech tagging and named entity identification operations on the input information through a data preprocessing layer to obtain entities for capturing information;
step 4, the word segmentation layer carries out word segmentation processing on the input information by using a word segmentation tool NLPIR, and the word segmentation tool NLPIR splits the original text into word sequences;
step 5, the Word vectorization layer adopts a deep learning Word2Vec Word vector model to calculate the weight of each Word of the input information so as to obtain the key Word of the input information, the Word2Vec Word vector model represents the Word in the text as a high-dimensional vector, and the relation of the words is calculated through the similarity of the high-dimensional vector so as to realize emotion analysis and text classification;
Step 6, the sentence representation layer converts the input information after word segmentation into vectors with fixed dimensions by adopting a convolutional neural network CNN and a cyclic neural network RNN model, the convolutional neural network CNN captures local information by carrying out convolutional operation on word vectors and compresses the word vectors into vectors with fixed lengths by adopting a pooling layer so as to obtain the representation of the whole sentence, and the cyclic neural network RNN model transmits context information and outputs the representation of the whole sentence by carrying out recursive calculation on the word vectors;
step 7, focusing attention on the input information keywords by adopting an attention mechanism layer so as to improve the accuracy of the extraction model;
step 8, deeply processing the segmented input information through a feedforward neural network and a cyclic neural network model to extract abstract semantic features of the text;
and 9, outputting keywords, entities and topics of the captured information through an output layer.
In a specific embodiment, through a deep multilingual extraction model, language identification and keyword, entity and theme extraction can be performed on the collected information. Specifically, the captured text data may be subjected to processing such as parsing, keyword extraction, and entity recognition using natural language processing techniques to extract useful information. Meanwhile, by combining the technologies of machine learning, deep learning, neural network and the like, more efficient and accurate information extraction can be performed. For example, models such as LSTM, BERT, etc. may be used to perform text classification, intent recognition, etc. to achieve more accurate information classification and analysis.
For the captured information, a CSS tag selector may be used for tag classification. The CSS tag selector can screen and classify according to tag attributes in the HTML document to tag identification of different information such as text, pictures and the like. In this way, classification and statistics of different types of information can be achieved, and powerful support is provided for subsequent data analysis.
In summary, by using the deep multi-language extraction model and the CSS tag selector, efficient information capturing and classification can be achieved, and data analysis efficiency is improved, and the efficiency of the data analysis process using the deep multi-language extraction model and the CSS tag selector and not using the deep multi-language extraction model and the CSS tag selector is shown in table 1.
Table 1 analysis speed versus statistics table
As can be seen from Table 1, by adopting the deep multilingual extraction model and the CSS tag selector, efficient information capture and classification can be realized, and the data analysis efficiency is improved.
In the above embodiment, the depth mining of the captured information by the adaptive policy optimization algorithm includes the following steps:
step one, determining a target, determining grabbing information as an objective function, setting the objective function as F (x), wherein x is a parameter to be optimized, industry rules and potential customers are parameters to be optimized, and setting an initial parameter vector x to be optimized 0 As an initial solution of the optimization process of the adaptive strategy optimization algorithm;
step two, implementing an optimization algorithm, estimating the dynamic situation and optional action of the current industry rule by adopting an adaptive strategy optimization algorithm according to the preprocessing result and the keywords, entities and subjects of the captured information, and carrying out iterative operation on the adaptive strategy optimization algorithm according to the scale and complexity of the captured information data set to realize parameter adjustment, wherein in each iteration, the adaptive strategy optimization algorithm is carried out according to the current parameter vector x n Calculate the objective function value F (x) n ) And according to F (x n ) For x n Decision making, generating a parameter vector x of the next round of optimization iteration n+1 Current parameter vector x n For the nth industry rule and potential customer, n is the number of parameter vectors, and the objective function value F (x n ) For the nth grab information, parameter vector x n+1 Is the n+1th industry rule and potential customer;
thirdly, data modeling and evaluation are carried out, wherein the data model of the self-adaptive strategy optimization algorithm is evaluated and verified in the process of data modeling and analysis through optimizing the quality change of the data model of the iterative self-adaptive strategy optimization algorithm, so that the accuracy and the effectiveness of the data model of the self-adaptive strategy optimization algorithm are ensured, the next layer analysis and mining are carried out on the result of the self-adaptive strategy optimization algorithm, a Gaussian optimization strategy is selected to carry out optimization iteration on a parameter vector, and the parameter vector x generated in the iteration is adopted n+1 Judging the effectiveness of the Gaussian optimization strategy, and optimizing the objective function F (x of iteration in the next round n+1 ) Is smaller than the objective function F (x n ) Then use the parameter vector x n+1 Continuing the optimization iteration, F (x n+1 ) Greater than F (x) n ) Then the parameter vector x is retained n Switching different strategies to perform optimization iteration;
step four, optimization and improvement, in the process of deep excavation, the prior is introducedKnowledge, proper initial solution selection, improved fitness function and design parameter control strategy are adopted to optimize and improve the data model of the self-adaptive strategy optimization algorithm so as to improve the accuracy and efficiency of data mining, when the accuracy or time limit predefined is reached, iteration is stopped, and the parameter vector x generated by the last iteration is generated n As an optimal industry rule or potential customer output.
In a specific embodiment, when capturing information for deep mining, the adaptive strategy optimization algorithm can help to quickly find out the characteristics and change rules of the information, so that analysis efficiency and accuracy are improved. The depth mining of the captured information by using the adaptive strategy optimization algorithm and the depth mining of the captured information by using the traditional depth mining algorithm are shown in table 2.
Table 2 comparative statistics table
As can be seen from table 2, the depth mining of the captured information by using the adaptive strategy optimization algorithm can help to quickly find out the characteristics and change rule of the information, thereby improving the analysis efficiency and accuracy.
In a specific embodiment, the historical superposition prediction algorithm is a commonly used time series prediction method, which builds a mathematical model of the trend of data change through analysis of historical data, and uses the model to predict future data change. Firstly, an effective data source is required to be selected, and data is cleaned and processed to meet the requirements of a history superposition prediction algorithm. The aim of the cleaning and processing is to remove outliers, noise and other interference factors, and prevent the factors from affecting the accuracy and reliability of data analysis. Establishing a mathematical model is a key step in a history superposition prediction algorithm, and can model and predict data by selecting an appropriate model. An autoregressive moving average model (ARIMA) is a widely used time series model, and can consider a plurality of factors such as data change trend, seasonal period and the like, so that the model is a common algorithm for predicting market trend and industry dynamics. After the mathematical model is established, parameter adjustment is needed to be carried out on the model so that the model is more fit with the actual situation. And then, future market trend and industry dynamics can be predicted through the model, so that corresponding market strategies are formulated and product strategies are adjusted, and the market competitiveness and production efficiency of enterprises are improved. Analysis and evaluation of the results are required after prediction. A model with a smaller prediction error may be selected for prediction and then the difference between the predicted result and the actual result is compared. If the prediction error is large, the model needs to be corrected to improve the accuracy and reliability of the prediction.
In a word, the historical superposition prediction algorithm can accurately predict market trend and industry dynamics, improve decision making capability and market predicting capability of enterprises, optimize production and sales plans of the enterprises, further improve competitiveness of the enterprises, and the effects of accurately predicting the market trend and the industry dynamics by adopting the historical superposition prediction algorithm and predicting the market trend and the industry dynamics by adopting the traditional prediction algorithm are shown in a table 3.
Table 3 comparison of prediction accuracy statistics
As shown in table 3, the prediction accuracy of the history superimposed prediction algorithm model is greatly improved compared with the conventional prediction model.
In the above embodiment, the visual monitoring center realizes real-time remote monitoring of foreign trade transaction flow information through a high-speed wireless two-way communication network, and adopts a visual insight platform QlikView to visually display the foreign trade transaction flow information, wherein the foreign trade transaction flow information comprises contract creation information, payment settlement information and logistics tracking information.
In a specific embodiment, the visual monitoring center realizes real-time remote monitoring of foreign trade transaction flow information through a high-speed wireless two-way communication network, so that related personnel can know the progress situation of the foreign trade transaction flow in time, and corresponding measures can be taken in time. The high-speed wireless two-way communication network can transmit and receive data through radio waves and support two-way communication functions, namely, the high-speed wireless two-way communication network can transmit and receive data. The network has the advantages of high transmission speed, wide coverage, high flexibility and the like. Meanwhile, the visual monitoring center adopts a visual insight platform QlikView to visually display foreign trade transaction flow information, so that data information can be presented more intuitively, and people can understand the relationship and trend among data better. QlikView is a business intelligence tool that can extract, transform and load data from multiple data sources and convert it into forms of charts, tables, reports, etc. that are easy to understand and use. It also supports custom queries, filters, ranker functions, etc., allowing users to freely organize and analyze data as desired.
The foreign trade transaction flow information includes contract creation information, payment settlement information, and logistics tracking information. The contract creation information comprises contract numbers, first party names, second party names, signing time and the like; the payment settlement information comprises payment amount, payment mode, collection account number and other contents; the logistics tracking information comprises contents such as transportation modes, cargo states, expected arrival time and the like. The information is an important link in the foreign trade process, and is very critical for monitoring and managing the foreign trade process.
In a word, the visual monitoring center realizes real-time remote monitoring of foreign trade transaction flow information through the high-speed wireless two-way communication network, and adopts the visual insight platform QlikView to visually display the foreign trade transaction flow information, so that people can better know and manage the foreign trade transaction flow, and the working efficiency and the decision level are improved.
In the above embodiment, the high-speed wireless bidirectional communication network adopts the MQTT lightweight bottom layer protocol, the UDP transport layer protocol, the HTTP/2 secure transport protocol and the WebSocket bidirectional communication protocol to implement real-time data interaction between the client and the server, so as to reduce network communication delay, and distributes data to the transmission nodes through the server load balancing logic and the message queue service, so that rapid retransmission of node faults is implemented.
In particular embodiments, the MQTT is a lightweight messaging protocol specifically designed for internet of things applications. It is capable of transmitting data with very low network bandwidth and CPU overhead while providing a reliable message transmission mechanism. The protocol can effectively reduce the load and delay of data transmission and improve the expandability and flexibility of the system. The UDP protocol is one of transport layer protocols, and compared with the TCP protocol, it can implement faster data transmission, and is generally used in a scenario with low requirements for reliability of data, where data can be distributed in a broadcast manner, and data can be transmitted in a peer-to-peer manner, so that the UDP protocol has higher real-time performance and reliability. The HTTP/2 protocol is a secure transmission protocol, and encrypts data when transmitting by adopting a transport layer security protocol TLS, so that the efficiency and the speed of data transmission can be improved while the data security is ensured. The WebSocket bidirectional communication protocol can realize bidirectional real-time interaction between the client and the server, avoid multiple responses of the server, and reduce the time and network delay of data transmission.
In order to further improve the efficiency and reliability of data transmission, the high-speed wireless two-way communication network adopts server load balancing logic and message queue service to distribute data to transmission nodes so as to realize rapid retransmission of node faults. The load balancing logic can distribute the request to the optimal node according to the load conditions of different nodes, so that the rapid transmission and distribution of the data are realized. Meanwhile, through the message queue service, the request of the peak elimination client can be buffered, so that the performance and the reliability of the system are improved.
In the above embodiment, the visualization insight platform QlikView acquires the related data of the mass data source based on the related data model to realize multidimensional data related analysis, and adopts the interactive chart, the heat point diagram, the map and the instrument panel to realize the real-time monitoring of the trend, the relationship and the change rule of the data, and the visualization insight platform QlikView adopts the Token user identity verification mechanism to verify the identity of the accessing user so as to improve the security of information access.
In a specific embodiment, one of key characteristics of the visual insight platform QlikView is to acquire mass data source association data based on an association data model so as to realize multidimensional data association analysis. The QlikView can be used for quickly connecting a plurality of data sources with different formats, converting the data and then presenting the converted data in a form of a relevance data model. The relevance data model can automatically process the relation in the data, thereby realizing multidimensional data relevance analysis and providing more accurate and fine data support for decision making of enterprise business. In addition, qlikView also provides a flexible authority management mechanism, and enterprises can carry out different-level authority management on users according to actual conditions, so that the security and privacy of data are ensured.
In a word, qlikView serves as an advanced visual insight platform, and through the functions of multidimensional data association analysis, rich visual presentation tools, a safe user identity verification mechanism and the like, accurate, efficient and safe data analysis and decision support services are provided for enterprises.
In the above embodiment, the dual-layer application firewall adopts the SSL secure socket layer protocol acceleration card to improve the secure access speed and the device performance processing capability, and the SSL secure socket layer protocol acceleration card accelerates the processing procedure of the connection between the secure socket layer and the transport layer, so as to shorten the user access time and reduce the load of the server inside the firewall, and the dual-layer application firewall classifies, monitors and protects the sensitive data in the internal network by cooperating with the anti-leak attack APT and the data security service platform security device, and recognizes the SQL injection, cross-site script attack XSS and command injection attack behaviors by the intelligent security engine, and recognizes the unknown threat and the unblemented Ding Loudong attack behaviors, so as to improve the network security and the system stability.
In a specific embodiment, the dual-layer application firewall refers to a firewall security mechanism, which consists of two independent firewalls, one located at the boundary of the internal network and the other located at the boundary of the external network.
An external firewall (also referred to as a boundary firewall) is located at the edge of the network to protect the internal network from threats from external networks, such as unauthorized remote access, malware, and attacks. The external firewall filters and restricts external traffic from entering the internal network.
The internal firewall is located at the edge of the internal network, and segments the internet into multiple security areas to improve network security. It can help protect important applications and sensitive data in internal networks from internal attacks, intrusions, and abuse. The internal firewall may also help control traffic and access rights between different departments in the internal network.
The advantage of a double-layer application firewall is that it provides multiple layers of protection against external attacks and internal threats effectively. Even if the external firewall is attacked and bypassed, the internal firewall may still provide an additional protection layer to ensure network and system security. The dual-layer application firewall may also provide better scalability and manageability because the internal and external firewalls may be managed separately, facilitating flexible network security policies and updates.
While specific embodiments of the present invention have been described above, it will be understood by those skilled in the art that these specific embodiments are by way of example only, and that various omissions, substitutions, and changes in the form and details of the methods and systems described above may be made by those skilled in the art without departing from the spirit and scope of the invention. For example, it is within the scope of the present invention to combine the above-described method steps to perform substantially the same function in substantially the same way to achieve substantially the same result. Accordingly, the scope of the invention is limited only by the following claims.

Claims (8)

1. A foreign trade content data processing system based on AI technology is characterized in that: the system comprises:
the data acquisition module is used for collecting and storing foreign trade content data in real time, and the data acquisition module is used for capturing foreign trade quotation, quotation and transaction flow information in parallel in real time through crawler tools PySpider and storing, managing and backing up the captured information by adopting a cloud storage server AWS 3;
the data preprocessing module is used for cleaning and converting the captured data, and the data preprocessing module adopts a data preprocessing tool Trifaca to rapidly clean, convert and sort large-scale captured information so as to improve the data quality;
the labeling classification module is used for translating and labeling the collected foreign trade content information, the labeling classification module identifies keywords, entities and topics of the captured information through the deep multi-language extraction model, and the CSS label selector is used for labeling the captured information so as to improve the data analysis efficiency;
the data analysis module is used for processing and analyzing the captured data, the data analysis module adopts a self-adaptive strategy optimization algorithm to deeply mine the captured information so as to accurately grasp industry rules and potential customers, and adopts a history superposition prediction algorithm to predict the data change trend so as to accurately predict market trend and industry dynamics;
The history superposition prediction algorithm sets the data sets of the history grabbing data and the real-time grabbing data as data setsT is the moment of capturing data, and the matrix expression for dividing the captured data sample into different characteristic data sets according to the parameter characteristics affecting market trend and industry dynamics is as follows:
（1）
in the formula (1), n is the number of parameter characteristics affecting market trend and industry dynamics, i is more than or equal to 1 and less than or equal to n, m is the number of data of each parameter characteristic, j is more than or equal to 1 and less than or equal to m, and the parameter characteristic data set affecting market trend and industry dynamics isThe ith market trend and industry dynamic parameter feature data set isIth influenceThe output function formula of market trend and industry dynamic parameter characteristic data trend prediction is as follows:
（2）
in the formula (2) of the present invention,for the ith market trend and industry dynamic parameter feature data trend prediction result,predicting a weighting function for the ith market trend and industry dynamic parameter characteristic data trend, and +_>For auxiliary weighting parameters->For the jth data in the ith market trend and industry dynamic parameter feature dataset,the (j-1) th data in the (j) th data set for influencing market trend and industry dynamic parameter characteristic data set, and the data set for influencing market trend and industry dynamic parameter characteristic data trend prediction is + - >The trend prediction output function formula of the captured data at the time t+1 is as follows:
（3）
in the formula (3) of the present invention,for the trend of the data captured at time t+1, < >>For the capture data at time t, +.>For the capture data at time t-1, +.>Predicting a weighting function for capturing the trend of the data at time t+1, < >>For auxiliary weighting parameters->Is a maximum function;
the visual monitoring center is used for remotely monitoring foreign trade transaction flow information;
the security reinforcement module is used for protecting the security of foreign trade content data, the security of a system and the security of a kernel platform, and the security reinforcement module performs security reinforcement through a double-layer application firewall;
the output end of the data acquisition module is connected with the input end of the data preprocessing module, the output end of the data preprocessing module is connected with the input end of the labeling classification module, the output end of the labeling classification module is connected with the input end of the data analysis module, the data analysis module is in bidirectional connection with the visual monitoring center, the output end of the data acquisition module is connected with the input end of the visual monitoring center, and the safety reinforcement module works in the whole course.
2. The AI-technology-based foreign trade content data processing system of claim 1, wherein: the deep multi-language extraction model comprises an input layer, a language identification layer, a data preprocessing layer, a word segmentation layer, a word vectorization layer, a sentence representation layer, an attention mechanism layer, a neural network layer and an output layer, wherein the output end of the input layer is connected with the input end of the language identification layer, the output end of the language identification layer is connected with the input end of the data preprocessing layer, the output end of the data preprocessing layer is connected with the input end of the word segmentation layer, the output end of the word segmentation layer is connected with the input end of the word vectorization layer, the output end of the word vectorization layer is connected with the input end of the sentence representation layer, the output end of the sentence representation layer is connected with the input end of the attention mechanism layer, the output end of the attention mechanism layer is connected with the input end of the neural network layer, and the output end of the neural network layer is connected with the input end of the output layer.
3. The AI-technology-based foreign trade content data processing system of claim 1, wherein: the working method of the deep multilingual extraction model comprises the following steps:
step 1, format conversion is carried out on the information of the captured foreign trade quotation, quotation and transaction flow, and the information is input into a deep multilingual clustering model through an input layer;
step 2, language recognition is carried out on input text data through a language recognition layer, the language recognition layer judges languages of input information through a convolutional neural network language recognition model and translates the languages, the convolutional neural network language recognition model represents an input information sequence into a vector with a fixed length, features are extracted through multi-layer convolution and pooling operation, on the last layer of full-connection layer, the input information sequence outputs fractional values corresponding to language categories, and a softmax activation function is adopted to convert the fractional values into probability values of the language categories;
step 3, removing stop words, part-of-speech tagging and named entity identification operations on the input information through a data preprocessing layer to obtain entities for capturing information;
step 4, the word segmentation layer carries out word segmentation processing on the input information by using a word segmentation tool NLPIR, and the word segmentation tool NLPIR splits the original text into word sequences;
Step 5, the Word vectorization layer adopts a deep learning Word2Vec Word vector model to calculate the weight of each Word of the input information so as to obtain the key Word of the input information, the Word2Vec Word vector model represents the Word in the text as a high-dimensional vector, and the relation of the words is calculated through the similarity of the high-dimensional vector so as to realize emotion analysis and text classification;
step 6, the sentence representation layer converts the input information after word segmentation into vectors with fixed dimensions by adopting a convolutional neural network CNN and a cyclic neural network RNN model, the convolutional neural network CNN captures local information by carrying out convolutional operation on word vectors and compresses the word vectors into vectors with fixed lengths by adopting a pooling layer so as to obtain the representation of the whole sentence, and the cyclic neural network RNN model transmits context information and outputs the representation of the whole sentence by carrying out recursive calculation on the word vectors;
step 7, focusing attention on the input information keywords by adopting an attention mechanism layer so as to improve the accuracy of the extraction model;
step 8, deeply processing the segmented input information through a feedforward neural network and a cyclic neural network model to extract abstract semantic features of the text;
And 9, outputting keywords, entities and topics of the captured information through an output layer.
4. The AI-technology-based foreign trade content data processing system of claim 1, wherein: the adaptive strategy optimization algorithm performs deep mining on the captured information, and comprises the following steps:
step one, determining a target, determining grabbing information as an objective function, setting the objective function as F (x), wherein x is a parameter to be optimized, industry rules and potential customers are parameters to be optimized, and setting an initial parameter vector x to be optimized 0 As an initial solution of the optimization process of the adaptive strategy optimization algorithm;
step two, implementing an optimization algorithm, estimating the dynamic situation and optional action of the current industry rule by adopting the self-adaptive strategy optimization algorithm according to the preprocessing result and the keywords, entities and subjects of the captured information, and carrying out iterative operation on the self-adaptive strategy optimization algorithm according to the scale and complexity of the captured information data set to realize parameter adjustmentIn each iteration, the adaptive strategy optimization algorithm is based on the current parameter vector x n Calculate the objective function value F (x) n ) And according to F (x n ) For x n Decision making, generating a parameter vector x of the next round of optimization iteration n+1 Current parameter vector x n For the nth industry rule and potential customer, n is the number of parameter vectors, and the objective function value F (x n ) For the nth grab information, parameter vector x n+1 Is the n+1th industry rule and potential customer;
thirdly, data modeling and evaluation are carried out, wherein the data model of the self-adaptive strategy optimization algorithm is evaluated and verified in the process of data modeling and analysis through optimizing the quality change of the data model of the iterative self-adaptive strategy optimization algorithm, so that the accuracy and the effectiveness of the data model of the self-adaptive strategy optimization algorithm are ensured, the next layer analysis and mining are carried out on the result of the self-adaptive strategy optimization algorithm, a Gaussian optimization strategy is selected to carry out optimization iteration on a parameter vector, and the parameter vector x generated in the iteration is adopted n+1 Judging the effectiveness of the Gaussian optimization strategy, and optimizing the objective function F (x of iteration in the next round n+1 ) Is smaller than the objective function F (x n ) Then use the parameter vector x n+1 Continuing the optimization iteration, F (x n+1 ) Greater than F (x) n ) Then the parameter vector x is retained n Switching different strategies to perform optimization iteration;
optimizing and improving the data model of the self-adaptive strategy optimization algorithm by introducing priori knowledge, selecting proper initial solution, improving fitness function and designing parameter control strategy in the process of depth mining so as to improve the accuracy and efficiency of data mining, stopping iteration when reaching the predefined accuracy or time limit, and generating a parameter vector x by the last iteration n As an optimal industry rule or potential customer output.
5. The AI-technology-based foreign trade content data processing system of claim 1, wherein: the visual monitoring center realizes real-time remote monitoring of foreign trade transaction flow information through a high-speed wireless two-way communication network, and adopts a visual insight platform QlikView to visually display the foreign trade transaction flow information, wherein the foreign trade transaction flow information comprises contract creation information, payment settlement information and logistics tracking information.
6. The AI-technology-based foreign trade content data processing system of claim 5, wherein: the high-speed wireless two-way communication network adopts an MQTT lightweight bottom layer protocol, a UDP transport layer protocol, an HTTP/2 secure transport protocol and a WebSocket two-way communication protocol to realize real-time data interaction between a client and a server so as to reduce network communication delay, and distributes data to a transmission node through server load balancing logic and message queue service so as to realize rapid retransmission of node faults.
7. The AI-technology-based foreign trade content data processing system of claim 5, wherein: the visual insight platform QlikView acquires mass data source associated data based on the associated data model to realize multidimensional data associated analysis, and adopts an interactive chart, a hot point diagram, a map and an instrument board to realize real-time monitoring of trend, relationship and change rule of data, and the visual insight platform QlikView adopts a Token user identity verification mechanism to verify the identity of an access user so as to improve the safety of information access.
8. The AI-technology-based foreign trade content data processing system of claim 1, wherein: the double-layer application firewall adopts an SSL secure socket layer protocol acceleration card to improve the secure access speed and the device performance processing capacity, the SSL secure socket layer protocol acceleration card shortens the user access time and lightens the load of a server in the firewall by accelerating the processing process of the connection of a secure socket layer and a transmission layer, and classifies, monitors and protects sensitive data in an internal network by cooperating with anti-leak attack APT and data security service platform security devices, and recognizes SQL injection, cross-site script attack XSS and command injection attack behaviors by an intelligent security engine, and recognizes unknown threat and unblemented Ding Loudong attack behaviors to improve network security and system stability.
CN202310942859.3A 2023-07-31 2023-07-31 Foreign trade content data processing system based on AI technology Active CN116680459B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310942859.3A CN116680459B (en) 2023-07-31 2023-07-31 Foreign trade content data processing system based on AI technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310942859.3A CN116680459B (en) 2023-07-31 2023-07-31 Foreign trade content data processing system based on AI technology

Publications (2)

Publication Number Publication Date
CN116680459A CN116680459A (en) 2023-09-01
CN116680459B true CN116680459B (en) 2023-10-13

Family

ID=87785813

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310942859.3A Active CN116680459B (en) 2023-07-31 2023-07-31 Foreign trade content data processing system based on AI technology

Country Status (1)

Country Link
CN (1) CN116680459B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117194576A (en) * 2023-10-07 2023-12-08 贵州电网有限责任公司信息中心 Power grid customer information data integration processing method and system
CN117114226B (en) * 2023-10-20 2024-01-30 无锡宇拓物联信息科技有限公司 Intelligent dynamic optimization and process scheduling system of automation equipment
CN117407179B (en) * 2023-12-15 2024-04-02 成都凯迪飞研科技有限责任公司 Acceleration sub-card processing method and system based on cooperative processing
CN117633636B (en) * 2024-01-25 2024-05-03 江苏省特种设备安全监督检验研究院 Cloud interconnected special detection data processing system and processing equipment thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007147166A2 (en) * 2006-06-16 2007-12-21 Quantum Leap Research, Inc. Consilence of data-mining
CN109639762A (en) * 2018-11-07 2019-04-16 重庆光电信息研究院有限公司 City Internet of Things information grading processing system and method
WO2020162943A1 (en) * 2019-02-07 2020-08-13 Thogamalai Krishnakumar Arumugham Holistic intelligence and autonomous information system and method thereof
CN116448419A (en) * 2023-03-14 2023-07-18 太原科技大学 Zero sample bearing fault diagnosis method based on depth model high-dimensional parameter multi-target efficient optimization

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007147166A2 (en) * 2006-06-16 2007-12-21 Quantum Leap Research, Inc. Consilence of data-mining
CN109639762A (en) * 2018-11-07 2019-04-16 重庆光电信息研究院有限公司 City Internet of Things information grading processing system and method
WO2020162943A1 (en) * 2019-02-07 2020-08-13 Thogamalai Krishnakumar Arumugham Holistic intelligence and autonomous information system and method thereof
CN116448419A (en) * 2023-03-14 2023-07-18 太原科技大学 Zero sample bearing fault diagnosis method based on depth model high-dimensional parameter multi-target efficient optimization

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
大数据时代的新型电力供需预测方法探究;钱语眉;;机电信息(35);全文 *
电子商务用户反馈文本数据的可视化研究与实现;何媛;陈亮;李丰妤;张志远;;包装工程(10);全文 *

Also Published As

Publication number Publication date
CN116680459A (en) 2023-09-01

Similar Documents

Publication Publication Date Title
CN116680459B (en) Foreign trade content data processing system based on AI technology
Haghighat et al. Intrusion detection system using voting-based neural network
Park et al. An enhanced AI-based network intrusion detection system using generative adversarial networks
US20220210190A1 (en) Systems and methods for intelligent phishing threat detection and phishing threat remediation in a cyber security threat detection and mitigation platform
WO2021159834A1 (en) Abnormal information processing node analysis method and apparatus, medium and electronic device
CN111327608A (en) Application layer malicious request detection method and system based on cascade deep neural network
WO2023246146A1 (en) Target security recognition method and apparatus based on optimization rule decision tree
CN115941322A (en) Attack detection method, device, equipment and storage medium based on artificial intelligence
Wang et al. An unknown protocol syntax analysis method based on convolutional neural network
CN113282920B (en) Log abnormality detection method, device, computer equipment and storage medium
Samaan et al. Feature-based real-time distributed denial of service detection in SDN using machine learning and Spark
US20210264033A1 (en) Dynamic Threat Actionability Determination and Control System
Sekar et al. Prediction of distributed denial of service attacks in SDN using machine learning techniques
CN111797997A (en) Network intrusion detection method, model construction method, device and electronic equipment
Ding et al. A data-driven based security situational awareness framework for power systems
CN113706207B (en) Order success rate analysis method, device, equipment and medium based on semantic analysis
CN113037555B (en) Risk event marking method, risk event marking device and electronic equipment
CN113127640A (en) Malicious spam comment attack identification method based on natural language processing
US11888895B2 (en) Using neural networks to process forensics and generate threat intelligence information
Hu et al. A robust IoT device identification method with unknown traffic detection
US11835989B1 (en) FPGA search in a cloud compute node
CN116775889B (en) Threat information automatic extraction method, system, equipment and storage medium based on natural language processing
Vulfin Detection of Network Attacks in a Heterogeneous Industrial Network Based on Machine Learning
US20230289227A1 (en) Multi-Computer System for Forecasting Data Surges
CN113688924B (en) Abnormal order detection method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant