CN117235243A - Training optimization method for large language model of civil airport and comprehensive service platform - Google Patents
Training optimization method for large language model of civil airport and comprehensive service platform Download PDFInfo
- Publication number
- CN117235243A CN117235243A CN202311524475.6A CN202311524475A CN117235243A CN 117235243 A CN117235243 A CN 117235243A CN 202311524475 A CN202311524475 A CN 202311524475A CN 117235243 A CN117235243 A CN 117235243A
- Authority
- CN
- China
- Prior art keywords
- language model
- large language
- airport
- civil
- civil airport
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000012549 training Methods 0.000 title claims abstract description 62
- 238000005457 optimization Methods 0.000 title claims description 24
- 230000004044 response Effects 0.000 claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000002360 preparation method Methods 0.000 claims abstract description 11
- 238000004891 communication Methods 0.000 claims abstract description 6
- 238000004806 packaging method and process Methods 0.000 claims abstract description 6
- 239000013598 vector Substances 0.000 claims description 79
- 239000011159 matrix material Substances 0.000 claims description 41
- 238000005516 engineering process Methods 0.000 claims description 31
- 230000008569 process Effects 0.000 claims description 24
- 238000001914 filtration Methods 0.000 claims description 22
- 230000011218 segmentation Effects 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000007667 floating Methods 0.000 claims description 12
- 230000007246 mechanism Effects 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000005538 encapsulation Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 3
- 238000013079 data visualisation Methods 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000013479 data entry Methods 0.000 claims description 2
- 238000003032 molecular docking Methods 0.000 claims description 2
- 238000012552 review Methods 0.000 abstract description 2
- 238000004590 computer program Methods 0.000 description 10
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 238000007689 inspection Methods 0.000 description 3
- 230000010006 flight Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 101150054987 ChAT gene Proteins 0.000 description 1
- 101100203187 Mus musculus Sh2d3c gene Proteins 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Machine Translation (AREA)
Abstract
The invention belongs to the technical field of large language model data processing, and discloses a training and optimizing method for a large language model of a civil airport and a comprehensive service platform. The method comprises the steps of performing data preparation and training on a large language model, constructing a civil airport large language model, and performing fine adjustment on the constructed civil airport large language model; service packaging is carried out on the constructed large language model of the civil airport, a comprehensive service platform is established, response service is provided for passengers and security personnel, and the problems and response contents are interacted with the passengers through a plurality of service contacts; and establishing association with the user terminal based on the established comprehensive service platform, and carrying out communication links of different user use requirements in different scenes. The invention covers the business knowledge of the whole flow of the airport flight guarantee, provides the services of rapid data review and intelligent expert treatment opinion for the staff, and further improves the airport flight guarantee efficiency.
Description
Technical Field
The invention belongs to the technical field of large language model data processing, and particularly relates to a training optimization method and a comprehensive service platform for a large language model of a civil airport.
Background
With the technical application of ChatGPT, the technical field of large language models is rapidly developed, internet manufacturers rapidly perform technical layout, and specific industry manufacturers also output special large models with industry characteristics, and related industries are continuously developed. Civil aviation is taken as an important component of comprehensive traffic, and as throughput is continuously increased and passenger service quality is continuously improved, relatively high pressure is brought to airport operation, and how to improve passenger service quality and production operation efficiency are the urgent problems to be solved by airport users.
The current large language model technology is in a rapid development period, the construction of a related technology system is in an exploration construction stage, most of base large language models are easy to generate word interpretation deviation in language translation, meanwhile, the general base large language models can well represent the general field, but concepts and nouns in the professional field cannot be well interpreted, so that the special large language models of civil airports in the prior art cannot provide accurate interpretation information for passengers and airport security personnel, the operation efficiency of the airports is affected to a certain extent, and the passengers are inconvenient to travel. Meanwhile, the large language model of the prior art has poor effect on identifying and processing the access and answering information of some airport services.
Disclosure of Invention
In order to overcome the problems in the related art, the embodiment of the invention discloses a training and optimizing method for a large language model of a civil airport and a comprehensive service platform, in particular relates to the fields of airport passenger service and production operation, and belongs to the field of important business of airports.
The technical scheme is as follows: a training optimization method for a large language model of a civil airport comprises the following steps:
s1, carrying out civil airport data preparation and training on a large language model, constructing a large language model of a civil airport, and carrying out fine adjustment on the constructed large language model of the civil airport; wherein, civil airport data includes: civil internet data, general internet data, civil aviation policy data, airport passenger service data and airport security personnel data;
s2, carrying out service encapsulation on the constructed large language model of the civil airport, establishing a comprehensive service platform, providing response service for passengers and security personnel, and carrying out problem and response content interaction with the passengers through a multi-service contact;
s3, based on the established comprehensive service platform, establishing association with different types of user terminals, and carrying out communication links of different users in different scenes; wherein, different types of user terminals include: weChat applet user terminal, intelligent navigation display screen user terminal, question-answering robot user terminal.
In step S1, civil airport data preparation is carried out on a large language model, and civil airport data acquisition is carried out through a plurality of acquisition technologies; the method specifically comprises the following steps:
obtaining text data by interfacing with a civil aviation information system;
by extracting from the operational database of the civil airport;
and (3) performing information capture on a system which cannot realize interface docking through a crawler software technology, and performing data entry on paper files archived by a business department of an airport terminal.
In step S1, civil airport data preparation for the large language model is performed before training: data cleaning, sentence level filtering, content duplication removal, text word segmentation and data vectorization;
the data cleansing includes: establishing a URL filtering blacklist, bringing forbidden websites, websites with low content quality, websites with low content correlation and websites with heavy charts and light texts into the blacklist, matching and checking the URL sources of articles collected on a network, deleting the content matched to the URL filtering blacklist, and simultaneously carrying out preliminary search and matching on the article content to delete the whole articles;
the sentence-level filtering includes: filtering and deleting sentences consisting of pure numbers;
The content deduplication comprises: converting text into a set for representation, converting a high-dimensional vector into a low-dimensional hash signature, calculating hash signature similarity, focusing on candidate hash signatures from similar documents, performing text deletion on similar content, and representing a text set matrix as followsThe specific distance calculation formula is set as follows:
；
in the method, in the process of the invention,representing a set matrix->Distance between->Representation pair->First row index value of 1 after scrambling every column, +.>Representation pair->A first row index value of 1 after each column is scrambled;
the text word segmentation includes: adopting unicode codes as minimum word segmentation granularity to process, avoiding the uncommon words, and establishing different types of data sets;
the data vectorization includes: an encoder and decoder architecture is adopted, a self-attention mechanism is input, and when the encoder encodes a specific word, information of other words in an input sentence is used for carrying out auxiliary operation; the decoder adds a coding and decoding attention layer on the basis of two layers of the coder, so that the decoder is assisted in focusing on the part needing to be focused in the input sentence.
Further, the input self-attention mechanism is a sequenceEach word is converted into a word vector through a word embedding algorithm and recorded as: / >Each->All pass through->Three matrix processing to obtain three vectors +.>The calculation formula is as follows:
；
；
；
in the method, in the process of the invention,for the input word vector, ++>Is a weight matrix>To match query vectors of other units, +.>For a key value vector matched by other units, +.>Is the extracted information vector; />For inputting the +.>Query vector of matching other units of the individual elements, +.>For inputting the +.>Key value vectors of individual elements that are matched by other units,for inputting the +.>The extracted information vector of the individual elements, +.>Is->Weights of query vectorsThe matrix is formed by a matrix of,is->Weight matrix of key value vector,>is->Extracting weight matrix of vector,>for inputting the +.>Element(s)>Is the first->An element;
the self-attention mechanism formula is:
；
wherein,is square matrix and is filled with->The result of inner product operation of the query vector of each input vector element matched with other units and the key value vector matched with other units of other input vector elements is stored in the memory, and the similarity degree between the query vector and other vectors is achieved; />Is->And->For reducing +.>For a word vector of the combination of the attention weight of the input vector element and the extracted information vector of the input vector element, a word vector of +. >Is the attention weight of the input vector element.
In the step S1, a civil airport large language model is built, firstly, unsupervised training is carried out through civil aviation professional knowledge of the Internet, and then, secondary training is carried out by using manually marked civil airport data to serve as correction; after correction, the construction of the civil airport large language model is completed, and the method specifically comprises the following steps:
a bypass is added beside the original base model to realize the operation of dimension reduction and dimension increase, the parameters of the base of the original large language model are fixed during training, and the dimension reduction matrix is initialized by random Gaussian distributionInitializing the up-dimension matrix with 0 matrix +.>The input and output dimensions of the original large language model are unchanged, and a dimension-reducing matrix is +.>And up-dimension matrix->Superposing the base parameters of the original large language model;
recording deviceIs the original base model initialization parameter, +.>Is to updateIs to be trained by the original large language model>All updates, updates in constructed civil airport big language model +.>The method comprises the steps of carrying out a first treatment on the surface of the The matrix of the original base model isThe constructed civil airport large language model is expressed as:
；
；
in the method, in the process of the invention,initializing parameters for the original base model, +.>For parameters that need to be updated +.>For the upword matrix, ++>For the dimension-reducing matrix >For the dimension of the additionally constructed bypass dimension-reducing neural network, comparing the dimension of the trainable layer of the base large model with the dimension of the base large model +.>Dimension reduction to dimension->，/>For the additionally structured bypass dimension-increasing neural network dimension, the trainable layer dimension r is increased to dimension k,/and->For neural network dimension, <' > for example>Trainable layer dimension for base large model, +.>For dimension reduction, let us go of>In order to be in the dimension of the dimension up, and (2)>Representing from->Selecting the minimum value;
after the civil airport large language model is built, knowledge implication capability effect evaluation is carried out, civil airport related business knowledge selection questions are built, different knowledge types are classified according to the selection question types, selection question data are written into the promt, the civil airport large language model is counted to output answers, all the answers are summarized, and the accuracy of the answers on different selection question types is counted.
In step S1, fine tuning is performed on the constructed large language model of the civil airport, including: performing instruction alignment to form an instruction set, performing supervised learning on a large language model of the civil airport, and performing user response; the instruction set includes: instruction, input and output.
Further, performing instruction alignment includes: converting the calculated floating point number into 8bit number, and recording the upper floating point number as By a scaling factor->Mapping to a range +.>Inner->Representation->The expression is:
；
in the method, in the process of the invention,meaning rounded to integer, +.>Representing truncation of outliers to +.>Within the range of>Is within [ -128,127]The 8 bits in the inner output integer, < >>To input floating point numbers, < >>Is a scaling factor;
for the followingIs calculated by the following formula:
；
；
in the method, in the process of the invention,to solve for the maximum in the floating point vector, +.>To calculate->Absolute value.
The invention further aims to provide a comprehensive service platform which is realized by applying the training and optimizing method of the large language model of the civil airport, and the comprehensive service platform finally provides service access service of the large language model of the civil airport for passengers and airport security personnel by carrying out service packaging on the large language model of the civil airport.
Further, the comprehensive service platform is sequentially connected with a key technical layer, a civil airport large language model layer, a technical support layer, a functional service layer and a user end layer from bottom to top through communication links.
Further, the key technology layer utilizes the carried neural network, image recognition, robot interaction, GIS, data visualization and data acquisition integration to implement corresponding functions;
the civil airport large language model layer comprises a large language model and a large language model training optimization module, the civil airport large language model performs parallelization operation by utilizing each CPU core through a distributed acceleration technology, and simultaneously performs continuous monitoring by monitoring the operation real-time state through the state of the large language model so as to prevent the large language model from falling into an abnormal state; the training optimization of the large language model comprises pre-training, parameter optimization and instruction fine adjustment, and continuous optimization of the large language model of the civil airport is carried out according to the requirements;
Technical support layer: providing a packaging foundation for the access of a civil airport large language model;
functional service layer: providing service access for passengers and airport security personnel, and providing passenger response access service and security personnel response access service;
user side layer: different service contacts are provided for users, and the service contacts comprise self-service equipment, mobile APP, an applet, a webpage and an API.
By combining all the technical schemes, the invention has the advantages and positive effects that: according to the invention, a large language model technology is adopted to provide intelligent customer service response service for passengers, solutions related to travel are provided for the passengers at contacts such as mobile APP, applet and self-service terminal, and the service experience of the passengers is improved; meanwhile, a large language model technology is adopted, so that more flexible knowledge base learning, difficult problem solving and other services are provided for production security personnel, and the airport production operation efficiency is improved.
The invention is positioned to construct a special large language model of a civil airport, and simultaneously establishes a complete service technology system for passengers and airport security personnel. According to the invention, a large language model of a civil airport is constructed by adopting a large language model technology, service packaging is performed, response service is provided for passengers and security personnel, interaction is performed with the passengers through a plurality of service contacts, the problems of the passengers in a whole travel scene are intelligently solved, association is established with an airport related system, related service is automatically triggered, and comprehensive and efficient service experience is provided for users. The model covers the business knowledge of the whole flow of the airport flight guarantee, provides the services of rapid data review and intelligent expert treatment opinion for the staff, and further improves the airport flight guarantee efficiency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure;
FIG. 1 is a flowchart of a training optimization method for a large language model of a civil airport provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a large language model of a civil airport, which is specific to the civil airport, provided by the embodiment of the invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit or scope of the invention, which is therefore not limited to the specific embodiments disclosed below.
Embodiment 1 as shown in fig. 1, a training optimization method for a large language model of a civil airport includes:
s1, carrying out civil airport data preparation and training on a large language model, constructing a large language model of a civil airport, and carrying out fine adjustment on the constructed large language model of the civil airport;
The civil airport data comprise Internet civil airport data, internet general data and civil aviation policy data, airport passenger service data and airport security personnel data;
it is understood that Large Language Models (LLMs) are common names in the art; the preparation of civil airport data for the large language model comprises the steps of acquiring the civil airport data by adopting a plurality of acquisition technologies; the method specifically comprises the following steps:
obtaining text data by interfacing with a civil aviation information system; by extracting from the operational database of the civil airport; information grabbing is carried out on a system which cannot realize interface butt joint through crawler software; the method comprises the steps of manually combing a large amount of paper files archived by a business department of airport navigation to input data;
after preparing civil airport data for the large language model, training the large language model of the civil airport, wherein the building of the large language model of the civil airport comprises the following steps:
firstly, carrying out unsupervised training through mass civil aviation related professional knowledge of the Internet, and then carrying out secondary training on civil airport data marked manually after training to serve as correction; and (5) after correction, building a large language model of the civil airport.
The distinction between large language models in the broad sense is therefore that the field of knowledge of the invention is purely civil aviation, and therefore the invention is defined as a civil airport large language model.
S2, carrying out service encapsulation on the constructed large language model of the civil airport, establishing a comprehensive service platform, providing response service for passengers and security personnel, and carrying out problem and response content interaction with the passengers through a multi-service contact;
and S3, establishing association with different types of user terminals based on the established comprehensive service platform, and carrying out communication links of different scene use requirements of different users. The different types of user terminals comprise a WeChat applet user terminal, a intelligent navigation display screen user terminal and a question-answering robot user terminal.
In step S1, data preparation for a large language model includes: acquiring text data related to civil airport service by adopting a plurality of acquisition technologies, wherein the text data related to the civil airport service comprises:
the system comprises Internet civil airport data, internet general data, civil aviation policy data, airport related passenger service data and airport related security personnel related data.
In step S1, before training the large language model, it is necessary to: data cleaning, sentence level filtering, content duplication removal, text word segmentation and data vectorization;
the data cleansing includes: establishing a URL filtering blacklist, bringing forbidden websites, websites with low content quality, websites with low content correlation and websites with heavy charts and light texts into the blacklist, matching and checking the URL sources of articles collected on a network, deleting the content matched to the URL filtering blacklist, and simultaneously carrying out preliminary search and matching on the article content to delete the whole articles;
The filtering rules of sentence-level filtering include: filtering and deleting sentences consisting of pure numbers;
the content deduplication converts text into a set representation, converts a high-dimensional vector into a low-dimensional hash signature, calculates hash signature similarity, focuses on a stack of candidate hash signatures from similar documents, performs text deletion on similar content, and specifically assumes a text set matrixThe specific distance calculation formula is set as follows:
；
wherein the method comprises the steps of，Representation pair->First row index value of 1 after scrambling every column, +.>Representation pair->First row index value of 1 after scrambling every column, +.>Representing a set matrix->A distance therebetween;
meanwhile, by utilizing manual examination and check, relevant data which are directly oriented to passengers and security personnel are examined, and errors, inconsistencies and missing values in the relevant data are removed; in the manual examination, a three-level examination system is established to carry out comprehensive examination, key contents are subjected to key examination, and the questioning contents are judged whether to be removed or not;
the text word segmentation adopts unicode coding as minimum word segmentation granularity to process and avoid the uncommon words; after text word segmentation, establishing different types of data sets;
The data vectorization includes: an encoder and decoder architecture is adopted, a self-attention mechanism is input, and when the encoder encodes a specific word, information of other words in an input sentence is used for carrying out auxiliary operation; the decoder adds a coding and decoding attention layer on the basis of two layers of the coder, so that the decoder is assisted in focusing on the part needing to be focused in the input sentence.
It can be understood that the key service pain point of the direct civil aviation provided by the invention takes the new technology as a fulcrum, improves the service quality and the guarantee efficiency of passengers, provides a construction thought for digital transformation of the civil aviation airport, and contributes to the development of the civil aviation. By constructing a comprehensive service platform taking a large language model as a base, a new service form is provided for the travel service of the passengers, and the passenger travel quality is further improved through fine access and differentiated service. Through the form of multi-contact service, a channel for supporting multiple services is provided for passengers, the travel requirements of different types of passengers are met, and passengers can travel better, so that the local travel and business service quality is improved, and more relevant industries are driven to increase. Through the special large language model of civil aviation, provide the work helper for airport staff, build stronger data and consult and intelligent expert suggestion handling ability fast, provide more help for staff's business development, improve work development efficiency and unusual business scenario's quick handling ability to further promote airport overall operation efficiency, with new technology for airport operation optimization provides new thinking.
The civil aviation large language model AI engine of the invention adopts a Large Language Model (LLMs) technology to decompose and refine the passenger whole-flow service scene and the flight guarantee whole-flow scene, focus on the use of pain points by users, and collect and sort the passenger service and operation guarantee related data in the airport service field, so that the invention can solve the task and application with wide range, and the previous research has not been related to the field.
Example 2 as another implementation manner of the present embodiment, the training optimization method for a large language model of a civil airport provided by the present embodiment includes: the civil airport large language model training construction process and the comprehensive service platform built around the civil airport large language model, wherein the civil airport large language model construction comprises the steps of data preparation, model training, model fine tuning and the like. The method comprises the following steps:
s101, data preparation, wherein large language model training is required to rely on a large amount of text data, and the invention comprehensively adopts various acquisition technologies to acquire text data related to civil airport business, and specifically comprises the following steps:
the Internet civil airport data specifically comprises a civil aviation office network, a civil aviation resource network, an air traffic network, a North China regional administration office network, a south China regional administration office network, a southwest regional administration office network, a North China regional administration office network, a Xinjiang administration office network, an air traffic administration office network, a civil aviation university office network, a civil aviation department college office network, a civil aviation related WeChat public number and the like, and the data is captured and stored by adopting a web crawler technology.
The internet general data is searched on the platforms of microblogs, hundred degrees, weChat search, necessary, red books and the like through keywords such as civil aviation, airports, passengers, flights, apron, ferrying vehicles, passenger complaints, passenger boarding vehicles, aircrafts, corridor bridges, check-in locations, boarding gates, check-in, boarding, duty and the like, and the searched content is grabbed and stored.
And the civil aviation policy data is used for downloading and collecting relevant files such as relevant laws and regulations of civil aviation, normative files, standard specifications, conventions, official documents and the like.
The airport related passenger service data are collected and arranged, and texts are extracted and stored in the way of airport information such as brief introduction, position, conventional traffic mode, surrounding hotel information, control area shop information, terminal building navigation information, passenger service flow information, passenger abnormal service regulations and the like.
The airport related security personnel related data are used for collecting and sorting airport related department responsibilities, post operation regulations, business forms, key business processes, exception handling, machine maintenance, airport basic resource information, airport operation other basic data information and the like, and extracting texts for storage.
S102, after data are collected, data are cleaned, a URL filtering blacklist is firstly established, forbidden websites, websites with low content quality, websites with low content correlation, websites with heavy charts and light texts and the like are brought into the blacklist, matching check is carried out on the URL sources of articles collected on the network, and the content matched to the URL filtering blacklist is deleted. And meanwhile, carrying out preliminary search matching on the article content to delete the whole article, which specifically comprises the following steps:
(1) The invention adopts the finite automaton related technology to realize the filtering of forbidden words and sensitive words, the technology has a finite state set and a plurality of edges leading from one state to the other, each edge is marked with a symbol, one state is an initial state, and some states are final states, and compared with each article, the article is searched item by item in a huge amount of forbidden words and sensitive word libraries, so the searching of the forbidden words and sensitive words is very rapid, and the forbidden words and the sensitive word patterns are as follows:
{
"Wide": {
"report": {
"push": {
"Wide": {
"is_end": True
},
"is_end": False
},
"sensitive": {
"feel": {
"word": {
"is_end": True
},
"is_end": False
},
"is_end": False
},
"is_end": False
},
"is_end": False
},
"people": {
"navigation": {
"special": {
"color": {
"is_end": True
},
"is_end": False
},
"is_end": False
},
"is_end": False
}
}
(2) The method comprises the steps of stepping on and dislike articles, customizing and grabbing different websites according to the condition that user evaluation articles are good or bad in specific websites, and deleting articles with lower user evaluation.
(3) Articles with more charts and very few text contents are identified by HTML tags, and the articles are deleted.
(4) Articles that capture erroneous information, such as those that include capture timeout, deny access, etc., will be deleted.
(5) Articles with very few concept words related to civil airports, for example, the articles hardly contain phrases such as civil aviation, airports, passengers, flights, apron, ferry, passenger complaints, passenger boarding vehicles, aircraft, corridor bridges, boarding positions, boarding gates, check-in, security check-in, boarding and check-out, and the like, and the articles are considered to have weak relevance to the civil airports and are deleted.
S103, after deleting the whole article, sentence level filtering is carried out, and the filtering rules are as follows:
Sentences composed of pure numbers, which are usually meaningless, need to be filtered out.
Sentences containing keywords such as "focus", "forward", "praise" and the like are filtered and deleted.
Keywords including "or end of sentence" expansion "," more ", etc., are filtered and deleted.
S104, after filtering the articles and sentences, performing content deduplication, performing MinHash value calculation on the filtered articles by using a MinHash method, wherein each article uses 9000 Hash values (20 barrels and 450 values for each barrel) and uses a deterministic deduplication method, and deleting articles with repeated fragments exceeding 50 token.
Meanwhile, by means of manual examination and check, related data which directly face passengers and guarantee personnel are carefully examined, errors, inconsistencies and missing values in the related data are removed, and in order to ensure the quality of manual check, a three-level examination system is established.
S105, after the data processing is completed, text word segmentation is carried out, unicode coding is adopted as the minimum word segmentation granularity, so that quick processing is facilitated, rarely used words and the like can be avoided, for example, a word to be processed can be decomposed into 0xE5, 0xBE and 0x85, and specific word segmentation is carried out according to the number of Chinese character unicode code word segments; in order to effectively reduce the difficulty of model training and reduce the problem of messy codes in model training, vocabulary expansion operation is carried out on Chinese characters which are easy to generate messy codes and have different byte digits of other Chinese characters, and the Chinese characters are counted into a specific character table, wherein the index numerical range is more than or equal to 143859.
For effectively managing the data after the processing is finished, the subsequent expansion and maintenance are convenient, meanwhile, the influence of the size of the data set on the model in the training process is relieved, and different types of data sets are established. In particular table 1 below.
Table 1 sets of data of different types constructed
| Numbering device | Data collection | Sampling ratio | Training times |
| 1 | Internet airport data | 30% | 1 |
| 2 | Internet retrieval data | 20% | 1 |
| 3 | Civil aviation policy data | 10% | 2 |
| 4 | Passenger service data | 20% | 3 |
| 5 | Airport security data | 20% | 3 |
。
S106, vectorizing data before training, dividing long articles according to seq_len (2048), and transmitting the cut vectors to a model for training; as shown in fig. 2, the large language model of civil airport of the invention adopts encoder and decoder architecture, and the input first flows into the self-attention mechanism layer, so that the encoder can use the information of other words in the input sentence when encoding specific words; the decoder adds a codec attention layer on the basis of two layers of the encoder to help the decoder focus on the part of the input sentence that needs to be focused.
The input self-attention mechanism is a sequenceConverting each word into a word vector through a word embedding algorithm, and recording the word vector as +.>Then each->All pass through->Three matrix processes to obtain three vectorsThe calculation formula is as follows:
；
；
；
in the method, in the process of the invention,for the input word vector, ++>Is a weight matrix>To match query vectors of other units, +.>For a key value vector matched by other units, +.>Is the extracted information vector; />For inputting the +.>Query vector of matching other units of the individual elements, +.>For inputting the +.>Key value vectors of individual elements that are matched by other units,for inputting the +.>The extracted information vector of the individual elements, +.>Is->The weight matrix of the vector is queried,is->Weight matrix of key value vector,>is->Extracting weight matrix of vector,>for inputting the +.>Element(s)>Is the first->An element;
the self-attention mechanism formula is:
；
wherein,is square matrix and is filled with->The result of inner product operation of the query vector of each input vector element matched with other units and the key value vector matched with other units of other input vector elements is stored in the memory, and the similarity degree between the query vector and other vectors is achieved; / >Is->And->For reducing +.>For a word vector of the combination of the attention weight of the input vector element and the extracted information vector of the input vector element, a word vector of +.>Is the attention weight of the input vector element.
S107, training the large language model of the civil airport, adding a bypass beside the original base model to perform a dimension-reducing and dimension-increasing operation, fixing the parameters of the base of the original large language model during training, and assuming a dimension-reducing matrix to reduce the training cost of the large language model of the civil airportAnd up-dimension matrix->Initializing +.>Initializing +.0 matrix>The input/output dimension of the original large language model is unchanged, and the output is performed by +.>Superposing the base parameters of the original large language model;
recording deviceIs the original base model initialization parameter, +.>Is a parameter to be updated, and the original large language model is trained by +.>All updates are needed to be updated in the constructed large language model of the civil airport>Assume that the matrix of the original base model is +.>The constructed civil airport large language model is expressed as:
；
wherein the rank is；
In the method, in the process of the invention,initializing parameters for the original base model, +.>For parameters that need to be updated +. >For the upword matrix, ++>For the dimension-reducing matrix>For the dimension of the additionally constructed bypass dimension-reducing neural network, comparing the dimension of the trainable layer of the base large model with the dimension of the base large model +.>Dimension reduction to dimension->，/>For the additionally structured bypass dimension-increasing neural network dimension, the trainable layer dimension r is increased to dimension k,/and->For neural network dimension, <' > for example>Trainable layer dimension for base large model, +.>For dimension reduction, let us go of>In order to be in the dimension of the dimension up, and (2)>Representing from->Selecting the minimum value;
it can be appreciated that in order to reduce the pre-training cost, the original base model can use the large model with an open source as the base to perform bypass fine tuning, so that the training cost is saved.
S108, after training of the large language model of the civil airport is completed, knowledge accumulation capability effect evaluation is carried out, a huge number of knowledge selection questions of related business of the civil airport are constructed manually, different knowledge types are classified according to the types of the selection questions, selection question data are written into a prompt, the large language model of the civil airport is counted to output answers, all the answers are summarized, the accuracy of the counted answers on different selection question types is evaluated objectively, the current level of the large language model of the civil airport is evaluated, and the related knowledge data collection, processing and training are carried out on the performance level of different types of knowledge based on the large language model of the civil airport, so that the large language model of the civil airport can meet the actual use requirements.
S109, next, instruction fine adjustment is carried out on the large language model of the civil airport, instruction alignment is needed, an instruction set is formed in a manual labeling mode, and supervision learning is carried out on the large language model of the civil airport, so that the large language model of the civil airport can better carry out user response. The instruction set includes instructions, inputs, outputs, and specific examples are as follows:
sample 1:
instructions to: when the passenger goes out inside the terminal building.
Input: the security check is carried out on the position of the channel.
And (3) outputting: the international passenger security inspection channel is positioned at 3F and at the left side of the check-in island, and is combined with units such as customs, side inspection and the like; the domestic passenger security inspection channel is positioned at 4F and is positioned right in front of the F value machine island.
Sample 2:
instructions to: the aircraft connecting personnel are connected with the aircraft.
Input: and acquiring 737-800 machine connection operation rules.
And (3) outputting: after the aircraft is seen by the aircraft receiving personnel, the aircraft receiving personnel can tell whether the aircraft at the airport correctly runs on the sliding line through the left and right swing arms, and after the aircraft reaches the corresponding front wheel stop line of the aircraft position, the aircraft receiving personnel can make a stop gesture, and after the aircraft is flameout, the wheel gear and the conical barrel are placed, so that the completion is ensured.
In order to improve the reasoning speed of the large language model of the civil airport, the invention converts the calculated floating point number into 8bit number, and the floating point number is recorded as By a scaling factor->Mapping to a range +.>Inner->Representation->The method comprises the following steps:
；
in the method, in the process of the invention,meaning rounded to integer, +.>Representing truncation of outliers to +.>Within the range of>Is within [ -128,127]The 8 bits in the inner output integer, < >>To input floating point numbers, < >>Is a scaling factor; for the followingIs calculated by the following formula:
；
；
in the method, in the process of the invention,to solve for the maximum in the floating point vector, +.>To calculate->Absolute value.
The training and optimizing of the large language model of the civil airport are completed.
S110, the invention constructs a comprehensive service platform surrounding the large language model, and the platform finally provides the service access service of the large language model of the civil airport for passengers and airport security personnel by carrying out service encapsulation on the large language model of the civil airport. The comprehensive service platform comprises a key technical layer, a civil airport large language model layer (civil airport large language model layer), a technical support layer, a functional service layer and a user side layer, and is specifically as follows:
key technical layer: the layer contains key technologies required by the platform, such as a neural network for providing a civil airport large language model bottom foundation; the image recognition is used for the face recognition of the passengers in the control area and is used for recognizing the identity of the user; the robot interaction technology is used for chat interaction between different terminal users and a large language model of a civil airport, so that inquiry response can be carried out more smoothly; GIS technology provides a basic technical foundation for passenger indoor navigation; the data visualization technology is used for service access analysis and is used for visually analyzing service access conditions; the data acquisition integration technology is used for implementing service operation data for acquiring other service systems of the airport and providing comprehensive dynamic inquiry for passengers; the video processing is used for capturing the terminal passenger image at the server. The technology forms the basis of the comprehensive service platform.
Civil airport large language model layer: the layer comprises a large language model and a large language model training optimization module, wherein the large language model of the civil airport fully utilizes each CPU core through a distributed acceleration technology to operate in parallel as much as possible, and simultaneously continuously monitors the operation real-time state of the large language model of the civil airport through the state monitoring of the large language model of the civil airport to prevent the large language model of the civil airport from being in an abnormal state; the training and optimizing of the civil airport large language model comprises specific functions of pre-training, parameter optimization, instruction fine adjustment and the like, and continuous optimization of the civil airport large language model can be carried out according to the needs.
Technical support layer: the layer provides a packaging foundation for the large language model access of the civil airport, for example, the model access control provides fine granularity authority identification, so that the large language model of the civil airport is prevented from being excessively accessed; the request queue carries out service flow limiting according to the access limit of the large language model of the civil airport, so that the condition that the large language model of the civil airport is accessed to too many requests at one time to cause service unavailability is prevented; the problem response screening establishes a problem response content screening through a sensitive word library, cuts off the content which does not meet the requirements, and prevents further influence; the real-time channel provides a bottom layer real-time stream request response technology, can continuously acquire the output byte stream of the large language model of the civil airport, and responds to the user demand in real time.
Functional service layer: the layer provides service access for passengers and airport security personnel, passenger response access service and security personnel response access service provide final civil airport large language model access service package, response evaluation provides a function of evaluating response contents for users, feedback of low quality of the user feedback response contents is recorded in the last day, manual marking is carried out on questions and the response contents again, and subsequent instruction fine adjustment is carried out on the civil airport large language model so as to ensure that better effects are achieved.
User side layer: the layer provides different service contacts for users, specifically comprises forms of self-service equipment, mobile APP, applet, webpage, API and the like, and can meet the use requirements of different scenes of different users.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
The content of the information interaction and the execution process between the devices/units and the like is based on the same conception as the method embodiment of the present invention, and specific functions and technical effects brought by the content can be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. For specific working processes of the units and modules in the system, reference may be made to corresponding processes in the foregoing method embodiments.
According to an embodiment of the present application, there is also provided a computer apparatus including: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, which when executed by the processor performs the steps of any of the various method embodiments described above.
Embodiments of the present application also provide a computer readable storage medium storing a computer program which, when executed by a processor, performs the steps of the respective method embodiments described above.
The embodiment of the application also provides an information data processing terminal, which is used for providing a user input interface to implement the steps in the method embodiments when being implemented on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer and a switch.
The embodiment of the application also provides a server, which is used for realizing the steps in the method embodiments when being executed on the electronic device and providing a user input interface.
Embodiments of the present application also provide a computer program product which, when run on an electronic device, causes the electronic device to perform the steps of the method embodiments described above.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc.
While the invention has been described with respect to what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention.
Claims (10)
1. A training and optimizing method for a large language model of a civil airport is characterized by comprising the following steps:
s1, carrying out civil airport data preparation and training on a large language model, constructing a large language model of a civil airport, and carrying out fine adjustment on the constructed large language model of the civil airport; wherein, civil airport data includes: civil internet data, general internet data, civil aviation policy data, airport passenger service data and airport security personnel data;
s2, carrying out service encapsulation on the constructed large language model of the civil airport, establishing a comprehensive service platform, providing response service for passengers and security personnel, and carrying out problem and response content interaction with the passengers through a multi-service contact;
s3, based on the established comprehensive service platform, establishing association with different types of user terminals, and carrying out communication links of different users in different scenes; wherein, different types of user terminals include: weChat applet user terminal, intelligent navigation display screen user terminal, question-answering robot user terminal.
2. The method for training and optimizing a large language model of a civil airport according to claim 1, wherein in step S1, the large language model is subjected to civil airport data preparation for acquiring the civil airport data by a plurality of acquisition technologies; the method specifically comprises the following steps:
obtaining text data by interfacing with a civil aviation information system;
by extracting from the operational database of the civil airport;
and (3) performing information capture on a system which cannot realize interface docking through a crawler software technology, and performing data entry on paper files archived by a business department of an airport terminal.
3. The method for training and optimizing large language model of civil airport according to claim 1, wherein in step S1, the large language model is prepared for civil airport data and is performed before training: data cleaning, sentence level filtering, content duplication removal, text word segmentation and data vectorization;
the data cleansing includes: establishing a URL filtering blacklist, bringing forbidden websites, websites with low content quality, websites with low content correlation and websites with heavy charts and light texts into the blacklist, matching and checking the URL sources of articles collected on a network, deleting the content matched to the URL filtering blacklist, and simultaneously carrying out preliminary search and matching on the article content to delete the whole articles;
The sentence-level filtering includes: filtering and deleting sentences consisting of pure numbers;
the content deduplication comprises: converting text into a set for representation, converting a high-dimensional vector into a low-dimensional hash signature, calculating hash signature similarity, focusing on candidate hash signatures from similar documents, performing text deletion on similar content, and representing a text set matrix as followsThe specific distance calculation formula is set as follows:
；
in the method, in the process of the invention,representing a set matrix->Distance between->Representation pair->First row index value of 1 after scrambling every column, +.>Representation pair->A first row index value of 1 after each column is scrambled;
the text word segmentation includes: adopting unicode codes as minimum word segmentation granularity to process, avoiding the uncommon words, and establishing different types of data sets;
the data vectorization includes: an encoder and decoder architecture is adopted, a self-attention mechanism is input, and when the encoder encodes a specific word, information of other words in an input sentence is used for carrying out auxiliary operation; the decoder adds a coding and decoding attention layer on the basis of two layers of the coder, so that the decoder is assisted in focusing on the part needing to be focused in the input sentence.
4. A civil airport large language model training optimization method of claim 3, wherein the input self-attention mechanism is a sequenceEach word is converted into a word vector through a word embedding algorithm and recorded as:each->All pass through->Three matrix processing to obtain three vectors +.>The calculation formula is as follows:
；
；
；
in the method, in the process of the invention,for the input word vector, ++>Is a weight matrix>To match query vectors of other units, +.>For a key value vector matched by other units, +.>Is the extracted information vector; />For inputting the +.>Query vector of matching other units of the individual elements, +.>For inputting the +.>Key value vector of the individual element matched by other units,/->For inputting the +.>The extracted information vector of the individual elements, +.>Is->Query the weight matrix of the vector,>is thatWeight matrix of key value vector,>is->Extracting weight matrix of vector,>for inputting the +.>Element(s)>Is the first->An element;
the self-attention mechanism formula is:
；
wherein,is square matrix and is filled with->The result of inner product operation of the query vector of each input vector element matched with other units and the key value vector matched with other units of other input vector elements is stored in the memory, and the similarity degree between the query vector and other vectors is achieved; / >Is->And->For reducing +.>For a word vector of the combination of the attention weight of the input vector element and the extracted information vector of the input vector element, a word vector of +.>Is the attention weight of the input vector element.
5. The civil airport large language model training optimization method according to claim 1, wherein in step S1, the civil airport large language model is constructed by performing unsupervised training through civil aviation expertise of the internet, and then performing secondary training with manually marked civil airport data as correction; after correction, the construction of the civil airport large language model is completed, and the method specifically comprises the following steps:
a bypass is added beside the original base model to realize the operation of dimension reduction and dimension increase, the parameters of the base of the original large language model are fixed during training, and the dimension reduction matrix is initialized by random Gaussian distributionInitializing the up-dimension matrix with 0 matrix +.>The input and output dimensions of the original large language model are unchanged, and a dimension-reducing matrix is +.>And up-dimension matrix->Superposing the base parameters of the original large language model;
recording deviceIs the original base model initialization parameter, +.>Is a parameter to be updated, and the original large language model training will All updates, updates in constructed civil airport big language model +.>The method comprises the steps of carrying out a first treatment on the surface of the The matrix of the original base model isThe constructed civil airport large language model is expressed as:
；
；
in the method, in the process of the invention,initializing parameters for the original base model, +.>For parameters that need to be updated +.>For the upword matrix, ++>For the dimension-reducing matrix>For the dimension of the additionally constructed bypass dimension-reducing neural network, comparing the dimension of the trainable layer of the base large model with the dimension of the base large model +.>Dimension reduction to dimension->，/>For the additionally structured bypass dimension-increasing neural network dimension, the trainable layer dimension r is increased to dimension k,/and->For neural network dimension, <' > for example>Trainable layer dimension for base large model, +.>For dimension reduction, let us go of>In order to be in the dimension of the dimension up,representing from->Selecting the minimum value;
after the civil airport large language model is built, knowledge implication capability effect evaluation is carried out, civil airport related business knowledge selection questions are built, different knowledge types are classified according to the selection question types, selection question data are written into the promt, the civil airport large language model is counted to output answers, all the answers are summarized, and the accuracy of the answers on different selection question types is counted.
6. The training optimization method for large language model of civil airport according to claim 1, wherein in step S1, fine tuning is performed on the constructed large language model of civil airport, comprising: performing instruction alignment to form an instruction set, performing supervised learning on a large language model of the civil airport, and performing user response; the instruction set includes: instruction, input and output.
7. The method of training optimization of a large language model of a civil airport of claim 6, wherein performing instruction alignment comprises: converting the calculated floating point number into 8bit number, and recording the upper floating point number asBy a scaling factor->Mapping to a range +.>Inner->Representation->The expression is:
；
in the method, in the process of the invention,meaning rounded to integer, +.>Representing truncation of outliers to +.>Within the scope of this invention,is within [ -128,127]The 8 bits in the inner output integer, < >>To input floating point numbers, < >>Is a scaling factor;
for the followingIs calculated by the following formula:
；
；
in the method, in the process of the invention,to solve for the maximum in the floating point vector, +.>To calculate->Absolute value.
8. The comprehensive service platform is characterized by being realized by applying the training optimization method of the large language model of the civil airport according to any one of claims 1-7, and finally provides service access service of the large language model of the civil airport for passengers and airport security personnel by carrying out service encapsulation on the large language model of the civil airport.
9. The integrated service platform of claim 8, wherein the integrated service platform has a key technology layer, a civil airport large language model layer, a technology support layer, a functional service layer, and a user side layer sequentially connected from bottom to top through a communication link.
10. The integrated service platform of claim 9, wherein the key technology layer implements corresponding functions using a onboard neural network, image recognition, robot interaction, GIS, data visualization, data collection integration;
the civil airport large language model layer comprises a large language model and a large language model training optimization module, the civil airport large language model performs parallelization operation by utilizing each CPU core through a distributed acceleration technology, and simultaneously performs continuous monitoring by monitoring the operation real-time state through the state of the large language model so as to prevent the large language model from falling into an abnormal state; the training optimization of the large language model comprises pre-training, parameter optimization and instruction fine adjustment, and continuous optimization of the large language model of the civil airport is carried out according to the requirements;
technical support layer: providing a packaging foundation for the access of a civil airport large language model;
functional service layer: providing service access for passengers and airport security personnel, and providing passenger response access service and security personnel response access service;
user side layer: different service contacts are provided for users, and the service contacts comprise self-service equipment, mobile APP, an applet, a webpage and an API.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311524475.6A CN117235243A (en) | 2023-11-16 | 2023-11-16 | Training optimization method for large language model of civil airport and comprehensive service platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311524475.6A CN117235243A (en) | 2023-11-16 | 2023-11-16 | Training optimization method for large language model of civil airport and comprehensive service platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117235243A true CN117235243A (en) | 2023-12-15 |
Family
ID=89084834
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311524475.6A Pending CN117235243A (en) | 2023-11-16 | 2023-11-16 | Training optimization method for large language model of civil airport and comprehensive service platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117235243A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117475678A (en) * | 2023-12-26 | 2024-01-30 | 青岛民航凯亚系统集成有限公司 | Airport flight area rule monitoring method and system based on large model |
| CN117709909A (en) * | 2024-02-06 | 2024-03-15 | 绵阳师范学院 | Business data processing method based on large language model |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595517A (en) * | 2018-03-26 | 2018-09-28 | 南京邮电大学 | A kind of extensive document similarity detection method |
| CN110442684A (en) * | 2019-08-14 | 2019-11-12 | 山东大学 | A kind of class case recommended method based on content of text |
| CN111104492A (en) * | 2019-09-19 | 2020-05-05 | 中国民航信息网络股份有限公司 | Hierarchical Attention mechanism-based automatic question-answering method in civil aviation field |
| WO2021243706A1 (en) * | 2020-06-05 | 2021-12-09 | 中山大学 | Method and apparatus for cross-language question generation |
| CN116385906A (en) * | 2023-03-10 | 2023-07-04 | 西南交通大学 | Side slope disaster light sensing system, equipment and platform based on deep learning |
| CN116628172A (en) * | 2023-07-24 | 2023-08-22 | 北京酷维在线科技有限公司 | Dialogue method for multi-strategy fusion in government service field based on knowledge graph |
| CN116842152A (en) * | 2023-06-09 | 2023-10-03 | 福建省科学技术信息研究所（福建省生产力促进中心） | Science and technology policy question-answering method and device for fine-tuning language big model |
-
2023
- 2023-11-16 CN CN202311524475.6A patent/CN117235243A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595517A (en) * | 2018-03-26 | 2018-09-28 | 南京邮电大学 | A kind of extensive document similarity detection method |
| CN110442684A (en) * | 2019-08-14 | 2019-11-12 | 山东大学 | A kind of class case recommended method based on content of text |
| CN111104492A (en) * | 2019-09-19 | 2020-05-05 | 中国民航信息网络股份有限公司 | Hierarchical Attention mechanism-based automatic question-answering method in civil aviation field |
| WO2021243706A1 (en) * | 2020-06-05 | 2021-12-09 | 中山大学 | Method and apparatus for cross-language question generation |
| CN116385906A (en) * | 2023-03-10 | 2023-07-04 | 西南交通大学 | Side slope disaster light sensing system, equipment and platform based on deep learning |
| CN116842152A (en) * | 2023-06-09 | 2023-10-03 | 福建省科学技术信息研究所（福建省生产力促进中心） | Science and technology policy question-answering method and device for fine-tuning language big model |
| CN116628172A (en) * | 2023-07-24 | 2023-08-22 | 北京酷维在线科技有限公司 | Dialogue method for multi-strategy fusion in government service field based on knowledge graph |
Non-Patent Citations (2)
| Title |
|---|
| 杨曦等: "《多模态数据融合与检索技术》", 30 June 2021, pages: 116 * |
| 申蕙;韩田;: "基于顾客感知的民用机场旅客服务水平评价", 首都经济贸易大学学报, no. 06, pages 122 - 123 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117475678A (en) * | 2023-12-26 | 2024-01-30 | 青岛民航凯亚系统集成有限公司 | Airport flight area rule monitoring method and system based on large model |
| CN117709909A (en) * | 2024-02-06 | 2024-03-15 | 绵阳师范学院 | Business data processing method based on large language model |
| CN117709909B (en) * | 2024-02-06 | 2024-04-09 | 绵阳师范学院 | Business data processing method based on large language model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110334178B (en) | Data retrieval method, device, equipment and readable storage medium | |
| CN110347894A (en) | Knowledge mapping processing method, device, computer equipment and storage medium based on crawler | |
| CN111783394B (en) | Training method of event extraction model, event extraction method, system and equipment | |
| CN117235243A (en) | Training optimization method for large language model of civil airport and comprehensive service platform | |
| CN107705839A (en) | Disease automatic coding and system | |
| CN113011189A (en) | Method, device and equipment for extracting open entity relationship and storage medium | |
| CN106126619A (en) | A kind of video retrieval method based on video content and system | |
| WO2023108991A1 (en) | Model training method and apparatus, knowledge classification method and apparatus, and device and medium | |
| CN110781671A (en) | Knowledge mining method for intelligent IETM fault maintenance record text | |
| CN115858758A (en) | Intelligent customer service knowledge graph system with multiple unstructured data identification | |
| CN111813874B (en) | Terahertz knowledge graph construction method and system | |
| CN115203421A (en) | Method, device and equipment for generating label of long text and storage medium | |
| CN110659392B (en) | Retrieval method and device, and storage medium | |
| CN112989058B (en) | Information classification method, test question classification method, device, server and storage medium | |
| CN114547303A (en) | Text multi-feature classification method and device based on Bert-LSTM | |
| CN113378024A (en) | Deep learning-based public inspection field-oriented related event identification method | |
| CN111104492B (en) | Civil aviation field automatic question and answer method based on layering Attention mechanism | |
| CN110889717A (en) | Method and device for filtering advertisement content in text, electronic equipment and storage medium | |
| CN116701604A (en) | Question and answer corpus construction method and device, question and answer method, equipment and medium | |
| CN114492362B (en) | Method and system for generating research and report questions and answers and computer readable storage medium | |
| CN115761235A (en) | Zero sample semantic segmentation method, system, equipment and medium based on knowledge distillation | |
| CN115270746A (en) | Question sample generation method and device, electronic equipment and storage medium | |
| CN112749556B (en) | Multi-language model training method and device, storage medium and electronic equipment | |
| CN112668284B (en) | Legal document segmentation method and system | |
| Rani et al. | Eliminating data duplication in CQA platforms using deep neural model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |