CN117370932A - Traffic information processing and sensing method based on multi-mode data fusion sensing - Google Patents

Abstract

The invention discloses a transportation intelligence processing and sensing method based on multi-modal data fusion perception, which includes: the user inputs the field to which the information needs to be obtained to determine the scope of the intelligence acquisition; and calls a large general language model to give the corresponding information. The domain's hypersynonym set Set _up , hyposynonym set Set _down , and synonym Set _syn are used as a basis to generate an initial _A0 generalized search keyword library; through the crawler tool, the _Ai- level generalized search keywords are generated based on this The library crawls data on the network to build an A _i- level intelligence database; imports the crawled data into the multi-modal data fusion sensing system to generate an A _i- level generalized search keyword database, repeating until i is greater than the number of iterations limited by the user; The final A _i- level intelligence knowledge base content is generated into a classified intelligence knowledge base based on institutional sources. Through time scoring and word frequency statistical results, the development trend of the user's field of concern is obtained. The invention can realize an effective screening mechanism to screen the obtained information and improve the efficiency of related work.

Description

Transportation intelligence processing and perception method based on multi-modal data fusion perception

Technical field

The invention belongs to the field of data processing technology, and specifically relates to a transportation information processing and sensing method based on multi-modal data fusion sensing.

Background technique

For the transportation industry, in addition to materials, structures, performance, patents and algorithms from a technical perspective, its research content also includes but is not limited to national strategies, regulations, industry specifications, structures of highway-related departments in various countries, etc. that are top-level designs. Intelligence, as well as the pilot and implementation of various high-tech technologies. This information can, to a certain extent, reflect a country's development level in the field of transportation and the development trend of the industry around the world. It is key intelligence in the field of transportation engineering.

However, this series of information exists on the Internet in various forms such as text, pictures, publications, radio recordings, video information, etc. Obtaining intelligence through traditional crawlers will ignore other forms of information besides text. On the one hand, the amount of information is huge. Obtaining the content through traditional crawler technology and then manually filtering it one by one is tantamount to finding a needle in a haystack. At this stage, with the development of self-media, a large number of self-media personnel in the transportation field have begun to publish videos or pdf formats. It tracks the subdivisions of a certain transportation industry and provides unique insights. This information is actually filtered high-quality intelligence, but it is difficult to obtain this information through traditional intelligence acquisition methods. In addition, since the transportation engineering industry is inseparable from the development of national infrastructure construction, its research direction is not only limited to technology, but also includes policies, regulations, etc. The value of the same information to different research directions is significantly different, and an effective method is needed. The screening mechanism screens the obtained intelligence to improve the efficiency of related work.

At this stage, there is no system that can collect, filter, and trend intelligence for the transportation industry. Conventional intelligence collection and trend sensing only rely on a large number of manual searches, or crawlers to capture information and then manually filter it, which is time-consuming and labor-intensive. , and the retrieval efficiency is low, there is a lot of duplicate content, and it takes a long time.

(1) From the perspective of search methods, existing crawlers can only search in conventional search engines or user-specified domain names based on the keywords entered by the user. The information obtained by this search method greatly depends on the user’s Familiarity with the research field. All content crawled by the crawler is retrieved based on the regular expression entered by the user. For the transportation industry, the same content entity has different expressions, which will significantly affect the retrieval results. Therefore, the results obtained by conventional crawlers have low fault tolerance and high repeatability, resulting in unsatisfactory actual use results.

(2) From the perspective of the data types crawled by the crawler, although it can save text data, picture data, publication data, and audio and video data, for the transportation field, the types of data are rich and each type of data is They are all huge in size. Manually filtering out useful parts from these results is equivalent to finding a needle in a haystack. There is a lack of an effective means to clean, classify, and refine the data, as well as to search for multiple modal data during the search process. Perform fusion and utilization to reversely improve the efficiency and accuracy of search.

(3) The intelligence results obtained by conventional crawlers are often only a list containing web page titles, data and original links. The content is complex and difficult to directly present the development trend of the industry. It requires a large amount of manpower for screening and verification, which is not conducive to timely grasping. Trends in foreign transportation fields.

(4) Since the transportation engineering industry is inseparable from the development of national infrastructure construction, its research direction is not only limited to technology, but also includes policies, regulations, etc. The value of the same information to different research directions is significantly different, and a method is needed The effective screening mechanism automatically scores and screens the value of the obtained intelligence to improve the efficiency of related work, which is not available in crawler technology.

Therefore, from the perspective of data acquisition and data screening, the current technology cannot meet the needs of effective intelligence acquisition in the transportation field. There is an urgent need to develop a new type of multi-modal big data fusion perception for the transportation field. Only intelligence collection and trend perception methods can meet the current needs of intelligence collection, screening and trend perception in the transportation industry.

Contents of the invention

In order to solve the problems existing in the existing technology, the present invention provides a transportation intelligence processing and sensing method based on multi-modal data fusion sensing, which realizes an effective screening mechanism to filter the obtained intelligence and improves the efficiency of related work.

An embodiment of the present invention provides a transportation information processing and sensing method based on multi-modal data fusion sensing, which includes the following steps:

Step S1: The user inputs the field in which the intelligence needs to be obtained to determine the scope of intelligence acquisition;

Step S2, call the general language large model to give the hyperonym set Set _up , hyponym set _down , and synonym Set _syn in the corresponding field, and generate an initial A ₀ generalized search keyword database based on this;

Step S3: Use a crawler tool to crawl data on the network based on the Ai _- level generalized search keyword database to build an _Ai- level intelligence database;

Step S4, import the crawled data into the multi-modal data fusion sensing system, generate an A _i- level generalized search keyword database, repeat the step S3 until i is greater than the number of iterations defined by the user, and jump out of the loop; the multi-modal The data fusion perception system includes: multimodal data semantic subsystem and multimodal semantic data fusion subsystem, among which,

The multi-modal data semantic subsystem is used to perform recognition and conversion processing on video data, picture data, pdf data and audio data respectively, and generate the processed database A _i-1 ;

The multimodal semantic data fusion subsystem is used to construct a scoring system according to the characteristics of the transportation industry, evaluate the validity of the content A _i-1 retrieved by the multimodal data semantic subsystem, and generate each piece of information According to the validity score of the data, the intelligence is sorted and processed, and the top-ranked words are displayed to the user for users to filter. The user's filtered results are used to generate a new generalized search keyword database A _i+1 . Jump to step S2, replace A ₀ with A _i+1 , and use the results of the first search to expand the search database by extracting keywords through data fusion and scoring optimization;

Among them, the multi-modal semantic data fusion subsystem evaluates the validity of the retrieved content, including:

(1) Keyword recurrence score p _k : evaluated based on the number of times the keyword appears in the text data;

(2) Timeliness score p _t : Evaluate the value of the retrieved information from the perspective of timeliness;

(3) Institutional authority score p _a : From the perspective of the authoritativeness of the text source, determine the value of the e-th text under the needs of the user;

The final effectiveness score of the e-th intelligence in A _i-1 is

score＝p _k +p _t +p _a ;

Step S5: Generate a classified intelligence knowledge base based on the institutional source of the final A _i- level intelligence knowledge base content, and obtain the development trend of the user's field of concern through time scoring and word frequency statistical results.

Preferably, in step S2, the initial A ₀ generalized search keyword library includes:

{A ₀ }＝Set _up ∪Set _down ∪Set _syn ,

Among them, the set of upper meaning words: Set _up = {u ₁ , u ₂ , u ₃ ...... u _i };

Set of hyponyms: Set _down = {d ₁ , d ₂ , d ₃ ...d _j };

Synonym set: Set _syn = {s ₁ , s ₂ , s ₃ ......s _k };

Then the cross expansion set is expressed as:

Set _expand ={u ₁ d ₁ , u ₁ d ₂ , u ₁ d ₃ ....u ₂ d ₁ , u ₂ d ₂ ...u ₃ d ₁ ...u _i d _j ...u _{i sk} }.

In any of the above solutions, preferably, in step S3, the Ai _- level intelligence database includes multiple pieces of intelligence data, and each piece of intelligence data includes: keywords used in retrieval, and the original link of the web page. , the main domain name, the initial launch time of the web page and the intelligence data in the web page.

In any of the above solutions, it is preferred that (1) for video data, the video is automatically captured frame by frame based on the pythonCV2 library and the obtained frame-by-frame data is added to the picture data of the corresponding column; the audio is captured based on the python moviepy library Separate it and save it to the audio data in the corresponding column;

(2) For image data and PDF data, the OCR method is used to identify the data based on the python Tesseract library, and the generated text content is added to the text data in the same column;

(3) For audio data, call the speech recognition API to convert the audio into text and add it to the text data in the same column.

In any of the above solutions, it is preferred that the multi-modal semantic data fusion subsystem performs keyword recurrence scoring, including the following steps:

Suppose the number of occurrences of the j-th keyword in A _i in the e-th piece of text data is a _l , and the number of words in this piece of text data is AWN, then the keyword recurrence score p _k of the e-th piece of text data is as follows :

In any of the above solutions, it is preferred that the multi-modal semantic data fusion subsystem performs timeliness scoring, including the following steps:

First, determine whether the e pieces of information are directly equal. If they are the same, delete the duplicates in A _i-1 and extract the duplicates into a subset set; then, perform general weight reduction on the reduced A _i-1. ; Finally, rate the timeliness of the information in A _i-1 after weight reduction.

In any of the above solutions, it is preferred that the multi-modal semantic data fusion subsystem performs general weight reduction on the weight-reduced A _i-1 , including the following steps:

First, according to the size of the computer storage space and the volume of the retrieved text data, set the window length and overlap length, disassemble the e-th text information, and set _te the set of i _e segment elements;

Then, each segment element in the set Set _te is cut by word, stems are extracted, word frequency is calculated and hash coding is performed, and the compressed coded data of the segment element is obtained;

Finally, the Hamming distance of the compression codes of each segment element in the set is calculated, the similarity is calculated, and the duplicates are extracted into a subset set.

In any of the above solutions, it is preferred that the multi-modal semantic data fusion subsystem performs timeliness scoring on the intelligence in A _i-1 after weight reduction, including the following steps:

Calculate the time span coefficient T _range of each piece of information in A _i-1 ;

According to the initial online time of the web page, extract the earliest online time of each piece of information, construct a vector, perform normalization operations, and calculate the timeliness coefficient T _early ;

According to the time span coefficient T _range and the timeliness coefficient T _early , the timeliness score p _t is calculated as:

p _t =k _range T _range +k _early T _early ;

Among them, k _range is the time span coefficient weight; k _early is the aging coefficient weight.

In any of the above solutions, it is preferable that the data sources selected by the institution authority score p _a according to the main domain name include: news category, forum category, government category, think tank category, and technology category.

The transportation information processing and sensing method based on multi-modal data fusion sensing according to the embodiment of the present invention has the following beneficial effects:

(1) By borrowing mature large-scale language models, by associating the hyponyms and synonyms of the keywords entered by the user, and using algorithms to build a keyword retrieval library based on this, we can expand the search with the user's keywords as the center. The scope can, to a certain extent, reduce the problem of low-quality intelligence collected during the first search due to the user's lack of familiarity with the field.

(2) In view of the problem that information in the field of transportation exists in various forms, but traditional crawlers can only download data and cannot parse the data, the present invention builds a multi-modal data semantic system based on speech recognition technology and OCR recognition technology, integrating multiple Modal data is converted into text data, making full use of existing types of intelligence.

(3) In view of the differences in research issues in the field of transportation, a multi-modal semantic data fusion perception system is designed, innovatively targeting the two main directions of policy research and technology research in the field of transportation, from the perspective of topic relevance, information timeliness (time span Based on three perspectives, including time priority) and the value of intelligence sources, an intelligence value evaluation algorithm in the field of transportation is designed. The semanticized intelligence data is deeply screened according to the conventional understanding of the transportation industry, and ranked by value. , intelligence timeliness scoring and word frequency statistics to achieve trend perception in two basic areas.

(4) Based on the industry trends analyzed by the multi-modal semantic data fusion perception system, further using natural language processing technology to provide users with recommended keyword categories after the initial search, thereby using a semi-supervised method based on the user's choice A keyword library for the next step of search is constructed to achieve real-time directional keyword expansion based on the knowledge of the transportation industry, which greatly increases the search scope, reduces the collection of invalid information, and improves search efficiency.

Description of the drawings

Figure 1 is a flow chart of a transportation intelligence processing and sensing method based on multi-modal data fusion sensing according to an embodiment of the present invention.

Detailed ways

In order to further understand the content of the present invention, the present invention will be described in detail below with reference to specific embodiments.

As shown in Figure 1, the transportation intelligence processing and sensing method based on multi-modal data fusion sensing according to the embodiment of the present invention includes the following steps:

In step S1, the user inputs the field in which the information needs to be obtained, thereby determining the scope of information acquisition.

Step S2: Call the general language large model to give the hypersynonym set Set _up , hyponym set set _down , and synonym Set _syn in the corresponding field, and generate an initial A ₀ generalized search keyword library based on this.

Among them, the general large language model can be any open source or commercial large language model; the keywords in the initial generalized search keyword database include

{A ₀ }＝Set _up ∪Set _down ∪Set _syn ,

Among them, the set of upper meaning words: Set _up = {u ₁ , u ₂ , u ₃ ...... u _i };

Set of hyponyms: Set _down = {d ₁ , d ₂ , d ₃ ...d _j };

Synonym set: Set _syn = {s ₁ , s ₂ , s ₃ ......s _k };

Then the cross expansion set is expressed as:

Set _expand ={u ₁ d ₁ , u ₁ d ₂ , u ₁ d ₃ ....u ₂ d ₁ , u ₂ d ₂ ...u ₃ d ₁ ...u _i d _j ...u _{i sk} }.

Step S3: Use a crawler tool to crawl data on the Internet based on the Ai _- level generalized search keyword database to build an _Ai- level intelligence database.

In this step, the _Ai- level intelligence database includes multiple pieces of intelligence data. Each piece of intelligence data includes: the keywords used in the search, the original link to the webpage, the main domain name, the initial launch time of the webpage, and the intelligence data in the webpage. .

Specifically, the crawler tool can obtain the text information in the web page, download pictures, PDFs, audio and video and other data, and can also read the online time of the web page, and finally the captured data forms an _Ai- level intelligence library. Each intelligence data item includes: the keywords used in the search, the original link of the webpage, the main domain name, the initial online time of the webpage and the intelligence data in the webpage. Among them, the intelligence data in the web page includes: text data, picture data, audio and video data, pdf publications and other data forms.

Step S4: Import the crawled data into the multi-modal data fusion sensing system to generate an A _i- level generalized search keyword database. Repeat step S3 until i is greater than the number of iterations defined by the user, and then break out of the loop.

In embodiments of the present invention, the multimodal data fusion perception system includes: a multimodal data semantic subsystem and a multimodal semantic data fusion subsystem.

The multi-modal data semantic subsystem is used to perform recognition and conversion processing on video data, picture data, pdf data and audio data respectively, and generate the processed database A _i-1 .

(1) For video data, first use the python CV2 library to automatically capture the video frame by frame and add the obtained frame-by-frame data to the picture data of the corresponding column; use the python moviepy library to separate the audio part and save it to the corresponding column. audio data.

(2) For image data and PDF data, they are identified through the OCR method based on the python Tesseract library, and the generated text content is added to the text data in the same column.

(3) For audio data, call the speech recognition API to convert the audio into text and add it to the text data in the same column.

(4) The processed database is named A _i-1 .

The multimodal semantic data fusion subsystem is used to construct a scoring system based on the characteristics of the transportation industry, evaluate the validity of the retrieved content, generate a validity score for each piece of intelligence data, sort and process the intelligence based on the obtained information, and The top-ranked words are displayed to the user for filtering, and a new generalized search keyword library A _i+1 is generated based on the user's filtered results, and the process jumps to step S3.

The multimodal semantic data fusion subsystem evaluates the validity of the retrieved content, including:

(1) Keyword recurrence score p _k : evaluated based on the number of times the keyword appears in the text data.

Suppose the number of occurrences of the j-th keyword in A _i in the e-th piece of text data is a _l , and the number of words in this piece of text data is AWN, then the keyword recurrence score p _k of the e-th piece of text data is as follows :

(2) Timeliness score p _t : Evaluate the value of the retrieved information from the perspective of timeliness.

First, determine whether the e pieces of information are directly equal. If they are the same, delete the duplicates in A _i-1 and extract the duplicates into a subset set; then, perform general weight reduction on the reduced A _i-1. ; Finally, rate the timeliness of the information in A _i-1 after weight reduction.

The multi-modal semantic data fusion subsystem performs generalized weight reduction on A _i-1 after weight reduction, including the following steps:

First, chunk segmentation is performed. Determine the sliding window length a and overlap length b according to the size of the computer storage space and the volume of retrieved text data. Among them, a is much larger than b. Therefore, by disassembling the e-th piece of text information with length d, a set Set _te containing i _e segment elements will be obtained.

Then, compression encoding is performed. Each segment element in the set Set _te is cut by word, word stems are extracted, word frequency is calculated and hash coding is performed, and the compressed coded data of the segment element is obtained.

Specifically, based on the open source corpus, the PythonNLTK library is used to cut each segment element in the set Set _te by word, and perform word stem extraction to eliminate the influence of word form and non-substantial words. Calculate the word frequency, perform ordinary hash transformation on the words with the first a*20% of the word frequency, and obtain the hash code corresponding to each word; the hash is a binary code. When it is 0, it takes -1, and when it is 1, it takes 1, that is, 00101---> [-1,-1,1,-1,1], store the transformed result as a row vector, perform column summation on it, and obtain the compression encoding of the segment elements.

Finally, the Hamming distance of the compression coding of each segment element in the set is calculated, the similarity is calculated, and the duplicates are extracted into a subset set.

Specifically, the Hamming distance of the compression codes of each segment element in the set is calculated (exclusive OR operation is performed between the two codes, and the number of results 1 is calculated), the calculation results are reversed, and the top c% combinations are considered to be similar. (c is determined by the user), remove duplicates in A _i-1 based on similar situations, and extract duplicates into a subset set.

The multi-modal semantic data fusion subsystem scores the timeliness of the information in A _i-1 after the weight reduction. The e-th piece of information in A _i-1 after the weight reduction is mainly based on the time duration and first time of the information. Evaluate its timeliness based on the time it appears. For policy research, the wider the time span in which policy documents appear in intelligence, the greater its support for the industry and the higher the timeliness score. However, the opposite is true for technical research. The larger the time span, the higher the timeliness score. The more mature the content, the less innovative it is, so it needs to be evaluated separately based on different user needs.

Specifically, timeliness scoring includes the following steps:

(1) Calculate the time span coefficient T _range of each piece of information in A _i-1 .

Based on the initial launch time of the web page, calculate the time span of each piece of information in A _i-1 . For information with no duplicates, its time span is 0. For information with similar subsets, its time span is the latest time in the similar subset. The difference between the intelligence and the earliest intelligence (recorded in days), construct a new vector from the time span of all intelligence entries, perform normalization operation, and the obtained value is the time span coefficient.

(2) According to the initial online time of the web page, extract the earliest online time of each piece of information, construct a vector, perform normalization operations, and calculate the timeliness coefficient T _early .

According to the time span coefficient T _range and the timeliness coefficient T _early , the timeliness score p _t is calculated as:

p _t =k _range T _range +k _early T _early ;

Among them, k _range is the time span coefficient weight; k _early is the aging coefficient weight.

Among them, for high-tech intelligence tasks, k _range = 0.2, k _early = 0.8; for mature technology intelligence tasks, k _range = 0.6, k _early = 0.4; for policy support intelligence, k _range = 0.7, k _early = 0.3; for general policies Task k _range =0.6, k _early =0.4.

(3) Institutional authority score p _a : From the perspective of the authoritativeness of the text source, determine the value of the e-th text under the needs of the user.

In the embodiment of the present invention, the data sources selected by the institutional authority score p _a according to the main domain name include: news category, forum category, government category, think tank category, technology category, etc. It should be noted that the data source of the institution's authoritative rating p _a is not limited to the above examples, and can also include other categories, and the settings can be adjusted by the user as needed, which will not be described again here.

Specifically, according to the main domain name, the sources of data can be mainly divided into five major categories: news, forums, government, think tanks, and technology. The intelligence value of different sources has obvious differences in different research directions. Transportation The transportation industry mainly includes technical research and policy research. The scoring tables of the five sources under different research directions are shown in Table 1 below. In actual application, the source of the data institution is determined based on the main domain name entry of the data and the preset domain name. -Organization mapping library for judgment. As shown in Table 1, 1-news p _news , 2-forum p _blog , 3-government p _governance , 4-think tank p _thinktank , 5-technology p _tech .

Table 1 Rating form for five sources

P _news P _blog P _governance P _thinktank P _tech Technology category 0.2 0.2 0.4 0.6 0.8 Policy 0.6 0.6 0.8 0.6 0.2

(4) The final validity score of the e-th information in A _i-1 is

score=p _k +p _t +p _a .

Sort the intelligence according to the score, retain the top e*b (b is a user-defined proportion) intelligence data, extract phrases based on the pythonnltk library, calculate word frequency, sort, and retain the top c% (c is a user-defined proportion) Phrases are classified based on the Hamming distance of the phrase source when calculating similarity. The top ten words with the highest word frequency in each category are displayed to the user as representatives of the subcategory, allowing users to quickly identify keywords. Filter, the filtered result generates a new generalized search keyword database A _i+1 , jump to step 2. Replace A _i+1 with A ₀ and use the results of the first search to expand the search database by extracting keywords through data fusion and scoring optimization.

Step S5: Generate a classified intelligence knowledge base based on the institutional source of the final A _i- level intelligence knowledge base content, and obtain the development trend of the user's field of concern through time scoring and word frequency statistical results.

According to the transportation intelligence processing and sensing method based on multi-modal data fusion sensing according to the embodiment of the present invention, it can solve the problem that the conventional intelligence collection system in the field of transportation has low intelligence collection efficiency and little effective information, which makes it difficult for researchers in related industries to Issues related to providing policy makers with effective industry development trend information, implementing an effective screening mechanism to automatically score and filter the obtained intelligence, improving the efficiency of related work, using effective means to clean, classify and refine data, and During the search process, data from multiple modalities are integrated and utilized to reversely improve the efficiency and accuracy of the search, enabling timely grasp of trends in the field of foreign transportation.

The transportation information processing and sensing method based on multi-modal data fusion sensing according to the embodiment of the present invention has the following beneficial effects:

(1) By borrowing mature large-scale language models, by associating the hyponyms and synonyms of the keywords entered by the user, and using algorithms to build a keyword retrieval library based on this, we can expand the search with the user's keywords as the center. The scope can, to a certain extent, reduce the problem of low-quality intelligence collected during the first search due to the user's lack of familiarity with the field.

(2) In view of the problem that information in the field of transportation exists in various forms, but traditional crawlers can only download data and cannot parse the data, the present invention builds a multi-modal data semantic system based on speech recognition technology and OCR recognition technology, integrating multiple Modal data is converted into text data, making full use of existing types of intelligence.

(3) In view of the differences in research issues in the field of transportation, a multi-modal semantic data fusion perception system is designed, innovatively targeting the two main directions of policy research and technology research in the field of transportation, from the perspective of topic relevance, information timeliness (time span Based on three perspectives, including time priority) and the value of intelligence sources, an intelligence value evaluation algorithm in the field of transportation is designed. The semanticized intelligence data is deeply screened according to the conventional understanding of the transportation industry, and ranked by value. , intelligence timeliness scoring and word frequency statistics to achieve trend perception in two basic areas.

(4) Based on the industry trends analyzed by the multi-modal semantic data fusion perception system, further using natural language processing technology to provide users with recommended keyword categories after the initial search, thereby using a semi-supervised method based on the user's choice A keyword library for the next step of search is constructed to achieve real-time directional keyword expansion based on the knowledge of the transportation industry, which greatly increases the search scope, reduces the collection of invalid information, and improves search efficiency.

Special note: The technical solution of the present invention involves many parameters, and the synergy between each parameter needs to be comprehensively considered in order to obtain the beneficial effects and significant progress of the present invention. Moreover, the value range of each parameter in the technical solution was obtained through a large number of tests. For each parameter and the combination of each parameter, the inventor has recorded a large amount of test data. Due to space limitations, the specific test data will not be disclosed here.

It is easy for those skilled in the art to understand that the transportation information processing and sensing method based on multi-modal data fusion sensing of the present invention includes the above-mentioned content of the invention and specific embodiments of the description of the present invention and each part shown in the drawings. Any combination, limited by space and to make the description concise, each solution composed of these combinations is not described one by one. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (9)

1. A traffic information processing and sensing method based on multi-mode data fusion sensing is characterized by comprising the following steps:

step S1, inputting the field to which the information needs to be acquired by a user so as to determine the range of acquiring the information;

step S2, calling a general language big model to give an upper sense word Set in the field _up Hyponym Set _down With synonym Set _syn And based thereon generating initial A ₀ Generalized search keyword library;

step S3, through a crawler tool, according to the A _i Step A, crawling data on a network by a level generalized search keyword library to construct step A _i A level information library;

step S4, importing the crawling data into a multi-mode data fusion sensing system to generate A _i Step S3, the keyword library is searched in a level generalized mode, and the step S3 is repeated until i is larger than iteration times defined by a user, and a loop is jumped out; the multi-modal data fusion awareness system includes: a multi-modal data semanticalization subsystem and a multi-modal semantic data fusion subsystem, wherein,

the multi-mode data semanteme subsystem is used for carrying out identification conversion processing on video data, picture data, pdf data and audio data respectively and rapidly, and generating a processed database A _i-1 ；

The multi-mode semantic data fusion subsystem is used for constructing a scoring system according to the characteristics of the transportation industry and retrieving the content A obtained by the multi-mode data semantic subsystem _i-1 Performing validity assessment, generating a validity score of each piece of information data, sorting and processing the information according to the obtained score, displaying the words with the top ranking to a user for the user to screen, and generating a new generalized search keyword library A from the screened result of the user _i+1 Jumping to the step S2, and setting A _i+1 Alternative A ₀ Utilizing the result of the first retrieval, extracting the keywords through data fusion and scoring optimization to enlarge the retrieval database;

the multimode semantic data fusion subsystem performs validity evaluation on the retrieved content, and the multimode semantic data fusion subsystem comprises:

(1) Keyword recurrence score p _k : evaluating from the number of times the keyword appears in the text data;

(2) Timeliness score p _t : evaluating the value of the retrieved information from the time-efficiency point of view;

(3) Authority score p _a : judging the value of the e text under the requirement of the user from the perspective of authority of the text source;

final A _i-1 The effectiveness score of the e-th information in (a) is as follows

score＝p _k +p _t +p _a ；

Step S5, for the finally obtained A _i The level information knowledge base content generates a classified information knowledge base according to the organization sources, and the development trend of the user attention field is obtained through time scoring and word frequency statistics results.

2. The traffic intelligence processing and sensing method based on multi-modal data fusion sensing as set forth in claim 1, wherein in said step S2, said initial a ₀ The generalized search keyword library includes:

{A ₀ }＝Set _up ∪Set _down ∪Set _syn ，

wherein, the word set is defined in the upper sense: set (Set) _up ＝{u ₁ ，u ₂ ，u ₃ ......u _i }；

The hyponym set: set (Set) _down ＝{d ₁ ，d ₂ ，d ₃ ......d _j }；

Synonym set: set (Set) _syn ＝{s ₁ ，s ₂ ，s ₃ ......s _k }；

The cross-augmentation set is expressed as:

Set _expand ＝{u ₁ d ₁ ，u ₁ d ₂ ，u ₁ d ₃ ....u ₂ d ₁ ，u ₂ d ₂ ...u ₃ d ₁ ...u _i d _j ...u _i s _k }。

3. the traffic information processing and sensing method based on multi-modal data fusion sensing as set forth in claim 1, wherein in said step S3, said a _i The level intelligence library comprises a plurality of pieces of intelligence data, and each piece of intelligence data comprises: the method comprises the steps of adopting keywords, original links of the webpage, a main domain name, initial online time of the webpage and information data in the webpage during retrieval.

4. The traffic information processing and sensing method based on multi-modal data fusion sensing as claimed in claim 1, wherein,

(1) For video data, automatically capturing images frame by frame based on a python CV2 library, and adding the obtained frame by frame data to picture data of a corresponding column; separating audio from the audio based on the python movie library, and storing the audio data in a corresponding column;

(2) For the picture data and pdf data, identifying based on a python Tesseact library by an OCR method, and adding the generated text content into the text data in the same column;

(3) For audio data, call speech recognition API to convert audio into text and add to the text data in the same column.

5. The traffic intelligence processing and sensing method based on multi-modal data fusion sensing as set forth in claim 1, wherein the multi-modal semantic data fusion subsystem performs keyword recurrence scoring, comprising the steps of:

let A _i The number of occurrences of the jth keyword in the jth text data is a _l The number of words of the text data is AWN, and the keyword reproduction score p of the e text data _k The following is shown:

6. the traffic intelligence processing and sensing method based on multi-modal data fusion sensing as set forth in claim 1, wherein the multi-modal semantic data fusion subsystem performs timeliness scoring, comprising the steps of:

firstly, judging whether the e pieces of information are directly equal, if so, then at A _i-1 Repeating items are deleted, and the repeating items are extracted as a subset set; then, for A after weight reduction _i-1 Generalized weight reduction is carried out; finally, for A after weight reduction _i-1 And (5) carrying out timeliness scoring on the information in the process.

7. The traffic information processing and sensing method based on multi-modal data fusion sensing as set forth in claim 6, wherein said multi-modal semantic data fusion subsystem is configured to reduce weight of A _i-1 The generalized weight reduction is carried out, and the method comprises the following steps:

firstly, according to the size of computer storage space and the volume of the retrieved text data, setting window length and overlap length, and dismantling the e-th text information, i _e Set of individual sentence segment elements _te ；

Then to the Set _te Each sentence segment element is cut according to words, word stem extraction is carried out, word frequency is calculated, hash coding is carried out, and compression coding data of the sentence segment element are obtained;

and finally, calculating the hamming distance of each sentence segment element compression coding in the set, performing similarity calculation, and extracting the repeated item as a subset set.

8. The traffic information processing and sensing method based on multi-modal data fusion sensing as set forth in claim 6, wherein said multi-modal semantic data fusion subsystem pair weight-reduced a _i-1 The time-based scoring of the information in the step comprises the following steps:

calculation A _i-1 Time span coefficient T of each piece of information _range ；

Extracting according to the initial online time of the webpageThe earliest time of online of each piece of information, constructing vectors, carrying out normalization operation, and calculating to obtain an aging coefficient T _early ；

According to the time span coefficient T _range And ageing coefficient T _early Calculating a timeliness score p _t The method comprises the following steps:

p _t ＝k _range T _range +k _early T _early ；

wherein k is _range Weighting the time span coefficient; k (k) _early Is the ageing coefficient weight.

9. The traffic intelligence processing and sensing method based on multi-modal data fusion sensing as claimed in, wherein the authority score p _a The data sources selected according to the main domain name comprise: news, forum, government, intelligent, technical.