CN111242581A - Automatic analysis and BUG intelligent identification system for satellite navigation mass data - Google Patents
Automatic analysis and BUG intelligent identification system for satellite navigation mass data Download PDFInfo
- Publication number
- CN111242581A CN111242581A CN202010042189.6A CN202010042189A CN111242581A CN 111242581 A CN111242581 A CN 111242581A CN 202010042189 A CN202010042189 A CN 202010042189A CN 111242581 A CN111242581 A CN 111242581A
- Authority
- CN
- China
- Prior art keywords
- data
- satellite navigation
- bug
- detected
- monitoring data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Databases & Information Systems (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- General Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Computer Networks & Wireless Communication (AREA)
- Software Systems (AREA)
- Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a satellite navigation mass data automatic analysis and BUG intelligent identification system, which comprises an initial information configuration unit, a data storage processing server and a full-automatic processing module running in the data storage processing server, wherein when a plurality of devices to be detected are detected simultaneously, the full-automatic processing module is used for decoding satellite navigation monitoring data acquired by each device to be detected and carrying out BUG analysis to obtain corresponding detection results, and all the detection results are fed back to corresponding managers according to corresponding initial information. Therefore, the intelligent and automatic system for simultaneously detecting the devices to be detected with different product types and different data formats and automatically identifying the data abnormal BUG in the satellite navigation data is realized.
Description
Technical Field
The invention relates to the technical field of satellite navigation data analysis, in particular to a satellite navigation mass data automatic analysis and BUG intelligent identification system.
Background
With the construction of the Beidou third-generation basic system announced by the China satellite navigation system management office in 12/27/2018, the Beidou satellite system can provide global service, complete high-strength density networking and form global service capability, and the Beidou third-generation global service is widely applied and is going out of the country to benefit the world, but with the continuous development of the satellite navigation system, the product types of the peripheral equipment tend to be diversified, specifically, multiple product types such as common navigation, precise time service, combined navigation, high-precision RTK, orientation, original observation quantity and the like exist, and the data formats of satellite navigation data generated by the equipment corresponding to each product type are different, such as an RTCM2.x protocol, an RTCM3.x protocol, a NovAtel 6 protocol, an OEM-blox private protocol, a Hemisphere protocol, a Rinex protocol, a Septensrio private protocol, an NMEA0183 protocol and private protocols of various manufacturers, therefore, the processing process of the satellite navigation data by each device is different, and then:
if the device is detected, in a detection process, only one device or a plurality of devices of the same product type in the same data format can be tested at the same time, so that the detection efficiency is low, in the test process, the satellite navigation data output by each device can be processed by data processing software, and then the data abnormal BUG can be found, so that the intelligent degree and the automatic degree are low.
Therefore, how to implement an intelligent and automatic system for simultaneously detecting devices to be detected of different product types and adopting different data formats and automatically identifying abnormal data BUGs in satellite navigation data is a technical problem to be solved urgently in the industry.
Disclosure of Invention
The invention aims to solve the technical problem of the prior art and provides a satellite navigation mass data automatic analysis and BUG intelligent identification system.
The technical scheme of the satellite navigation mass data automatic analysis and BUG intelligent identification system is as follows:
the system comprises an initial information configuration unit, a data storage processing server and a full-automatic processing module running in the data storage processing server;
the initial information configuration unit configures corresponding initial information in the data storage processing server according to the connected equipment to be detected with different product types;
in the detection process, each device to be detected respectively sends the acquired satellite navigation monitoring data with different data formats to the storage processing server for storage according to corresponding initial information;
the full-automatic processing module is used for decoding and BUG analyzing each satellite navigation monitoring data in sequence to obtain a corresponding detection result, and feeding back all the detection results to a manager of each device to be detected according to corresponding initial information.
On the basis of the scheme, the satellite navigation mass data automatic analysis and BUG intelligent identification system can be further improved as follows.
Further, the full-automatic processing module comprises a data storage unit;
the data storage unit names each satellite navigation monitoring data according to a preset naming standard and stores the satellite navigation monitoring data according to a preset storage mode;
the data storage unit also detects the residual disk space of the storage processing server, and if the residual disk space is smaller than a preset threshold, alarm information of insufficient residual disk space is sent out.
Further, the full-automatic processing module also comprises a data preprocessing unit;
the data preprocessing unit identifies the data format of each satellite navigation monitoring data, puts the satellite navigation monitoring data with the same data format into the same group, and marks each group;
the data preprocessing unit also identifies different product types corresponding to each satellite navigation monitoring data in each group, puts the satellite navigation monitoring data of the same product type into corresponding subgroup, and marks each subgroup.
Further, the full-automatic processing module also comprises a data analysis processing unit;
the data analysis processing unit reads the satellite navigation monitoring data corresponding to each subgroup in parallel in a binary single byte data reading mode, decodes each satellite navigation monitoring data in a coding mode of a corresponding data format to generate a plurality of satellite navigation decoding data correspondingly, and then performs BUG analysis on each satellite navigation decoding data.
Further, still include: the data analysis processing unit reads monitoring data of each index of satellite navigation in each satellite navigation decoding data, judges whether the monitoring data of each index is in a standard data range corresponding to each index, if not, judges that data abnormity BUG exists in the monitoring data, and calls a preprocessing scheme corresponding to each data abnormity BUG from a preprocessing scheme library for processing.
Further, still include: the data analysis processing unit also counts the test information of each device to be tested and the error information of each index, the test information comprises test duration, epoch number, positioning satellite number, positioning proportion and de-positioning proportion, and the error information comprises effective positioning rate, horizontal error and normal error.
Further, the full-automatic processing module also comprises a data mark construction unit;
and the data mark construction unit counts the name, the detection time, the file size, the corresponding product type and the storage position of each satellite navigation monitoring data in the data storage processing server.
Further, the full-automatic processing module also comprises a data report automatic generation unit and a report mail automatic sending unit;
the data report automatic generation unit generates a detection report, namely a detection result, corresponding to each device to be detected according to the information output by the data analysis processing unit and the data mark construction unit;
the initial information comprises an outbox and an inbox of a manager of each device to be detected, and the report mail automatic sending unit automatically sends each detected report to the corresponding manager.
Further, still include: and the report mail automatic sending unit sets different reminding levels according to the number of the data abnormal BUGs and/or the severity of each data abnormal BUG in each detection report.
Further, the data storage processing server further comprises a display, and the initial information configuration unit configures the initial information of each device to be detected in the data storage processing server through the display.
The system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG has the beneficial effects that:
when a plurality of devices to be detected are detected simultaneously, the satellite navigation monitoring data acquired by each device to be detected are decoded and BUG analyzed through a full-automatic processing module to obtain corresponding detection results, and all the detection results are fed back to corresponding managers according to corresponding initial information. Therefore, the intelligent and automatic system for simultaneously detecting the devices to be detected with different product types and different data formats and automatically identifying the data abnormal BUG in the satellite navigation data is realized.
Drawings
FIG. 1 is a schematic structural diagram of an automatic analysis and BUG intelligent identification system for satellite navigation mass data according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a method for automatically analyzing satellite navigation mass data and intelligently identifying a BUG according to an embodiment of the present invention.
Detailed Description
The system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG comprises an initial information configuration unit 100, a data storage processing server 101 and a full-automatic processing module 102 running in the data storage processing server 101, wherein the initial information configuration unit 100 is shown in figure 1;
the initial information configuration unit 100 configures corresponding initial information in the data storage processing server 101 according to a plurality of connected devices to be detected of different product types;
in the detection process, each device to be detected sends the acquired satellite navigation monitoring data with different data formats to a storage processing server for storage according to corresponding initial information;
the full-automatic processing module 102 decodes and BUG analyzes each satellite navigation monitoring data to obtain a corresponding detection result, and feeds back all the detection results to a manager of each device to be detected according to corresponding initial information.
When a plurality of devices to be detected are detected simultaneously, the satellite navigation monitoring data acquired by each device to be detected is decoded and BUG analyzed by the full-automatic processing module 102 to obtain corresponding detection results, and all the detection results are fed back to corresponding managers according to corresponding initial information. Therefore, the intelligent and automatic system for simultaneously detecting the devices to be detected with different product types and different data formats and automatically identifying the data abnormal BUG in the satellite navigation data is realized.
As shown in fig. 1, 6 devices to be tested can be set as an example for explanation, specifically:
1) marking 6 devices to be detected as a first device to be detected 109, a second device to be detected 110, a third device to be detected 111, a fourth device to be detected 112, a fifth device to be detected 113 and a sixth device to be detected 114 respectively;
respectively recording satellite navigation monitoring data acquired by 6 devices to be detected as first satellite navigation monitoring data, second satellite navigation monitoring data, third satellite navigation monitoring data, fourth satellite navigation monitoring data, fifth satellite navigation monitoring data and sixth satellite navigation monitoring data;
2) assuming that the product types of the first device to be detected 109, the second device to be detected 110 and the third device to be detected 111 are all common navigation, and the data formats of the first satellite navigation monitoring data, the second satellite navigation monitoring data and the third satellite navigation monitoring data are NMEA0183 protocols; the data format of the fourth satellite navigation monitoring data, the fifth satellite navigation monitoring data and the sixth satellite navigation monitoring data is an RTCM3.2 protocol, the product types of the fourth device to be detected 112 and the fifth device to be detected 113 are both precise time service, and the product type of the sixth device to be detected 114 is combined navigation, so:
the initial information configuration unit 100 configures the initial information of the 6 devices to be detected in the data storage processing server 101, and the initial information includes:
1) the storage positions of the first satellite navigation monitoring data, the second satellite navigation monitoring data, the third satellite navigation monitoring data, the fourth satellite navigation monitoring data, the fifth satellite navigation monitoring data and the sixth satellite navigation monitoring data in the data storage processing server 101;
2) the information of the managers of the first device to be detected 109, the second device to be detected 110, the third device to be detected 111, the fourth device to be detected 112, the fifth device to be detected 113 and the sixth device to be detected 114, such as an inbox, a telephone, a mobile phone number, a name and the like;
the initial information can be configured on a self-contained display screen or a display of the storage processing server, or the initial information can be configured on a computer connected with the storage processing server.
The full-automatic processing module 102 decodes and BUG analyzes the first satellite navigation monitoring data, the second satellite navigation monitoring data, the third satellite navigation monitoring data, the fourth satellite navigation monitoring data, the fifth satellite navigation monitoring data and the sixth satellite navigation monitoring data to obtain corresponding detection results, and feeds back all the detection results to the manager of the equipment to be detected according to corresponding initial information, such as obtaining an inbox of each manager, feeding back in a mail manner, such as obtaining a mobile phone number of each manager, feeding back in a short message or multimedia message manner, and the like.
Wherein, the equipment to be detected specifically can be the module product of chip level, also can be the high accuracy integrated circuit board that is used for measuring the survey and drawing, and the quantity of the equipment to be detected can be 1, 2, 10 and more, N is 1, 2 … … 100 … … promptly, and N does not have the upper limit, can understand, in order to guarantee the reliability of connecting between circuit and the hardware in the entire system of this application, the suggestion is gone on the quantity of the equipment to be detected that detects simultaneously and is not more than 20.
Preferably, in the above technical solution, the full-automatic processing module 102 includes a data storage unit 103;
the data storage unit 103 names each satellite navigation monitoring data according to a preset naming specification and stores the named data according to a preset storage mode;
the data storage unit 103 further detects a remaining disk space of the storage processing server, and if the remaining disk space is smaller than a preset threshold, sends alarm information indicating that the remaining disk space is insufficient, specifically:
1) when 6 devices to be detected are detected simultaneously, firstly, the residual disk space of a storage processing server is detected, if the residual disk space is smaller than a preset threshold, alarm information that the residual disk space is insufficient is sent, for example, the residual disk space is 2G, the preset threshold is 1.5G, at the moment, the alarm information that the residual disk space is insufficient can be sent, the alarm information can prompt a detection person through a pop-up window on a display of a computer, and if the storage processing server is provided with a display screen or a display, the storage processing server can prompt the detection person through the pop-up window on the display screen or the display;
the preset threshold can be set by a user, and if the user detects 6 devices to be detected simultaneously, the data size of corresponding 6 satellite navigation monitoring data can be predicted to be 10G according to experience, and then the preset threshold can be set to be 10G;
2) in the process of simultaneously detecting 6 devices to be detected, whether the residual disk space is smaller than a preset threshold or not is judged in real time, if yes, alarm information that the residual disk space is insufficient is sent to detection personnel, and the detection personnel expand the disk space or delete previous old data to ensure that the residual disk space is not smaller than the preset threshold.
The preset naming specification can be in various forms, such as naming 6 satellite navigation monitoring data by using the forms of data specification, product type, serial number of equipment to be detected or time, data specification, product type, serial number of equipment to be detected.
The preset storage manner can be understood as:
1) the first satellite navigation monitoring data may be stored by time, such as daily or hourly, in particular: if the first satellite navigation monitoring data is stored every hour, when the first device to be detected 109 detects for three hours, 3 files are generated to store the first satellite navigation monitoring data;
2) the first satellite navigation monitoring data may be stored according to a file size, specifically: if the first satellite navigation monitoring data is stored according to the file size of 1G, and if the first satellite navigation monitoring data is 4G, generating 4 files for storing the first satellite navigation monitoring data;
by the preset storage mode, the problem that the data size of a single file is too large to cause easy occurrence of a problem in the processing process when the first device to be detected 109 is subjected to long-term stability test is avoided.
Preferably, in the above technical solution, the fully automatic processing module 102 further includes a data preprocessing unit 104;
the data preprocessing unit 104 identifies the data format of each satellite navigation monitoring data through a fast data browsing technology, puts the satellite navigation monitoring data in the same data format into the same group, and marks each group;
the data preprocessing unit 104 further identifies different product types corresponding to each satellite navigation monitoring data in each group through a key element identification technology, puts the satellite navigation monitoring data of the same product type into corresponding subgroups of the group, and marks each subgroup, specifically:
1) because the data formats of the first satellite navigation monitoring data, the second satellite navigation monitoring data and the third satellite navigation monitoring data are NMEA0183 protocols, and the data formats of the fourth satellite navigation monitoring data, the fifth satellite navigation monitoring data and the sixth satellite navigation monitoring data are RTCM3.2 protocols, then:
firstly, the first satellite navigation monitoring data, the second satellite navigation monitoring data and the third satellite navigation monitoring data are classified into the same group and marked as a first group; the fourth satellite navigation monitoring data, the fifth satellite navigation monitoring data and the sixth satellite navigation monitoring data are classified into the same group and marked as a second group;
2) because the product types of the first device to be detected 109, the second device to be detected 110 and the third device to be detected 111 are all common navigation, the product types of the fourth device to be detected 112 and the fifth device to be detected 113 are both precise time service, and the product type of the sixth device to be detected 114 is combined navigation, then:
the first satellite navigation monitoring data, the second satellite navigation monitoring data and the third satellite navigation monitoring data belong to the same subclass and are marked as a first subgroup; classifying the fourth satellite navigation monitoring data and the fifth satellite navigation monitoring data into the same subclass, and marking the subclass as a second subgroup; and classifying the sixth satellite navigation monitoring data into the same subclass, and marking the sixth satellite navigation monitoring data as a third subgroup.
Among these, the fast data browsing techniques can be understood as: for example, the data format is quickly identified by reading a synchronous header 0xD3 for RTCM3.2 protocol, or the data format is quickly identified by reading GGA characteristic characters for NMEA0183 message.
The key element identification technology can be understood as follows: for the equipment to be detected with the product type of precise time service, the message related to the pulse per second can be output; or aiming at the equipment to be detected with the product type of integrated navigation, outputting an inertial navigation initialization related message; or aiming at the high-precision equipment to be detected of the product type, the product type can be quickly identified through the positioning type; and then or aiming at the equipment to be detected with the product type as the original observation quantity, the original observation quantity message output by an RTCM3.2 protocol is used for quickly identifying.
Preferably, in the above technical solution, the fully automatic processing module 102 further includes a data analysis processing unit 105; the data analysis processing unit 105 reads the satellite navigation monitoring data corresponding to each sub-group in parallel in a binary single byte data reading mode, decodes each satellite navigation monitoring data in a coding mode of a corresponding data format to generate a plurality of satellite navigation decoding data correspondingly, and then performs a BUG analysis on each satellite navigation decoding data.
At present, the problems of analyzing and processing Satellite Navigation data include the problem of batch processing of mass data of a global Navigation Satellite System GNSS (GNSS is a short writing of the global Navigation Satellite System), the problem of compatibility of data formats, and the problem of concern in the process of processing mass data collected by devices of different product types, specifically:
1) the problem of batch processing of mass data of a Global Navigation Satellite System (GNSS) is as follows: the device to be detected adopting a high-precision carrier phase difference technology RTK (RTK is short for Real-Time Kinematic) is taken as an example for explanation, the output frequency is assumed to be 10Hz, the data format of the output satellite navigation data comprises an NMEA0183 protocol, an RTCM3.2 protocol and a custom debugging data format, the size of the satellite navigation data collected every hour is about 420MB, the size of the data volume of the satellite navigation data collected every 24 hours is about 10GB, if parallel testing of a plurality of devices to be detected is involved, massive data can be collected, and the problems of low efficiency and even crash of the data processing software and the like can exist in batch processing of the massive data by adopting current data processing software;
2) compatibility issues with data formats: the formats of the satellite navigation data comprise an RTCM2.x protocol, an RTCM3.x protocol, a NovAtel OEM6 protocol, a u-blob private protocol, a Hemisphere protocol, a Rinex protocol, a Septentrio private protocol, an NMEA0183 protocol and private protocols of various manufacturers, and the conventional manufacturers can only carry out compatible design aiming at one or more protocols;
3) concern in the process of processing mass data collected by devices of different product types: with the continuous development of satellite navigation systems, the types of terminal products are diversified, from the perspective of product types, there are multiple product types such as common navigation, precise time service, combined navigation, high-precision RTK, orientation, original observation and the like, and for each product type, the focus in the data processing process of satellite navigation data is different, and conventional data processing software can only provide an analysis result for a certain product type.
In the present application, the data preprocessing unit 104 divides the 6 satellite navigation monitoring data into 3 subgroups, namely a first subgroup, a second subgroup and a third subgroup, according to the data format and the product type, and the data analyzing and processing unit 105 reads the satellite navigation monitoring data corresponding to each subgroup in parallel in a binary single byte data reading manner, which can be understood as: the data analysis processing unit 105 simultaneously and parallelly starts three threads to process the first sub-group, the second sub-group and the third sub-group, and the three threads call the same encoding method, specifically:
the first thread calls an encoding mode of an NMEA0183 protocol to decode the first satellite navigation monitoring data, the second satellite navigation monitoring data and the third satellite navigation monitoring data;
the second thread calls a coding mode of an RTCM3.2 protocol to decode the fourth satellite navigation monitoring data and the fifth satellite navigation monitoring data;
the second thread calls an encoding mode of an RTCM3.2 protocol to decode the sixth satellite navigation monitoring data;
the high efficiency of batch processing of the 6 satellite navigation monitoring data is ensured, and the problems of breakdown and the like are avoided;
the data format library for storing various data formats can be constructed on the data storage processing server 101, and the data format library is provided with an interface for inputting an RTCM2.x protocol, an RTCM3.x protocol, a NovAtel OEM6 protocol, a u-blox private protocol, a Hemisphere protocol, a Rinex protocol, a Septentrio private protocol, an NMEA0183 protocol, private protocols of various manufacturers and the like, so that the compatibility problem of the data formats is solved.
Moreover, the satellite navigation monitoring data of the first device to be detected 109, the second device to be detected 110, the third device to be detected 111, the fourth device to be detected 112, the fifth device to be detected 113 and the sixth device to be detected 114 for different product types can be decoded to obtain corresponding detection results, namely, the detection results of the devices to be detected for different product types can be provided by the method.
The main purpose of using the above binary single byte data reading method is to compatibly process data fused with multiple types of data, and separate different types of data, where the different types of data can be understood as: the system comprises NMEA0183 protocol data of an ASCII encoding mode, receiver custom data of hexadecimal encoding and the like.
Preferably, in the above technical solution, the method further comprises: the data analysis processing unit 105 reads monitoring data of each index of satellite navigation from each satellite navigation decoding data, determines whether the monitoring data of each index is within a standard data range corresponding to each index, if not, determines that data exception BUG exists in the monitoring data, and invokes a corresponding preprocessing scheme corresponding to each data exception BUG from a preprocessing scheme library for processing, specifically:
the standard data ranges of the indexes may be pre-stored in the data storage and processing server 101, or the standard data ranges of the indexes may be configured in the data storage and processing server 101 through the initial information configuration unit 100, and the analysis is performed by taking the first device to be detected 109 as an example, then:
the satellite navigation monitoring data comprises cold-start First-Time positioning Time TTFF, when the detection environment is assumed to be an open environment, the First-Time positioning Time TTFF (TTFF is short for Time to First Fix) of cold-start of the First equipment to be detected 109 is used as an index, the detection times are 10 times, the First-Time positioning Time TTFF of each cold-start and the effective positioning rate are obtained, and the effective positioning rate is also used as the index; the standard data range of the cold-start first positioning time TTFF is less than 35 seconds, and the standard data of the effective positioning rate is 100%;
if the first time TTFF of the cold start positioning is greater than 35 seconds or the effective positioning rate does not reach 100% in 10 detections, it may be identified that there is a TTFF data abnormal BUG, that is, the location of the data abnormal BUG is determined, and then the TTFF data abnormal BUG is further analyzed, specifically:
if a processing scheme of the TTFF data abnormal BUG is stored in the preprocessing scheme library, assuming that the processing scheme requires to call another 'xxx' monitoring data, the 'xxx' monitoring data is read and fed back to a manager of the first device to be detected 109, after receiving the feedback, the manager performs more detailed analysis and troubleshooting on the TTFF data abnormal BUG, and may also record relevant contents such as a time point of the TTFF data abnormal BUG, thereby improving intelligence of identifying the data abnormal BUG, and for other indexes, the above contents may be referred to, and details are not repeated here.
Preferably, in the above technical solution, the method further comprises: the data analysis processing unit 105 further counts test information of each device to be tested and error information of each index, wherein the test information includes test duration, epoch number, positioning satellite number, positioning proportion and de-positioning proportion, and the error information includes effective positioning rate, horizontal error and normal error.
Wherein the positioning proportion comprises RTD differential positioning proportion, and the horizontal error comprises horizontal error RMS, horizontal error CEP50, horizontal error CEP95, horizontal error CEP99, horizontal maximum error and horizontal minimum error; the normal errors include normal error RMS, normal error CEP50, normal error CEP95, normal error CEP99, normal maximum error, and normal minimum error, the test information and the error information can be obtained through time conversion, coordinate conversion, variance calculation and other techniques, and the time conversion, coordinate conversion, variance calculation and other techniques are conventional techniques and are not described herein.
Taking the first device to be detected 109 as an example, the test information and the error information of each index are shown in table 1 below:
table 1:
preferably, in the above technical solution, the full-automatic processing module 102 further includes a data mark construction unit 106; the data tag construction unit 106 counts the name, the detection time, the file size, the corresponding product type, and the storage location in the data storage processing server 101 of each satellite navigation monitoring data. Taking the first device to be detected 109 as an example, specifically:
at present, the problem of satellite navigation data analysis and processing is the problem of intelligent identification of abnormal BUG: the traditional data processing software mainly analyzes and counts the position information, the speed information and the direction information of the equipment to be detected, and when the equipment to be detected has problems, the collected original satellite navigation data can only be checked and analyzed in a manual mode, so that the analysis efficiency is low;
the data marker establishing unit 106 is configured to count attributes of the first device to be detected 109 and the first satellite navigation monitoring data acquired by the first device to be detected, and specifically includes: the file name and the detection time of the first satellite navigation monitoring data are, for example, 2020, 01, the corresponding product type, that is, the common product, and the storage location in the data storage processing server 101, and the attribute is stored in an attribute database pre-stored in the data storage processing server 101, and similarly, the attributes of other devices to be detected and the satellite navigation monitoring data acquired by the devices to be detected are also stored in the attribute database, and all the attributes are fed back to the administrator, so that the administrator can quickly access the data during data backtracking and query in the future.
Preferably, in the above technical solution, the full-automatic processing module 102 further includes an automatic data report generating unit 107 and an automatic report mail sending unit 108;
the data report automatic generation unit 107 generates a detection report, i.e. a detection result, corresponding to each device to be detected according to the information output by the data analysis processing unit 105 and the data mark construction unit 106;
the initial information includes an outbox and an inbox of a manager of each device to be detected, the automatic report sending unit 108 automatically sends each detected report to a corresponding manager, which is described by taking the first device to be detected 109 as an example, specifically:
the detection report of the first device to be detected 109 includes: the method includes the steps of testing duration, the number of epochs, the number of positioning satellites, an RTD differential positioning ratio, a solution positioning ratio, an effective positioning rate, a horizontal error RMS, a horizontal error CEP50, a horizontal error CEP95, a horizontal error CEP99, a horizontal maximum error and a horizontal minimum error, a normal error RMS, a normal error CEP50, a normal error CEP95, a normal error CEP99, a normal maximum error and a normal minimum error, a file name of first satellite navigation monitoring data, detection time such as 2020, 01 month and 01 day and the like, corresponding product types, namely common products, storage positions of the data storage processing server 101 and other data, and sending a detection report to a manager of the first device to be detected 109 through mails, specifically:
the mail subject can be set according to the YYYYMMDD _ TestReport format, the mail content takes the detection report of the first device to be detected 109 as an attachment or directly as a mail body, the mail sending can be a mail of a detector, and a mail sending mailbox can also be fixedly set.
Preferably, in the above technical solution, the method further comprises: the report mail automatic sending unit 108 sets different reminding levels according to the number of the data abnormal BUGs in each detected report and/or the severity of each data abnormal BUG, specifically:
1) when different reminding levels are set according to the number of the data abnormal BUGs in each detection report, specifically:
the method can be set in one test, when the number of the data abnormal BUGs is less than 10, the third reminding level is set, and words such as 'the number of the data abnormal BUGs is less than 10' and the like are attached to the mail content;
when the number of the data abnormal BUGs is more than 10 and less than 30, the number is a second reminding level, and words such as ' the number of the data abnormal BUGs is more than 10 and less than 30 ' and the like ' can be attached to the mail content;
when the number of the data abnormal BUGs is more than 30, the first reminding level is set, word patterns such as 'the number of the data abnormal BUGs is more than 30 and the data abnormal BUGs are preferentially checked' can be added to the mail content, and a manager can be reminded of preferentially checking the data abnormal BUGs by increasing fonts or thickening the fonts and the like;
2) setting different reminding levels according to the severity of each data abnormal BUG in each detection report, specifically:
the method can be set in one test, any abnormal data BUG hinders the development of a device to be tested and causes the test to be incapable of being carried out continuously, the severity degree is very serious and can also be recorded as the fatality degree, and the method is a first reminding level, can attach the word patterns of 'the very serious or fatal abnormal data BUG exists and the like and preferentially check' in the mail content, and can also remind a manager of preferentially checking in the modes of increasing the font or thickening the font and the like;
although all the data abnormal BUGs affect the function or operation of the equipment to be detected, namely the main functions of the equipment to be detected have serious defects, the stability of the equipment to be detected is not affected, the serious degree of the equipment to be detected is a second reminding grade, the second reminding grade can also be marked as a serious grade, and words such as 'serious data abnormal BUGs' and the like can be added to the mail content;
all the abnormal data BUGs only affect the interface of the equipment to be detected, the performance and the function of the equipment to be detected are slightly affected, the severity of the equipment to be detected is general, a third reminding level is obtained, and words such as 'general data abnormal BUGs exist' can be attached to the mail content;
all data abnormity BUG do not generate influence on the equipment to be detected, the problem of usability and advisability is only put forward to a manager of the equipment to be detected, the severity of the problem is advice, the severity is a fourth reminding level, and words such as 'advising the manager to perform certain processing' and the like can be added to the mail content.
3) Setting different reminding levels according to the number of the data abnormal BUGs in each detection report and the severity of each data abnormal BUG, and in detail:
when any abnormal data BUG is in a fatal degree, the abnormal data BUG is in a first reminding level, and words such as 'serious or fatal abnormal data BUG exists, and the abnormal data BUG is preferably checked' can be added to the mail content, and a manager can be reminded of the preferential checking in a way of increasing fonts or thickening fonts and the like;
when there is no data abnormal BUG to be the fatal degree, the above manner can be referred to, and the grade distribution is performed according to the number of the data abnormal BUGs.
Preferably, in the above technical solution, the data storage processing server 101 further includes a display, and the initial information configuration unit 100 configures the initial information of each device to be detected in the data storage processing server 101 through the display, so as to facilitate the operation of the detection personnel.
In still another embodiment, the satellite navigation mass data automatic analysis and BUG intelligent recognition system is used in a GNSS satellite navigation module vehicle-mounted front-loading project, because the vehicle-mounted front-loading project puts higher requirements on the design, test, reliability and product quality of a product, a large number of internal field and external field test cases and test samples are added for the project, the data size is extremely large, and meanwhile, the project time emergency also puts fast, accurate and efficient requirements on the analysis of the product performance, specifically:
firstly, by adopting the automatic analysis of the satellite navigation mass data and the mode of comparing and verifying the BUG intelligent identification system with the traditional manual mode, the test and verification of about one month time are carried out, the processing efficiency of the satellite navigation mass data is improved by 12 times in comparison with the manual mode, the accumulated processing data volume is 120GB, the single-day processing data volume is about 4G, the processing time of the satellite navigation mass data is less than 5 minutes, and the manual processing time exceeds 1 hour.
Compared with a manual method, the accuracy and efficiency of the BUG recognition are obviously improved, the accuracy of the BUG recognition is equivalent to that of the BUG recognition by manual processing, but the efficiency is improved by more than 10 times compared with the manual method. In the embodiment, for one month of data processing, the system identifies the number of data abnormality BUGs as 36, wherein the number of data abnormality BUGs is 6 for the severity, 24 for the general degree, and 6 for the recommended degree.
The invention is implemented and tested and verified on the vehicle-mounted front-mounted project, so that the advantages of the system in the aspects of data processing efficiency, BUG identification and the like are proved, and a plurality of projects are applied to the invention at present, so that the verification cost is greatly saved, and the testing efficiency and the accuracy of BUG identification are improved.
The automatic analysis and BUG intelligent recognition system for the satellite navigation mass data can greatly improve the analysis efficiency of satellite navigation test working data, and meanwhile, the BUG recognition can assist in quickly and effectively recognizing design defects in the algorithm development process. The method comprises the following specific steps:
1) after the application is adopted, the satellite navigation data are summarized and grouped by the hardware resources of the high-efficiency data storage processing server 101 and the data segmentation storage technology in the data storage stage, namely a preset storage mode, the data preprocessing unit adopts the technologies of rapid data browsing, key element identification and the like, and the data analysis processing unit is operated in parallel aiming at each satellite navigation data in each subgroup, and the operation efficiency of the satellite navigation data processing tool or software and other systems in the market is more than 10 times of that of the traditional data analysis, namely the manual mode;
2) through abnormal data BUG identification in the application, when satellite navigation data analysis is carried out, configuration parameters in configuration files are loaded, and the configuration parameters specifically comprise: the method is characterized in that abnormal configurations of different product types are set, design defects of products can be effectively and rapidly identified in the algorithm development process, nearly 100 BUGs are proposed to report for testing abnormity judgment of time service products, inertial navigation products and high-precision products compared with manual analysis since the system operates at present, and the accuracy of reported abnormal data BUGs reaches over 95% through analysis of research and development personnel.
3) This application adopts the index technique of fully transferring the database promptly data mark construction unit 106, realizes the convenience of satellite navigation data backtracking in satellite navigation data storage, abnormal data BUG location, compares the mode that traditional tester finds abnormal data BUG and then provides the satellite navigation data that test data was gathered promptly, and the efficiency of analysis is again improved more than 3 times to the backtracking of satellite navigation data in this application.
As shown in fig. 2, the method for automatically analyzing satellite navigation mass data and intelligently identifying a BUG according to the embodiment of the present invention includes the following steps:
s1, configuring corresponding initial information in the data storage processing server according to the connected equipment to be detected with different product types;
s2, in the detection process, enabling each device to be detected to respectively send the acquired satellite navigation monitoring data with different data formats to the storage processing server for storage according to corresponding initial information;
s3, decoding and BUG analyzing each satellite navigation monitoring data to obtain a corresponding detection result, and feeding back all the detection results to a manager of each device to be detected according to the corresponding initial information.
When a plurality of devices to be detected are detected simultaneously, the satellite navigation monitoring data acquired by each device to be detected are decoded and BUG analyzed to obtain corresponding detection results, and all the detection results are fed back to corresponding managers according to the corresponding initial information. Therefore, the intelligent and automatic method for simultaneously detecting the devices to be detected with different product types and different data formats and automatically identifying the data abnormal BUG in the satellite navigation data is realized.
Preferably, in the above technical solution, the method further comprises the following steps:
naming each satellite navigation monitoring data according to a preset naming standard and storing the named data according to a preset storage mode;
and detecting the residual disk space of the storage processing server, and if the residual disk space is smaller than a preset threshold, sending alarm information of insufficient residual disk space.
Preferably, in the above technical solution, the method further comprises the following steps:
identifying the data format of each satellite navigation monitoring data, classifying the satellite navigation monitoring data with the same data format into the same group, and marking each group;
and identifying different product types corresponding to each satellite navigation monitoring data in each group, classifying the satellite navigation monitoring data of the same product type into corresponding subgroup, and marking each subgroup.
Preferably, in the above technical solution, the method further comprises the following steps:
and reading the satellite navigation monitoring data corresponding to each subgroup in parallel in a binary single byte data reading mode, decoding each satellite navigation monitoring data by using a coding mode of a data format corresponding to the satellite navigation monitoring data to generate a plurality of satellite navigation decoding data correspondingly, and performing BUG analysis on each satellite navigation decoding data.
Preferably, in the above technical solution, the method further comprises the following steps:
and reading monitoring data of each index of satellite navigation in each satellite navigation decoding data, judging whether the monitoring data of each index is in a standard data range corresponding to each index, if not, judging that data abnormity BUG exists in the monitoring data, and calling a corresponding preprocessing scheme corresponding to each data abnormity BUG from a preprocessing scheme library for processing.
Preferably, in the above technical solution, the method further comprises the following steps:
and counting the test information of each device to be detected and the error information of each index, wherein the test information comprises test duration, epoch number, positioning satellite number, positioning proportion and de-positioning proportion, and the error information comprises effective positioning rate, horizontal error and normal error.
Preferably, in the above technical solution, the method further comprises the following steps:
and counting the name, the detection time, the file size, the corresponding product type and the storage position of each satellite navigation monitoring data in the data storage processing server.
Preferably, in the above technical solution, the method further comprises the following steps:
and generating a detection report corresponding to each device to be detected, namely a detection result, according to the test information of each device to be detected, the error information of each index and the data exception BUG.
Preferably, in the above technical solution, the method further comprises the following steps:
the initial information comprises an outbox and an inbox of a manager of each device to be detected, and each detection report is automatically sent to the corresponding manager through an email.
Preferably, in the above technical solution, the method further comprises the following steps:
and setting different reminding levels according to the number of the data abnormal BUGs in each detection report and/or the severity of each data abnormal BUG.
Preferably, in the above technical solution, the method further comprises the following steps: the data storage processing server further comprises a display, and initial information of each device to be detected is configured in the data storage processing server through the display.
The above steps for implementing the corresponding functions in the method for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG according to the present invention refer to the above parameters, modules and units in the embodiment of the system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG, which are not described herein again.
In the present invention, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A satellite navigation mass data automatic analysis and BUG intelligent identification system is characterized by comprising an initial information configuration unit, a data storage processing server and a full-automatic processing module running in the data storage processing server;
the initial information configuration unit configures corresponding initial information in the data storage processing server according to the connected equipment to be detected with different product types;
in the detection process, each device to be detected respectively sends the acquired satellite navigation monitoring data with different data formats to the storage processing server for storage according to corresponding initial information;
and the full-automatic processing module decodes and BUG analyzes each satellite navigation monitoring data to obtain a corresponding detection result, and feeds back all the detection results to a manager of each device to be detected according to the corresponding initial information.
2. The system for automatically analyzing satellite navigation mass data and intelligently identifying the BUG according to claim 1, wherein the full automation processing module comprises a data storage unit;
the data storage unit names each satellite navigation monitoring data according to a preset naming standard and stores the satellite navigation monitoring data according to a preset storage mode;
the data storage unit also detects the residual disk space of the storage processing server, and if the residual disk space is smaller than a preset threshold, alarm information of insufficient residual disk space is sent out.
3. The system for automatically analyzing satellite navigation mass data and intelligently identifying the BUG according to claim 2, wherein the full automation processing module further comprises a data preprocessing unit;
the data preprocessing unit identifies the data format of each satellite navigation monitoring data, puts the satellite navigation monitoring data with the same data format into the same group, and marks each group;
the data preprocessing unit also identifies different product types corresponding to each satellite navigation monitoring data in each group, puts the satellite navigation monitoring data of the same product type into corresponding subgroup, and marks each subgroup.
4. The system for automatically analyzing satellite navigation mass data and intelligently identifying the BUG according to claim 3, wherein the full automation processing module further comprises a data analysis processing unit;
the data analysis processing unit reads the satellite navigation monitoring data corresponding to each subgroup in parallel in a binary single byte data reading mode, decodes each satellite navigation monitoring data in a coding mode of a corresponding data format to generate a plurality of satellite navigation decoding data correspondingly, and then performs BUG analysis on each satellite navigation decoding data.
5. The system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG according to claim 4, further comprising: the data analysis processing unit reads monitoring data of each index of satellite navigation in each satellite navigation decoding data, judges whether the monitoring data of each index is in a standard data range corresponding to each index, if not, judges that data abnormity BUG exists in the monitoring data, and calls a preprocessing scheme corresponding to each data abnormity BUG from a preprocessing scheme library for processing.
6. The system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG according to claim 5, further comprising: the data analysis processing unit also counts the test information of each device to be tested and the error information of each index, the test information comprises test duration, epoch number, positioning satellite number, positioning proportion and de-positioning proportion, and the error information comprises effective positioning rate, horizontal error and normal error.
7. The system for automatically analyzing satellite navigation mass data and intelligently identifying the BUG according to claim 6, wherein the full automation processing module further comprises a data mark construction unit;
and the data mark construction unit counts the name, the detection time, the file size, the corresponding product type and the storage position of each satellite navigation monitoring data in the data storage processing server.
8. The system for automatically analyzing satellite navigation mass data and intelligently identifying the BUG according to claim 7, wherein the full automation processing module further comprises a data report automatic generation unit and a report mail automatic sending unit;
the data report automatic generation unit generates a detection report, namely a detection result, corresponding to each device to be detected according to the information output by the data analysis processing unit and the data mark construction unit;
the initial information comprises an outbox and an inbox of a manager of each device to be detected, and the report mail automatic sending unit automatically sends each detected report to the corresponding manager.
9. The system for automatically analyzing the satellite navigation mass data and intelligently identifying the BUG according to claim 7, further comprising: and the report mail automatic sending unit sets different reminding levels according to the number of the data abnormal BUGs and/or the severity of each data abnormal BUG in each detection report.
10. The system for automatically analyzing satellite navigation mass data and intelligently identifying BUG according to any one of claims 1 to 9, wherein the data storage processing server further comprises a display, and the initial information configuration unit configures the initial information of each device to be detected in the data storage processing server through the display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010042189.6A CN111242581A (en) | 2020-01-15 | 2020-01-15 | Automatic analysis and BUG intelligent identification system for satellite navigation mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010042189.6A CN111242581A (en) | 2020-01-15 | 2020-01-15 | Automatic analysis and BUG intelligent identification system for satellite navigation mass data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111242581A true CN111242581A (en) | 2020-06-05 |
Family
ID=70872369
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010042189.6A Withdrawn CN111242581A (en) | 2020-01-15 | 2020-01-15 | Automatic analysis and BUG intelligent identification system for satellite navigation mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111242581A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220721A (en) * | 2021-04-06 | 2021-08-06 | 中国人民解放军63921部队 | Mass data stream processing method and device for satellite navigation monitoring system |
-
2020
- 2020-01-15 CN CN202010042189.6A patent/CN111242581A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220721A (en) * | 2021-04-06 | 2021-08-06 | 中国人民解放军63921部队 | Mass data stream processing method and device for satellite navigation monitoring system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109947746B (en) | Data quality control method and system based on ETL flow | |
| CN106294134B (en) | The collapse localization method and device of code | |
| CN104202201B (en) | A kind of log processing method, device and terminal | |
| WO2021218178A1 (en) | Automatic report generation method, device, computer apparatus, and storage medium | |
| CN111240876B (en) | Fault positioning method and device for micro-service, storage medium and terminal | |
| CN101188523A (en) | Generation method and generation system of alarm association rules | |
| US11799748B2 (en) | Mitigating failure in request handling | |
| CN109885453B (en) | Big data platform monitoring system based on stream data processing | |
| CN107193714B (en) | Alarm display method and device | |
| CN107659443A (en) | The monitoring method and its system of a kind of real time business | |
| CN111385148A (en) | Service alarm method, device, computer equipment and storage medium | |
| CN109885537B (en) | Log display method, system and computer readable storage medium | |
| CN114648393A (en) | Data mining method, system and equipment applied to bidding | |
| CN112380533A (en) | Method for checking security baseline of computer terminal | |
| CN110647472A (en) | Breakdown information statistical method and device, computer equipment and storage medium | |
| CN111242581A (en) | Automatic analysis and BUG intelligent identification system for satellite navigation mass data | |
| CN111061802A (en) | Power data management processing method and device and storage medium | |
| CN107968727A (en) | A kind of detection method, device and the medium of CIFS services | |
| CN110717130B (en) | Dotting method, dotting device, dotting terminal and storage medium | |
| CN112241362A (en) | Test method, test device, server and storage medium | |
| CN112486841A (en) | Method and device for checking data collected by buried point | |
| CN113535034B (en) | Method, apparatus, system and medium for fault warning | |
| CN112116314B (en) | Sample weighing data management system | |
| CN112559368A (en) | Interface automation test method and its use case generation method, equipment and storage medium | |
| KR101377241B1 (en) | System and method for analyzing trip criticality by failure of equipment in nuclear power plant |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200605 |
|
| WW01 | Invention patent application withdrawn after publication |