CN110147370B - Train data storage method based on producer or consumer task scheduling mode - Google Patents
Train data storage method based on producer or consumer task scheduling mode Download PDFInfo
- Publication number
- CN110147370B CN110147370B CN201910350764.6A CN201910350764A CN110147370B CN 110147370 B CN110147370 B CN 110147370B CN 201910350764 A CN201910350764 A CN 201910350764A CN 110147370 B CN110147370 B CN 110147370B
- Authority
- CN
- China
- Prior art keywords
- task
- running
- data
- receiving
- producer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000013500 data storage Methods 0.000 title claims abstract description 17
- 238000012544 monitoring process Methods 0.000 claims abstract description 29
- 238000007726 management method Methods 0.000 claims abstract description 16
- 238000012216 screening Methods 0.000 claims description 14
- 239000003990 capacitor Substances 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 238000004092 self-diagnosis Methods 0.000 claims description 3
- 230000001934 delay Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1479—Generic software techniques for error detection or fault masking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/40—Business processes related to the transportation industry
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a train data storage method based on a producer or consumer task scheduling mode, which is designed according to a software structure and prevents a plurality of tasks from simultaneously accessing the same shared cache queue through semaphore, and the method specifically comprises the following steps: 1) an initialization stage: the initialization task completes the initialization of software and hardware environment, generates a receiving task, a storage task and a monitoring task according to configuration data, or generates an interception task, performs file management initialization according to files on a disk, and starts clock interruption; 2) and (3) a normal operation stage: and running the receiving task, the storing task and the monitoring task in a normal mode, and running the intercepting task in an intercepting mode. Compared with the prior art, the invention has the advantages of high efficiency and the like.
Description
Technical Field
The invention relates to a train data storage method, in particular to a train data storage method based on a producer or consumer task scheduling mode.
Background
The state data generated by each subsystem in the running process of the train plays an important role in guaranteeing the safe operation of the train, the state data is reliably and efficiently stored, and important guarantee can be provided for evaluating the health condition of the train and analyzing the cause of train failure.
Because the number of subsystems of the train is large, the covered line is long in the running process, the running period is long, the generated data volume is very large, the capacity of the train storage equipment is limited, and abnormal data storage or data loss is easy to occur if an effective storage method is not adopted. If the storage device does not have power failure protection measures under the condition of power failure of the train, data cannot be completely stored, and data loss is caused.
At present, the FLASH storage technology has the characteristics of large storage, good performance, low energy consumption and the like, and is widely applied to train storage equipment, but the defect that data is easy to lose when power fails exists, and an effective storage task management mechanism is required to be relied on for ensuring.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a train data storage method based on a producer or consumer task scheduling mode.
The purpose of the invention can be realized by the following technical scheme:
a train data storage method based on a producer or consumer task scheduling mode is designed according to a software structure, and prevents a plurality of tasks from simultaneously accessing the same shared cache queue through semaphores, and specifically comprises the following steps:
1) an initialization stage:
the initialization task completes the initialization of software and hardware environment, generates a receiving task, a storage task and a monitoring task according to configuration data, or generates an interception task, performs file management initialization according to files on a disk, and starts clock interruption;
2) and (3) a normal operation stage:
and running the receiving task, the storing task and the monitoring task in a normal mode, and running the intercepting task in an intercepting mode.
Preferably, the file management initialization includes a filtered file management initialization and a complete file management initialization.
Preferably, the receiving task operation specifically includes:
firstly, waiting for receiving semaphore resources required by task operation, generating self diagnosis information according to previous-period operation data after a clock is interrupted to release the semaphore resources, managing a time synchronization state, receiving an operation log message packet sent by vehicle-mounted equipment, processing the data packet, and updating an equipment connection state; receiving a task as a producer, and writing the diagnostic information and the running log into a cache queue; and releasing the semaphore resource of the storage task after the task is executed.
Preferably, the storage task operation specifically includes:
firstly, waiting for storing semaphore resources required by task running, executing disk management after receiving the semaphore resources released by the task, taking the stored task as a consumer, reading the diagnostic information and the running log in the cache queue, and storing the diagnostic information and the running log in a disk file.
Preferably, the monitoring task operation specifically includes:
firstly, waiting for semaphore resources required by the running of a monitoring task, taking the monitoring task as a consumer to access a disk after a clock is interrupted to release the semaphore resources, adding a complete file into a complete file queue and monitoring the number of files in a screening queue, and adding the screening files in the disk into the screening queue when the number of files is smaller than a set threshold value.
Preferably, the screening file is only stored for one day, so that the disk space sufficiency is ensured.
Preferably, the monitoring task is restarted when the receiving task or the storing task is found to be in the suspend state.
Preferably, the interception task operation specifically includes:
and acquiring semaphore resources required by the running of the SPY task, copying data in the array buffer queue to a data buffer, and storing the data in the data buffer to a file.
Preferably, the method includes the steps that a magnetic disk cannot be written in time after a chassis is powered off in order to prevent the received data volume from being large, a capacitor is installed in a hardware board card, the equipment is powered by the capacitor after the chassis is powered off to delay the running time of 1s, and the software running period is 100ms, so that the integrity of data storage of the chassis after the chassis is powered off is guaranteed.
Compared with the prior art, the invention has the following advantages:
1. high efficiency: and the task management is realized through a producer/consumer task scheduling mechanism, and the efficiency is high.
2. Reliability: the work can be delayed after power failure, so that the data can be stored in the file in the program running period, and the data loss is avoided.
3. The storage space utilization rate is high: the stored data are screened, the storage time of the non-important data is short, and the storage integrity of the important data is ensured.
Drawings
FIG. 1 is a schematic diagram of a software architecture suitable for use with the present invention;
FIG. 2 is a flow chart of an initialization phase of the present invention;
FIG. 3 is a flow chart of the main task phase of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of the present invention.
The invention is designed according to a software structure, prevents a plurality of tasks from simultaneously accessing the same shared cache queue through semaphore, and carries out data error storage through a task scheduling mode of a producer/consumer mode, wherein the data error storage is mainly divided into the following two stages:
an initialization stage:
the initialization task completes initialization of software and hardware environments, generates a receiving task, a storage task and a monitoring task according to configuration data, or generates an interception task, performs file management initialization including screened files and complete files according to files on a disk, and starts clock interruption.
And (3) a normal operation stage:
and running the receiving task, the storing task and the monitoring task in a normal mode, and running the intercepting task in an intercepting mode.
The receiving task firstly waits for receiving semaphore resources required by the task operation, generates self diagnosis information according to the previous period operation data after a clock is interrupted to release the semaphore resources, manages a time synchronization state, receives an operation log message packet sent by the vehicle-mounted equipment, processes the data packet, and updates the equipment connection state. The receiving task serves as a producer and writes the diagnostic information and the running log into a cache queue. And releasing the semaphore resource of the storage task after the task is executed.
The storage task firstly waits for semaphore resources required by the running of the storage task, executes disk management after receiving the semaphore resources released by the task, takes the storage task as a consumer, reads the diagnostic information and the running log in the cache queue and stores the diagnostic information and the running log in a disk file.
The method comprises the steps that a monitoring task firstly waits for semaphore resources required by the running of the monitoring task, the monitoring task serves as a consumer to access a disk after a clock is interrupted to release the semaphore resources, complete files are added to a complete file queue, the number of the files in a screening queue is monitored, when the number of the files is smaller than a set threshold value, the screening files in the disk are added into the screening queue, and the screening files are only stored for one day, so that the space sufficiency of the disk is guaranteed. The software is restarted when either the receive task or the store task is found to be in the suspend state.
The interception task acquires semaphore resources required by the running of the SPY task, copies data in the array cache queue into the data cache, and stores the data in the data cache into a file
In order to prevent the received data from being large in size and incapable of being written into a disk in time after the chassis is powered off, a capacitor is installed in a hardware board card, the device delays the running time for 1s by utilizing the power supply of the capacitor after the chassis is powered off, and the software running period is 100ms, so that the integrity of data storage after the chassis is powered off is guaranteed.
The receiving task is driven by a clock interrupt, and the relationship between the receiving task and the storing task is as follows:
1) receiving a task waiting semaphore resource;
2) storing a task waiting semaphore resource;
3) clock interruption releases semaphore resources;
4) receiving a task to obtain semaphore resources, executing the task in the period, and releasing the semaphore resources of the storage task after the execution is finished;
5) and the storage task obtains the semaphore resource and executes the task in the period.
The monitoring task is driven by a clock interrupt:
1) monitoring a task waiting semaphore resource;
2) clock interruption releases semaphore resources;
3) and the monitoring task obtains semaphore resources and executes the periodic task.
The interception task obtains intercepted IP data packets from the network tasks, and the relationship between the two tasks is as follows:
1) the network task stores the detected data packet into an array cache queue, and releases the semaphore when the cache node in the queue is full;
2) and after the interception task obtains the semaphore, storing the data in the cache node into a data cache for processing.
FIGS. 2 and 3 show a specific process of the present invention. With reference to fig. 1, fig. 2, and fig. 3, the following embodiments of the present invention are described by taking the vehicle-mounted storage software DLU to which the present invention is applied as an example:
as shown in fig. 1, for a system structure suitable for this solution, the initialization task is responsible for initializing the software and hardware environment of the DLU and generating each task. The receiving task is responsible for functions of receiving, processing and time synchronization of the diagnostic information. The storage task is responsible for managing the disk storage space and storing the diagnostic data received by the receiving task to the disk. And when the state monitoring task finds that the receiving task or the storing task is in a suspend state, the DLU software is restarted, and meanwhile, when the number of the files in the screening file queue is smaller than a set threshold value, the corresponding files in the directory are added into the screening file queue. The listening task is responsible for receiving listening packets from the network task.
FIG. 2 is a flow chart of the initialization phase of the present invention, described in detail as follows:
step 1: the software hardware environment is initialized.
And 2, step: and creating a monitoring task.
And step 3: a receiving task is created.
And 4, step 4: a storage task is created.
And 5: a listening task is created.
Step 6: a clock interrupt is initiated.
FIG. 3 is a flow chart of the main task phase of the present invention, described in detail as follows:
step 1: judging the current mode state, if the current mode state is the normal mode, entering the step 2, otherwise, entering the step 6;
and 2, step: waiting for the clock to interrupt and release the semaphore resource;
and step 3: and if the semaphore released by the clock interrupt is the monitoring task semaphore, entering the step 4.
No step 5 is entered.
And 4, step 4: the monitoring task obtains semaphore resources, executes disk management, stores diagnostic information and an operation log into a disk file, and simultaneously monitors a receiving task and a storing task;
and 5: receiving a task to obtain semaphore resources, executing the task in the period, storing the received diagnostic information and the generated running log into a cache queue, and releasing the semaphore resources of the stored task after the execution is finished;
and 5: the storage task obtains semaphore resources, executes the periodic task, takes out data in the cache queue and stores the data in a disk file;
and 6: waiting for the network task to release the semaphore;
and 7: and after the interception task obtains the semaphore, storing the data in the cache node into a cache queue for processing.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (6)
1. A train data storage method based on a producer or consumer task scheduling mode is characterized in that the method is designed according to a software structure, and prevents a plurality of tasks from simultaneously accessing the same shared cache queue through semaphores, and the method specifically comprises the following steps:
1) an initialization stage:
the initialization task completes the initialization of software and hardware environment, generates a receiving task, a storage task and a monitoring task according to configuration data, or generates a monitoring task, initializes file management according to files on a disk, and starts clock interruption;
2) and (3) a normal operation stage:
running a receiving task, a storing task and a monitoring task in a normal mode, and running an interception task in an interception mode;
the file management initialization comprises screened file management initialization and complete file management initialization;
the monitoring task operation specifically comprises the following steps:
firstly, waiting for semaphore resources required by the running of a monitoring task, taking the monitoring task as a consumer to access a disk after a clock is interrupted to release the semaphore resources, adding a complete file to a complete file queue and monitoring the number of files in a screening queue, and adding screening files in the disk into the screening queue when the number of files is smaller than a set threshold value;
the screening files are only stored for one day, so that the disk space sufficiency is ensured.
2. The train data storage method based on the producer or consumer task scheduling mode according to claim 1, wherein the receiving task operation specifically comprises:
firstly, waiting for receiving semaphore resources required by task operation, generating self diagnosis information according to previous-period operation data after a clock is interrupted to release the semaphore resources, managing a time synchronization state, receiving an operation log message packet sent by vehicle-mounted equipment, processing the data packet, and updating an equipment connection state; receiving a task as a producer, and writing the diagnostic information and the running log into a cache queue; and releasing the semaphore resource of the storage task after the task is executed.
3. The train data storage method based on the producer or consumer task scheduling mode according to claim 1, wherein the storage task operation specifically comprises:
firstly, waiting for storing semaphore resources required by task running, executing disk management after receiving the semaphore resources released by the task, taking the stored task as a consumer, reading the diagnostic information and the running log in the cache queue, and storing the diagnostic information and the running log in a disk file.
4. The train data storage method based on the producer or consumer task scheduling mode as claimed in claim 1, wherein the monitoring task is restarted when the receiving task or the storing task is found to be in a suspend state.
5. The train data storage method based on the producer or consumer task scheduling mode according to claim 1, wherein the listening task operation specifically comprises:
and acquiring semaphore resources required by the running of the SPY task, copying data in the array buffer queue to a data buffer, and storing the data in the data buffer to a file.
6. The train data storage method based on the producer or consumer task scheduling mode as claimed in claim 1, wherein in order to prevent that the received data volume is large and the data cannot be written into a disk in time after the power failure of the chassis, a capacitor is installed in a hardware board card, the device delays the running time of 1s by using the power supplied by the capacitor after the power failure of the chassis, and the software running period is 100ms, so that the integrity of data storage after the power failure of the chassis is ensured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910350764.6A CN110147370B (en) | 2019-04-28 | 2019-04-28 | Train data storage method based on producer or consumer task scheduling mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910350764.6A CN110147370B (en) | 2019-04-28 | 2019-04-28 | Train data storage method based on producer or consumer task scheduling mode |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110147370A CN110147370A (en) | 2019-08-20 |
| CN110147370B true CN110147370B (en) | 2022-08-30 |
Family
ID=67594055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910350764.6A Active CN110147370B (en) | 2019-04-28 | 2019-04-28 | Train data storage method based on producer or consumer task scheduling mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110147370B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8812875B1 (en) * | 2010-04-12 | 2014-08-19 | Stephen Melvin | Virtual self-destruction of stored information |
| CN106685856A (en) * | 2016-12-30 | 2017-05-17 | 国网浙江省电力公司绍兴供电公司 | Method, system and equipment for processing intelligent substation mass network messages |
-
2019
- 2019-04-28 CN CN201910350764.6A patent/CN110147370B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8812875B1 (en) * | 2010-04-12 | 2014-08-19 | Stephen Melvin | Virtual self-destruction of stored information |
| CN106685856A (en) * | 2016-12-30 | 2017-05-17 | 国网浙江省电力公司绍兴供电公司 | Method, system and equipment for processing intelligent substation mass network messages |
Non-Patent Citations (2)
| Title |
|---|
| Reducing misses to external memory accesses in task-level pipelining;Ali Azarian et al.;《2015 IEEE International Symposium on Circuits and Systems (ISCAS)》;20150730;全文 * |
| 基于MongoDB的光纤传感海量监测数据存储方法研究;易诗;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20180115;第2018年卷(第1期);第I137-50页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110147370A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108664496B (en) | Data migration method and device | |
| US7698602B2 (en) | Systems, methods and computer products for trace capability per work unit | |
| US8132043B2 (en) | Multistage system recovery framework | |
| US8132190B2 (en) | Predictive method for managing logging or replaying non-deterministic operations within the execution of an application process | |
| US8539434B2 (en) | Method for the management, logging or replay of the execution of an application process | |
| US9098439B2 (en) | Providing a fault tolerant system in a loosely-coupled cluster environment using application checkpoints and logs | |
| US11526411B2 (en) | System and method for improving detection and capture of a host system catastrophic failure | |
| US8219851B2 (en) | System RAS protection for UMA style memory | |
| US10146653B2 (en) | Automated system-level failure and recovery | |
| US7962782B2 (en) | Modifying connection records | |
| JP2012533796A5 (en) | ||
| JP2012503826A (en) | Evaluating the effectiveness of memory management techniques that use selective mitigation to reduce errors | |
| Liu et al. | Optimizing shuffle in wide-area data analytics | |
| JP4322240B2 (en) | Reboot method, system and program | |
| CN111858077A (en) | Recording method, device and equipment for IO request log in storage system | |
| US9015535B2 (en) | Information processing apparatus having memory dump function, memory dump method, and recording medium | |
| CN107943567B (en) | High-reliability task scheduling method and system based on AMQP protocol | |
| CN110147370B (en) | Train data storage method based on producer or consumer task scheduling mode | |
| JP2007242010A (en) | Analytic computer system for logging information collected by common logging | |
| US20070174688A1 (en) | Method for optimizing the transmission of logging data in a multi-computer environment and a system implementing this method | |
| US20170091038A1 (en) | Fault tolerant event management system | |
| US20180309702A1 (en) | Method and device for processing data after restart of node | |
| CN115562933A (en) | Processing method and device of operation monitoring data, storage medium and electronic equipment | |
| CN115391106A (en) | Method, system and device for pooling backup resources | |
| CN112286727A (en) | Space-time isolation domain fast recovery method and system based on incremental snapshot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |