CN108107750B - Real-time IO data processing method and system for power system simulation - Google Patents
Real-time IO data processing method and system for power system simulation Download PDFInfo
- Publication number
- CN108107750B CN108107750B CN201711224511.1A CN201711224511A CN108107750B CN 108107750 B CN108107750 B CN 108107750B CN 201711224511 A CN201711224511 A CN 201711224511A CN 108107750 B CN108107750 B CN 108107750B
- Authority
- CN
- China
- Prior art keywords
- fifo
- file
- thread
- real
- socket
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B17/00—Systems involving the use of models or simulators of said systems
- G05B17/02—Systems involving the use of models or simulators of said systems electric
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Multi Processors (AREA)
Abstract
The application provides a real-time IO data processing method and a system for power system simulation, wherein the method comprises the following steps: when the power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread; and the calculation thread writes the result file through the IO process and carries out bidirectional communication with the upper computer interface. The application provides an independent IO program which is communicated with a real-time simulation calculation program to complete IO operation required by the real-time calculation program, and realizes that a real-time calculation thread writes a result file through an IO process and carries out bidirectional communication with an upper computer interface.
Description
Technical Field
The application relates to power system simulation, in particular to a real-time IO data processing method and system for power system simulation.
Background
In the real-time simulation calculation process of the power system, necessary data IO operations, such as file reading and writing, communication with a control interface and the like, need to be performed. Since these data IOs belong to non-real-time operations and cannot be completed within a certain time, they cannot be put into the real-time computing process and must be put into a special data IO process or thread.
In a common CLUSTER CLUSTER, an RTLINUX operating system is used, real-time calculation and IO operation are two threads of one real-time thread, real-time calculation is a real-time thread, IO operation threads are non-real-time threads, and a real-time IO interface between the two threads is realized through RTFIFO provided by RTLINX.
In an SGI parallel machine simulation platform, no suitable thread real-time control method is found at present, one thread in a process is set as a real-time thread, and an operating system ensures the real-time performance; the other thread is a non-real-time thread, and the calculation time can be divided into some when the real-time thread is not busy.
Therefore, in the SGI real-time calculation, the IO interface scheme on the CLUSTER cannot be simply applied, and needs to be realized again.
Disclosure of Invention
In order to solve the defects in the prior art, the application provides a real-time IO data processing method and system for power system simulation, which realize an independent IO program, communicate with a real-time simulation calculation program and complete IO operation required by the real-time calculation program. And can run on two parallel machine simulation platforms, namely SGI and CLUSTER.
The technical scheme provided by the application is as follows: a real-time IO data processing method for power system simulation, the method comprising:
when the power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
and the calculation thread writes the result file through the IO process and carries out bidirectional communication with the upper computer interface.
Preferably, the calculating thread writes the result file through the IO process and performs bidirectional communication with the upper computer interface, including:
the computing thread constructs a write file FIFO based on a basic first-in first-out queue FIFO through an IO process, and the IO process is realized to write the executing process of the computing thread into a file;
the computing thread constructs a control FIFO based on the basic FIFO to realize the establishment, real-time communication and ending communication of communication between the IO process and the computing thread;
and the calculation thread constructs a communication FIFO based on the basic FIFO through the IO process, so that the calculation thread and the upper computer interface are in bidirectional communication.
Preferably, the data contents transmitted in the control FIFO, the write file FIFO and the communication FIFO are packaged according to the message, and share the data communication channel and process according to the message type.
Preferably, the computing thread constructs a write file FIFO based on a FIFO by an IO process, and the IO process writes the computing thread execution process into a file, including:
the IO process registers the write file FIFO based on the register write file FIFO command sent by the computing thread;
the IO process opens the write file FIFO based on an open file IO command sent by the computing thread;
the IO process writes the message content into the write file FIFO based on a write file command sent by the computing thread;
the IO process closes the corresponding file based on a file closing command sent by the computing thread.
Preferably, the IO process registers the write FIFO based on a register write FIFO command sent by the computing thread, including:
the computing thread sends a register write file FIFO command to the IO process, and the IO process registers corresponding resources for the write file FIFO after receiving the register write file FIFO command;
returning a corresponding registration ID through an IO process return command;
when the file IO registration fails, the IO process return command further includes: error information.
Preferably, the IO process opens the write file FIFO based on an open file IO command sent by a computing thread, including:
before the computing thread needs to write a file FIFO, sending a file opening IO command to an IO process, and opening a file by using a resource registered by the file writing FIFO;
and after receiving the file opening IO command, the IO process tries to open the write file FIFO and returns a mark whether the file FIFO is successfully opened or not.
Preferably, the writing, by the IO process, the message content into the write file FIFO based on a write file command sent by the computing thread includes:
after the IO process opens a file, the computing thread sends a file writing command to the IO process;
after the IO process receives the file writing command, writing the message content into the file writing FIFO;
if the writing of the file fails, an error identification is returned.
Preferably, the IO process closes the corresponding file based on a close file command sent by the computing thread, including:
after the IO process finishes writing the file, the computing thread sends a file closing command to the IO process;
and after receiving the file closing command, the IO process tries to close the corresponding file and returns an identification of whether the file is successfully closed or not.
Preferably, the computing thread constructs a control FIFO based on the basic FIFO, comprising:
after the computing thread is started, when the computing thread uses data transmitted in non-real time, the computing thread constructs a control FIFO; the control FIFO includes: a pair of elementary FIFOs;
the IO process establishes and maintains a dynamic array for each basic FIFO;
the IO process determines the size of the dynamic array in the IO process according to the number of the computing threads;
the IO process writes the message communicated by the computing thread into a first array through the control FIFO;
and the IO process reads message data communicated with an upper computer interface in a second array, and sends the read data to the corresponding calculation thread through a control FIFO.
Preferably, the communication message includes a data header and data content;
the data header includes a data type and a length;
the data content comprises: a data content length and a data content format; the data content format is determined according to the data type.
Preferably, the computing thread constructs a communication FIFO based on a basic FIFO through an IO process, including:
the computing thread constructs a pair of SOCKET FIFOs based on the basic FIFO through the IO process;
when the communication is carried out with an upper computer interface, the IO process presses the information in the first array into a buffer area of a first SOCKET FIFO, and sends the information to the upper computer interface through SOCKET, so that the communication between the calculation thread and the upper computer interface is realized;
before the IO process monitors that the upper computer interface transmits a control command, the IO process establishes a link with the upper computer interface;
and the IO process writes the message data sent by the upper computer interface into the second array through a second SOCKET FIFO, and sends the message data to the responding computing thread through the control FIFO.
Preferably, the socks FIFO adopts a non-blocking communication mode, and if the buffer is full, no data is transmitted any more.
Preferably, the computing thread constructs a pair of socks FIFOs based on the basic FIFOs through the IO process, including:
the IO process registers SOCKET FIFO on the shared FIFO based on the register SOCKET FIFO command sent by the computing thread;
the IO process opens the SOCKET FIFO based on an SOCKET IO opening command sent by the computing thread;
the IO process writes the message content into the SOCKET based on a write file command sent by a computing thread;
and the IO process closes the SOCKET FIFO based on a closing file command sent by the computing thread.
Preferably, the IO process registers a SOCKET FIFO based on a register SOCKET FIFO command sent by the computing thread, including:
the computing thread sends a registration SOCKET FIFO command to the IO process;
after receiving the SOCKET FIFO command, the IO process registers corresponding resources for the SOCKET FIFO and returns the registration ID of the SOCKET FIFO;
if registration fails, an error message flag is returned.
Preferably, the IO process opens the SOCKET FIFO based on an open SOCKET IO command sent by the computing thread, including:
before the computing thread needs SOCKET IO, sending a link establishment command to the IO process, and opening the SOCKET link by using registered resources;
after receiving the link establishment command message, the IO process tries to open the corresponding SOCKET connection with the upper computer and returns a mark whether the connection is successfully opened or not;
the interface allows SOCKET FIFO to be opened in the real-time calculation process to perform SCOCKET writing operation;
if the calculation thread receives the SOCKET opening failure message after a plurality of steps, the SOCKET opening failure message is written into an error log.
Preferably, the writing, by the IO process, the message content into the SOCKET based on a write file command sent by the computing thread includes:
after the IO process opens the SOCKET FIFO, the computing thread sends a message to the upper computer through the IO process, and after the IO process receives the message, the IO process writes the message content into the SOCKET;
when the message length is greater than the threshold value, sending the message to an upper computer; if the writing of SOCKET fails, a failure identifier is returned.
Preferably, the IO process closes the SOCKET FIFO based on a close file command sent by a computing thread, including:
after the SOCKET communication is finished, the computing thread sends a closing file command to the IO process to close the SOCKET FIFO; and after receiving the file closing command, the IO process tries to close the SOCKET FIFO and returns an identification of whether the closing is successful or not.
Preferably, the data content includes: and recording result data of each step of simulation calculation, monitored state variable data and control command data sent by an interface.
Preferably, the system comprises:
based on the same conception, the application also provides a real-time IO data processing system for power system simulation, which comprises:
and (3) a building module: the method comprises the steps that when a power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
and an access module: and the computing thread writes the result file through the IO process and carries out bidirectional communication with an upper computer interface.
Preferably, the functional module includes:
writing a file unit: the method comprises the steps of constructing a unidirectional write file FIFO based on a basic FIFO, and enabling the IO process to write the execution process of the computing thread into a file;
and a control unit: the control FIFO is used for constructing a control FIFO based on the basic FIFO to realize the establishment of communication between the IO process and the computing thread, real-time communication and ending communication;
a communication unit: and the system is used for constructing a bidirectional communication FIFO based on the basic FIFO to realize bidirectional communication between the computing thread and the upper computer interface.
Compared with the closest prior art, the technical scheme provided by the application has the following beneficial effects:
according to the technical scheme provided by the application, an independent IO program is provided, a calculation thread writes a result file through an IO process and carries out two-way communication with an upper computer interface, IO operation required by the real-time calculation program is completed, and the real-time calculation thread writes the result file through the IO process and carries out two-way communication with the upper computer interface.
The technical scheme provided by the application adopts the shared memory as a data exchange mode of the real-time computing thread and the non-real-time IO thread or socket thread, and has the characteristics of highest performance and small influence on the computing program.
According to the technical scheme provided by the application, the message record class is designed based on the basic FIFO function, the debugging information in the real-time simulation process is recorded, and the interference of the debugging information record with the test result is avoided; and can run on two parallel machine simulation platforms, namely SGI and CLUSTER.
Drawings
FIG. 1 is a flow chart of the present application;
FIG. 2 is a schematic diagram illustrating the relationship between an IO process and other related processes according to the present application;
FIG. 3 is a flow diagram of the basic FIFO transmission and reception based on asynchronous MPI of the present application;
FIG. 4 is a schematic diagram of the shared memory organization of the present application;
FIG. 5 is a flow chart of the process of writing the simulation data or the monitoring variable of each step of the real-time thread into the shared memory;
FIG. 6 is a schematic diagram of an IO thread processing flow in accordance with the present application;
FIG. 7 is a schematic diagram of a communication thread processing flow according to the present application.
Detailed Description
For a better understanding of the present application, reference is made to the following description, drawings and examples.
Example 1,
Fig. 1 is a flowchart of a real-time IO data processing method for power system simulation, where the real-time IO data processing method for power system simulation may include:
when the power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
and the calculation thread writes the result file through the IO process and carries out bidirectional communication with the upper computer interface.
In the real-time simulation calculation process, the data IO may include:
(1) And writing a result file and recording a simulation calculation process. The result file may be multiple.
(2) And communicating with the interface program, transmitting the monitored state variable data to the interface, and receiving the control command sent by the interface.
In order to ensure the real-time performance of the simulation calculation, a process is set up to be responsible for data IO according to the characteristics of a real-time operating system, and data exchange is carried out between the process and the calculation process through a shared memory or MPI.
In parallel simulation calculation, a plurality of calculation processes exist, each process may have a data IO requirement, and the calculation process and the IO process have a bidirectional IO relationship. The IO process writes the result file in a single direction and the communication with the monitoring interface is bi-directional. The unidirectional data transmission between the IO process and the computing process is realized by adopting a first-in first-out queue (FIFO), and if bidirectional, two unidirectional FIFOs are needed to be created.
The application may include two functions:
(1) Calculating a process side IO interface function;
(2) And (5) realizing IO process.
First, based on MPI asynchronous communication mode, basic FIFO function is realized, then control FIFO, unidirectional write file FIFO and bidirectional communication FIFO are constructed based on basic FIFO. And the built various FIFO functions are utilized to realize the establishment, real-time communication and ending of IO communication.
Considering that different channels are adopted for control, file output and SOCKET, synchronization among the different channels is difficult, the embodiment of the application can be realized in a channel sharing mode, and data contents transmitted on the channels are packaged according to the messages and processed according to the message types. Compared with the channel separation mode, the communication times are reduced, and the synchronism is ensured.
The application will be described in further detail in this embodiment with reference to the accompanying drawings.
FIG. 2 is a schematic diagram showing the relationship between an IO process and other related processes, wherein the IO process and the plurality of computing threads are in bidirectional communication as shown in FIG. 2; a plurality of computing threads realize unidirectional writing of corresponding result files through IO processes; and the plurality of computing threads realize bidirectional communication with an upper computer interface through the IO process.
In order to ensure the real-time performance of the simulation calculation, a process is independently set up to be responsible for data IO according to the characteristics of a real-time operating system, and data exchange is carried out between the process and the calculation process through a shared memory or MPI.
Fig. 3 is a flow diagram of asynchronous MPI-based basic FIFO transmission and reception, as shown in fig. 3,
the computing thread or IO process realizes data transmission based on basic FIFO, receives data, ends the transmission and ends the receiving flow, as follows:
transmitting data: when data is transmitted, judging whether the last transmission is finished or not, and if not, judging whether the buffer space is enough or not; if yes, processing the cache, and judging whether the cache space is enough after the sent part is marked to be available;
if the buffer space is enough, the data to be buffered is put into the buffer, otherwise, the transmission fails;
judging whether the last transmission is completed or not, if so, transmitting the data, otherwise, discarding the data.
Receiving data: judging whether data are to be received or not, if so, judging whether the buffer space is enough or not, otherwise, continuing waiting;
if the buffer space is enough, the data is successfully received into the buffer, otherwise, the data is failed to be received.
Ending the transmission: and judging whether the transmission of the cache data is completed, if so, ending the transmission, otherwise, ending the transmission after completing the transmission of the cache data.
Ending the receiving: judging whether data is being received, if so, trying to receive the data once, and if not, ending the receiving;
and if the primary data receiving fails, canceling the data receiving and ending the data receiving, otherwise ending the data receiving.
The embodiment also provides a design code for realizing the file writing process and the two-way communication.
1.1, CMPIFIFOBase class design implements basic FIFO functions
The CMPIFIFOBase type package related data, buffer operation, MPI communication and other functions, and provides a basic FIFO interface function.
(1) Principal member (S)
1.2, file IO management
And the file IO management class records information such as file names, file pointers and the like, encapsulates the processes of file creation, file writing and closing, and provides FIFO management class use.
2.1 CMPIPIFIFOFile class design
1.3, SOCKET communication management realizes bidirectional communication between IO process and communication service program
The SOCKET communication management class manages SOCKET port information and encapsulates basic functions of SOCKET establishment, receiving and sending and closing.
5.3.1 CMPIPIFIFOSocket class design
The CMPIF socket type package provides communication establishment, sending, receiving and closing functions of an IO process side and data sending and receiving functions of a computing program side.
The CMPIF foSocket adopts a non-blocking communication mode and is provided with a transmission buffer area. The data to be transmitted is first pressed into the buffer area and then transmitted through SOCKET communication. If the buffer is full, no more data can be sent.
/>
1.4 debug File information record management
In the RTLinux environment, the real-time simulation thread cannot directly write the file, and a means for recording debugging information in the real-time simulation process needs to be provided. Although the MPIFifo can also meet the requirements, in order to avoid the interference of the debug information record with the test result, a control type CFifoMsgLog based on a basic FIFO is provided, and interfaces such as information reading and writing, file recording and the like are packaged.
1.4.1CFifoMsgLog class design
/>
1.5 control FIFO
The control FIFO is a bidirectional FIFO established by the computing process and the IO process after being started, and is responsible for establishing, managing and canceling subsequent data communication and transmitting data. The FIFO related functions are performed in the control FIFO by sending, receiving and executing control commands.
1.5.1 basic procedure
1.5.1.1 control FIFO set-up and release
After the program is started, the control FIFO is built and completed before the first IO is carried out; and releasing the last IO before the program is finished.
The control FIFO needs to establish a bidirectional communication connection between the computing process and the IO process: for each computing process, a pair of cmpisifobase classes needs to be established; for IO processes, two sets of arrays of CMPIF ifoBase need to be maintained. The established basic parameters of the CMPIFIFOBase class are set as follows:
nMaxByte, sent by the computing side, 1024 x 10 bytes; the computing side receives 1024 bytes. Since command messages are generally not long.
The iMsgTag,1000+iprocno (computing process number), and the computing side sends; 1000+iprocno, computing side reception.
The IO process listens for messages from the computing process using 1000+iprocno (IO process number):
1.5.1.2 File IO
(1) The computing process sends an SMT_regfifo message to the IO process, and registers the IO function of a file on a shared FIFO; and after receiving the message, the IO process registers corresponding resources for the file IO. The corresponding registration ID is returned via rmt_retregfifo message for use in writing files later. If registration fails, an error message is also returned via rmt_retregfifo.
(2) Before the file IO is needed, an SMT_OpenFile message is sent, and the file is opened by using the registered resources. After the IO process receives the message, it tries to open the corresponding file, and returns an RMT_ERRORNO indicating whether the file is successfully opened. The interface allows the file to be opened during the real-time computing process, and the subsequent file writing operation can be performed without waiting for the return of the mark of the successfully opened file. If a file open failure message is received after several steps, corresponding error processing is required.
(3) After opening the file, the smt_writefile write file may be sent. And after the IO process receives the message, writing the content of the message into a file. If the write file fails, return rmt_error.
(4) And after the writing of the file is finished, sending the SMT_CloseIO closed file. After the IO process receives the message, it tries to close the corresponding file and returns RMT_ERRORNO to indicate whether the closing is successful. After closing the file, the registered resources can be used to open other files continuously, and the file writing operation can be performed continuously.
1.5.1.3SOCKET IO
(1) The computing process sends an SMT_regfifo message to the IO process, and registers the IO function of a SOCKET on the shared FIFO; and after receiving the message, the IO process registers corresponding resources for the SOCKET IO. The corresponding registration ID is returned via rmt_retregfifo message for subsequent socks communication. If registration fails, an error message is also returned via rmt_retregfifo.
(2) Before the SOCKET IO is needed, a ConnectSocket message is sent, and the SOCKET link is opened by using the registered resources. After the IO process receives the message, it tries to open the corresponding SOCKET connection, and returns RMT_ERRORNO to indicate whether the connection is successfully opened. The interface allows the SOCKET to be opened in the real-time computing process, and the subsequent SCOCKET writing operation can be performed without waiting for the returned mark of the successfully opened SOCKET to return. If the SOCKET opening failure message is received after a plurality of steps, corresponding error processing is needed.
(3) After the SOCKET is opened, the SMT_WriteSocket write file may be sent. After the IO process receives the message, the message content is written into SOCKET. If SOCKET fails, return RMT_ERRORNO. The SOCKET adopts a non-blocking mode, and if all data cannot be successfully written into the SOCKET, the data is left in a buffer area; if the buffer is full, the message is discarded and the discard is recorded in the log file.
(4) After the SOCKET communication is finished, the SMT_CloseIO is sent to close the SOCKET. After the IO process receives the message, it tries to close the corresponding SOCKET and returns RMT_ERRORNO, indicating whether the closing is successful. After the SOCKET is closed, other SOCKET can be opened by using the registered resources continuously, and the SOCKET communication is continuously carried out.
1.5.2 control Command message design
1.5.2.1 data packet format
The control command transmitted between the processes is in a binary format, and the content of the control command is composed of the following two parts.
(1) A data header including a data type and a length.
The data header is 8 bytes in length and its data structure is as follows:
(2) The data content is as follows: the number of bytes of the data content is uPacketLength. The data content format is defined in terms of message type, as described in detail below. The data packet format is schematically shown in the following table:
| data head | Data content |
1.5.2.2 computing Process sends data of IO Process
A data structure description;
(1) SOCKET connection information
/>
(2) SOCKET connection information
1.5.2.3IO Process sends data for computing Process
A data structure description;
(1) FIFO information
(2) Error information
1.5.3 class structural design
The CMPIF foCtrl type package provides functions of building control FIFO at IO process side and computing process side, sending, receiving and executing control command, creating file FIFO, SOCKET FIFO, reading and writing operation and the like.
/>
/>
/>
/>
/>
1.6 interface with Fortran language
In order to meet the requirements of Fortran language call, the functions are required to be packaged into a standard C interface. Since there is only one control class in a process, this object is set as a global variable.
5.6.1C interface declaration
/>
1.6.2Fortran interface declaration (applicable to more than Intel Fortran for Windows V10.01.01 version)
/>
/>
/>
/>
FIG. 4 is a schematic diagram of the organization of the shared memory, as shown in FIG. 4: in the organization of shared memory, the organization of data is similar in that both IO data and data of monitored variables need to be exported from a real-time computing thread to a non-real-time thread.
Each data segment is capable of holding several steps of data, where S (1), S (2) each represent one step of recorded data. Each data segment corresponds to a status flag indicating whether the data segment has been written to. If the current data segment is already full, the current data segment is marked as full first, and then the data is written into the next data segment. For a full segment of data, the IO thread will write the data to disk in time, and the communication thread will send the data to the network in time. The IO thread and the communication thread judge whether the data segment is fully written or not through polling the state mark of the data segment.
Each data segment is 1M in length, and 1 real-time calculation thread corresponds to 3 data segments.
As the number of MPI processes increases, the number of shared memory segments required also increases.
The shared memory mechanism is capable of sharing data space among multiple threads or different processes.
The shared memory is used as a data exchange mode of a real-time computing thread and a non-real-time IO thread or socket thread, and the method has the characteristics of highest performance and small influence on a computing program.
FIG. 5 is a diagram of the real-time thread per step emulation data or monitoring variables written to shared memory, as described in FIG. 5:
the computing process determines whether the current data segment has sufficient space,
if so, copying the data to the shared memory, ending the writing process,
otherwise, the state marking bit of the current data segment is set to be full, the next data segment is pointed, the data is copied to the shared memory, and the writing process is ended.
(1)Writestr2sharememory(type,str,len)
The functions are as follows: writing simulation data or monitoring variables of each step of real-time thread into shared memory
Parameter description:
type, reshape, type of current string, 0 is data IO,1 is monitoring variable
str, character string, each simulation step records or monitors data in the format of' file name: the content is as follows: or file ID: the content is as follows: "
len, shaping, length of character string to be written
The implementation flow is shown in fig. 5.
(2)QueryCtrlinfo(Offset,size)
The functions are as follows: querying whether a region of shared memory has been set
The realization is as follows: and comparing the value of a certain area of the shared memory with a set value.
FIG. 6 is a schematic diagram of IO process flow, as shown in the following:
the IO process determines if there is a full segment of data,
if so, writing the full data segment into the file, setting the juxtaposition state to be empty, enabling the write file FIFO to go to dormancy,
otherwise, the IO process directly enters a dormant state, and the IO process continues to poll.
And the data in the real-time thread is exported through the shared memory and written into the file. But only to one file.
Since it typically takes tens or hundreds of microseconds to write a data segment, the sleep time may be on the order of milliseconds so that the polling frequency is not too high.
The current sleep time is 5 milliseconds.
Performance: about 80Mb of data is written in about 5 seconds.
After the simulation is finished, post-processing is carried out on the file data: splitting a file into multiple files by parsing or sending the file to a designated node.
Fig. 7 is a schematic diagram of a communication thread processing flow, as shown,
the IO process determines if there is a full segment of data,
if so, the full data segment is sent to the upper computer interface through the SOCKET FIFO, the juxtaposition state is empty, the SOCKET FIFO enters dormancy,
otherwise, the IO process directly enters a dormant state, and the IO process continues to poll.
The communication thread opens a socket to monitor, and is used for:
(1) Receiving control commands from a management interface
(2) And sending the monitored variable data from the real-time calculation thread.
And (2) after the communication thread receives the control command, correspondingly setting the shared memory area for control so as to be inquired by the real-time thread.
For (2), the data exchange between the real-time computing thread and the communication thread is similar to the exchange between the real-time computing thread and the IO thread, except that after the data segment is fully written, the data segment is sent to the management interface through a socket.
Based on the same conception, the embodiment also provides a real-time IO data processing system for power system simulation, which comprises:
and (3) a building module: the method comprises the steps that when a power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
and an access module: and the computing thread writes the result file through the IO process and carries out bidirectional communication with an upper computer interface.
Preferably, the functional module includes:
writing a file unit: the method comprises the steps of constructing a unidirectional write file FIFO based on a basic FIFO, and enabling the IO process to write the execution process of the computing thread into a file;
and a control unit: the control FIFO is used for constructing a control FIFO based on the basic FIFO to realize the establishment of communication between the IO process and the computing thread, real-time communication and ending communication;
a communication unit: and the system is used for constructing a bidirectional communication FIFO based on the basic FIFO to realize bidirectional communication between the computing thread and the upper computer interface.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is illustrative of the present application and is not to be construed as limiting thereof, but rather as providing for the use of additional embodiments and advantages of all such modifications, equivalents, improvements and similar to the present application are intended to be included within the scope of the present application as defined by the appended claims.
Claims (16)
1. A real-time IO data processing method for power system simulation, the method comprising:
when the power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
the computing thread writes a result file through an IO process and carries out bidirectional communication with an upper computer interface;
the calculation thread writes a result file through the IO process and carries out bidirectional communication with an upper computer interface, and the method comprises the following steps:
the computing thread constructs a write file FIFO based on a basic first-in first-out queue FIFO through an IO process, and the IO process is realized to write the executing process of the computing thread into a file;
the computing thread constructs a control FIFO based on the basic FIFO to realize the establishment, real-time communication and ending communication of communication between the IO process and the computing thread;
the computing thread constructs a communication FIFO based on a basic FIFO through an IO process, so that the computing thread and the upper computer interface are in bidirectional communication;
the data contents transmitted in the control FIFO, the write file FIFO and the communication FIFO are packaged according to the information, share a data communication channel and are processed according to the information types;
the computing thread constructs a write file FIFO based on a first-in first-out queue FIFO through an IO process, the IO process writes the executing process of the computing thread into a file, and the method comprises the following steps:
the IO process registers the write file FIFO based on the register write file FIFO command sent by the computing thread;
the IO process opens the write file FIFO based on an open file IO command sent by the computing thread;
the IO process writes the message content into the write file FIFO based on a write file command sent by the computing thread;
the IO process closes the corresponding file based on a file closing command sent by the computing thread;
2. the real-time IO data processing method of claim 1 wherein the IO process registers a write file FIFO based on a register write file FIFO command sent by a compute thread, comprising:
the computing thread sends a register write file FIFO command to the IO process, and the IO process registers corresponding resources for the write file FIFO after receiving the register write file FIFO command;
returning a corresponding registration ID through an IO process return command;
when the file IO registration fails, the IO process return command further includes: error information.
3. The real-time IO data processing method of claim 1 wherein the IO process opens the write file FIFO based on an open file IO command sent by a compute thread, comprising:
before the computing thread needs to write a file FIFO, sending a file opening IO command to an IO process, and opening a file by using a resource registered by the file writing FIFO;
and after receiving the file opening IO command, the IO process tries to open the write file FIFO and returns a mark whether the file FIFO is successfully opened or not.
4. The real-time IO data processing method of claim 1 wherein the IO process writing message content into the write file FIFO based on a write file command sent by a compute thread comprises:
after the IO process opens a file, the computing thread sends a file writing command to the IO process;
after the IO process receives the file writing command, writing the message content into the file writing FIFO;
if the writing of the file fails, an error identification is returned.
5. The real-time IO data processing method of claim 1, wherein the IO process closes the corresponding file based on a close file command sent by the computing thread, comprising:
after the IO process finishes writing the file, the computing thread sends a file closing command to the IO process;
and after receiving the file closing command, the IO process tries to close the corresponding file and returns an identification of whether the file is successfully closed or not.
6. The real-time IO data processing method of claim 1 wherein the compute thread constructs a control FIFO based on a basic FIFO, comprising:
after the computing thread is started, when the computing thread uses data transmitted in non-real time, the computing thread constructs a control FIFO; the control FIFO includes: a pair of elementary FIFOs;
the IO process establishes and maintains a dynamic array for each basic FIFO;
the IO process determines the size of the dynamic array in the IO process according to the number of the computing threads;
the IO process writes the message communicated by the computing thread into a first array through the control FIFO;
and the IO process reads message data communicated with an upper computer interface in a second array, and sends the read data to the corresponding calculation thread through a control FIFO.
7. The method for real-time IO data processing according to claim 6, wherein,
the communication message comprises a data head and data content;
the data header includes a data type and a length;
the data content comprises: a data content length and a data content format; the data content format is determined according to the data type.
8. The real-time IO data processing method of claim 6 wherein the computing thread constructs a communication FIFO based on a basic FIFO through an IO process, comprising:
the computing thread constructs a pair of SOCKET FIFOs based on the basic FIFO through the IO process;
when the communication is carried out with an upper computer interface, the IO process presses the information in the first array into a buffer area of a first SOCKET FIFO, and sends the information to the upper computer interface through SOCKET, so that the communication between the calculation thread and the upper computer interface is realized;
before the IO process monitors that the upper computer interface transmits a control command, the IO process establishes a link with the upper computer interface;
and the IO process writes the message data sent by the upper computer interface into the second array through a second SOCKET FIFO, and sends the message data to the responding computing thread through the control FIFO.
9. The method for real-time IO data processing according to claim 6, wherein the SOCKET FIFO uses a non-blocking communication mode, and if the buffer is full, no data is transmitted.
10. The real-time IO data processing method of claim 8, wherein the computing thread constructs a pair of socks FIFOs based on a basic FIFO through an IO process, comprising:
the IO process registers SOCKET FIFO on the shared FIFO based on the register SOCKET FIFO command sent by the computing thread;
the IO process opens the SOCKET FIFO based on an SOCKET IO opening command sent by the computing thread;
the IO process writes the message content into the SOCKET based on a write file command sent by a computing thread;
and the IO process closes the SOCKET FIFO based on a closing file command sent by the computing thread.
11. The method for real-time IO data processing according to claim 10, wherein the IO process registers a socks FIFO based on a registration socks FIFO command sent by a computing thread, comprising:
the computing thread sends a registration SOCKET FIFO command to the IO process;
after receiving the SOCKET FIFO command, the IO process registers corresponding resources for the SOCKET FIFO and returns the registration ID of the SOCKET FIFO;
if registration fails, an error message flag is returned.
12. The method of real-time IO data processing according to claim 10, wherein the IO process opens the socks FIFO based on an open socks IO command sent by a computing thread, comprising:
before the computing thread needs SOCKET IO, sending a link establishment command to the IO process, and opening the SOCKET link by using registered resources;
after receiving the link establishment command message, the IO process tries to open the corresponding SOCKET connection with the upper computer and returns a mark whether the connection is successfully opened or not;
the interface allows SOCKET FIFO to be opened in the real-time calculation process to perform SCOCKET writing operation;
if the calculation thread receives the SOCKET opening failure message after a plurality of steps, the SOCKET opening failure message is written into an error log.
13. The real-time IO data processing method of claim 10 wherein the IO process writing message content into the SOCKET based on a write file command sent by a computing thread comprises:
after the IO process opens the SOCKET FIFO, the computing thread sends a message to the upper computer through the IO process, and after the IO process receives the message, the IO process writes the message content into the SOCKET;
when the message length is greater than the threshold value, sending the message to an upper computer; if the writing of SOCKET fails, a failure identifier is returned.
14. The real-time IO data processing method of claim 10 wherein the IO process closes the socks FIFO based on a close file command sent by a computing thread, comprising:
after the SOCKET communication is finished, the computing thread sends a closing file command to the IO process to close the SOCKET FIFO; and after receiving the file closing command, the IO process tries to close the SOCKET FIFO and returns an identification of whether the closing is successful or not.
15. The real-time IO data processing method of claim 1, wherein the data content comprises: and recording result data of each step of simulation calculation, monitored state variable data and control command data sent by an interface.
16. A real-time IO data processing system for power system simulation for implementing a real-time IO data processing method for power system simulation as claimed in claim 1, said system comprising:
and (3) a building module: the method comprises the steps that when a power system simulation real-time calculation thread uses data transmitted in non-real time, an IO process is established for the calculation thread;
and an access module: the computing thread writes a result file through an IO process and carries out bidirectional communication with an upper computer interface;
the access module comprises:
writing a file unit: the method comprises the steps that a write file FIFO is constructed by the computing thread through an IO process based on a basic first-in first-out queue FIFO, so that the IO process writes the executing process of the computing thread into a file;
and a control unit: the control FIFO is used for the computing thread to construct a control FIFO based on the basic FIFO, so that the communication between the IO process and the computing thread is established, the communication is performed in real time, and the communication is ended;
a communication unit: and the communication FIFO is constructed on the basis of the basic FIFO by the computing thread through the IO process, so that the computing thread and the upper computer interface are in bidirectional communication.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711224511.1A CN108107750B (en) | 2017-11-29 | 2017-11-29 | Real-time IO data processing method and system for power system simulation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711224511.1A CN108107750B (en) | 2017-11-29 | 2017-11-29 | Real-time IO data processing method and system for power system simulation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108107750A CN108107750A (en) | 2018-06-01 |
| CN108107750B true CN108107750B (en) | 2023-10-20 |
Family
ID=62208620
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711224511.1A Active CN108107750B (en) | 2017-11-29 | 2017-11-29 | Real-time IO data processing method and system for power system simulation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108107750B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115996197B (en) * | 2023-03-17 | 2023-06-16 | 之江实验室 | Distributed computing flow simulation system and method with preposed flow congestion |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1218565A (en) * | 1996-03-13 | 1999-06-02 | 钻石多媒体系统公司 | Multi-threaded fifo pool buffer and bus transfer control system |
| CN101030864A (en) * | 2006-03-03 | 2007-09-05 | 中兴通讯股份有限公司 | Method for transmitting data file between computer and mobile terminal |
| CN101198936A (en) * | 2005-05-02 | 2008-06-11 | 阿塞米克两合公司 | Method and apparatus for emulating a programmable unit |
| CN101340404A (en) * | 2008-08-14 | 2009-01-07 | 山东大学 | Embedded gateway for wireless network protocol conversion between WiFi and ZigBee |
| CN102117342A (en) * | 2011-01-21 | 2011-07-06 | 中国科学院上海技术物理研究所 | Peripheral component interconnect (PCI) Express bus-based multiband infrared image real-time acquisition system and method |
| CN102811152A (en) * | 2012-08-31 | 2012-12-05 | 南京鑫三强科技实业有限公司 | Method for realizing real-time transaction and data exchange of multiple main bus network communication |
| CN103472733A (en) * | 2013-09-03 | 2013-12-25 | 国家电网公司 | Digital real-time simulation physical port device of power system based on optical fiber communication |
| CN104009976A (en) * | 2014-05-09 | 2014-08-27 | 中国人民银行印制科学技术研究所 | Real-time communication method and equipment used among multiple main station systems |
| CN105843693A (en) * | 2016-03-22 | 2016-08-10 | 同济大学 | High-speed maglev transportation simulation oriented memory sharing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8792945B2 (en) * | 2006-10-31 | 2014-07-29 | Motorola Mobility Llc | Methods and devices for dual mode bidirectional audio communication |
-
2017
- 2017-11-29 CN CN201711224511.1A patent/CN108107750B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1218565A (en) * | 1996-03-13 | 1999-06-02 | 钻石多媒体系统公司 | Multi-threaded fifo pool buffer and bus transfer control system |
| CN101198936A (en) * | 2005-05-02 | 2008-06-11 | 阿塞米克两合公司 | Method and apparatus for emulating a programmable unit |
| CN101030864A (en) * | 2006-03-03 | 2007-09-05 | 中兴通讯股份有限公司 | Method for transmitting data file between computer and mobile terminal |
| CN101340404A (en) * | 2008-08-14 | 2009-01-07 | 山东大学 | Embedded gateway for wireless network protocol conversion between WiFi and ZigBee |
| CN102117342A (en) * | 2011-01-21 | 2011-07-06 | 中国科学院上海技术物理研究所 | Peripheral component interconnect (PCI) Express bus-based multiband infrared image real-time acquisition system and method |
| CN102811152A (en) * | 2012-08-31 | 2012-12-05 | 南京鑫三强科技实业有限公司 | Method for realizing real-time transaction and data exchange of multiple main bus network communication |
| CN103472733A (en) * | 2013-09-03 | 2013-12-25 | 国家电网公司 | Digital real-time simulation physical port device of power system based on optical fiber communication |
| CN104009976A (en) * | 2014-05-09 | 2014-08-27 | 中国人民银行印制科学技术研究所 | Real-time communication method and equipment used among multiple main station systems |
| CN105843693A (en) * | 2016-03-22 | 2016-08-10 | 同济大学 | High-speed maglev transportation simulation oriented memory sharing method |
Non-Patent Citations (7)
| Title |
|---|
| Bo Yu，等.Novel mixed-voltage I/O buffer with thin-oxide CMOS transistors.Journal of Semiconductors.2009,1-4. * |
| 刘健，等.远程智能I／O 装置在DCS 中的应用.黑龙江电力.2004,第26卷(第26期),302-304. * |
| 张博文，等.特高压输变电设备状态监测数据通信仿真研究.中国电力.2017,第50卷(第50期),11-15. * |
| 戴志涛.PCI总线双向FIFO缓冲并行通信的实现.微处理机.2002,(第4期),53-55. * |
| 李 伟,等.基于hadoop 的电网大数据处理探究.网络与信息工程.2014,74-77. * |
| 田芳，等.电力系统全数字实时仿真装置.电 网 技 术.2008,第32卷(第32期),17-22. * |
| 陈迪仕，等.基于RTLinux 的飞控系统进程间通信方案.计算机工程.2012,第38卷(第38期),249-251. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108107750A (en) | 2018-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109558344B (en) | DMA transmission method and DMA controller suitable for network transmission | |
| CN110807063B (en) | Substation real-time data rapid distribution synchronization system and method based on edge calculation | |
| CN112039722A (en) | MQTT protocol simulation method and simulation equipment | |
| US8799546B2 (en) | Method and apparatus for monitoring single board running state | |
| US10942771B2 (en) | Method, apparatus and system for multi-module scheduling | |
| CN113422707B (en) | In-band network remote measuring method and equipment | |
| CN108062235B (en) | Data processing method and device | |
| EP4246965A2 (en) | Method and device, equipment, and storage medium for data processing | |
| CN116225992A (en) | NVMe verification platform and method supporting virtualized simulation equipment | |
| CN104866528A (en) | Multi-platform data acquisition method and system | |
| CN108107750B (en) | Real-time IO data processing method and system for power system simulation | |
| CN114363269B (en) | Message transmission method, system, equipment and medium | |
| CN112416557A (en) | Method and device for determining call relation, storage medium and electronic device | |
| CN112181806A (en) | Embedded software testing device and method based on TFTP protocol | |
| CN107911317B (en) | Message scheduling method and device | |
| CN108153597B (en) | LINUX equipment management device and method | |
| JP7182744B1 (en) | Deterministic Reproduction of Events Between Software Entities | |
| CN112148537A (en) | Bus monitoring device and method, storage medium, and electronic device | |
| CN115344522A (en) | Message conversion channel, message conversion device, electronic equipment and switching equipment | |
| CN105187519A (en) | Socket transmission system based on AIO | |
| US7006442B1 (en) | Communication control system and control method thereof | |
| CN117499351A (en) | Message forwarding device and method, communication chip and network equipment | |
| CN111182047B (en) | Method and system for transferring files between large data platforms across a network | |
| CN111314805B (en) | Debugging method and device for intercommunication OMCI (OMCI) messages of GPON (gigabit passive optical network) system | |
| CN108809405B (en) | Communication method, server, storage medium and computer equipment for unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |