GB2280974A - A data logging and processing apparatus - Google Patents

Abstract

An apparatus (1) has a source data bank (2) connected to a time stamp device (3) for rapid reception and non-volatile storage of data from disparate sources. Fixed program processors (7) are separated from the data bank by a one-way interface (6) which prevents feedback of data to the bank, to help ensure data integrity. Each processor (7) relates, for example, to a particular product in an industrial process. This provides an interim output on a line (8). A variable processor (9) includes a respective hard-wired data cell associated with each output field of a processor (7) to provide a second interim output on a line (11) at a higher level than the first interim output. An interpreter (13), e.g an expert system, provides a further, higher level of processing according to interactive inputs on a line (14) to provide a final output on a line (15). The fixed processors (7) and variable processor (9) provide an ideal combination for obtaining data specific to particular products or any other parameter and a higher level of processed data to provide overview information. Faults may be easily tracked to their source. <IMAGE>

Description

"A data Processing aPParatus The invention relates to data processing where data is being logged at a fast rate into a storage device of an apparatus and the apparatus must carry out processing of the data to provide an output.

At present, data loggers, or indeed minicomputers connected to a large number of on-line data input terminals have the capability to log data which is being received and to carry out subsequent processing to provide the output. However, where a substantial amount of data processing is required which includes the carrying out of fixed processes and also variable processes where the actual processing and the data being processed may vary according to auxiliary inputs is required, it is very difficult to guarantee data integrity. This is because one piece of faulty auxiliary input data, or indeed one piece of faulty original or source data would cause a "chain reaction" and corrupt subsequent data downstream which is generated from the processing operations.

Another problem with existing apparatus where there is a very large amount of processing of data which is being received at a rapid rate is the lack of versatility in the processing operations. Generally, the programs operate in a manner which can only be changed by changing of certain parameters and thus the degree of change is quite limited.

European Patent Specification No. EP-B1-0173534 (GEC Avionics Limited) describes a data processing arrangement for operation in an environment in which large quantities of data from various sources are arriving rapidly such as in an airplane. Data is received from sources such as radar, thermal imaging sensors and electronic signal monitoring receivers. Addresses are assigned to selected portions of the data and the portions are processed together to generate an output dependent on the nature of the selected portions. This arrangement appears to be particularly useful for the avionics industry, however, in other industries such as data logging and processing for industrial process control where not only is a large amount of processing required, but the actual processes may vary depending on circumstances, problems may arise in providing the necessary data integrity and processing versatility.

The invention is directed towards providing a data processing apparatus which is capable of the following functions: the logging of data being received from disparate sources at a rapid rate; processing of the data in a versatile manner whereby the processes may vary from time to time depending on interactive inputs without the requirement for a very large processing capacity; and maintenance of data integrity by minimising the effects of spurious inputs.

According to the invention, there is provided a data processing apparatus comprising:- a source data bank comprising non-volatile storage devices and an associated controller, the controller being connected at an input port to a time stamp device for automatic insertion of a time data element in data as it is being received, the time stamp device having a plurality of ports for reception of data from a plurality of disparate sources; a one-way interface device connected to the source data bank by a data bus, the one-way interface device being a data storage device and a controller programmed to write retrieved data from the data bank as a sequence of data values in flat format separated by delimiters; a set of processors, each having fixed programs stored on a non-volatile device for processing data in a fixed manner to generate processed data values in data fields; an instruction controller connected to the source data bank, to the one-way interface device and to the set of processors, and being programmed to control multiple data retrieval passes in sequence for each relevant processor in turn so that the one-way interface handles data for one processor only at any one time; an output device connected to the set of fixed processors for outputting of a first interim data output; a variable processor comprising a non-volatile memory circuit and an associated controller connected to the set of fixed processors and being programmed to assign groups of memory locations of the memory circuit so that each group forms a cell in a grid format, the controller being programmed to recognise a unique code which cross-links an output data field of a fixed processor with a cell so that there is one cell linked to each output data field of a fixed processor in a one-to-one relationship, the controller also being programmed to generate a set of cells for reception of auxiliary input data, wherein programs for carrying out the various processes may be amended interactively using user inputs; an interpreter connected to the variable processor and being programmed to interpret the output of the variable processor to provide a high-level of processed data to provide a final output, the interpreter being programmed to provide outputs according to interactively received user instructions; and a report generator connected to the interpreter for providing control instructions for printing devices.

Preferably, the source data bank has a capacity in the range of 4 to 8KB.

In another embodiment, the source data bank controller is programmed to store data in the bank in sequential, fixedlength files and to stored index files for fast data access.

In a further embodiment, the interpreters connected to the variable processor by a protocol converter for fast data format conversion.

The invention will be more clearly understood from the following description of some preferred embodiments thereof, given by way of example only with reference to the accompanying drawings in which: Fig. 1 is a schematic diagram showing data flow between portions of a data processing apparatus of the invention; and Fig. 2 is a more detailed diagram showing the apparatus.

Referring to the drawings, there is shown a data processing apparatus of the invention, indicated generally by the reference numeral 1. The overall data flows and primary inputs and outputs are shown in Fig. 1 which illustrates in particular the manner in which data integrity is maintained and by which the processing is carried out in a way which minimises the processing capacities required and also provides the necessary versatility. A source data bank 2 receives data from a time stamp device 3, which in turn receives data from an input line 4. The input line 4 is in this embodiment connected to a large number of terminals (100) and also to modulation/demodulation circuits for reception of data from remote sources. An example of the sources from which data is received would be motor car production plants at various locations throughout the world, and also a set of terminals located in an office building for an organisation. It will be appreciated that a large amount of data is received, usually on a 24-hour basis. The data reception rate is 9600 band. Input terminals 24 and a remote data generator 25 are shown in Fig. 2 in which the source data bank 2 is shown as a set of three fixed disk banks 21, 22 and 23 and associated drivers. The bank 2 has a total capacity of 5 GB, and in general is preferably in the range of 4GB to 8GB. The controller of the bank 2 is programmed to store data in the bank in sequential, fixed-length files and to store index files for fast data access.

The only external link which the source data bank 2 has is a link 5 to a one-way interface 6, and a link to an instruction controller 26 (see Fig. 2). The one-way interface 6 is in this embodiment a non-volatile memory device having a stored flat file structure without a header or other program-specific instructions. The file structure and associated control programs are for the retrieval of data from the source data bank 2 and writing to the flat file in ASCII format, the data values being separated by delimiters. This is under instructions from the instruction controller 26. The control programs and circuits are constructed for retrieval of data from the source data bank 2 and writing to the flat file in sequences, values being separated by delimiters. There are no circuits or programs for carrying out the reverse operations of separating the data and writing back to the source data bank 2. Thus, the flat file and associated control circuits act as a one-way interface for the source data bank 2.

The output of the one-way interface 6 is connected to a set of processors 7 indicated by the letters A, B, C, ..... O. These processors are in the form of a microprocessor connected to a storage device and operating according to database programs whereby there is a set of records for each process to be carried out. The set of stored records and the processing circuits together form a processor to carry out fixed processes according to programs stored in non-volatile memory. It will be appreciated, however, that implementation of the set of fixed processors 7 may vary, depending on the circumstances. For example, each processor may be a separate hard-wired circuit. The important point is that the process which is carried out is fixed as it will not vary according to user interactive inputs. Each processor 7 relates to a single objection, such as an end product in an industrial process. An output line 8 provides an interim output &num;1 so that users may view the results of the processing carried out by the processors 7. The output line 8 may, of course, also be connected to printers for printing of the output.

The apparatus 1 also includes a variable processor 9 which comprises a random access memory circuit, a storage device and a microprocessor circuit which is programmed to assign a set of locations in the random access memory circuit as being an individual cell of data and to arrange the cells in a grid format. There is a unique identifier linking each output data field of each processor 7 with a cell in the variable processor 9. Because the programs which link the fields and the cells are stored in non-volatile memory with write-protect security, the fields and cells should be regarded as being hard-wired together. Thus, a certain cell in the variable processor 9 will always be associated with data for one particular field in one of the processors 7. However, the programs which retrieve data from cells in the variable processor 9 and carry out additional processing may be easily changed interactively by the user.

In addition, there is a set of auxiliary cells which are provided for reception of auxiliary input data on an input line 10. Such auxiliary input data may include "gut feel" industrial process data inputted by a control engineer as to the manner in which a process will perform, or to the manner in which materials will be required or to which end products will be required in an industrial process. As stated above, the control circuits connected to the random access memory device and to the non-volatile storage device of the variable processor 9 run according to programs which may be varied by the user. When these programs run, they change the values of data in cells and this data may be used to provide a second interim output &num;2 on an output line 11.

A protocol converter 12 converts the data from the cells to a format which may be understood by an interpreter 13 which in this embodiment is an expert system having rules for interpretation of received data and manipulation of the data to provide a high-level output on a final output line 15. This output may be varied interactively by the user using interactive inputs on an input line 14. A report generator, in this embodiment a word processor 16 is provided for generating reports including the final output data on an output line 17 is connected to a printer for printing of the reports.

The apparatus 1 operates in such a way that data is continuously received at a rapid rate into the source data bank 2. This is because there is a large number of data sources. The instruction controller 26 is connected to the data bank 2, to the one-way interface 6 and to the processors 7 to transmit instructions to them to control data retrieval passes from the data bank 2 to the flat file 6 and subsequently to the relevant processor 7. Each data retrieval pass is for a single processor 7, A, or B, or C ..... N or 0 and involves a maximum of 5ORB of data.

Thus, in a single pass the instruction controller 26 retrieves a set of values from the data bank 2, writes them to the flat file 6 in sequential format separated by delimiters and subsequently writes the data to the relevant input field in the relevant processor 7. The instruction controller 26 only transmits instructions and cannot be used for feeding back of data from a processor 6 to the data bank 2. Thus, data in the data bank 2 may not be corrupted from inadvertent feedback from the rest of the operators 1 and may be regarded at all times as being source or "raw" data as received from the external sources.

The time stamp device 3 inserts a time stamp element into each item of data so that if data received from an external source is incorrect, it may be tracked back to the source relatively quickly. This aspect of the invention is important as it ensures that data integrity is maintained. For example, if one of the processors 7 corrupts data during it's operation, the data in the data bank 2 remains safe as it is protected by the one-way interface 6.

The user or operator has the choice of three different outputs from the apparatus 1, namely, interim output #1, interim output &num;2, and the final output on the line 17.

In certain circumstances, all that is required is the processing carried out by the fixed processor 7. Because output fields of the processors 7 are hard-wired to cells of the variable processor 9, the variable processing may be controlled in an easy and simple manner and mistakes would not arise because of the manner in which the cells are hard-wired. This provides for maximum versatility in the programs which are used to carry out the next higher level of processing.

Of course, the interpreter 13 allows the operator to achieve interpretation of the process data according to any required criteria. The use of an expert system with stored rules is the most efficient way of generating this level of processing. Because the interpreter, the variable processor and the set of fixed processors are all separate, the extent of processing power required is kept to a minimum and the control which is provided to a user is maximised. Any faulty processing operations may be easily tracked to a single device. Indeed, the data transfer controller 27 which accesses the variable processor 9, the protocol converter 12, the interpreter 13 and the word processor 16 logs accesses which are made to these devices to facilitate tracking at a later stage.

The invention is not limited to the embodiments hereinbefore described, but may be varied in construction and detail.

Claims (5)

1. A data processing apparatus comprising: a source data bank comprising non-volatile storage devices and an associated controller, the controller being connected at an input port to a time stamp device for automatic insertion of a time data element in data as it is being received, the time stamp device having a plurality of ports for reception of data from a plurality of disparate sources; a one-way interface device connected to the source data bank by a data bus, the one-way interface device being a data storage device and a controller programmed to write retrieved data from the data bank as a sequence of data values in flat format separated by delimiters; a set of processors, each having fixed programs stored on a non-volatile device for processing data in a fixed manner to generate processed data values in data fields; an instruction controller connected to the source data bank, to the one-way interface device and to the set of processors, and being programmed to control multiple data retrieval passes in sequence for each relevant processor in turn so that the one-way interface handles data for one processor only at any one time; an output device connected to the set of fixed processors for outputting of a first interim data output; a variable processor comprising a non-volatile memory circuit and an associated controller connected to the set of fixed processors and being programmed to assign groups of memory locations of the memory circuit so that each group forms a cell in a grid format, the controller being programmed to recognise a unique code which cross-links an output data field of a fixed processor with a cell so that there is one cell linked to each output data field of a fixed processor in a one-to-one relationship, the controller also being programmed to generate a set of cells for reception of auxiliary input data, wherein programs for carrying out the various processes may be amended interactively using user inputs; an interpreter connected to the variable processor and being programmed to interpret the output of the variable processor to provide a high-level of processed data to provide a final output, the interpreter being programmed to provide outputs according to interactively received user instructions; and a report generator connected to the interpreter for providing control instructions for printing devices.

2. An apparatus as claimed in claim 1, wherein the source data bank has a capacity in the range of 4 to 8KB.

3. An apparatus as claimed in claims 1 or 2, wherein the source data bank controller is programmed to store data in the bank in sequential, fixed-length files and to stored index files for fast data access.

4. An apparatus as claimed in any preceding claim, wherein the interpreters connected to the variable processor by a protocol converter for fast data format conversion.

5. An apparatus substantially as hereinbefore described, with reference to and as illustrated in the accompanying drawings.