GB2293032A - Handling extremely large quantities of data - Google Patents
Handling extremely large quantities of data Download PDFInfo
- Publication number
- GB2293032A GB2293032A GB9418181A GB9418181A GB2293032A GB 2293032 A GB2293032 A GB 2293032A GB 9418181 A GB9418181 A GB 9418181A GB 9418181 A GB9418181 A GB 9418181A GB 2293032 A GB2293032 A GB 2293032A
- Authority
- GB
- United Kingdom
- Prior art keywords
- data
- control apparatus
- process control
- code
- comparator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/17—Interprocessor communication using an input/output type connection, e.g. channel, I/O port
Abstract
Apparatus for efficiently receiving, validating, storing and processing an extremely large quantity of data comprises a data entry device for receiving and locally validating the data; an activity monitor 3 for transmitting the data over a local-area-network cable 9, at a communication rate of greater than 4 MHz, during periods of reduced network activity; devices 4 and 5 which further validate the transmitted data by comparison with stored data history information, convert the data from 16-bit to 32-bit format, assign a control code to the data, then transmit the data and code; and a reduced-instruction-set processor 6, operating at greater than 30 MHz, which receives the 32-bit data and code, stores the data at locations determined by the code in a fixed-disk array of at least 2 gigabytes capacity, processes the data in real time by use of a co-processor 7 which refers to a modification matrix array held in a dedicated non-volatile memory, and generates reports in response to requests from attached workstations. <IMAGE>
Description
"A process control apparatus" The present invention relates to a process control apparatus. More particularly, the invention relates to such an apparatus used to efficiently control extremely large quantities of data. By control is meant the reception, internal communication, temporary storage, permanent storage and both direct processing and monitoring of data throughput.
Presently, processing and analysis of such a large quantity of information such as is normally gathered in a market survey is only possible using extremely large, complex and powerful computers.
This presents a problem in that the availability of such computer systems is limited and specialist knowledge is required to configure and maintain them. Accordingly systems of this type are developed and offered on a commercial basis to a number of users frequently with a variety of different requirements, who are often in remote locations. As a result of the processing demands placed on these systems by the large numbers of users, system response is often slow. Further, as the connection of the system circuits and associated software elements is a complex process it is rarely cost efficient to reconfigure the system for a once-off analysis request. In addition updating the various permitted parameters associated with a given data element cannot be easily done, adversely affecting result quality.
European patent specification No. EP-B1-017534 describes a data processing arrangement. This specification is particularly directed to the avionics industry where there is a need to process data from a large number of sensors each presenting data to the arrangement for processing at the same time. The arrangement described achieves this and overcomes the need for an excessively large computer. Such an arrangement offers a solution to efficiently processing the bursts of information made available by a large number of sensors, each relaying a small amount of information, in real time, however it is not suitable for efficient processing of extremely large amounts of data.
The present invention is directed towards providing a process control apparatus to overcome the disadvantages of the aforementioned systems, in particular the excessive processing capacity requirements when dealing with extremely large quantities of data. It is a further objective to provide all the functionality of the systems described on a smaller scale apparatus in a responsive and easy to modify manner.
Accordingly, there is provided a process control apparatus comprising:
a data entry device for receiving entered data
and locally validating the data for efficient
transmission by an associated network activity
monitoring device across a coaxial local area
network operating at a communication rate of
greater than four megahertz to a data validation
and control device;
the data validation and control device having a
validation and control node connected to a
comparison means with an associated sixteen to
thirty two bit converter for further validating
the data and a control code generator for
assigning a control code to the data,
communicating with a processing device;;
the processing device having a receiving means
for receiving the data and data control code, an
associated storage device having a reconfigurable
capacity of at least two gigabytes, a code reader
for decoding the control code and determining a
storage location for the data within the storage
device and a processor with a reduced instruction
set operating at a speed of speed in excess of
thirty megahertz for processing of the data in
real time.
A process control apparatus formed in accordance with the invention has a number of advantages. As the processing is not centralised at a single processor or group of processors the response time of the apparatus is greatly enhanced. The distribution of the validation stages across the apparatus ensures that modification of the various validation parameters can be easily effected as the size of each set of parameters is reduced.
Preferably the data processing device has a code reader for receiving a control code from the data validation and control device connected to the control main processor and a dedicated data modification processor in turn connected to a modification matrix array located in a dedicated permanent non-volatile memory circuit for modifying the data in accordance with predefined parameters.
Ideally the code reader has means for decoding the control code to determine a storage location for the data in the data processing device. Thus, the data entered is coded and can be easily be grouped for efficient retrieval.
Preferably the modification processor has means for retrieval of data from the storage device and the modification matrix array located in a dedicated nonvolatile memory circuit, on receipt of a request from a request generation and reporting means, for modifying the data and returning it to the main processor for supply to the reporting means. Thus, data grouped on the storage device for expedient retrieval and modification in accordance with predefined parameters in response to requests.
Normally the request generation and reporting means includes a plurality of workstations with request entry means and reporting means incorporating printing and display means.
Preferably the comparison means comprises a sixteen bit data comparator with associated memory circuits and dedicated cache memory of in excess of two hundred kilobytes, a sixteen to thirty two bit converter with an accompanying sixteen bit latch and a comparator controller for connection to the sixteen bit data comparator, the sixteen to thirty two bit converter and a control code generator. Thus further verification is possible by checking the data entered against its associated permitted attributes information to determine that all required relevant data has been entered and that it is within allowable ranges.
Preferably the comparator controller comprises
means for reception of a conversion complete
signal from the sixteen to thirty-two bit
converter;
means for reception of a validity signal from the
sixteen bit data comparator for indicating a
valid information condition based on parameters
and data history located within a comparator
memory circuit; and
means for supplying a signal to the code
generator.
Thus subsequent verification is conducted at the validation and control node and these data verification parameters are also easily modified.
Preferably the code generator has receiving means for receiving the signal from the comparator controller, and transmission means for transmitting a control code to the data processing device in response to the signal received. Thus, coding of the data with reference to data history stored in the comparator's memory circuits prior to storage ensures that related data is stored in logical groups for efficient retrieval.
Ideally the validation and control node has a fixed disk with a capacity of greater than eighty megabytes, a data entry means, a display, and a memory circuit in excess of eight megabytes having a processor operating at greater than twenty five megahertz connected to a coaxial local area network operating at a speed of greater than four megahertz.
Ideally the network activity monitoring device incorporates means for receiving data from the data entry device and transmitting means for transmission of the data across the local area network during periods of reduced network activity. Thus, ensuring that the data is transmitted in a manner which will not adversely affect the responsiveness of the apparatus.
Ideally the dedicated network activity monitoring device is directly connected to the data entry device and has an associated local storage device. Thus, the data may be stored until a period of diminished network traffic occurs.
Normally the data entry device comprises a plurality of workstations, each workstation having a data entry means, an initial data validation means, a fixed disk, a display, and a memory circuit. Thus, validation procedures in the data entry device allow that data may be rejected before being received into the apparatus if incorrect, thereby providing considerable initial data verification.
In one arrangement the comparator incorporates data transfer means for receiving data transmitted from the main processor and transmitting the data to the comparator associated memory circuits to form a data entry history. Advantageously this allows data to be classified at the validation stage to determine its suitability for inclusion into one or more data groupings. By determining this using the data entry history before the information is stored processing requirements on the system are further reduced.
Preferably the data entry means is provided by a pattern recognition device for automatically transferring the data from a written format to a format suitable for use by the data entry device. This reduces the data entry personnel requirements by having the data entered automatically.
In one embodiment the pattern recognition device is a scanner allowing the information to be entered in a fast and efficient manner.
In an alternative embodiment the pattern recognition device is a bar code reader allowing the information to be entered in a simple and easy to modify manner.
The invention will be more clearly understood from the following description of an embodiment thereof given by way of example only, with reference to the accompanying drawings in which:
Fig. 1 is a general block diagram of a process
control apparatus in accordance with the
invention;
Fig. 2. is a detailed block diagram of a data
entry device portion of the invention;
Fig. 3. is a detailed block diagram of a data
validation and control device portion of the
invention; and
Fig. 4. is a detailed block diagram of a data
processing device portion of the invention.
Referring to the drawings and initially to Figs. 1 and 2 there is provided a process control apparatus according to the invention, indicated generally by the reference numeral 1 for entering, validating, storing and processing data. The apparatus comprises a data entry device 2 with an associated network activity monitor 3, a data validation and control device 4 with an accompanying comparator device 5 and a data processing device 6 with an associated modification processor element 7.
The data entry device 2 comprises a number of identical workstations 8, connected to a co-axial local area network cable 9 operating at a speed of sixteen megahertz. The workstation 8 has a fixed disk 10, a data entry means provided by a keyboard 11 and a monitor 12.
The workstation 8 further has a sixteen Megabyte read / write memory circuit 13, and an initial data validation means provided by a look up table listing stored in the read / write memory circuit 13.
Connected to the workstation 8 and the local area network cable 9 there is a network activity monitor 14 with an associated local storage device provided by a fixed disk 15. The network activity monitoring device 14 has an input / output buffer 16 for receiving data from the workstation and for transmission of the data across the local area network cable 9 during periods of reduced network activity to the data validation and control device 4.
Referring now to Figs. 3 and 4. The data validation and control device 4 has a validation and control node 20 connected to the local area network cable 9 and a data processing device 6.
The data validation and control device 4 also has a comparator device provided by a sixteen bit data comparator 21 with an associated hard disk 22 and two hundred and fifty six kilobytes of cache memory 23. The sixteen bit data comparator 21 is connected to a sixteen to thirty two bit converter 24 with an accompanying sixteen bit latch 2Sa. A comparator controller 26 is connected to the sixteen bit data comparator 21, the sixteen to thirty two bit converter 24 and a control code generator 25. The comparator controller 26 has, an input port 28 for receiving a conversion complete signal from the sixteen to thirty two bit converter 25, and a second input port 27 for receiving a validity signal from the sixteen bit data comparator 21 indicating a valid data condition based on parameters and data history located within the fixed disk 22 and the cache memory 23.The comparator controller 26 also has a an output port 29 for supplying a signal to the code generator 25 based on the signals received.
The data processing device 6 has thirty two bit reduced instruction set main processor 30 operating at sixty six megahertz, connected to a request generation and reporting network 31. The main processor 30 accesses a thirty two megabyte read/write memory circuit 32, a monitor 33 and a three gigabyte fixed disk array 34. The fixed disk array 34 is connected to a code reader 35 through the main processor 30 to receive a data control code from the data validation and control device 4. The code reader 35 decodes the control code generated by the control code generator 25 to determine a storage location for the data within the fixed disk array 34.
The data processing device 6 also has a dedicated thirty two bit data modification processor 36 connected to a modification matrix array 37 located in a read only memory circuit.
The request generation and reporting network 31 has a number of similar workstations 39 and reporting means provided by a dot matrix printer 40 and a laser printer 41.
In use, data is entered at the keyboard 11 of the workstation 8. As the data is entered it is passed through a software data filter in the read / write memory circuit 13 of the workstation 8 to provide initial data validation.
Having been thus validated the data is transmitted to the network activity monitor 14 and temporarily stored on the fixed disk 15. When network traffic on the local area network cable 9 is acceptably low the data is retrieved from the fixed disk 15 and transmitted to the validation and control node 20.
History files and accepted data parameters are stored in the hard disk 22 and the cache memory 23. This data is compared with the data supplied from the validation and control node 20 to validate the data and to determine the appropriate control code. The validated data is passed to the sixteen to thirty two bit converter 24 and the latch 25a and the appropriate control and data valid signal is sent to the input port 27 of the comparator controller 26. The first data element is latched into the latch 25a and ignored by the sixteen to thirty two bit converter 24. When the second data element is received from the sixteen bit data comparator 21, the first data element is latched through the latch 25a and combined with the second data element in the sixteen to thirty two bit converter 24. This thirty two bit word is then connected to the data processing device 6.
The thirty two bit data word is received by main processor 30 and stored on the fixed disk array 34 having its location determined by the code from the control code generator 25. The location selected is then reported back to the hard disk 22 to update the history files to ensure continued correct code generation from the 16 bit data comparator 22.
Upon receipt of a request from the request generation and reporting network 31, the main processor 30 retrieves the stored data from the fixed disk array 34 and supplies it to the modification processor 36. The modification processor 36 retrieves the modification matrix array 38 and processes the information. When processing is complete the results are returned from the modification processor 36 to the main processor 30.
These results may then be then be sent to the request generation and reporting network 31 for printing on either the dot matrix printer 40 or the laser printer 41. Alternatively the results may be viewed on the monitor of the workstations 39.
The invention is not limited to the embodiment hereinbefore described but may be varied in construction and detail. For example, it is envisaged that the data entry means may be provided by a bar code reader such as a pen based system or by a scanning device rather than a keyboard.
Claims (17)
1. A process control apparatus comprising:
a data entry device for receiving entered
data and locally validating the data for
efficient transmission by an associated
network activity monitoring device across a
coaxial local area network operating at a
communication rate of greater than four
megahertz to a data validation and control
device;
the data validation and control device
having a validation and control node
connected to a comparison means with an
associated sixteen to thirty two bit
converter for further validating the data
and a control code generator for assigning
a control code to the data, communicating
with a processing device;;
the processing device having a receiving
means for receiving the data and data
control code, an associated storage device
having a reconfigurable capacity of at
least two gigabytes, a code reader for
decoding the control code and determining a
storage location for the data within the
storage device and a processor with a
reduced instruction set operating at a
speed of speed in excess of thirty
megahertz for processing of the data in
real time.
2. A process control apparatus as claimed in claim 1
wherein the data processing device has a code
reader for receiving a control code from the data
validation and control device connected to the
control main processor and a dedicated data
modification processor in turn connected to a
modification matrix array located in a dedicated
permanent non-volatile memory circuit for
modifying the data in accordance with predefined
parameters.
3. A process control apparatus as claimed in claim 2
wherein the code reader has means for decoding
the control code to determine a storage location
for the data in the data processing device.
4. A process control apparatus as claimed in claims
2 or 3 wherein the modification processor has
means for retrieval of data from the storage
device and the modification matrix array located
in a dedicated non-volatile memory circuit, on
receipt of a request from a request generation
and reporting means, for modifying the data and
returning it to the main processor for supply to
the reporting means.
5. A process control apparatus as claimed in claim 4
wherein the request generation and reporting
means includes a plurality of workstations with
request entry means and reporting means
incorporating printing and display means.
6. A process control apparatus as claimed in any
preceding claim wherein the comparison means
comprises a sixteen bit data comparator with
associated memory circuits and dedicated cache
memory of in excess of two hundred kilobytes, a
sixteen to thirty two bit converter with an
accompanying sixteen bit latch and a comparator
controller for connection to the sixteen bit data
comparator, the sixteen to thirty two bit
converter and a control code generator.
7. A process control apparatus as claimed in claim 6
wherein the comparator controller comprises
means for reception of a conversion
complete signal from the sixteen to thirty
two bit converter;
means for reception of a validity signal
from the sixteen bit data comparator for
indicating a valid information condition
based on parameters and data history
located within a comparator memory circuit;
and
means for supplying a signal to the code
generator.
8. A process control apparatus as claimed in claim
7, wherein the code generator has receiving means
for receiving the signal from the comparator
controller, and transmission means for
transmitting a control code to the data
processing device in response to the signal
received.
9. A process control apparatus as claimed in any
preceding claim wherein the validation and
control node has a fixed disk with a capacity of
greater than eighty megabytes, a data entry
means, a display, and a memory circuit in excess
of eight megabytes having a processor operating
at greater than twenty five megahertz connected
to a co-axial local area network operating at a
speed of greater than four megahertz.
10. A process control apparatus as claimed in any
preceding claim wherein the network activity
monitoring device incorporates means for
receiving data from the data entry device and
transmitting means for transmission of the data
across the local area network during periods of
reduced network activity.
11. A process control apparatus as claimed in claim
10 wherein the dedicated network activity
monitoring device is directly connected to the
data entry device and has an associated local
storage device.
12. A process control apparatus as claimed in claim
11, wherein the data entry device comprises a
plurality of workstations, each workstation
having a data entry means, an initial data
validation means, a fixed disk, a display, and a
memory circuit;
13. A process control apparatus as claimed in any
preceding claim wherein the comparator
incorporates data transfer means for receiving
data transmitted from the main processor and
transmitting the data to the comparator
associated memory circuits to form a data entry
history.
14. A process control apparatus as claimed in any
preceding claim wherein the data entry means is
provided by a pattern recognition device for
automatically transferring the data from a
written format to a format suitable for use by
the data entry device.
15. A process control apparatus as claimed in claim
14, wherein the pattern recognition device is a
scanner.
16. A process control apparatus as claimed in claim
14, wherein the pattern recognition device is a
bar code reader.
17. A process control apparatus as hereinbefore
described, with reference to and as illustrated
in the accompanying drawings.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9418181A GB2293032B (en) | 1994-09-09 | 1994-09-09 | A process control apparatus for handling large quantities of data |
| BE9400863A BE1006934A6 (en) | 1994-09-09 | 1994-09-23 | Automatic control process. |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB9418181A GB2293032B (en) | 1994-09-09 | 1994-09-09 | A process control apparatus for handling large quantities of data |
| BE9400863A BE1006934A6 (en) | 1994-09-09 | 1994-09-23 | Automatic control process. |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| GB9418181D0 GB9418181D0 (en) | 1994-10-26 |
| GB2293032A true GB2293032A (en) | 1996-03-13 |
| GB2293032B GB2293032B (en) | 1999-07-14 |
Family
ID=25662923
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB9418181A Expired - Fee Related GB2293032B (en) | 1994-09-09 | 1994-09-09 | A process control apparatus for handling large quantities of data |
Country Status (2)
| Country | Link |
|---|---|
| BE (1) | BE1006934A6 (en) |
| GB (1) | GB2293032B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2384877A (en) * | 2002-02-01 | 2003-08-06 | Micron Technology Inc | System and Method for Incorporating Validated Cells into a Library |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4603232A (en) * | 1984-09-24 | 1986-07-29 | Npd Research, Inc. | Rapid market survey collection and dissemination method |
| GB2266171A (en) * | 1992-04-10 | 1993-10-20 | Gandon Treasury Services | Transaction processing. |
| GB2273587A (en) * | 1992-12-21 | 1994-06-22 | Calder Ltd | Data communications and document generation |
-
1994
- 1994-09-09 GB GB9418181A patent/GB2293032B/en not_active Expired - Fee Related
- 1994-09-23 BE BE9400863A patent/BE1006934A6/en not_active IP Right Cessation
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4603232A (en) * | 1984-09-24 | 1986-07-29 | Npd Research, Inc. | Rapid market survey collection and dissemination method |
| GB2266171A (en) * | 1992-04-10 | 1993-10-20 | Gandon Treasury Services | Transaction processing. |
| GB2273587A (en) * | 1992-12-21 | 1994-06-22 | Calder Ltd | Data communications and document generation |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2384877A (en) * | 2002-02-01 | 2003-08-06 | Micron Technology Inc | System and Method for Incorporating Validated Cells into a Library |
| GB2384877B (en) * | 2002-02-01 | 2004-12-15 | Micron Technology Inc | System and method for generating high-quality libraries |
Also Published As
| Publication number | Publication date |
|---|---|
| BE1006934A6 (en) | 1995-01-31 |
| GB2293032B (en) | 1999-07-14 |
| GB9418181D0 (en) | 1994-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US4041472A (en) | Data processing internal communications system having plural time-shared intercommunication buses and inter-bus communication means | |
| US4964038A (en) | Data processing system having automatic address allocation arrangements for addressing interface cards | |
| US4523272A (en) | Bus selection control in a data transmission apparatus for a multiprocessor system | |
| US4403286A (en) | Balancing data-processing work loads | |
| US4814981A (en) | Cache invalidate protocol for digital data processing system | |
| US3866175A (en) | Data communication system between a central computer and a plurality of data terminals | |
| US5448702A (en) | Adapters with descriptor queue management capability | |
| JP2644554B2 (en) | Information processing system | |
| US5146607A (en) | Method and apparatus for sharing information between a plurality of processing units | |
| US4814982A (en) | Reconfigurable, multiprocessor system with protected, multiple, memories | |
| US4982321A (en) | Dual bus system | |
| US4591973A (en) | Input/output system and method for digital computers | |
| EP0121373A2 (en) | Multilevel controller for a cache memory interface in a multiprocessing system | |
| JPH02288927A (en) | Shared memory management system | |
| US6530043B1 (en) | Write data error checking in a PCI Bus system | |
| TW359777B (en) | Arbitration apparatus using least recently used algorithm | |
| AU597980B2 (en) | Apparatus and method for interprocessor communication | |
| EP0265575B1 (en) | Data processing system having automatic address allocation arrangements for addressing interface cards | |
| AU603433B2 (en) | Bus adapter unit for digital data processing system | |
| GB2293032A (en) | Handling extremely large quantities of data | |
| US5379389A (en) | Method for transmitting commands excluded from a predefined command set | |
| CN101751311A (en) | Request processing device, request processing system, and access testing method | |
| US5721821A (en) | Information processing system having ring fashioned bus connection | |
| IE72083B1 (en) | A process control apparatus | |
| IES61898B2 (en) | "A process control apparatus" |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PCNP | Patent ceased through non-payment of renewal fee |
Effective date: 20000909 |