GB2295699A - Handling different data formats in inter-computer communications - Google Patents

Abstract

An inter-computer communications apparatus provides transparent communication between a number of data sources and a host computer where the data sources have a variety of different hardware and software configurations. The data is received by the apparatus and its type determined by comparison with a number of stored data types using a combination of a store 30, a latch 25, a page decoder 29 and a comparator 33. The data is then converted at 32 to the host format using an import filter from non-volatile store 38 in accordance with the identified data type and is then available for processing by the host computer and re-transmission where required thereby, providing inter-computer communication in an automatic and transparent manner. <IMAGE>

Description

"An inter-computer communications aPParatus1 The present invention relates to a method for intercomputer communications and more particularly to such methods used for improving the transparency and efficiency of communications between computer systems.

The term "inter-computer communications" refers to communication between a number of data processing entities distributed across a number of computers.

In many distributed data processing systems, it is common to transfer data from a number of disparate and often geographically remote sources to a central or target computer system. These sources frequently use different hardware and software platforms to other data sources and to the central target computer. The data may be transferred for storage, used to maintain an upto-date nucleus of information or for centralised processing to provide a result which is then retransferred to the source.

When the hardware and software platforms are identical this transfer of data is relatively simple and error free. However, as the number of different hardware and software platforms increase so do the problems increase and it becomes virtually impossible to transfer data in a transparent manner because of the conversions required. Data may be converted using an output filter, at each source, into a format suitable for use by the central target computer however, this is not suitable when the sources may be connected to a number of centralised target computers and cannot take the time to translate the data into a format suitable for use by each type of target computer.

Alternatively, the hardware and software platforms used by the sources may be changed to conform to the central computer, thus providing a homogeneous environment for data transfer. This is frequently impractical as the data sources may be numerous and developed over a long period of time, hence the cost of replacing the hardware and/or software becomes prohibitive. Additionally, all of the sources may not be owned by the owner of the central target computer placing translation or replacement requirements beyond their control and thus impossible.

Another solution is for the central computer to receive data from the disparate sources and store it for translation using an import filter at some later time.

This solution, while often the most achievable, requires user intervention at the central computer to identify the source of the information, recognise the format used by that source and select an import filter accordingly.

This adversely affects the functionality of the central target computer and more particularly precludes effective real time operation, so frequently required in such systems.

There is therefore a need for a communication apparatus which will provide communications between disparate data sources which will overcome the aforementioned problems.

Accordingly, there is provided an inter-computer communications apparatus comprising: means for identifying the host computer format signature; storage means for a multiplicity of different input data format signatures; an associated import filter for each data format signature; a data identifier for recognising the format signature of inputted data; format matching means for pairing the format of the inputted data signature with the associated import filter to generate a format match signal; a format converter for converting the format of the inputted data into the host computer data format on receipt of the format match signal; and means for transferring the inputted data, in host computer format, to the host computer.

An inter-computer communications apparatus formed in accordance with the invention has a number of advantages. Inputted data may be accepted in any format known to the host computer thus obviating the need for a homogenous data transfer environment and for manual intervention when importing data to the host computer.

In this way the host computer may communicate in an automatic way with a great variety of disparate systems.

Preferably, the apparatus comprises additional means for receiving the previously inputted data, post processing from the host computer, and re-formatting the data for transfer. Data transferred from a source computer, to the target or host computer for processing may have the results of that processing returned the source computer in a format suitable for use by the course computer without further processing.

Preferably, there is provided: classification means for typing the inputted data, in host computer format; and means for directing the typed data to a portion of the host computer memory appropriate to the typed data.

In this way processing requirements on the host computer system are minimised, by ensuring the data received from the disparate sources is routed to the most appropriate area of system memory. Thus, searching of the fixed disks and/or system volatile memory is minimised.

Ideally the storage means is provided by a temporary storage buffer for temporarily storing inputted data from another computer. In this way identification of the inputted data source can begin as soon as the first data word is received.

Preferably, the temporary storage buffer is of variable size for efficiently storing inputted data. Thus, system resources can be allocated as and when required.

Preferably, the temporary storage buffer temporarily stores inputted format data in thirty-two bit data words, thus data is immediately available for use.

Ideally, the means for transferring the inputted data to the host computer includes a host temporary storage buffer. In this way, the converted data is immediately communicated to the host computer ensuring that the system is optimally available for further data conversion.

Preferably, the host temporary storage buffer is of variable size for efficiently storing data in host computer format in thirty-two bit data words.

Preferably, the data identifier has a data latch for holding a portion of the inputted data, thereby holding the portion of the inputted data containing the data signature constant until a match is found.

Ideally, the data latch is edge triggered, thus, the data signature is held until a data match signal is received.

Preferably, the data latch comprises a plurality of D type registers having a common clock input connected to the system clock, thereby latching through the inputted data upon receipt of the system clock pulse train.

In one embodiment the data identifier includes a page decoder connected to the storage means. Thus, the storage means may be accessed in an efficient manner.

In a preferred embodiment the format matching means comprises: a comparator having a pair of input ports, one input port fed with the inputted data signature; and a counter for incrementally addressing the storage means to feed input data signatures to the other input port.

In this way the match for the inputted data format is found in a relatively simple and highly efficient manner.

Preferably, the comparator has a match output port for generating the format match signal connected to a counter input and a converter output port, connected to the format converter.

Ideally, the match output port is also connected to the system clock and the data identifier, thereby allowing the signal on the match output port to override the system clock and prevent release of the inputted data to the converter until the format match signal is generated.

According to the invention there is provided a method for inter-computer communications comprising the steps of: receiving inputted data; temporarily storing the inputted data; identifying the inputted data format signature; matching the format signature of the inputted data with one of a plurality of previously stored data format signatures; generating a format match signal; releasing the inputted data; converting the format of the inputted data to a host computer data format; and transferring the inputted data in host computer data format to the host computer.

In one embodiment of the invention after processing in the host computer the data is additionally reformatted into the original data format and returned.

In a preferred arrangement that portion of the temporarily stored inputted data providing the inputted data format signature is held separately to await the format match signal initiating release of the inputted data.

In a particularly preferred arrangement the following steps are performed of: incrementally retrieving previously stored data format signatures; comparing each retrieved data format signature with the inputted data format signature; and generating the format match signal when the format signatures are identical.

In a preferred embodiment the transferring of the inputted data in host computer format to the host computer comprising the steps of: typing the inputted data by matching the data with a previously stored data type; and storing the data in host computer memory.

The only use of latches and comparators heretofore in tasks such as this is described British Patent Specification No. GB 2 193 017B describes the use of latches and comparators in combination with a computer system. The methods and apparatus described efficiently address memory blocks and significantly improve access speeds, however, it does not address the problem of transferring data between a variety of computer systems.

The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which: Fig. 1 is a general block diagram of a communications apparatus according to the invention; Fig. 2 is a more detailed block diagram illustrating the pre-processing stages of Fig. 1; and Fig. 3 is a more detailed block diagram illustrating the processing and post processing stages of Fig. 1.

For the purposes of this description, specific architectures, processors, memory devices, timing and performance details have been omitted in order not to unnecessarily obscure the present invention. Thus the constituent components of the invention have been described in terms of functionality as many ways of achieving said functionality will be readily apparent to those skilled in the art and are of no importance to the operation of the invention. This is particularly the case as in most instances system requirements will dictate choice of components.

Referring now to Fig. 1, there is provided an intercomputer communications apparatus according to the invention, indicated generally by the reference numeral 1. Pre-processing stages indicated at 2 shows a data source 3 electrically connected to a temporary storage area 4 in turn electrically connected to a data type identification unit 5 and a data translator 6.

Processing and post processing stages indicated at 7 comprise a data processing request identification unit 8 connected to a data processor 9. The data processor 9 has an output result buffer 10 electrically connected to the data translator 6 in turn connected to the data source 3.

Referring to Fig. 2, the pre-processing stages 2 are shown in more detail. The pre-processing stages 2 include a storage means for different input data format signatures provided by a thirty-two bit lookaside buffer 30, an associated import filter for each data format signature stored in a dedicated non-volatile memory 38 and a data identifier comprising a thirty-two bit edge triggered latch 25 and a page decoder 29. The preprocessing stages 2 also include a format matching means comprising a thirty-two bit comparator 27 and a counter 34 each communicating with the thirty-two bit lookaside buffer 30.

Four disparate data sources 20 are shown, each having different hardware and software characteristics communicating with a data multiplexer 21. The data multiplexer 21 is connected to a validity checker 22 formed from a number of programable read memory devices having an associated cache memory 23 and an enable signal 23A. The validity checker 22 is also connected to a thirty-two bit extendable temporary storage buffer 24 which in turn is electrically connected to the thirty-two bit edge triggered latch 25. The thirty-two bit edge triggered latch 25 is a non- transparent latch formed from an array of D type registers having a common clock input 26. The thirty-two bit edge triggered latch 25 is connected to a thirty-two bit comparator 27, by a thirty-two bit data bus 28.The two most significant data lines D31 and D30 of the thirty-two bit data bus 28 are connected to the page decoder 29. The page decoder 29 is in turn connected to the thirty-two bit look aside buffer 30. The thirty-two bit look aside buffer 30 is itself connected to the thirty-two bit comparator 27 by a second thirty-two bit data bus 31 similar to the thirty-two bit data bus 28. The thirty-two bit data bus 28 and the second thirty-two bit data bus 31 are connected to the inputs of the thirty-two bit comparator 27.

The output from the thirty-two comparator 27 is electrically connected to a format converter 32. The thirty-two bit comparator 27 also has a format match output 33 which is electrically connected to, the counter 34, a system clock 35 and the latch clock input 26. The counter 34 is connected to the thirty-two bit look aside buffer 30 and an address decoder 36. The address decoder 36 is in turn connected to an import filter select 37 communicating with the dedicated nonvolatile memory 38.

Referring now to Fig. 3 the processing and postprocessing stages 7 are shown in more detail. The address decoder 36 of Fig. 2 is connected to an output filter select 39.

The format converter 32 (see Fig. 2) is connected to a second thirty-two bit latch 41. The thirty-two bit latch 41 is a transparent latch whose output directly follows the applied input. The thirty-two bit latch 41 has an output bus 42 connected to a system volatile memory 43 and a memory address decode unit 44. The memory address decode unit 44 has an output thirty-two bit address bus 45 and a memory decode disable input 46.

The disable input 46 is connected to a system processor 52 itself connected to a system bus 47.

The output thirty-two bit address bus 45 is connected to a system memory 46 in turn connected to the system volatile memory 43 by the system bus 47. The system memory 46 as is conventional stores of programs and data. The system processor 52 is also connected to an output buffer 48. The output buffer 48 is also connected to an output converter 49. The output converter 49 has an output converter filter input port 50 connected to the output filter select 39 and a converter output connection 51 connected to the data multiplexer 21.

In use, data is transmitted from the data sources 20 by means of direct connection, modem and network connections to the data multiplexer 21. The data multiplexer 21 receives the data from the data source 20 and routes it to the validity checker 22. The validity checker 22 receives the data and places it in temporary storage in the cache memory 23. The data is then processed in accordance with the instructions stored in the programmable read only memory devices to determine a data valid condition. When a data valid condition is recognised, the enable signal 23A is switched to a logic one or high state by the validity checker 22 to identify a data valid condition. The presence of this signal enables the temporary storage buffer 24.

On receipt of the data valid signal from the validity checker 22, the temporary storage buffer 24 begins to store data. The data is stored in a matrix of data words thirty-two bits long. The temporary storage buffer 24 uses a first-in-first-out (FIFO) data storage system. The first word of the incoming data stream is immediately passed to the thirty-two bit edge triggered latch 25 and latched through. The first thirty-two bit data word is latched through in this way due to an indeterminate state being present on the format match output 33 allowing the system clock 35 to act upon the common clock input 26. This data word is then held on the thirty-two bit data bus 28.

This first thirty-two bit data word from the edge triggered latch 25 switches the format match output 33 to a fixed logic one or high state. This over rides the action of the system clock 35 on the edge triggered latch 25, by tying the clock input 26 high. In this way no data words are lost as further information cannot be latched through.

The two most significant data bits, of the digital data word held on the data lines D31 and D30 of the thirty-two bit data bus 28 are then passed to the page decoder 29.

The values present on the data lines D31 and D30 being 00, 01, 10 or 11 are used to determine the output from the page decoder 29 selecting one of four pages from the thirty-two bit look aside buffer 30. The data contents of the location being addressed within the thirty-two bit look aside buffer 30 is then output to the thirtytwo bit data bus 31. The output from the counter 34 is initially set at 0, thus, the first entry of the page selected by the page decoder 29 is addressed.

This output is compared by the thirty-two bit comparator 27 with the data word on the thirty-two bit data bus 28.

If the data words do not match then the process is repeated. The output from the counter 34 is used to sequentially index the data entries within the previously selected page. This is achieved by incrementing the counter 34 to address the next location address on the given page. The comparison is repeated with the new data word on the second thirty-two bit data bus 31. To avoid counter over-run the counter 34 is incremented after a timed delay greater than the time taken to address the next location within the thirty-two bit look aside buffer 30 providing the format match output 33 remains high.

This incremental comparison cycle process is repeated until the digital words on the thirty-two bit data bus 31 and the thirty-two bit data bus 28 are the same.

When the data words are the same, the format match output 33 is switched to an open collector state thereby disabling the counter 34 and preventing further operation. When the format match output 33 is switched to open collector the system clock 35 can latch through data from the temporary storage buffer 24. The data is latched directly through the thirty-two bit comparator 27 to the format converter 32 on each successive rising and falling edge of the system clock 35.

The output from the counter 34 is sent to the address decoder 36. The output from the address decoder 36 feeds the input filter select 37 and the appropriate import filter is retrieved from the dedicated non volatile memory 38 and then fed to the format converter 32. As data passes through the format converter 32 it is converted into a format suitable for use by the system processor 52. Having being converted by the format converter 32, the data is then sent to the second thirty-two bit latch 41.

The first data word is latched through the second thirty-two bit latch 41 on to the output bus 42 and hence to the memory address decode unit 44. The memory address decode unit 44 receives the first data word and stores it until the disable input 46 is activated. The first data word stored in the memory address decode unit 44 is then passed on to the output thirty-two bit address bit 45 and is used to select an appropriate memory location in the system memory 46. The memory location in the system memory 46 is relayed to the system processor 52 on the system bus 47. Upon receipt of this location address the system processor 52 generates a logic one or high state on an output to activate the disable input 46 of the memory address decode unit 44 preventing another data word from changing the memory location within the system memory 46 from being addressed.

The remaining words of the converted data stream are passed directly from the second thirty-two bit latch 41 to the system volatile memory 43 by the output bus 42.

Processing is then conducted by the system processor 52 using functions stored in the system memory 46 and the data stored in the system volatile memory 43. The system memory 46 has banks of associated functions or programs stored in adjacent memory blocks so that data arriving is addressed in the most efficient manner. In this way, the required functions are the only ones addressed as they have been selected using the data word stored in the memory address decode unit 44.

The final result of the processing request from a source is then passed from the system processor 52 to the output buffer 48. The output buffer 48 routes the result to the output converter 49 which is then converted using the output filter provided by the output filter select 39 and non-volatile memory unit 40 which is selected using the contents of the counter 34. This converted result is then passed to the output converter connection 51 until the next available transmission period.

When the data may be transmitted the data multiplexer 21 selects the appropriate data source 20 to which the information is to be routed and transmission occurs as before.

It will be appreciated by one skilled in the art, that the invention herein described provides for a significantly improved method of transferring data between disparate sources in a transparent manner. In this way the data may be transferred from any data source whose characteristics are stored in the thirtytwo bit look aside buffer. It will be further appreciated that the addition of a new characteristic transfer format may be easily added by the simple addition of an entry to the lookaside buffer.

It will be appreciated that the memory address decode unit may be configured in isolation or in combination with another device, to store inputted data in host computer format, in a manner responsive to one or more rules. These rules, in turn, may be configured in either hardware or software and may be easily modified or replaced allowing inputted data to be efficiently grouped for subsequent processing.

This grouping of data is particularly useful and efficient when used for data reconciliation. For example, inputted data containing invoice information may be grouped and matched with corresponding data relating to purchase orders, held within the system memory.

It will thus be evident that large volumes of data may be automatically internally reconciled and therefore reduced. This reduction results in a small number of exceptions for which no data match exists, which may then be checked manually. This manual checking is only made possible because of the reduced number.

It will also be appreciated that the speed of the system may be greatly enhanced by limiting the number of entries in the look aside buffer and significantly increasing the number of pages which may be addressed.

Thereby eliminating the need for numerous incremental comparison cycles. It will also be appreciated that the identification of the data type prior to the processing stage enables the data to be stored in a suitable format and also for the function or operation being called, to be efficiently accessed from the system memory.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail. For example, it is envisaged that the validity checker be of any suitable type conducted either in hardware or software. Further, it is envisaged that the apparatus described may easily be modified and could be used for operations other than thirty-two bit data transfers. The number of bits in each data word may be changed to sixteen bits and the components changed accordingly. Similarly, the thirtytwo bit latch may be of any suitable type using existing latches or application specific integrated circuits. It will be appreciated that the performance of the apparatus may be improved by integrating the functions of a number of the components hereinbefore described onto a single integrated circuit using VLSI techniques.

Claims (16)

Claims

1. An inter-computer communications apparatus for a host computer comprising: means for identifying the host computer format signature; storage means for a multiplicity of different input data format signatures; an associated import filter for each data format signature; a data identifier for recognising the format signature of inputted data; format matching means for pairing the format of the inputted data signature with the associated import filter to generate a format match signal; a format converter for converting the format of the inputted data into the host computer data format on receipt of the format match signal means for transferring the inputted data, in host computer format, to the host computer; means for receiving the previously inputted data, post processing from the host computer, and re-formatting the data for transfer.

2. An inter-computer communications apparatus as claimed in claim 1 in which there is provided classification means for typing the inputted data, in host computer format; and means for directing the typed data to a portion of the host computer memory appropriate to the typed data.

3. An inter-computer communications apparatus as claimed in any preceding claim wherein the storage means is provided by a thirty-two bit temporary storage buffer of variable size for efficiently and temporarily storing inputted data from another computer in thirty two bit data words.

4. An inter-computer communications apparatus as claimed in any preceding claim in which the means for transferring the inputted data to the host computer includes a host temporary storage buffer of variable size for efficiently storing data in host computer format.

5. An inter-computer communications apparatus as claimed in claim 8 wherein the host temporary storage buffer temporarily stores host computer format data in thirty-two bit data words.

6. An inter-computer communications apparatus as claimed in any preceding claim wherein the data identifier has a data latch for holding a portion of the inputted data.

7. An inter-computer communications apparatus as claimed in claim 6 wherein the data latch is edge triggered comprising a plurality of D type registers having a common clock input connected to the system clock.

8. An inter-computer communications apparatus as claimed in any of claim 6 or claim 7 wherein the data identifier includes a page decoder connected to the storage means.

9. An inter-computer communications apparatus as claimed in any preceding claim wherein the format matching means comprises: a comparator having a pair of input ports, one input port fed with the inputted data signature; a match output port for generating the format match signal connected to a counter input; a converter output port, connected to the format converter; and a counter for incrementally addressing the storage means to feed input data signatures to the other input port.

10. An inter-computer communications apparatus as claimed in claim 9 wherein the match output port is also connected to the system clock and the data identifier.

11. A method for inter-computer communications comprising the steps of: receiving inputted data; temporarily storing the inputted data; identifying the inputted data format signature; matching the format signature of the inputted data with one of a plurality of previously stored data format signatures; generating a format match signal; releasing the inputted data; converting the format of the inputted data to a host computer data format; transferring the inputted data in host computer data format to the host computer; processing the inputted data in host computer format; reformatting the processed data into the original data format and returned.

12. A method as claimed in claim 11 in which that portion of the temporarily stored inputted data providing the inputted data format signature is held separately to await the format match signal initiating release of the inputted data.

13. A method as claimed in any of claim 12 in which the following steps are performed of: incrementally retrieving previously stored data format signatures; comparing each retrieved data format signature with the inputted data format signature; and generating the format match signal when the format signatures are identical.

14. A method as claimed in any of claims 11 to 13 in which the transferring of the inputted data in host computer format to the host computer comprising the steps of: typing the inputted data by matching the data with a previously stored data type; and storing the data in host computer memory.

15. A method as claimed in claim 14 wherein the data is stored in a portion of the host computer memory according to the data type.

16. A method as claimed in any of claims 11 to 15 wherein inputted data received from another computer is validated before being temporarily stored.