JPH0527945A - Record transforming device - Google Patents

Abstract

PURPOSE:To transform the record of variable length data dividing the inside into plural variable length fields to the fixed length record of a record length and a field length designated in advance based on communication with another processor in the record transforming device for used data processing, etc., utilizes a computer. CONSTITUTION:According to the communication with the other processor, a control part 104 outputs a select signal based on inputted data and data stored in a data format storage part 105, a stream selection part 103 outputs either the inputted data or complementary data stored in a complementary data storage part 101 based on the select signal and thus, the record of the variable length data can be transformed to the record of the fixed length data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a record conversion device used for data processing using a computer.

[0002]

2. Description of the Related Art In a 100% search of a database, a column of variable length data called a record may be processed. In order to improve the performance of processing on a record string, it has become common to use a processing device composed of dedicated hardware.

In order to further improve the processing performance, a plurality of processing devices having different functions may be connected and pipeline processing may be performed on a data stream of a record string. In the case of performing pipeline processing in this way, in a processing device having a certain function, the data accepted by the processing device may be fixed length data due to restrictions resulting from the improvement of the circuit configuration and the efficiency of the algorithm. is there. In such a case, it is necessary to convert a record string composed of variable length data into a record string of fixed length data and input it to the processing device.

For example, a record 906 having a format as shown in FIG. 9, that is, a certain variable-length record is further divided into a plurality of variable-length fields, and a record boundary 902 and a field boundary 903 are unit data 904.
It is necessary to convert the record that can be recognized by the value of the tag 905 attached for each into a record having a record length and a field length specified in advance.

Conventionally, such record conversion processing is often realized by software.

[0006]

However, in the above-mentioned conventional method, the record conversion process by software becomes a bottleneck and it is difficult to sufficiently bring out the performance of the processing device constituted by hardware.

The present invention solves the above-mentioned problems of the prior art, so that a record composed of variable length data can be a record having a predetermined record length and field length.
Excessive input data will be truncated, and data shortage will be complemented by pre-designated data for record conversion.
In addition, it is another object of the present invention to provide a record conversion device that transfers data while communicating with other processing devices placed before and after.

[0008]

To achieve this object, the present invention firstly provides a complementary data storage unit for storing complementary data, and a stream storage unit for inputting and holding data based on a control signal. A stream selection unit that selects and outputs either one of the data in the complementary data storage unit and the data held in the stream storage unit based on a selection signal; a data format storage unit that stores information regarding the format of output data; The control unit outputs a control signal to the stream storage unit based on data in the stream storage unit and the data format storage unit, and outputs a selection signal to the stream selection unit.

Secondly, the control unit controls the input / output of data based on the signals from the first processing unit whose output is the input of the stream storage unit and the second processing unit whose input is the output of the stream selection unit. It is configured to do.

[0010]

According to the present invention, according to the above configuration, firstly, the stream storage unit inputs and holds data based on the control signal and outputs it to the stream selection unit, and the control unit stores the input data and the data format storage unit. Based on the data, the stream storage unit outputs a control signal related to data input / output and a selection signal for selecting an output to the stream selection unit, and the stream selection unit outputs the stream selection unit based on the selection signal from the control unit. By outputting either one of the data input to and the complementary data stored in the complementary data storage unit, the record of the variable length data can be converted into the record of the fixed length data.

Secondly, the control unit communicates with other processing units placed before and after the present apparatus for data transfer, thereby connecting processing units having different functions to each other. It becomes possible to perform pipeline processing on the stream.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a record conversion apparatus according to an embodiment of the present invention. In FIG. 1, 1
01 is a complementary data storage unit, 102 is a stream storage unit,
103 is a stream selection unit, 104 is a control unit, 105 is a data format storage unit, 106 is an input stream, 107 is a stream input control signal, 108 is an output stream, 1
09 is a stream output control signal, 110 is an input control signal, 111 is a data type, 112 is a selection signal, and 113 is a clock.

Before explaining the operation of the record converting apparatus,
The format of data handled by the record conversion apparatus in this embodiment will be described. FIG. 2 is a structural diagram of data handled by the record conversion apparatus in this embodiment. In FIG. 2, 2
01 is a stream, 202 is a record, 203 is a field, 204 is data, 205 is a tag, 206 is a field terminator, 207 is a record terminator, and 208 is a stream terminator.

The stream 201 is a concatenation of records 202 composed of variable length data, and is terminated by a stream terminator 208. Record 202 is field terminator 2
It is divided into a plurality of fields 203 by 06 and ends with a record terminator 207. The range of the record 202 is from the beginning of the stream 201 or immediately after the record terminator 207 of the preceding record 202 to the record terminator 207 encountered first. The range of the field 203 is from immediately after the field terminator 206 of the beginning or the preceding field 203 of the record 202 including the field 203 to the field terminator 206 or the record terminator 207 that is encountered first.

The field terminator 206, the record terminator 207, and the stream terminator 208 are distinguished by the tag attached to the data. The tag used in the record conversion apparatus in this embodiment is attached to data of every 2 bytes and represents four types of data types: general data, field terminator, record terminator, stream terminator.

In the record conversion apparatus of this embodiment, the field terminator 206 and the record terminator 207 are used.
And the data part of the stream terminator 208 is not used.

Next, the operation of the record conversion device will be described.
First, the operation of each block in FIG. 1 will be described, and then the operation of the entire record conversion apparatus will be described using a flowchart.

The complementary data storage unit 101 is a storage device for storing complementary data for compensating for the shortage of output records when converting into an output record of a size larger than the input record, and before the conversion processing. It has a function to store arbitrary data. A tag representing general data is attached to the complementary data.

The stream storage unit 102 holds the stream element of the input stream 106 according to the input control signal 110, and sends it to the stream selection unit 103. Further, the data type 111 of the held stream element is set to the control unit 10.
Tell 4. Here, a stream element is a pair of 2-byte data and a tag.

The stream selection unit 103 selects one of the complementary stream element from the complementary data storage unit 101 or the stream element from the stream storage unit 102 according to the selection signal 112, and outputs it as the output stream 108.

The data format storage unit 105 is a storage device for storing information regarding the data structure of the record after conversion, and like the complementary data storage unit 101, stores arbitrary data before conversion processing. It has a function for In the record conversion device of this embodiment, the end position of each field is used as information regarding the data structure of the converted record. That is, the position of the field terminator is represented by an offset value from the beginning of the record, and the fields are stored in the data format storage unit 105 in an array in the order of appearance of the fields.

The operation of the control unit 104 will be described with reference to the drawings. FIG. 3 is a block diagram showing the configuration of the control unit. In FIG. 3, reference numeral 301 denotes a sequence controller, 30
2 is a field number counter, 303 is a record length counter, 304 is a comparator, 107a is a stream input completion signal, 107b is a stream input permission signal, 109a is an output stream valid signal, 109b is an output stream request signal, 114a is an address, 114b. Is the request end child position.

The sequence control unit 301 is constructed by using a state transition circuit and realizes a processing flow described later.

FIG. 4 is a diagram showing a data reference relationship between the field number counter 302, the record length counter 303, and the record format recording unit 105.

The field number counter 302 is a circuit for counting the number of fields that have already been output in the record being converted. Field number counter 30
The value of 2 is used as the address 114a when referring to the data format storage unit 105.

The record length counter 303 is a circuit for counting the number of stream elements already output in the record under conversion processing.

The comparator 304 compares the value obtained by referring to the data format storage unit 105 at the address 114a, that is, the request end child position 114b of the field and the value of the record length counter 303, and the values of both are matched. The sequence control unit 301 is notified of whether or not it is done.

The stream input completion signal 107a and the stream input permission signal 107b constitute the stream input control signal 107 in FIG. The output stream valid signal 109a and the output stream request signal 109b are shown in FIG.
The stream output control signal 109 in
14a and the request end child position 114b are 114 in FIG.
Make up. The stream input control signal 107 and the stream output control signal 109 are signals for controlling the exchange of streams with other devices connected before and after the record conversion device.

The operation of the control unit 104 will be described with reference to FIGS. The sequence control unit 301 uses the stream input permission signal 107b and the output stream request signal 109b.
Is enabled, the input control signal 110 is enabled and the stream element of the input stream 106 is held in the stream storage unit 102, and then the stream input completion signal 107a.
To enable. Next, the selection signal 112 is determined based on the comparison result in the comparator 304 and the data type 111, and the output stream valid signal 109a is validated.

The field number counter 302 includes a comparator 3
A value is updated by detecting a change when the output stream request signal 109b becomes valid in a state where the coincidence signal is output from 04, that is, in a state in which the stream element for one field is completely output. At this time, if the data type 111 represents a record terminator, the field number counter 3
02 and the record length counter 303 are both cleared.

The record length counter 303 detects a change when the output stream request signal 109b becomes valid and updates the value.

The operation of each block in FIG. 1 has been described above. Next, the operation of the entire record conversion apparatus in this embodiment will be described with reference to the drawings.

FIG. 5 is a flow chart showing the operation of the entire record conversion apparatus in this embodiment. The record conversion process is composed of a transfer process and a complementary process. FIG. 6 is a flowchart regarding the transfer processing, and FIG. 7 is a flowchart regarding the complementing processing.

In the initialization shown in FIG. 5, the record conversion process is set to the transfer process, and the field number counter 302 and the record length counter 303 are cleared. After the initialization, the record conversion processing continues until the output of the stream end element 208 is completed.

In the transfer process (FIG. 6), it is first determined whether or not the value of the record length counter 303 matches the request end child position 114b of the corresponding field. When they match, the stream element of the input stream is continuously input until the end element of the record or field in the input stream is detected (end element search), and when the end element is detected, the end element is output. If the terminator output is a field terminator, 1 is added to the field number counter 302, and if it is not a field terminator, it is regarded as a record terminator and the field number counter 302 and the record length counter 30
3 is cleared and the transfer processing is ended.

If they do not match, the stream element is read from the input stream 106, the data type is checked, and if it is the end element of the record or field, the state is changed so as to shift to the complementary processing, and either If it is not the terminator, the read stream element is output as the output stream 108, and the transfer process ends.

In the complementing process (FIG. 7), it is first determined whether or not the value of the record length counter 303 matches the request end child position 114b of the corresponding field. If they match, the stream element read at the end of the transfer process, that is, the end element of the record or field is output as the output stream 108, and if the output end element is the field end element, the field number counter 302 If 1 is not added to the field terminator, it is regarded as a record terminator, the field number counter 302 and the record length counter 303 are cleared, the state is changed so as to shift to the transfer processing, and the complementary processing is terminated.

If they do not match, the complementary data stored in the complementary data storage unit 101 is output as the output stream 108, the record length counter is updated, and the complementary processing ends.

FIG. 8 is a diagram showing the correspondence between records before conversion and records after conversion. Reference numeral 801 is a record before conversion, and 802 is a record after conversion. Reference numeral 803 is complementary data complemented by the complementary processing, and 804 is data truncated by the termination search processing.

[0040]

As described above, according to the present invention, first, a record of variable length data can be converted into a record of fixed length data at high speed.

Secondly, it becomes possible to perform pipeline processing by connecting processing devices having different functions with different data formats.

[Brief description of drawings]

FIG. 1 is a block diagram of a record conversion device according to an embodiment of the present invention.

FIG. 2 is an input data structure diagram according to an embodiment of the present invention.

FIG. 3 is a block diagram of a control unit of a record conversion device according to an embodiment of the present invention.

FIG. 4 is a data reference relationship diagram according to an embodiment of the present invention.

FIG. 5 is a flowchart of a record conversion device according to an embodiment of the present invention.

FIG. 6 is a flowchart of a transfer process according to an embodiment of the present invention.

FIG. 7 is a flowchart of a complementary process in the embodiment of the present invention.

FIG. 8 is a correspondence diagram of input data and output data according to an embodiment of the present invention.

FIG. 9 is a data structure diagram of data to be processed by the record conversion device.

[Explanation of symbols]

101 Complementary data storage unit 102 stream storage unit 103 Stream selection unit 104 control unit 105 data format storage unit 106 input stream 107 Stream input control signal 108 output stream 109 Stream output control signal 110 Input control signal 111 data type 112 selection signal 113 clock 201 stream 202 records 203 field 204 data 205 tags 206 Field Terminator 207 record terminator 208 Stream terminator 301 Sequence control unit 302 field number counter 303 record length counter 304 comparator 305 Input control signal 306 data type 307 selection signal 308 Stream input completion signal 309 Stream input permission signal 310 Output stream valid signal 311 Output stream request signal 312 clock 313 address 314 Request end child position 801 Record before conversion 802 Conversion word record 901 stream 902 record boundary 903 field boundary 904 unit data 905 tags 906 record

Continued front page    (72) Inventor Noboru Tamura             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor ▲ Tsuru ▼ Takeshi Hayashi             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd.

Claims (2)

[Claims] 1. A complementary data storage unit that stores complementary data, a stream storage unit that inputs and holds data based on a control signal, and one of data of the complementary data storage unit and data of the stream storage unit is selected. A stream selection unit that selects and outputs based on a signal, a data format storage unit that stores information regarding the format of output data, and a control signal to the stream storage unit based on the data of the stream storage unit and the data format storage unit. A record conversion device comprising: a control unit that outputs and outputs a selection signal to the stream selection unit. 2. A control unit performs data input / output control based on signals from a first processing device that outputs an input of a stream storage unit and an output of a stream selection unit and a second processing device that receives an input of the stream selection unit. The record conversion device according to claim 1, wherein