JP2003044459A - Method for compressing and exchanging structured data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for compressing and exchanging data, by which a data amount can be reduced and data can be exchanged after simultaneously applying enciphering by using the structure information of structured data. SOLUTION: In a compressing (enciphering) module, the internal expression data of the structured data are separated to structure information and contents by using previously applied syntax designation information and further, they are compressed (enciphered) together. The compressed (enciphered) data are delivered from a transmitting side system through a network to a receiving side system. In an extending (deciphering) module, the received compressed (enciphered) data are restored into internal expression data of the structured data by using the syntax designation information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a compression method and a data exchange method in a system using structured data such as data exchange using structured data, application integration, system integration, and data storage.

[0002]

2. Description of the Related Art Conventionally, various methods such as run length, Huffman coding, and LZ (Zip-Lempel code) are known as lossless compression processing methods for text data. These are compression processing methods for general text data. When the structured document is in the form of text data, the above compression processing method for general text data can be used for the structured document. In this case, the compression process does not effectively use the information that the document is structured.

On the other hand, data exchange using structured data such as XML is becoming popular. This is basically text-formatted data.

[0004]

[Problems to be Solved by the Invention] As described above, XML
Since structured data such as is in text format,
It is becoming common to perform data exchange by compressing using the above-described compression processing method for general text data, but since structured data is basically text format data, there are the following problems. .

(1) Since the structured information is added as a tag, the amount of data increases. (2) Since the tag and content can be read as text, there is a risk that the content of the data will be stolen by eavesdropping or the like. (3) When using the data after the exchange, it is necessary to perform parsing processing such as lexical analysis and syntax analysis, which may cause processing overhead.

The present invention provides a compression method and a data exchange method for structured data, which makes it possible to exchange data after reducing the amount of data and simultaneously performing encryption by using the structure information of the structured data. The purpose is to do.

[0007]

In order to achieve the above object, the invention according to claim 1 is a data compression method for structured data, wherein a plurality of pieces of information defining the structure of the structured data to be processed are provided. Individual grammar rules can be specified by the identification information in the syntax specification information including the grammar rules, and the syntax tree representing the structure of the structured data is expressed by using the identification information specifying the grammar rules. According to, the structure information of the structured data and the content information are separated.

The invention according to claim 2 is the data compression method for structured data according to claim 1, characterized in that the content information obtained by separating from the structured data is further classified according to the data type. And

The invention according to claim 3 is the data compression method for structured data according to claim 1, wherein the data appearance position of the content data included in the content information is represented by a variable.

A fourth aspect of the present invention is a method for exchanging structured data, wherein the data compression method according to any one of the first to third aspects is used, and the data is data to be exchanged. The encrypted data is separated into structure information and content information, the separated structure information and content information are respectively compressed by a predetermined compression method or encrypted by a predetermined encryption method, and the compressed or encrypted structure information and content information are transmitted. It is characterized by doing.

According to a fifth aspect of the present invention, there is provided a data compression method for structured data, wherein syntax specification information including a plurality of grammatical rules, which are information defining a structure of structured data to be processed, is provided with grammars thereof. The step of holding the rule in the storage means so that the rule can be identified by the identification information, the grammatical rule representing the structure of the structured data to be processed is obtained, and the identification information of all the obtained grammatical rules is arranged to obtain the structural information. For those grammatical rules that are accompanied by content data while being generated, the content data is stored in the content information, and the identification information arranged in the structure information is provided with an index indicating that the content data is associated. And a step.

According to a sixth aspect of the present invention, there is provided a data compression or encryption method for structured data, wherein syntax designation information including a plurality of grammatical rules, which is information defining a structure of structured data to be processed, is provided. A step of holding the grammatical rules in a storage means so that they can be specified by the identification information, a grammatical rule representing the structure of the structured data to be processed is obtained, and the identification information of all the obtained grammatical rules are arranged. An index indicating that the structure information is generated and the content data is stored in the content information for those grammatical rules associated with the content data, and the identification information arranged in the structure information is associated with the content data. And a step of compressing or encrypting the structure information and the content information.

The invention according to claim 7 is a data decompression or decryption method for decompressing or decrypting structure information and content information compressed or encrypted by the data compression or encryption method according to claim 6. Decompressing or decrypting the compressed or encrypted structure information and content information, and extracting the identification information of the grammar rule from the expanded or decrypted structure information, and referring to the syntax designation information, the extracted identification information And a step of acquiring the grammatical rule corresponding to the grammar rule, the structured data having the data structure represented by the acquired grammatical rule is restored, and the content data associated with the grammatical rule is extracted from the corresponding content data. Restore the structured data by setting it to the corresponding position in the structured data. Characterized in that a step.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 and 2 are diagrams showing a basic system configuration and a processing procedure according to an embodiment of the present invention.

FIG. 1 shows a system configuration and a processing procedure for compressing or encrypting structured data. When compressing / encrypting, first the lexical / syntactic analysis module (103)
Analyzes the input structured data (101) on the basis of the syntax designation information (102) given in advance, and outputs a partial syntax tree string (104) appearing in the structured data (101).
And the content (105) that appears there are retrieved. These data are passed to the syntax compression (encryption) module (106) and the content compression (encryption) module (107), respectively. The syntax compression (encryption) module (106) compresses or encrypts the partial syntax tree sequence (104). Content compression (encryption) module (1
07) compresses or encrypts the content (105). The results of compression (encryption) by these modules are combined and output as compressed (encrypted) data (108).

FIG. 2 shows a system configuration and a processing procedure when decompressing or decrypting compressed (encrypted) data. In the case of decompression / decryption, contrary to the processing shown in FIG. 1, compressed (encrypted) data (201) is received as an input, and a partial syntax tree decompression (decryption) module (202) and content decompression (decryption) are performed. ) Deliver corresponding portions of the compressed (encrypted) data 201 to the module (203). Partial syntax tree decompression (decoding) module (202)
Decompresses (decrypts) the compressed (encrypted) partial syntax tree string based on the syntax designation information, and
04) is output. The content decompression (decryption) module (203) decompresses (decrypts) the compressed (encrypted) content and outputs the content (205). Finally, the composition module (206) embeds the content (205) on the partial syntax tree string (204) to obtain the original structured data (207).

FIG. 3 is a diagram showing a system configuration and a processing procedure when data exchange using structured data is performed using the system shown in FIGS. 1 and 2. First, the syntax designation information (301) for the structured data to be transmitted
And (302) in advance by the data transmission side system (30
3), the compression (encryption) module (304), the decompression (decryption) module (305), and the data receiving side system (306).

In the system shown in this figure, two copies of the same syntax designation information (301) and (302) are prepared. Data transmission system (303) and compression (encryption)
The module (304) uses the syntax designation information (301), and the decompression (decoding) module (305) and the lexical / syntax analysis module (313) in the data receiving system (306) use the syntax designation information (302). The case where it uses is illustrated.

The data transmission side system (303) parses the given input data based on the syntax designation information (301) and holds it in the form of internal data (307) by a syntax tree or the like. Structured data creation module (3
08) outputs structured data (309) in the text format from this internal data (307). The compression (encryption) module (304) that receives this separates the structured data (309) into structural information and content and compresses (encrypts) each as described in FIG. ) Data (310) and network (31
1) Send to the receiving side via.

On the receiving side, the received compressed (encrypted) data (310) is expanded (decrypted) module (305).
Then, the data is decompressed (decoded) as described in FIG. 2, restored to the original structured data (312), and then passed to the data receiving system (306). Data receiving system (306)
The lexical / syntactic analysis module (313) therein converts the structured data (312) into internal data (314) and then proceeds with data processing.

In this system configuration, since the structured data (309) is compressed when exchanging data, the amount of communication data passing through the network (311) is reduced.

FIG. 4 is an example of a system configuration in which the data exchange system of FIG. 3 is improved and a purging process is incorporated to eliminate the overhead of data compression (encryption) / decompression (decryption). Instead of the structured data creation module (308) in FIG. 3, compression (encryption)
The module (304) is connected to the data transmission system (30
The data transmission side system (401) of FIG. 4 is directly incorporated into 3). Similarly, decompression (decoding) instead of the lexical / syntactic analysis module (313) in FIG.
The module (305) is connected to the data receiving system (30
The data receiving side system (402) of FIG. 4 is directly incorporated in 6).

By adopting such a system configuration, the internal representation of the data is compressed (encrypted) data without being converted into structured data, so that the overhead generated in the data exchange system of FIG. 3 is eliminated. It Furthermore, rather than simply eliminating the overhead, rather than sending the structured data directly without any compression (encryption) module (304) and decompression (decryption) module (305), It is possible to improve the performance. In fact, since the compressed (encrypted) data (403) transmitted in FIG. 4 already includes syntax information, the internal data (404) that is a syntax tree or the like without performing the lexical / syntactic analysis in the data receiving system 402. ) Can be obtained.

5 and 6 show specific examples of compression / encryption.

In FIG. 5, (5.1) is syntax designation information (102, 301, 302, etc. in FIGS. 1 to 4) consisting of a plurality of syntax definition information for creating an address book, which is XML.
It is described in the definition format called DTD for documents.
(5.2) is XML structured data (101, 207, 309, 312 in FIGS. 1 to 4) using this DTD, and is assumed to be a search result from the address book database. For later reference, the syntax specification information (5.1) includes
They are numbered 1 to 15. This number identifies one line of information in the syntax designation information (5.1), and this number data is not included in the syntax designation information (5.1). As can be seen from the structured data (5.2), in the structured data (5.2), more than half of the data is the tag definition that defines the syntax, and similar syntax appears repeatedly.

FIG. 6 shows the structured data (5.2) as the syntax (104, 204 in FIGS. 1, 2) and the content (FIGS. 1, 2).
105, 205) of the above. (6.1)
Is the separated syntactic information, structured data (5.2)
Information on which of the 15 grammar rules numbered in the syntax specification information (5.1) was used for parsing, and PCDATA which is normal character string data as contents, or character string data including special characters. It also outputs the information indicating whether any CDATA is included. That is, in the syntax (6.1), a variable p is included when PCDATA is included, and a variable c is included when CDATA is included.

For example, "1, 2, 3" from the beginning of the syntax (6.1) is structural information (specifically, based on the definition rules of the first to third lines of the syntax designation information (5.1). Is from <address-book> on the first line of (5.2) to the third line of
(up to <name>) comes first. Next `` 4
“P” indicates that the structure information based on the definition rule on the fourth line of the syntax designation information (5.2) comes next including PCDATA. This is the (5.2) <firstname> John </ firstname
Corresponds to the> part.

As can be seen by comparing the structured data (5.2) and (6.1), with regard to the syntax information, the tag is replaced by the number only with this conversion, so that the compression effect may already be obtained. Recognize. (6.2) and (6.3) are PCDA
TA and CDATA are concatenated in the order of appearance with appropriate delimiters. The XML specification specifies that PCDATA does not include '>' and CDATA does not include ']]>' as character strings, so these are used as delimiters here.

In order to perform the data compression of the structured data (5.2), the separated syntax (6.1) as shown in FIG.
PCDATA (6.2) and CDATA (6.3) may be compressed. In the case of this example, the structured data (5.
The syntax of 2) is expected to repeat data of almost the same format. In (6.1), since the contents of the data are variable and only the syntax is taken out, it is expected that this part will be the same or almost the same repetition of character strings. Therefore, a sufficient compression effect can be obtained even with an elementary compression method such as run length. PCDATA
As for the text data of (6.2) and CDATA (6.3), the same tag portion of the structured data (5.2) is taken out, though there is no significant data bias as described above. It can be expected that words of the same category such as an address if it is an address and a person's name if it is a person's name will be gathered. Therefore, a high compression effect can be obtained by using a compression method effective when the appearance rate of the same character string is locally high, for example, LZ77 (Zip-Lampel) code.

Although the DTD is used as the definition format in this example, XML Schema and Relax are also proposed as new definition formats that can define the data format more finely. In such a case, the local bias of the data can be further increased, so that the compression effect is expected to be further enhanced. Further, since data can be collected for each type, it is possible to further improve the compression efficiency by adopting the encoding according to the data type. In fact
Working Draft (h of XML Schema of April 7, 2000)
(ttp: //www.w3.org/TR/2000/WD-xmlschema-0-20000407/)
For more than 40 simple types such as character strings, logical values, floating point decimals, double precision real numbers, decimal numbers, dates, and periods, are predefined as built-in types. It is possible to significantly increase locality.

In order to decompress the structured data compressed in this way, decompression is performed for the adopted compression method,
The syntax as shown in FIG. 6 (6.1), PCDATA (6.
2) and CDATA (6.3) classification, then PCDATA (6.2) and CDATA in the variable part of the syntax (6.1)
The data of (6.3) may be substituted in order while dividing the delimiter. In this substitution, first, a finite state machine corresponding to each grammatical rule of the syntax designation information (5.1) is constructed. This can be constructed with standard algorithms because each grammar rule is written in regular expressions. Furthermore, the finite state machine thus obtained from each grammatical rule of the DTD must be deterministic.
It is defined by the L specification.

The procedure for obtaining the data classification shown in FIG. 6 will be described below. Structured data such as XML is designed so that there is no syntactic ambiguity, so a parser can be created with the LALR (1) parser generator, such as YACC. That is, the parsing rule is BN
It is possible to generate a program for performing syntax analysis by describing in F format and specifying an action when the rule is applied to each rule. Therefore, if the syntax analysis rule is constructed from the DTD in the following procedure, the analysis program (103 in FIG. 1) for obtaining the data classification in FIG. 6 can be generated.

Step 1: <! ELEMENT tag_name in DTD
For each element definition in the form of body>, lex_id (tag_name)
→ Create a parsing rule of the form trans (body). However, lex_id is the ID for the tag_name returned by the lexical analyzer, and trans (body) is the bod obtained in step 2 and below.
Let y be the conversion result. Also, the corresponding action is an operation that returns the following string: <! ELEMENT tag_name b
A string obtained by concatenating the string obtained as the action of trans (body) after the number given to ody>.

Step 2: The transformation trans is recursively determined as follows.

(1) trans (nil) = nil. This action is an operation that returns an empty string "".

(2) In (x .y), after the head element x, list y
Let be a concatenated list of trans (x .y)
= (trans_each (x) .trans (y)). The action is an operation that returns the character string obtained by concatenating the character strings obtained as a result of the trans_each (x) and trans (y) actions. However, trans_each is defined as follows.

(2-1) trans_each (#PCDATA) = lex_id
(#PCDATA), the action is an operation that returns the character string "p". (2-2) trans_each (#CDATA) = lex_id (#CDATA), the action is an operation that returns the character string "c". (2-3) trans_each (tag_name) = lex_id (tag_nam
e), the action is an operation that returns an empty string "".

(2-4) trans_each (tag_name *) = make_
new_symbol (lex_id (tag_name), _ list), action is the identity map. However, make_new_symbol is a function that connects two symbols given as arguments to create a new symbol. Furthermore, in this case, tn = lex_id (tag_na
me), tn_list = make_new_symbol (ln, _list) add the following new rule. (2-4-1) tn_list → nil, and the action is an operation that returns an empty character string "". (2-4-2) tn_list → tn_list tn, the action is an operation of returning a character string that is a concatenation of two character strings returned as a result of the action on the right side.

(2-5) trans_each (tag_name +) = make_
new_symbol (lex_id (tag_name), _ list1), the action is the identity map. In this case, tn = lex_id (tag_name), t
n_list1 = Add the following new rule for make_new_symbol (ln, _list1). (2-5-1) tn_list1 → tn, and the action is the identity map. (2-5-2) tn_list1 → tn_list tn, the action is an operation that returns a character string that is a concatenation of two character strings returned as a result of the action on the right side.

(2-6) trans_each (tag_name?) = Make_
new_symbol (lex_id (tag_name), _ opt), action is the identity map. In this case, tn = lex_id (tag_name), tn_
Add the following new rule for opt = make_new_symbol (ln, _opt). (2-6-1) tn_opt → nil, and the action is an operation that returns an empty character string "". (2-6-2) tn_opt → tn, and the action is the identity map.

Although the case of mainly compressing / decompressing has been described above, the same applies to the case of encryption / decryption. That is, after classifying as shown in FIG. 6, the syntax (6.
1), PCDATA (6.2), and CDATA (6.3) may be encrypted and transmitted, and the receiving side may decrypt them. If the syntax designation information (5.1) is secret to parties other than the data exchanger, it is considered that the structured information has already been encrypted even if it is in the form of the syntax (6.1). be able to.

[0043]

As described above, according to the present invention, the structured data is separated into the syntax and the content according to the syntax designation information and compressed and encrypted. Therefore, the structured data is not compressed as it is. Also, it becomes possible to effectively reduce the amount of communication data. Further, at the time of data exchange, by adopting the compression / encryption method according to the present invention, in addition to the effect of reducing the amount of communication data and improving the safety, data without overhead due to the purging of structured data is further provided. Exchange can be realized.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration and a processing procedure when compressing or encrypting structured data.

FIG. 2 is a diagram showing a system configuration and a processing procedure when decompressing or decrypting compressed (encrypted) data.

FIG. 3 is a diagram showing a system configuration and a processing procedure when exchanging data using structured data.

FIG. 4 is a diagram showing an example of a system configuration in which overhead is eliminated by incorporating a purging process.

FIG. 5 is a diagram showing an application example for illustrating an outline of a compression / encryption method.

FIG. 6 is a diagram showing a result of separating structured data into syntax and content.

[Explanation of symbols]

101 ... Structured data, 102 ... Syntax designation information, 103
... Lexical / syntactic analysis, 104 ... Partial syntax tree string, 105 ... Content, 106 ... Syntax compression (encryption) module, 10
7 ... Content compression (encryption) module, 108 ... Compressed (encrypted) data, 201 ... Compressed (encrypted) data, 202
... Partial syntax tree expansion (decoding) module, 203 ... Content expansion module, 204 ... Partial syntax tree string, 205 ...
Contents, 206 ... Compositing module, 207 ... Structured data, 301, 302 ... Syntax designation information, 303 ... Data transmission side system, 304 ... Compression (encryption) module, 305 ... Decompression (decryption) module, 306 ... Data reception Side system, 313 ... Lexical / syntactic analysis module, 307 ... Internal data, 308 ... Structured data creation module, 309 ... Structured data, 310 ... Compressed (encrypted) data, 311 ... Network, 312 ... Structured data, 313 ... Lexical / syntactic analysis module, 314 ... Internal data, 401 ... Data transmission side system, 402 ... Data reception side system, 403 ... Compressed (encrypted) data,
404 ... Internal data, 5.1 ... Syntax designation information, 5.2 ...
Structured data.

Claims (7)

[Claims] 1. A data compression method for structured data, wherein individual grammar rules are specified by identification information in syntax specification information including a plurality of grammar rules that are information defining a structure of structured data to be processed. The structure information of the structured data and the content information are separated by expressing the syntax tree representing the structure of the structured data by using the identification information for specifying the grammar rule. Data compression method for structured data. 2. The data compression method for structured data according to claim 1, wherein the content information obtained by separating from the structured data is further classified according to a data type. Data compression method. 3. The data compression method for structured data according to claim 1, wherein the data appearance position of the content data included in the content information is represented by a variable. 4. A method for exchanging structured data, wherein the structured data, which is the target data for exchanging data, is converted into structure information by using the data compression method according to claim 1. Separated into content information, the separated structure information and content information are respectively compressed by a predetermined compression method or encrypted by a predetermined encryption method, and the compressed or encrypted structure information and content information are transmitted. Data exchange method. 5. A data compression method for structured data, wherein syntax specification information including a plurality of grammatical rules, which is information defining a structure of structured data to be processed, is specified by identification information. As possible, a step of retaining in a storage means, obtaining grammatical rules representing the structure of the structured data to be processed, generating identification information of all the obtained grammatical rules, and generating the structural information. Of the grammatical rules, if the content data is attached, the content data is stored in the content information, and the identification information arranged in the structure information is provided with an index indicating that the content data is attached. A data compression method for structured data. 6. A data compression or encryption method for structured data, wherein syntax specification information including a plurality of grammatical rules, which is information defining a structure of structured data to be processed, is identified. A step of storing the information in a storage means so that it can be specified by information, obtaining grammatical rules representing the structure of the structured data to be processed, and generating structural information by arranging identification information of all the obtained grammatical rules. For those grammatical rules accompanied by content data, the content data is stored in the content information, and the identification information arranged in the structure information is provided with an index indicating that the content data is associated, A step of compressing or encrypting the structure information and the content information. Data compression or encryption method. 7. A data decompression or decryption method for decompressing or decrypting structure information and content information compressed or encrypted by the data compression or encryption method according to claim 6, which is compressed or encrypted. The step of decompressing or decoding the structure information and the content information, and extracting the identification information of the grammar rule from the expanded or decrypted structure information, referring to the syntax designation information, and determining the grammar rule corresponding to the extracted identification information. Acquiring step and restoring the structured data of the data structure represented by the acquired grammar rule, and if the grammatical rule is accompanied by the content data, extract the corresponding content data from the content information and correspond the structured data. Setting the position to restore the structured data. Data decompression or decoding method for structured data, wherein.