JP2009128945A - Data processing apparatus, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that compression of binary XML including the data size of a schemer is not good in efficiency since the data size of the schemer is still large during binarization of an XML document. <P>SOLUTION: A data processing apparatus which binarizes at least part of the XML document described in text format using the schemer where a structure that the XML document possibly has is described compresses the amount of information of the schemer by deleting or altering elements not relating to the XML document. The schemer where the amount of information is compressed is used to binarize the XML document. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for binarizing structured data described in a text format.

  The information described in the XML document is written in a text format and thus has a high readable specification. On the other hand, the data size tends to be larger than that described in binary data. In order to solve this problem, an approach has been proposed in which an XML document is converted into another format, in particular, binary data to reduce the data size.

  As a compression technique, in addition to general-purpose lossless compression that has been used in the past, not only XML, but also compression techniques by converting redundant XML descriptions into other notations (Patent Documents 4, 5, 6, etc.) Proposed. Also, a technique for compressing an XML document at a higher compression rate after referring to data describing a grammar rule of an XML document called a schema and excluding more information than an XML document to be compressed (Patent Documents 1, 2, 7 etc.) have also been proposed. In such a compression technique, the compression result is generally output as binary data in order to increase the compression rate.

  The schema used in such a compression technique is generally published in advance and widely shared, but a schema (Patent Document 3) that dynamically generates from an XML document is used. May not be shared.

JP 2002-244894 A JP 2004-032774 A JP 2006-0923329 A JP 2004-302868 A JP 2005-018672 A JP 2005-310064 A JP 2001-217720 A

  However, in the conventional XML document binarization technology using a schema, compression of the XML document is considered, but compression of the schema itself is not considered. If the schema is distributed to devices that handle binary XML in advance, there is not much need to consider schema compression. However, when the schema is distributed together with the XML document itself as in DTD, or when a schema that is not generally distributed such as Patent Document 3 is used, include the data size of the schema itself. Efficient operation is not possible without considering compression. In general, the schema has a very large data size, and with conventional technology, compression of binary XML including the schema data size has not been efficient. In order to improve efficiency, the schema itself may be binarized in the same way as an XML document, but the mechanism for generating binary XML becomes complicated, such as the need for a schema schema.

  Therefore, the present invention has an object of reducing the amount of schema information when binarizing an XML document and reducing the total data size including the schema.

  According to one aspect of the present invention, there is provided a data processing apparatus that binarizes at least a part of an XML document described in a text format using a schema that describes a structure that an XML document can take. The XML document is binarized using a schema compression unit that compresses the information amount of the schema by deleting or changing an element not related to the XML document, and a schema whose information amount is compressed by the schema compression unit. There is provided a data processing apparatus characterized by comprising binarization means.

  According to the present invention, when an XML document is binarized, the amount of schema information can be reduced, and the total data size including the schema can be reduced.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following embodiment, It shows only the specific example advantageous for implementation of this invention. In addition, not all combinations of features described in the following embodiments are indispensable as means for solving the problems of the present invention.

<Device configuration>
FIG. 1A is a block diagram conceptually showing a functional configuration of a computer device as two data processing devices that transmit and receive an XML document in the present embodiment.

  The first computer device 10 on the data transmission side includes a binary XML generation processor 11 that binarizes an input XML document (text XML) and a schema binarization unit 12 that binarizes the input schema. Here, the schema is data describing a grammar rule of an XML document, that is, a structure that the XML document can take. The schema binarization unit 12 includes one or more schema compression units 13 having different compression algorithms for compressing the amount of schema information. Thereby, the schema binarization unit 12 can change the compression algorithm according to the binarization algorithm adopted by the schema binarization unit 12. The compression schema is sent to the binary XML generation processor 11 as indicated by arrow 14. However, in another embodiment, as indicated by an arrow 15, a configuration may be adopted in which the data is transmitted to the second computer device 20 on the data receiving side in advance independently of the binary XML. On the other hand, the second computer device 20 on the data receiving side includes a binary XML parser 21 as an analysis unit that analyzes the XML document (binary XML) received from the first computer device 10 and generates text XML.

  The configuration shown in FIG. 1A is for convenience of explanation. Normally, each computer device includes a binary XML generation processor 11 and a binary XML parser 21, and both computer devices are provided on the data transmission side. Needless to say, it can also be a receiver.

  FIG. 1B is a block diagram showing the hardware configuration of the first and second computer apparatuses 10 and 20. Each computer device may be realized by a single computer, or may be realized as a computer system including a plurality of computers and having functions distributed to each computer. When configured with a plurality of computers, the computers are connected to each other via a local area network (LAN) or the like.

  In FIG. 1, reference numeral 101 denotes a central processing unit (CPU) that controls the entire computer apparatus. Reference numeral 102 denotes a read only memory (ROM) that stores programs and data that do not need to be changed. A random access memory (RAM) 103 temporarily stores programs and data supplied from the outside. A storage unit 104 includes a hard disk, a memory card, and the like that are fixedly installed in the computer apparatus. The storage unit 104 may be configured by an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be attached to and detached from the computer apparatus, a magnetic or optical card, an IC card, a memory card, and the like. The storage unit 104 stores a program for causing the CPU 101 to function as the binary XML generation processor 11, the schema binarization unit 12, and the binary XML parser 21. An operation unit 105 includes a pointing device and a keyboard. A display unit 106 displays data held by the computer apparatus and supplied data. Reference numeral 107 denotes a network interface (i / f) for connecting to a network line such as the LAN 203. A system bus 108 connects the units 101 to 107 so that they can communicate with each other.

  Hereinafter, a basic XML binarization procedure in this embodiment will be described, and then a schema compression method will be specifically described.

<Basic XML binarization procedure>
Hereinafter, a procedure in which the above-described computer device binarizes an XML document using a compressed schema and a procedure in which an analysis is performed using a schema in which the XML document binarized by the procedure is compressed will be described. An example of the schema compression procedure will be described later, and here, one or a plurality of compression algorithms described later are used in combination. In this embodiment, a compressed schema is generated by compressing the schema according to the binarization process shown in FIG. 2, and binary XML is analyzed using the compressed schema according to the analysis process shown in FIG. The schema used in this embodiment is stored in advance in the storage unit 104, ROM 102, or RAM 103 of the computer device that performs binarization.

  First, the processing procedure of the computer device that performs binarization will be described with reference to FIG.

  When instructed to binarize the XML document, the computer apparatus according to the present embodiment checks whether there is a schema corresponding to the XML document to be binarized (S202). Each schema describes a namespace for specifying a target XML document, and a namespace for specifying the schema is also described on the XML document side. Therefore, whether there is a schema corresponding to the XML document is confirmed by comparing the namespace used in the XML document with the technology related to the namespace of each schema stored in the computer device. I can judge. If a pair of an XML document and a schema whose namespace declaration matches is not found, it is determined that there is no corresponding schema. If there is no schema, it is checked whether the binarization without using the schema is possible (S203). As a confirmation method, for example, there is a method of actually applying Fast Infoset or the algorithm disclosed in Patent Document 4. For information on Fast Infoset, see http://java.sun.com/developper/technicalArticles/xml/fastinfoset/. Alternatively, there is a method of checking whether or not a description to be binarized (element contents composed only of numbers) is included from an XML document to be binarized. If binarization is possible without using the schema, binary XML is generated using only the XML document (S204).

  On the other hand, if there is a schema in S202, the schema is read (S205), and it is determined whether there is a schema compression effect (S206). Judgment methods include the method of actually executing the schema compression algorithm, which will be described later, and the description of the schema required for each schema compression algorithm ("interleave" element is required, elements that mean repetition are required, etc.) There is a method to judge by. If there is no compression effect, binarization using a normal schema is performed (S207).

  If the schema information amount of the XML document can be compressed, a compression schema is generated by compressing the schema information amount (S208) and output to the storage unit 104 or the RAM 103 in the computer apparatus. Further, the XML document is compressed using the stored compression schema (S209). Note that the binarization algorithm in S209 is exactly the same as the binarization algorithm in S207.

  When the computer apparatus according to the present embodiment performs binarization a plurality of times, the following process may be executed before performing the above process. First, it is confirmed whether a compression schema has already been generated and stored in the storage unit 104 or the RAM 103 (S210). If a compression schema has already been generated and stored, it is determined whether or not the compression schema is appropriate for the current binarization algorithm (S211). If the compression schema is appropriate, S202-S208, which is a series of processes for generating the compression schema, is omitted, and the XML document is immediately binarized (S209).

  Thereafter, when the binary XML is handled by another computer device, the binary XML itself is transferred. At this time, when there is no schema or compression schema at the transfer destination, it is necessary to transfer it together. The schema or the compressed schema can be transferred at the same time as the binary XML, or transferred when the computer that analyzes the binary XML actually needs the schema information (not shown). The computer apparatus that has received the compression schema stores the received compression schema in the storage unit 104 or the RAM 103.

  Next, the binary XML analysis processing procedure in the computer apparatus will be described with reference to FIG.

  When instructed to analyze binary XML, the computer device in this embodiment checks whether the binary XML compression schema to be analyzed is stored in the storage unit 104 or the RAM 103 (S302). Specifically, as in the case of binarization, the namespace declaration on the binary XML side is compared with the namespace declaration on the compression schema side, and it can be determined whether or not they match. If there is a compression schema that can be used, the compression schema and binary XML are combined, and the information of the XML document before binarization is analyzed and acquired (S303-S304).

  On the other hand, if there is no compression schema but a normal schema can be used (S305, YES), the schema and binary XML are combined, and the information of the XML document before binarization is analyzed and acquired (S306-S307). You can also determine whether it can be used by comparing the binary XML and the namespace declaration of the schema.

  If there is no schema, the binary XML alone is analyzed (S308-S309).

<Schema compression method>
The schema compression algorithm according to this embodiment will be described below. The language for describing the schema is RELAX NG. An example of the schema to be compressed is shown in FIG. 12, and an example of the XML document to be binarized is shown in FIG. The same shall apply hereinafter). The schema example shown in FIG. 12 basically conforms to the specification of SVG (Scalable Vector Graphics) 1.1, but is simplified for explanation. Actually, SVG has more child elements / attributes, and child elements have more child elements / attributes, but they have been omitted for clarity.

  Various other schema compression algorithms can be used regardless of lossless / lossy compression. In the example shown below, the compression schema is kept in the text format, but it is possible to combine the conversion technology to the binary format.

  The schema binarization algorithm in this embodiment is, for example, as follows.

(1) List the choices of child elements for each element defined in the schema, and assign IDs in the order of declaration.
(2) Calculate the minimum number of bits required to store the ID.
(3) When binarizing an XML document, an ID to be replaced is selected from a combination of a parent element and a child element described in the XML document and replaced with the ID. The ID is described by securing an area with the minimum number of bits.

  From the schema of FIG. 12, it can be seen that the SVG element has 12 options such as “rect” and “path” as child elements. As a result, when IDs are actually generated, 12 types of IDs are allocated. These 12 options are counted by the number described in the schema regardless of which child element is actually used. The actually allocated result is shown in FIG. Since there are 12 types of options, a 4-bit area is required to actually describe these options as data. In the XML document shown in FIG. 13, the “svg” element has a “rect” element and an “image” element as child elements. When the XML document shown in FIG. 13 is binarized, it is necessary to assign values “1” and “6” to the child elements. Therefore, values are set after securing a 4-bit area.

  The schema compression algorithm for the above binarization will be described below.

  In this compression algorithm, only the elements used in the XML document to be binarized are left in the schema, and other information is deleted as long as it does not violate the schema syntax. That is, the schema is converted according to the following pattern.

(1) For each element, list possible child element types.
Examples of the listing method include the following methods.
A method of receiving a list of child elements used from an application or user.
A method for listing the parent-child relationships of elements from one or more XML documents to be binarized and generating a list of actually used child elements.
(2) The “element” child element that is not used as a child element in the XML document to be binarized is deleted from the “element” element in the schema.

  When the child elements of the “svg” element are only the “rect” element and the “Image” element, among the 05th to 16th lines in FIG. 12, the only remaining lines are the 05th and 10th lines. . FIG. 15 shows a compression schema generated by deleting unnecessary lines. It can be seen that the original schema consisted of 20 lines, but the compressed schema was compressed to 10 lines.

  Actually, when the binarization algorithm in this embodiment is applied to the compression schema, the result is as follows.

  From the 05th and 06th lines in FIG. 15, it can be seen that the “svg” element has two child elements, “rect” element and “image” element, as options. As a result, two types of IDs are allocated. An example of the ID allocation result is shown in FIG. Since there are only two options, only one bit area is required to actually describe this option as data. As described above, in the XML document shown in FIG. 13, the “svg” element has only “rect” and “image” elements as child elements. When the XML document shown in FIG. 13 is binarized, the values “1” and “2” need to be allocated to the child elements, respectively, so that a value is set after securing only a 1-bit area.

  By using the schema compression algorithm as described above, binary XML can be generated and used using a compression schema having a size smaller than the original schema. In addition, since the same binarization algorithm can be used as before, it is not necessary to add a processing system for the binarization algorithm.

<Modification 1>
In the following example, the schema to be compressed is the schema shown in FIG. 4, and the XML document to be binarized is the XML document shown in FIG.

  The binarization algorithm in this example is as follows.

(1) Assign IDs in the order of declaration to element names, attribute names, and constants defined in the schema.
(2) When the XML document is binarized, the element name, attribute name, and constant described in the XML document are replaced with the ID.

In order to assign IDs to element names, attribute names, and constants from the schema shown in FIG. FIG. 6 shows the result of obtaining element names, attribute names, and constants from the top of the schema in FIG. 4 and assigning IDs. The obtained string is
・ "Top" on line 02
・ "Int_value" on line 04
・ "Ubyte_value" on line 07
・ "Int_list" on line 10
・ "Value_foo" on line 19
・ "Foo" on line 20
ID is assigned in order from 1.

On the other hand, FIG. 7 shows the result of binarizing the XML document of FIG. 5 using the correspondence table of FIG. As shown in FIG.
・ "Top" on line 01 is 1
・ "Int_value" on line 02 is 2
・ "Ubyte_value" on line 03 is 3
・ "Int_list" on line 04 is 4
-"Value_foo" on line 05 is 5
・ "Foo" on line 05 is 6
・ "Top" on line 06 is 1
Respectively. In FIG. 7, the replacement result is described with underlined numbers. Actually, the underlined numbers are managed in a format that is distinguished from normal characters.

  When acquiring the information of the original XML document from binary XML, similarly generate an ID correspondence table from the schema, and replace the underlined numbers in binary XML shown in FIG. 7 with the character strings in the ID correspondence table. .

  The schema compression algorithm for the above binarization will be described below.

  In this compression algorithm, only the elements and attributes necessary for describing the character string related to the element name, attribute name, and element content to be binarized are left in the compression schema, and other information is within the scope of the schema of the schema. Delete with. That is, the schema is converted according to the following pattern.

-Leave the start element.
・ Delete interleave, oneOrMore, and list because they do not affect the ID allocation order.
-Leave the element element and name attribute pair to get the string.
-Since the data element has no character string to be acquired, delete it.
• Leave the value element to have a string to allocate the ID.
• Replace the start tag and end tag pairs that no longer have element content with empty element tags.

  As a result, the lines to be left in FIG. 4 are the 01 line, 02 line, 04 line, 06 line, 07 line, 09 line, 10 line, 18 line, 19 line, 20 line, Only the 21st, 23rd and 24th lines. Since the data element on the 05th line is deleted, the set of start / end tags of the element element on the 04th and 06th lines is replaced with one empty element tag. The same applies to the 07/09 line group and the 10/18 line group. As a result, the schema with 24 lines is converted into a 10-line compressed schema. The generated compression schema is shown in FIG.

  Applying the binarization algorithm in this example to the compression schema is as follows. Six character strings “top”, “int_value”, “ubyte_value”, “int_list”, “value_foo”, and “foo” are acquired in order from the second line in FIG. 8. If IDs are assigned in this order, an ID correspondence table identical to the ID correspondence table shown in FIG. 6 is generated. Since the same ID correspondence table is used, the remaining processing of the binarization algorithm has exactly the same result, and the binary XML shown in FIG. 7 is generated. When parsing binary XML, the same ID correspondence table is generated and the same result is obtained.

  For the binarization algorithm in this example, using the schema compression algorithm shown above, it is possible to generate and use binary XML of the same size using a compression schema of a size smaller than the original schema. become. In addition, since a common binarization algorithm can be used, it is not necessary to add a new processing system for the binarization algorithm.

<Modification 2>
In the following example, the following binary algorithm is adopted.

(1) If there is a description (loop statement) that means a loop in the schema, describe the number of loop iterations at the beginning of the corresponding location in binary XML.
(2) A character string with a specified data type is converted to the specified data type.

  The looping description in the schema of FIG. 4 is a oneOrMore element starting from the 11th line, and it is defined that “a sequence in which int type element contents are continued three times” is repeated. The corresponding part in FIG. 5 is the 05th line, and “an array of int type element contents three times in succession” is repeated twice. When this is binarized, the value “2” is placed at the beginning, followed by a total of six values stored as int type values. FIG. 9 shows a part of the result of binarization. The underlined parts are actually treated as values.

  When acquiring the information of the original XML document from the binary XML, the information in which the corresponding location “a sequence in which the int type element contents are repeated three times” is repeated is acquired from the schema of FIG. From the value of “2”, it is determined that the number of repetitions is 2. As a result, it is determined that a total of six ints are arranged, and six int areas are read from the subsequent areas, and each is analyzed as an int value, whereby the original information can be acquired correctly.

  The schema compression algorithm for the above binarization will be described below.

In this compression algorithm, the schema description is changed so that only the number of loops in binarization changes. That is, the schema is converted according to the following pattern.
-Find the elements (oneOrMore, zeroOrMore) that mean the loop.
-If the pattern in the loop is repeated ("int type is 3 times" or "byte type and String type pair is 2 times"), the pattern is 1 time ("int type is 1 time") or "byte" Type / String type pair once ”etc.).

  As a result, the 11th to 17th lines in the schema of FIG. 4 are replaced with the 3rd line of the 11th to 13th lines of FIG. 10, resulting in a compressed schema that repeats the int type element contents one or more times. If only the above rule is applied, the list element remains as the parent element of the data element. However, the meaning of the list element having only one child element does not change even if the list element is deleted.

  Applying the binarization algorithm in this example using a compression scheme results in the following.

  From the 11th to 13th lines in FIG. 10, it can be seen that one int type element content of the int_list element is repeated one or more times. On the other hand, since the XML document shown in FIG. 5 to be binarized lists six numerical values as element contents, it is converted into data of the number of repetitions “6” and six int type values. The conversion result is shown in FIG. Unlike FIG. 9, the number of repetitions described at the beginning of the 05th line changes from “2” to “6”, but the size of the binary XML itself is the same. When parsing binarized XML, it is possible to extract 6 int areas by using the compression schema, so the same result as before the schema change can be obtained.

  For the binarization algorithm in this example, using the schema compression algorithm shown above, it is possible to generate and use binary XML of the same size using a compression schema of a size smaller than the original schema. become. In addition, since a common binarization algorithm can be used, it is not necessary to add a new processing system for the binarization algorithm.

<Other embodiments>
As mentioned above, although embodiment of this invention was explained in full detail, this invention may be applied to the system comprised from several apparatuses, and may be applied to the apparatus which consists of one apparatus.

  In the present invention, a program for realizing each function of the above-described embodiments is supplied directly or remotely to a system or apparatus, and a computer included in the system or apparatus reads and executes the supplied program code. Can also be achieved.

  Therefore, since the functions and processes of the present invention are implemented by a computer, the program code executable by the computer also implements the present invention. That is, the computer program itself for realizing the functions and processes is also one aspect of the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  Examples of the computer-readable recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, an MO, a CD-ROM, a CD-R, and a CD-RW. Further, examples of the recording medium include a magnetic tape, a nonvolatile memory card, a ROM, and a DVD (DVD-ROM, DVD-R).

  The program may be downloaded from a homepage on the Internet using a browser on a client computer. That is, the computer program itself of the present invention or a compressed file including an automatic installation function may be downloaded from a home page to a recording medium such as a hard disk. Further, it is also possible to divide the program code constituting the program of the present invention into a plurality of files and download each file from a different home page. In other words, a WWW server that allows a plurality of users to download a program file for realizing the functions and processing of the present invention on a computer may be a constituent requirement of the present invention.

  Further, the program of the present invention may be encrypted and stored in a computer-readable storage medium such as a computer-readable CD-ROM and distributed to users. In this case, only the user who cleared the predetermined condition is allowed to download the key information to be decrypted from the homepage via the Internet, decrypt the program encrypted with the key information, execute it, and install the program on the computer May be.

  Further, the functions of the above-described embodiments may be realized by the computer executing the read program. Note that an OS or the like running on the computer may perform part or all of the actual processing based on the instructions of the program. Of course, also in this case, the functions of the above-described embodiments can be realized.

  Furthermore, the program read from the recording medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Based on the instructions of the program, a CPU or the like provided in the function expansion board or function expansion unit may perform part or all of the actual processing. In this way, the functions of the above-described embodiments may be realized.

It is a block diagram which shows notionally the function structure of the two computer apparatuses which transmit / receive an XML document in embodiment. It is a figure which shows the structure of the computer apparatus which concerns on embodiment. It is a flowchart which shows the binary XML production | generation process in embodiment. It is a flowchart which shows the binary XML analysis process in embodiment. It is a figure which shows the example of the schema before compression in embodiment. It is a figure which shows the example of the XML document of the binarization object in embodiment. FIG. 5 is a diagram illustrating a correspondence between an element name / attribute name / constant acquired from the schema of FIG. 4 and an ID. 7 is a diagram illustrating an example of binary XML obtained by binarizing the XML document of FIG. 5 using the ID allocation result of FIG. 6. FIG. 5 is a diagram illustrating an example of a compression schema of the schema in FIG. 4. It is a figure which shows a part of result binarized using the schema of FIG. FIG. 5 is a diagram illustrating an example of a compression schema of the schema in FIG. 4. It is a figure which shows the example of binary XML obtained from the XML document of FIG. It is a figure which shows the example of the schema before compression in embodiment. It is a figure which shows the example of the XML document of the binarization object in embodiment. It is a figure which shows the example of the conversion table of the choice of the child element and ID in embodiment. It is a figure which shows the example of the compression schema of the schema of FIG. FIG. 16 is a diagram showing an example of correspondence between child element options and IDs for the compression schema of FIG. 15.

Explanation of symbols

101 CPU
102 ROM
103 RAM
104 storage unit 105 operation unit 106 display unit 107 network i / f
108 System bus

Claims (7)

A data processing apparatus that binarizes at least a part of an XML document described in a text format using a schema that describes a structure that an XML document can take.
Schema compression means for compressing the amount of information of the schema by deleting or changing elements not related to the XML document from the schema;
Binarization means for binarizing the XML document using a schema whose amount of information is compressed by the schema compression means;
A data processing apparatus comprising:   The data processing apparatus according to claim 1, wherein the schema compression unit changes a compression algorithm according to a binarization algorithm adopted by the binarization unit.   The data processing apparatus according to claim 1, wherein a language describing the schema is RELAX NG.   The schema compression unit compresses the information amount of the schema by deleting a description related to a child element and an attribute that are not used in the XML document among elements included in the schema. Data processing equipment.   The data processing apparatus according to claim 1, wherein the schema compression unit replaces the pattern with one description of the pattern when a predetermined pattern repeatedly appears in a loop statement included in the schema. A data processing method for binarizing at least a part of an XML document described in a text format using a schema describing a structure that an XML document can take.
A schema compression step, wherein the schema compression means compresses the amount of information of the schema by deleting or changing an element not related to the XML document from the schema;
A binarizing unit that binarizes the XML document using the schema in which the information amount is compressed in the schema compressing step;
A data processing method characterized by comprising:   6. A computer-executable program that causes a computer that executes the program to function as the data processing apparatus according to claim 1.

Images