JP4394964B2 - Data compression apparatus, data restoration apparatus, template generation apparatus, and data compression system - Google Patents

Description

  The present invention relates to an electronic data compression apparatus, decompression apparatus, template generation apparatus, and data compression system.

  In recent years, with the spread of the World Wide Web (WWW), data exchange using structured documents such as HTML (Hyper Text Markup Language) and XML (Extensible Markup Language) is increasing. In particular, XML is attracting attention as a next-generation language that supplements HTML, and is expected to be most popular in the field of information exchange on the Internet.

  XML is a language with a data expression format representing a hierarchical structure of elements, and a document (XML document) using XML is described as shown in FIG. 32, for example. FIG. 32 is a diagram showing the XML document 10. As shown in FIG. 32, XML is roughly divided into markup and text information. In the XML document 10 shown in FIG. 32, the markup includes an element start symbol (start tag) Ma, an element end symbol (end tag) Mb, and an empty element symbol (empty element tag) Mc. In FIG. 32, <book>, <title>, <authors>, <author>, <contents>, and <chapter> represent the element start symbol Ma. Also, </ book>, </ title>, </ authors>, </ author>, </ contents>, and </ chapter> represent the element end symbol Mb, and <misc /> represents the empty element symbol Mc. ing. The area from the element start symbol Ma to the corresponding element end symbol Mb, or the empty element symbol Mc represents an element (information unit that is the basis of XML).

  In addition to other element symbols, text information can be described between the element start symbol Ma and the element end symbol Mb. For example, in the XML document 10 shown in FIG. 32, the character string “XML basics” is included in the element <title>, and the character string “Taro Yamada” is included in the first element <author> that appears in the element <authors>. Are defined as text information.

Parent-child relationships and sibling relationships are defined between elements and text information. In the case of the XML document 10 shown in FIG. 32, the element start symbol Ma starts with <book>, and the element end symbol Mb starts with </ book>.
Includes an element (element <book>) that ends with <element> and an element start symbol Ma that starts with <title> and an element end symbol Mb that ends with </ title> (element <title>). At this time, the element <book> is said to be a parent element of the element <title>, and the element <title> is said to be a child element of the element <book>. This is the parent-child relationship of elements.

The element <title> and the element <authors> have the same parent element <book> and are continuous. At this time, element <title> and element <authors> are said to be siblings, and element <title> is element
The previous sibling of <authors>, element <authors> is said to be the next sibling of element <title>. This is an element sibling.

  In general, when XML is communicated between computers or stored in a hard disk device or flash memory, XML is expressed in a text format as an XML document 10 shown in FIG. On the other hand, when it is used for searching and correction in the computer, it is analyzed and converted into a data structure suitable for the computer.

  FIG. 33 is a diagram showing a data structure 11 obtained by analyzing the XML document 10 shown in FIG. 32 and converting it into a format suitable for internal use of the computer. In FIG. 33, each element and text information are described as vertices 301 to 317 having types and values. The type is described on the left side of each of the vertices 301 to 317. “E” represents an element, and “T” represents text information. For example, at the vertex 301, the type 301a is “E”. The value is described on the right side of the vertex. For example, in the vertex 301, the value 301b is “book”. When the vertex represents an element, the element name (element name) is described in the value. When the vertex represents text information, a character string is described. For example, the vertex 302 represents the element name <title>, and the vertex 306 represents the text information “XML basics”.

  Each vertex 301 to 317 represents four references of a parent reference, a child reference, a next sibling reference, and a previous sibling reference in order to express the parent-child relationship and sibling relationship of the original (before conversion) XML document 10. Has reference information. In the case of the XML document 10 described above, the element <title> is a child element of the element <book>, and the element <book> is a parent element of the element <title>. Therefore, in the data structure 11 illustrated in FIG. The vertices 301 and 302 each have a child reference P1 from <book> to <title> and a parent reference P2 from <title> to <book>, which are represented by arrows. The element <book> also has an element <authors> as a child element next to <title>. In this case, for the vertices 302 and 303, the next sibling reference P3 from the element <title> to the element <authors> and the previous sibling reference P4 from the element <authors> to the element <title> are held. In the case of the data structure 11, elements that are in a sibling relationship do not directly have a parent reference other than the first child element (for example, element <title>).

  In the data structure, reference information between vertices and element names and text information can be managed separately. For example, each of them is expressed as shown in FIGS. 34 (a) and 34 (b). Can do. Here, FIG. 34 (a) is a diagram showing cross-reference relationship data 400 having reference information between vertices, and FIG. 34 (b) has a type and value set in either element or text information. It is a figure which shows the table 450 which shows the collection (it is also called vertex group) of a some vertex.

  However, since the capacity of a storage device such as a memory is finite, when storing a data structure, it is required to efficiently compress and store the data structure. In this regard, Non-Patent Document 1 discloses a method for compressing element names and text information as shown in FIG. In Non-Patent Document 1, the element name and text information held by each vertex are separately accumulated as a dictionary, and each vertex has a dictionary index so that a plurality of the same character strings are not accumulated. Is disclosed.

On the other hand, Non-Patent Document 2 discloses a method for compressing an XML document by reusing a partial structure in the XML document. In this method, the original XML document is separated into the structure, element name information, and text information, and each is compressed by a general compression algorithm such as LZ77 (for details on LZ77, refer to Jacob Ziv). Abraham Lempel: A Universal Algorithm for Sequential Data Compression, see IEEE Transactions on Information Theory 23 (3): 337-343 (1977)).
Here, the compression method disclosed in Non-Patent Document 2 will be described. In this compression method, first, an element start symbol and an empty element symbol are replaced with short element names such as “# 1” and “# 2”, respectively, and an element end symbol is replaced with “/”. The text information is replaced with “C”.
When the above compression method is applied to the separated XML document 10, the data structure 12, the element name information 13, and the text information 14 after the separation are expressed as shown in FIGS. 35, 36, and 37, respectively.

  In the compression method described in Non-Patent Document 2, each is independently compressed using a compression algorithm typified by LZ77 or the like. Here, an outline of the compression algorithm will be described. A compression algorithm such as LZ77 finds a partial pattern included in the original input information, and performs compression by repeatedly reusing it as a template. For example, the compression of the data structure 12 shown in FIG. 35 will be described. Assume that templates X, Y, Z, W, and V are used as templates, and the assignment of each template is X = “# 1 # 2 C / # 3. "= Y #" # 4C / ", Z =" / # 5 ", W =" # 6 C / ", V =" / # 7 // ", the data structure 12 shown in FIG. Can be expressed as “X Y Y Y Z W W V”. This uses Y and W a plurality of times as templates representing a part of the document structure. In this way, if the template can be used repeatedly and the original document can be expressed with a small number of templates, the amount of information representing the original XML document can be reduced, so that compression is possible.

Mathias Neumuller and John N. Wilson: “Compact In-Memory Representation of XML” Internal Report of University of strathclyde Hartmut Liefke and Dan Suciu. : “XMill: An Efficient Compressor for XML Data”, In proceedings of ACM SIGMOD International Conference on Management of Data, 2000

  However, in the conventional technology, when the templates that can be used are limited due to the limitation of the amount of memory or the like, sufficient compression cannot be performed. Therefore, it is desirable to perform recompression on a data structure compressed using a template so that the data compressed efficiently by the recompression can be restored.

  Accordingly, the present invention has been made to solve the above-described problems. The data structure of an XML document is efficiently compressed using a template, the efficiently compressed data is restored using a template, and efficient compression and It is an object of the present invention to provide a data compression apparatus having a configuration that enables restoration, a data restoration apparatus that restores the compressed data structure, a template generation apparatus that generates a template used for the compression, and a data compression system.

In order to solve the above problems, the present invention provides input data having a plurality of vertices each having a type and a value and reference information between the vertices, cross-reference relationship data having reference information between vertices, a type and a value. separated into a vertex group consisting of a plurality of vertices having a separating means for outputting the data of the separated vertices, template storage for storing a template representing a specific partial pattern of the reference information between vertices Means for detecting a matching point corresponding to the template stored in the template storage unit from the cross-reference relationship data separated by the separation unit, and the cross-reference relationship data separated by the separation unit Of these, when replacing the matching part detected by the template matching part detection means with a template, Provided instructions with vertices indicate where the matching portion has a portion other than the matching portion of the cross reference data, the data comprising an instruction with the vertex, and a template replacing means to enable re-replacement by a template, The data compression apparatus includes output means for outputting cross-reference relationship data repeatedly replaced by the template replacement means to a storage device in a state where the reference information between the vertices can be restored by the template .
In this data compression apparatus, the replacement location can be expressed with the same configuration as the apex, and the template can be recursively applied, so that high compression efficiency can be realized with a small number of templates.

In addition, when the template matching location detection means detects a matching location corresponding to the template stored in the template storage means when the input data has a rooting tree structure , the vertex at the deepest position of the rooting tree structure is detected. It is preferable to detect only a coincident portion including a certain leaf vertex.
As a result, when the compressed cross-reference data is restored and some vertices of the restored cross-reference data are referenced, only the templates necessary for referencing the vertex are expanded. It can be done at high speed.

The present invention includes a template storage means for storing a template representing a specific partial pattern of the reference information between the vertices of the multiple has reference information between the vertices and compressed mutual substituted by template Input the reference relationship data, expand the compressed cross-reference relationship data using the template, and when the expanded cross-reference relationship data includes the indicated vertex , the expanded cross-reference relationship data is the template. Repeat the re-expansion using, and input the expansion means for restoring the cross-reference relation data before compression from the compressed cross-reference relation data and the data of the vertex group consisting of a plurality of vertices. Combining the data with the uncompressed cross-reference relationship data restored by the decompression means, and having the synthesis means for outputting the synthesized data Providing data restoration device.
Since this data restoration device repeatedly re-expands the input cross-reference relationship data using the template, the template is recursively applied, and the cross-reference relationship data compressed with a high compression ratio using a small number of templates. Can be expanded and restored.

In addition, when a vertex to be expanded using a template is specified in the cross-reference relationship data, the expansion means may perform reexpansion using a template necessary for referring to the vertex to be referred to.
As a result, only a template necessary for referring to the vertex to be referred to can be developed, and data can be referred to at high speed from the compressed cross-reference relationship data.

And this invention inputs the data of the netsuke tree structure which has a plurality of vertices and reference information between the vertices, acquires the input data, and each of the input data acquired by the means Template candidate detection means for detecting the reference information of the rooted tree structure included in the reference information between vertices as a template candidate, a template candidate acquired by the means, and a template candidate storage for storing the appearance frequency of the template candidate in the input data The template candidate is selected based on the appearance frequency from the template, the template replacement means for replacing the input data according to the template candidate stored in the means, and the template candidate stored in the template candidate storage means, A template selection means for outputting the selected template candidates; Providing a template generator having.
The template generation device detects reference information of the rooted tree structure included in the input data as a template candidate, stores it in the template candidate storage means, and replaces the input data using the template candidate. Further, a template candidate is selected from the accumulated template candidates based on the appearance frequency, and is output as a template.

Further, the template selecting means can select only the template candidates used by the template replacing means for replacing the input data when selecting the template candidates.
In this way, it is possible to output reference information of a necessary portion of given input data as a template candidate.
Further, the template candidate detection means can detect, as a template candidate, reference information of a portion having a minimum tree structure including all vertices having the same depth as the shallowest leaf vertex from the root vertex in the input data.
Furthermore, reference information of a portion having a rooted tree structure at a location not detected by the template candidate detection unit in the input data may be input to the template candidate detection unit.
When the template generation apparatus is configured in this way, a rooted tree structure in which only root vertices have connection information can be generated from input data as a template, so that a template effective for recursive compression can be generated. .

Then, the template candidate detecting means starts from the leaf vertex first encountered in the depth-first search from the root vertex of the input data, and sets the reference information of the rooted tree structure portion having the starting vertex and its parent vertex as a template. It is also possible to repeat the extension candidate detection using the reference information of the rooted tree structure part obtained by adding one vertex to the reference information as a candidate and a new template candidate.
By configuring the template generation device in this way, it is possible to verify the match with the template by performing the structural analysis of the input data without sequentially confirming the match between the template and the input data. Can be generated.
Further, the template candidate detection means may limit the detection of a new template candidate by the number of vertices, and may limit the detection of a new template candidate by the height of the reference information of the rooted tree structure portion.
By doing so, it is possible to limit the size of the generated template and suppress the generation of a large template with a low appearance frequency.

Furthermore, when the template candidate detection means performs extension candidate detection, it does not extend to a new vertex whose depth difference from the starting vertex is larger than a predetermined value, but detects an extension candidate starting from the new vertex. Is preferably repeated.
By configuring the template generation device in this way, not only can the size of the generated template be restricted, but also the vertices that are descendants of the vertices whose depth difference is greater than a predetermined value are separated from the vertices as the starting point. Can be treated as leaf vertices. Since a rooted tree structure in which only root vertices have connection information can be generated from input data as a template, a template effective for recursive compression can be dynamically generated.

The present invention is a data compression system for compressing first input data having a plurality of vertices each having a type and a value, and reference information between the vertices, and has the same structure as the first input data enter a cross reference data with reference information between extracted vertex from the second input data having a lutein plates generator to generate a template representing a specific partial pattern of the reference information between the vertices the any one of the data compression apparatus comprising a template storing means for storing a template generated by template generator, with recoverable state by references to the template between vertices, the stored template to the template storage means the cross reference data are repeatedly replaced by the data compression apparatus using output as the first output data, the Providing data compression system and an output means for outputting the data of the vertex group consisting of a plurality of vertices each having a separate type and value from the input data as the second output data.
By configuring the data compression system in this way, it is possible to compress the input data using the generated template while dynamically generating a template suitable for compression of the given input data. High data compression can be realized.

  According to the present invention, the data structure of an XML document can be efficiently recursively compressed using a small number of templates, and the efficiently compressed data structure can be restored.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Embodiment of data compression apparatus (first embodiment)
FIG. 1 is a block diagram showing a configuration of a data compression apparatus 101 according to the present embodiment. As shown in FIG. 1, the data compression apparatus 101 includes a template storage unit 102, a template matching location detection unit 103, a separation unit 107, a template replacement unit 111, and a switching unit 104. The data compression apparatus 101 receives input data 108 having a plurality of vertices (vertex group) each having a type and a value, and reference information between the vertices, and outputs first output data 109 and second output data. Data 110 is output.

  In the present embodiment, the details of the invention will be described with a procedure for compressing the XML document 20 shown in FIG. FIG. 2A is a diagram illustrating an example of a text representation of the XML document 20. The XML document 20 has a data structure 21 suitable for internal use of a computer as shown in FIG. 2B by a known method (for example, Xerces shown in http://xml.apache.org/xerces2-j/). Can be converted to Therefore, hereinafter, the compression process after the XML document 20 is converted into the data structure 21 shown in FIG. 2B will be described. This data structure 21 has a plurality of vertices (vertex group) each having a type and a value, and reference information between the vertices.

  In the data compression apparatus 101, the separating unit 107 inputs the data structure 21 shown in FIG. 2B as input data 108, and the input data 108 is cross-reference relationship data Rd having reference information between vertices, type, It is separated into vertex group data Td consisting of a plurality of vertices having values. That is, the separating means 107 generates the cross-reference relationship data 1000 shown in FIG. 3 and the table 900 shown in FIG. 4 as follows, and the table 900 is a set of vertices 1001 to 1007 (vertices having types and values). Group)), the data structure 21 is separated. The cross-reference relationship data 1000 is generated by sequentially assigning each vertex 1001 to 1007 an ID (vertex ID) that can uniquely identify each vertex. Further, the table 900 generates a list of the assigned vertex IDs and the types and values originally possessed by the corresponding vertices 1001 to 1007. There are a breadth-first search, a depth-first search, and the like as a method of assigning the vertex IDs to the vertices 1001 to 1007. Here, the depth-first search is used. Further, the separating unit 107 outputs the vertex group data obtained by the separation as the second output data 110.

The cross-reference relationship data 1000 and the table 900 separated from the XML document 20 are as shown in FIGS. 3 and 4, respectively. Here, the table 900 shown in FIG. 4 is expressed in a format in which the vertex ID 901, the type 902, and the value 903 are one line.
The template storage unit 102 stores templates and template entities in advance prior to compression. At this time, the template storage unit 102 stores, for example, a high-frequency template that is known to be frequently applied in advance as a template or a template entity, or a template generated by a template generation device 1601 described later. Yes. Examples of such templates and template entities include a template 1105 and a template entity 1109 shown in FIGS. 5 (a) and 5 (b).

  The template 1105 has a template ID 1106, connection information 1107, and pattern information 1108. The template ID 1106 is used to uniquely identify each template when a plurality of templates are stored in the template storage unit 102. The pattern information 1108 represents a reference information pattern between vertices represented by the template, and includes a plurality of vertices and mutual reference information. The reference information between vertices included in the pattern information 1108 includes four types of references: parent reference, child reference, next sibling reference, and previous sibling reference. Of the references that do not have a vertex at the connection destination, a reference is used in the pattern information 1108 for a reference that is not used when the template is applied and the cross-reference relationship data 1000 is compressed as described later. This can be realized, for example, by defining an invalid vertex and making a reference to that vertex. The connection information 1107 lists information indicating connections with other templates and vertices when the template 1105 is actually applied and the cross-reference relationship data 1000 is compressed.

  Since such a template 1105 is configured by distinguishing connection information 1107 and pattern information 1108 having reference information, the pattern information 1108 can be shared by different templates 1105. That is, by making the connection information 1107 different, vertices and other templates that can be connected are made different, and even if the pattern information 1108 is the same, it can be used like another template. Then, the reference information between the vertices included in the template can be omitted, and the memory usage (storage area) of the template storage unit 102 can be used efficiently.

  In FIG. 5A, “T1” is set as a specific value in the template ID 1106. The pattern information 1108 includes four vertices 1101 to 1104 and reference information indicating references between them, and the references are described by arrows. The reference type is described as an arrow, the parent reference is p, the child reference is c, the next sibling reference is ns, and the previous sibling reference is ps. For example, the child reference c of the vertex 1101 designates the vertex 1102, and the next sibling reference ns of the vertex 1103 designates the vertex 1104.

  In addition, a reference indicating that a template is applied and is not used when the cross-reference relationship data 1000 is compressed is described by using “x” as an end point, and when the template is applied and the cross-reference relationship data 1000 is compressed, References indicating that they are connected to templates and vertices are described with “◯” at the end points. For the four references corresponding to the latter, that is, the parent reference of the vertex 1101, the child reference of the vertex 1102, the vertex 1103, and the vertex 1104, the vertex information and the type of reference are listed in the connection information 1107.

A template entity 1109 shown in FIG. 5B is used to indicate that a template has been applied to the input data 108 when the cross-reference relationship data 1000 is compressed (template applied). The template entity 1109 has a template entity ID 1110, a use template ID 1111 representing a template to be applied, entity connection information 1112, and entity information 1113.
The template entity ID 1110 is used to uniquely identify the application location of the template 1105 when the template 1105 indicated by the template ID 1111 is applied and the cross-reference relationship data 1000 is compressed. The entity connection information 1112 lists vertices to be connected when the template 1105 is applied and the cross-reference relationship data 1000 is compressed. As the entity information 1113, when the template 1105 is applied and the cross-reference relationship data 1000 is compressed, vertex IDs (vertex IDs) included in the template 1105 are accumulated. Specific values entering the entity connection information 1112 and the entity information 1113 will be described later.

A template matching location detection unit (hereinafter referred to as “matching location detection unit”) 103 detects a matching location corresponding to the template stored in the template storage unit 102 from the cross-reference relationship data 1000 shown in FIG. In addition to the template 1105, it is expected that a plurality of templates are stored in the template storage unit 102. Therefore, it is considered that there are a plurality of detection results by the coincidence point detection unit 103. However, for example, according to the matching location detection procedure shown in FIG. 12, the detection result is uniquely determined.
In addition, the matching location detection procedure shown in FIG. 12 is as follows.
In step 1 after the processing is started, the templates stored in the template storage means are selected one by one in descending order of the number of vertices, and the following processing is repeated.
The selected template is set as Pj, and the process proceeds to Step 2.
In the subsequent step 2, the combinations for selecting the vertices that match the number of vertices of the selected template P from the vertices included in the cross-reference relationship data are X1, X2, and Xm, and one of them is selected one by one. repeat.
Let Xk be the selected combination.
Next, the process proceeds to step 3, where it is determined whether or not all the vertices included in Xk have a replaced mark. If not, the process proceeds to step 4; otherwise, the process proceeds from step 3 to step 5 until k reaches m. repeat.
In step 4, it is determined whether Xk is the same type as Pj. If it is the same type, the process proceeds to step 5; otherwise, k is incremented by one and the process returns to step 3.
In step 5, the templates Pj and Xk are registered in the pattern matching information as matching locations, and the vertices included in Xk are marked as replaced.

In the present embodiment, when only the template 1105 is stored in the template storage unit 102, the matching location obtained by the matching location detection procedure shown in FIG. The matching part of the template indicates, for example, a template ID 1702 indicating the used template and assignment from the vertex of the template to the vertex of the original cross-reference relationship data 1000 as in the matching part information 1701 shown in FIG. It can be represented by vertex correspondence information 1703.
Here, as an example of the result of detecting the matching part corresponding to the template in the cross-reference relationship data 1000, the matching part information 1704 is shown in FIG. When only the template 1105 is stored in the template storage unit 102, the matching location information 1704 detects a matching location corresponding to the template 1105 in the cross-reference relationship data 1000 by the matching location detection procedure shown in FIG. It is a result and it has shown that there was one coincidence part. The matching location information 1704 is input from the matching location detection unit 103 to the template replacement unit 111.

The template replacement unit 111 receives the matching part information 1704 (matching part information Ud shown in FIG. 1) from the matching part detection unit 103, and uses the corresponding vertex as a starting point and corresponds to the vertex included in the pattern information of the template. Are replaced with the template entity 1109, and the result of the replacement is output as compressed cross-reference relationship data Cd. This replacement can be realized, for example, by the matching location detection procedure shown in FIG.
Here, the result of the replacement is shown in FIG. In FIG. 6, a template entity 1202 exists as a template entity. The template entity 1202 has a template entity ID 1203, a use template ID 1204, entity connection information 1205, and entity information 1206. “X1” is set in the template entity ID 1203, and “T1” is set in the usage template ID 1204. As a result, the template entity 1202 indicates that the template 1105 shown in FIG.

The entity information 1206 indicates the correspondence between the vertices included in the used template (pattern information) and the vertices of the cross-reference relationship data 1000 before the template is applied. For example, in the case of the template entity 1202, the vertices 1101, 1102, 1103, and 1104 of the template 1105 correspond to the vertex 1001, the vertex 1002, the vertex 1006, and the vertex 1007 of the cross-reference relationship data 1000 before compression, respectively. Yes.
The entity connection information 1205 indicates connection information with other template entities and vertices in the template entity. Here, as described above, the template applied by the template entity 1202 is the template 1105, but it is described in the connection information 1107 in the template 1105 that the template 1105 holds four references that can be connected to the outside. (See FIG. 5A).

The template entity 1202 describes which vertex is the reference destination. That is, it is described that the child reference of the vertex 1102 is connected to the vertex 1003, and the parent reference of the vertex 1101, the child reference of the vertex 1103, and the child reference of the vertex 1104 are described as not connected to any vertex. .
When replacing, the template replacement unit 111 can re-compress the cross-reference relationship data 1000 after replacement with the template by expressing the information indicating the replacement with the template entity 1202 with the same configuration as the vertex. It is data.

  In the present embodiment, when the template replacement unit 111 replaces the cross-reference relationship data 1000 using the template 1105, it is not replaced with the template entity 1202 indicating the application of the template 1105, but as follows. Yes. That is, as shown in FIG. 6, the template replacement unit 111 provides a vertex with pointer (vertex with instruction) 1207 indicating the entity ID (X1) set in the template entity ID 1203, so that the replacement by the template entity 1202 is performed. On the other hand, the vertices other than the pointed vertex 1207, that is, the vertex 1003, the vertex 1004, and the vertex 1005 have their respective references as before the compression, and the replacement using the template 1105 is performed. . As a result, the vertex 1207 with a pointer representing the replacement with the template entity 1202 has the same configuration as the other vertices (vertex 1003, vertex 1004, vertex 1005) not replaced by the template 1105, and the template entity 1202 is one of the vertices. Can be expressed as In this way, the template replacement unit 111 can realize re-compression of the cross-reference relationship data 1000 after compression using the template.

  The switching unit 104 outputs the compressed cross-reference relationship data 1201 (Cd shown in FIG. 1), which has been compressed with respect to the cross-reference relationship data 1000 before compression, as the first output data, or the matching location detection unit 103. Is switched to perform recompression (output or recompression) and both can be selected. This switching means 104 outputs using a determination result of whether or not there is room for recompression (presence / absence of recompression room), the number of recompressions specified in advance (hereinafter referred to as “specified number of times”), and the like. Or recompression can be selected. Whether or not there is room for recompression can be determined by checking whether or not the template matching location information Ud output by the matching location detection means 103 is empty.

In this embodiment, output or recompression is selected using the designated number of times. As an example of the input data, the data structure 21 of the XML document shown in FIG. 2B is given, and the operation when only the template 1105 shown in FIG.
The switching unit 104 selects output or recompression of the compressed cross-reference relationship data 1201 using the specified number of times. Here, for example, it is assumed that the designated number of times is N, and the current compression is the nth time.

The output is selected when n ≧ N. At this time, the switching unit 104 outputs the cross-reference relationship data 1201 as the first output data 109.
Recompression is selected when n <N. At this time, the switching unit 104 inputs the cross-reference relationship data 1201 to the matching location detection unit 103. Since the cross-reference relationship data 1201 expresses the application of the template entity 1202 in the same configuration as the vertex, the application location of the template is regarded as one vertex, and for example, the matching location information is obtained by the matching location detection procedure shown in FIG. Can be derived. When the cross-reference relationship data 1201 shown in FIG. 6 is input to the matching part detection unit 103, the detected matching part detection information becomes matching part information 1710 shown in FIG. 11C.

  FIG. 7 is a diagram showing cross-reference relationship data 1301 obtained by recompressing the cross-reference relationship data 1201 shown in FIG. The template replacement unit 111 starts from the vertex 1207 indicated in the matching part detection information in the cross-reference relationship data 1201 shown in FIG. 6, and the vertex group corresponding to the pattern information of the template 1205, that is, the vertex 1207, the vertex 1003, vertex 1004, and vertex 1005 are replaced using the template entity 1302. As a result of this replacement, as shown in FIG. 7, cross-reference relationship data 1301 including only a single vertex 1307 is obtained. In this embodiment, for example, the vertex 1307 holds the template entity ID 1303 of the template entity 1302 to indicate that the template entity 1302 is referred to, and has the same configuration as the vertex that is not replaced by another template.

Note that the number of connection information in the template may be larger than the number of connection information at the vertex (one each for parent reference, child reference, next sibling reference, and previous sibling reference). The vertices with a large number of points may be excluded from matching point detection by the matching point detection unit 103.
As described above, the data compression apparatus, when replacing the matching part detected by the matching part detection unit 103 with the template, provides the designated vertex 1207 indicating the replacement part by the template, and enables re-substitution by the template. , So that templates can be applied recursively. Therefore, the same template can be applied a plurality of times, and a high compression rate can be realized with a smaller number of templates.

(Second Embodiment)
By referencing a part of the vertices in the cross-reference relationship data having a plurality of vertices each having a type and a value and reference information between the vertices by performing the recompression as described above, the compressed mutual It is necessary to perform a repetitive expansion process on the reference relationship data using a template and perform a recursion process recursively. Therefore, as the number of recompression increases, it may take a significant time to refer to the vertex. Therefore, it is preferable that the time required to refer to the vertex can be shortened as much as possible.
For this purpose, when the cross-reference relationship data provided as input data has a rooted tree structure, the vertex from the shallowest position (root vertex) of the original cross-reference relationship data having the tree structure to the deepest position. If it is possible to sequentially restore to a certain vertex (leaf vertex), that is, if the template is present at the leaf vertex in the restored cross-reference data, only the points that need to be restored are selected. It can be restored. Therefore, it is considered that the time required for reference can be shortened.

This is because, for example, the matching part detection unit 103 detects only the matching part including the leaf vertex of the tree structure in the detection of the matching part, and the template replacement unit 111 recursively replaces the matching part with the template. Can be realized.
Here, for example, it is assumed that the cross-reference relationship data 1000 shown in FIG. 3 is input to the coincidence portion detection unit 103 and the template 1105 shown in FIG. At this time, the matching point detection unit 103 prioritizes the leaf vertex in the detection of the matching point, and therefore detects the matching point from the leaf vertex 1005. Then, since the vertex 1002 is detected as a matching location including the leaf vertex 1005, the matching location detection unit 103 inputs the vertex 1002 to the template replacement unit 111 as the matching location information Vd.

Then, the template replacement unit 111 recompresses the vertex group corresponding to the pattern information 1108 of the template 1105, that is, the vertex 1002, the vertex 1003, the vertex 1004, and the vertex 1005, with the vertex 1002 as the starting point, using the input matching part information Vd. Replace with templates as possible.
On the other hand, the switching unit 104 selects the output or recompression of the compressed cross-reference relationship data replaced with the template. Here, assuming that recompression is selected, the switching unit 104 inputs the cross-reference relationship data replaced with the template to the matching location detection unit 103.

  FIG. 8 is a diagram showing the cross-reference relationship data 1401 when recompression (priority compression) that prioritizes leaf vertices in the manner described above. When the cross-reference relationship data 1401 is compared with the cross-reference relationship data 1301 that has been recompressed without considering the recompression order as shown in FIG. 7, there are differences in the following points. The latter cross-reference relationship data 1301 cannot restore the root vertex of the cross-reference relationship data 1000 before compression, that is, the vertex 1001, unless the template is completely restored to the original cross-reference relationship data. Therefore, it is unclear at which position in the original cross-reference relationship data 1000 the restored location was located. On the other hand, the former cross-reference relationship data 1401 can restore the (root) vertex 1001 only by restoring the template to the cross-reference relationship data once. It is possible to restore while confirming. Therefore, if the priority compression is performed as described above, the time required to refer to the vertex can be shortened.

(Third embodiment)
Since the first output data 109 output from the data compression apparatus 101 expresses the template entity indicating the application of the template as a vertex, the first output data 109 is input to the coincidence portion detection unit 103 as with the cross-reference relationship data before compression. be able to. Therefore, recompression of cross-reference relationship data can be performed by connecting a plurality of data compression devices in series. Here, FIG. 15 is a block diagram showing a configuration of a data compression apparatus 2104 in which a plurality of data compression means are connected in series. As shown in the figure, the data compression apparatus 2104 includes a first compression unit 2101 having a matching point detection unit 2107 and a template replacement unit 2108, a second compression unit 2102,..., An Nth compression unit 210n. Connected in series continuously. The first to Nth compression units 2101 to 210n all refer to the same template storage unit 2109. Each of the compression means 2101 to 210n inputs cross-reference relationship data, compresses the cross-reference relationship data using a template, and outputs the compressed cross-reference relationship data as output data. . The data compression apparatus 2104 having such a configuration can re-compress cross reference data.

Embodiment of Data Restoration Apparatus FIG. 9 is a block diagram showing a configuration of a data restoration apparatus 1501 according to an embodiment of the present invention. The data restoration apparatus 1501 includes a template storage unit 1502, a template development unit 1505, a switching unit 1503, and a synthesis unit 1504. This data decompression apparatus 1501 uses the compressed cross-reference relationship data as the first input data 1506 and the vertex group data having the type and value as the second input data 1507. The cross-reference relationship data is restored and combined with the vertex group data input as the second input data 1507, and output data 1508 is output. In this embodiment, the cross-reference relationship data 1301 shown in FIG. 7 is input as the first input data 1506 and the table 900 shown in FIG. 3 is input as the second input data 1507 to the data restoration device 1501. Then, the data restoration device 1501 will be described by taking as an example a case where only the template 1105 shown in FIG.

The template expansion unit 1505 expands the cross-reference relationship data 1301 (the template application location) given as the first input data 1506 using the template 1105 stored in the template storage unit 1502. The development can be performed by, for example, the replacement procedure shown in FIG.
The replacement procedure is as follows.
In the replacement procedure, after the processing is started, the process proceeds to step 6, i is set to 0, and the process proceeds to step 7.
In step 7, the processing of step 8 and step 9 is repeated one by one for all the matching locations included in the template matching location information.
In step 8, one template entity is created and entity ID = i. This template entity is set to Oi, and the following processing is performed. The usage template ID is duplicated from the usage template ID of the matching location Mi.
The entity information is duplicated from the vertex correspondence information of Mi. For the entity connection information, the original reference is substituted as it is based on the correspondence described in the entity information.
In step 9, i = i + 1 is calculated.
After executing Step 7 and repeating Step 8 and Step 9, in Step 10, the created template entities are selected one by one, and the following processing is repeated. Let the selected template entity be Oi.
Next, the process proceeds to step 11, and if the connection destination vertex of the reference described in the entity connection information is included in another template entity, it is replaced with a set of the template entity ID and the template vertex.
When the cross-reference relationship data 1301 is expanded by the restoration procedure shown in FIG. 14, the cross-reference relationship data 1201 shown in FIG. 6 is obtained.
The restoration procedure is as follows.
In FIG. 14, in step 12 after the start, all template entities included in the compressed cross-reference relationship data are X1, X2, and Xn, and the following is performed for all.
Let Xi be the selected template entity.
In step 13, the reference information between the vertices of the template used by the template entity Xi is copied, and the vertex ID described in the entity information of the template entity Xi is assigned.
Next, the process proceeds to step 14, and if the vertex described in the entity connection information of the template entity Xi is a vertex included in another template entity Xm, the vertex ID described in the template entity Xm is replaced.

The switching unit 1503 determines whether or not there is a further template application location in the expanded cross-reference relationship data 1201 (presence / absence of a re-application location). When the switching unit 1503 determines that there is a re-applied portion, the cross-reference relationship data as a result of the expansion is input to the template expansion unit 1505 for further expansion. If the switching unit 1503 determines that there is no re-applied portion, the switching unit 1503 switches the output destination of the cross-reference relationship data as a result of the expansion to the combining unit 1504. Note that the reapplied portion can be detected by detecting, for example, a vertex having a reference to the template entity among all the vertices.
The template expansion unit 1505 and the switching unit 1503 repeatedly execute the above-described expansion until there are no more application locations.
In the case of the present embodiment, the first development result using the template 1105 is the cross-reference relationship data 1201 shown in FIG. Since the cross-reference relationship data 1201 has a reference to the template entity 1202 (because there is a vertex 1207 with a pointer), it still has a reapplied portion. Therefore, the switching unit 1503 inputs the cross-reference relationship data 1201 as a result of the expansion to the template expansion unit 1505, and the template expansion unit 1505 expands the reapplied portion again. Then, the cross-reference relationship data 1000 shown in FIG. 3 is restored, and the cross-reference relationship data 1000 is input to the switching unit 1503. Since the cross-reference relationship data 1000 does not include a reapplied part, the switching unit 1503 switches the output destination to the synthesis unit 1504.

  The synthesizing unit 1504 synthesizes the input cross-reference relationship data 1000 and the table 900 input as the second input data 1507. In the table 900 shown in FIG. 4, a vertex ID 901 corresponding to each vertex is assigned. Therefore, the synthesizing unit 1504 applies the information set in the type 902 and the value 903 to each vertex having the same vertex ID 901 in the cross-reference relationship data 1000, thereby obtaining the cross-reference relationship data 1000 and the table 900. Synthesize. By performing this synthesis, it is possible to restore the data structure 21 having vertex group data each having a type and a value and reference information between the vertices.

In FIG. 9, the data restoration device 1501 is configured as a single device in which the respective units are integrated. However, the data restoration device 1501 is not necessarily realized as a single device, and a communication unit that does not illustrate a plurality of devices is illustrated. It is also possible to implement by connecting with. For example, after separating the template storage unit 1502 from the data restoration device 1501, the template storage unit 1502 may be realized as another single device, and the two devices may be connected by a communication unit (not shown). Then, the template storage unit 1502 can be shared among a plurality of data restoration apparatuses that do not have the template storage unit 1502.
In the present embodiment, the switching unit 1503 switches the input to the template expanding unit 1505 and the output to the combining unit 1504, so that the template expanding process in the template expanding unit 1505 is repeatedly executed. In addition, a switching unit may be used in which the expanded cross-reference relationship data is always input to the template expansion unit 1505 and output to the synthesis unit 1504 when the template expansion is completed.

  Furthermore, the switching unit 1503 may be configured to provide data such as the reference position of the cross-reference relationship data and the number of times of template expansion from the outside of the data restoration device 1501. In this way, the template expansion means 1505 should refer to a data structure having data of vertices each having a type and a value that can be obtained by expanding the application location of the template, and reference information between the vertices. It is possible to perform only the expansion using a template necessary for referencing a vertex (which needs to be referred to).

Embodiment of template generation apparatus (first embodiment)
FIG. 10 is a block diagram showing a configuration of template generation apparatus 1601 according to the present embodiment. The template generation device 1601 includes a template candidate detection unit 1602, a template replacement unit 1603, an input data acquisition unit 1604, a template candidate storage unit 1605, and a template selection unit 1606. The template generation device 1601 receives an input data group 1607 and outputs output data (template group) 1608.

The input data group 1607 is composed of a plurality of cross-reference relationship data having a netting tree structure. In the present embodiment, two cross-reference relationship data 2900 and 3000 shown in FIGS. 23 and 24 separated from the XML document 30 shown in FIG. 21 and the XML document 40 shown in FIG. The generation device 1601 will be described.
In order to separate the cross-reference relationship data 2900 and 3000 from the XML documents 30 and 40 shown in FIG. 21 and FIG. 22, it may be performed in the same manner as in the first embodiment. That is, in this separation, a vertex ID that can uniquely identify each vertex is assigned to each vertex in order to generate cross-reference relationship data 2900, 3000, and the assigned vertex ID and the corresponding vertex originally possessed. A table can be generated by listing pairs of types and values, and the table can be easily used as a vertex group.

A plurality of procedures are conceivable as procedures for realizing the present invention. In this embodiment, it is assumed that the first to third procedures of template generation shown in FIGS. 25, 26, and 27 are used.
First, a first procedure for generating a template shown in FIG. 25 will be described. After starting the processing of the first procedure, the template generation device 1601 proceeds to step 30, and the input data acquisition unit 1604 extracts one cross-reference relationship data from the input data group 1607 given to the template generation device 1601. Reference relationship data is input to the template candidate detection means 1602. The template candidate detecting unit 1602 sets the input cross-reference relationship data to the retained data D until the next cross-reference relationship data is input. The selection order of the cross-reference relationship data to the template candidate detection unit 1602 may be any order, but in this embodiment, the cross-reference relationship data 2900 and 3000 are input in this order.

(Intermediate procedure 1)
Next, the process proceeds to step 31 where the template generation device 1601 calls the second procedure using the root vertex of the retained data D as an argument. Here, the root vertex 2901 of the cross-reference relationship data 2900 is set as an argument, and the second procedure is called. In the following step 32, it is determined whether or not the cross reference relationship data exists in the input data group 1607. If there is cross reference relationship data, the process returns to step 30; As a result of this step 32, as long as the input data group 1607 includes cross-reference relationship data, the template generation device 1601 repeatedly calls the second procedure using each cross-reference relationship data (root vertex) as an argument.
On the other hand, when there is no cross-reference relationship data in the input data group 1607, the input data acquisition unit 1604 detects this and notifies the template selection unit 1606. When the notification is received, the template generation apparatus 1601 proceeds to step 33, where the template selection unit 1606 selects a template from the template candidates stored in the template candidate storage unit 1605, and outputs the selected template as a template group 1608. To do.

Next, the second procedure will be described. In the second procedure, the vertex of the retained data D is acquired from the first procedure or the third procedure and set as the argument cursor. This argument “cursor” represents which position of the retained data D is traced in the second procedure.
Here, as shown in the intermediate procedure 1 described above, the root vertex 2901 of the retained data D is set as an argument when the second procedure is called.
As shown in FIG. 26, when the second procedure is started, the process proceeds to step 40, and in the retained data D, the partial reference information that forms the tree structure rooted in the argument cursor is set to T. Here, since the root vertex 2901 of the retained data D is set in the argument “cursor”, the partial reference information T matches the retained data D (here, the cross-reference relationship data 2900).

Next, proceeding to step 41, the value of the argument cursor is set to the variable R. This variable R indicates the root vertex of the partial reference information T of the retained data D.
Subsequently, the process proceeds to step 42, where the vertex forming the tree structure in the partial reference information T is searched by the depth-first search using the vertex indicated by the variable R as the starting point, and the first (leaf) vertex encountered is set as the argument cursor. . At this stage, the vertex 2904 is set in the argument cursor. The variable R indicates the vertex 2901.

Next, in step 43, it is determined whether or not there is a parent vertex in the argument cursor. Here, since the vertex 2904 indicated by the argument “cursor” has a parent reference indicating the vertex 2903, the process proceeds to step 44. At this time, the template generation device 1601 sets a candidate pattern to be a pattern having the vertex indicated by the argument cursor and the vertex indicated by the parent reference of the argument cursor by the template candidate detection unit 1602, and this candidate pattern exists in the retained data D Whether or not to do is detected. Here, a pattern having a vertex 2904 and a vertex 2903 is set as a candidate pattern.
In step 44, the template generation device 1601 sets a candidate pattern composed of the argument cursor and the vertex indicated by the parent reference as the first argument, and sets the variable R and the argument cursor as the second argument and the second argument, respectively. Set the third argument to call the third procedure. In the third procedure, the first argument is expressed as C, the second argument, and the third argument as R and cursor, respectively.

(Intermediate procedure 2)
When the template generating apparatus 1601 calls the third procedure, the process proceeds to step 50 shown in FIG. 27, in which the third argument cursor is set in the data S and stored. Although this step 50 is not necessarily required, in the present embodiment, the data S is used for determining the interruption condition for the template candidate search in step 57 described later.
Then, the template generation device 1601 proceeds to step 51 and determines whether or not a template candidate having the pattern (candidate pattern C) indicated by the first argument C is accumulated in the template candidate accumulation unit 1605. Here, since the template candidate having the candidate pattern C is not accumulated in the template candidate accumulating unit 1605, the template generating apparatus 1601 proceeds to step 53 and sets the template candidate having the candidate pattern C to “1” as the number of passages (appearance number). "Is set and stored in the template candidate storage means 1605.

There are various methods for storing the template candidates in the template candidate storage unit 1605. In this embodiment, the template candidates are stored while building a tree structure in the format shown in FIG.
Here, the configuration of the template candidates will be described with reference to FIG. A template candidate 3401 shown in FIG. 28 has pattern information 3402, an identification ID 3403 for uniquely identifying each template candidate, a pass count 3405, and a replacement count 3406. The number of passes 3405 indicates the total number of times that can be replaced by the template candidates counted in the process of searching for replaceable template candidates in the cross-reference relationship data input to the template generation device 1601. This number of passes 3405 is initialized to “1” when the template is stored. The number of replacements 3406 indicates the number of times that the template is actually applied. The number of replacements 3406 is initialized to “0” when the template is stored.

  The template generation device 1601 stores a plurality of template candidates in the template candidate storage unit 1605. In the present embodiment, as shown in FIG. 29, a plurality of template candidates are accumulated in a tree structure in which each template candidate is a vertex. In FIG. 29, the arrows between the template candidates indicate that, by adding one vertex to a certain template candidate, the same structure as the template candidate indicated by the arrow is obtained. A label N (M) attached to the arrow means that a vertex is added at the reference N to the vertex M. Reference N is described in three types: parent reference p, child reference c, and next sibling reference ns. Since the number of passes and the number of replacements of the template candidate change every moment, the illustration is omitted in FIG.

  When the template generation device 1601 calls the third procedure for the first time, it proceeds to step 53. Then, as shown in FIG. 29, since the candidate pattern C matches the pattern information of the template candidate 3501, the template candidate 3501 is added to the template candidate accumulation unit 1605. At this time, since the candidate pattern C can be replaced at the vertex of the retained data D indicated by the third argument “cursor”, the number of times the template candidate 3501 is passed is set to “1”.

When the template generating apparatus 1601 proceeds to step 54, it determines whether or not a stop condition is satisfied for the template candidate having the candidate pattern C, and determines whether or not to generate more template candidates based on the stop condition. Although multiple stop conditions can be considered, in this embodiment, all vertices in the partial reference information in which the vertex corresponding to the second argument R (vertex R) is the root vertex of the tree structure among the vertices are template candidates. The replacement condition can be a stop condition. This can be determined by checking whether or not the third argument cursor is the last vertex of the subtree below the vertex R (stop condition 1). Further, since the storage capacity of the template candidate accumulation unit 1605 is limited, it may be set as a stop condition to stop the detection of the template candidate by the number of vertices in the template candidate, the height of the tree, or the like (stop condition 2). The stop condition 2 can be determined as to whether or not the number of vertices included in the candidate pattern C of the template candidate is 5 or more, but is not limited thereto.
(Intermediate procedure 3)
Here, the candidate pattern C matches the pattern information of the template candidate 3501, and the number of vertices in the template candidate is “2”. The second argument R indicates the vertex 2901,
Since candidate pattern C does not include all vertices less than or equal to second argument R, either stop condition 2 (the number of vertices included in template candidate pattern C is 5 or more) and stop condition 1 Is not established. Therefore, the process of the template generation device 1601 proceeds to step 55.

  In step 55, the template generation device 1601 traces the retained data D by the depth-first search from the third argument “cursor”, adds the next vertex encountered at that time to the candidate pattern C, and adds the added pattern to the candidate pattern C. Candidate pattern C is set. As a result, the template generation device 1601 adds a vertex to the candidate pattern C and detects a new template candidate. Thereby, extended candidate detection in the present invention is performed. Here, the third argument “cursor” indicates the vertex 2904. The candidate pattern C matches the pattern of the template candidate 3501 and matches the vertex 2903 and the vertex 2904 on the retained data D. Therefore, by the depth-first search, the same structure as that of the template candidate 3502 shown in FIG. 29, that is, the structure obtained by adding the vertex 2905, which is the next vertex of the vertex (vertex 2904) indicated by the third argument cursor, to the candidate pattern C is obtained. A pattern is detected by the template candidate detection unit 1602 and the detected pattern is newly set as a candidate pattern C.

Further, the process proceeds to step 56, where the newly added vertex in the retained data D is set as the third argument cursor. Here, the vertex 2905 is set in the third argument “cursor”.
Next, proceeding to step 57, it is determined whether or not the interruption condition is satisfied for the template candidate having the candidate pattern C. If the interruption condition is satisfied, the process proceeds to step 58, and if not satisfied, the process returns to step 51. By adding the determination of the interruption condition in step 57 and the processing in step 58, it is possible to limit the size of the template and suppress the generation of a template with a low appearance frequency. As the interruption condition, the difference between the height of the vertex indicated by the third argument “cursor” in the retained data D and the height of the retained data D of the vertex (vertex S) indicated by the data S is N or more. It is possible to use a plurality of methods and combinations thereof, such as the number of vertices included is N or less.

(Intermediate procedure 4)
In the present embodiment, the interruption condition in step 57 is as follows. That is, the height of the vertex indicated by the third argument “cursor” in the retained data D is the same as the height of the vertex indicated by the data S, and the vertex indicated by the third argument “cursor” has a child vertex. The interruption condition is set.
When this interruption condition is satisfied, the process proceeds to step 58, in which the third argument “cursor” is set as an argument and the above-described second procedure is called. When the second procedure ends, the process returns to step 55. When the interruption condition is satisfied in this way, the process proceeds to step 58, and another template candidate (candidate pattern) is repeatedly detected starting from the third argument cursor. As a result, the template candidate detection unit 1602 limits the size of the template candidates to be detected.

  Here, as shown in FIG. 23, the third argument “cursor” indicates a vertex 2905, and the vertex 2905 has no child vertex. Therefore, the interruption condition is not satisfied, and the template generation device 1601 returns from step 57 to step 51. The candidate pattern C at this point matches the pattern of the template candidate 3502 shown in FIG. Further, only the template candidate 3501 is stored in the template candidate storage unit 1605. Therefore, the template generating apparatus 1601 proceeds from step 51 to step 53, and the template candidate detecting unit 1602 sets the template candidate 3502 having the candidate pattern C to “1” as the number of appearances and adds it to the template candidate accumulating unit 1605. . In addition, the template candidate detection unit 1602 provides an arrow between the template candidate 3501 and the template candidate 3502 at the time of this addition, and a label ns (3602) indicating that a vertex has been added to the pattern as the next sibling reference of the vertex 3602 Is attached.

  Then, the template generation device 1601 repeatedly executes step 54 to step 57 as described above. At this time, a structure similar to the pattern of the template candidate 3503 is acquired as the candidate pattern C, and steps 51 to 53 are executed. As a result, a template candidate 3503 is added to the template candidate storage unit 1605, an arrow is set between the template candidate 3502 and the template candidate 3503, and a label ns (3603) is attached.

In the template candidate accumulation means 1605 at this stage, only the template candidate 3501, the template candidate 3502, and the template candidate 3503 are accumulated among the template candidates shown in FIG. Further, the number of times each of the template candidates 3501, the template candidates 3502, and the template candidates 3503 is “1”.
(Intermediate procedure 5)
Then, after executing Steps 54 to 56 and proceeding to Step 57, the third argument “cursor” indicates the vertex 2906, and the interruption condition shown in the intermediate procedure 4 is satisfied. Therefore, the template generation device 1601 interrupts template detection and proceeds to step 58, sets the third argument cursor as an argument, and calls the second procedure.

(Intermediate procedure 6)
Here, the vertex 2906 is set as an argument, and the second procedure is called. Thereby, a template candidate is searched for the subtree of the retained data D having the vertex 2906 as a root.
When the template generation apparatus 1601 calls the second procedure, it executes steps 40 to 43 shown in FIG. 26 in the same manner as described above. Then, the process proceeds to step 44, in which the same structure as the pattern of the template candidate 3501 is set as the first argument, the vertex 2906 is set as the second argument, and the vertex 2907 is set as the third argument. Call the procedure.

Then, by calling the third procedure, the template generation device 1601 acquires a pattern similar to the pattern included in the template candidate 3501 as the candidate pattern C. Also, the vertex 2906 is acquired as the second argument R, and the vertex 2907 is acquired as the third argument cursor.
Here, since the candidate pattern C has already been stored as the template candidate 3501 in the template candidate storage unit 1605, the template generation apparatus 1601 proceeds from step 51 to step 52 in FIG. Add “1” to “2”.

The template generation device 1601 returns to step 51 when the processing from step 54 to step 57 is executed. At this time, the candidate pattern C matches the template candidate of the template candidate 3502, and the third argument cursor is the vertex 2908. Is shown. At this time, the template generation device 1601 adds “1” to the number of times the template candidate 3502 has passed to make “2”.
At this point, the third argument cursor indicates vertex 2908. This satisfies the above-described stop condition 1 shown in the intermediate procedure 3, that is, the argument “cursor” is the last vertex of the subtree with the vertex R or less. Therefore, the process of the template generation device 1601 proceeds from step 54 to step 59.

(Intermediate procedure 7)
When the template generating apparatus 1601 proceeds to step 59, it determines whether or not replacement with a template candidate having the candidate pattern C is possible as it is. If it is determined that the replacement is possible, the process proceeds to step 60; otherwise, the process proceeds to step 62.
Here, whether or not replacement with the candidate pattern C is possible matches the number of child vertices of the root vertex X in the candidate pattern C and the root vertex X when the candidate pattern C matches the retained data D. The determination can be made based on whether or not the number of child vertices of the vertex Y in the retained data D matches (hereinafter referred to as “replacement determination criterion”). Here, the candidate pattern C has the same structure as the pattern of the template candidate 3502, and the root vertex 3601 has two child vertices, the vertex 3602 and the vertex 3603, and the child vertex of the vertex 2906 in the retained data D Are two of vertex 2907 and vertex 2908. Therefore, the number of child vertices of the root vertex 3601 matches the number of child vertices at the vertex 2906. Therefore, the template generation device 1601 advances the process from step 59 to step 60.

(Intermediate procedure 8)
When the template generating apparatus 1601 proceeds to step 60, the corresponding part in the retained data D is replaced with the template candidate, the applied part is represented by a vertex, and the resulting data is input to the input data acquiring unit 1604. That is, the template replacement unit 1603 replaces the vertex of the retained data D with a template candidate having the same structure as the candidate pattern C, and inputs the replacement result to the input data acquisition unit 1604.
The replacement by the template candidate is performed using the template entity in the same manner as the replacement by the template in the first embodiment. However, in this embodiment, for the sake of simplicity, it is expressed as “candidate replacement”.

Proceeding to step 62, the template candidate stored in the template candidate storage means 1605 is traced upward from the latest template candidate, and the template candidate of the first parent reference or child reference connection encountered is used. The retained data D is replaced and expressed by vertices, and the data obtained as a result is input to the input data acquisition means 1604.
The input data acquisition unit 1604 determines that the template can be further applied unless the replacement holding data D is represented by one vertex, and inputs the holding data D to the template candidate detection unit 1602 to detect the template candidate. Continue. At this time, “1” is added to the number of replacements of the template candidate that matches the candidate pattern C stored in the template candidate storage unit 1605, that is, the template candidate 3502 to be “1”.

  23. From the cross-reference relationship data 2900 shown in FIG. 23, the replacement results are obtained by excluding the vertices 2907 and 2908, and the vertices 2906 are vertices (vertices with instructions) representing replacement with the template candidates 3502, as in the first embodiment. Can be expressed. Also, because of the replacement by the template candidate 3502, the vertex 2908 referred to by the argument cursor no longer exists. Therefore, the argument cursor is set so as to refer to the vertex 2906 after replacement.

In step 61, it is determined whether or not the third argument cursor and the second argument R are different. Here, if the two match, the third procedure ends, and if different, the process proceeds to step 63. If the template candidates are substituted as shown in the intermediate procedure 8 above, all the vertices below the second argument R are included. Therefore, the third argument “cursor” and the second argument “R” both coincide with each other with reference to the vertex 2906. Therefore, the template generation device 1601 ends the third procedure without executing steps 61 to 63.
On the other hand, the third procedure is called up recursively from step 44 in the second procedure as shown in the intermediate procedure 6 described above, but after the processing in step 44 in the second procedure is completed. Then, step 58 of the third procedure that is the caller of the second procedure, that is, the template detection interrupted in the intermediate procedure 5 is resumed.
At this time, the values after the return match the candidate pattern C with the template candidate 3503, the third argument cursor indicates the vertex 2906, the second argument R indicates the vertex 2901, and the data S indicates the vertex 2904. Yes.

  When the template generation device 1601 returns to step 55, the template candidate detection unit 1602 adds a vertex to the candidate pattern C and tries to detect a new template candidate. However, since the vertex 2906 indicated by the third argument “cursor” has already been replaced by the template candidate as described above, and has neither a child reference nor a next sibling reference, the addition of the vertex is performed in the third argument “cursor”. This is performed further toward the parent reference direction of the parent vertex. By adding this vertex, the candidate pattern C obtains the same structure as the template candidate of the template candidate 3505, and the third argument cursor is moved to the vertex 2902 in step 56.

The vertex 2902 indicated by the third argument “cursor” is stored in the above-described intermediate procedure 2 and is shown in the intermediate procedure 4 because the vertex (vertex S) indicated by the data S as the starting point, that is, the vertex 2904 is different in height. The interruption condition is not satisfied. Therefore, the processing of the template generation device 1601 returns from step 57 to step 51, and further proceeds to step 53, where the template candidate 3505 is stored in the template candidate storage unit 1605 with the number of appearances set to “1”.
In step 54, the candidate pattern C matches the pattern of the template candidate 3505, the number of vertices is “5”, and the stop condition shown in the intermediate procedure 3 is satisfied. Therefore, the processing of the template generation device 1601 proceeds to step 59.

In step 59, it is determined whether or not replacement with a template candidate having the candidate pattern C is possible as it is. Here, the replacement criterion shown in the intermediate procedure 7 described above is applied, and it is determined that the template candidate 3505 can be replaced.
Further, in the subsequent step 60, the template generation device 1601 replaces the vertex of the retained data D with the template candidate 3505 using the template replacement unit 1603, and inputs the replacement result to the input data acquisition unit 1604.

The input data acquisition unit 1604 determines that the template can be further applied unless the replacement holding data D is represented by one vertex, and inputs the holding data D to the template candidate detection unit 1602 to detect the template candidate. Continue.
At this time, since the candidate pattern C matches the pattern of the template candidate 3505, “1” is added to the number of times the template candidate 3505 has passed to make “1”.

At this time, the third argument cursor indicates the vertex 2902 after replacement, but the second argument R indicates the vertex 2901. Therefore, in step 61, it is determined that the third argument “cursor” and the second argument R are different, and the process proceeds to step 63 where the second procedure is called using the second argument R as an argument. This means that an attempt is made to detect a template candidate for the original retained data D that has already been replaced by the template candidate.
In the description so far of the present embodiment, template replacement has already been performed twice for the retained data D. At this stage, the retained data D is a vertex belonging to areas 2912 and 2915 in FIG. Have only.

As in the past, the template generation device 1601 recursively calls the second procedure and the third procedure, further advances the replacement of the retained data D with the template candidate, finally ends the first procedure, and retains it. Data D is represented by one vertex. When the input data acquisition unit 1604 detects that the retained data D is represented by one vertex, the input data acquisition unit 1604 acquires the next cross-reference relationship data 3000 from the input data group 1607.
For the cross-reference relationship data 2900 shown in FIG. 23, the status of the template candidate storage unit 1605 at the stage where the replacement with the template candidate is completed is as shown in FIG.

  Then, the input data acquisition unit 1604 starts inputting the cross-reference relationship data 3000 shown in FIG. 24 since the input of the holding data D shown in FIG. 23 is completed. For the cross-reference relationship data 3000 shown in FIG. 24, the same procedure as the cross-reference relationship data 2900 shown in FIG. 23 is repeated. Here, when the argument cursor matches the vertex 3006, the variable R matches the vertex 3008, and the candidate pattern C matches the template candidate 3510 of FIG. 29, the number of vertices of the candidate pattern C is “5”. The stop condition 2 shown in the intermediate procedure 3 is satisfied.

Therefore, the template generation device 1601 advances the processing to step 59 and determines whether or not the template candidate 3510 can be applied as it is. As shown in FIG. 24, since the next sibling vertex 3007 exists at the vertex 3006, the replacement with the template candidate 3510 is impossible, and the process proceeds to Step 62.
Therefore, the template generation device 1601 traces the tree structure upward from the position of the template candidate 3510 in the template candidate storage unit 1605 and finds the template candidate of the parent reference or child reference connection that is encountered first. At this time, “1” is subtracted from the number of times the candidate template is present on the traced path.

In the case of the present embodiment, this is a template candidate 3501 shown in FIG. The subsequent processing proceeds in the same manner as when the template replacement is performed on the cross-reference relationship data 2900.
The cross-reference relationship data 3000 can also be finally expressed by one vertex. At the stage expressed by one vertex, the input data acquisition unit 1604 detects that there is no more input, and inputs that fact to the template selection unit 1606.

FIG. 31 shows the status of the template candidate accumulating unit 1605 at the time when the input of the cross-reference relationship data 2900, 3000 shown in FIGS. Reference numerals 3701 to 3709 indicate patterns.
The template selection unit 1606 selects a template from the template candidates stored in the template candidate storage unit 1605 and outputs the template. At this time, a plurality of template selection methods are conceivable. In the present embodiment, a template candidate whose replacement count is not “0” is selected. At this time, a pattern is extracted from the template candidates, and the extracted pattern is used as a template. This is easily possible if a template is configured as shown in the first embodiment.

In this way, the template generation device 1601 extracts the pattern 3701, the pattern 3702, the pattern 3704, the pattern 3705, and the pattern 3709 from the template candidate storage unit 1605 by using the template selection unit 1606, and each extracted pattern is a template. And the converted templates are output as output data 1608.
The template generation apparatus 1601 can generate a template by the procedure described above.

(Second Embodiment)
As described above, in the present invention, data compression by recursive application of a template, that is, by applying a template and expressing the template as a new vertex, data compression is performed recursively. Considering this, the template holds the reference information held between the vertices in the template and the vertices outside the template as connection information, but the type and number of the connection information is the original cross-reference relationship data. It is desirable not to exceed the types and number of reference information possessed by vertices (one parent reference, one child reference, one previous sibling reference, and one next sibling reference if the vertex is a tree structure) .

In addition, since the efficiency of compression by replacement using a template increases as the type and number of connection information increases, it is desirable that the template be provided so that the connection information is reduced as much as possible.
Therefore, in the present embodiment, the template is generated so as to reduce the connection information as much as possible. That is, when the cross-reference relationship data (input data) input as the data compression target has a rooted tree structure, when detecting a template candidate from the input data in the template generation device, the vertex having the input data is rooted. As a vertex, a partial tree structure formed by descendant vertices of the vertex is set as pattern information of a template candidate to be extracted.

  Since the template generated in this way has a rooted tree structure in which the vertex connected to the outside of the template is only the root vertex, even if the template is regarded as one vertex when recompressing, The type and number of connection information of the template does not exceed the type and number of reference information that each vertex has in the original input data. Moreover, since only the root vertices in the template are interconnected with the outside of the template, when recompressing the input data, it is replaced with the template in order from the deepest vertex (leaf vertex) in the tree structure of the input data As a result, the template expressed as a vertex always becomes a vertex (leaf vertex) at a position deeper than the upper vertex of the place where the template is applied. Therefore, when all the template candidates detected by the template candidate detection unit 1602 are used as templates, it is possible to finally represent input data with one vertex.

FIG. 16 is a flowchart showing an operation procedure in the template candidate detection unit 1602. The template candidate detection unit 1602 receives input data having a rooted tree structure, and as a template candidate, a minimum tree including all vertices having the same depth as the leaf vertex at the shallowest position counted from the root vertex of the input data Detect partial reference information of structure.
If there are vertices having descendant vertices among the vertices having the same depth as the leaf vertices, the partial reference information of the rooted tree structure formed by the descendant vertices of the vertices is used as a template. By inputting again into the candidate detecting means, a template candidate is separately detected.

  Here, for example, a template candidate is detected from the cross-reference relationship data 2008 shown in FIG. The first leaf vertex encountered in the breadth-first search is vertex 2006. The template candidate detection unit 1602 has partial reference information formed by a group of vertices existing from the root vertex 2001 to the depth of the vertex 2006, that is, a rooted tree structure formed by the vertex 2001, the vertex 2002, the vertex 2006, and the vertex 2007. Partial reference information is extracted as a template candidate and stored in the template candidate storage unit 1605.

  A vertex 2002 is a vertex that exists at the same depth as the vertex 2006 from the root vertex 2001 and has descendant vertices. In this case, the partial reference information formed by the vertex 2002 as a root vertex and its descendant vertices, that is, the partial reference information having a rooted tree structure formed by the vertex 2002, the vertex 2003, the vertex 2004, and the vertex 2005. As an argument, the template candidate detecting means 1602 is recursively called. As a result, the obtained template candidates are the first template candidate 2009 and the second template candidate 2010 shown in FIGS.

  In this embodiment, template candidates are extracted based on the leaf vertex at the shallowest position counted from the root vertex of the cross-reference relationship data 2008. However, the first leaf vertex encountered in the depth-first search is used as the reference. Alternatively, partial reference information of a minimum tree structure including all vertices that are shallower than or at the same depth as the leaf vertex may be detected as a template candidate. Further, the depth counted from the root vertex may be set in advance, and the partial reference information of the minimum tree structure including all the vertices at the set depth may be used as the template candidate.

  Further, the vertex at the deepest position in the tree structure of the cross-reference relationship data 2008 is used as a reference vertex, and partial reference information of the minimum tree structure including all the vertices at the same depth as the vertex is detected as a template candidate. May be. In this case, the partial reference information detected as the template candidate is represented only by the root vertex of the partial reference information, and the descendant vertex of the root vertex is not set as the reference vertex search range. Also, the depth (set depth) counted from the vertex at the deepest position is set in advance, and it exists from the vertex existing at the deepest position to the vertex existing within the set depth A partial reference information group of a rooted tree structure formed by vertices may be used as a template candidate.

Embodiment of Data Compression System FIG. 20 is a system configuration diagram of a data compression system 2601 according to an embodiment of the present invention. The data compression system 2601 includes a template generation device 2602 and a data compression device 2603, and performs data compression by inputting first input data 2605 and second input data 2604. Then, the data compression system 2601 outputs the template-applied cross-reference relationship data replaced by the template as first output data 2606, and the vertex group having a plurality of vertices separated from the input cross-reference relationship data. The data is output as second output data 2607.

The template generation device 2602 receives, as second input data 2604, cross-reference relationship data having a plurality of vertices each having a type and a value, and reference information in which the reference relationship between the vertices has a rooted tree structure. The second input data 2604 is input for generating a template used for compression in the data compression device 2603.
Further, the template generation device 2602 uses the partial reference information of the rooted tree structure included in the reference information between the vertices of the second input data as the appearance frequency and the template when the partial reference information is set as a pattern. The partial reference information corresponding to the compression effect is output as a template. That is, the template generation device 2602 outputs partial reference information having a high appearance frequency and a high compression effect as a template.

  Then, for example, when the data structure 21 shown in FIG. 2B is input and compressed, the cross-reference relationship data 2008 separated from the data structure 21 by the separating means 107 is used as the second input data 2604. The data is input to the template generation device 2602. Here, if the partial reference information having a high appearance frequency and a high compression effect is the second template candidate 2010 shown in FIG. 19, the template generation device 2602 outputs the second template candidate 2010 as a template. The output template is stored in the template storage unit 102 of the data compression device 2603. This template is commonly used when other cross-reference relationship data is input.

  The data compression device 2603 inputs data having the same structure as the second input data 2604 as the first input data 2605. The data compression device 2603 has the same configuration as the data compression device 101 shown in FIG. That is, the data compression apparatus 2603 includes a separating unit 107, a template accumulating unit 102, a matching location detecting unit 103, a template replacing unit 111, and a switching unit 104. The template storage unit 102 stores templates output from the template generation device 2602. The coincidence part detecting means 103 detects only the coincidence part including the leaf vertex of the tree structure from the input cross-reference relationship data, and performs re-replacement to replace the template.

  The matching part detection unit 103 detects a matching part corresponding to the template stored in the template storage unit 102 among the vertices included in the cross-reference relationship data 2008 shown in FIG. At this time, if the matching part detection unit 103 detects only the matching part of the template including the leaf vertex, it detects partial reference information formed by the vertex 2002, the vertex 2003, the vertex 2004, and the vertex 2005 as the matching part.

  The template replacement unit 111 applies the template to the detected matching part. At this time, the template is expressed as a new vertex, and the cross-reference relationship data to which the template has been applied is output. When the output is selected by the switching means 104, the cross-reference relationship data to which the template has been applied is output as the first output data 2606. When re-compression is selected by the switching unit 104, the re-compression is performed, so that the cross-reference relationship data to which the template has been applied is recursively input to the matching location detection unit 103, and the template is applied again. The first output data 2606 is like the cross-reference relationship data 1401 shown in FIG.

  By configuring the data compression system 2601 in this way, a template suitable for compression of given input data can be dynamically generated, and the input data can be compressed using the generated template. Therefore, the data compression system 2601 can realize data compression with a high compression effect.

It is a block diagram which shows the structure of the data compression apparatus which concerns on embodiment of this invention. (A) is a figure which shows an example of an XML document, (b) is a figure which shows the data structure of the XML document of (a). It is a figure which shows the cross-reference relationship data before compression. It is a figure which shows the table of the set of vertices. (A) is a block diagram showing the configuration of the template, (b) is a block diagram showing the configuration of the template entity. It is a figure which shows the cross reference relationship data compressed using the template shown in FIG.5 (b). It is a figure which shows the cross-reference relationship data further compressed using the template shown in FIG.5 (b). It is a figure which shows another cross-reference relationship data further compressed using the template shown in FIG.5 (b). It is a block diagram which shows the structure of the data restoration apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the template production | generation apparatus which concerns on embodiment of this invention. (A) is a block diagram showing the configuration of matching location information, (b) is a block diagram showing the configuration of matching location information corresponding to FIG. 6, (c) is a block diagram showing the configuration of another matching location information It is. It is a figure which shows an example of a matching location detection procedure. It is a figure which shows an example of the replacement procedure. It is a figure which shows an example of a decompression | restoration procedure. It is a block diagram which shows the structure of another data compression apparatus. It is a flowchart which shows an example of the operation | movement procedure of a template candidate detection means. It is a figure which shows another cross-reference relationship data before compression. It is a figure which shows the structure of a 1st template candidate. It is a figure which shows the structure of a 2nd template candidate. It is a system configuration figure of a data compression system. It is a figure which shows another XML document. It is a figure which shows another XML document. It is a figure which shows the cross reference relationship data isolate | separated from the XML document of FIG. It is a figure which shows the cross reference relationship data isolate | separated from the XML document of FIG. It is a flowchart which shows the 1st procedure of template production | generation. It is a flowchart which shows the 2nd procedure of template production | generation. It is a flowchart which shows the 3rd procedure of template production | generation. It is a block diagram which shows the structure of a template candidate. It is a figure which shows the internal structure of a template candidate storage means. It is a figure which shows another internal structure of a template candidate storage means. It is a figure which shows another internal structure of a template candidate storage means. It is a figure which shows another example of an XML document. It is a figure which shows the data structure of the XML document of FIG. (A) is a figure which shows the cross-reference relationship data isolate | separated from the XML document of FIG. 33, (b) is a figure which shows the table | surface of the vertex set. It is a figure which shows the data structure separated from the XML document of FIG. It is a figure which shows the element name information isolate | separated from the XML document of FIG. It is a figure which shows the text information isolate | separated from the XML document of FIG.

Explanation of symbols

20, 30, 40 ... XML document 21, 31 ... Data structure 101, 2104, 2603 ... Data compression device 102, 1502 ... Template storage means 103, 2107 ... Matching location detection means 107, 2106 ... Separation means 108, 2105 ... Input data 109, 2110, 2606 ... first output data 110, 2111, 2607 ... second output data 111, 1603, 2108 ... template replacement means 1000, 1201 ... cross-reference relation data 1301, 1401, 2008 ... cross-reference relation data 1001 , 1002, 1003, 1004 ... vertex 1005, 1006, 1007 ... vertex 900 ... table 1105 ... template 1109, 1202, 1302 ... template entity 1501 ... data restoration device, 1504 ... synthesis means 150 ... Template expansion means 1506, 2605 ... First input data 1507, 2604 ... Second input data 1508 ... Output data 1601, 2602 ... Template generation device 1602 ... Template candidate detection means 1604 ... Input data acquisition means 1605 ... Template candidate accumulation Means 1606 ... Template selection means 1607 ... Input data group, 1608 ... Output data 2009 ... First template candidate 2010 ... Second template candidate 2101 ... First compression means 2109 ... Template storage means 2601 ... Data compression system 2900, 3000 ... Cross-reference relationship data 3401 ... Template candidates

Claims (5)

Input data having a plurality of vertices each having a type and a value, and reference information between the vertices, cross-reference relationship data having reference information between the vertices, and a vertex comprising a plurality of vertices having the type and value Separating means for separating into groups and outputting data of the separated vertex groups;
A template storage means for storing a template representing a specific partial pattern of the reference information between previous SL vertices,
A template matching point detecting unit for detecting a matching point corresponding to the template stored in the template storing unit from the cross-reference relationship data separated by the separating unit;
Among the cross-reference relationship data separated by the separation unit, when replacing the matching part detected by the template matching part detection unit with the template, the indicated vertex indicating the replacement part by the template is matched. provided at a position, a portion other than the matching portion of the cross reference data, the data comprising said instructions with the vertex, and a template replacing means to enable re-replacement by the template,
A data compression apparatus comprising: output means for outputting cross-reference relationship data repeatedly replaced by the template replacement means to a storage device in a state where reference information between the vertices can be restored by the template . When the input data has a rooted tree structure, the template matching position detecting means is at the deepest position of the rooted tree structure when detecting a matching position corresponding to the template stored in the template storing means. 2. The data compression apparatus according to claim 1, wherein only a coincidence portion including a leaf vertex which is a vertex is detected. A template storage means for storing a template representing a specific partial pattern of the reference information between the vertices of multiple,
The compressed cross-reference relationship data having reference information between the vertices and replaced by the template is input, the compressed cross-reference relationship data is expanded using the template, and the expanded mutual reference When the designated vertex is included in the reference relationship data, the expanded cross-reference relationship data is repeatedly expanded using the template, and the cross-reference before compression is performed from the compressed cross-reference relationship data. Deployment means to restore the relevant data;
Input data of the vertex group composed of the plurality of vertices, synthesize the data of the vertex group and the cross-reference relationship data before compression restored by the expansion means, and output the synthesized data And a data recovery device. The expansion means performs the re-expansion using the template necessary for referring to the vertex to be referred to when the vertex to be expanded using the template is specified in the cross-reference relationship data. 4. The data restoration apparatus according to claim 3, wherein A data compression system for compressing first input data having a plurality of vertices each having a type and a value, and reference information between the vertices,
The cross-reference relation data having the reference information between the vertices extracted from the second input data having the same structure as the first input data is input, and a specific partial pattern of the reference information between the vertices is input. and lutein plates generator to generate a template representing,
The data compression apparatus according to claim 1, further comprising a template storage unit that stores the template generated by the template generation apparatus.
In a state where the reference information between the vertices can be restored by the template , the cross-reference relation data repeatedly replaced by the data compression device using the template stored in the template storage means is output as first output data. And a data compression system comprising: output means for outputting, as second output data, data of a plurality of vertices each having the type and value separated from the first input data.

Images