CN103544262A - XML-based stream page release method and system - Google Patents

Abstract

The invention discloses an XML-based stream page release method and system. The method includes the steps of 1, streaming an XML input document that meets predetermined segmenting conditions, namely segmenting and reconfiguring, and selectively performing streaming again; 2, quickly paging the XML input document that meets predetermined partitioning conditions, namely multiple binary tree partitioning and reconfiguring; 3, according to a conversion pattern table provided by a terminal device, converting the input document into documents of other standard formats to output; 4, transmitting the documents of different standard formats to the corresponding terminal devices. The system comprises a streaming unit, a fast paging unit, an XSLT converter and a release server, wherein the streaming unit comprises a segmenting device and a reconfiguring device, and the fast paging unit comprises a partitioning device and a reconfiguring device. The method and system is applicable to super-large XML documents, with improved conversion reliability and fault tolerance, and is well flexible and widely applicable.

Description

An XML-based streaming pagination publishing method and system

technical field

The invention relates to an XML-based paging publishing method and system.

Background technique

With the rapid development of information technology, more and more enterprises and institutions need to operate massive data, such as medical data of hospitals, traffic data of traffic bureaus, power data of power bureaus, planning data of planning bureaus, Hydrological and water conservancy data, meteorological data of the Meteorological Bureau, these data are often stored in the server in the form of XML, and users only need to access the documents on the server to obtain the data. However, when users access documents on the server through different terminal devices such as PCs, handheld devices, and smart phones, due to the differences in terminal display formats and software system storage and reading formats, it is necessary to correctly receive and display data. A format conversion must be performed on the document on the server. At present, XML document format conversion tools mainly include: DOM, SAX and XSLT, among which, XSLT, as one of the most popular XML document format conversion technologies, is very powerful, but its working principle is relatively simple, as shown in Figure 1.

Because in the conversion process, the XML source document first needs to be parsed into a DOM tree and stored in the memory, if the document is too large, it will inevitably cause memory overflow. Therefore, when users use terminal devices such as PCs, handheld devices, and smart phones to read large data, they often cannot receive and display the data correctly due to insufficient memory or too small a display screen.

And because the traditional pagination process only realizes the function of the segmentation processor, that is, iteratively segmenting the input document, so all the small XML documents obtained are not "well-formed", making the next conversion The operation is not relatively independent, and the reliability and fault tolerance are also poor. In addition, the iterative processing method also greatly reduces the speed of segmented processing.

Contents of the invention

In order to overcome the shortcomings of the existing XML-based paging publishing method and system that cannot be applied to the occasions where the XML document is too large, and the conversion reliability, fault tolerance, flexibility, and applicability are high, the present invention provides a An XML-based streaming pagination publishing method and system for occasions where XML documents are too large and require high conversion reliability, fault tolerance, flexibility, and applicability.

The technical solution adopted by the present invention to solve its technical problems is:

An XML-based streaming paging publishing method, the publishing method comprising the following steps:

(1) Fluidization process:

For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then proceed to step (2) for processing ; Conversely, if the file size exceeds the preset segment read threshold, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, and will generate two well-formed XML document, one size is equal to T _m , and the other size is equal to T _s -T _m , the former will be sent to step (2) for processing, and the latter will be sent to the streaming processor for judgment, segmentation and reassessment structural processing;

(2) Fast paging process:

If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 ＞>T, the XML document F _s0,1 will be divided and reconstructed in the first round to generate two "Well-formed" new XML documents F _s1,1 and F _s1,2 ; next, the two newly generated documents F _s1,1 and F _s1,2 are judged and the second round of segmentation and reconstruction processing is performed, That is, if the two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then the two documents should be segmented and Refactoring process, generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeat judgment, segmentation and reconstruction until a certain round of segmentation The size of all generated XML documents does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends;

(3) XSLT conversion process: compare the conversion style sheet provided by the terminal device, convert the input document into a document output in other standard formats;

(4) Publishing process: Send documents with different standard formats to corresponding terminal devices.

Further, in the step (1), the stream processing process includes a segmentation processing process and a reconstruction processing process, and the segmentation processing process:

Assume that there is an XML document F _s with a size of T _s , and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is far greater than the maximum available memory of the streaming processor, that is, T _s >>T _m , or in other words, satisfy the condition: T _s ≈ pT _m , p>>1, then use the segmenter in the fluidization processor to segment it, which specifically includes the following three steps:

First, read the XML document F _s ;

Second, set the segmentation reading threshold T _d =T _m ;

Third, segment processing to generate two non-"well-formed" XML documents:

①F _s1 , the size is recorded as T _s1 , T _s1 =T _d =T _m ;

②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m .

Furthermore, the reconstruction process includes two steps of initial reconstruction and further reconstruction, the process is as follows:

1.1) Read the XML document F _s1 generated in the third step;

1.2) Position the pointer to the end;

1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ ;

1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ , there are two possibilities at this time:

If the end tag ">" can be found, then the value of _L2 is the position value of the tag;

Conversely, if the end tag ">" cannot be found, then the pointer should be positioned at L ₁ , and step 1.3) should be performed again, and after obtaining the new value of L ₁ , perform step 1.4) again to obtain a new L ₂ value, this new L ₂ value is the real position of the end mark in this case;

1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 ; after that, the XML document F _s1 with incomplete data deleted and the XML document F _s2 with incomplete data added will be obtained;

1.6) Get the label names of all ancestor nodes missing due to the split:

1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False;

1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list;

1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that are missing due to splitting, put these label names into another list, and when obtaining the label names of these nodes, keep their original order in the list unchanged;

1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document:

1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted;

1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added;

1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2).

Further, in the step (2), suppose there is an XML document F _s0,1 whose size is T _s0,1 , and the required memory of the terminal device is T. If the XML document is very large, far larger than the terminal device Require memory, that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q＞＞1, and perform the first round of segmentation and reconstruction processing on this XML document, the process is as follows:

2.1) Read the XML document F _s0,1 ;

2.2) Set a segmentation threshold

2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f0,1 ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ;

2.4) Refactor the two XML documents F _s1,1 and F _s1,2 generated in step 2.3), and after processing, two new "well-formed" XML documents will be obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ;

和

对步骤2.4）生成的两个XML文档F_s1,1和F_s1,2进行第二轮的分割和重构处理，处理后将得到四个“形式良好”的文档F_s2,1、F_s2,2、F_s2,3、F_s2,4，大小分别等于T_f1,1、T_f1,1、T_f1,2、T_f1,2，依此类推，设定2^n-1个分割阈值

对2^n-1个XML文档F_s(n-1),k,k=1,…,2^n-1进行第n轮的分割和重构处理，处理后将得到2ⁿ个“形式良好”的XML文档F_sn,k,k=1,…,2ⁿ，至此，所有文档的大小都不超过终端设备的需求内存，即T_sn,k≤T,k=1,…,2ⁿ，不再满足分割条件，分割和重构处理结束。At this point, the first round of segmentation and reconstruction processing is over, and then we will set two segmentation thresholds

and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,1 and F _s1,2 generated in step 2.4). After processing, four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 , the size is equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 respectively, and so on, set 2 ^n-1 segmentation thresholds

Perform the nth round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(n-1),k ,k=1,…,2 ^n-1 , and get 2 ⁿ “well-formed” documents after processing XML document F _sn,k ,k=1,…,2 ⁿ , so far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,…,2 ⁿ , not Then the segmentation condition is satisfied, and the segmentation and reconstruction process ends.

An XML-based streaming paging publishing system, the publishing system includes:

Streaming processor: For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then the This document is processed by the fast pager; on the contrary, if the document size exceeds the preset threshold for segmented reading, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, processing Two well-formed XML documents will be generated, one with size T _m and the other with size T _s -T _m , the former will be sent to the fast pager for processing, and the latter will be sent to the streaming processor for another ground judgment, segmentation and reconstruction processing;

Fast pager: If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 >>T, then perform the first round of segmentation and reconstruction processing on the XML document F _s0,1 , generate two _" well-formed" new XML documents F _s1,1 and F _s1,2 ; then judge and second round of _segmentation and Refactoring processing, that is, if two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then these two documents should be processed at the same time The document is divided and reconstructed to generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeatedly judging, dividing and reconstructing until The size of all XML documents generated by a certain round of segmentation does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends;

XSLT Converter: It is used to convert the input document into document output in other standard formats according to the conversion style sheet provided by the terminal device;

Publisher: used to send documents with different standard formats to corresponding terminal devices.

Further, the streaming processor includes a segmenter and a reconstructor, wherein,

In the segmenter, it is assumed that there is an XML document F _s with a size of T _s , and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is far greater than the maximum available memory of the streaming processor , that is, T _s ＞＞T _m , or in other words, satisfy the condition: T _s ≈pT _m , p＞＞1, then use the segmenter in the fluidization processor to segment it, specifically including the following three steps :

First, read the XML document F _s ;

Second, set the segmentation reading threshold T _d =T _m ;

Third, segment processing to generate two non-"well-formed" XML documents:

①F _s1 , the size is recorded as T _s1 , T _s1 =T _d ＝T _m ;

②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m .

Further, in the reconstructor, the processing process includes two steps of preliminary reconstruction and re-construction, and the process is as follows:

1.1) Read the XML document F _s1 generated in the third step;

1.2) Position the pointer to the end;

1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ ;

1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ , there are two possibilities at this time:

If the end tag ">" can be found, then the value of _L2 is the position value of the tag;

Conversely, if the end tag ">" cannot be found, then the pointer should be positioned at L ₁ , and step 1.3) should be performed again, and after obtaining the new value of L ₁ , perform step 1.4) again to obtain a new L ₂ value, this new L ₂ value is the real position of the end mark in this case;

1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 ; after that, the XML document F _s1 with incomplete data deleted and the XML document F _s2 with incomplete data added will be obtained;

1.6) Get the label names of all ancestor nodes missing due to the split:

1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False;

1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list;

1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that are missing due to splitting, put these label names into another list, and when obtaining the label names of these nodes, keep their original order in the list unchanged;

1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document:

1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted;

1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added;

1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2).

Still further, the fast pager includes a splitter and a reconstructor, wherein,

In the splitter, it is assumed that there is an XML document F _s0,1 whose size is T _s0,1 and the required memory of the terminal device is T. If the XML document is very large, it is far larger than the required memory of the terminal device, that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q＞＞1, perform the first round of segmentation and reconstruction processing on this XML document, the process is as follows:

2.1) Read the XML document F _s0,1 ;

2.2) Set a segmentation threshold

2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f0,1 ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ;

2.4) The two XML documents F _s1,1 and F _s1,2 generated in step 2.3) are reconstructed in the reconstructor, and two new "well-formed" XML documents will be obtained after processing:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ;

对步骤2.4）生成的两个XML文档F_s1,2和F_s1,2进行第二轮的分割和重构处理，处理后将得到四个“形式良好”的文档F_s2,1、F_s2,2、F_s2,3、F_s2,4，大小分别等于T_f1,1、T_f1,1、T_f1,2、T_f1,2，依此类推，设定2^n-1个分割阈值

对2^n-1个XML文档F_s(m-1),k,k=1,…,2^n-1进行第n轮的分割和重构处理，处理后将得到2ⁿ个“形式良好”的XML文档F_sn,k，k=1,…,2ⁿ，至此，所有文档的大小都不超过终端设备的需求内存，即T_sn,k≤T,k=1,…,2ⁿ，不再满足分割条件，分割和重构处理结束。At this point, the first round of segmentation and reconstruction processing is over, and then we will set two segmentation thresholds and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,2 and F _s1,2 generated in step 2.4). After processing, four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 , the size is equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 respectively, and so on, set 2 ^n-1 segmentation thresholds

Carry out the n-th round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(m-1),k ,k=1,…,2 ^n-1 , and get 2 ⁿ "well-formed" documents after processing XML document F _sn,k , k=1,…,2 ⁿ , so far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,…,2 ⁿ , not Then the segmentation condition is satisfied, and the segmentation and reconstruction process ends.

The technical idea of the present invention is: use the streaming processor and the fast pager to divide this large XML document into multiple small XML documents that do not exceed the memory limit of the terminal device, and then complete the conversion work through the XSLT converter. After the above analysis and research, we can draw a process diagram of reading big data in pages, as shown in Figure 2.

As can be seen from the figure above, this process is completed under the joint action of the XML file server, the streaming paging server and the publishing server, where the XML file server is used to store and send heterogeneous XML documents; the streaming paging server is responsible for completing Segmentation, reconstruction and conversion of XML documents; the purpose of the publishing server is to send small XML documents in corresponding standard formats to terminal devices such as PCs, handheld devices, and smart phones, as long as these terminal devices request the XSLT converter to exchange data, And provide its own conversion style sheet.

The beneficial effects of the present invention are mainly manifested in: (1) The system uses the streaming paging server to paginate the data, so that the data can be sent to the terminal device in the form of "page", which solves the problem of , Display size and specifications and other limitations make it impossible to obtain or display data correctly;

(2) The system can meet the requirements of multiple users, that is, multiple different terminals or multiple terminals of the same type with different format standards, to request access to XML documents on the file server at the same time, with high flexibility and wide application.

(3) The system can use the streaming paging server to convert XML documents of any size into a format and size that can be received and displayed by terminal devices. That is, the system can be applied to XML documents of any size and has universal applicability.

(4) Compared with the traditional paging publishing system that only realizes the function of the segment processor, the system not only adds a reconstructor, but makes the output XML documents "well-formed", improving the reliability of parsing and conversion and fault tolerance; a fast pager is also added to make paging processing faster and more efficient.

On this basis, this system adds a reconstructor, the purpose is to convert all these non-"well-formed" small XML documents into "well-formed" small XML documents, so that the next step of parsing and conversion is possible;

This system also adds a fast pager, which uses a binary tree-based segmentation algorithm, which can quickly segment and reconstruct all the above-mentioned "well-formed" small XML documents, and generally speed up the paging process. That is, the expected paging effect can be achieved more quickly.

(5) All XML documents output by any link in the system, including streaming processor, fast pager and XSLT converter, are "well-formed" and can be parsed and converted independently. Among them, the processing or transmission error of any one document will not affect the correct parsing and format conversion of other documents, that is, the error will be effectively isolated in the corresponding error document, and will not spread to other correct documents; Error correction can be combined with other correct documents to form a complete document, which effectively improves the fault tolerance of parsing and format conversion.

(6) Since all XML documents received by end-user devices are also "well-formed", it can quickly split, parse and assemble these documents.

Description of drawings

Figure 1 is a schematic diagram of XSLT transformation.

Figure 2 is a diagram of the paged reading process of big data.

Fig. 3 is a block diagram of the streaming pagination publishing system.

Fig. 4 is a block diagram of the streaming processor.

Fig. 5 is a block diagram of the fast pager.

Figure 6 is a schematic diagram of streaming processing, in which (a) indicates the situation when the document size does not exceed the preset segmented reading threshold, and (b) indicates the situation when the document size exceeds the preset segmented reading threshold .

Fig. 7 is a flowchart of the reconstruction processing procedure of F _s1 and F _s2 .

FIG. 8 is a schematic diagram of a fast paging algorithm based on a binary tree, wherein each ellipse node represents an XML document used in a corresponding round of processing.

FIG. 9 is a flow chart of the fast paging process based on the binary tree.

Fig. 10 is a flow chart of fast paging processing.

Fig. 11 is a flowchart of the first round of segmentation and reconstruction processing.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

Example 1

Referring to Figures 1 to 11, an XML-based streaming paging publishing method, the publishing method includes the following steps:

(1) Fluidization process:

For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then proceed to step (2) for processing ; Conversely, if the file size exceeds the preset segment read threshold, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, and will generate two well-formed XML document, one size is equal to T _m , and the other size is equal to T _s -T _m , the former will be sent to step (2) for processing, and the latter will be sent to the streaming processor for judgment, segmentation and reassessment structural processing;

(2) Fast paging process:

If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 ＞>T, the XML document F _s0,1 will be divided and reconstructed in the first round to generate two "Well-formed" new XML documents F _s1,1 and F _s1,1 ; next, the two newly generated documents F _s1,1 and F _s1,2 are judged and the second round of segmentation and reconstruction processing is performed, That is, if the two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then the two documents should be segmented and Refactoring process, generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeat judgment, segmentation and reconstruction until a certain round of segmentation The size of all generated XML documents does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends;

(3) XSLT conversion process: compare the conversion style sheet provided by the terminal device, convert the input document into a document output in other standard formats;

(4) Publishing process: Send documents with different standard formats to corresponding terminal devices.

Further, in the step (1), the stream processing process includes a segmentation processing process and a reconstruction processing process, and the segmentation processing process:

Assume that there is an XML document F _s with a size of T _s , and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is far greater than the maximum available memory of the streaming processor, that is, T _s >>T _m , or in other words, satisfy the condition: T _s ≈ pT _m , p>>1, then use the segmenter in the fluidization processor to segment it, which specifically includes the following three steps:

First, read the XML document F _s ;

Second, set the segmentation reading threshold T _d =T _m ;

Third, segment processing to generate two non-"well-formed" XML documents:

①F _s1 , the size is recorded as T _s1 , T _s1 =T _d =T _m ;

②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m .

Furthermore, the reconstruction process includes two steps of initial reconstruction and further reconstruction, the process is as follows:

1.1) Read the XML document F _s1 generated in the third step;

1.2) Position the pointer to the end;

1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ ;

1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ , there are two possibilities at this time:

If the end tag ">" can be found, then the value of _L2 is the position value of the tag;

Conversely, if the end tag ">" cannot be found, then the pointer should be positioned at L ₁ , and step 1.3) should be performed again, and after obtaining the new value of L ₁ , perform step 1.4) again to obtain a new L ₂ value, this new L ₂ value is the real position of the end mark in this case;

1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 ; after that, the XML document F _s1 with incomplete data deleted and the XML document F _s2 with incomplete data added will be obtained;

1.6) Get the label names of all ancestor nodes missing due to the split:

1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False;

1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list;

1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that are missing due to splitting, put these label names into another list, and when obtaining the label names of these nodes, keep their original order in the list unchanged;

1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document:

1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted;

1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added;

1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2).

Further, in the step (2), suppose there is an XML document F _s0,1 whose size is T _s0,1 , and the required memory of the terminal device is T. If the XML document is very large, far larger than the terminal device Require memory, that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q＞＞1, and perform the first round of segmentation and reconstruction processing on this XML document, the process is as follows:

2.1) Read the XML document F _s0,1 ;

2.2) Set a segmentation threshold

2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f0,1 ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ;

2.4) Refactor the two XML documents F _s1,1 and F _s1,2 generated in step 2.3), and after processing, two new "well-formed" XML documents will be obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ;

和

对步骤2.4）生成的两个XML文档F_s1,1和F_s1,2进行第二轮的分割和重构处理，处理后将得到四个“形式良好”的文档F_s2,1、F_s2,2、F_s2,3、F_s2,4，大小分别等于T_f1,1、T_f1,1、T_f1,2、T_f1,2，依此类推，设定2^n-1个分割阈值

对2^n-1个XML文档F_s(n-1),k,k=1,…,2^n-1进行第n轮的分割和重构处理，处理后将得到2ⁿ个“形式良好”的XML文档F_sn,k,k=1,…,2ⁿ，至此，所有文档的大小都不超过终端设备的需求内存，即T_sn,k≤T,k=1,…,2ⁿ，不再满足分割条件，分割和重构处理结束。At this point, the first round of segmentation and reconstruction processing is over, and then we will set two segmentation thresholds

and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,1 and F _s1,2 generated in step 2.4). After processing, four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 , the size is equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 respectively, and so on, set 2 ^n-1 segmentation thresholds

Perform the nth round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(n-1),k ,k=1,…,2 ^n-1 , and get 2 ⁿ “well-formed” documents after processing XML document F _sn,k ,k=1,…,2 ⁿ , so far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,…,2 ⁿ , not Then the segmentation condition is satisfied, and the segmentation and reconstruction process ends.

Example 2

Referring to Figures 1 to 11, an XML-based streaming paging publishing system, the publishing system includes:

Streaming processor: For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then the This document is processed by the fast pager; on the contrary, if the document size exceeds the preset threshold for segmented reading, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, processing Two well-formed XML documents will be generated, one with size T _m and the other with size T _s -T _m , the former will be sent to the fast pager for processing, and the latter will be sent to the streaming processor for another ground judgment, segmentation and reconstruction processing;

Fast pager: If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 >>T, then perform the first round of segmentation and reconstruction processing on the XML document F _s0,1 , generate two _" well-formed" new XML documents F _s1,1 and F _s1,2 ; then judge and second round of _segmentation and Refactoring processing, that is, if two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then these two documents should be processed at the same time The document is divided and reconstructed to generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeatedly judging, dividing and reconstructing until The size of all XML documents generated by a certain round of segmentation does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends;

XSLT Converter: It is used to convert the input document into document output in other standard formats according to the conversion style sheet provided by the terminal device;

Publisher for sending documents with different standard formats to corresponding terminal devices.

In this embodiment, the streaming paging publishing system is composed of a streaming paging server, including a streaming processor, a quick pager, an XSLT converter, and a publishing server, as shown in Figure 3. The streaming paging server splits, reconstructs and transforms large XML documents, and generates multiple "well-formed" small XML target documents. The size and data format of these documents depend on the available memory provided by the terminal device and the conversion style sheet, respectively. , and then send these small XML documents to the corresponding terminal devices in the form of "pages" through the publishing server.

The streaming processor is composed of a segmenter and a reconstructor, as shown in Figure 4. The basic idea of the work is to first judge the size of the input XML document. If the document does not meet the preset segmentation conditions, it will not Do any processing and send it to the fast pager; on the contrary, if the document meets the preset segmentation conditions, the document is read in segments to ensure that the size of the read data does not exceed the preset segment read Take the threshold value, and then use the reconstructor to reconstruct the new XML document storing the above data, so that each XML document output by the streaming processor is in good form.

The fast pager is composed of a splitter and a reconstructor, as shown in Figure 5. The basic idea of the work is to first judge the size of the input XML document, and if the document meets the preset splitting conditions, it will perform a binary tree analysis on the document. Then, all the new XML documents generated by this splitting process are reconstructed. It is important to emphasize that the refactoring process here is the same as in the streaming processor.

The function of the XSLT converter is to convert the input document into a document output in other standard formats according to the conversion style sheet provided by the terminal device; the function of the publishing server is to send the documents with different standard formats to the corresponding terminal device.

The stream processing process is implemented by using a stream processor, and for each large XML input document, the stream processor must first judge its size. If the size of the document does not exceed the preset segment read threshold, that is, T _s ≤ T _m , the streaming processor will output the document to the fast pager without any processing, as shown in FIG. 6 a . Conversely, if the document size exceeds the preset segment read threshold, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, and two well-formed XMLs will be generated after processing Documents, one with a size approximately equal to T _m , and the other with a size approximately equal to T _s -T _m , the former will be sent to the fast pager, while the latter will be sent to the streaming processor for judgment, segmentation and reconstruction again processing, as shown in Figure 6b.

Segmented processing: Assume that there is an XML document F _s with a size of T _s and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is much larger than the maximum available memory of the streaming processor, that is T _s ＞＞T _m , or in other words, satisfy the condition: T _s ≈pT _m , p＞＞1, then we will use the segmenter in the stream processor to segment it, including the following three steps :

First, read the XML document F _s .

Second, set the segmentation reading threshold T _d =T _m .

Third, segment processing to generate two non-"well-formed" XML documents:

①F _s1 , the size is recorded as T _s1 , T _s1 =T _d =T _m ;

②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m .

The refactoring process is relatively complicated, and it includes two steps of initial refactoring and re-refactoring. The implementation process is shown in Figure 7, and the specific implementation methods are as follows:

1.1) Read the XML document F _s1 generated in the third step.

1.2) Position the pointer to the tail.

1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ .

1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ . At this point there are two possibilities:

If the end tag ">" can be found, then the value of _L2 is the position value of the tag;

Conversely, if the end tag ">" cannot be found, the pointer should be positioned at L ₁ , and execute again

Execute step 1.3) to get the new L ₁ value, then perform step 1.4) to get the new L ₂ value, this new L ₂ value

is the real position of the closing tag in this case.

1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 . Thereafter, an XML document F _s1 with incomplete data deleted and an XML document F _s2 with incomplete data added will be obtained.

1.6) Get the label names of all ancestor nodes missing due to the split:

1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False;

1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list;

1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that were missing due to the split, put these label names into another list. It is worth noting that when obtaining the label names of these nodes, their original order in the list should be kept unchanged.

1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document:

1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted;

1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added;

1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2).

The fast paging process is realized by using the fast pager. Its core is the binary tree segmentation algorithm, a special parallel segmentation algorithm, which is faster and more efficient than the traditional iterative segmentation algorithm.

Fast paging algorithm based on binary tree: The basic principle of fast paging algorithm based on binary tree is to first split and reconstruct an XML document into two new documents, and then split and reconstruct the two new XML documents into two other documents, that is, a total of Four XML documents are obtained, and so on, when all obtained documents no longer satisfy the segmentation conditions, the segmentation ends. Below we will use a simple diagram to illustrate the basic principle of the above algorithm, as shown in Figure 8.

In order to further explain the basic principle of the binary tree-based fast paging algorithm, we can convert Figure 8 into the document change graph shown in Figure 9, and name the document F _sn,k , where the subscript n represents segmentation and reconstruction processing The number of rounds, such as n=1 indicates that the document is generated in the first round of segmentation and reconstruction processing; the subscript k represents the serial number of the document generated after each round of segmentation and reconstruction processing, and its value range depends on For round n, that is, k=1,...,2 ⁿ , for example, when n=1, the value of k is 1 and 2, and F _s1,1 represents the first one generated after the first round of segmentation and reconstruction document, F _s1,2 represents the second document generated after the first round of segmentation and reconstruction processing.

Through the research and analysis of Figure 9, we can draw the overall flow of fast paging processing, as shown in Figure 10. Since the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 >>T, we need to perform the first round of segmentation and reconstruction on it to generate two "well-formed "'s new XML documents F _s1,1 and F _s1,2 . Next, we need to judge the two newly generated documents F _s1,1 and F _s1,2 and perform the second round of segmentation and reconstruction processing, that is, if the two newly generated documents F _s1,1 and F _{s1, 2} still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then the two documents should be segmented and reconstructed at the same time, and four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, judge, split and reconstruct repeatedly, until the size of all XML documents generated by a round of split does not exceed the required memory of the terminal device, split and the refactoring process ends.

Binary tree splitting and reconstruction process: Suppose there is an XML document F _s0,1 whose size is T _s0,1 and the required memory of the terminal device is T. If the XML document is very large, it is far larger than the required memory of the terminal device , that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q＞＞1, then we will use the fast pager, including splitter and reconstructor, to process the XML document for the first time A round of segmentation and reconstruction processing, the process flow of which is shown in Figure 11, specifically includes the following four steps:

2.1) Read the XML document F _s0,1 ;

2.2) Set a segmentation threshold

2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f0,1 ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ;

2.4) Refactor the two XML documents F _s1,1 and F _s1,2 generated in step 2.3), and after processing, two new "well-formed" XML documents will be obtained:

①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ;

②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ;

和

对步骤4）生成的两个XML文档F_s1,1和F_s1,2进行第二轮的分割和重构处理，处理后将得到四个“形式良好”的文档F_s2,1、F_s2,2、F_s2,3、F_s2,4，大小分别约等于T_f1,1、T_f1,1、T_f1,2、T_f1,2。依此类推，设定2^n-1个分割阈值

对2^n-1个XML文档F_s(n-1),k,k=1,…,2^n-1进行第n轮的分割和重构处理，处理后将得到2ⁿ个“形式良好”的XML文档F_sn,k,k=1,…,2ⁿ。至此，所有文档的大小都不超过终端设备的需求内存，即T_sn,k≤T,k=1,…,2ⁿ，不再满足分割条件，分割和重构处理结束。At this point, the first round of segmentation and reconstruction processing ends. Next we will set two segmentation thresholds

and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,1 and F _s1,2 generated in step 4), and four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 are approximately equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 in size respectively. And so on, set 2 ^n-1 segmentation thresholds

Carry out the nth round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(n-1),k ,k=1,…,2 ^n-1 , after processing, 2 ⁿ "well-formed" documents will be obtained The XML document F _sn,k ,k=1,…,2 ⁿ . So far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,...,2 ⁿ , the segmentation condition is no longer satisfied, and the segmentation and reconstruction process ends.

Claims (8)

1. an XML-based streaming paging release method, characterized in that: the release method may further comprise the steps: (1) Fluidization process: For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then proceed to step (2) for processing ; Conversely, if the file size exceeds the preset segment read threshold, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, and will generate two well-formed XML document, one size is equal to T _m , and the other size is equal to T _s -T _m , the former will be sent to step (2) for processing, and the latter will be sent to the streaming processor for judgment, segmentation and reassessment structural processing; (2) Fast paging process: If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 ＞>T, the XML document F _s0,1 will be divided and reconstructed in the first round to generate two "Well-formed" new XML documents F _s1,1 and F _s1,2 ; next, the two newly generated documents F _s1,1 and F _s1,2 are judged and the second round of segmentation and reconstruction processing is performed, That is, if the two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then the two documents should be segmented and Refactoring process, generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeat judgment, segmentation and reconstruction until a certain round of segmentation The size of all generated XML documents does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends; (3) XSLT conversion process: compare the conversion style sheet provided by the terminal device, convert the input document into a document output in other standard formats; (4) Publishing process: Send documents with different standard formats to corresponding terminal devices. 2. An XML-based stream pagination publishing method according to claim 1, characterized in that: in the step (1), the stream processing process includes segment processing and reconstruction processing, and the segment Segment processing: Assume that there is an XML document F _s with a size of T _s , and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is far greater than the maximum available memory of the streaming processor, that is, T _s >>T _m , or in other words, satisfy the condition: T _s ≈ pT _m , p>>1, then use the segmenter in the fluidization processor to segment it, which specifically includes the following three steps: First, read the XML document F _s ; Second, set the segmentation reading threshold T _d =T _m ; Third, segment processing to generate two non-"well-formed" XML documents: ①F _s1 , the size is recorded as T _s1 , T _s1 =T _d =T _m ; ②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m . 3. a kind of XML-based streaming pagination publishing method as claimed in claim 2, is characterized in that: described refactoring process comprises two steps of primary refactoring and refactoring, and the process is as follows: 1.1) Read the XML document F _s1 generated in the third step; 1.2) Position the pointer to the end; 1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ ; 1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ , there are two possibilities at this time: If the end tag ">" can be found, then the value of _L2 is the position value of the tag; Conversely, if the end tag ">" cannot be found, then the pointer should be positioned at L ₁ , and step 1.3) should be performed again, and after obtaining the new value of L ₁ , perform step 1.4) again to obtain a new L ₂ value, this new L ₂ value is the real position of the end mark in this case; 1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 ; after that, the XML document F _s1 with incomplete data deleted and the XML document F _s2 with incomplete data added will be obtained; 1.6) Get the label names of all ancestor nodes missing due to the split: 1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False; 1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list; 1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that are missing due to splitting, put these label names into another list, and when obtaining the label names of these nodes, keep their original order in the list unchanged; 1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document: 1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted; 1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added; 1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2). 4. An XML-based streaming paging publishing method according to any one of claims 1 to 3, characterized in that: in the step (2), it is assumed that there is now an XML document F _s0,1 whose size is T _s0,1 , the required memory of the terminal device is T, if the XML document is very large, far larger than the required memory of the terminal device, that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q>>1, the first round of segmentation and reconstruction processing is performed on this XML document, and the process is as follows: 2.1) Read the XML document F _s0,1 ; 2.2) Set a segmentation threshold

2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained: ①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f0,1 ; ②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ; 2.4) Refactor the two XML documents F _s1,1 and F _s1,2 generated in step 2.3), and after processing, two new "well-formed" XML documents will be obtained: ①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ; ②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ; At this point, the first round of segmentation and reconstruction processing is over, and then we will set two segmentation thresholds

and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,1 and F _s1,2 generated in step 2.4). After processing, four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 , the size is equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 respectively, and so on, set 2 ^n-1 segmentation thresholds

Perform the nth round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(n-1),k ,k=1,…,2 ^n-1 , and get 2 ⁿ “well-formed” documents after processing XML document F _sn,k ,k=1,…,2 ⁿ , so far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,…,2 ⁿ , not Then the segmentation condition is satisfied, and the segmentation and reconstruction process ends. 5. An XML-based streaming paging release system, characterized in that: the release system includes: Streaming processor: For each large XML input document, the streaming processor first needs to judge its size. If the document size does not exceed the preset segment reading threshold, that is, T _s ≤ T _m , then the This document is processed by the fast pager; on the contrary, if the document size exceeds the preset threshold for segmented reading, that is, T _s >T _m , then the streaming processor will segment and reconstruct the document, processing Two well-formed XML documents will be generated, one with size T _m and the other with size T _s -T _m , the former will be sent to the fast pager for processing, and the latter will be sent to the streaming processor for another ground judgment, segmentation and reconstruction processing; Fast pager: If the size of the XML document F _s0,1 far exceeds the required memory T of the terminal device, that is, T _s0,1 >>T, then perform the first round of segmentation and reconstruction processing on the XML document F _s0,1 , generate two _" well-formed" new XML documents F _s1,1 and F _s1,2 ; then judge and second round of _segmentation and Refactoring processing, that is, if two newly generated documents F _s1,1 and F _s1,2 still meet the segmentation conditions: T _s1,1 ＞＞T and T _s1,2 ＞＞T, then these two documents should be processed at the same time The document is divided and reconstructed to generate four "well-formed" documents F _s2,1 , F _s2,2 , F _s2,3 and F _s2,4 , and so on, repeatedly judging, dividing and reconstructing until The size of all XML documents generated by a certain round of segmentation does not exceed the required memory of the terminal device, and the segmentation and reconstruction process ends; XSLT Converter: It is used to convert the input document into document output in other standard formats according to the conversion style sheet provided by the terminal device; Publisher: used to send documents with different standard formats to corresponding terminal devices. 6. The streaming paging publishing system based on XML as claimed in claim 5, characterized in that: the streaming processor includes a segmenter and a reconstructor, wherein, In the segmenter, it is assumed that there is an XML document F _s with a size of T _s , and the maximum available memory of the streaming processor is T _m . If the XML document is very large, it is far greater than the maximum available memory of the streaming processor , that is, T _s ＞＞T _m , or in other words, satisfy the condition: T _s ≈pT _m , p＞＞1, then use the segmenter in the fluidization processor to segment it, specifically including the following three steps : First, read the XML document F _s ; Second, set the segmentation reading threshold T _d =T _m ; Third, segment processing to generate two non-"well-formed" XML documents: ①F _s1 , the size is recorded as T _s1 , T _s1 =T _d =T _m ; ②F _s2 , the size is recorded as T _s2 , T _s2 =T _s -T _d =T _s -T _m . 7. A kind of XML-based streaming paging publishing system as claimed in claim 6, characterized in that: in the reconstructor, the processing process includes two steps of preliminary reconstruction and reconstruction again, and the process is as follows: 1.1) Read the XML document F _s1 generated in the third step; 1.2) Position the pointer to the end; 1.3) Search forward for the start tag "</" of the end tag, and record its position as L ₁ ; 1.4) Search backward from L ₁ for the corresponding end tag ">", and record its position as L ₂ , there are two possibilities at this time: If the end tag ">" can be found, then the value of _L2 is the position value of the tag; Conversely, if the end tag ">" cannot be found, then the pointer should be positioned at L ₁ , and step 1.3) should be performed again, and after obtaining the new value of L ₁ , perform step 1.4) again to obtain a new L ₂ value, this new L ₂ value is the real position of the end mark in this case; 1.5) Move the incomplete data caused by segmentation from the end of F _s1 to the head of F _s2 ; after that, the XML document F _s1 with incomplete data deleted and the XML document F _s2 with incomplete data added will be obtained; 1.6) Get the label names of all ancestor nodes missing due to the split: 1.6.1) Set the read flag flag=True, when the length of the read value is equal to or equal to 0, flag=False; 1.6.2) Read the XML document F _s1 with deleted incomplete data generated in step 1.5), and add the tag name of each node, except for the empty tag name, to the list; 1.6.3) Count the different elements and their numbers in the list. According to the principle that the start tag and end tag of a well-formed XML document should match, and empty tags should be closed, elements with an odd number, except the first element, That is, the label names of the ancestor nodes that are missing due to splitting, put these label names into another list, and when obtaining the label names of these nodes, keep their original order in the list unchanged; 1.7) Construct the two XML documents F _s1 and F _s2 generated in step 1.5) into a well-formed XML document: 1.7.1) Use the elements in the list obtained in step 1.6.3) as end tags and add them in reverse order to the end of the XML document F _s1 with incomplete data deleted; 1.7.2) Add the elements in the list obtained in step 1.6.3), except the first element, as start tags to the header of the XML document F _s2 to which incomplete data has been added; 1.7.3) Add the first element in the list obtained in step 1.6.3), that is, the declaration tag name, as a start tag to the header of the XML document F _s2 obtained in step 1.7.2). 8. An XML-based streaming paging publishing system according to any one of claims 5 to 7, wherein the fast pager includes a splitter and a reconstructor, wherein, In the splitter, it is assumed that there is an XML document F _s0,1 whose size is T _s0,1 and the required memory of the terminal device is T. If the XML document is very large, it is far larger than the required memory of the terminal device, that is, T _s0,1 ＞＞T, or in other words, satisfy the condition: T _s0,1 ≈qT, q＞＞1, perform the first round of segmentation and reconstruction processing on this XML document, the process is as follows: 2.1) Read the XML document F _s0,1 ; 2.2) Set a segmentation threshold 2.3) Carry out the first round of segmentation processing. This round includes a segmentation process. After segmentation, two non-"well-formed" XML documents are obtained: ①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 =T _f01, ; ②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 =T _f0,1 ; 2.4) The two XML documents F _s1,1 and F _s1,2 generated in step 2.3) are reconstructed in the reconstructor, and two new "well-formed" XML documents will be obtained after processing: ①F _s1,1 , the size is recorded as T _s1,1 , T _s1,1 ≈ _{T f0,1} ; ②F _s1,2 , the size is recorded as T _s1,2 , T _s1,2 ≈ _{T f0,1} ; At this point, the first round of segmentation and reconstruction processing is over, and then we will set two segmentation thresholds

and

Carry out the second round of segmentation and reconstruction processing on the two XML documents F _s1,1 and F _s1,2 generated in step 2.4). After processing, four "well-formed" documents F _s2,1 and F _{s2, 2} , F _s2,3 , F _s2,4 , the size is equal to T _f1,1 , T _f1,1 , T _f1,2 , T _f1,2 respectively, and so on, set 2 ^n-1 segmentation thresholds Perform the nth round of segmentation and reconstruction processing on 2 ^n-1 XML documents F _s(n-1),k ,k=1,…,2 ^n-1 , and get 2 ⁿ “well-formed” documents after processing XML document F _sn,k ,k=1,…,2 ⁿ , so far, the size of all documents does not exceed the required memory of the terminal device, that is, T _sn,k ≤T,k=1,…,2 ⁿ , not Then the segmentation condition is satisfied, and the segmentation and reconstruction process ends.

Images