Summary of the invention
In view of the above problems, the present invention is proposed to provide a kind of method and apparatus content of XML file being imported database overcoming the problems referred to above or solve the problem at least in part.
According to one aspect of the present invention, provide a kind of method content of XML file being imported database, comprising: adopt stream mode to be loaded in internal memory by the XML data in XML file; From the described XML data internal memory, obtain data and be saved in specific data table; The data store organisation of described specific data table is identical with specified database; By in specified database described in the data importing in described specific data table.
Alternatively, the XML data in XML file is loaded into internal memory and comprises further by described employing stream mode: derive the method performing stream mode based on XPathDocument class, adopts stream mode the XML data in XML file to be loaded in internal memory by the method.
Alternatively, described from the described XML data internal memory, obtain data and be saved in specific data table comprise: determine the set of node in the described XML data in internal memory; Described XML data in traversal internal memory, reads each node in described set of node, obtains data and be saved in described specific data table from each node.
Alternatively, described XML data in described traversal internal memory, read each node in described set of node, from each node, obtain data and be saved in described specific data table and comprise: the method deriving the parent attribute that can obtain present node based on XPathNavigator class, traveled through each node in described set of node by the method, obtain the property value of each node and be saved in described specific data table.
Alternatively, described specified database described in data importing in described specific data table to be comprised: call SqlBulkCopy class by specified database described in the data importing in described specific data table.
Alternatively, call SqlBulkCopy class described in specified database described in the data importing in described specific data table is comprised further: the configuration file reading Oracle client, obtains the routing information of Oracle.DataAccess.dll file; Call described Oracle.DataAccess.dll file according to described routing information, call SqlBulkCopy class by specified database described in the data importing in described specific data table according to this file.
According to another aspect of the present invention, provide a kind of device content of XML file being imported database, comprising: loading unit, for adopting stream mode, the XML data in XML file being loaded in internal memory; Data capture unit, for obtaining data and being saved in specific data table from the described XML data in internal memory; The data store organisation of described specific data table is identical with specified database; Data importing unit, for by specified database described in the data importing in described specific data table.
Alternatively, described loading unit, derives the method performing stream mode based on XPathDocument class, adopts stream mode the XML data in XML file to be loaded in internal memory by the method.
Alternatively, described data capture unit, for determining the set of node in the described XML data in internal memory; Described XML data in traversal internal memory, reads each node in described set of node, obtains data and be saved in described specific data table from each node.
Alternatively, described data capture unit, derive the method for the parent attribute that can obtain present node based on XPathNavigator class, traveled through each node in described set of node by the method, obtain the property value of each node and be saved in described specific data table.
Alternatively, described data importing unit, calls SqlBulkCopy class by specified database described in the data importing in described specific data table.
Alternatively, described data importing unit, is further used for the configuration file reading Oracle client, obtains the routing information of Oracle.DataAccess.dll file; Call described Oracle.DataAccess.dll file according to described routing information, call SqlBulkCopy class by specified database described in the data importing in described specific data table according to this file.
From the above, technical scheme of the present invention, adopt stream mode, XML data segmentation in XML file is loaded in internal memory, effectively prevent the situation causing Installed System Memory to overflow because the XML data in XML file is excessive, from the XML data internal memory, obtain data again and be saved in the specific data table identical with the storage organization of specified database, finally by specified database described in the data importing in described specific data table.This technical scheme not only improves the speed of XML file when reading, and there is the beneficial effect reading and stablize and not easily collapse, and when the data importing will got in XML file is to database, first be saved in the specific data table identical with the storage organization of specified database, thus can submit in batches in data to data storehouse, substantially reduce the running time of database, improve the importing efficiency of data.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
Fig. 1 shows a kind of according to an embodiment of the invention process flow diagram content of XML file being imported the method for database, and as shown in Figure 1, the method comprises:
Step S110, adopts stream mode the XML data in XML file to be loaded in internal memory.Wherein, adopt the segmentation of stream mode to read XML data in XML file, following step can be facilitated to read node in XML file one by one, effectively prevent the situation causing Installed System Memory to overflow because the XML data in XML file is excessive.
Step S120, obtains data and is saved in specific data table from the XML data internal memory; The data store organisation of specific data table is identical with specified database.
Wherein, can obtain the property value of each node data in XML data as the data needing to preserve, the tables of data of specifying can create as required according to the difference of type of database.
Step S130, by the data importing specified database in specific data table.
Visible, method shown in Fig. 1, adopt stream mode, XML data segmentation in XML file is loaded in internal memory, from the XML data internal memory, obtain data again and be saved in the specific data table identical with the storage organization of specified database, finally by the data importing specified database in specific data table.This technical scheme not only improves the speed of XML file when reading, and there is the beneficial effect reading and stablize and not easily collapse, and when the data importing will got in XML file is to database, first be saved in the specific data table identical with the storage organization of specified database, thus can submit in batches in data to data storehouse, substantially reduce the running time of database, improve the importing efficiency of data.
Particularly, in one embodiment of the invention, in method shown in Fig. 1, step S110 adopts stream mode that the XML data in XML file is loaded into internal memory to comprise further: derive the method performing stream mode based on XPathDocument class, adopt stream mode the XML data in XML file to be loaded in internal memory by the method.
.NET in platform, the class under System.Xml.XPath NameSpace can be used to carry out the inquiry based on path to XML document, the inquiry string constructing similar SQL is needed in query script, this character string follows XPath grammer, and wherein XPathDocument and XMLDocument can realize the inquiry to XML content.But in many cases, only need not need to modify to data by the data importing stored in XML file to database, whole process as bill of quantities XML imported electronic cutting assessment of bids database only need read XML file data, write operation can not be carried out to XML, therefore can improving performance with read-only XPathDocument replacement XMLDocument.XPathDocument class uses XPath data model to provide the quick read-only representation of XML document in internal memory, can quick position to the node in Xml or attribute.
But XPathDocument class can not realize stream mode reads XML data, this just needs to derive based on XPathDocument class the method performing stream mode, XML data in XML file is loaded in internal memory, with solve XML file larger time file reading system performance glide fast, the problem that the unstable even internal memory of system overflows.
In one embodiment of the invention, in the method shown in Fig. 1, from the XML data internal memory, obtain data and be saved in specific data table and comprise: determine the set of node in the XML data in internal memory; XML data in traversal internal memory, reads each node in set of node, obtains data and be saved in specific data table from each node.
Such as, in e-bidding field, for the XML file of bill of quantities, need that pricing norms is installed and select set of node, quotation information as total in unit, branch's subitem inventory etc.In other words, the data stored in XML file might not be all the data needing to import in database, can determine according to modes such as contents the data that need to import in database, and determine the node preserving these data further, finally obtain the set of node comprising all above-mentioned nodes.Thus can the XML data in internal memory be traveled through, read each node in set of node, from each node, obtain data and be saved in specific data table.
In one embodiment of the invention, in said method, XML data in traversal internal memory, read each node in set of node, from each node, obtain data and be saved in specific data table and comprise: the method deriving the parent attribute that can obtain present node based on XPathNavigator class, each node concentrated by the method traverse node, obtains the property value of each node and is saved in specific data table.
XPathNavigator class (under being positioned at System.Xml.XPath NameSpace equally) is very similar with the working method of XmlDocument class mentioned above, it to be all loaded into all information in internal memory and to allow to move in node, and wherein key difference is that its uses the mode based on vernier to allow to use the statement of similar MoveToNext () to move between XML data.But traditional XPathNavigator object can only move down and get nodal community, define a query structure in the present embodiment, present node parent attribute can be got.Such as, shown below is the code realizing the method in the present invention's specific embodiment:
The inquiry field that } // circulate is all, if run into the attribute of beginning, then directly obtains the value of this nodal community, is then displaced to next property value by the GetAttribute method of XPathNodeIterator; If run into the property value started with " ../", then ../get rid of, after treating the circulation of all properties inquiry phase, moved to the father node of this node by MoveToParent (), circulate with this, when waiting for that all property values all get p=Length, jump out circulation.
Can be found out by above-mentioned code, the method is based on XPathNavigator class, when obtaining nodal community, still the GetAttribute method of Selection and call XPathNodeIterator obtains the value of this nodal community, improvement is the father node being determined node by signature, and return back to father node by MoveToParent () statement and circulate, solve XPathNavigator object traditional in prior art can only move down and get the problem of nodal community, improve the reading efficiency of node.
In one embodiment of the invention, in the method shown in Fig. 1, the data importing specified database in specific data table is comprised: call SqlBulkCopy class by the data importing specified database in specific data table.
The BCP agreement that SqlBulkCopy class have employed SQLServer carries out the batch duplicating of data, and wherein BCP is the command-line tool being responsible for importing and exporting data in SQLServer, can import and export large batch of data efficiently in a parallel fashion.For avoiding the too much data of disposable operation to cause system pressure excessive, mode in batches can be adopted in the data importing specified database in specific data table.Through test, the present embodiment adopts the method calling SqlBulkCopy class than directly to exceed 25 times nearly by the efficiency of Sql statement data inserting.
For the system using oracle database, the Oracle.DataAccess.dll of the Oracle client that the server of web program deployment is installed is needed to depend on owing to calling SqlBulkCopy in Oracle, therefore in said method, call SqlBulkCopy class the data importing specified database in specific data table is comprised further: the configuration file reading Oracle client, obtain the routing information of Oracle.DataAccess.dll file; Call Oracle.DataAccess.dll file according to routing information, call SqlBulkCopy class by the data importing specified database in specific data table according to this file.The method, for the otherness of the Oracle client installation path in each web server, adopts and reads configuration file, and the method for Oracle.DataAccess.dll file is called in reflection, meets the adaptivity to various client call.
Fig. 2 shows a kind of according to an embodiment of the invention structural representation content of XML file being imported the device of database, as shown in Figure 2, is comprised by the device 200 that the content of expandable mark language XML file imports database:
Loading unit 210, is loaded into the XML data in XML file in internal memory for adopting stream mode.Wherein, adopt the segmentation of stream mode to read XML data in XML file, following step can be facilitated to read node in XML file one by one, effectively prevent the situation causing Installed System Memory to overflow because the XML data in XML file is excessive.
Data capture unit 220, for obtaining data and being saved in specific data table from the XML data in internal memory; The data store organisation of specific data table is identical with specified database.Wherein, can obtain the property value of each node data in XML data as the data needing to preserve, the tables of data of specifying can create as required according to the difference of type of database.
Data importing unit 230, for by the data importing specified database in specific data table.
Visible, device shown in Fig. 2, adopt stream mode, XML data segmentation in XML file is loaded in internal memory by loading unit 210, data capture unit 220 obtains data again and is saved in the specific data table identical with the storage organization of specified database from the XML data internal memory, and final data imports unit 230 by the data importing specified database in specific data table.This technical scheme not only improves the speed of XML file when reading, and there is the beneficial effect reading and stablize and not easily collapse, and when the data importing will got in XML file is to database, first be saved in the specific data table identical with the storage organization of specified database, thus can submit in batches in data to data storehouse, substantially reduce the running time of database, improve the importing efficiency of data.
In one embodiment of the invention, in the device shown in Fig. 2, loading unit 210, derives the method performing stream mode based on XPathDocument class, adopts stream mode the XML data in XML file to be loaded in internal memory by the method.
In one embodiment of the invention, in the device shown in Fig. 2, data capture unit 220, for determining the set of node in the XML data in internal memory; XML data in traversal internal memory, reads each node in set of node, obtains data and be saved in specific data table from each node.
In one embodiment of the invention, in said apparatus, data capture unit 220, the method of the parent attribute that can obtain present node is derived based on XPathNavigator class, each node concentrated by the method traverse node, obtains the property value of each node and is saved in specific data table.
In one embodiment of the invention, in the device shown in Fig. 2, data importing unit 230, calls SqlBulkCopy class by the data importing specified database in specific data table.
In one embodiment of the invention, in said apparatus, data importing unit 230, is further used for the configuration file reading Oracle client, obtains the routing information of Oracle.DataAccess.dll file; Call Oracle.DataAccess.dll file according to routing information, call SqlBulkCopy class by the data importing specified database in specific data table according to this file.
It should be noted that, the embodiment of above-mentioned each device embodiment is identical with the embodiment of aforementioned each corresponding method embodiment, does not repeat them here.
In sum, technical scheme of the present invention, adopt stream mode, XML data segmentation in XML file is loaded in internal memory, from the XML data internal memory, obtaining data again and be saved in the specific data table identical with the storage organization of specified database, finally adopting as called the mode of SqlBulkCopy class by the data importing specified database in specific data table.This technical scheme not only improves the speed of XML file when reading, and there is the beneficial effect reading and stablize and not easily collapse, and when the data importing will got in XML file is to database, first be saved in the specific data table identical with the storage organization of specified database, thus can submit in batches in data to data storehouse, substantially reduce the running time of database, improve the importing efficiency of data.
The foregoing is only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.All any amendments done within the spirit and principles in the present invention, equivalent replacement, improvement etc., be all included in protection scope of the present invention.
It should be noted that:
Intrinsic not relevant to any certain computer, virtual bench or miscellaneous equipment with display at this algorithm provided.Various fexible unit also can with use based on together with this teaching.According to description above, the structure constructed required by this kind of device is apparent.In addition, the present invention is not also for any certain programmed language.It should be understood that and various programming language can be utilized to realize content of the present invention described here, and the description done language-specific is above to disclose preferred forms of the present invention.
In instructions provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can use with arbitrary array mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions of the some or all parts content of expandable mark language XML file imported in the device of database that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the embodiment of the present invention.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.