CN107562790A - A kind of method and system for realizing data processing batch storage - Google Patents

A kind of method and system for realizing data processing batch storage Download PDF

Info

Publication number
CN107562790A
CN107562790A CN201710641234.8A CN201710641234A CN107562790A CN 107562790 A CN107562790 A CN 107562790A CN 201710641234 A CN201710641234 A CN 201710641234A CN 107562790 A CN107562790 A CN 107562790A
Authority
CN
China
Prior art keywords
data
computer
sql scripts
storage
interface form
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710641234.8A
Other languages
Chinese (zh)
Other versions
CN107562790B (en
Inventor
拜山峰
高林武
毕永东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing VRV Software Corp Ltd
Original Assignee
Beijing VRV Software Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing VRV Software Corp Ltd filed Critical Beijing VRV Software Corp Ltd
Priority to CN201710641234.8A priority Critical patent/CN107562790B/en
Publication of CN107562790A publication Critical patent/CN107562790A/en
Application granted granted Critical
Publication of CN107562790B publication Critical patent/CN107562790B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a kind of method for realizing data processing batch storage, this method comprises the following steps:Log-on data processing service, SQL scripts then parsed and stored in internal memory;According to the SQL scripts after interface form Command Line Parsing;Data in multiple requests are sorted out according to interface form;The data of classification are pressed into queue;It will be matched with the SQL scripts postponed with data;And by the data with same interface form with being spliced with the SQL scripts postponed.The method of the present invention solves the too high situation of service end concurrent request number, makes the handling capacity of service end processing data increase, and solves service end processing concurrent data abnormal interruption and the unsafe problem of data processing.

Description

A kind of method and system for realizing data processing batch storage
Technical field
The present invention relates to database purchase field, more specifically, particularly relate to it is a kind of realize data processing batch storage Method and system.
Background technology
In Modern Information system, generally required substantial amounts of data are centrally stored, and counted according to actual business requirement It is exactly a kind of application system that can realize above-mentioned purpose according to storehouse system.Database be according to data structure come tissue, storage and Manage warehouse of the foundation of data on computer memory device.It is the file cabinet that can be considered electronization in itself in simple terms --- The place of e-file is stored, user, which can be increased newly to the data in file, intercept, update, delete etc., to be operated.
In the prior art, the collection of data and the write into Databasce generally use method of synchronization, it is this in database aspect The problem of realizing the mode of data syn-chronization and shortcoming are:
1. if in the case of high concurrent, read-write inconsistent (reading fast, that writes is slow) can cause loss of data;
After the completion of 2. synchronous insertion data are, it is necessary to handle, interface could return, and system low-response, handling capacity is low, concurrently It is low.
The content of the invention
In view of this, the purpose of the embodiment of the present invention is to propose a kind of method for realizing data processing batch storage and is System, the problem of service end interface concurrent request number is too high is can solve the problem that, makes the handling capacity of service end processing data increase, and energy Enough solve service end processing concurrent data abnormal interruption and the unsafe problem of data processing.
Based on above-mentioned purpose, the one side of the embodiment of the present invention provides a kind of side for realizing data processing batch storage Method, it the described method comprises the following steps:
Log-on data processing service, SQL scripts then parsed and stored in internal memory;
According to the SQL scripts after interface form Command Line Parsing;
Data in multiple requests are sorted out according to the interface form;
The data of classification are pressed into queue;
It will be matched with the SQL scripts postponed with the data;And
By the data with same interface form with being spliced with the SQL scripts postponed.
In some embodiments, the SQL scripts are parsed using asynchronous thread.
In some embodiments, the SQL scripts are resolved into SqlElement objects and by the SqlElement Object is stored into internal memory.
In some embodiments, what each interface form corresponded to a kind of business datum reports structure.
In some embodiments, parsing is polled to the data for being pressed into queue, and according to the interface form It will be matched with the SQL scripts postponed with the data.
In some embodiments, the data with same interface form are spliced into a SQL statement storage.
The another aspect of the embodiment of the present invention, additionally provide a kind of system for realizing data processing batch storage, the system System performs the above method.
The another aspect of the embodiment of the present invention, additionally provide a kind of computer equipment, including memory, at least one processing Device and the computer program that can be run on the memory and on the processor is stored in, described in the computing device The above method is performed during program.
The another aspect of the embodiment of the present invention, a kind of computer-readable recording medium is additionally provided, it is described computer-readable Storage medium is stored with computer program, and the computer program performs the above method when being executed by processor.
The another aspect of the embodiment of the present invention, additionally provide a kind of computer program product, the computer program product Including the calculation procedure being stored on computer-readable recording medium, the calculation procedure includes instruction, when the instruction is counted When calculation machine performs, the computer is set to perform the above method.
The present invention has following advantageous effects:The side provided in an embodiment of the present invention for realizing data processing batch storage Method and system, by using the technical side that batch data is spliced into a SQL statement storage in a manner of a kind of batch processing Case, it can realize that batch data is put in storage in the case where high concurrent is asked, make the handling capacity of service end processing data increase, and It can solve the problem that service end processing concurrent data abnormal interruption and the unsafe problem of data processing.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet according to the method for one embodiment of the invention;
Fig. 2 is the schematic flow sheet according to the high concurrent asynchronous process data loading of one embodiment of the invention;
Fig. 3 is the system configurations figure according to one embodiment of the invention;
Fig. 4 is the hardware configuration signal of one embodiment of the computer equipment of execution methods described provided by the invention Figure.
Embodiment
For the object, technical solutions and advantages of the present invention are more clearly understood, below in conjunction with specific embodiment, and reference Accompanying drawing, the embodiment of the present invention is further described.
It should be noted that all statements for using " first " and " second " are for differentiation two in the embodiment of the present invention The non-equal entity of individual same names or non-equal parameter, it is seen that " first " and " second " only for statement convenience, no It is interpreted as the restriction to the embodiment of the present invention, subsequent embodiment no longer illustrates one by one to this.
Based on above-mentioned purpose, the one side of the embodiment of the present invention, it is proposed that realize the side of data processing batch storage One embodiment of method, the problem of service end interface concurrent request number is too high is can solve the problem that, makes handling up for service end processing data Amount increase, and can solve the problem that service end processing concurrent data abnormal interruption and the unsafe problem of data processing.Shown in Fig. 1 It is the schematic flow sheet of method according to an embodiment of the invention.
Described method may include following steps:
Step S101, log-on data processing service, SQL scripts then parsed and stored in internal memory.Need what is illustrated It is, when handling the every kind of business of client, to first have to ensure that service end data processing service normally starts, then could Reported data, the SQL scripts corresponding to every a kind of product could be parsed, and with the storage of SqlElement objects in Deposit.Wherein it is preferred to SQL scripts are parsed using asynchronous thread.The structure of SQL scripts is as follows:
Step S102, according to the SQL scripts after interface form Command Line Parsing, it is necessary to explanation, each interface form A kind of corresponding business datum reports structure.Every a kind of interface is entered by configuring the SQL script corresponding with interface form Row is sorted out, and is configured inside SQL scripts and interface passes ginseng and unanimously, and must have the service logic of such interface.Interface form is such as Shown in following table:
Step S103, the data in multiple requests are sorted out, it is necessary to which explanation, concurrent in service end according to interface form In the case that number of request is higher, service end can be serviced by TCP and UDP services forward requests to processing at that time concurrently Task less data handles service end.
Step S104, the data of classification are pressed into queue, wherein it is preferred to, after the data check that client is reported, lead to Cross LinkedBlockingQueue<String[]>It is pressed into queue, pending data and poll during SQL scripts match.
Step S105, it will be matched with the SQL scripts postponed with data, wherein it is preferred to, the data to being pressed into queue Be polled parsing, and according to the MaxCode of agreement (such as:00FF0A00) and MinCode (such as:00016004) will match somebody with somebody The SQL scripts postponed are matched with request data, after matching, by existing SqlElement objects in internal memory, are started pair Sql sentences are spliced.
Step S106, by the data with same interface form with being spliced with the SQL scripts postponed.By reported data Interface form and SQL scripts match after, start to be spliced data and SQL scripts.The SQL statement spliced is, for example, insert into tableName (column1,column2...)values(value1,value2...),(value3, value4...)。
From above-described embodiment as can be seen that the method provided in an embodiment of the present invention for realizing data processing batch storage, pin The problem of conventional each framework synchronization process data loading, is optimized, by using in a manner of a kind of batch processing by number According to the technical scheme for being spliced into a SQL statement storage in batches, it can realize that batch data enters in the case where high concurrent is asked Storehouse, makes the handling capacity of service end processing data increase, and can solve the problem that service end processing concurrent data abnormal interruption and data Handle unsafe problem.
As shown in Fig. 2 the schematic flow sheet for the high concurrent asynchronous process data loading according to one embodiment of the invention.
First, service end receives the correctness of client request data and verification data, is then back to client-side information.Will High concurrent request data, uses LinkedBlockingQueue<String[]>It is pressed into queue, it is preferable that the inside of queue Structure is MaxCode, MinCode and the JSON data specifically reported, contains the verification of the JSON forms of reported data here, Verification passes through, then asynchronous return client data check results.
Then, Event nodes corresponding to generation, are added in SQL scripts, as follows:
Next, starting the data of snoop queue, and splice SQL statement.Detection is pressed into the data of queue, passes through after poll MaxCode+MinCode matches corresponding SqlElement objects, by srcData key and value and SqlElement couples The key and value of elephant are contrasted respectively, if identical, the field is spliced into insertion sentence will insert field correspondingly Position, such as:Insert into tableName (k1, k2, k3 ...) values (v1, v2, v3 ...), (v2-1, v2-2, v3-3…).So far, asynchronous process client request data, completed to perfection splicing SQL sentences, perform SQL and can reach simultaneously Hair slide enters effect.
Based on above-mentioned purpose, second aspect of the embodiment of the present invention, it is proposed that realize data processing batch storage is One embodiment of system, the problem of service end interface concurrent request number is too high is can solve the problem that, makes handling up for service end processing data Amount increase, and can solve the problem that service end processing concurrent data abnormal interruption and the unsafe problem of data processing.The system is held The above-mentioned method of row.
The system provided in an embodiment of the present invention for realizing data processing batch storage, for conventional each framework synchronization process The problem of data loading, is optimized, and batch data is spliced into a SQL language by using in a manner of a kind of batch processing The technical scheme of sentence storage, it can realize that batch data is put in storage in the case where high concurrent is asked, make service end processing data Handling capacity increase, and can solve the problem that service end processing concurrent data abnormal interruption and the unsafe problem of data processing.
It is important to note that the embodiment of said system employs the embodiment of the above method to illustrate each mould The course of work of block, those skilled in the art can be it is readily conceivable that other by these module applications to methods described be implemented In example.
As shown in figure 3, it is the system configurations figure according to one embodiment of the invention.
The exemplary architecture of the system of the present invention can be divided into primary service area and agency service region.Wherein primary service area It can include:Authorization service, address service, configuration service, buffer service etc..Agency service region mainly include one-level service, Forwarding service and secondary service.One-level service can include service upgrade service, O&M monitoring service etc..Forwarding service can wrap Include TCP forwardings service and UDP forwarding services.Secondary service can include strategy distribution service, data processing service, client liter Level service, patch service, block the 28 class services such as service, scan service, point-to-point services.Wherein, the work of data processing service With the service for being verified and being put in storage processing primarily directed to client reported data.In the higher situation of service end number of concurrent Under, service end can forward service and UDP to forward service to forward requests to the number that processing concurrent tasks are less at that time by TCP According to processing service end.
Based on above-mentioned purpose, a kind of the 3rd aspect of the embodiment of the present invention, it is proposed that computer for performing methods described One embodiment of equipment.
The computer equipment includes memory, at least one processor and storage on a memory and can be in processor The computer program of upper operation, any one above-mentioned method is performed during computing device program.
As shown in figure 4, the hardware knot of one embodiment for the computer equipment of execution methods described provided by the invention Structure schematic diagram.
By taking computer equipment as shown in Figure 4 as an example, include a processor 401 and one in the computer equipment Memory 402, and can also include:Input unit 403 and output device 404.
Processor 401, memory 402, input unit 403 and output device 404 can pass through bus or other modes Connect, in Fig. 4 exemplified by being connected by bus.
Memory 402 is used as a kind of non-volatile computer readable storage medium storing program for executing, available for storage non-volatile software journey Sequence, non-volatile computer executable program and module, as corresponding to the methods described in the embodiment of the present application programmed instruction/ Module.Processor 401 is stored in non-volatile software program, instruction and module in memory 402 by operation, so as to hold The various function application of row server and data processing, that is, that realizes above method embodiment realizes that data processing batch is put in storage Method.
Memory 402 can include storing program area and storage data field, wherein, storing program area can store operation system Application program required for system, at least one function;Storage data field can store is according to realize data processing batch storage System uses created data etc..In addition, memory 402 can include high-speed random access memory, can also include non- Volatile memory, for example, at least a disk memory, flush memory device or other non-volatile solid state memory parts. In some embodiments, memory 402 is optional including relative to the remotely located memory of processor 401, these remote memories Network connection to local module can be passed through.The example of above-mentioned network includes but is not limited to internet, intranet, local Net, mobile radio communication and combinations thereof.
Input unit 403 can receive the numeral or character information of input, and produce with the user of the system set with And the key signals input that function control is relevant.Output device 404 may include the display devices such as display screen.
Programmed instruction/module corresponding to methods described is stored in the memory 402, is held when by the processor 401 During row, the method in above-mentioned any means embodiment is performed.
Any one embodiment of the computer equipment, corresponding foregoing any means embodiment phase can be reached Same or similar effect.
Based on above-mentioned purpose, the 4th aspect of the embodiment of the present invention, it is proposed that a kind of computer-readable recording medium, institute Stating computer-readable recording medium storage has computer executable instructions, and the computer executable instructions can perform above-mentioned any side Method in method embodiment is with realizing the system in above-mentioned any system embodiment.The implementation of the computer-readable recording medium Example, can reach the corresponding foregoing any means effect identical or similar with system embodiment.
Based on above-mentioned purpose, the 5th aspect of the embodiment of the present invention, it is proposed that a kind of computer program product, the calculating Machine program product includes the calculation procedure being stored on computer-readable recording medium, and the calculation procedure includes instruction, when this refers to When order is computer-executed, the computer is performed the method in above-mentioned any means embodiment and realize that above-mentioned any system is real Apply the system in example.The embodiment of the computer program product, corresponding foregoing any means and system can be reached The identical or similar effect of embodiment.
Finally it should be noted that one of ordinary skill in the art will appreciate that realizing the whole in above-described embodiment method Or part flow, related hardware can be instructed to complete by computer program, described program can be stored in a computer In read/write memory medium, the program is upon execution, it may include such as the flow of the embodiment of above-mentioned each method.Wherein, it is described Storage medium can be magnetic disc, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access Memory, RAM) etc..The embodiment of the computer program, corresponding foregoing can be reached The identical or similar effect of embodiment of the method for anticipating.
In addition, typically, it can be various electric terminal equipments that the embodiment of the present invention, which discloses described system, equipment etc., example Such as mobile phone, personal digital assistant (PDA), tablet personal computer (PAD), intelligent television or large-scale terminal device, such as service Device etc., therefore protection domain disclosed in the embodiment of the present invention should not limit as certain certain types of system, equipment.
In addition, disclosed method is also implemented as the computer program performed by CPU according to embodiments of the present invention, should Computer program can store in a computer-readable storage medium.When the computer program is performed by CPU, the present invention is performed The above-mentioned function of being limited in method disclosed in embodiment.
In addition, above method step and system unit can also utilize controller and make it that controller is real for storing The computer-readable recording medium of the computer program of existing above-mentioned steps or Elementary Function is realized.
In addition, it should be appreciated that computer-readable recording medium (for example, memory) as described herein can be volatile Property memory or nonvolatile memory, or both volatile memory and nonvolatile memory can be included.As example Sub and nonrestrictive, nonvolatile memory can include read-only storage (ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory.Volatile memory can include depositing at random Access to memory (RAM), the RAM can serve as external cache.Nonrestrictive as an example, RAM can be with Diversified forms obtain, such as synchronous random access memory (DRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate SDRAM (DDR SDRAM), enhancing SDRAM (ESDRAM), synchronization link DRAM (SLDRAM) and directly Rambus RAM (DRRAM).The storage device of disclosed aspect is intended to the memory of including but not limited to these and other suitable type.
Those skilled in the art will also understand is that, the various illustrative logical blocks with reference to described by disclosure herein, mould Block, circuit and algorithm steps may be implemented as the combination of electronic hardware, computer software or both.It is hard in order to clearly demonstrate This interchangeability of part and software, enters with regard to the function of various exemplary components, square, module, circuit and step to it General description is gone.This function is implemented as software and is also implemented as hardware depending on concrete application and application Design constraint to whole system.Those skilled in the art can realize described in a variety of ways for every kind of concrete application Function, but this realize that decision should not be interpreted as causing a departure from scope of disclosure of the embodiment of the present invention.
Various illustrative logical blocks, module and circuit with reference to described by disclosure herein, which can utilize, to be designed to The following part of function described here is performed to realize or perform:General processor, digital signal processor (DSP), special collection Into circuit (ASIC), field programmable gate array (FPGA) or other PLDs, discrete gate or transistor logic, divide Any combinations of vertical nextport hardware component NextPort or these parts.General processor can be microprocessor, but alternatively, processing Device can be any conventional processors, controller, microcontroller or state machine.Processor can also be implemented as computing device Combination, for example, the combination of DSP and microprocessor, multi-microprocessor, one or more microprocessors combination DSP and/or any Other this configurations.
The step of method or algorithm with reference to described by disclosure herein, can be directly contained in hardware, be held by processor In capable software module or in combination of the two.Software module may reside within RAM memory, flash memory, ROM storages Device, eprom memory, eeprom memory, register, hard disk, removable disk, CD-ROM or known in the art it is any its In the storage medium of its form.Exemplary storage medium is coupled to processor so that processor can be from the storage medium Middle reading information writes information to the storage medium.In an alternative, the storage medium can be with processor collection Into together.Processor and storage medium may reside within ASIC.ASIC may reside within user terminal.In a replacement In scheme, processor and storage medium can be used as discrete assembly resident in the user terminal.
In one or more exemplary designs, the function can be real in hardware, software, firmware or its any combination It is existing.If realized in software, can be stored in using the function as one or more instruction or code computer-readable Transmitted on medium or by computer-readable medium.Computer-readable medium includes computer-readable storage medium and communication media, The communication media includes helping for computer program to be sent to any medium of another position from a position.Storage medium It can be any usable medium that can be accessed by a general purpose or special purpose computer.It is nonrestrictive as an example, the computer Computer-readable recording medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc memory apparatus, disk storage equipment or other magnetic Property storage device, or can be used for carry or storage form for instruction or data structure required program code and can Any other medium accessed by universal or special computer or universal or special processor.In addition, any connection can It is properly termed as computer-readable medium.If for example, use coaxial cable, optical fiber cable, twisted-pair feeder, digital subscriber line (DSL) or such as wireless technology of infrared ray, radio and microwave to send software from website, server or other remote sources, Then above-mentioned coaxial cable, optical fiber cable, twisted-pair feeder, DSL or such as wireless technology of infrared ray, radio and microwave are included in The definition of medium.As used herein, disk and CD include compact disk (CD), laser disk, CD, digital versatile disc (DVD), floppy disk, Blu-ray disc, wherein disk generally magnetically reproduce data, and CD reproduce data using laser optics.On The combination for stating content should also be as being included in the range of computer-readable medium.
Above is exemplary embodiment disclosed by the invention, it should be noted that in the sheet limited without departing substantially from claim On the premise of inventive embodiments scope of disclosure, it may be many modifications and change.According to open embodiment described herein The function of claim to a method, step and/or action be not required to perform with any particular order.In addition, although the present invention is implemented Element disclosed in example can be described or required in the form of individual, but be odd number unless explicitly limited, it is understood that be multiple.
It should be appreciated that it is used in the present context, unless context clearly supports exception, singulative " one It is individual " (" a ", " an ", " the ") be intended to also include plural form.It is to be further understood that "and/or" used herein is Referring to includes any of one or more than one project listed in association and is possible to combine.
The embodiments of the present invention disclose that embodiment sequence number is for illustration only, do not represent the quality of embodiment.
One of ordinary skill in the art will appreciate that hardware can be passed through by realizing all or part of step of above-described embodiment To complete, by program the hardware of correlation can also be instructed to complete, described program can be stored in a kind of computer-readable In storage medium, storage medium mentioned above can be read-only storage, disk or CD etc..
Those of ordinary skills in the art should understand that:The discussion of any of the above embodiment is exemplary only, not It is intended to imply that scope of disclosure of the embodiment of the present invention (including claim) is limited to these examples;In the think of of the embodiment of the present invention It under road, can also be combined, and exist as described above between the technical characteristic in above example or different embodiments Many other changes of the different aspect of the embodiment of the present invention, for simplicity, they are not provided in details.Therefore, it is all at this Spiritual and any omission within principle, made, modification, equivalent substitution, improvement of inventive embodiments etc., should be included in this hair Within the protection domain of bright embodiment.

Claims (10)

  1. A kind of 1. method for realizing data processing batch storage, it is characterised in that the described method comprises the following steps:
    Log-on data processing service, SQL scripts then parsed and stored in internal memory;
    According to the SQL scripts after interface form Command Line Parsing;
    Data in multiple requests are sorted out according to the interface form;
    The data of classification are pressed into queue;
    It will be matched with the SQL scripts postponed with the data;And
    By the data with same interface form with being spliced with the SQL scripts postponed.
  2. 2. according to the method for claim 1, it is characterised in that parsed the SQL scripts using asynchronous thread.
  3. 3. according to the method for claim 1, it is characterised in that the SQL scripts are resolved into SqlElement objects simultaneously By SqlElement objects storage into internal memory.
  4. 4. according to the method for claim 1, it is characterised in that each interface form corresponds to a kind of reporting for business datum Structure.
  5. 5. according to the method for claim 1, it is characterised in that parsing is polled to the data for being pressed into queue, and It will be matched according to the interface form with the SQL scripts postponed with the data.
  6. 6. according to the method for claim 1, it is characterised in that the data with same interface form are spliced into one Bar SQL statement is put in storage.
  7. 7. a kind of system for realizing data processing batch storage, it is characterised in that perform such as any one institute in claim 1-6 The method stated.
  8. 8. a kind of computer equipment, including memory, at least one processor and it is stored on the memory and can be in institute State the computer program run on processor, it is characterised in that such as claim is performed during the computing device described program Method in 1-6 described in any one.
  9. 9. a kind of computer-readable recording medium, the computer-readable recording medium storage has computer program, and its feature exists In method of the execution as described in any one in claim 1-6 when the computer program is executed by processor.
  10. 10. a kind of computer program product, it is characterised in that the computer program product includes being stored in computer-readable deposit Calculation procedure on storage media, the calculation procedure include instruction, when the instruction is computer-executed, make the computer Perform the method as described in any one in claim 1-6.
CN201710641234.8A 2017-07-31 2017-07-31 Method and system for realizing batch warehousing of data processing Active CN107562790B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710641234.8A CN107562790B (en) 2017-07-31 2017-07-31 Method and system for realizing batch warehousing of data processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710641234.8A CN107562790B (en) 2017-07-31 2017-07-31 Method and system for realizing batch warehousing of data processing

Publications (2)

Publication Number Publication Date
CN107562790A true CN107562790A (en) 2018-01-09
CN107562790B CN107562790B (en) 2020-05-01

Family

ID=60974720

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710641234.8A Active CN107562790B (en) 2017-07-31 2017-07-31 Method and system for realizing batch warehousing of data processing

Country Status (1)

Country Link
CN (1) CN107562790B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110580185A (en) * 2018-06-07 2019-12-17 中兴通讯股份有限公司 Data preprocessing method, device and storage medium
CN111831379A (en) * 2020-07-15 2020-10-27 中车大连机车车辆有限公司 Online dynamic data batch processing and displaying method and vehicle-mounted display screen device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7882144B1 (en) * 2003-06-02 2011-02-01 The Board Of Trustees Of The Leland Stanford Jr. University Computer systems and methods for the query and visualization of multidimensional databases
CN102945256A (en) * 2012-10-18 2013-02-27 福建省海峡信息技术有限公司 Method and device for merging and classifying massive SQL (Structured Query Language) sentences
CN103116595A (en) * 2012-10-24 2013-05-22 中国电力科学研究院 Realizing method of supervisory control and data acquisition (SCADA) history data distribution-type storage facing power grid
CN103412972A (en) * 2013-07-01 2013-11-27 北京理工大学 Method for packaging and integrating simulation models of distributed simulation platform of aircraft
CN104158757A (en) * 2014-08-21 2014-11-19 福建星海通信科技有限公司 GPS (Global Positioning System) data loading and processing method and system
CN105205122A (en) * 2015-09-11 2015-12-30 烽火通信科技股份有限公司 Telecommunication service retrieval method based on data synchronization
CN105740344A (en) * 2016-01-25 2016-07-06 中国科学院计算技术研究所 Sql statement combination method and system independent of database
CN105930511A (en) * 2016-05-12 2016-09-07 深圳市傲天科技股份有限公司 Answer big data subscription platform
CN106294423A (en) * 2015-05-25 2017-01-04 阿里巴巴集团控股有限公司 Data base divides wiring method and the device of table

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7882144B1 (en) * 2003-06-02 2011-02-01 The Board Of Trustees Of The Leland Stanford Jr. University Computer systems and methods for the query and visualization of multidimensional databases
CN102945256A (en) * 2012-10-18 2013-02-27 福建省海峡信息技术有限公司 Method and device for merging and classifying massive SQL (Structured Query Language) sentences
CN103116595A (en) * 2012-10-24 2013-05-22 中国电力科学研究院 Realizing method of supervisory control and data acquisition (SCADA) history data distribution-type storage facing power grid
CN103412972A (en) * 2013-07-01 2013-11-27 北京理工大学 Method for packaging and integrating simulation models of distributed simulation platform of aircraft
CN104158757A (en) * 2014-08-21 2014-11-19 福建星海通信科技有限公司 GPS (Global Positioning System) data loading and processing method and system
CN106294423A (en) * 2015-05-25 2017-01-04 阿里巴巴集团控股有限公司 Data base divides wiring method and the device of table
CN105205122A (en) * 2015-09-11 2015-12-30 烽火通信科技股份有限公司 Telecommunication service retrieval method based on data synchronization
CN105740344A (en) * 2016-01-25 2016-07-06 中国科学院计算技术研究所 Sql statement combination method and system independent of database
CN105930511A (en) * 2016-05-12 2016-09-07 深圳市傲天科技股份有限公司 Answer big data subscription platform

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张锐: "基于SOA的企业管理信息系统的设计", 《中国优秀硕士学位论文全文数据库信息科技辑》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110580185A (en) * 2018-06-07 2019-12-17 中兴通讯股份有限公司 Data preprocessing method, device and storage medium
CN110580185B (en) * 2018-06-07 2023-10-31 中兴通讯股份有限公司 Data preprocessing method, device and storage medium
CN111831379A (en) * 2020-07-15 2020-10-27 中车大连机车车辆有限公司 Online dynamic data batch processing and displaying method and vehicle-mounted display screen device
CN111831379B (en) * 2020-07-15 2023-06-30 中车大连机车车辆有限公司 Online dynamic data batch processing and displaying method and vehicle-mounted display screen device

Also Published As

Publication number Publication date
CN107562790B (en) 2020-05-01

Similar Documents

Publication Publication Date Title
US11704665B2 (en) Method and system for offline data transfer via machine-readable code
US11444777B2 (en) Method and system for providing validated, auditable, and immutable inputs to a smart contract
US11121870B2 (en) Method and system for interacting public and private blockchains with controlled participation
US10572361B2 (en) Concurrent production use of a production enterprise system and testing of a modified enterprise system
CN107423387A (en) The method and apparatus of table is divided in a kind of point storehouse based on middleware
US20210049715A1 (en) Blockchain-based data procesing method, apparatus, and electronic device
US11115804B2 (en) Subscription to dependencies in smart contracts
CN104270395A (en) Method, device and system for checking input data
CN106210032A (en) The method and device reported based on terminal data batch
CN107329735A (en) A kind of intranet patch update method and device
CN107330028A (en) Expansion application methods and system of a kind of Apache NiFi in terms of source data input database
CN114372006A (en) Method and system for automatically generating test case, computer equipment and storage medium
CN105684027A (en) House resource verifying method and system for real estate network
CN107562790A (en) A kind of method and system for realizing data processing batch storage
CN103984633B (en) A kind of bank main passes down the automatization test system of operation
US20240078324A1 (en) Fast access to a data resource update in a blockchain network
CN108399076A (en) A kind of firmware update and device based on UEFI
US11488145B2 (en) Method and system for user-based distributed ledgers
CN110442433A (en) A kind of method, equipment and storage medium executing SQL task
CN110633146A (en) Method, equipment and readable medium for multithread operation of IPMI command
US20230131813A1 (en) Method and system for authorization and settlement in blockchain transactions
WO2024007690A1 (en) Blockchain-based transaction processing method, apparatus and device, and computer-readable storage medium and computer program product
KR20240100350A (en) Method and system for pruning blocks from a blockchain for data retention and storage scalability purposes
US20190115095A1 (en) Methods for more accurately managing processing of medical bill data and devices thereof
CN115269660A (en) Cache data processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant