CN110020359A - Apply the data processing method in webpage front-end, device and storage medium - Google Patents
Apply the data processing method in webpage front-end, device and storage medium Download PDFInfo
- Publication number
- CN110020359A CN110020359A CN201711088531.0A CN201711088531A CN110020359A CN 110020359 A CN110020359 A CN 110020359A CN 201711088531 A CN201711088531 A CN 201711088531A CN 110020359 A CN110020359 A CN 110020359A
- Authority
- CN
- China
- Prior art keywords
- data
- thread
- conversion
- data block
- data conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000006243 chemical reaction Methods 0.000 claims abstract description 247
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000012545 processing Methods 0.000 claims description 34
- 230000008859 change Effects 0.000 claims description 11
- 238000012163 sequencing technique Methods 0.000 claims description 10
- 230000001174 ascending effect Effects 0.000 claims description 9
- 238000004064 recycling Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 238000007373 indentation Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 25
- 230000006870 function Effects 0.000 description 30
- 239000012634 fragment Substances 0.000 description 7
- 241001269238 Data Species 0.000 description 5
- 238000003491 array Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 239000000872 buffer Substances 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013499 data model Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000006062 fragmentation reaction Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000007620 mathematical function Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
- G06F16/972—Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The data processing method in webpage front-end, device and storage medium are applied this application provides a kind of.Wherein, this method comprises: the source data got from background service is divided into multiple data blocks, wherein above-mentioned multiple data blocks are provided with the label for unlabeled data block sequence；Data conversion is carried out to above-mentioned multiple data blocks parallel, multiple data blocks after being converted；According to above-mentioned label, multiple data blocks after above-mentioned conversion are ranked up, target data is obtained, wherein above-mentioned target data is shown for front end view.By the application, data conversion can be rapidly completed, improve the efficiency of data conversion, solve mass data of the prior art webpage front-end serial process from background service, the problem that treatment effeciency is low, system performance is poor.
Description
Technical field
This application involves technical field of data processing, particularly, are related to a kind of apply in the data processing side of webpage front-end
Method, device and storage medium.
Background technique
Webpage front-end (i.e. web front-end) is realized based on Html5 platform in the prior art, and web front-end js object uses
Ember.js frame, rear end run java service providing data, and both ends are based on websocket long connection progress data subscription and obtain
Take, connection object is Protocol Buffers(abbreviation PB), serializing and the antitone sequence of structural data are done based on PB, i.e.,
That web front-end receives is the byte arrays byte [] after serializing, needs for byte [] to be converted into front end js object.
Specifically, as shown in Figure 1, the method packet of data of the webpage front-end serial process from background service in the prior art
Include following steps:
Step S101:web page invocation background service interface.
Wherein, include business datum list in the view of web page, background service interface is called by web page, is carried out
Data subscription.
Step S103: data queue Array is got<byte [ ]>.
Optionally, after background service receives subscription request, data queue is sent to web front-end, web front-end receives
To the data queue Array of subscription<byte [ ]>, wherein the data queue includes multiple array objects, and multiple array objects are pressed
It is arranged according to time field descending, for example, the data queue for getting subscription is the data queue of 3000 temporally field descendings.
Step S105: the data queue that will acquire carries out data conversion treatment, the data queue that mainly will acquire
In each array object be converted into the js object of web front-end, in conversion process, to keep the js data queue after conversion to remain unchanged
Orderly, the general processing using to the serial conversion one by one of the array object in data queue in the prior art.
Step S107: output treated JS object data queue.
After converting, the js data queue caching cache after conversion is got up, for view binding display.
The efficiency of web front-end change data is about in 400/s at present, if in the data queue got including 3000 numbers
According to, then execute aforesaid operations need time-consuming 7s or more.
It follows that in the prior art, operating in the Html5 web front-end application on common PC configuration machine, handling number
According to about 400/s of efficiency, using the above scheme handle mass data (e.g., 3000) when serial conversion data (turn one by one
Change), it is taken a substantial amount of time in the conversion process of data queue, for example, 3000 byte [] are changed into 3000 js objects
Process take around 7s, handling mass data, time-consuming, performance is poor, treatment effeciency is low.
In view of the above-mentioned problems, currently no effective solution has been proposed.
Summary of the invention
The application provides a kind of data processing method and device, comes from for solving prior art webpage front-end serial process
The mass data of background service, the problem that treatment effeciency is low, system performance is poor.
To achieve the goals above, the embodiment of the present application discloses a kind of data processing method applied in webpage front-end,
This method comprises: the source data got from background service is divided into multiple data blocks, wherein above-mentioned multiple data blocks are provided with
Label for unlabeled data block sequence；Data conversion is carried out to above-mentioned multiple data blocks parallel, multiple numbers after being converted
According to block；According to above-mentioned label, multiple data blocks after above-mentioned conversion are ranked up, target data is obtained, wherein above-mentioned target
Data are shown for front end view.
Further, the source data got from background service is divided into multiple data blocks includes: according to above-mentioned source data
Array object in above-mentioned source data is divided into above-mentioned multiple data blocks, wherein each above-mentioned data by the sequence of middle array object
The quantity for the above-mentioned array object that block includes is not more than predetermined quantity, and above-mentioned predetermined quantity is based on individual data conversion thread in list
It is capable of the maximum quantity of change data in the time of position and determines, each above-mentioned data conversion thread is used for an above-mentioned data block
Carry out data conversion.
Further, carrying out data conversion to above-mentioned multiple data blocks parallel includes: the quantity called with above-mentioned data block
The data conversion thread being consistent；Data conversion is carried out to above-mentioned multiple data blocks by the data conversion thread of calling, wherein every
A above-mentioned data conversion thread is used to the array object in an above-mentioned data block being converted to front end JS object one by one, to carry out
Data conversion.
Further, call the data conversion thread that is consistent with the quantity of above-mentioned data block include: called from thread pool and
The quantity of above-mentioned data block is consistent, idle data converts thread, wherein above-mentioned thread pool is at least used to record each data and turns
The thread state of thread-changing journey, above-mentioned thread state include: idle, work and exception.
Further, above-mentioned thread pool is at least also used to create above-mentioned data conversion thread, and records above-mentioned data and turn
The thread state of thread-changing journey, wherein in the case where above-mentioned data conversion thread is called, above-mentioned thread pool is used for above-mentioned number
Work is recorded as according to the thread state of conversion thread；It completes in the conversion operation of above-mentioned data conversion thread, return thread pool
In the case of, above-mentioned thread pool is used to the thread state of above-mentioned data conversion thread being recorded as the free time.
Further, after the data conversion thread by calling carries out data conversion to above-mentioned multiple data blocks, on
State method further include: if above-mentioned data conversion thread completes the data conversion of corresponding above-mentioned data block, call back function is executed,
In, above-mentioned call back function is at least used to recycle above-mentioned data conversion thread, and the above-mentioned data conversion thread of recycling is returned to line
Cheng Chi.
Further, after multiple data blocks after being converted, the above method further include: will be upper by call back function
Multiple data blocks indentation data block queue after stating conversion；According to above-mentioned label, multiple data blocks after above-mentioned conversion are carried out
Sequence, obtaining target data includes: by above-mentioned call back function, data block in above-mentioned data block queue is arranged according to above-mentioned label
Number；Ascending sort is carried out to the data block in above-mentioned data block queue according to above-mentioned number, obtains above-mentioned target data.
To achieve the goals above, the embodiment of the present application discloses a kind of data processing equipment applied in webpage front-end,
The device includes: division unit, for the source data got from background service to be divided into multiple data blocks, wherein above-mentioned more
A data block is provided with the label for unlabeled data block sequence；Parallel transformation unit, for parallel to above-mentioned multiple data blocks
Carry out data conversion, multiple data blocks after being converted；Sequencing unit is used for according to above-mentioned label, after above-mentioned conversion
Multiple data blocks are ranked up, and obtain target data, wherein above-mentioned target data is shown for front end view.
Further, above-mentioned division unit includes: segmentation module, for according in above-mentioned source data array object it is suitable
Array object in above-mentioned source data is divided into above-mentioned multiple data blocks, wherein the above-mentioned number that each above-mentioned data block includes by sequence
The quantity of group objects is not more than predetermined quantity, and above-mentioned predetermined quantity is based on individual data conversion thread can turn within the unit time
It changes the maximum quantity of data and determines, each above-mentioned data conversion thread is used to carry out data conversion to an above-mentioned data block.
Further, above-mentioned Parallel transformation unit includes: calling module, is consistent for calling with the quantity of above-mentioned data block
Data conversion thread；Conversion module carries out data to above-mentioned multiple data blocks for the data conversion thread by calling and turns
It changes, wherein each above-mentioned data conversion thread is used to the array object in an above-mentioned data block being converted to front end JS one by one
Object, to carry out data conversion.
Further, above-mentioned calling module includes: calling submodule, for calling the number with above-mentioned data block from thread pool
Amount is consistent, idle data conversion thread, wherein above-mentioned thread pool is at least used to record the thread shape of each data conversion thread
State, above-mentioned thread state include: idle, work and exception.
Further, above-mentioned thread pool is at least also used to create above-mentioned data conversion thread, and records above-mentioned data and turn
The thread state of thread-changing journey, wherein in the case where above-mentioned data conversion thread is called, above-mentioned thread pool is used for above-mentioned number
Work is recorded as according to the thread state of conversion thread；It completes in the conversion operation of above-mentioned data conversion thread, return thread pool
In the case of, above-mentioned thread pool is used to the thread state of above-mentioned data conversion thread being recorded as the free time.
Further, above-mentioned apparatus further include: execution unit, for passing through the data conversion thread called to above-mentioned more
After a data block carries out data conversion, if above-mentioned data conversion thread completes the data conversion of corresponding above-mentioned data block, hold
Row call back function, wherein above-mentioned call back function is at least used to recycle above-mentioned data conversion thread, by the above-mentioned data conversion of recycling
Thread returns to thread pool.
Further, above-mentioned apparatus further include: indentation unit leads to after multiple data blocks after being converted
It crosses call back function and multiple data blocks after above-mentioned conversion is pressed into data block queue；Above-mentioned sequencing unit includes: setup module, is used
In by above-mentioned call back function, the number of data block in above-mentioned data block queue is set according to above-mentioned label；Sorting module is used for
Ascending sort is carried out to the data block in above-mentioned data block queue according to above-mentioned number, obtains above-mentioned target data.
To achieve the goals above, the embodiment of the present application discloses a kind of storage media, and above-mentioned storage medium, which is stored with, to be used for
Execute the above-mentioned program applied in the data processing method of webpage front-end.
Using above-described embodiment, the source data subscribed to from background service is divided into multiple data blocks, parallel to multiple
Data block carries out data conversion, and upon the transition, the data block being converted to sequentially is resequenced, target data is obtained.
In the above-described embodiments, without the array object in converting source data one by one, array object that can simultaneously in multiple data blocks
Conversion operation is executed, the efficiency of change data is improved；Meanwhile after converting, again by the data block converted
Sequence, it is ensured that the order of data.Through the foregoing embodiment, it can be rapidly completed in the case where guaranteeing that data are orderly
Data conversion improves the efficiency of data conversion, solves prior art webpage front-end serial process from a large amount of of background service
Data, the problem that treatment effeciency is low, system performance is poor.
Detailed description of the invention
The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as the limitation to the application.And whole
In a attached drawing, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 is the flow diagram of data of the webpage front-end serial process from background service in the prior art；
Fig. 2 is the network environment schematic diagram according to an embodiment of the present invention applied in the data processing method of webpage front-end；
Fig. 3 is the flow chart one according to an embodiment of the present invention applied in the data processing method of webpage front-end；
Fig. 4 is that the structure according to an embodiment of the present invention for applying the thread pool in the creation of the data processing method of webpage front-end is shown
It is intended to；
Fig. 5 is various processes according to an embodiment of the present invention and data format correspondence diagram；
Fig. 6 is the flowchart 2 according to an embodiment of the present invention applied in the data processing method of webpage front-end；And
Fig. 7 is the structural schematic diagram of the data update apparatus according to an embodiment of the present invention for distributed index service-Engine.
Specific embodiment
In order to make the above objects, features, and advantages of the present application more apparent, with reference to the accompanying drawing and it is specific real
Applying mode, the present application will be further described in detail.
Firstly, to this application involves term meaning be explained as follows:
Ember.js: being the MVC frame of a JavaScript.This JavaScript frame and other JavaScript frames
It compares, possesses more compact modular frame, template system very abundant possesses composed view and UI binding.Wherein, UI is
The abbreviation of User Interface (user interface).
Protocol Buffers: it is a kind of structural data storage format of portable and effective, can be used for structuring number
According to serializing.It is well suited for doing data storage or RPC data interchange format.It can be used for the fields such as communications protocol, data storage
Language is unrelated, platform is unrelated, expansible serializing structured data formats, abbreviation PB.
HTML5: one under the kernel language of WWW, standard generalized markup language applies hypertext markup language
(HTML) the 5th material alteration is now widely used for the technology platform of web front-end exploitation.
HTML5 Web Worker(abbreviation Worker in the embodiment of the present application): HTML5 Web Worker, current
JavaScript execution thread in an additional thread can newly to be opened up using this class of Worker specific to load and run
JavaScript file, can't interact between this new thread and the main thread of JavaScript and block execution
's；And the interface of data exchange between this new thread and JavaScript main thread is being provided in Web Worker:
PostMessage and onmessage event.Wherein, postMessage is one in Windows API (application programming interfaces)
A common mathematical function, for a piece of news to be put into message queue, onmessage is a self-defined message inside MFC
Map it is macro, its effect be bind a customized message and the message receptance function.
Data model: any business datum, the business data model got from background service are the words generated based on PB
Joint number group byte [], each data record are a byte arrays byte [].Html5 web front-end gets data team
It arranges Array<byte []>, needs to be converted into the JavaScript object i.e. Ember.Object of Ember.Js, target is to be used for
Web(, that is, webpage) the various presentations uses in front end.
Init: the user level process started by kernel.Kernel, which voluntarily starts, (to be had been loaded into memory, brings into operation, and
Initialize all device driver and data structure etc.) after, just by way of starting a user class program init,
Complete boot process.So init is first process always (its process number is always 1).
In the description of the present application, it is to be understood that term " first ", " second " are used for description purposes only, and cannot
It is interpreted as indication or suggestion relative importance or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " the
One ", the feature of " second " can explicitly or implicitly include one or more of the features.The meaning of " plurality " is two
Or it is more than two, unless otherwise specifically defined.The terms "include", "comprise" and similar terms are understood to out
The term of putting property, i.e., " including/including but not limited to ".Term "based" is " being based at least partially on ".Term " embodiment "
It indicates " at least one embodiment "；Term " another embodiment " expression " at least one other embodiment ".The phase of other terms
Pass definition provides in will be described below.
The application is applied can be applied in network environment shown in Fig. 2 in the data processing method of webpage front-end.Such as Fig. 2
Shown, which includes background service 10 and webpage front-end 20, and the background service and webpage front-end, should by network connection
Network can be computer network, such as local area network, Metropolitan Area Network (MAN), wide area network, for another example, and finite element network and wireless network etc., the application
This is not construed as limiting.Above-mentioned background service can provide data, the number that webpage front-end provides background service for webpage front-end
According to being shown.
Above-mentioned array object is corresponding with business datum, and every business datum can be by a character array (also referred to as
Be data object) indicate, each character array may include that (e.g., an array object may include multiple field attributes
100 field attributes), business datum mainly includes numerical value class data and character string class data, and array object adapts to any industry
Business, that is, it is not limited to a certain business, it is general a function and model.
Embodiments herein is applied to internet environment, is based on html5 technology frame, is front end web application, fortune
Row is in ordinary PC device；Browser end hardware is very popular, and ' 4G memory, CPU 4core i5 2.5Ghz, browser can be used
Chrome v53+ ' is used as standard.Wherein, background service and webpage front-end can rely on physical machine and exist, background service
It is generally located on webpage front-end on different machines, optionally, background service and webpage front-end also can be set in a machine
On device, the application is not construed as limiting this.
Optionally, webpage front-end (i.e. web front-end) is realized based on Html5 platform, and web front-end js object uses ember.js
Frame, background service run java service providing data, and both ends are based on websocket long connection (for realizing browser and clothes
The connection of business device full-duplex communication) data subscription acquisition is carried out, connection object is Protocol Buffers(abbreviation PB), it is based on
PB does serializing and the antitone sequence of structural data, i.e., that web front-end receives is the byte arrays byte [] after serializing, needs
Byte [] is converted into front end js object.
Optionally, it is provided by the present application apply webpage front-end data processing method can by it is as shown in Figure 3 with
Lower step is realized:
Step S301: the source data got from background service is divided into multiple data blocks, wherein above-mentioned multiple data block settings
There is the label for unlabeled data block sequence.
Optionally, webpage front-end can send data subscription request to background service, and background service receives data subscription
After request, data corresponding with data subscription request are obtained, the data that will acquire return to webpage front-end, then webpage
Front end can get source data from background service.
Wherein, which can be the data queue comprising multiple array objects, and the array object in data queue is pressed
It is arranged according to time sequencing descending.Webpage front-end, will be in the data queue that received after receiving the data queue of subscription
Array object is divided into multiple data blocks, contains at least one array object in each data block.
Optionally, because the array object in source data be it is sequential, in the process that source data is divided into multiple data blocks
In, the sequence of each data block can be determined based on the sequence of array object in data block, and it is suitable that the multiple data blocks of mark are arranged
The label of sequence, the label can number for letter or number, e.g., 1,2,3 ... ..., and for another example, A, B, C ... ....The application is to mark
The form of label is not especially limited.
Step S303: data conversion is carried out to above-mentioned multiple data blocks parallel, multiple data blocks after being converted.
Optionally, after the source data received is divided into multiple data blocks by webpage front-end, parallel to multiple data blocks
In array object carry out data conversion, the data block after obtaining each conversion.Above-mentioned conversion is specifically referred to structuring
Array object is converted to front end JS object.
Parallel processing refers to that batch processing is executed by independent asynchronous speed, in this application, parallel in computer field
Processing refers to while carrying out data conversion treatment to multiple data blocks.
It should be noted that here while be not limited to synchronization conversion operation executed parallel to multiple data blocks.
Here it can refer to while while initiate the data conversion treatment to multiple data blocks.For example, if system idling-resource is enough,
It can then start to execute conversion operation parallel to multiple data blocks in synchronization；It, may be right if system idling-resource is inadequate
The operation of partial data block Parallel transformation, and during executing conversion operation parallel to partial data block, real-time detecting system
It is to have new idling-resource, if system has new idling-resource, using the new idling-resource to not yet starting to execute conversion
One or more data blocks of processing carry out data conversion.
It, can be simultaneously to a plurality of data (such as multiple arrays pair by multiple data block parallel execution of data conversion operations
As) data conversion is carried out, improve treatment effeciency.
Step S305: according to above-mentioned label, being ranked up multiple data blocks after above-mentioned conversion, obtain target data,
Wherein, above-mentioned target data is shown for front end view.
Optionally, after completing to the data conversion of multiple data blocks, according to the sequence of data block to conversion
Data block rearrangement afterwards, the sequence phase of the sequence of the obtained array object in target data and array object in source data
The target data can be carried out view displaying by symbol, web front-end.
Through the above steps, sequence can be re-started to the data in multiple data blocks after completing data conversion,
To guarantee the order of data.
Using above-described embodiment, the source data subscribed to from background service is divided into multiple data blocks, parallel to multiple
Data block carries out data conversion, and upon the transition, the data block being converted to sequentially is resequenced, target data is obtained.
In the above-described embodiments, without the array object in converting source data one by one, array object that can simultaneously in multiple data blocks
Conversion operation is executed, the efficiency of change data is improved；Meanwhile after converting, again by the data block converted
Sequence, it is ensured that the order of data.Through the foregoing embodiment, it can be rapidly completed in the case where guaranteeing that data are orderly
Data conversion improves the efficiency of data conversion, solves prior art webpage front-end serial process from a large amount of of background service
Data, the problem that treatment effeciency is low, system performance is poor.
The efficiency of the big data processing of web front-end under html5 environment can be improved through the foregoing embodiment, and guarantee data
Order.In the case where improving efficiency, in order to guarantee that the preceding consistency with after of ordered data processing, the application also mention simultaneously
Supplied a kind of embodiment, in this embodiment, by the source data got from background service be divided into multiple data blocks include: according to
Array object in above-mentioned source data is divided into above-mentioned multiple data blocks by the sequence of array object in above-mentioned source data, wherein every
The quantity for the above-mentioned array object that a above-mentioned data block includes is not more than predetermined quantity.
Optionally, above-mentioned predetermined quantity is capable of the maximum of change data based on individual data conversion thread within the unit time
Quantity and determine, each above-mentioned data conversion thread is used to carry out data conversion to an above-mentioned data block.By the embodiment,
Can based on individual data convert thread maximum transfer capability (maximum amount of data that can be converted i.e. within the unit time) come
The quantity of array object in each data block is determined, so as to the processing speed of balanced each data block, to improve entirety
Treatment effeciency.Wherein, data conversion thread can parallel execution of data conversion operation.
It should be noted that parallel data processing interface is mounted in each data conversion thread, so that multiple data turn
Thread-changing journey can execute conversion operation parallel.
Optionally, the quantity of array object can be all or part of identical in each data block.For example, in N number of data block
Before in (N-1) a data block array object quantity it is identical, the quantity of array object (N-1) number with before in n-th data block
It is different according to the quantity of array object in block.
For example, the quantity of optimal processing array object of individual data conversion thread is 400, in this, as fragment according to
According to, i.e., every 400 are used as a data block, assign to last inadequate 400 also as a fragment, 3000 piecemeals of illustrating, and totally 8
Block, first 7 pieces are all 400, last block only has 200.The field number of every data is the same, and field type is the same, field
Value can be different.The movement of fragment piecemeal is serial, not time-consuming, the slice slice that js array Array can be called to carry
Source data is divided into multiple data blocks using the slice function by function.Each data fragmentation piecemeal, at a separate threads
Reason.
Wherein, the association attributes of slice (start, [end]) slice function: first parameter start represents start bit
It sets, second parameter [end] represents next position of end position, and the length for the character string for intercepting out is second parameter
With the difference between first parameter；If parameter value is negative, switch to positive value after which is added string length；If first
Parameter, which is equal to, is greater than second parameter, then returns to null character string.
According to the application alternative embodiment, carrying out data conversion to above-mentioned multiple data blocks parallel includes: to call
The data conversion thread being consistent with the quantity of above-mentioned data block；By the data conversion thread of calling to above-mentioned multiple data blocks into
Row data conversion, wherein each above-mentioned data conversion thread is used to carry out data conversion to an above-mentioned data block.Optionally,
Each data conversion thread is used to the array object in a data block being converted to front end JS object one by one, with into
Row data conversion.
Specifically, thread pool can be constructed when webpage front-end initializes, thread is managed by the thread pool, before webpage
After the source data received is divided into multiple data blocks by end, call multiple data conversion threads in thread pool parallel to multiple
Data block carries out data conversion.Multiple data blocks can be handled parallel by the thread being pre-created.
In an alternative embodiment, each above-mentioned data conversion thread carries out data conversion to an above-mentioned data block
It include: that the array object in corresponding data block is converted to by front end JS object by above-mentioned data conversion thread one by one.
In the above-described embodiments, according to the sequence serial conversion array pair of array object in each data conversion thread
As, and after being converted to multiple data blocks, it is only necessary to the order for ensuring that data is ranked up to data block.
Present invention also provides a kind of optional embodiments, call the data conversion line being consistent with the quantity of above-mentioned data block
Journey includes: to call be consistent with the quantity of above-mentioned data block, idle data conversion thread from thread pool, wherein above-mentioned thread pool
At least for recording the thread state of each data conversion thread, above-mentioned thread state includes: idle, work and exception.Pass through
Thread pool management thread may be implemented to avoid resource congestion so that data conversion thread orderly works.
It is alternatively possible to different thread states is represented using different identifiers, for example, " 0 " mark is idle, " 1 " table
Show work, " 2 " indicate abnormal.
Still optionally further, above-mentioned thread pool is at least also used to create above-mentioned data conversion thread, and records above-mentioned number
According to the thread state of conversion thread, wherein in the case where above-mentioned data conversion thread is called, above-mentioned thread pool be used for by
The thread state for stating data conversion thread is recorded as work；Conversion operation completion, recurrence thread in above-mentioned data conversion thread
In the case where pond, above-mentioned thread pool is used to the thread state of above-mentioned data conversion thread being recorded as the free time.By thread pool and
Shi Biangeng thread state, can more reasonable distribution thread, the availability of thread is improved, rationally to utilize system resource.
Optionally, thread pool workerpool is constructed in initialization, manages thread for producing, accessing, recycle etc.；It is raw
At number of threads set by exterior arrangement.It is alternatively possible to by initPool construct thread pool, creation Thread Count by
Configs.threadCount is determined, needs to configure in configuration file, default is 10 threads.It should be noted that thread
Several configuration quantity are not limited to 10, can modify.
Optionally, the thread pool of creation mainly includes 3 core interfaces: initialization thread pond (thread containing creation) interface,
Thread sets idle interface, obtains idle thread interface.
One kind it is alternatively possible to create thread pool in the following way:
var wpool=new Array(configs.threadCount);
/ * * * initialization thread pond, creates thread, and thread state status:0(is idle), 1(work), 2(it is abnormal)
*/
initPool: function() {
for(var i = 0; i < configs.threadCount; i++) {
var worker = {};
worker.work = new Worker("com_operator_worker.js");
worker.id = i;
worker.status = 0;
this.wpool.push(worker);
}
},
/**
* the thread for completing work is set to the free time
*/
rebackWorker:function(id){
varwork=this.getWorkerById(id);
if(work===undefined){
return false;
}
work.status=0;
return true;
},
/**
* idle thread is obtained from thread pool
*/
getWorker: function() {
for(var i = 0; i < this.wpool.length; i++) {
if(this.wpool[i].status === 0) {
this.wpool[i].status = 1;
return this.wpool[i];
}
}
return null;
}
Wherein, core script is com_operator_worker.js in any thread (i.e. data conversion thread), this script dress
The core interface protoAlarmConvert of parallel data processing (i.e. change data) is carried.
It is described in detail below with reference to Fig. 4, the implementation that thread pool is managed data conversion line journey, as shown in figure 4,
It may include multiple data conversion threads in thread pool, it is idle data conversion thread that system, which obtains multiple states, acquisition
Data conversion thread parallel to multiple data blocks execute data conversion treatment, each data conversion thread complete data conversion it
Afterwards, thread pool is returned.Wherein, it is work that thread pool, which will call the state recording for removing to execute the data conversion thread of data transformation operations,
Make, the state recording of the data conversion thread of recurrence is the free time by thread pool.
It should be noted that being order-assigned first, first having handled during obtaining multiple data conversion threads
Data conversion thread is first returned to thread pool, and reallocation when is exactly to be randomly assigned, and what is embodied at random is to find first sky
Not busy thread is returned to.
Present invention also provides a kind of embodiments, carry out in the data conversion thread by calling to above-mentioned multiple data blocks
After data conversion, the above method further include: if above-mentioned data conversion thread completes the data conversion of corresponding above-mentioned data block,
Execute call back function, wherein above-mentioned call back function is at least used to recycle above-mentioned data conversion thread, and the above-mentioned data of recycling are turned
Thread-changing journey returns to thread pool.
Can be after the data conversion that data conversion thread completes a data block by call back function, the quick tropic
Cheng Chi realizes the effective use of resource so as to down-stream use.
Further, after multiple data blocks after being converted, the above method further include: will be upper by call back function
Multiple data blocks indentation data block queue after stating conversion；According to above-mentioned label, multiple data blocks after above-mentioned conversion are carried out
Sequence, obtaining target data includes: by above-mentioned call back function, data block in above-mentioned data block queue is arranged according to above-mentioned label
Number；Ascending sort is carried out to the data block in above-mentioned data block queue according to above-mentioned number, obtains above-mentioned target data.?
In the embodiment, ascending sort is carried out to data block according to the number of data block, sequencing production is fast, error rate is low.
The optional implementation applied in the data processing method of webpage front-end below with reference to Fig. 5 and Fig. 6 to the application
Example is described in detail, and as shown in Figure 5 and Figure 6, which may include steps of:
Step S601: creation thread pool.Specific implementation is consistent with the implementation of above-mentioned corresponding embodiment, no longer superfluous herein
It states.
Step S602: thread is created according to configuration file.
Optionally, according to the Thread Count configured in configuration file, data conversion thread corresponding with the Thread Count is created.
Specific implementation is consistent with the implementation of above-mentioned corresponding embodiment, and details are not described herein.
Step S603: webpage front-end obtains data queue from background service.
Optionally, webpage front-end calls the background service interface on its page, obtains the data queue of background service.Here
Data queue be source data in above-described embodiment.
Step S604: multiple data blocks are obtained to data queue's fragment.
Specific implementation is consistent with the implementation of above-mentioned corresponding embodiment, and details are not described herein.
Step S605: data conversion thread is obtained on demand from thread pool.Wherein it is possible to be obtained and data block number from thread pool
The data conversion thread being consistent is measured, if data conversion Thread Count amount idle in thread pool is less than the quantity of data block, is continued
Whether occur new idle data conversion thread in detection thread pool, if occurring, calls the new idle data conversion
Thread, until the total amount of the data conversion thread called is consistent with the quantity of data block.
Step S606: parallel asynchronous carries out data conversion to multiple data blocks.Specific implementation with it is above-mentioned it is corresponding implement
The implementation of example is consistent, and details are not described herein.
Step S607: the data block after conversion is ranked up to obtain orderly data queue.
If Fig. 6 shows the corresponding relationship between data format and above-mentioned steps, to source data queue fragment/piecemeal,
Before number, the data in source data queue are byte [] x, wherein the sequence serial number of x expression array object.It is numbered in piecemeal
Later, multiple data block 1-M are obtained.After the parallel data processing after piecemeal, unordered output treated data block, according to
Piecemeal number sorting, the target data after obtaining data conversion, the data format in the target data are as follows: jsObject(, that is, JS
Object).
Step S608: thread is returned to thread pool.
Above-described embodiment is illustrated so that webpage front-end gets 3000 datas from background service as an example below.
For example, obtaining 3000 datas from background service, need to carry out the conversion of byte [] to js object.Single web
Worker data conversion thread optimal processing 400, the Thread Count of 3000 demands starting need to be calculated, i.e., obtains foot from thread pool
Enough threads go handle data transitions, wherein the number of thread pool is configuration, and number of threads is just fixed after initialization, will not
Become again.What is changed is the state of thread itself: work, idle, exception.Algorithm is as follows:
/**
* data conversion distributes thread
* pbArray: source data queue such as 3000, onceCount: single thread handles optimal number of data such as 400
* threadCount: the number of threads finally created
* callbackCount: the readjustment number after thread process, when equal with threadCount, that is, all threads all locate
Reason finishes
*/
protoDataConvert: function(pbArray) {
var n = pbArray.length / this.onceCount;
this.callbackCount = 0;
this.threadCount = n;
for(var k = 0; k < n; k++) {
if(k + 1 > n) {
this.threadCount = k + 1;
} else {
this.threadCount = k;
}
var start = k * this.onceCount;
var end = (k + 1) * this.onceCount;
this.dataConvertWorker(pbArray.slice(start, end), k);
Console.info (' start start thread '+k+', allow it carry out the work of data conversion ');
}
}
Wherein, one data conversion thread dataConvertWorker of every starting represents 400 datas of processing, this 400 are
The pbArray.slice (start, end) come out from 3000 middle data fragmentations, piecemeal, k are the number of deblocking.
To the 3000 data piecemeals got, totally 8 pieces, first 7 pieces are all 400, last block only has 200.This 8
Data block is that serial in 3000 orderly queues (sequence) is split, and is successively numbered to each data block, i.e., 1/2/3 ...
7/8.It is as follows to start single scene dataConvertWorker:
/**
* single thread starts the course of work
*/
dataConvertWorker: function(dataSliceArray, no) {
// from thread pool obtain idle worker thread
var convwork = this.wpool.getWorker();
if(convwork === undefined) {
Console.info (' thread can not be obtained, thread pool work has been saturated！');
return;
}
convwork.work.postMessage({
data: dataSliceArray,
no: no,
wid: convwork.id
});
convwork.work.onmessage = function(ndata) {
Readjustment after //convwork thread process
this.dataConverCallback(ndata);
};
Console.info (' starting thread '+no+', and start to work ');
}
After any single thread has handled data, receipt row adjusts back dataConverCallback, and tissue is needed inside readjustment
Data block that treated.When all asynchronous threads are not fully completed work, by the data block push handled well to queue
DataBlocks, and recycle thread and return to thread pool；When all threads complete work, the last one thread is recycled, it will
Then the last one data block push compiles data block queue dataBlocks according to blocking of record to queue dataBlocks
Number, carry out ascending sort, the data sequence consensus before accomplishing and handling.Algorithm is as follows:
/**
* the arbitrarily readjustment processing of conversion thread
*/
dataConverCallback: function(data) {
this.callbackCount++;
if(this.callbackCount === this.threadCount) {
Console.info (' readjustment handles number and parallel thread sum is consistent, i.e., whole thread asynchronous process completions ');
this.dataBlocks.push(data);
this.dataBlocks.sort(function(a, b) {
if(a['no'] >= b['no']) {
return 1;
} else {
return -1;
}
});
this.wpool.rebackWorker(data.wid);
this.callbackCount = 0;
Console.info (' this worker thread, resetting deblocking queue are returned, and return to the data team after sequence
Column ');
return this.dataBlocks;
} else {
this.dataBlocks.push(data);
this.wpool.rebackWorker(data.wid);
Console.info (' return this worker thread, piecemeal queue is added in new data block；There are also other threads just in work
Make ');
return null;
}
}
After all threads complete work, this means that all, treated that data block enters dataBlocks queue, needs pair
Data block in dataBlocks queue carries out ascending sort according to number.In this example, 8 data blocks are from orderly
Serial (sequence) is split in 3000 queues, is successively numbered to each data block, i.e., 1/2/3 ... 7/8, per thread maintenance
It is consistent before itself 400 treated sequence and processing, then being numbered to piecemeal after all thread process are complete and carrying out ascending order row
Sequence, then final process is complete in dataBlocks 3000 and 3000 sequence consensus before processing.
Wherein, above-mentioned dataBlocks is data block queue, when data conversion thread completes respective data block
After data conversion, the data block push after conversion is entered and (is pushed into) dataBlocks, by dataBlocks in queue
All block sequencings empty dataBlocks after sequence is completed or is initialized.
In the examples described above, thread pool is created, safeguards work, the free time, abnormality of thread；It is got from background service
3000 datas carry out piecemeal fragment to data, will obtain the responsible deblocking of an idle thread from thread pool respectively
Processing, to all threads complete work, that is, all threads be completed data byte [] arrive js object conversion；This
When the thread of all completions revert to thread pool, then the data block of each thread output is arranged according to data block number
Sequence is organized into the data queue with sequence consistent before parallel processing；Whole process is completed in 1s, so as to ensure Html5
Big data processing under environment, meets the treatment effeciency such as 3000/s.
In the above-described embodiments, by carrying out Thread Count calculating to big data, obtaining idle thread from thread pool, to big number
According to piecemeal, parallel processing, thread readjustment after give back thread to thread pool, maintenance data block queue, by piecemeal number sorting data,
Treated complete ordered data is returned, parallel, asynchronous place can be carried out to a large amount of ordered datas under Html5 technical system
The feasible program of reason can also accomplish so that operating on ordinary PC device, using the web front-end application of Html5 technical system
The treatment effeciency of orderly big data is greatly improved.
To achieve the goals above, the embodiment of the present application discloses a kind of data processing equipment applied in webpage front-end,
The device includes: division unit 71, for the source data got from background service to be divided into multiple data blocks, wherein above-mentioned
Multiple data blocks are provided with the label for unlabeled data block sequence；Parallel transformation unit 73, for parallel to above-mentioned multiple numbers
Data conversion is carried out according to block, multiple data blocks after being converted；Sequencing unit 75 is used for according to above-mentioned label, to above-mentioned turn
Multiple data blocks after changing are ranked up, and obtain target data, wherein above-mentioned target data is shown for front end view.
Further, above-mentioned division unit includes: segmentation module, for according in above-mentioned source data array object it is suitable
Array object in above-mentioned source data is divided into above-mentioned multiple data blocks, wherein the above-mentioned number that each above-mentioned data block includes by sequence
The quantity of group objects is not more than predetermined quantity.
Further, above-mentioned predetermined quantity based on individual data conversion thread within the unit time can change data most
Big quantity and determine, each above-mentioned data conversion thread is used to carry out data conversion to an above-mentioned data block.
Further, above-mentioned Parallel transformation unit includes: calling module, is consistent for calling with the quantity of above-mentioned data block
Data conversion thread；Conversion module carries out data to above-mentioned multiple data blocks for the data conversion thread by calling and turns
It changes, wherein each above-mentioned data conversion thread is used to carry out data conversion to an above-mentioned data block.
Further, above-mentioned conversion module includes: transform subblock, for will correspond to number by above-mentioned data conversion thread
Front end JS object is converted to one by one according to the array object in block.
Further, above-mentioned calling module includes: calling submodule, for calling the number with above-mentioned data block from thread pool
Amount is consistent, idle data conversion thread, wherein above-mentioned thread pool is at least used to record the thread shape of each data conversion thread
State, above-mentioned thread state include: idle, work and exception.
Further, above-mentioned thread pool is at least also used to create above-mentioned data conversion thread, and records above-mentioned data and turn
The thread state of thread-changing journey, wherein in the case where above-mentioned data conversion thread is called, above-mentioned thread pool is used for above-mentioned number
Work is recorded as according to the thread state of conversion thread；It completes in the conversion operation of above-mentioned data conversion thread, return thread pool
In the case of, above-mentioned thread pool is used to the thread state of above-mentioned data conversion thread being recorded as the free time.
Further, above-mentioned apparatus further include: execution unit, for passing through the data conversion thread called to above-mentioned more
After a data block carries out data conversion, if above-mentioned data conversion thread completes the data conversion of corresponding above-mentioned data block, hold
Row call back function, wherein above-mentioned call back function is at least used to recycle above-mentioned data conversion thread, by the above-mentioned data conversion of recycling
Thread returns to thread pool.
Further, above-mentioned apparatus further include: indentation unit leads to after multiple data blocks after being converted
It crosses call back function and multiple data blocks after above-mentioned conversion is pressed into data block queue；Above-mentioned sequencing unit includes: setup module, is used
In by above-mentioned call back function, the number of data block in above-mentioned data block queue is set according to above-mentioned label；Sorting module is used for
Ascending sort is carried out to the data block in above-mentioned data block queue according to above-mentioned number, obtains above-mentioned target data.
To achieve the goals above, the embodiment of the present application discloses a kind of storage media, and above-mentioned storage medium, which is stored with, to be used for
Execute the above-mentioned program applied in the data processing method of webpage front-end.
Using above-described embodiment, the source data subscribed to from background service is divided into multiple data blocks, parallel to multiple
Data block carries out data conversion, and upon the transition, the data block being converted to sequentially is resequenced, target data is obtained.
In the above-described embodiments, without the array object in converting source data one by one, array object that can simultaneously in multiple data blocks
Conversion operation is executed, the efficiency of change data is improved；Meanwhile after converting, again by the data block converted
Sequence, it is ensured that the order of data.Through the foregoing embodiment, it can be rapidly completed in the case where guaranteeing that data are orderly
Data conversion improves the efficiency of data conversion, solves prior art webpage front-end serial process from a large amount of of background service
Data, the problem that treatment effeciency is low, system performance is poor.
Through the foregoing embodiment, using the webpage thread pool web worker using Html5, thread pool worker is established
Thread worker is managed, can also carry out parallel, asynchronous process to big data in the web front-end of html5, without by
The serial process of a data improves the availability of thread so as to ensure under limited resource.Using to source data
Fragment, piecemeal are carried out, then the data block after parallel processing is carried out by block sequencing, with the scheme of orderly recombination data, not only
Data-handling efficiency can be improved, and can ensure data Ordinal Consistency before and after the processing.
It should be noted that above-mentioned apparatus embodiment belongs to preferred embodiment, related unit and module might not
It is necessary to the application.
For the various method embodiments described above, simple in order to describe, therefore, it is stated as a series of action combinations, but
It is that those skilled in the art should be aware of, the application is not limited by the described action sequence, because according to the application,
Certain steps can serially or simultaneously be executed using other；Secondly, those skilled in the art should also know that, the above method is implemented
Example belongs to preferred embodiment, necessary to related actions and modules not necessarily the application.
Disclosed herein as well is a kind of storage mediums for being recorded on the program for executing the above method.It is described to deposit
Storage media includes any mechanism being configured to by the readable form storage of computer (by taking computer as an example) or transmission information.Example
Such as, storage medium includes read-only memory (ROM), random-access memory (ram), magnetic disk storage medium, optical storage media, sudden strain of a muscle
Fast storage medium, electricity, light, sound or transmitting signal (for example, carrier wave, infrared signal, digital signal etc.) of other forms etc..
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.For the dress of the application
For setting embodiment, since it is basically similar to the method embodiment, so being described relatively simple, related place is referring to method reality
Apply the explanation of example part.Device and Installation practice described above is only schematical, wherein described be used as is divided
Module from part description may or may not be physically separated, and both can be located in one place or can also be with
It is distributed over a plurality of network elements.Some or all of the modules therein can be selected to realize this implementation according to the actual needs
The purpose of example scheme.Those of ordinary skill in the art can understand and implement without creative efforts.
Above to a kind of method and apparatus for distributed index service-Engine provided herein, carry out in detail
It introduces, specific examples are used herein to illustrate the principle and implementation manner of the present application, the explanation of above embodiments
It is merely used to help understand the present processes and its core concept；At the same time, for those skilled in the art, according to this
The thought of application, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification is not answered
It is interpreted as the limitation to the application.
Claims (15)
1. a kind of data processing method applied in webpage front-end characterized by comprising
The source data got from background service is divided into multiple data blocks, wherein the multiple data block is provided with for marking
Show the label of data block order；
Data conversion is carried out to the multiple data block parallel, multiple data blocks after being converted；
According to the label, multiple data blocks after the conversion are ranked up, target data is obtained, wherein the target
Data are shown for front end view.
2. data processing method according to claim 1, which is characterized in that by the source data got from background service point
Include: for multiple data blocks
According to the sequence of array object in the source data, the array object in the source data is divided into the multiple data
Block；
Wherein, the quantity for the array object that each data block includes is not more than predetermined quantity, the predetermined quantity base
It is capable of the maximum quantity of change data within the unit time in individual data conversion thread and determines, each data conversion line
Journey is used to carry out data conversion to a data block.
3. data processing method according to claim 1, which is characterized in that carry out data to the multiple data block parallel
Conversion includes:
Call the data conversion thread being consistent with the quantity of the data block；
Data conversion is carried out to the multiple data block by the data conversion thread of calling, wherein each data conversion
Thread is used to the array object in a data block being converted to front end JS object one by one, to carry out data conversion.
4. data processing method as claimed in any of claims 1 to 3, which is characterized in that
After multiple data blocks after being converted, the method also includes: it will be more after the conversion by call back function
A data block is pressed into data block queue；
According to the label, multiple data blocks after the conversion are ranked up, obtaining target data includes: by described time
The number of data block in the data block queue is arranged according to the label for letter of transfer number；According to the number to the data block
Data block in queue carries out ascending sort, obtains the target data.
5. data processing method according to claim 3, which is characterized in that calling was consistent with the quantity of the data block
Data conversion thread includes:
Be consistent with the quantity of the data block, idle data conversion thread is called from thread pool, wherein the thread pool is at least
For recording the thread state of each data conversion thread, the thread state includes: idle, work and exception.
6. data processing method according to claim 5, which is characterized in that the thread pool is at least also used to create described
Data conversion thread, and record the thread state of the data conversion thread, wherein it is called in the data conversion thread
In the case where, the thread pool is used to the thread state of the data conversion thread being recorded as work；In the data conversion
In the case that the conversion operation of thread is completed, returns thread pool, the thread pool is used for the thread of the data conversion thread
State recording is the free time.
7. data processing method according to claim 3, which is characterized in that in the data conversion thread by calling to institute
After stating multiple data blocks progress data conversions, the method also includes:
If the data conversion thread completes the data conversion of the corresponding data block, call back function is executed, wherein described time
Letter of transfer number is at least used to recycle the data conversion thread, and the data conversion thread of recycling is returned to thread pool.
8. a kind of data processing equipment applied in webpage front-end characterized by comprising
Division unit, for the source data got from background service to be divided into multiple data blocks, wherein the multiple data block
It is provided with the label for unlabeled data block sequence；
Parallel transformation unit, for carrying out data conversion to the multiple data block parallel, multiple data blocks after being converted；
Sequencing unit, for being ranked up to multiple data blocks after the conversion, obtaining target data according to the label,
Wherein, the target data is shown for front end view.
9. data processing equipment according to claim 8, which is characterized in that the division unit includes:
Array object in the source data is divided by segmentation module for the sequence according to array object in the source data
The multiple data block,
Wherein, the quantity for the array object that each data block includes is not more than predetermined quantity, the predetermined quantity base
It is capable of the maximum quantity of change data within the unit time in individual data conversion thread and determines, each data conversion line
Journey is used to carry out data conversion to a data block.
10. data processing equipment according to claim 8, which is characterized in that the Parallel transformation unit includes:
Calling module, for calling the data conversion thread being consistent with the quantity of the data block；
Conversion module carries out data conversion to the multiple data block for the data conversion thread by calling, wherein each
The data conversion thread is used to the array object in a data block being converted to front end JS object one by one, to be counted
According to conversion.
11. the data processing equipment according to any one of claim 8 to 10, which is characterized in that
Described device further include: indentation unit, after multiple data blocks after being converted, by call back function by institute
Multiple data blocks indentation data block queue after stating conversion；
The sequencing unit includes: setup module, for the data block to be arranged according to the label by the call back function
The number of data block in queue；Sorting module, for being risen according to the number to the data block in the data block queue
Sequence sequence, obtains the target data.
12. data processing equipment according to claim 10, which is characterized in that the calling module includes:
Submodule is called, for calling be consistent with the quantity of the data block, idle data conversion thread from thread pool,
In, the thread pool is at least used to record the thread state of each data conversion thread, and the thread state includes: idle, work
Make and abnormal.
13. data processing equipment according to claim 12, which is characterized in that the thread pool is at least also used to create institute
Data conversion thread is stated, and records the thread state of the data conversion thread, wherein is adjusted in the data conversion thread
In the case where, the thread pool is used to the thread state of the data conversion thread being recorded as work；Turn in the data
In the case that the conversion operation of thread-changing journey is completed, returns thread pool, the thread pool is used for the line of the data conversion thread
Journey state recording is the free time.
14. data processing equipment according to claim 10, which is characterized in that described device further include:
Execution unit, after carrying out data conversion to the multiple data block in the data conversion thread for passing through calling, if
The data conversion thread completes the data conversion of the corresponding data block, then executes call back function, wherein the call back function
At least for recycling the data conversion thread, the data conversion thread of recycling is returned into thread pool.
15. a kind of storage medium, which is characterized in that the storage medium is stored with for any one in perform claim requirement 1 to 7
The program applied in the data processing method of webpage front-end described in.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711088531.0A CN110020359B (en) | 2017-11-08 | 2017-11-08 | Data processing method, device and storage medium applied to front end of webpage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711088531.0A CN110020359B (en) | 2017-11-08 | 2017-11-08 | Data processing method, device and storage medium applied to front end of webpage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110020359A true CN110020359A (en) | 2019-07-16 |
| CN110020359B CN110020359B (en) | 2024-04-05 |
Family
ID=67185946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711088531.0A Active CN110020359B (en) | 2017-11-08 | 2017-11-08 | Data processing method, device and storage medium applied to front end of webpage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110020359B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158939A (en) * | 2019-12-31 | 2020-05-15 | 中消云（北京）物联网科技研究院有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1990004236A1 (en) * | 1988-10-04 | 1990-04-19 | Gems Of Cambridge Limited | Improved data processing |
| JP2005150843A (en) * | 2003-11-11 | 2005-06-09 | Canon Inc | Image data coding apparatus and method, computer program, and computer-readable storage medium |
| US20050198568A1 (en) * | 2003-01-30 | 2005-09-08 | Fujitsu Limited | Table display switching method, text data conversion program, and tag program |
| CN1858735A (en) * | 2005-12-30 | 2006-11-08 | 华为技术有限公司 | Method for processing mass data |
| US20070220063A1 (en) * | 2005-12-30 | 2007-09-20 | O'farrell William J | Event data translation system |
| US20090177728A1 (en) * | 2007-12-20 | 2009-07-09 | Pottenger William M | Peer-to-peer indexing-based marketplace |
| CN101621353A (en) * | 2008-07-03 | 2010-01-06 | 华为技术有限公司 | Method, device and system for random linear network encoding |
| CN101860533A (en) * | 2010-05-26 | 2010-10-13 | 卓望数码技术（深圳）有限公司 | Data transmission method based on C/S architecture browser and server |
| US20120233417A1 (en) * | 2011-03-11 | 2012-09-13 | Microsoft Corporation | Backup and restore strategies for data deduplication |
| CN103167348A (en) * | 2011-12-13 | 2013-06-19 | 深圳长城开发科技股份有限公司 | Method for downloading update data of set top box software in the air |
| CN103218264A (en) * | 2013-03-26 | 2013-07-24 | 广东威创视讯科技股份有限公司 | Multi-thread finite state machine switching method and multi-thread finite state machine switching device based on thread pool |
| US20140324785A1 (en) * | 2013-04-30 | 2014-10-30 | Amazon Technologies, Inc. | Efficient read replicas |
| CN104166551A (en) * | 2014-08-14 | 2014-11-26 | 华为技术有限公司 | Application interface developing method and developing device |
| CN104216768A (en) * | 2014-09-22 | 2014-12-17 | 北京金山安全软件有限公司 | Data processing method and device |
| CN105224332A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | A kind of application program generates method and apparatus |
| CN106790022A (en) * | 2016-12-14 | 2017-05-31 | 福建天泉教育科技有限公司 | Communication means and its system based on many inquiry threads |
| US20190028668A1 (en) * | 2016-02-01 | 2019-01-24 | Alibaba Group Holding Limited | System and method for streaming multimedia data |
-
2017
- 2017-11-08 CN CN201711088531.0A patent/CN110020359B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1990004236A1 (en) * | 1988-10-04 | 1990-04-19 | Gems Of Cambridge Limited | Improved data processing |
| US20050198568A1 (en) * | 2003-01-30 | 2005-09-08 | Fujitsu Limited | Table display switching method, text data conversion program, and tag program |
| JP2005150843A (en) * | 2003-11-11 | 2005-06-09 | Canon Inc | Image data coding apparatus and method, computer program, and computer-readable storage medium |
| CN1858735A (en) * | 2005-12-30 | 2006-11-08 | 华为技术有限公司 | Method for processing mass data |
| US20070220063A1 (en) * | 2005-12-30 | 2007-09-20 | O'farrell William J | Event data translation system |
| US20090177728A1 (en) * | 2007-12-20 | 2009-07-09 | Pottenger William M | Peer-to-peer indexing-based marketplace |
| CN101621353A (en) * | 2008-07-03 | 2010-01-06 | 华为技术有限公司 | Method, device and system for random linear network encoding |
| CN101860533A (en) * | 2010-05-26 | 2010-10-13 | 卓望数码技术（深圳）有限公司 | Data transmission method based on C/S architecture browser and server |
| US20120233417A1 (en) * | 2011-03-11 | 2012-09-13 | Microsoft Corporation | Backup and restore strategies for data deduplication |
| CN102736961A (en) * | 2011-03-11 | 2012-10-17 | 微软公司 | Backup and restore strategies for data deduplication |
| CN103167348A (en) * | 2011-12-13 | 2013-06-19 | 深圳长城开发科技股份有限公司 | Method for downloading update data of set top box software in the air |
| CN103218264A (en) * | 2013-03-26 | 2013-07-24 | 广东威创视讯科技股份有限公司 | Multi-thread finite state machine switching method and multi-thread finite state machine switching device based on thread pool |
| US20140324785A1 (en) * | 2013-04-30 | 2014-10-30 | Amazon Technologies, Inc. | Efficient read replicas |
| CN104166551A (en) * | 2014-08-14 | 2014-11-26 | 华为技术有限公司 | Application interface developing method and developing device |
| CN104216768A (en) * | 2014-09-22 | 2014-12-17 | 北京金山安全软件有限公司 | Data processing method and device |
| CN105224332A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | A kind of application program generates method and apparatus |
| US20190028668A1 (en) * | 2016-02-01 | 2019-01-24 | Alibaba Group Holding Limited | System and method for streaming multimedia data |
| CN106790022A (en) * | 2016-12-14 | 2017-05-31 | 福建天泉教育科技有限公司 | Communication means and its system based on many inquiry threads |
Non-Patent Citations (6)
| Title |
|---|
| ANNETTE FENG: "Block-based programming abstractions for explicit parallel computing", 《2015 IEEE BLOCKS AND BEYOND WORKSHOP (BLOCKS AND BEYOND)》, pages 71 - 75 * |
| MINGHUI DONG等: "Frame labeling and mapping for non-parallel voice conversion", 《2017 IEEE 2ND INTERNATIONAL CONFERENCE ON SIGNAL AND IMAGE PROCESSING (ICSIP)》, pages 361 - 365 * |
| 丁有伟等: "一种异构集群中能量高效的大数据处理算法", 《计算机研究与发展》, vol. 52, no. 2, pages 377 - 390 * |
| 夏瑀登: "基于Wikidata和标签云的搜索算法研究", 《软件导刊》, vol. 15, no. 8, pages 42 - 46 * |
| 宋江洪: "图像分块分层结构在海量数据处理中的应用", 《 计算机工程与应用》, no. 33, pages 31 - 33 * |
| 杨甲森等: "用于数据交换的XML文档和关系数据库转换", 《计算机工程与设计》, vol. 27, no. 5, pages 857 - 859 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158939A (en) * | 2019-12-31 | 2020-05-15 | 中消云（北京）物联网科技研究院有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110020359B (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110447010A (en) | Matrix multiplication is executed within hardware | |
| CN106250100B (en) | Analyzing logic control method and device | |
| EP3007113A1 (en) | Event processing with enhanced throughput | |
| CN101114273A (en) | Executing an allgather operation with an alltoallv operation in a parallel computer | |
| CN104899085B (en) | A kind of data processing method and device | |
| CN111814959A (en) | Model training data processing method, device and system and storage medium | |
| US20140195510A1 (en) | Partitioning a search space for distributed crawling | |
| CN109697266A (en) | Page rendering system, page rendering method and page rendering device | |
| CN109471626A (en) | Page logic structure, page generation method, page data processing method and device | |
| CN110780921A (en) | Data processing method and device, storage medium and electronic device | |
| US20180109641A1 (en) | Data Processing Method and Apparatus, Server, and Controller | |
| CN109754072A (en) | Processing method, artificial intelligence process device and the Related product of internet off-line model | |
| CN116991560B (en) | Parallel scheduling method, device, equipment and storage medium for language model | |
| CN107273339A (en) | A kind of task processing method and device | |
| CN110941655A (en) | Data format conversion method and device | |
| CN109669678A (en) | Template engine integration method, device, electronic equipment and storage medium | |
| CN113676344B (en) | Network virtualization method and device for heterogeneous optical network, electronic equipment and storage medium | |
| Chen et al. | A hybrid dynamic harmony search algorithm for identical parallel machines scheduling | |
| CN110020359A (en) | Apply the data processing method in webpage front-end, device and storage medium | |
| CN111489279B (en) | GPU acceleration optimization method and device and computer storage medium | |
| CN111814044A (en) | Recommendation method and device, terminal equipment and storage medium | |
| Sax et al. | Aeolus: An optimizer for distributed intra-node-parallel streaming systems | |
| US20060004807A1 (en) | Performing recursive database operations | |
| CN110222286A (en) | Information acquisition method, device, terminal and computer readable storage medium | |
| CN114070901B (en) | Data transmitting and receiving method, device and equipment based on multi-data alignment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |