CN105577804B - Big data processing method and device - Google Patents
Big data processing method and device Download PDFInfo
- Publication number
- CN105577804B CN105577804B CN201511008580.XA CN201511008580A CN105577804B CN 105577804 B CN105577804 B CN 105577804B CN 201511008580 A CN201511008580 A CN 201511008580A CN 105577804 B CN105577804 B CN 105577804B
- Authority
- CN
- China
- Prior art keywords
- data
- server
- request
- user terminal
- pending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 230000005540 biological transmission Effects 0.000 claims description 21
- 230000001960 triggered effect Effects 0.000 claims description 8
- 238000000034 method Methods 0.000 abstract description 26
- 230000008569 process Effects 0.000 abstract description 19
- 238000005516 engineering process Methods 0.000 abstract description 13
- 239000002699 waste material Substances 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 17
- 238000002224 dissection Methods 0.000 description 12
- 238000004458 analytical method Methods 0.000 description 6
- 241001269238 Data Species 0.000 description 4
- 238000009434 installation Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a big data processing method and a big data processing device, wherein the big data processing method comprises the following steps: sending a data request to a server so that the server feeds back corresponding data to be processed according to the data request; receiving the data to be processed, and analyzing the data to be processed to generate processed data; and transmitting the processed data back to the server. According to the technical scheme, the server can process the mass data stored in the server through the user side, so that the server does not need to process the mass data, hardware resources of the server are saved, the concurrent processing capacity of the server is improved, and meanwhile, the mass data stored in the server is processed by the client side according to requirements, so that the problem of resource waste caused by full disk processing of the mass data by the server in the related technology is solved.
Description
Technical field
The present invention relates to technical field of data processing, big in particular to the processing method and one kind of a kind of big data
The processing unit of data.
Background technique
In the related art, server generallys use following two method when handling big data:
Method one: parsing data while receiving reported data, and what current most of producers reported is all that serial ports refers to
Data are enabled, the information for including is also more, parses these information and needs to consume certain computer resource；
Method two: the program for receiving reported data and processing reported data is separated, program is received and directly receives and deposit
Reported data is stored up, does not make other any processing, processing routine is according to regular individually parsing reported data, and by the result of parsing
It stores, for user query.
For both methods, have the following deficiencies:
1, server needs to handle received data while receiving data, this just needs to consume certain computer money
Source, therefore concurrent processing reports the quantity of refrigerator that must reduce, but also the case where blocking easy to form；
2, processing routine system needs to handle a large amount of data, although can need to put into certain with distributed computing
Server hardware resource have longer delay if data volume is bigger.In addition, according to sixteen principles, in system
Have a large amount of data may nobody inquiry can be gone to use, therefore, going to handle these data totally, not only to consume server hard
Part resource, also will cause the wasting of resources.
Therefore, how while saving server hardware resource, the oncurrent processing ability for improving server becomes urgently
The technical issues of solution.
Summary of the invention
The present invention is directed to solve at least one of the technical problems existing in the prior art or related technologies.
For this purpose, user can be passed through an object of the present invention is to provide a kind of processing scheme of new big data
The mass data stored in the processing server of end, so that server itself is not necessarily to handle mass data, to save
Server hardware resource, improves the oncurrent processing ability of server, while by client according to demand to storing in server
Mass data handled, avoid server in the related technology and mass data handled totally and is resulted in waste of resources
The problem of.
To achieve the above object, according to the embodiment of the first aspect of the invention, a kind of processing side of big data is proposed
Method is suitable for user terminal, comprising: request of data is sent to server, so that the server is fed back according to the request of data
Corresponding pending data；The pending data is received, and the pending data is parsed, treated to generate
Data；Will treated the data back to the server.
The processing method of the big data of embodiment according to the present invention, user terminal by server send request of data,
And the pending data of server feedback is received, and parse to pending data, to generate treated data, will locate
Data back after reason is to server, data that server stores that treated according to actual needs, after processing
Data carry out data update, the mass data by storing in user terminal processing server is realized, so that server itself
Without handling mass data, to save server hardware resource, the oncurrent processing ability of server is improved, together
When the mass data stored in server is handled according to demand by client, avoid in the related technology server to sea
The problem of amount data are handled totally and are resulted in waste of resources.
The processing method of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, the step of sending request of data to server, specifically includes: when detecting
When stating user terminal unlatching, Xiang Suoshu server sends the request of data；And/or work as the target solution detected in the user terminal
When analysis thread is triggered, Xiang Suoshu server sends the request of data；And/or when detecting the inquiry operation of user, to
The server sends the request of data.
The processing method of the big data of embodiment according to the present invention, user terminal can be sent to server in real time according to demand
Request of data specifically when may be provided at user terminal unlatching, sends request of data to server, may also be arranged on user installation
The parsing thread for parsing data when being triggered, send request of data to server, can also be provided at and detect user's
When inquiry operation, for example, detecting the clicking operation etc. of the inquiry button on the interface of user to user end, number is sent to server
According to request, send the setting of request of data to server under what conditions for user terminal, can by the manufacturer of user terminal Lai
Setting, can also be by user Lai customized.
According to one embodiment of present invention, before being parsed to the pending data, further includes: when detecting
When the inquiry operation of user, Xiang Suoshu server sends the request of data, and receives the server and asked according to the data
It negates the pending data of feedback；Judge whether all data in the pending data have been resolved；Described in judgement
When all data in pending data have been resolved, all data in the pending data are shown, for institute
User is stated to check；When all data or partial data in the judgement pending data do not parse, execute to described
The step of pending data is parsed.
The processing method of the big data of embodiment according to the present invention may when user inquires certain data by user terminal
The data stored in server are at this time by all data judged in pending data by user terminal dissection process mistake
It is no to be resolved, and when determining that all data in the pending data have been resolved, to all in pending data
Data are shown, and when determining that all data in pending data or partial data do not parse, then to number to be processed
According to being parsed, both avoided and repeated resolution is carried out to parsed data, and reduced the operating load of user terminal, improve simultaneously
To the displaying efficiency of user query data.
Embodiment according to the second aspect of the invention proposes a kind of processing method of big data, is suitable for server,
It include: the request of data for receiving each user terminal and sending；It is corresponding to each client feeds back according to the request of data
Pending data, so that each user terminal parses the pending data received, the number that obtains that treated
According to, and will treated the data back to the server；Receiving each user terminal passback described, treated
Data；Data update is carried out according to treated the data.
The processing method of the big data of embodiment according to the present invention is asked by receiving the data that each user terminal is sent
It asks, and according to request of data to the corresponding pending data of each client feeds back, and receives the processing of each user terminal passback
Data afterwards realize the mass data that will be stored in server and are distributed to so that data carry out data update according to treated
Each user terminal is handled the data of demand by each user terminal so that server itself be not necessarily to mass data into
Row processing, to save server hardware resource, improves the oncurrent processing ability of server, while by client according to need
It asks and the mass data stored in server is handled, avoid server in the related technology and mass data is located totally
The problem of managing and resulting in waste of resources.
The processing method of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, according to the request of data, to each client feeds back it is corresponding to
The step of handling data, specifically includes: according to default feedback strategy, to the corresponding number to be processed of each client feeds back
According to；Wherein, default feedback strategy includes the data category of feedback and/or the data bulk of feedback.
The processing method of the big data of embodiment according to the present invention, when receiving the request of data of each user terminal,
Server can be according to default feedback strategy to the corresponding pending data of each client feeds back, for example, in advance by server
In pending data be divided into personal data, alive data and other data etc., server is in the request for receiving user terminal
When instruction, the corresponding personal data of the user terminal are first fed back into the user terminal and are parsed, then are taken out at random from alive data
A certain amount of (such as 40) data feedback is taken to be parsed to user terminal, it is resolved or countless in personal data or alive data
According to when, a certain amount of (such as 40) data feedback is randomly selected from other data and is parsed to user terminal.Wherein, it presets anti-
Feedback strategy includes but is not limited to the data bulk of the data category of feedback, feedback, and the data category of feedback can be from multiple dimensions
It distinguishes.
According to one embodiment of present invention, before receiving the request of data that each user terminal is sent, further includes: receive
The running state data that household appliance uploads；The running state data is stored, using the running state data as described in
Pending data.
The processing method of the big data of embodiment according to the present invention, server can receive the upload of magnanimity household appliance
Running state data, and the progress of these running state datas is only stored, to carry out dissection process to user terminal, itself is not to it
Dissection process is carried out, to realize that magnanimity household appliance is handled by mass users end.
Embodiment according to the third aspect of the invention we proposes a kind of processing unit of big data, is suitable for user terminal,
It include: the first transmission unit, for sending request of data to server, so that the server is fed back according to the request of data
Corresponding pending data；Receiving unit, for receiving the pending data；Resolution unit, for the number to be processed
According to being parsed, with the data that generate that treated；Second transmission unit, for will treated the data back to the clothes
Business device.
The processing unit of the big data of embodiment according to the present invention, user terminal by server send request of data,
And the pending data of server feedback is received, and parse to pending data, to generate treated data, will locate
Data back after reason is to server, data that server stores that treated according to actual needs, after processing
Data carry out data update, the mass data by storing in user terminal processing server is realized, so that server itself
Without handling mass data, to save server hardware resource, the oncurrent processing ability of server is improved, together
When the mass data stored in server is handled according to demand by client, avoid in the related technology server to sea
The problem of amount data are handled totally and are resulted in waste of resources.
The processing unit of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, first transmission unit is specifically used for: when detecting that the user terminal opens
Qi Shi, Xiang Suoshu server send the request of data；And/or works as the target parsing thread detected in the user terminal and touched
When hair, Xiang Suoshu server sends the request of data；And/or when detecting the inquiry operation of user, Xiang Suoshu server
Send the request of data.
The processing unit of the big data of embodiment according to the present invention, user terminal can be sent to server in real time according to demand
Request of data specifically when may be provided at user terminal unlatching, sends request of data to server, may also be arranged on user installation
The parsing thread for parsing data when being triggered, send request of data to server, can also be provided at and detect user's
When inquiry operation, for example, detecting the clicking operation etc. of the inquiry button on the interface of user to user end, number is sent to server
According to request, send the setting of request of data to server under what conditions for user terminal, can by the manufacturer of user terminal Lai
Setting, can also be by user Lai customized.
According to one embodiment of present invention, further includes: judging unit, for detecting use when first transmission unit
The inquiry operation at family, Xiang Suoshu server sends the request of data, and the receiving unit receives the server according to institute
When stating the pending data of request of data feedback, judge whether all data in the pending data have been resolved；
Display unit, for when the judging unit determines that all data in the pending data have been resolved, to it is described to
All data in processing data are shown, so that the user checks；The resolution unit is specifically used for, described
When judging unit determines that all data or partial data in the pending data do not parse, the pending data is carried out
Parsing.
The processing unit of the big data of embodiment according to the present invention may when user inquires certain data by user terminal
The data stored in server are at this time by all data judged in pending data by user terminal dissection process mistake
It is no to be resolved, and when determining that all data in pending data have been resolved, to all data in pending data
Be shown, and determine pending data in all data or partial data do not parse when, then to pending data into
Row parsing, had both avoided and has carried out repeated resolution to parsed data, and reduced the operating load of user terminal, while improving pair
The displaying efficiency of user query data.
Embodiment according to the fourth aspect of the invention, it is also proposed that a kind of processing unit of big data is suitable for service
Device, comprising: the first receiving unit, the request of data sent for receiving each user terminal；Transmission unit, for according to the number
According to request, to the corresponding pending data of each client feeds back, so that each user terminal is to described in receiving
Pending data is parsed, the data that obtain that treated, and will treated the data back to the server；Second
Receiving unit, for the data that receive described in each user terminal passback that treated；Updating unit, for according to the place
Data after reason carry out data update.
The processing unit of the big data of embodiment according to the present invention is asked by receiving the data that each user terminal is sent
It asks, and according to request of data to the corresponding pending data of each client feeds back, and receives the processing of each user terminal passback
Data afterwards realize the mass data that will be stored in server and are distributed to so that data carry out data update according to treated
Each user terminal is handled the data of demand by each user terminal so that server itself be not necessarily to mass data into
Row processing, to save server hardware resource, improves the oncurrent processing ability of server, while by client according to need
It asks and the mass data stored in server is handled, avoid server in the related technology and mass data is located totally
The problem of managing and resulting in waste of resources.
The processing unit of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, the transmission unit is specifically used for: according to default feedback strategy, to described every
The corresponding pending data of a client feeds back；Wherein, default feedback strategy includes the data category of feedback and/or the number of feedback
Data bulk.
The processing unit of the big data of embodiment according to the present invention, when receiving the request of data of each user terminal,
Server can be according to default feedback strategy to the corresponding pending data of each client feeds back, for example, in advance by server
In pending data be divided into personal data, alive data and other data etc., server is in the request for receiving user terminal
When instruction, the corresponding personal data of the user terminal are first fed back into the user terminal and are parsed, then are taken out at random from alive data
A certain amount of (such as 40) data feedback is taken to be parsed to user terminal, it is resolved or countless in personal data or alive data
According to when, a certain amount of (such as 40) data feedback is randomly selected from other data and is parsed to user terminal.Wherein, it presets anti-
Feedback strategy includes but is not limited to the data bulk of the data category of feedback, feedback, and the data category of feedback can be from multiple dimensions
It distinguishes.
According to one embodiment of present invention, further includes: third receiving unit, for being received in first receiving unit
Before the request of data that each user terminal is sent, the running state data that household appliance uploads is received；Storage unit is used for
The running state data is stored, using the running state data as the pending data.
The processing unit of the big data of embodiment according to the present invention, server can receive the upload of magnanimity household appliance
Running state data, and the progress of these running state datas is only stored, to carry out dissection process to user terminal, itself is not to it
Dissection process is carried out, to realize that magnanimity household appliance is handled by mass users end.
Embodiment according to the fifth aspect of the invention, it is also proposed that a kind of user terminal, comprising: as appointed in above-described embodiment
The processing unit of big data described in one.
Embodiment according to the sixth aspect of the invention, it is also proposed that a kind of server, comprising: as appointed in above-described embodiment
The processing unit of big data described in one.
A kind of embodiment according to the seventh aspect of the invention, it is also proposed that system, comprising: as described in above-described embodiment
User terminal, the server and household appliance, wherein the server is for receiving and storing the household appliance
The running state data of biography.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect of the invention and advantage will become from the description of the embodiment in conjunction with the following figures
Obviously and it is readily appreciated that, in which:
Fig. 1 shows the schematic flow diagram of the processing method of big data according to an embodiment of the invention；
Fig. 2 shows the schematic flow diagrams of the processing method of big data according to another embodiment of the invention；
Fig. 3 shows the schematic block diagram of the processing unit of big data according to an embodiment of the invention；
Fig. 4 shows the schematic block diagram of the processing unit of big data according to another embodiment of the invention；
Fig. 5 shows the schematic block diagram of the user terminal of embodiment according to the present invention；
Fig. 6 shows the schematic block diagram of server according to an embodiment of the present invention；
Fig. 7 shows schematic block diagram according to the system in the embodiment of the present invention；
Fig. 8 shows the process flow diagram of the data of embodiment according to the present invention；
Fig. 9 shows the interaction diagrams between browser and server according to an embodiment of the present invention；
Figure 10 shows the process flow diagram of server distribution program according to an embodiment of the present invention.
Specific embodiment
To better understand the objects, features and advantages of the present invention, with reference to the accompanying drawing and specific real
Applying mode, the present invention is further described in detail.It should be noted that in the absence of conflict, the implementation of the application
Feature in example and embodiment can be combined with each other.
In the following description, numerous specific details are set forth in order to facilitate a full understanding of the present invention, still, the present invention may be used also
To be implemented using other than the one described here other modes, therefore, protection scope of the present invention is not by described below
Specific embodiment limitation.
Fig. 1 shows the schematic flow diagram of the processing method of big data according to an embodiment of the invention.
As shown in Figure 1, the processing method of big data according to an embodiment of the invention, is suitable for user terminal, comprising:
Step 102, request of data is sent to server, so that the server is corresponding according to request of data feedback
Pending data；
Step 104, the pending data is received, and the pending data is parsed, treated to generate
Data；
Step 106, will treated the data back to the server.
User terminal receives the pending data of server feedback, and treat by sending request of data to server
Processing data are parsed, to generate treated data, will treated data back to server, server is according to reality
Demand storage treated data, according still further to treated, data carry out data update, realize by user terminal
The mass data stored in reason server, so that server itself is not necessarily to handle mass data, thus the service of having saved
Device hardware resource improves the oncurrent processing ability of server, while by client according to demand to the sea stored in server
Amount data are handled, and are avoided server in the related technology and are handled totally mass data and what is resulted in waste of resources asks
Topic.
The processing method of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, the step of sending request of data to server, specifically includes: when detecting
When stating user terminal unlatching, Xiang Suoshu server sends the request of data；And/or work as the target solution detected in the user terminal
When analysis thread is triggered, Xiang Suoshu server sends the request of data；And/or when detecting the inquiry operation of user, to
The server sends the request of data.
User terminal can send request of data to server in real time according to demand, specifically, when may be provided at user terminal unlatching,
Request of data is sent to server, when may also be arranged on the parsing thread for parsing data of user installation and being triggered, to clothes
Business device sends request of data, when can also be provided at the inquiry operation for detecting user, for example, detecting user to user end interface
On inquiry button clicking operation etc., to server send request of data, for user terminal under what conditions to server
The setting for sending request of data, can be arranged by the manufacturer of user terminal, can also be by user Lai customized.
According to one embodiment of present invention, before being parsed to the pending data, further includes: when detecting
When the inquiry operation of user, Xiang Suoshu server sends the request of data, and receives the server and asked according to the data
It negates the pending data of feedback；Judge whether all data in the pending data have been resolved；Described in judgement
When all data in pending data have been resolved, all data in the pending data are shown, for institute
User is stated to check；When all data or partial data in the judgement pending data do not parse, execute to described
The step of pending data is parsed.
When user inquires certain data by user terminal, the data stored in possible server are by user terminal dissection process
It crosses, at this time by judging whether all data in pending data have been resolved, and is determining owning in pending data
When data have been resolved, all data in pending data are shown, and are determining owning in pending data
When data or partial data do not parse, then pending data is parsed, has both avoided and parsed data are repeated
Parsing, reduces the operating load of user terminal, while improving the displaying efficiency to user query data.
Fig. 2 shows the schematic flow diagrams of the processing method of big data according to another embodiment of the invention.
As shown in Fig. 2, the processing method of big data according to another embodiment of the invention, is suitable for server, packet
It includes:
Step 202, the request of data that each user terminal is sent is received；
Step 204, according to the request of data, to the corresponding pending data of each client feeds back, for institute
It states each user terminal to parse the pending data received, the data that obtain that treated, and will be after the processing
Data back to the server；
Step 206, treated the data of each user terminal passback are received；
Step 208, data update is carried out according to treated the data.
The request of data sent by receiving each user terminal, and it is corresponding to each client feeds back according to request of data
Pending data, and the passback of each user terminal is received treated data, with according to treated, data carry out data update,
It realizes the mass data that will be stored in server and is distributed to each user terminal, carried out by each user terminal come the data to demand
Processing, so that server itself, to save server hardware resource, improves service without handling mass data
The oncurrent processing ability of device, while the mass data stored in server is handled according to demand by client, it avoids
The problem of server handles mass data totally and is resulted in waste of resources in the related technology.
The processing method of big data according to the abovementioned embodiments of the present invention can also have following technical characteristic:
According to one embodiment of present invention, according to the request of data, to each client feeds back it is corresponding to
The step of handling data, specifically includes: according to default feedback strategy, to the corresponding number to be processed of each client feeds back
According to；Wherein, default feedback strategy includes the data category of feedback and/or the data bulk of feedback.
When receiving the request of data of each user terminal, server can be according to default feedback strategy to each user terminal
Feed back corresponding pending data, for example, in advance by the pending data in server be divided into personal data, alive data and
Other data etc., server first feed back to the corresponding personal data of the user terminal in the request instruction for receiving user terminal
The user terminal is parsed, then is randomly selected a certain amount of (such as 40) data feedback from alive data and solved to user terminal
Analysis randomly selects a certain amount of (such as 40 when personal data or alive data be resolved or no data from other data
Item) data feedback parsed to user terminal.Wherein, presetting feedback strategy includes but is not limited to the data category fed back, feedback
Data bulk, the data category of feedback can distinguish from multiple dimensions.
According to one embodiment of present invention, before receiving the request of data that each user terminal is sent, further includes: receive
The running state data that household appliance uploads；The running state data is stored, using the running state data as described in
Pending data.
Server can receive the running state data of magnanimity household appliance upload, and only store these running state datas
It carries out, to carry out dissection process to user terminal, itself does not carry out dissection process to it, to realize magnanimity household appliance by sea
Amount user terminal is handled.
Fig. 3 shows the schematic block diagram of the processing unit of big data according to an embodiment of the invention.
As shown in figure 3, the processing unit 300 of big data according to an embodiment of the invention, is suitable for user terminal, packet
It includes: the first transmission unit 302, receiving unit 304, resolution unit 306 and the second transmission unit 308.
Wherein, the first transmission unit 302, for sending request of data to server, so that the server is according to
Request of data feeds back corresponding pending data；Receiving unit 304, for receiving the pending data；Resolution unit 306,
For being parsed to the pending data, to generate treated data；Second transmission unit 308 is used for the place
Data back after reason is to the server.
User terminal receives the pending data of server feedback, and treat by sending request of data to server
Processing data are parsed, to generate treated data, will treated data back to server, server is according to reality
Demand storage treated data, according still further to treated, data carry out data update, realize by user terminal
The mass data stored in reason server, so that server itself is not necessarily to handle mass data, thus the service of having saved
Device hardware resource improves the oncurrent processing ability of server, while by client according to demand to the sea stored in server
Amount data are handled, and are avoided server in the related technology and are handled totally mass data and what is resulted in waste of resources asks
Topic.
The processing unit 300 of big data according to the abovementioned embodiments of the present invention, can also have following technical characteristic:
According to one embodiment of present invention, first transmission unit 302 is specifically used for: when detecting the user terminal
When unlatching, Xiang Suoshu server sends the request of data；And/or work as the target parsing thread quilt detected in the user terminal
When triggering, Xiang Suoshu server sends the request of data；And/or when detecting the inquiry operation of user, to the service
Device sends the request of data.
User terminal can send request of data to server in real time according to demand, specifically, when may be provided at user terminal unlatching,
Request of data is sent to server, when may also be arranged on the parsing thread for parsing data of user installation and being triggered, to clothes
Business device sends request of data, when can also be provided at the inquiry operation for detecting user, for example, detecting user to user end interface
On inquiry button clicking operation etc., to server send request of data, for user terminal under what conditions to server
The setting for sending request of data, can be arranged by the manufacturer of user terminal, can also be by user Lai customized.
According to one embodiment of present invention, further includes: judging unit 310, for being examined when first transmission unit 302
The inquiry operation of user is measured, Xiang Suoshu server sends the request of data, and the receiving unit 304 receives the service
When the pending data that device is fed back according to the request of data, judge all data in the pending data whether
It is resolved；Display unit 312, for determining that all data in the pending data have been solved in the judging unit 310
When analysis, all data in the pending data are shown, so that the user checks；The resolution unit
306 are specifically used for, and determine that all data or partial data in the pending data do not parse in the judging unit 310
When, the pending data is parsed.
When user inquires certain data by user terminal, the data stored in possible server are by user terminal dissection process
It crosses, at this time by judging whether all data in pending data have been resolved, and is determining owning in pending data
When data have been resolved, all data in pending data are shown, and are determining owning in pending data
When data or partial data do not parse, then pending data is parsed, has both avoided and parsed data are repeated
Parsing, reduces the operating load of user terminal, while improving the displaying efficiency to user query data.
Fig. 4 shows the schematic block diagram of the processing unit of big data according to another embodiment of the invention.
As shown in figure 4, the processing unit 400 of big data according to another embodiment of the invention, is suitable for server,
It include: the first receiving unit 402, transmission unit 404, the second receiving unit 406 and updating unit 408.
First receiving unit 402, the request of data sent for receiving each user terminal；Transmission unit 404 is used for basis
The request of data, to the corresponding pending data of each client feeds back, so that each user terminal is to receiving
The pending data parsed, the data that obtain that treated, and will treated the data back to the service
Device；Second receiving unit 406, for the data that receive described in each user terminal passback that treated；Updating unit 408,
For treated according to, data carry out data update.
The request of data sent by receiving each user terminal, and it is corresponding to each client feeds back according to request of data
Pending data, and the passback of each user terminal is received treated data, with according to treated, data carry out data update,
It realizes the mass data that will be stored in server and is distributed to each user terminal, carried out by each user terminal come the data to demand
Processing, so that server itself, to save server hardware resource, improves service without handling mass data
The oncurrent processing ability of device, while the mass data stored in server is handled according to demand by client, it avoids
The problem of server handles mass data totally and is resulted in waste of resources in the related technology.
The processing unit 400 of big data according to the abovementioned embodiments of the present invention, can also have following technical characteristic:
According to one embodiment of present invention, the transmission unit 404 is specifically used for: according to default feedback strategy, to institute
State the corresponding pending data of each client feeds back；Wherein, default feedback strategy includes the data category and/or feedback of feedback
Data bulk.
When receiving the request of data of each user terminal, server can be according to default feedback strategy to each user terminal
Feed back corresponding pending data, for example, in advance by the pending data in server be divided into personal data, alive data and
Other data etc., server first feed back to the corresponding personal data of the user terminal in the request instruction for receiving user terminal
The user terminal is parsed, then is randomly selected a certain amount of (such as 40) data feedback from alive data and solved to user terminal
Analysis randomly selects a certain amount of (such as 40 when personal data or alive data be resolved or no data from other data
Item) data feedback parsed to user terminal.Wherein, presetting feedback strategy includes but is not limited to the data category fed back, feedback
Data bulk, the data category of feedback can distinguish from multiple dimensions.
According to one embodiment of present invention, further includes: third receiving unit 410, in first receiving unit
Before receiving the request of data that each user terminal is sent, the running state data that household appliance uploads is received；Storage unit
412, for storing the running state data, using the running state data as the pending data.
Server can receive the running state data of magnanimity household appliance upload, and only store these running state datas
It carries out, to carry out dissection process to user terminal, itself does not carry out dissection process to it, to realize magnanimity household appliance by sea
Amount user terminal is handled.
Fig. 5 shows the schematic block diagram of the user terminal of embodiment according to the present invention.
As shown in figure 5, the user terminal 500 of embodiment according to the present invention, comprising: the processing of big data as shown in Figure 3
Device 300.
Fig. 6 shows the schematic block diagram of server according to an embodiment of the present invention.
As shown in fig. 6, the server 600 of embodiment according to the present invention, comprising: the processing of big data as shown in Figure 4
Device 400.
Fig. 7 shows schematic block diagram according to the system in the embodiment of the present invention.
As shown in fig. 7, the system 700 of embodiment according to the present invention, comprising: user terminal 500 as shown in Figure 5, Fig. 6 institute
The server 600 and household appliance 702 shown, wherein the server 600 is for receiving and storing the household appliance 702
The running state data of upload.
Technical solution of the present invention is described further below in conjunction with Fig. 8 to Figure 10.
In the present embodiment, it is illustrated using refrigerator as household appliance, refrigerator timing reports operation data, and server connects
Reported data is received, but directly stores serial ports message data with no treatment；When user opens system by browser, user
It can choose download and inquiry and accelerate program, inquiry accelerates program that can start with starting operating system, and a solution is opened after starting
Thread is analysed, if JS parses thread；If user does not download " inquiry accelerates program ", system will start a JS by browser
Parse thread, " JS parse thread " to server request data message, after server receives request, according to allocation rule wait locate
The message data of reason returns to JS parsing thread, and JS parses thread for the data back handled well to server later；If with
Family carries out inquiry operation, and JS parsing thread will stop work at present, and then remove the number not being resolved in parsing current queries result
According to parsing result being showed user, while parsing result is also returned to server after being parsed.
Specifically, as shown in figure 8, the data processing system includes browser, query service device, database, reports service
Device and refrigerator, wherein query service device, database and report server can integrate for the same server, specific data
Process flow includes:
S802: operation data is reported to server by refrigerator.For example, the serial ports that the data that refrigerator reports are 16 systems refers to
It enables, is exactly as follows a complete refrigerator running state data: " aa, 2f, ca, e5,00,00,43,00,00,04,01,01,
03,10,05,00,00,00,00,00,28,28,28,28,9b, 28,64e, 6e, 6e, 6e, 64,00,00,00,00,00,00,
00,02,01,00,00,00,00,00,00, db ", this instruction the inside contain temperature, humidity, load switch etc. more than 100
Information, it is assumed that there are 500,000 refrigerators to report primary information every 5 minutes, will be to the pressure of server it is huge, therefore, this reality
It applies the server process program in example and avoids parsing operation.
S804, server is reported directly to distribute one to database, while to the record for the serial ports reported instruction storage
Unique id.
When S806, user open " inquiry system " by browser, user can choose download and inquiry and accelerate program.
S808, inquiry accelerate program that can start with starting operating system, and a JS is opened after starting and parses thread；If
User does not have download and inquiry to accelerate program, and system will start a JS by browser and parse thread.
S810, JS are parsed after thread starts to query service device request data.
S812, inquiry database.
The data that S814, parsing receive.
S816, after being parsed, parsing result is submitted into the storage of query service device.
S818, more new database.
S820, user query data.
S822, querying condition is submitted.
S824, inquiry database, return to query result.
If there are not resolved data in S826, query result, JS parsing thread will stop current work, preferentially
Parse the data that user needs.
S828, by the data after parsing, show user, if being all the data parsed, be exposed directly to user,
Namely show query result.
S830, it whole parsing results is mentioned is back to query service device, to be updated to database.
As shown in figure 9, the interactive process of browser and server, specifically includes:
1, after user opens inquiry system, prompt the user whether that download and inquiry accelerates program, if user does not select down
It carries inquiry and accelerates program, then starting JS is defaulted by browser and parse thread, the thread is actively to server request data, " clothes
It is engaged in device distribution program " data to be resolved are returned, after JS parsing thread receives data, execute parsing work, and by the knot of parsing
Fruit returns to server, and parsing result example is as follows: and " temperature of refrigerating chamber ": 2, " freezer temperature ": -15, " press gear ":
2 ... ..., " freezing chamber Fahrenheit respective notch ": including 100 multinomial parsing results in parsing result 3 } at present, subsequent expansible；
If 2, user has downloaded inquiry acceleration program, which accelerates program that can start with computer booting, always can
From server request data, and the work of parsing is executed, but only take up a small amount of computer resource, does not influence user experience；
If 3, user has submitted query service, and inquires and returns the result the middle data for existing and not being resolved, JS parsing
Thread will stop at once current parsing work, preferential to parse the data not being resolved in query result, after the completion of to be resolved, then
Continue previous work.
As shown in Figure 10, the specific works of server distribution program include:
1), client (namely user terminal) request analysis data inquire the client preferentially in data pool (i.e. database)
The personal data at end then return to result data to be resolved if there is personal data to client；
2) if the personal data of the client, are not present in step 1), from " alive data ", by the side Hash
Method takes out 40 data to be resolved at random and returns to client；
If 3), 2) in or data are not present, from " other data ", pass through Hash method, it is random to take out 40
Data to be resolved return to client；
If 4), 3) in or data are not present, returning to empty result to client；
Above-mentioned " alive data " refers to: according to the liveness of user query, for example, the user being queried in current 30 days
Data, it is classified as " alive data "；
Above-mentioned " other data " refer to: other data in addition to " personal data " and " alive data "；
In the above-described embodiments, client is realized using HTML5 and JS technology, and Python, lua, C++ also can be used,
The same client functionality of the realizations such as C#；Inquiry accelerates program with starting operating system self-starting, can also manually boot, or
Configure setup rule；What is parsed is the serial ports instruction of 16 systems, is also possible to the machine instruction of other forms；
Server distribution program splits data into: " personal data ", " alive data ", " other data ", can also be from it
He sorts data into dimension, such as: by date, by area etc..
Through the foregoing embodiment, the data for realizing bulk device generation, are handled by the client of magnanimity, have saved clothes
Business device hardware resource；Improve server oncurrent processing ability；Client real-time query from mass data is realized simultaneously to go out
As a result.
The technical scheme of the present invention has been explained in detail above with reference to the attached drawings, and the invention proposes a kind of places of new big data
Reason scheme, can be by the mass data that stores in user terminal processing server, so that server itself is not necessarily to mass data
Handled, to save server hardware resource, improve the oncurrent processing ability of server, at the same by client according to
Demand handles the mass data stored in server, avoids server in the related technology and carries out totally to mass data
The problem of handling and resulting in waste of resources.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (8)
1. a kind of processing method of big data is suitable for user terminal characterized by comprising
Request of data is sent to server, so that the server feeds back corresponding pending data according to the request of data；
The pending data is received, and the pending data is parsed, to generate treated data；
Will treated the data back to the server；
Before being parsed to the pending data, further includes:
When detecting the inquiry operation of user, Xiang Suoshu server sends the request of data, and receives the server root
The pending data fed back according to the request of data；
Judge whether all data in the pending data have been resolved；
When determining that all data in the pending data have been resolved, to all data in the pending data into
Row is shown, so that the user checks；
When determining that all data in the pending data or partial data do not parse, execute to the pending data into
The step of row parsing.
2. the processing method of big data according to claim 1, which is characterized in that send the step of request of data to server
Suddenly, it specifically includes:
When detecting that the user terminal is opened, Xiang Suoshu server sends the request of data；And/or
When detecting that the target parsing thread in the user terminal is triggered, Xiang Suoshu server sends the request of data.
3. a kind of processing method of big data is suitable for server characterized by comprising
Receive the request of data that each user terminal is sent；
According to the request of data, to the corresponding pending data of each client feeds back, for each user terminal
The pending data received is parsed, the data that obtain that treated, and will treated the data back extremely
The server；
Receive treated the data of each user terminal passback；
Data update is carried out according to treated the data；
Before receiving the request of data that each user terminal is sent, further includes:
Receive the running state data that household appliance uploads；
The running state data is stored, using the running state data as the pending data.
4. the processing method of big data according to claim 3, which is characterized in that according to the request of data, Xiang Suoshu
It the step of each client feeds back corresponding pending data, specifically includes:
According to default feedback strategy, to the corresponding pending data of each client feeds back；
Wherein, default feedback strategy includes the data category of feedback and/or the data bulk of feedback.
5. a kind of processing unit of big data is suitable for user terminal characterized by comprising
First transmission unit, for sending request of data to server, so that the server is fed back according to the request of data
Corresponding pending data；
Receiving unit, for receiving the pending data；
Resolution unit, for being parsed to the pending data, to generate treated data；
Second transmission unit, for will treated the data back to the server；
Judging unit, for detecting the inquiry operation of user when first transmission unit, described in Xiang Suoshu server is sent
Request of data, and when the receiving unit receives the pending data that the server is fed back according to the request of data,
Judge whether all data in the pending data have been resolved；
Display unit, for when the judging unit determines that all data in the pending data have been resolved, to institute
All data stated in pending data are shown, so that the user checks；
The resolution unit is specifically used for, and determines all data or part number in the pending data in the judging unit
When according to not parsing, the pending data is parsed.
6. the processing unit of big data according to claim 5, which is characterized in that first transmission unit is specifically used
In:
When detecting that the user terminal is opened, Xiang Suoshu server sends the request of data；And/or
When detecting that the target parsing thread in the user terminal is triggered, Xiang Suoshu server sends the request of data.
7. a kind of processing unit of big data is suitable for server characterized by comprising
First receiving unit, the request of data sent for receiving each user terminal；
Transmission unit, for according to the request of data, to the corresponding pending data of each client feeds back, for institute
It states each user terminal to parse the pending data received, the data that obtain that treated, and will be after the processing
Data back to the server；
Second receiving unit, for the data that receive described in each user terminal passback that treated；
Updating unit, for treated according to, data carry out data update；
Third receiving unit, for before the request of data that first receiving unit receives that each user terminal is sent,
Receive the running state data that household appliance uploads；
Storage unit, for storing the running state data, using the running state data as the pending data.
8. the processing unit of big data according to claim 7, which is characterized in that the transmission unit is specifically used for:
According to default feedback strategy, to the corresponding pending data of each client feeds back；
Wherein, default feedback strategy includes the data category of feedback and/or the data bulk of feedback.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511008580.XA CN105577804B (en) | 2015-12-25 | 2015-12-25 | Big data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511008580.XA CN105577804B (en) | 2015-12-25 | 2015-12-25 | Big data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105577804A CN105577804A (en) | 2016-05-11 |
| CN105577804B true CN105577804B (en) | 2019-07-09 |
Family
ID=55887448
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511008580.XA Active CN105577804B (en) | 2015-12-25 | 2015-12-25 | Big data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105577804B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108063746B (en) * | 2016-11-08 | 2020-05-15 | 北京国双科技有限公司 | Data processing method, client, server and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127784A (en) * | 2007-09-29 | 2008-02-20 | 网秦无限（北京）科技有限公司 | Method and system for quickly obtaining network information service at mobile terminal |
| CN101741653A (en) * | 2008-11-21 | 2010-06-16 | 爱思开电讯投资（中国）有限公司 | Client server, intelligent terminal, online game system and method |
| CN101754430A (en) * | 2009-12-31 | 2010-06-23 | 魏新成 | System and method for dial-up networking via telephone website |
| CN102238223A (en) * | 2010-05-06 | 2011-11-09 | 清华大学 | Networked personal data management method for mobile device |
| CN102354178A (en) * | 2011-08-02 | 2012-02-15 | 常州节安得能源科技有限公司 | Energy efficiency monitoring system |
| CN104092770A (en) * | 2014-07-22 | 2014-10-08 | 中国电建集团华东勘测设计研究院有限公司 | Inner-enterprise address book management method and system based on cloud computing |
| CN104717286A (en) * | 2015-03-03 | 2015-06-17 | 百度在线网络技术（北京）有限公司 | Data processing method, terminal, server and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070226226A1 (en) * | 2006-03-23 | 2007-09-27 | Elta Systems Ltd. | Method and system for distributing processing of computerized tasks |
-
2015
- 2015-12-25 CN CN201511008580.XA patent/CN105577804B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101127784A (en) * | 2007-09-29 | 2008-02-20 | 网秦无限（北京）科技有限公司 | Method and system for quickly obtaining network information service at mobile terminal |
| CN101741653A (en) * | 2008-11-21 | 2010-06-16 | 爱思开电讯投资（中国）有限公司 | Client server, intelligent terminal, online game system and method |
| CN101754430A (en) * | 2009-12-31 | 2010-06-23 | 魏新成 | System and method for dial-up networking via telephone website |
| CN102238223A (en) * | 2010-05-06 | 2011-11-09 | 清华大学 | Networked personal data management method for mobile device |
| CN102354178A (en) * | 2011-08-02 | 2012-02-15 | 常州节安得能源科技有限公司 | Energy efficiency monitoring system |
| CN104092770A (en) * | 2014-07-22 | 2014-10-08 | 中国电建集团华东勘测设计研究院有限公司 | Inner-enterprise address book management method and system based on cloud computing |
| CN104717286A (en) * | 2015-03-03 | 2015-06-17 | 百度在线网络技术（北京）有限公司 | Data processing method, terminal, server and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105577804A (en) | 2016-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11513846B1 (en) | Distributed data acquisition, indexing and search system | |
| US10503531B2 (en) | Loading runtime configuration files into virtual machine instances which when executed transform a stored virtual machine image into a customized configuration | |
| CN101821728B (en) | Batch processing system | |
| CN109274730B (en) | Internet of things system, MQTT message transmission optimization method and device | |
| US9137172B2 (en) | Managing multiple proxy servers in a multi-tenant application system environment | |
| CN105117289A (en) | Task allocation method, device and system based on cloud testing platform | |
| CN104932941B (en) | A kind of distributed message processing system and device and method therein | |
| CN105279017A (en) | Cloud test platform based task distribution method, apparatus and system | |
| CN110753099B (en) | Distributed cache system and cache data updating method | |
| EP3128416B1 (en) | Sdn application integration, management and control method, system and device | |
| CN106325908B (en) | A kind of application package installation method and application package mounting device | |
| CN105183564B (en) | Equipment scheduling method, apparatus and system based on cloud test platform | |
| US8032587B2 (en) | Method and apparatus for client-side aggregation of asynchronous fragmented requests | |
| CN102916953A (en) | Method and device for realizing concurrent service on basis of TCP (transmission control protocol) connection | |
| CN106411778B (en) | The method and device of data forwarding | |
| CN105577804B (en) | Big data processing method and device | |
| CN111209310A (en) | Service data processing method and device based on stream computing and computer equipment | |
| CN110413388A (en) | Multi-task processing method, device, equipment and storage medium based on operation system | |
| US8788625B2 (en) | Displaying application content in synchronously opened window | |
| US20150281264A1 (en) | Security data processing method and system | |
| KR20180048940A (en) | Pre-instantiation of native applications in the background | |
| CN102023997B (en) | Data query system, construction method thereof and corresponding data query method | |
| US20210132925A1 (en) | Software provisioning agent residing in trusted execution environment | |
| EP3014483A1 (en) | Augmented reality | |
| CN105335362B (en) | The processing method and system of real time data, instant disposal system for treating |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180207 Address after: 230088 Building No. 198, building No. 198, Mingzhu Avenue, Anhui high tech Zone, Anhui Applicant after: Hefei Midea Intelligent Technology Co., Ltd. Address before: 230601 Hefei economic and Technological Development Zone, Fairview Road, Anhui Applicant before: Hefei Hualing Co., Ltd. Applicant before: Midea Group Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |