CN109634724A - Collecting method, device, equipment and computer storage medium - Google Patents
Collecting method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN109634724A CN109634724A CN201811205507.5A CN201811205507A CN109634724A CN 109634724 A CN109634724 A CN 109634724A CN 201811205507 A CN201811205507 A CN 201811205507A CN 109634724 A CN109634724 A CN 109634724A
- Authority
- CN
- China
- Prior art keywords
- thread
- task
- data acquisition
- data
- idle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000003860 storage Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 97
- 238000004458 analytical method Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012544 monitoring process Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of collecting methods, comprising: in the operational order that reception terminal is sent based on default shared interface, obtains the waiting task in operational order；Asynchronism callback mode is set by the tupe of processor, and thread is handled by task and executes waiting task；In task processing thread implementation procedure, the state of each thread in processor is monitored, and judge whether there is idle thread；Idle thread if it exists, then using idle thread as data acquisition line journey, and the service related data uploaded by data acquisition line journey acquisition terminal based on default shared interface；When task processing thread execution is completed, default shared interface is adjusted back, by presetting processing result of the shared interface to terminal transmission operational order.The invention also discloses a kind of data acquisition device, equipment and computer storage mediums.The present invention is optimized by data acquisition flow, so that service related data acquisition is more reasonable.
Description
Technical field
The present invention relates to data collecting fields more particularly to collecting method, device, equipment and computer storage to be situated between
Matter.
Background technique
With the fast development of computer technology, data analysis is carried out by computer and is become increasingly prevalent, but by
Limited in the data analysis algorithm of computer, data source and data acquisition just become particularly important.
For service optimization and business development, it usually needs be analyzed and processed for service related data, wherein business
Related data includes: the user behavior data in business datum (such as financial product purchase data), the product page, in business
Related data needs to carry out the acquisition of service related data before being analyzed, current business related data generallys use concentration and adopts
The mode of collection, that is, carried out in service related data of the special time period server to user mobile phone, such service related data
There are drawbacks for acquisition method, and e.g., carrying out data concentrated collection in business flow processing may interfere with normal operation flow,
In addition, data concentrated collection also needs to carry out intensive data transmission, more memory can be occupied, how influences the performance of server
Above situation is avoided, the acquisition for reasonably carrying out service related data becomes current technical problem urgently to be resolved.
Summary of the invention
The main purpose of the present invention is to provide a kind of collecting method, device, equipment and computer storage medium, purports
The problem when solving current server and carrying out data acquisition, so that service related data acquisition is more reasonable.
To achieve the above object, the present invention provides a kind of collecting method, and the collecting method includes following step
It is rapid:
The operational order that terminal is sent based on default shared interface is received, to be processed in the operational order is obtained
Business；
Asynchronism callback mode is set by the tupe of processor, and described to be processed by task processing thread execution
Task；
In task processing thread implementation procedure, the state of each thread in the processor is monitored, and judge whether
There are idle threads；
Idle thread if it exists then using the idle thread as data acquisition line journey, and passes through the data acquisition line
Journey acquires the service related data that the terminal is uploaded based on the default shared interface；
When task processing thread executes completion, the default shared interface is adjusted back, is connect by default share
Mouth sends the processing result of the operational order to the terminal.
Optionally, described to set asynchronism callback mode for the tupe of processor, and thread is handled by task and is held
The step of row waiting task, comprising:
Asynchronism callback mode is set by the tupe of processor, and counts the task quantity of the waiting task；
When the task quantity is greater than or equal to two, asynchronous thread pond is created；
Thread is handled using the idle thread in the asynchronous thread pond as task, and thread is handled by task and executes institute
State waiting task.
Optionally, it is described the task quantity be greater than or equal to two when, create asynchronous thread pond the step of after, packet
It includes:
Count the number of threads of idle thread in the asynchronous thread pond, and by the number of threads and the task quantity
It is compared；
If the number of threads is less than the task quantity, the waiting task more than the number of threads is added to
In default queue；
If the number of threads is greater than or equal to the task quantity, executes and make the idle thread in asynchronous thread pond
Thread is handled for task, and the step of thread executes the waiting task is handled by task.
Optionally, the idle thread if it exists, then using the idle thread as data acquisition line journey, and by described
Data acquisition line journey acquires the step of service related data that the terminal is uploaded based on the default shared interface, comprising:
Idle thread if it exists then judges in the default queue with the presence or absence of waiting task；
If there are waiting tasks in the default queue, thread is handled using the idle thread as task, is passed through
Task handles thread and executes the waiting task；
If waiting task is not present in the default queue, using the idle thread as data acquisition line journey, and
The service related data that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey.
It is optionally, described to monitor the state of each thread in the processor in task processing thread implementation procedure,
And the step of judging whether there is idle thread, comprising:
In task processing thread implementation procedure, the execution time of the task processing thread is estimated；
If the execution time is more than prefixed time interval, the state of each thread in the processor is monitored, and judge
With the presence or absence of idle thread.
Optionally, the idle thread if it exists, then using the idle thread as data acquisition line journey, and by described
Data acquisition line journey acquires the step of service related data that the terminal is uploaded based on the default shared interface, comprising:
Idle thread if it exists then obtains the number of threads of the idle thread and the core amounts of the processor；
When the number of threads is greater than the core amounts, terminating is more than the core amounts more than idle thread；
When the number of threads is less than or equal to the core amounts, using the idle thread as data acquisition line
Journey, and the service related data that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey.
Optionally, the idle thread if it exists, then using the idle thread as data acquisition line journey, and by described
After data acquisition line journey acquires the step of service related data that the terminal is uploaded based on the default shared interface, packet
It includes:
The total amount of data is compared the total amount of data for counting each service related data with presupposition analysis amount；
When the total amount of data is more than the presupposition analysis amount, the priority of each service related data is set；
Data are handled by the target that the priority of each service related data extracts the presupposition analysis amount corresponding number,
To carry out data processing.
In addition, to achieve the above object, the present invention also provides a kind of data acquisition device, the data acquisition device packet
It includes:
Command reception module, the operational order sent for receiving terminal based on default shared interface, obtains the operation
Waiting task in instruction；
Mode setting module for setting asynchronism callback mode for the tupe of processor, and is handled by task
Thread executes the waiting task；
Thread monitoring modular, for monitoring each thread in the processor in task processing thread implementation procedure
State, and judge whether there is idle thread；
Data acquisition module then using the idle thread as data acquisition line journey, and leads to for idle thread if it exists
It crosses the data acquisition line journey and acquires the service related data that the terminal is uploaded based on the default shared interface；
As a result sending module, for adjusting back the default shared interface, leading to when task processing thread executes completion
Cross the processing result that the default shared interface sends the operational order to the terminal.
In addition, to achieve the above object, the present invention also provides a kind of data acquisition equipments；
The data acquisition equipment includes: memory, processor and is stored on the memory and can be in the processing
The computer program run on device, in which:
The step of computer program realizes collecting method as described above when being executed by the processor.
In addition, to achieve the above object, the present invention also provides computer storage mediums；
Computer program, the realization when computer program is executed by processor are stored in the computer storage medium
Such as the step of above-mentioned collecting method.
A kind of collecting method, device, equipment and the computer storage medium that the embodiment of the present invention proposes, the present embodiment
During server carries out business processing, asynchronism callback mode is set by the tupe of processor, at task
Lineation journey (being called main thread) processing business process monitors the processor during task processing thread executes
In the state of each thread using the idle thread as data acquisition line journey, and adopted by the data when there is free idle thread
Collect thread capturing service related data, the distributed capture of business datum is realized, so that the acquisition of service related data will not shadow
It rings to task and handles thread, the process of data acquisition is more reasonable；It is carried out in addition, shared interface is preset in unification used herein
The acquisition and transmission of operational order processing and service related data, by reducing the quantity of data-interface in server, to subtract
The operating pressure of server is lacked, so that data-handling efficiency is higher.
Detailed description of the invention
Fig. 1 is the apparatus structure schematic diagram for the hardware running environment that the embodiment of the present invention is related to；
Fig. 2 is the flow diagram of collecting method first embodiment of the present invention；
Fig. 3 is the functional block diagram of one embodiment of data acquisition device of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The operational order that server receiving terminal is sent based on default shared interface in the present invention obtains the operational order
In waiting task；Asynchronism callback mode is set by the tupe of processor, and thread is handled by task and executes institute
State waiting task；In task processing thread implementation procedure, the state of each thread in the processor is monitored, and judge
With the presence or absence of idle thread；Idle thread if it exists then using the idle thread as data acquisition line journey, and passes through the number
The service related data that the terminal is uploaded based on the default shared interface is acquired according to collecting thread；Line is handled in the task
When Cheng Zhihang is completed, the default shared interface is adjusted back, the operation is sent to the terminal by the default shared interface
The processing result of instruction.
The corresponding business processing flow of operational order and the data acquisition flow of service related data are point in the prior art
It opens, i.e., two processing processing are mutually indepedent, only just will do it service related data acquisition on other occasions, counted
When according to analysis, server also needs the service related data that will be acquired in conjunction with concrete scene, in addition, operational order is corresponding
Business processing flow and the data acquisition flow of service related data pass through respective data-interface processing business data, terminal
Need to carry out frequent data-interface calling when carrying out data transmission, server, which is also required to corresponding data are arranged, to be connect
Mouthful, the data-interface quantity of server is more, causes the operating pressure of server big；The distribution of business datum is realized in the present invention
Acquisition, so that the acquisition of service related data does not interfere with task processing thread, so that data acquisition is more reasonable, in addition,
The acquisition and transmission that shared interface carries out operational order processing and service related data are preset in unification used herein, by subtracting
The quantity of data-interface in few server, to reduce the operating pressure of server, so that data-handling efficiency is higher.
As shown in Figure 1, the server that Fig. 1 is the hardware running environment that the embodiment of the present invention is related to (is called business phase
Close data equipment, wherein service related data equipment can be to be made of individual service related data device, be also possible to by
Other devices are combined with service related data device and to be formed) structural schematic diagram.
Server of the embodiment of the present invention refers to a management resource and provides the computer of service for user, is generally divided into file
Server, database server and apps server.The computer or computer system for running the above software are also referred to as
Server.For common PC (personal computer) personal computer, server is in stability, safety, property
Energy etc. requires higher；As shown in Figure 1, the server may include: processor 1001, such as central processing unit
(Central Processing Unit, CPU), network interface 1004, user interface 1003, memory 1005, communication bus
1002, hardware such as chipset, disk system, network etc..Wherein, communication bus 1002 is for realizing the connection between these components
Communication.User interface 1003 may include display screen (Display), input unit such as keyboard (Keyboard), optional user
Interface 1003 can also include standard wireline interface and wireless interface.Network interface 1004 optionally may include having for standard
Line interface, wireless interface (such as Wireless Fidelity WIreless-FIdelity, WIFI interface).Memory 1005 can be high speed with
Machine accesses memory (random access memory, RAM), is also possible to stable memory (non-volatile
), such as magnetic disk storage memory.Memory 1005 optionally can also be the storage dress independently of aforementioned processor 1001
It sets.
Optionally, server can also include camera, RF (Radio Frequency, radio frequency) circuit, sensor, sound
Frequency circuit, WiFi module；Input unit, than display screen, touch screen；Network interface can be blue in blanking wireless interface in addition to WiFi
Tooth, probe etc..It will be understood by those skilled in the art that server architecture shown in Fig. 1 does not constitute the restriction to server,
It may include perhaps combining certain components or different component layouts than illustrating more or fewer components.
As shown in Figure 1, the computer software product, which is stored in a storage medium, (storage medium: is called computer storage
Medium, computer media, readable medium, readable storage medium storing program for executing, computer readable storage medium are directly medium etc., storage
Medium can be non-volatile readable storage medium, such as RAM, magnetic disk, CD) in, including some instructions use is so that an end
End equipment (can be mobile phone, computer, server, air conditioner or the network equipment etc.) executes each embodiment institute of the present invention
The method stated, as may include operating system, network communication module, use in a kind of memory 1005 of computer storage medium
Family interface module and computer program.
In server shown in Fig. 1, network interface 1004 be mainly used for connect background data base, with background data base into
Row data communication；User interface 1003 is mainly used for connection client, and (client, is called user terminal or terminal, and the present invention is implemented
Example terminal can be also possible to mobile terminal with fixed terminal, e.g., intelligent air condition, intelligent electric lamp, intelligent power with network savvy,
Intelligent sound box, autonomous driving vehicle, PC, smart phone, tablet computer, E-book reader, portable computer etc., are wrapped in terminal
Containing sensor such as optical sensor, motion sensor and other sensors, details are not described herein), data are carried out with client
Communication；And processor 1001 can be used for calling the computer program stored in memory 1005, and it is real to execute the present invention or less
Step in the service related data method of example offer is provided.
The first embodiment of the present invention proposes that a kind of collecting method, the collecting method are applied to such as Fig. 1 institute
The server shown, specifically includes:
The operational order that terminal is sent based on default shared interface is received, to be processed in the operational order is obtained
Business；
Asynchronism callback mode is set by the tupe of processor, and described to be processed by task processing thread execution
Task；
In task processing thread implementation procedure, the state of each thread in the processor is monitored, and judge whether
There are idle threads；
Idle thread if it exists then using the idle thread as data acquisition line journey, and passes through the data acquisition line
Journey acquires the service related data that the terminal is uploaded based on the default shared interface；
When task processing thread executes completion, the default shared interface is adjusted back, is connect by default share
Mouth sends the processing result of the operational order to the terminal.
In the present embodiment data acquisition flow optimization is embedded in whole operation flow in an orderly manner, realizes that distributed data is adopted
Collection is allowed by starting asynchronism callback mode while the corresponding main business thread of operational order carries out and carries out service related data
Acquisition, effectively improve server service treatment effeciency, specifically:
Referring to Fig. 2, in a kind of first embodiment of collecting method of the present invention, the collecting method includes:
Step S10 receives the operational order that sends based on default shared interface of terminal, obtain in the operational order to
Processing task.
User triggers the relevant operational order of business at the terminal, and terminal sends operational order by default shared interface
To server, the operational order that server receiving terminal is sent based on default shared interface, server parses the operational order
Obtain the waiting task in the operational order, wherein default shared interface refers to data biography predetermined in server
Defeated interface, the data-interface can be not only used for carrying out the processing of operational order corresponding data, can be also used for the biography of data acquisition
It is defeated.
Step S20 sets asynchronism callback mode for the tupe of processor, and handles thread by task and execute institute
State waiting task.
The tupe of own processor is set asynchronism callback mode by server, that is, refers to being not received by operation
When enabling, the tupe of own processor is set blocking model by server, and realization rapidly sends and receives data, connecing
When receiving operational order, the tupe of processor is set asynchronism callback mode by server, to carry out multiple processing simultaneously
Thread, that is, the waiting task is executed by task processing thread (main thread), other sub threads can not also be stopped, with
By improving the resource utilization of processor, the treatment effeciency of Lai Tigao data.Operational order progress will be used in the present embodiment
Task processing thread as main business thread, using the corresponding data acquisition line journey of service related data as from thread (or
Sub thread).
In addition, after setting asynchronism callback mode for the tupe of processor, it is also necessary to determine waiting task
Quantity, in such a way that the quantity according to waiting task determines alignment processing, specifically:
The tupe of processor is set asynchronism callback mode by step a1, and counts appointing for the waiting task
Business quantity.
Step b1 creates asynchronous thread pond when the task quantity is greater than or equal to two.
Step c1 handles thread for the idle thread in the asynchronous thread pond as task, and handles line by task
Waiting task described in Cheng Zhihang.
That is, after setting asynchronism callback mode for the tupe of processor in the present embodiment, described in server statistics
The task quantity of waiting task judges whether the task quantity is multiple；When task quantity is multiple, asynchronous line is created
Cheng Chi；Thread is handled using the idle thread in the asynchronous thread pond as task, and is handled described in thread execution by task
Waiting task, when task quantity is one, the directly corresponding task of creation waiting task handles thread, and passes through task
It handles thread and executes the waiting task.It is wrapped in there are multiple operational orders or an operational order in the present embodiment
Asynchronous thread pond is created when containing multiple operation tasks, to carry out the efficient process of waiting task.
Step S30 monitors the state of each thread in the processor in task processing thread implementation procedure, and
Judge whether there is idle thread.
Specifically, comprising:
Step a2 estimates the execution time of the task processing thread in task processing thread implementation procedure.
Step b2 monitors the state of each thread in the processor if the execution time is more than prefixed time interval,
And judge whether there is idle thread.
That is, server estimates the execution time of the task processing thread in task processing thread implementation procedure,
That is, server pre-sets treatment schedule, it include the processing time of different type task in treatment schedule, server can
The mission bit stream in thread is handled according to task, query processing timetable obtains the execution time of task processing thread, server
Time and prefixed time interval (prefixed time interval refers to the pre-set time, for example, being set as 0.2s) will be executed, in institute
It states and executes the time no more than prefixed time interval, that is, determine that the processing time of current task is shorter, server normally executes this
Business processing thread；It is more than prefixed time interval in the execution time, that is, determine that the processing time of current task is longer, service
Device monitors the state of each thread in the processor, and judges whether there is idle thread, when there are idle thread, server
It can use idle thread and execute data acquisition session, specifically:
Step S40, idle thread, then using the idle thread as data acquisition line journey, and pass through the number if it exists
The service related data that the terminal is uploaded based on the default shared interface is acquired according to collecting thread.
Determine that there are idle threads in server, then server needs to call idle thread to carry out task processing, specifically
Ground:
Step a3, idle thread, then judge in the default queue with the presence or absence of waiting task if it exists；
Step b3 handles line for the idle thread as task if there are waiting tasks in the default queue
Journey handles thread by task and executes the waiting task；
Step c3 is acquired if waiting task is not present in the default queue using the idle thread as data
Thread, and the business dependency number that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey
According to.
That is, when there are idle thread, then server first determines whether that (default queue refers to each for the default queue of processor
The corresponding queue of thread) in whether there is waiting task, if there are waiting tasks in the default queue, by the sky
Idle thread handles thread as task, handles thread by task and executes the waiting task；Exist in default queue to
When the processing of processing task is completed, data acquisition session is being executed using idle thread；If there is no wait locate in the default queue
Reason task then using the idle thread as data acquisition line journey, and acquires the terminal base by the data acquisition line journey
In the service related data that the default shared interface uploads, wherein service related data refer to page data in terminal and
User behavior data on the page.
Step S50 adjusts back the default shared interface, by described pre- when task processing thread executes completion
If shared interface sends the processing result of the operational order to the terminal.
When task processing thread executes completion, server receives processing result, and described preset of server readjustment shares
Interface sends the processing result of the operational order by the default shared interface to the terminal.
During server carries out business processing in the present embodiment, set the tupe of processor to asynchronous time
Mode transfer formula handles thread (being called main thread) processing business process by task, in the process that task processing thread executes
In, the state of each thread in the processor is monitored, when there is free idle thread, using the idle thread as data acquisition line
Journey, and by the data acquisition line journey capturing service related data, the distributed capture of business datum is realized, so that business phase
The acquisition for closing data does not interfere with task processing thread, and the process of data acquisition is more reasonable；In addition, system used herein
One default shared interface carries out the acquisition and transmission of operational order processing and service related data, by reducing data in server
The quantity of interface, to reduce the operating pressure of server, so that data-handling efficiency is higher.
Further, on the basis of first embodiment of the invention, propose collecting method of the present invention second is real
Apply example.
The present embodiment is the refinement of step S20 in first embodiment, specifically illustrates in the present embodiment and receives in server
Processing mode when to multiple waiting tasks, specifically, comprising:
The tupe of processor is set asynchronism callback mode by step S21, and counts appointing for the waiting task
Business quantity.
The tupe of processor is set asynchronism callback mode by server, waiting task described in server statistics
Task quantity, server judge the task quantity whether at least two, that is, multiple to be processed are likely to be received on server
Business, server determine corresponding processing step according to the quantity of the waiting task received, specifically:
Step S22 creates asynchronous thread pond when the task quantity is greater than or equal to two.
If server determines that the task quantity is at least two, that is, when server receives multiple waiting tasks, clothes
Business device creates asynchronous thread pond, includes multiple idle threads, processing while to carry out multiple tasks, tool in asynchronous thread pond
Body:
Step S23, counts the number of threads of idle thread in the asynchronous thread pond, and by the number of threads with it is described
Task quantity is compared, to determine that waiting task goes out processing step according to comparison result.
Specifically, after creating asynchronous thread pond, the thread of idle thread in asynchronous thread pond described in server statistics
Quantity, and the number of threads is compared with the task quantity, if the number of threads is less than the task quantity,
Waiting task more than the number of threads is added to default queue (the default queue in default queue and first embodiment
It is identical) in；If the number of threads be greater than or equal to the task quantity, using the idle thread in asynchronous thread pond as times
Business processing thread, and thread is handled by task and executes the waiting task.There is also extra idle thread, services at this time
Device carries out data acquisition using extra idle thread, corresponding according to different task processing quantity settings in the present embodiment
Processing strategie guarantees server resource utilization rate, to improve the treatment effeciency of server.
Further, on the basis of first embodiment of the invention, the third for proposing collecting method of the present invention is real
Apply example.
The present embodiment is that step S40 is refined in first embodiment, and the present embodiment includes:
Step S41, idle thread, then obtain the number of threads of the idle thread and the core of the processor if it exists
Quantity.
If server determines that server obtains the number of threads and the processing of the idle thread there are idle thread
The core amounts of device, the core amounts of processor are that finger processor is monokaryon, double-core, four cores or eight cores etc., in the present embodiment
According to the core amounts of the number of threads of idle thread and the processor, the tupe of each thread is determined.
The core amounts are compared, to determine processing mode according to comparison result by step S42 with number of threads.
The core amounts are compared by the thread of core processing one of server one, server with number of threads,
The number of threads is greater than the core amounts, and server not may be performed simultaneously the corresponding all processing lines of number of threads
Journey, server will be more than terminating more than idle thread for the core amounts；If the core amounts are greater than or equal to the line
Number of passes amount then using the idle thread as data acquisition line journey, and acquires the terminal base by the data acquisition line journey
In the service related data that the default shared interface uploads.It is determined and is corresponded to according to the core amounts of processor in the present embodiment
Process flow, the utilization rate of server can be made higher, to improve the treatment effeciency of data.
Further, in the fourth embodiment of collecting method of the present invention, server can be based on collected terminal
Upload data acquisition specific processing method include:
Step S60 counts the total amount of data of each service related data, by the total amount of data and presupposition analysis amount into
Row compares.
The total amount of data of each service related data of server statistics, server is by the total amount of data and presupposition analysis
Amount is compared, wherein presupposition analysis amount refers to pre-set data amount of analysis critical value, is less than or equal in total amount of data
When presupposition analysis amount, server needs to analyze whole service related datas, is greater than presupposition analysis amount in total amount of data
When, the service related data of server selected part is analyzed.
The excellent of each service related data is arranged when the total amount of data is more than the presupposition analysis amount in step S70
First grade.
That is, each service related data is arranged in server when the total amount of data is more than the presupposition analysis amount
Priority (significance level that priority is used to embody business datum), so that server is according to the excellent of each service related data
First grading row data decimation.
Step S80 is extracted the target of the presupposition analysis amount corresponding number by the priority of each service related data
Data are handled, to carry out data processing.
Server is extracted by the priority of each service related data at the target of the presupposition analysis amount corresponding number
Manage data, that is, service related data is according to priority ranked up by server, business personality number of the server from highest priority
According to selecting corresponding target to handle data, to carry out data processing.In collection of server to more business phase in the present embodiment
When closing data, server can select data according to priority, carry out data processing and make data processing amount smaller, data processing
It is high-efficient.
In addition, the embodiment of the present invention also proposes that a kind of data acquisition device, the data acquisition device include: referring to Fig. 3
Command reception module 10, the operational order sent for receiving terminal based on default shared interface, obtains the behaviour
Waiting task in instructing；
Mode setting module 20, for setting asynchronism callback mode for the tupe of processor, and by task
Lineation journey executes the waiting task；
Thread monitoring modular 30, for monitoring each line in the processor in task processing thread implementation procedure
The state of journey, and judge whether there is idle thread；
Data acquisition module 40, for idle thread if it exists, then using the idle thread as data acquisition line journey, and
The service related data that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey；
As a result sending module 60, for adjusting back the default shared interface when task processing thread executes completion,
The processing result of the operational order is sent to the terminal by the default shared interface.
Optionally, the mode setting module 20, comprising:
Mode setting unit for setting asynchronism callback mode for the tupe of processor, and counts described wait locate
The task quantity of reason task；
Thread pool creating unit, for creating asynchronous thread pond when the task quantity is greater than or equal to two；
Task executing units for handling thread for the idle thread in the asynchronous thread pond as task, and pass through
Task handles thread and executes the waiting task.
Optionally, the mode setting module 20, comprising:
Quantity statistics unit, for counting the number of threads of idle thread in the asynchronous thread pond, and by the thread
Quantity is compared with the task quantity；
Task adding unit will be more than the number of threads if being less than the task quantity for the number of threads
Waiting task be added in default queue；
If the number of threads is greater than or equal to the task quantity, task executing units are executed, are used for asynchronous line
The idle thread of Cheng Chizhong handles thread as task, and handles the step of thread executes the waiting task by task.
Optionally, the data acquisition module 40, comprising:
Task judging unit then judges in the default queue for idle thread if it exists with the presence or absence of to be processed
Business；
Execution unit, if for there are waiting tasks in the default queue, using the idle thread as task
Thread is handled, thread is handled by task and executes the waiting task；
Data acquisition unit, if making the idle thread for waiting task to be not present in the default queue
For data acquisition line journey, and the industry that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey
Business related data.
Optionally, the thread monitoring modular 30, comprising:
Time estimates unit, for estimating the task processing thread in task processing thread implementation procedure
Execute the time；
Thread monitoring unit monitors each in the processor if being more than prefixed time interval for the execution time
The state of thread, and judge whether there is idle thread.
Optionally, the data acquisition module 40, further includes:
Acquiring unit, for idle thread if it exists, then obtain the idle thread number of threads and the processor
Core amounts；
Thread Termination unit, for when the number of threads is greater than the core amounts, terminating to be more than the core number
Amount is more than idle thread；
Data acquisition unit is used for when the number of threads is less than or equal to the core amounts, by the idle line
Cheng Zuowei data acquisition line journey, and the terminal is acquired by the data acquisition line journey and is based on the default shared interface upload
Service related data.
Optionally, the data acquisition device, comprising:
Comparison module is counted, for counting the total amount of data of each service related data, by the total amount of data and in advance
If amount of analysis is compared；
Priority setup module, for each business to be arranged when the total amount of data is more than the presupposition analysis amount
The priority of related data；
Data selecting module corresponds to number for extracting the presupposition analysis amount by the priority of each service related data
The target of amount handles data, to carry out data processing.
Wherein, the step of each Implement of Function Module of data acquisition device can refer to each of collecting method of the present invention
A embodiment, details are not described herein again.
In addition, the embodiment of the present invention also proposes a kind of computer storage medium.
Computer program, the realization when computer program is executed by processor are stored in the computer storage medium
Operation in collecting method provided by the above embodiment.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body/operation/object is distinguished with another entity/operation/object, without necessarily requiring or implying these entity/operations/
There are any actual relationship or orders between object；The terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that the process, method, article or the system that include a series of elements not only include that
A little elements, but also including other elements that are not explicitly listed, or further include for this process, method, article or
The intrinsic element of system.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged
Except there is also other identical elements in process, method, article or the system for including the element.
For device embodiment, since it is substantially similar to the method embodiment, related so describing fairly simple
Place illustrates referring to the part of embodiment of the method.The apparatus embodiments described above are merely exemplary, wherein making
It may or may not be physically separated for the unit of separate part description.In can selecting according to the actual needs
Some or all of the modules realize the purpose of the present invention program.Those of ordinary skill in the art are not making the creative labor
In the case where, it can it understands and implements.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of collecting method, which is characterized in that the collecting method the following steps are included:
The operational order that terminal is sent based on default shared interface is received, the waiting task in the operational order is obtained；
Asynchronism callback mode is set by the tupe of processor, and thread is handled by task and executes described to be processed
Business；
In task processing thread implementation procedure, the state of each thread in the processor is monitored, and judge whether there is
Idle thread；
Idle thread if it exists then using the idle thread as data acquisition line journey, and is adopted by the data acquisition line journey
Collect the service related data that the terminal is uploaded based on the default shared interface；
Task processing thread execute complete when, adjust back the default shared interface, by the default shared interface to
The terminal sends the processing result of the operational order.
2. collecting method as described in claim 1, which is characterized in that described to set different for the tupe of processor
Call back scheme is walked, and the step of thread executes the waiting task is handled by task, comprising:
Asynchronism callback mode is set by the tupe of processor, and counts the task quantity of the waiting task；
When the task quantity is greater than or equal to two, asynchronous thread pond is created；
Using the idle thread in the asynchronous thread pond as task handle thread, and by task handle thread execute described in
Processing task.
3. collecting method as claimed in claim 2, which is characterized in that described to be greater than or equal to two in the task quantity
When a, create asynchronous thread pond the step of after, comprising:
The number of threads of idle thread in the asynchronous thread pond is counted, and the number of threads and the task quantity are carried out
Compare；
If the number of threads is less than the task quantity, the waiting task more than the number of threads is added to default
In queue；
If the number of threads be greater than or equal to the task quantity, execute using the idle thread in asynchronous thread pond as times
Business processing thread, and the step of thread executes the waiting task is handled by task.
4. collecting method as claimed in claim 3, which is characterized in that the idle thread if it exists, then by the sky
Idle thread acquires the terminal by the data acquisition line journey and is based on the default shared interface as data acquisition line journey
The step of service related data of upload, comprising:
Idle thread if it exists then judges in the default queue with the presence or absence of waiting task；
If there are waiting tasks in the default queue, thread is handled using the idle thread as task, passes through task
It handles thread and executes the waiting task；
If waiting task is not present in the default queue, using the idle thread as data acquisition line journey, and pass through
The data acquisition line journey acquires the service related data that the terminal is uploaded based on the default shared interface.
5. collecting method as described in claim 1, which is characterized in that described to handle thread implementation procedure in the task
In, monitor the state of each thread in the processor, and the step of judging whether there is idle thread, comprising:
In task processing thread implementation procedure, the execution time of the task processing thread is estimated；
If the execution time is more than prefixed time interval, the state of each thread in the processor is monitored, and judge whether
There are idle threads.
6. collecting method as described in claim 1, which is characterized in that the idle thread if it exists, then by the sky
Idle thread acquires the terminal by the data acquisition line journey and is based on the default shared interface as data acquisition line journey
The step of service related data of upload, comprising:
Idle thread if it exists then obtains the number of threads of the idle thread and the core amounts of the processor；
When the number of threads is greater than the core amounts, terminating is more than the core amounts more than idle thread；
When the number of threads is less than or equal to the core amounts, using the idle thread as data acquisition line journey, and
The service related data that the terminal is uploaded based on the default shared interface is acquired by the data acquisition line journey.
7. collecting method as described in claim 1, which is characterized in that the idle thread if it exists, then by the sky
Idle thread acquires the terminal by the data acquisition line journey and is based on the default shared interface as data acquisition line journey
After the step of service related data of upload, comprising:
The total amount of data is compared the total amount of data for counting each service related data with presupposition analysis amount；
When the total amount of data is more than the presupposition analysis amount, the priority of each service related data is set；
By the priority of each service related data extract the presupposition analysis amount corresponding number target handle data, with into
Row data processing.
8. a kind of data acquisition device, which is characterized in that the data acquisition device includes:
Command reception module, the operational order sent for receiving terminal based on default shared interface, obtains the operational order
In waiting task；
Mode setting module for setting asynchronism callback mode for the tupe of processor, and handles thread by task
Execute the waiting task；
Thread monitoring modular, for monitoring the shape of each thread in the processor in task processing thread implementation procedure
State, and judge whether there is idle thread；
Data acquisition module then using the idle thread as data acquisition line journey, and passes through institute for idle thread if it exists
It states data acquisition line journey and acquires the service related data that the terminal is uploaded based on the default shared interface；
As a result sending module, for adjusting back the default shared interface, passing through institute when task processing thread executes completion
State the processing result that default shared interface sends the operational order to the terminal.
9. a kind of data acquisition equipment, which is characterized in that the data acquisition equipment includes: memory, processor and is stored in
On the memory and the computer program that can run on the processor, in which:
The data acquisition as described in any one of claims 1 to 7 is realized when the computer program is executed by the processor
The step of method.
10. a kind of computer storage medium, which is characterized in that be stored with computer program, institute in the computer storage medium
State the step of collecting method as described in any one of claims 1 to 7 is realized when computer program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811205507.5A CN109634724A (en) | 2018-10-16 | 2018-10-16 | Collecting method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811205507.5A CN109634724A (en) | 2018-10-16 | 2018-10-16 | Collecting method, device, equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109634724A true CN109634724A (en) | 2019-04-16 |
Family
ID=66066340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811205507.5A Pending CN109634724A (en) | 2018-10-16 | 2018-10-16 | Collecting method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109634724A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263618A (en) * | 2019-04-30 | 2019-09-20 | 阿里巴巴集团控股有限公司 | The alternative manner and device of one seed nucleus body model |
| CN111339005A (en) * | 2020-02-19 | 2020-06-26 | 上海新物科技有限公司 | Data acquisition method and device and computer readable storage medium |
| CN111597364A (en) * | 2020-03-27 | 2020-08-28 | 广联达科技股份有限公司 | Engineering quantity calculation intermediate layer generation method and device, storage medium and electronic equipment |
| CN112148493A (en) * | 2020-09-30 | 2020-12-29 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device and data server |
| CN112546617A (en) * | 2020-12-29 | 2021-03-26 | 珠海金山网络游戏科技有限公司 | Task processing method and device |
| CN112735115A (en) * | 2020-12-28 | 2021-04-30 | 新天科技股份有限公司 | Multithreading business processing method and device, server and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591721A (en) * | 2011-12-30 | 2012-07-18 | 北京新媒传信科技有限公司 | Method and system for distributing thread execution task |
| CN108279977A (en) * | 2017-12-29 | 2018-07-13 | 深圳市德兰明海科技有限公司 | A kind of data processing method, device and controller based on RTOS |
-
2018
- 2018-10-16 CN CN201811205507.5A patent/CN109634724A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102591721A (en) * | 2011-12-30 | 2012-07-18 | 北京新媒传信科技有限公司 | Method and system for distributing thread execution task |
| CN108279977A (en) * | 2017-12-29 | 2018-07-13 | 深圳市德兰明海科技有限公司 | A kind of data processing method, device and controller based on RTOS |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263618A (en) * | 2019-04-30 | 2019-09-20 | 阿里巴巴集团控股有限公司 | The alternative manner and device of one seed nucleus body model |
| CN110263618B (en) * | 2019-04-30 | 2023-10-20 | 创新先进技术有限公司 | Iteration method and device of nuclear body model |
| CN111339005A (en) * | 2020-02-19 | 2020-06-26 | 上海新物科技有限公司 | Data acquisition method and device and computer readable storage medium |
| CN111339005B (en) * | 2020-02-19 | 2023-09-08 | 上海新物科技有限公司 | Data acquisition method, device and computer readable storage medium |
| CN111597364A (en) * | 2020-03-27 | 2020-08-28 | 广联达科技股份有限公司 | Engineering quantity calculation intermediate layer generation method and device, storage medium and electronic equipment |
| CN111597364B (en) * | 2020-03-27 | 2024-01-30 | 广联达科技股份有限公司 | Engineering quantity calculation middle layer generation method and device, storage medium and electronic equipment |
| CN112148493A (en) * | 2020-09-30 | 2020-12-29 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device and data server |
| CN112735115A (en) * | 2020-12-28 | 2021-04-30 | 新天科技股份有限公司 | Multithreading business processing method and device, server and storage medium |
| CN112546617A (en) * | 2020-12-29 | 2021-03-26 | 珠海金山网络游戏科技有限公司 | Task processing method and device |
| CN112546617B (en) * | 2020-12-29 | 2022-08-30 | 珠海金山数字网络科技有限公司 | Task processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109634724A (en) | Collecting method, device, equipment and computer storage medium | |
| CN109672627A (en) | Method for processing business, platform, equipment and storage medium based on cluster server | |
| CN111459665A (en) | Distributed edge computing system and distributed edge computing method | |
| CN107943577A (en) | Method and apparatus for scheduler task | |
| US11537862B2 (en) | Neural network processor and control method of neural network processor | |
| CN107729139A (en) | A kind of method and apparatus for concurrently obtaining resource | |
| CN108683720A (en) | A kind of container cluster service configuration method and device | |
| CN106557369A (en) | A kind of management method and system of multithreading | |
| CN110427252A (en) | Method for scheduling task, device and the storage medium of task based access control dependence | |
| JP2014531081A (en) | Method, apparatus and system for scheduling processor cores in a multiprocessor core system | |
| CN108012156A (en) | A kind of method for processing video frequency and control platform | |
| CN108540568B (en) | Computing capacity sharing method and intelligent equipment | |
| CN110427256A (en) | Job scheduling optimization method, equipment, storage medium and device priority-based | |
| CN105068865A (en) | Task scheduling method and apparatus | |
| CN102999385A (en) | Multiprocessor co-processing method in computation equipment | |
| CN109634738A (en) | Asynchronous processing method, server, storage medium and device based on micro services | |
| CN109445921A (en) | A kind of distributed data task processing method and device | |
| CN107729137A (en) | Server, the method and storage medium of the decryption of block chain sign test | |
| CN104461710A (en) | Method and device for processing tasks | |
| CN108829519A (en) | Method for scheduling task, cloud platform and computer readable storage medium based on cloud platform | |
| CN107911547A (en) | Interactive system, the method for interface layout | |
| CN108920274A (en) | Performance optimization and device for image processing server end | |
| CN110532559A (en) | The processing method and processing device of rule | |
| CN109889406A (en) | For managing method, apparatus, equipment and the storage medium of network connection | |
| CN109542867A (en) | Distribution type data collection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |