CN107483532B - Behavior data uploading method, resource server and storage medium - Google Patents
Behavior data uploading method, resource server and storage medium Download PDFInfo
- Publication number
- CN107483532B CN107483532B CN201710506118.5A CN201710506118A CN107483532B CN 107483532 B CN107483532 B CN 107483532B CN 201710506118 A CN201710506118 A CN 201710506118A CN 107483532 B CN107483532 B CN 107483532B
- Authority
- CN
- China
- Prior art keywords
- data
- identification
- information
- uploaded
- attribute information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/62—Establishing a time schedule for servicing the requests
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a behavior data uploading method, a resource server and a storage medium, wherein the method comprises the following steps: acquiring a target preset identifier and target attribute information which are matched with data identifiers in behavior data uploaded by a client from a configuration file; judging whether behavior data need to be uploaded immediately or not according to the target attribute information; if so, uploading the data identification and the target attribute information to a platform server; if not, adding the data identification and the target attribute information to the data container; judging whether the information quantity in the data container is greater than M; judging whether the time length of the information uploaded for the last time from the data container is greater than T; and if the quantity of the information in the data container is greater than M, or the time length from the last time of uploading the information in the data container to the platform server is greater than T, uploading the information in the data container to the platform server. The invention realizes the technical effects of reducing the difficulty of later-stage data maintenance and modification and reducing network congestion and CPU consumption of the platform server.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a behavior data uploading method, a resource server and a storage medium.
Background
In order to facilitate manufacturers to improve and know the using habits of users, the flash player can collect a large amount of user behavior data or behavior data of the flash player to upload the user behavior data or the behavior data to the platform server, so that the platform server can conveniently analyze and mine data with valuable content.
The current collection method is that the flash player directly uploads the collected behavior data to the platform server.
However, in the process of collecting data, the collected data is difficult to maintain because of the numerous and scattered business scenarios. Particularly, when data collected in a certain type at a later stage needs to be modified, a large amount of labor cost needs to be consumed to modify the data one by one in the process of re-collection.
In addition, in a certain service scenario, a flash player may report a large amount of data to a platform server at the same time, which may cause a peak value of network communication between the flash player or a resource server and the platform server, thereby increasing bandwidth cost.
Therefore, the existing method for uploading behavior data by a flash player has the technical problems of difficult data maintenance and modification, easy network blockage and promotion of CPU consumption of a platform server.
Disclosure of Invention
The embodiment of the application provides an uploading method of behavior data, a resource server and a storage medium, and solves the technical problems that in the prior art, a flash player collects behavior data, data maintenance and modification are difficult, network congestion is easily caused, and CPU consumption of a platform server is easily increased.
In a first aspect, the present application provides a behavior data uploading method, which is applied to a resource server, and includes:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to the platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
Optionally, the obtaining of the target preset identifier ID matched with the data identifier ID in the behavior data uploaded by the client and the target attribute information corresponding to the preset identifier ID includes: receiving the behavior data sent by a client, wherein the behavior data comprises the data identification ID; judging whether the target preset identification ID matched with the data identification ID exists in the N preset identification IDs of the configuration file; attribute information corresponding to each preset identification ID in the N preset identification IDs is also preset in the configuration file; n is a positive integer; and if a target preset identification ID matched with the data identification ID exists in the configuration file, acquiring the target attribute information corresponding to the target preset identification ID.
Optionally, the determining whether there is the target preset identifier ID matching the data identifier ID in the N preset identifier IDs of the configuration file further includes: and if the target preset identification ID matched with the data identification ID does not exist in the configuration file, refusing to upload the data identification ID.
Optionally, before the receiving the behavior data sent by the client, the method further includes: loading the configuration file using an object of a system-level loader class; the method for triggering loadComplete comprises the steps of firstly converting the configuration file into JSON data, and then storing the JSON data into a logData variable; the judging whether a target preset identification ID matched with the data identification ID exists in the N preset identification IDs of the configuration file comprises the following steps: calling an Addlog method, and comparing the N preset identification IDs in the logData by traversing the N preset identification IDs so as to judge whether a target preset identification ID matched with the data identification ID exists; the acquiring of the target attribute information corresponding to the target preset identifier ID includes: and acquiring target attribute information corresponding to the target preset identification ID, and assigning the target attribute information as a variable Singledata.
Optionally, before obtaining the target preset identifier ID matched with the data identifier ID in the behavior data uploaded by the client, the method further includes: loading and analyzing the configuration file; initializing a reported data timer; the reported data timer is used for judging whether the time length of the latest information uploaded from the data container is greater than T; initializing a collected data manager; the data manager is used for receiving the behavior data sent by the client.
Optionally, the loading and parsing the configuration file includes: loading the configuration file using an object of a system-level loader class; and triggering the loadComplete method, firstly converting the configuration file into JSON data, and then storing the JSON data into a logData variable.
Optionally, the determining, according to the target attribute information, whether the behavior data needs to be uploaded immediately includes: using a preset variable Bolan as a variable for representing whether instant uploading is required, and using Bolan ═ singledata? Judging whether the behavior data needs to be uploaded immediately by using a true flash command; when the istimely is 1, representing that the real-time uploading is needed, wherein the command execution result is true; when the istimely is 0, representing that instant uploading is not needed, wherein the command execution result is a flash; wherein, the value of the istimely is preset in the target attribute information of the configuration file.
In a second aspect, a resource server is provided, including:
the acquisition unit is used for acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
the first judging unit is used for judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
the uploading unit is used for directly uploading the data identification ID and the target attribute information to the platform server if the behavior data needs to be uploaded instantly;
the adding unit is used for adding the data identification ID and the target attribute information to a data container if the behavior data does not need to be uploaded instantly;
the second judging unit is used for judging whether the quantity of the information in the data container is greater than M or not; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and the emptying unit is used for uploading the information in the data container to the platform server and emptying the data container if the quantity of the information in the data container is greater than M or the time length from the last information uploading of the data container to the platform server is greater than T.
In a third aspect, there is provided a resource server, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the program:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to the platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to the platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
according to the method, the resource server and the storage medium provided by the embodiment of the application, on one hand, the uploaded target attribute information replaces the uploaded behavior data, and the target attribute information is set in the configuration file in advance, so that the uniformity and regularity of the uploaded data can be effectively improved, the difficulty of subsequent data maintenance is reduced, the uploaded data can be modified by only modifying the configuration file, and the modification efficiency is improved. On the other hand, whether the behavior data needs to be uploaded immediately is judged according to the target attribute information; if necessary, directly uploading the data identification ID and the target attribute information to the platform server; if not, adding the data identification ID and the target attribute information to a data container; the files with low real-time requirements are stored in a data container, so that a large number of files needing to be uploaded in one time period are avoided, and network congestion and platform server CPU consumption are reduced; further, when the number of information in a data container is larger than M, or the time length of last information uploading from the data container is larger than T, the information in the data container is uploaded to the platform server, the data container is emptied, ordered uploading is further guaranteed by combining two uploading rules, and network congestion and CPU consumption of the platform server are reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a behavior data uploading method in an embodiment of the present application;
fig. 2 is a schematic structural diagram of a resource server according to a second embodiment of the present application;
fig. 3 is a schematic structural diagram of a resource server in a third embodiment of the present application;
fig. 4 is a schematic structural diagram of a computer-readable storage medium 400 according to a fourth embodiment of the present application.
Detailed Description
The embodiment of the application provides an uploading method of behavior data, a resource server and a storage medium, solves the technical problems that a flash player in the prior art is difficult to maintain and modify data and network congestion is easily caused and CPU consumption of a platform server is easily increased in a method for collecting behavior data, and achieves the technical effects of reducing the difficulty in maintaining and modifying data in the later period and reducing the network congestion and the CPU consumption of the platform server.
In order to solve the technical problems, the general idea of the embodiment of the application is as follows:
a behavior data uploading method is applied to a resource server and comprises the following steps:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to the platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
Specifically, on one hand, the uploading target attribute information replaces the uploading behavior data, the target attribute information is preset in the configuration file, the uniformity and regularity of the uploading data can be effectively improved, the difficulty of subsequent data maintenance is reduced, the uploading data can be modified according to the permission only by modifying the configuration file, and the modification efficiency is improved. On the other hand, whether the behavior data needs to be uploaded immediately is judged according to the target attribute information; if necessary, directly uploading the data identification ID and the target attribute information to the platform server; if not, adding the data identification ID and the target attribute information to a data container; the files with low real-time requirements are stored in a data container, so that a large number of files needing to be uploaded in one time period are avoided, and network congestion and platform server CPU consumption are reduced; further, when the number of information in a data container is larger than M, or the time length of last information uploading from the data container is larger than T, the information in the data container is uploaded to the platform server, the data container is emptied, ordered uploading is further guaranteed by combining two uploading rules, and network congestion and CPU consumption of the platform server are reduced.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
Example one
As shown in fig. 1, the present embodiment provides a behavior data uploading method, which is applied in a resource server, and includes:
step S101, acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
step S102, judging whether the behavior data needs to be uploaded instantly or not according to the target attribute information;
step S103, if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
step S104, if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
step S105, judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and step S106, if the quantity of the information in the data container is greater than M, or the time length from the last time of uploading the information in the data container to the platform server is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
It should be noted that, in the embodiment of the present application, the resource server may be a computer device, a cloud, or a group of computer devices, which is not limited herein. The client may be a desktop, a smart phone, or a tablet computer, which is not limited herein.
The detailed implementation steps of the behavior data uploading method provided by the present application are described in detail below with reference to fig. 1:
firstly, step S101 is executed to obtain a target preset identifier ID matching a data identifier ID in behavior data uploaded by a client and target attribute information corresponding to the preset identifier ID from a pre-stored configuration file.
In this embodiment, the method for acquiring the target preset identifier ID and the target attribute information includes:
and receiving behavior data sent by a client, wherein the behavior data comprises a data Identification (ID).
In this embodiment of the application, before the server receives the behavior data sent by the client, the method may further include:
loading and analyzing the configuration file;
initializing a reported data timer; the reported data timer is used for judging whether the time length of the latest information uploaded from the data container is greater than T; wherein, T is a positive number, and the unit can be seconds, minutes, and hours, which is not limited herein;
initializing a collected data manager; the data manager is used for receiving the behavior data sent by the client.
In this embodiment of the present application, the configuration file is preset in the server, and the configuration file needs to be loaded and analyzed before receiving behavior data sent by a client, where a specific loading and analyzing method is as follows:
loading the configuration file by using an object of a loader class of a system level;
for example: the configuration file is loaded by adopting the following codes:
loader＝new URLLoader()；
loader.addEventListener(Event.COMPLETE,loadComplete).
loader.load(urlrequest)；
the Urlrequest is a system interface network request, and the behavior data configuration can be requested through the method to complete the loading of the configuration file.
And then, loading the configuration file, triggering a loadComplete method after the configuration file is requested to be configured to behavior data, converting the configuration file into JSON data, and storing the JSON data into a logData variable.
Next, judging whether a target preset identification ID matched with the data identification ID exists in the N preset identification IDs of the configuration file; attribute information corresponding to each preset identification ID in the N preset identification IDs is also preset in the configuration file; n is a positive integer.
In this embodiment of the application, the target preset identifier ID matched with the data identifier ID may refer to a target preset identifier ID identical to the data identifier ID; or the target preset identification ID is equal to a calculation result obtained after the preset rule calculation is performed on the data identification ID, and the calculation may be any one or a combination of more of the following: addition, subtraction, multiplication, division, exponentiation, shifting, removing partial or sequential changes, and the like.
In the embodiment of the application, in the configuration file, a worker records in advance the IDs of behavior data to be collected, that is, the N preset identification IDs, and in a specific implementation process, the worker determines which IDs of data to be collected according to factors such as importance degree and data category, and records the ID in the configuration file.
In the configuration file, attribute information attribute corresponding to each preset identifier ID in the N preset identifier IDs is further correspondingly recorded, and in a specific implementation process, the attribute information may include any one or a combination of multiple pieces of information: the meaning of the behavior corresponding to the preset identification ID, the user operation corresponding to the preset identification ID, the data source corresponding to the preset identification ID, whether the data corresponding to the preset identification ID needs to be uploaded instantly or not, and the like. The staff can adopt the preset rule code or rule description as the attribute information to avoid the complexity, fragmentation and different expression modes of the attribute information, thereby facilitating the subsequent inquiry and management.
It should be noted that, the preset identification ID of each piece of behavior data to be uploaded is strictly configured in the configuration file in advance, and attribute information is set correspondingly.
In this embodiment of the application, the method for determining whether there is a target preset identifier ID matching the data identifier ID may specifically be:
assuming that the collected data manager is a datamanager, an Addlog method exists in the datamanager, the Addlog method takes the collected data identifier ID as a parameter and is used for collecting dotting data, and after the Addlog method is called, the Addlog method compares the data identifier ID with the N preset identifier IDs in the logData variable by traversing the N preset identifier IDs, so as to judge whether a target preset identifier ID matching the data identifier ID exists.
And if the configuration file has a target preset identification ID matched with the data identification ID, acquiring target attribute information corresponding to the target preset identification ID.
Specifically, the method of acquiring the target attribute information may be: and acquiring target attribute information corresponding to the target preset identification ID, and assigning the target attribute information as a variable Singledata. After the matched target preset identification ID is found by traversing the logData variable, the corresponding target attribute information needing to be collected is found according to the target preset identification ID, and the target attribute information is assigned as the variable Singledata. For example, traversing the logData variable and assigning as variable Singledata can be implemented using the following code:
then, uploading the data identifier ID and the target attribute information to a platform server, which may specifically be: and uploading the Singledata variable to a platform server.
In the implementation process of the application, the situation that a flash player reports a large amount of data to a platform server at the same time in a certain service scene is considered, which may cause a peak value of network communication between the flash player or a resource server and the platform server, and increase bandwidth cost, and meanwhile, after a certain time node of the platform server suddenly receives a large amount of data of the flash player, service logic is performed, so that the CPU consumption of the platform server is easily increased. Therefore, the application sets whether to upload data instantly or not according to the real-time attribute of the data, and the data which does not need to be uploaded instantly is uploaded temporarily through the data container of the resource server, so that the CPU consumption of the platform server is reduced, the network congestion is avoided, and the detailed description is as follows:
and step S102 is executed, and whether the behavior data need to be uploaded instantly is judged according to the target attribute information.
In a specific implementation process, when the target attribute information includes a description of a real-time attribute, it may be determined whether the behavior data needs to be uploaded immediately according to the target attribute information.
If the behavior data needs to be uploaded instantly, executing step S103, and directly uploading the data identification ID and the target attribute information to a platform server;
if the behavior data does not need to be uploaded instantly, executing step S104, and adding the data identification ID and the target attribute information to a data container;
specifically, because the behavior data of some service scenarios is time-efficient, it is necessary to upload data information in time for analysis and processing, that is, it is necessary to determine whether the target attribute information (e.g., Singledata variable) is a real-time attribute to determine whether the behavior data obtained this time needs to be uploaded in time.
For example, the istimely is a real-time attribute code configured in a behavior data file, when the behavior data needs to be uploaded in time, the istimely is 1, otherwise, the istimely is 0, so that the effective attribute of the behavior data can be modified only by modifying the value of the istimely of the configuration file in the later maintenance. Of course, other codes may be set to characterize the real-time attributes, and are not limited herein.
Assuming that a preset variable blan is used as a variable for representing whether instant uploading is required, the corresponding steps can be as follows: bolan ═ singledata > 0? Judging real-time attributes by using a flash command, representing that real-time uploading is required when the istimely is 1, and obtaining a true command execution result; when the istimely is 0, representing that instant uploading is not needed, wherein the command execution result is a flash; wherein, the value of the istimely is preset in the target attribute information of the configuration file.
And when Bolan is true, directly sending an http request to upload the data identification ID and the target attribute information (Singledata variable), and otherwise, adding the data identification ID and the target attribute information (Singledata variable) into a data container LogList.
Further, in order to avoid the jam of resource server and platform server that leads to is uploaded in the concentration of information in the data container, and further avoid platform server CPU peak value to consume too high, this application still sets up:
step S105, judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and step S106, if the quantity of the information in the data container is greater than M, or the time length from the last time of uploading the information in the data container to the platform server is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
Specifically, the method for judging whether the amount of information in the data container is larger than M comprises the following steps: in the previous operation, the program stores Singledata variable data in the LogList of the data container, and if the storage of the Singledata variable data is not increased in time, the processing causes the memory of the LogList to be increased infinitely, so the maximum length of the LogList is set to be N (for example, N is equal to 20), and when the length of the LogList exceeds N, the data in the LogList is sent one by one and emptied, so that the memory overflow is avoided. Let a variable needSeed characterize whether LogList length is exceeded, then with needSeed ═ LogList. False can be judged, and when the needSeed is true, the data in the LogList is sent one by one and is emptied.
For example, it may be set that each time a Singledata variable is uploaded, a corresponding data object in the LogList is cleaned up, a sendData method may be programmed to represent a specific method for uploading, parameters of the method are single behavior data objects, and specific codes are as follows:
when the LogList.length is larger than 0 by using the While function, the logic in the parenthesis is executed in a circulating way, the _dataobtains the first element of the LogList and calls the sendData method to upload the first element to the platform server, and the LogList.hit () function can delete the first element of the LogList, so that the LogList.length can be reduced by 1.
In a specific implementation process, the determination of whether the time length from the last time of uploading the information to the data container is greater than T is implemented by a previously initialized data reporting timer. Specifically, the reporting data timer may avoid excessive non-uploaded behavior data accumulated in the data container. Assume that the timer is Delaytimer. The time it triggers is T seconds (e.g. 120 seconds), then Delaytimer is new Timer (120 × 1000,0), and the second parameter is set to 0 when Delaytimer is initialized, so that Delaytimer can respond indefinitely. Meanwhile, the timer can execute uploading and emptying of the data in the data container each time the timer is triggered.
Specifically, the method and the device process the uploading of the non-timely data information in the data container by combining two judgment rules of the number of the information in the data container and the time length of the latest information uploading from the data container, can effectively avoid the blockage of a resource server and a platform server caused by the centralized uploading of the information in the data container, and further avoid the overhigh peak value consumption of a CPU (Central processing Unit) of the platform server.
In addition, if the target preset identification ID matched with the data identification ID does not exist in the configuration file, the data identification ID is refused to be uploaded. And uploading unimportant data is avoided, and the complexity of subsequent data management is reduced.
Specifically, in order to solve the problem that flash is difficult to maintain data collection in a large number of service scenes, a section of data collection logic is not inserted into each service node when behavior data are collected, behavior data are collected through a scheme of a configuration file and a data collection manager, namely, a preset identification ID and corresponding attribute information of each piece of data to be uploaded are written in the configuration file, the data collection manager reads corresponding target attribute information in the configuration file according to the data identification ID of the obtained behavior data, and complexity and fragmentation degree of the uploaded data are reduced by uploading the target attribute information, so that later maintenance is facilitated.
Further, whether the behavior data is the behavior data (with real-time property) needing to be uploaded in time is judged firstly, if the behavior data is the behavior data with real-time property, the behavior data is uploaded to a platform server, and if not, a non-timely scheme is executed: and firstly putting the attribute files which do not need to be uploaded immediately into a data container LogList for unified storage, and waiting for uploading to a platform server. And if the number of the behavior data containers exceeds N, uploading all data in the behavior data containers and emptying the behavior data containers, and meanwhile, setting the time length of each T of the timer to upload all data in the behavior data containers and emptying the behavior data containers. The two uploading rules are combined to process non-timely data, so that the phenomenon that the uploading peak value is too high, the network is blocked and the CPU consumption of the platform server is too high can be avoided.
Based on the same inventive concept, the application provides a device corresponding to the embodiment, which is detailed in the second embodiment.
Example two
As shown in fig. 2, the present embodiment provides a resource server, including:
an obtaining unit 201, configured to obtain, from a pre-stored configuration file, a target preset identifier ID that matches a data identifier ID in behavior data uploaded by a client, and target attribute information corresponding to the preset identifier ID;
a first judging unit 202, configured to judge whether the behavior data needs to be uploaded immediately according to the target attribute information;
an uploading unit 203, configured to directly upload the data identifier ID and the target attribute information to a platform server if the behavior data needs to be uploaded immediately;
an adding unit 204, configured to add the data identifier ID and the target attribute information to a data container if the behavior data does not need to be uploaded immediately;
a second judging unit 205, configured to judge whether the amount of information in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T;
an emptying unit 206, configured to upload information in the data container to the platform server and empty the data container if the amount of information in the data container is greater than M, or a time length from the last time that the information was uploaded by the data container is greater than T.
Since the resource server described in this embodiment is a device used for implementing an uploading method of behavior data in this embodiment, based on the method described in the first embodiment of this application, a person skilled in the art can understand a specific implementation manner of the resource server in this embodiment and various variations thereof, so that a detailed description of how the resource server implements the method in this embodiment of this application is omitted here. The equipment used by those skilled in the art to implement the methods in the embodiments of the present application is within the scope of the present application.
Based on the same inventive concept, the application provides a resource server corresponding to the third embodiment.
EXAMPLE III
As shown in fig. 3, the present embodiment provides a resource server, which includes a memory 310, a processor 320, and a computer program 311 stored in the memory 320 and running on the processor 320, wherein the processor 320 executes the computer program 311 to implement the following steps:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
In particular, when the processor 320 executes the computer program 311, any one of the first embodiment can be implemented.
Since the resource server described in this embodiment is a device used for implementing an uploading method of behavior data in this embodiment, based on the method described in the first embodiment of this application, a person skilled in the art can understand a specific implementation manner of the resource server in this embodiment and various variations thereof, so that a detailed description of how the resource server implements the method in this embodiment of this application is omitted here. The equipment used by those skilled in the art to implement the methods in the embodiments of the present application is within the scope of the present application.
Based on the same inventive concept, the application provides a storage medium corresponding to the fourth embodiment, which is described in detail in the fourth embodiment.
Example four
As shown in fig. 4, the present implementation provides a computer-readable storage medium 400, on which a computer program 411 is stored, which computer program 411, when being executed by a processor, realizes the following steps:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
In a specific implementation, when the computer program 411 is executed by a processor, any one of the first embodiment may be implemented.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A behavior data uploading method is applied to a resource server and is characterized by comprising the following steps:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file; the target attribute information includes any one or a combination of a plurality of pieces of information: the meaning of the behavior corresponding to the preset identification ID, the user operation corresponding to the preset identification ID, the data source corresponding to the preset identification ID and whether the data corresponding to the preset identification ID need to be uploaded instantly or not;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T; m is a positive integer, T is a positive number;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
2. The method of claim 1, wherein the obtaining of the target preset identification ID matched with the data identification ID in the behavior data uploaded by the client and the target attribute information corresponding to the preset identification ID comprises:
receiving the behavior data sent by a client, wherein the behavior data comprises the data identification ID;
judging whether the target preset identification ID matched with the data identification ID exists in the N preset identification IDs of the configuration file; attribute information corresponding to each preset identification ID in the N preset identification IDs is also preset in the configuration file; n is a positive integer;
and if a target preset identification ID matched with the data identification ID exists in the configuration file, acquiring the target attribute information corresponding to the target preset identification ID.
3. The method as claimed in claim 2, wherein said determining whether there is a target preset identification ID matching the data identification ID in the N preset identification IDs of the configuration file further comprises:
and if the target preset identification ID matched with the data identification ID does not exist in the configuration file, refusing to upload the data identification ID.
4. The method of claim 2,
before the receiving the behavior data sent by the client, the method further includes:
loading the configuration file using an object of a system-level loader class; the loadComplete method is triggered, the configuration file is converted into JSON data, and then the JSON data is stored in a logData variable;
the judging whether a target preset identification ID matched with the data identification ID exists in the N preset identification IDs of the configuration file comprises the following steps:
calling an Addlog method, and comparing the N preset identification IDs in the traversal logData variable with the data identification IDs so as to judge whether a target preset identification ID matched with the data identification IDs exists or not;
the acquiring of the target attribute information corresponding to the target preset identifier ID includes:
and acquiring target attribute information corresponding to the target preset identification ID, and assigning the target attribute information as a variable Singledata.
5. The method of claim 1, wherein before obtaining the target preset identification ID matching the data identification ID in the behavior data uploaded by the client, the method further comprises:
loading and analyzing the configuration file;
initializing a reported data timer; the reported data timer is used for judging whether the time length of the latest information uploaded from the data container is greater than T;
initializing a collected data manager; the data manager is used for receiving the behavior data sent by the client.
6. The method of claim 5, wherein the loading and parsing the configuration file comprises:
loading the configuration file using an object of a system-level loader class;
and the loadComplete method is triggered, the configuration file is converted into JSON data, and then the JSON data is stored in a logData variable.
7. The method of claim 1, wherein the determining whether the behavior data needs to be uploaded immediately according to the target attribute information comprises:
using a preset variable Bolan as a variable for representing whether instant uploading is required, and using Bolan ═ singledata? Judging whether the behavior data needs to be uploaded immediately by using a true flash command;
when the istimely is 1, representing that the real-time uploading is needed, wherein the command execution result is true;
when the istimely is 0, representing that instant uploading is not needed, wherein the command execution result is a flash;
wherein, the value of the istimely is preset in the target attribute information of the configuration file.
8. A resource server, comprising:
the acquisition unit is used for acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file; the target attribute information includes any one or a combination of a plurality of pieces of information: the meaning of the behavior corresponding to the preset identification ID, the user operation corresponding to the preset identification ID, the data source corresponding to the preset identification ID and whether the data corresponding to the preset identification ID need to be uploaded instantly or not;
the first judging unit is used for judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
the uploading unit is used for directly uploading the data identification ID and the target attribute information to a platform server if the behavior data needs to be uploaded instantly;
the adding unit is used for adding the data identification ID and the target attribute information to a data container if the behavior data does not need to be uploaded instantly;
the second judging unit is used for judging whether the quantity of the information in the data container is greater than M or not; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and the emptying unit is used for uploading the information in the data container to the platform server and emptying the data container if the quantity of the information in the data container is greater than M or the time length from the last information uploading of the data container to the platform server is greater than T.
9. A resource server comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file; the target attribute information includes any one or a combination of a plurality of pieces of information: the meaning of the behavior corresponding to the preset identification ID, the user operation corresponding to the preset identification ID, the data source corresponding to the preset identification ID and whether the data corresponding to the preset identification ID need to be uploaded instantly or not;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
10. A computer-readable storage medium, on which a computer program is stored, which program, when executed by a processor, carries out the steps of:
acquiring a target preset identification ID matched with a data identification ID in behavior data uploaded by a client and target attribute information corresponding to the preset identification ID from a prestored configuration file; the target attribute information includes any one or a combination of a plurality of pieces of information: the meaning of the behavior corresponding to the preset identification ID, the user operation corresponding to the preset identification ID, the data source corresponding to the preset identification ID and whether the data corresponding to the preset identification ID need to be uploaded instantly or not;
judging whether the behavior data needs to be uploaded immediately or not according to the target attribute information;
if the behavior data needs to be uploaded instantly, the data identification ID and the target attribute information are directly uploaded to a platform server;
if the behavior data does not need to be uploaded instantly, adding the data identification ID and the target attribute information to a data container;
judging whether the information quantity in the data container is greater than M; judging whether the time length from the last time of uploading the information of the data container is greater than T;
and if the quantity of the information in the data container is greater than M, or the time length from the last information uploading of the data container to the data container is greater than T, uploading the information in the data container to the platform server, and emptying the data container.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710506118.5A CN107483532B (en) | 2017-06-28 | 2017-06-28 | Behavior data uploading method, resource server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710506118.5A CN107483532B (en) | 2017-06-28 | 2017-06-28 | Behavior data uploading method, resource server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107483532A CN107483532A (en) | 2017-12-15 |
| CN107483532B true CN107483532B (en) | 2020-05-08 |
Family
ID=60594878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710506118.5A Active CN107483532B (en) | 2017-06-28 | 2017-06-28 | Behavior data uploading method, resource server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107483532B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108377270B (en) * | 2018-03-21 | 2021-02-26 | 耘申信息科技（上海）有限公司 | Method and system for reporting and managing equipment use data |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102984244A (en) * | 2012-11-21 | 2013-03-20 | 用友软件股份有限公司 | Uploading system and uploading method of bill data |
| CN103166827A (en) * | 2011-12-09 | 2013-06-19 | 北京神州泰岳软件股份有限公司 | Method and system for user behavior data reporting |
| CN104539739A (en) * | 2015-01-26 | 2015-04-22 | 浙江大学 | System, method and device for uploading files |
| WO2015196711A1 (en) * | 2014-06-24 | 2015-12-30 | 深圳市志友企业发展促进中心 | Method, device and system for tracking resource transmission |
| CN105610915A (en) * | 2015-12-15 | 2016-05-25 | 深圳市慧动创想科技有限公司 | Data reporting method and system |
| CN106878397A (en) * | 2017-01-21 | 2017-06-20 | 浙江沛宏网络科技有限公司 | A kind of WEB user behaviors feedback method and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101534415B (en) * | 2009-04-23 | 2011-03-16 | 天柏宽带网络科技（北京）有限公司 | Method and system for transmitting file in real time |
| CN102480481B (en) * | 2010-11-26 | 2015-01-07 | 腾讯科技（深圳）有限公司 | Method and device for improving security of product user data |
| CN105933368A (en) * | 2015-12-24 | 2016-09-07 | 中国银联股份有限公司 | User behavior data uploading method and device |
| CN106790380A (en) * | 2016-11-21 | 2017-05-31 | 广州华多网络科技有限公司 | Data reporting method and device |
-
2017
- 2017-06-28 CN CN201710506118.5A patent/CN107483532B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103166827A (en) * | 2011-12-09 | 2013-06-19 | 北京神州泰岳软件股份有限公司 | Method and system for user behavior data reporting |
| CN102984244A (en) * | 2012-11-21 | 2013-03-20 | 用友软件股份有限公司 | Uploading system and uploading method of bill data |
| WO2015196711A1 (en) * | 2014-06-24 | 2015-12-30 | 深圳市志友企业发展促进中心 | Method, device and system for tracking resource transmission |
| CN104539739A (en) * | 2015-01-26 | 2015-04-22 | 浙江大学 | System, method and device for uploading files |
| CN105610915A (en) * | 2015-12-15 | 2016-05-25 | 深圳市慧动创想科技有限公司 | Data reporting method and system |
| CN106878397A (en) * | 2017-01-21 | 2017-06-20 | 浙江沛宏网络科技有限公司 | A kind of WEB user behaviors feedback method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107483532A (en) | 2017-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106302632B (en) | Downloading method of basic mirror image and management node | |
| CN106201661B (en) | Method and apparatus for elastic telescopic cluster virtual machine | |
| US20130232254A1 (en) | Cloud resource utilization management | |
| CN107872402A (en) | The method, apparatus and electronic equipment of global traffic scheduling | |
| WO2017166643A1 (en) | Method and device for quantifying task resources | |
| CN111262726B (en) | Configuration information updating method and device and computer readable storage medium | |
| CN104410543A (en) | An automatic testing method and system based on cloud resource | |
| CN111858067B (en) | Data processing method and device | |
| US10891164B2 (en) | Resource setting control device, resource setting control system, resource setting control method, and computer-readable recording medium | |
| CN106685894B (en) | Risk identification method, device and system | |
| CN107493315B (en) | Behavior data collection method, resource server and storage medium | |
| CN110933178A (en) | Method for adjusting node configuration in cluster system and server | |
| CN109756533B (en) | Mirror image acceleration method and device and server | |
| CN111258746A (en) | Resource allocation method and service equipment | |
| CN110009347B (en) | Block chain transaction information auditing method and device | |
| CN107483532B (en) | Behavior data uploading method, resource server and storage medium | |
| CN111124791A (en) | System testing method and device | |
| US8667141B2 (en) | Method and system for handling load on a service component in a network | |
| US11216352B2 (en) | Method for automatically analyzing bottleneck in real time and an apparatus for performing the method | |
| CN111683280A (en) | Video processing method and device and electronic equipment | |
| CN110716938A (en) | Data aggregation method and device, storage medium and electronic device | |
| CN109462510B (en) | CDN node quality evaluation method and device | |
| CN108463813A (en) | A kind of method and apparatus carrying out data processing | |
| CN115794382A (en) | Data processing method, device and nonvolatile storage medium | |
| CN109257256A (en) | Apparatus monitoring method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230718 Address after: 332000 Room 201, Building 1, next to Sanli Primary School, Sanli Village, Baishuihu Street, Xunyang District, Jiujiang City, Jiangxi Province Patentee after: Jiangxi Minghang Information Technology Co.,Ltd. Address before: 430000 East Lake Development Zone, Wuhan City, Hubei Province, No. 1 Software Park East Road 4.1 Phase B1 Building 11 Building Patentee before: WUHAN DOUYU NETWORK TECHNOLOGY Co.,Ltd. |