CN115292330A - Data collection method and device - Google Patents
Data collection method and device Download PDFInfo
- Publication number
- CN115292330A CN115292330A CN202211186783.8A CN202211186783A CN115292330A CN 115292330 A CN115292330 A CN 115292330A CN 202211186783 A CN202211186783 A CN 202211186783A CN 115292330 A CN115292330 A CN 115292330A
- Authority
- CN
- China
- Prior art keywords
- user
- data
- information
- acquiring
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a data collection method and a device, wherein the method comprises the following steps: acquiring configuration information configured by a user through a configured GUI interface; monitoring user behaviors, and acquiring flow data of the user behaviors in real time; processing and integrating the configuration information and the flow data to obtain integrated data; distributing corresponding target indexes for the integrated data according to the access party of the user behavior; and storing the integrated data into a database according to the target index. Therefore, the method can automatically collect data, the collected data is accurate, the method is suitable for the existing service scene, and the flexibility is good.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data collection method and apparatus.
Background
At present, various current products are a vital item for collecting and analyzing user behavior data, which directly reflects whether the design of the products meets the user requirements, obtains user groups, and plays a very important role in improving user experience and further transforming product revenues. The existing data collection method usually collects data through an open source tool with a single function and cannot be attached to the existing business scene, so that the collected data are inaccurate and poor in flexibility.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data collection method and apparatus, which can automatically collect data, the collected data is accurate, and the flexibility is good when the data is attached to an existing service scene.
A first aspect of an embodiment of the present application provides a data collection method, including:
acquiring configuration information configured by a user through a configured GUI interface;
monitoring user behaviors, and acquiring flow data of the user behaviors in real time;
processing and integrating the configuration information and the flow data to obtain integrated data;
distributing a corresponding target index for the integrated data according to the access party of the user behavior;
and storing the integrated data into a database according to the target index.
In the implementation process, the method can preferentially acquire the configuration information configured by the user through the configured GUI interface; monitoring user behaviors and acquiring flow data of the user behaviors in real time; then, processing and integrating the configuration information and the flow data to obtain integrated data; distributing corresponding target indexes for the integrated data according to the access party of the user behavior; and finally, storing the integrated data into a database according to the target index. Therefore, by implementing the implementation mode, the data can be automatically collected, the collected data is accurate, the existing business scene is attached, and the flexibility is good.
Further, the configuration information at least comprises a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry and a probe identification.
Further, the processing and integrating the configuration information and the flow data to obtain integrated data includes:
acquiring a current probe identifier in the flow data according to the configuration information;
acquiring user behavior information and scene module information according to the current probe identification;
analyzing the flow data according to the configuration information to obtain user account information;
acquiring detailed information of a login person according to the user account information;
acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
Further, the method further comprises:
matching a behavior tag for the consolidated data;
and generating a user portrait according to the behavior tag, and storing the user portrait.
Further, the method further comprises:
receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
acquiring query data from the database according to the query request;
and generating a data report according to the query data.
A second aspect of the embodiments of the present application provides a data collection device, including:
the acquisition unit is used for acquiring configuration information configured by a user through a configured GUI interface;
the monitoring unit is used for monitoring user behaviors;
the acquisition unit is used for acquiring the flow data of the user behavior in real time;
the processing integration unit is used for processing and integrating the configuration information and the flow data to obtain integrated data;
the distribution unit is used for distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
and the storage unit is used for storing the integrated data into a database according to the target index.
In the implementation process, the device can acquire the configuration information configured by the user through the configured GUI interface through the acquisition unit; monitoring user behaviors through a monitoring unit; acquiring flow data of user behaviors in real time through an acquisition unit; processing and integrating the configuration information and the flow data through a processing and integrating unit to obtain integrated data; distributing corresponding target indexes for the integrated data according to the access party of the user behavior through a distribution unit; and storing the integrated data into a database according to the target index through a storage unit. Therefore, the implementation of the implementation mode can automatically collect data, the collected data is accurate, the existing business scene is attached, and the flexibility is good.
Further, the configuration information at least comprises a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry and a probe identification.
Further, the processing integration unit comprises:
the acquisition subunit is used for acquiring the current probe identifier in the flow data according to the configuration information; acquiring user behavior information and scene module information according to the current probe identification;
the analysis subunit is used for analyzing the flow data according to the configuration information to obtain user account information;
the acquisition subunit is further configured to acquire detailed information of a login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
Further, the data collection apparatus further includes:
a matching unit for matching behavior tags for the integration data;
the storage unit is further used for generating a user portrait according to the behavior tag and storing the user portrait.
Further, the data collection apparatus further includes:
the receiving unit is used for receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
the query unit is used for acquiring query data from the database according to the query request;
and the generating unit is used for generating a data report according to the query data.
A third aspect of embodiments of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to cause the electronic device to execute the data collection method described in any one of the first aspect of embodiments of the present application.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the data collection method according to any one of the first aspect of the embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of a data collection method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart diagram of another data collection method provided by an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a data collection device according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of another data collection device provided in an embodiment of the present application;
fig. 5 is a system architecture diagram of a system according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a data collection method according to an embodiment of the present application. Wherein, the data collection method comprises the following steps:
s101, acquiring configuration information configured by a user through a configured GUI interface.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
S102, monitoring user behaviors and acquiring flow data of the user behaviors in real time.
And S103, processing and integrating the configuration information and the flow data to obtain integrated data.
And S104, distributing corresponding target indexes for the integrated data according to the access party of the user behavior.
And S105, storing the integrated data into a database according to the target index.
In this embodiment, the execution subject of the method may be a computing device such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
Therefore, the data collection method described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design development; meanwhile, the configuration flexibility can be improved, so that the configuration of the basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 2
Referring to fig. 2, fig. 2 is a flow chart illustrating a data collection method according to an embodiment of the present application. Wherein, the data collection method comprises the following steps:
s201, obtaining configuration information configured by a user through a configured GUI interface.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
In this embodiment, a Graphical User Interface (GUI) refers to a computer operation User Interface displayed in a Graphical manner.
In this embodiment, the configuration information includes a function module; functional module types, such as: static pages, menu pages, timed tasks, etc.; user behavior types, such as: page access, card occupation, account occupation, and the like; the function module associates user behaviors such as: the module A associates A1/A2 user behaviors and the like; a function execution entry, namely a request path, and a back-end interceptor identifies the use of the request; the probe identification, i.e. the globally unique character string, is used for identifying the identification of the execution entry in the non-filter scene.
S202, monitoring user behaviors and acquiring flow data of the user behaviors in real time.
In this embodiment, the user behavior is customized by the user, for example: page access, jump in to the current page from that page, query, file submission, register card, activate card, and the like.
And S203, acquiring the current probe identification in the flow data according to the configuration information.
And S204, acquiring user behavior information and scene module information according to the current probe identification.
And S205, analyzing the flow data according to the configuration information to obtain user account information.
And S206, acquiring the detailed information of the login person according to the user account information.
And S207, acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person.
In this embodiment, the integration data at least includes menu/function module information, user operation behavior, user information, and user behavior trigger time.
In this embodiment, the processing logic of the integration process is as follows: when the system intercepts the flow, the probe identification in the flow is obtained, and the corresponding user behavior, scene module and other information are correlated through the probe; meanwhile, the login state of the user is analyzed to obtain account information of the user, and then a CMS (Content Management System) related party System is used for obtaining detailed information of a login person through the login account of the user; and finally, identifying corresponding index information through the functional module. It can be seen that the information stored in the index after final processing is as follows:
(1) menu/function module information (e.g., name, number);
(2) user action (e.g., access or specific action);
(3) user information (e.g., name, account number, department, group, etc.);
(4) a trigger time.
And S208, distributing corresponding target indexes for the integrated data according to the access party of the user behavior.
S209, storing the integrated data into a database according to the target index.
And S210, matching behavior labels for the integrated data.
S211, generating a user portrait according to the behavior tag, and storing the user portrait in a database.
In this embodiment, the granularity of the behavior tag in the system to which the method is applied is relatively coarse, and is primarily determined primarily according to the user behavior. The user behavior comprises: access, operation, occupation; the three categories are three latitudes provided by the currently docked platform; and finally mapping (drawing) three types of users: visitors, general guest groups, deep users;
(1) visitor: only browsing people whose system function does not do any operation;
(2) general guest groups: in particular to a person who needs to rely on the internal functions of the platform to achieve the self appeal;
(3) depth user: in particular to a person who completely relies on the data provided by the platform and possesses the data.
And S212, receiving a query request input by a user.
In this embodiment, the query request includes a query dimension, and the query dimension includes one or more of a department dimension, a personal dimension, a grouping dimension, a scene dimension, a group dimension, and a module dimension.
And S213, acquiring query data from the database according to the query request.
And S214, generating a data report according to the query data.
In this embodiment, the method may implement configuration, collection, storage, calculation, and visualization according to a modular development module by using a modular design development concept, and finally combine an engine system for completing user behavior collection (calculation). Referring to fig. 5, fig. 5 shows a system architecture diagram of the system.
In this embodiment, the modules may communicate with each other through an API network, so as to implement interactive transmission on data and high-cohesion low-coupling between systems. Meanwhile, the system provides configuration capability, maintains relevant basic configuration, provides a visual GUI interface, and is more convenient and faster. An API (Application Program Interface) is defined as a standard set of commands and information that an Application can use to exchange information with a computer operating system.
In this embodiment, the method can use an autonomic development collector engine to provide multiple data collection modes, pouring in business logic zeros; meanwhile, the collector engine provides various different docking modes, and a dockee can conveniently and quickly access the collector.
In the embodiment, the method can also realize the storage of large data volume and high-efficiency aggregation calculation by means of the mass storage and strong calculation capability of the ES; meanwhile, for a calculation result, a visual interface display is provided, and generation of various data reports is supported; in addition, the user images can be dynamically aggregated into a single user image according to the user behavior data.
For example, the engine system applied by the method can be divided into the following modules: the system comprises a basic configuration module, a cache (cache) preheating module, a collector, a storage/calculation unit and an operation view report.
(1) A basic configuration module: the module is basic information of the whole system, and a user needs to maintain the module in advance through a configured GUI (graphical user interface); data collection at the upper layer needs to be associated with configuration at the lower layer, and if configuration information is not associated, the system automatically skips the collection process. The method comprises the following specific steps:
a functional module: the module is the definition of a butt-joint system module and mainly comprises the name and the number of the module and the priority of the internal function of the module; priority is a protection mechanism of the system, namely: when the data volume is overloaded, the purpose of current-limiting protection can be achieved according to the priority of the module, the higher the priority is, the system is preferentially ensured, and the risk of data loss can exist when the priority is low;
module classification: for module classification, the collector can match different collection modes according to different classifications, and the system defines three categories at present: the system comprises a static page, a functional module and a timing task module, wherein if a new category needs to be expanded, the new category can be added through a back management page;
and (4) behavior classification: for distinguishing user's operation behavior, for example: access, commit, delete, query, jump, etc.;
marking a probe: the system uses a 10-bit random character string calculated by a hash algorithm through module numbering and classification (modules and behaviors), and guarantees global uniqueness; the probe is a unique identifier for distinguishing different scene classifications by the collector and is also a tangent point of the collector;
associating the functional module with the module class and the behavior class (1 n), generating the corresponding probe identification through the buttons provided on the page, and the underlying configuration is completed, if any.
(2) Cache (Cache) warm-up: in order to improve the performance of configuration reading, a cache module is developed by a system through an open source component Caffeine, after a project is started each time, basic configuration is automatically loaded into a cache, the change of a configuration item is monitored in real time, and the cache is refreshed in time.
(3) A collector: monitoring the triggering of user behaviors, acquiring user flow in real time, synchronously matching configuration items in a cache, processing and integrating configuration information and flow data, and pushing the configuration information and the flow data to a storage unit; wherein, the collector provides two kinds of butt joint modes altogether: a. deploying the service of the collector, realizing data collection by providing an http request interface, and b, packaging the collector into an independent jar package integrally, and synchronously sharing a warehouse, so that a user can access to the project of the user for use in an sdk (Software Development Kit) mode. Four different collectors are as follows:
http (hypertext Transfer Protocol) interface: the interface can be directly called by the butt-joint party, and the corresponding data is directly transmitted to the storage unit;
an interceptor: intercepting a user network request by using an interceptor mechanism of a springMVC frame, and asynchronously transmitting request data to a core processing module for processing, wherein the interceptor is mainly suitable for static resources; the springMVC framework is a Java framework and provides a full-function MVC module for constructing a Web application program.
Cutting into noodles: the section is a transverse development mode in the technical field of Java, is realized by introducing an Aspect frame with an open source, and is mainly used for zero-entering of service logic in the scenes of functional modules and complex services;
event monitoring: in the system, a set of service mechanism for monitoring and event sending is built, developers call an API for sending events at the position of codes needing to be triggered and collected, and meanwhile, when the system is started, a synchronous probe is embedded between codes in each row, and when the codes execute event sending logic, the probe is packaged in an event object through the API and sent to a server.
(4) A storage/calculation unit: the user behavior data are stored by relying on the processing capacity of the elastic search massive database, different access parties distribute different indexes to store corresponding data, data isolation is achieved, and meanwhile the high availability of the environment is guaranteed by using a cluster deployment mode; processing aggregation and real-time calculation of data are realized based on an API provided by es, an interface is packaged to provide service capability, an operation data background is butted, and a background page is rendered in real time;
(5) reporting an operation view: based on the result of the storage unit aggregation, a set of operation data pages is independently developed, the behavior data of the user can be checked in real time, various behavior tags are set, and the client portrait is aggregated through the tags; meanwhile, a data report can be provided, real-time aggregation can be performed according to different conditions and latitudes selected by a user, and a downloading and exporting entrance can be provided.
In this embodiment, the execution subject of the method may be a computing device such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
Therefore, by implementing the data collection method described in the embodiment, the system coupling degree can be reduced and the system stability can be improved based on the idea of modular design and development; meanwhile, the configuration flexibility can be improved, so that the configuration of basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 3
Please refer to fig. 3, fig. 3 is a schematic structural diagram of a data collection device according to an embodiment of the present disclosure. As shown in fig. 3, the data collection apparatus includes:
an obtaining unit 310, configured to obtain configuration information configured by a user through a configured GUI interface;
a monitoring unit 320, configured to monitor user behavior;
an obtaining unit 330, configured to obtain flow data of a user behavior in real time;
a processing integration unit 340, configured to process and integrate the configuration information and the flow data to obtain integrated data;
an allocating unit 350, configured to allocate a corresponding target index to the integrated data according to an access party of the user behavior;
the storage unit 360 is used for storing the integrated data into the database according to the target index.
In this embodiment, for the explanation of the data collection device, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the data collection device described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design development; meanwhile, the configuration flexibility can be improved, so that the configuration of basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 4
Referring to fig. 4, fig. 4 is a schematic structural diagram of a data collection device according to an embodiment of the present disclosure. As shown in fig. 4, the data collection apparatus includes:
an obtaining unit 310, configured to obtain configuration information configured by a user through a configured GUI interface;
a monitoring unit 320, configured to monitor user behavior;
an obtaining unit 330, configured to obtain traffic data of a user behavior in real time;
a processing integration unit 340, configured to process and integrate the configuration information and the flow data to obtain integrated data;
an allocating unit 350, configured to allocate a corresponding target index to the integrated data according to an access party of the user behavior;
the storage unit 360 is used for storing the integrated data into the database according to the target index.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
As an alternative embodiment, the process integration unit 340 includes:
an obtaining subunit 341, configured to obtain, according to the configuration information, a current probe identifier inside the flow data; acquiring user behavior information and scene module information according to the current probe identification;
the parsing subunit 342 is configured to parse the traffic data according to the configuration information to obtain user account information;
the obtaining subunit 341 is further configured to obtain detailed information of the login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
As an optional implementation, the data collection apparatus further comprises:
a matching unit 370 for matching behavior tags for the integration data;
and the storage unit 360 is also used for generating a user portrait according to the behavior tag and storing the user portrait.
As an optional implementation, the data collection apparatus further comprises:
a receiving unit 380, configured to receive a query request input by a user; the query request comprises query dimensions, wherein the query dimensions comprise one or more of department dimensions, individual dimensions, grouping dimensions, scene dimensions, group dimensions and module dimensions;
a query unit 390, configured to obtain query data from a database according to a query request;
a generating unit 400, configured to generate a data report according to the query data.
In this embodiment, for the explanation of the data collection device, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the data collection device described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design and development; meanwhile, the configuration flexibility can be improved, so that the configuration of the basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
An embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the data collection method in embodiment 1 or embodiment 2 of the present application.
An embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions execute the data collection method in embodiment 1 or embodiment 2 of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A method of data collection, comprising:
acquiring configuration information configured by a user through a configured GUI interface;
monitoring user behaviors, and acquiring flow data of the user behaviors in real time;
processing and integrating the configuration information and the flow data to obtain integrated data;
distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
and storing the integrated data into a database according to the target index.
2. The data collection method of claim 1, wherein the configuration information includes at least a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry, and a probe identification.
3. The data collection method of claim 1, wherein the processing and integrating the configuration information and the flow data to obtain integrated data comprises:
acquiring a current probe identifier in the flow data according to the configuration information;
acquiring user behavior information and scene module information according to the current probe identification;
analyzing the flow data according to the configuration information to obtain user account information;
acquiring detailed information of a login person according to the user account information;
acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
4. The data collection method of claim 1, further comprising:
matching a behavior tag for the consolidated data;
and generating a user portrait according to the behavior tag, and storing the user portrait.
5. The data collection method of claim 1, further comprising:
receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
acquiring query data from the database according to the query request;
and generating a data report according to the query data.
6. A data collection device, the data collection device comprising:
the acquisition unit is used for acquiring configuration information configured by a user through a configured GUI interface;
the monitoring unit is used for monitoring user behaviors;
the acquisition unit is used for acquiring the flow data of the user behavior in real time;
the processing integration unit is used for processing and integrating the configuration information and the flow data to obtain integrated data;
the distribution unit is used for distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
and the storage unit is used for storing the integrated data into a database according to the target index.
7. The data collection device of claim 6, wherein the process integration unit comprises:
the acquisition subunit is used for acquiring the current probe identifier in the flow data according to the configuration information; acquiring user behavior information and scene module information according to the current probe identification;
the analysis subunit is used for analyzing the flow data according to the configuration information to obtain user account information;
the obtaining subunit is further configured to obtain detailed information of a login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
8. The data collection device of claim 6, further comprising:
the matching unit is used for matching behavior labels for the integrated data;
the storage unit is further used for generating a user portrait according to the behavior tag and storing the user portrait.
9. An electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to cause the electronic device to perform the data collection method of any of claims 1 to 5.
10. A readable storage medium having stored therein computer program instructions which, when read and executed by a processor, perform the data collection method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186783.8A CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186783.8A CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115292330A true CN115292330A (en) | 2022-11-04 |
| CN115292330B CN115292330B (en) | 2022-12-20 |
Family
ID=83833943
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211186783.8A Active CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115292330B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563190A (en) * | 2020-04-07 | 2020-08-21 | 中国电子科技集团公司第二十九研究所 | Multi-dimensional analysis and supervision method and system for user behaviors of regional network |
| CN112104523A (en) * | 2020-09-11 | 2020-12-18 | 中国联合网络通信集团有限公司 | Detection method, device and equipment for flow transparent transmission and storage medium |
| CN114461644A (en) * | 2022-01-30 | 2022-05-10 | 中国农业银行股份有限公司 | Data acquisition method and device, electronic equipment and storage medium |
| CN115001967A (en) * | 2022-05-30 | 2022-09-02 | 平安科技（深圳）有限公司 | Data acquisition method and device, electronic equipment and storage medium |
-
2022
- 2022-09-28 CN CN202211186783.8A patent/CN115292330B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563190A (en) * | 2020-04-07 | 2020-08-21 | 中国电子科技集团公司第二十九研究所 | Multi-dimensional analysis and supervision method and system for user behaviors of regional network |
| CN112104523A (en) * | 2020-09-11 | 2020-12-18 | 中国联合网络通信集团有限公司 | Detection method, device and equipment for flow transparent transmission and storage medium |
| CN114461644A (en) * | 2022-01-30 | 2022-05-10 | 中国农业银行股份有限公司 | Data acquisition method and device, electronic equipment and storage medium |
| CN115001967A (en) * | 2022-05-30 | 2022-09-02 | 平安科技（深圳）有限公司 | Data acquisition method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115292330B (en) | 2022-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2343537C2 (en) | Computer search with help of associative links | |
| US20140101134A1 (en) | System and method for iterative analysis of information content | |
| CN109840782A (en) | Clicking rate prediction technique, device, server and storage medium | |
| Ji et al. | Epidemic outbreak and spread detection system based on twitter data | |
| CN111405030B (en) | Message pushing method and device, electronic equipment and storage medium | |
| US10540386B2 (en) | Method for processing and displaying real-time social data on map | |
| CN115422169B (en) | Data warehouse construction method and device based on commercial advertisement scene | |
| CN109977296A (en) | A kind of information-pushing method, device, equipment and storage medium | |
| CN110300084A (en) | A kind of IP address-based portrait method and apparatus | |
| JPWO2011043429A1 (en) | Information management apparatus, data processing method thereof, and computer program | |
| CN115292330B (en) | Data collection method and device | |
| CN109815278A (en) | A kind of method for exhibiting data and its equipment, storage medium, electronic equipment | |
| CN115018473A (en) | Service processing method, device, storage medium and equipment | |
| Antunes et al. | Semantic-based publish/subscribe for M2M | |
| Kaufhold et al. | Big data and multi-platform social media services in disaster management | |
| Kaufhold et al. | Cross-Media Usage of Social Big Data for Emergency Services and Volunteer Communities: Approaches, Development and Challenges of Multi-Platform Social Media Services | |
| CN112230902A (en) | Software development method and system based on nail customization | |
| CN109614566A (en) | Traffic source analysis method, system and device | |
| Kpiebaareh et al. | User-connection behaviour analysis in service management using bipartite labelled property graph | |
| CN111466102B (en) | Method, system and apparatus for providing a set of context keywords for a communication event in a multiple communication platform environment | |
| Farooq et al. | Smartphone based interface for epidemic surveillance system | |
| WO2016199401A1 (en) | Information processing device, information analysis device, information processing method, information analysis method, information processing program, and information analysis program | |
| CN115718825B (en) | Method and device for determining duration label and electronic equipment | |
| Gerontini et al. | Large scale geospatial analysis on mobile application usage | |
| US20220038892A1 (en) | Mathematical Summaries of Telecommunications Data for Data Analytics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |