CN115292330B - Data collection method and device - Google Patents
Data collection method and device Download PDFInfo
- Publication number
- CN115292330B CN115292330B CN202211186783.8A CN202211186783A CN115292330B CN 115292330 B CN115292330 B CN 115292330B CN 202211186783 A CN202211186783 A CN 202211186783A CN 115292330 B CN115292330 B CN 115292330B
- Authority
- CN
- China
- Prior art keywords
- user
- data
- information
- acquiring
- behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a data collection method and a device, wherein the method comprises the following steps: acquiring configuration information configured by a user through a configured GUI interface; monitoring user behaviors, and acquiring flow data of the user behaviors in real time; processing and integrating the configuration information and the flow data to obtain integrated data; distributing corresponding target indexes for the integrated data according to an access party of the user behavior; and storing the integrated data into a database according to the target index. Therefore, the method can automatically collect data, the collected data is accurate, the method is attached to the existing business scene, and the flexibility is good.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data collection method and apparatus.
Background
At present, various current products are a vital item for collecting and analyzing user behavior data, which directly reflects whether the design of the products meets the user requirements, obtains user groups, and plays a very important role in improving user experience and further transforming product revenues. The existing data collection method usually collects data through an open source tool with a single function and cannot be attached to the existing business scene, so that the collected data are inaccurate and poor in flexibility.
Disclosure of Invention
The embodiment of the application aims to provide a data collection method and device, which can automatically collect data, are accurate in collected data, fit with the existing business scene and are good in flexibility.
A first aspect of an embodiment of the present application provides a data collection method, including:
acquiring configuration information configured by a user through a configured GUI interface;
monitoring user behaviors, and acquiring flow data of the user behaviors in real time;
processing and integrating the configuration information and the flow data to obtain integrated data;
distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
and storing the integrated data into a database according to the target index.
In the implementation process, the method can preferentially acquire the configuration information configured by the user through the configured GUI interface; monitoring user behaviors and acquiring flow data of the user behaviors in real time; then, processing and integrating the configuration information and the flow data to obtain integrated data; distributing corresponding target indexes for the integrated data according to the access party of the user behavior; and finally, storing the integrated data into a database according to the target index. Therefore, the implementation of the implementation mode can automatically collect data, the collected data is accurate, the existing business scene is attached, and the flexibility is good.
Further, the configuration information at least comprises a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry and a probe identification.
Further, the processing and integrating the configuration information and the flow data to obtain integrated data includes:
acquiring a current probe identifier in the flow data according to the configuration information;
acquiring user behavior information and scene module information according to the current probe identification;
analyzing the flow data according to the configuration information to obtain user account information;
acquiring detailed information of a login person according to the user account information;
acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
Further, the method further comprises:
matching a behavior tag for the consolidated data;
and generating a user portrait according to the behavior tag, and storing the user portrait.
Further, the method further comprises:
receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
acquiring query data from the database according to the query request;
and generating a data report according to the query data.
A second aspect of the embodiments of the present application provides a data collection device, including:
the acquisition unit is used for acquiring configuration information configured by a user through a configured GUI interface;
the monitoring unit is used for monitoring user behaviors;
the acquisition unit is used for acquiring the flow data of the user behavior in real time;
the processing integration unit is used for processing and integrating the configuration information and the flow data to obtain integrated data;
the distribution unit is used for distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
and the storage unit is used for storing the integrated data into a database according to the target index.
In the implementation process, the device can acquire the configuration information configured by the user through the configured GUI interface through the acquisition unit; monitoring user behaviors through a monitoring unit; acquiring flow data of user behaviors in real time through an acquisition unit; processing and integrating the configuration information and the flow data through a processing and integrating unit to obtain integrated data; distributing corresponding target indexes for the integrated data according to the access party of the user behavior through a distribution unit; and storing the integrated data into a database according to the target index through a storage unit. Therefore, the implementation of the implementation mode can automatically collect data, the collected data is accurate, the existing business scene is attached, and the flexibility is good.
Further, the configuration information at least comprises a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry and a probe identification.
Further, the processing integration unit comprises:
the acquisition subunit is used for acquiring the current probe identifier in the flow data according to the configuration information; acquiring user behavior information and scene module information according to the current probe identification;
the analysis subunit is used for analyzing the flow data according to the configuration information to obtain user account information;
the obtaining subunit is further configured to obtain detailed information of a login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
Further, the data collection apparatus further includes:
the matching unit is used for matching behavior labels for the integrated data;
the storage unit is further used for generating a user portrait according to the behavior tag and storing the user portrait.
Further, the data collection apparatus further includes:
the receiving unit is used for receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
the query unit is used for acquiring query data from the database according to the query request;
and the generating unit is used for generating a data report according to the query data.
A third aspect of embodiments of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the data collection method described in any one of the first aspect of embodiments of the present application.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the data collection method according to any one of the first aspect of the embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of a data collection method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart diagram illustrating another data collection method provided by an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a data collection device according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of another data collection device provided in an embodiment of the present application;
fig. 5 is a system architecture diagram of a system according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a data collection method according to an embodiment of the present application. Wherein, the data collection method comprises the following steps:
s101, acquiring configuration information configured by a user through a configured GUI interface.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
S102, monitoring user behaviors and acquiring flow data of the user behaviors in real time.
And S103, processing and integrating the configuration information and the flow data to obtain integrated data.
And S104, distributing corresponding target indexes for the integrated data according to the access party of the user behavior.
And S105, storing the integrated data into a database according to the target index.
In this embodiment, the execution subject of the method may be a computing device such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
Therefore, by implementing the data collection method described in the embodiment, the system coupling degree can be reduced and the system stability can be improved based on the idea of modular design and development; meanwhile, the configuration flexibility can be improved, so that the configuration of the basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 2
Referring to fig. 2, fig. 2 is a flow chart illustrating a data collection method according to an embodiment of the present application. Wherein, the data collection method comprises the following steps:
s201, obtaining configuration information configured by a user through a configured GUI interface.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
In this embodiment, a Graphical User Interface (GUI) refers to a computer operation User Interface displayed in a Graphical manner.
In this embodiment, the configuration information includes a function module; functional module types, such as: static pages, menu pages, timed tasks, etc.; user behavior types, such as: page access, card occupation, account occupation, and the like; the function module associates user behaviors such as: the module A associates A1/A2 user behaviors and the like; the function execution entrance is a request path, and the back-end interceptor identifies the request for use; the probe identification, i.e. the globally unique character string, is used for identifying the identification of the execution entry in the non-filter scene.
S202, monitoring user behaviors and acquiring flow data of the user behaviors in real time.
In this embodiment, the user behavior is customized by the user, for example: page access, jump in to the current page from that page, query, file submission, register card, activate card, and the like.
And S203, acquiring the current probe identification in the flow data according to the configuration information.
And S204, acquiring user behavior information and scene module information according to the current probe identification.
And S205, analyzing the flow data according to the configuration information to obtain user account information.
And S206, acquiring the detailed information of the login person according to the user account information.
And S207, acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person.
In this embodiment, the integration data at least includes menu/function module information, user operation behavior, user information, and user behavior trigger time.
In this embodiment, the processing logic of the integrated process is as follows: when the system intercepts the flow, the probe identification in the flow is obtained, and the corresponding user behavior, scene module and other information are correlated through the probe; meanwhile, the login state of the user is analyzed to obtain account information of the user, and then a CMS (Content Management System) related party System is used for obtaining detailed information of a login person through the login account of the user; and finally, identifying corresponding index information through the functional module. It can be seen that the information stored in the index after final processing is as follows:
(1) menu/function module information (e.g., name, number);
(2) user action (e.g., access or specific action);
(3) user information (e.g., name, account number, department, group, etc.);
(4) and (4) triggering time.
And S208, distributing corresponding target indexes for the integrated data according to the access party of the user behavior.
And S209, storing the integrated data into a database according to the target index.
And S210, matching behavior labels for the integrated data.
S211, generating a user portrait according to the behavior tag, and storing the user portrait in a database.
In this embodiment, the granularity of the behavior tag in the system to which the method is applied is relatively coarse, and is primarily determined primarily according to the user behavior. The user behavior comprises: access, operation, occupation; the three types are three latitudes provided by the currently docked platform; and finally mapping (drawing) three types of users: visitors, general guest groups, deep users;
(1) visitor: only browse people who do not do any operation on the system function;
(2) general guest group: in particular to a person who needs to rely on the internal functions of the platform to achieve the self appeal;
(3) depth user: in particular to a person who completely relies on the data provided by the platform and possesses the data.
S212, receiving a query request input by a user.
In this embodiment, the query request includes a query dimension, and the query dimension includes one or more of a department dimension, a person dimension, a group dimension, a scene dimension, a group dimension, and a module dimension.
And S213, acquiring query data from the database according to the query request.
And S214, generating a data report according to the query data.
In this embodiment, the method may implement configuration, collection, storage, calculation, and visualization according to a modular development module by using a modular design development concept, and finally combine an engine system for completing user behavior collection (calculation). Referring to fig. 5, fig. 5 shows a system architecture diagram of the system.
In this embodiment, the modules may communicate with each other through an API network, so as to implement interactive transmission on data and high-cohesion low-coupling between systems. Meanwhile, the system provides configuration capability, maintains relevant basic configuration, provides a visual GUI interface, and is more convenient and faster. An API (Application Program Interface) is defined as a standard set of commands and information that an Application can use to exchange information with a computer operating system.
In this embodiment, the method can use an autonomic development collector engine to provide multiple data collection modes, pouring in business logic zeros; meanwhile, the collector engine provides various different docking modes, and a dockee can conveniently and quickly access the collector.
In the embodiment, the method can also realize the storage of large data volume and high-efficiency aggregation calculation by means of the mass storage and strong calculation capability of the ES; meanwhile, for a calculation result, a visual interface display is provided, and generation of various data reports is supported; in addition, the user images can be dynamically aggregated into a single user image according to the user behavior data.
For example, the engine system applied by the method can be divided into the following modules: the system comprises a basic configuration module, a cache (cache) preheating module, a collector, a storage/calculation unit and an operation view report.
(1) A basic configuration module: the module is basic information of the whole system, and a user needs to maintain the module in advance through a configured GUI (graphical user interface); data collection at the upper layer needs to be associated with configuration at the lower layer, and if configuration information is not associated, the system automatically skips the collection process. The method comprises the following specific steps:
a functional module: the module is the definition of a butt-joint system module and mainly comprises the name and the number of the module and the priority of the internal function of the module; priority is a protection mechanism of the system, namely: when the data volume is overloaded, the purpose of current-limiting protection can be achieved according to the priority of the module, the higher the priority is, the system is preferentially ensured, and the risk of data loss can exist when the priority is low;
module classification: for module classification, the collector can match different collection modes according to different classifications, and the system defines three categories at present: the system comprises a static page, a functional module and a timing task module, wherein if a new category needs to be expanded, the new category can be added through a back management page;
and (4) behavior classification: for distinguishing user's operation behavior, for example: access, commit, delete, query, jump, etc.;
marking by using a probe: the system uses a 10-bit random character string calculated by a hash algorithm through module numbering and classification (modules and behaviors), and guarantees global uniqueness; the probe is a unique identifier for distinguishing different scene classifications by the collector and is also a tangent point of the collector;
the functional module is associated with a module classification and a behavior classification (1:n), and the corresponding probe identification is generated by a button provided by the page, and the basic configuration is completed.
(2) Cache (Cache) warm-up: in order to improve the performance of configuration reading, a cache module is developed by a system through an open source component Caffeine, after a project is started each time, basic configuration is automatically loaded into a cache, the change of a configuration item is monitored in real time, and the cache is refreshed in time.
(3) A collector: monitoring the triggering of user behaviors, acquiring user flow in real time, synchronously matching configuration items in the cache, processing and integrating configuration information and flow data, and pushing the configuration information and the flow data to a storage unit; wherein, the collector provides two kinds of butt joint modes altogether: a. deploying the collectors in a service mode, realizing data collection by providing an http request interface, and b, integrally packaging the collectors into independent jar packages, and synchronously sharing a warehouse, so that a user can access to own engineering for use in a mode of sdk (Software Development Kit). Four different collectors are as follows:
http (hypertext Transfer Protocol) interface: the butt joint party can directly call the interface and directly transmit the corresponding data to the storage unit;
an interceptor: intercepting a user network request by using an interceptor mechanism of a springMVC frame, and asynchronously transmitting request data to a core processing module for processing, wherein the interceptor is mainly suitable for static resources; the springMVC framework is a Java framework and provides a full-function MVC module for constructing a Web application program.
Cutting: the section is a transverse development mode in the technical field of Java, is realized by introducing an Aspect frame with an open source, and is mainly used for zero-entering of service logic in the scenes of functional modules and complex services;
event monitoring: in the system, a set of service mechanisms for monitoring and event sending are built, developers call an API for sending events at the position of codes needing to be triggered to collect, and meanwhile, when the system is started, a synchronous probe is embedded between every two lines of codes, and when the codes execute an event sending logic, the probe can be packaged in an event object through the API and sent to a server side.
(4) A storage/calculation unit: the processing capacity of the elastic search massive database is used for storing user behavior data, different access parties allocate different indexes to store corresponding data, data isolation is achieved, and meanwhile a cluster deployment mode is used, so that high availability of the environment is guaranteed; processing aggregation and real-time calculation of data are realized based on an API provided by es, an interface is packaged to provide service capability, an operation data background is connected in a butt joint mode, and real-time rendering is carried out on a background page;
(5) reporting an operation view: based on the result of the storage unit aggregation, a set of operation data pages is independently developed, the behavior data of the user can be checked in real time, various behavior tags are set, and the customer portrait is aggregated through the tags; meanwhile, a data report can be provided, real-time aggregation can be performed according to different conditions selected by a user, and an entrance for downloading and exporting can be provided.
In this embodiment, the execution subject of the method may be a computing device such as a computer and a server, and is not limited in this embodiment.
In this embodiment, an execution subject of the method may also be an intelligent device such as a smart phone and a tablet computer, which is not limited in this embodiment.
Therefore, the data collection method described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design development; meanwhile, the configuration flexibility can be improved, so that the configuration of basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 3
Please refer to fig. 3, fig. 3 is a schematic structural diagram of a data collection device according to an embodiment of the present application. As shown in fig. 3, the data collection apparatus includes:
an obtaining unit 310, configured to obtain configuration information configured by a user through a configured GUI interface;
a monitoring unit 320, configured to monitor user behavior;
an obtaining unit 330, configured to obtain traffic data of a user behavior in real time;
a processing integration unit 340, configured to process and integrate the configuration information and the flow data to obtain integrated data;
an allocating unit 350, configured to allocate a corresponding target index to the integrated data according to an access party of the user behavior;
and the storage unit 360 is used for storing the integrated data into the database according to the target index.
In this embodiment, for the explanation of the data collection device, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the data collection device described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design and development; meanwhile, the configuration flexibility can be improved, so that the configuration of the basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
Example 4
Referring to fig. 4, fig. 4 is a schematic structural diagram of a data collection device according to an embodiment of the present application. As shown in fig. 4, the data collection apparatus includes:
an obtaining unit 310, configured to obtain configuration information configured by a user through a configured GUI interface;
a monitoring unit 320, configured to monitor user behavior;
an obtaining unit 330, configured to obtain traffic data of a user behavior in real time;
a processing integration unit 340, configured to process and integrate the configuration information and the flow data to obtain integrated data;
an allocating unit 350, configured to allocate a corresponding target index to the integrated data according to an access party of the user behavior;
the storage unit 360 is used for storing the integrated data into the database according to the target index.
In this embodiment, the configuration information at least includes a function module, a function module type, a user behavior type, a function module-associated user behavior, a function execution entry, and a probe identifier.
As an alternative embodiment, the process integration unit 340 includes:
an obtaining subunit 341, configured to obtain, according to the configuration information, a current probe identifier inside the flow data; acquiring user behavior information and scene module information according to the current probe identification;
the parsing subunit 342 is configured to parse the traffic data according to the configuration information to obtain user account information;
the obtaining subunit 341 is further configured to obtain detailed information of the login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
As an optional implementation, the data collection apparatus further comprises:
a matching unit 370 for matching behavior tags for the integration data;
the storage unit 360 is further used for generating a user portrait according to the behavior tag and storing the user portrait.
As an optional implementation, the data collection apparatus further comprises:
a receiving unit 380, configured to receive a query request input by a user; the query request comprises query dimensions, wherein the query dimensions comprise one or more of department dimensions, individual dimensions, grouping dimensions, scene dimensions, group dimensions and module dimensions;
a query unit 390, configured to obtain query data from a database according to a query request;
the generating unit 400 is configured to generate a data report according to the query data.
In this embodiment, for the explanation of the data collection device, reference may be made to the description in embodiment 1 or embodiment 2, and details are not repeated in this embodiment.
Therefore, the data collection device described in the embodiment can reduce the system coupling degree and improve the system stability based on the idea of modular design development; meanwhile, the configuration flexibility can be improved, so that the configuration of the basic information can be realized more conveniently and quickly; in addition, various butt joint modes can be used, so that the effects of simplifying butt joint cost and improving development efficiency are achieved; finally, a friendly visual interface can be provided, so that the effect of displaying and counting the user behavior data in real time is realized.
An embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the data collection method in embodiment 1 or embodiment 2 of the present application.
An embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions, and when the computer program instructions are read and executed by a processor, the computer program instructions execute the data collection method in embodiment 1 or embodiment 2 of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
Claims (8)
1. A method of data collection, comprising:
acquiring configuration information configured by a user through a configured GUI interface;
monitoring user behaviors, and acquiring flow data of the user behaviors in real time;
processing and integrating the configuration information and the flow data to obtain integrated data;
distributing a corresponding target index for the integrated data according to the access party of the user behavior;
storing the integrated data into a database according to the target index;
wherein, the processing and integrating the configuration information and the flow data to obtain integrated data includes:
acquiring a current probe identifier in the flow data according to the configuration information;
acquiring user behavior information and scene module information according to the current probe identification;
analyzing the flow data according to the configuration information to obtain user account information;
acquiring detailed information of a login person according to the user account information;
acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
2. The data collection method of claim 1, wherein the configuration information includes at least a function module, a function module type, a user behavior type, a function module associated user behavior, a function execution entry, and a probe identification.
3. The data collection method of claim 1, further comprising:
matching a behavior tag for the consolidated data;
and generating a user portrait according to the behavior tag, and storing the user portrait.
4. The data collection method of claim 1, further comprising:
receiving a query request input by a user; wherein the query request includes a query dimension, the query dimension including one or more of a department dimension, a personal dimension, a group dimension, a scene dimension, a group dimension, and a module dimension;
acquiring query data from the database according to the query request;
and generating a data report according to the query data.
5. A data collection device, the data collection device comprising:
the acquisition unit is used for acquiring configuration information configured by a user through a configured GUI interface;
the monitoring unit is used for monitoring user behaviors;
the acquisition unit is used for acquiring the flow data of the user behavior in real time;
the processing integration unit is used for processing and integrating the configuration information and the flow data to obtain integrated data;
the distribution unit is used for distributing corresponding target indexes to the integrated data according to the access party of the user behavior;
the storage unit is used for storing the integrated data into a database according to the target index;
wherein, the processing integration unit includes:
the acquisition subunit is used for acquiring the current probe identifier in the flow data according to the configuration information; acquiring user behavior information and scene module information according to the current probe identification;
the analysis subunit is used for analyzing the flow data according to the configuration information to obtain user account information;
the acquisition subunit is further configured to acquire detailed information of a login person according to the user account information; acquiring integrated data according to the user behavior information, the scene module information and the detailed information of the login person; the integrated data at least comprises menu/function module information, user operation behaviors, user information and user behavior trigger time.
6. The data collection device of claim 5, further comprising:
the matching unit is used for matching behavior labels for the integrated data;
the storage unit is further used for generating a user portrait according to the behavior tag and storing the user portrait.
7. An electronic device, characterized in that the electronic device comprises a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the data collection method of any of claims 1 to 4.
8. A readable storage medium having stored therein computer program instructions which, when read and executed by a processor, perform the data collection method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186783.8A CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186783.8A CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115292330A CN115292330A (en) | 2022-11-04 |
| CN115292330B true CN115292330B (en) | 2022-12-20 |
Family
ID=83833943
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211186783.8A Active CN115292330B (en) | 2022-09-28 | 2022-09-28 | Data collection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115292330B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461644A (en) * | 2022-01-30 | 2022-05-10 | 中国农业银行股份有限公司 | Data acquisition method and device, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563190B (en) * | 2020-04-07 | 2023-03-14 | 中国电子科技集团公司第二十九研究所 | Multi-dimensional analysis and supervision method and system for user behaviors of regional network |
| CN112104523B (en) * | 2020-09-11 | 2022-04-12 | 中国联合网络通信集团有限公司 | Detection method, device and equipment for flow transparent transmission and storage medium |
| CN115001967B (en) * | 2022-05-30 | 2023-08-29 | 平安科技（深圳）有限公司 | Data acquisition method and device, electronic equipment and storage medium |
-
2022
- 2022-09-28 CN CN202211186783.8A patent/CN115292330B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461644A (en) * | 2022-01-30 | 2022-05-10 | 中国农业银行股份有限公司 | Data acquisition method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115292330A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309264B (en) | Method and device for acquiring geographic product data based on knowledge graph | |
| RU2343537C2 (en) | Computer search with help of associative links | |
| Beheshti et al. | iprocess: Enabling iot platforms in data-driven knowledge-intensive processes | |
| US20140101134A1 (en) | System and method for iterative analysis of information content | |
| Ji et al. | Epidemic outbreak and spread detection system based on twitter data | |
| CN109840782A (en) | Clicking rate prediction technique, device, server and storage medium | |
| US10540386B2 (en) | Method for processing and displaying real-time social data on map | |
| CN115422169B (en) | Data warehouse construction method and device based on commercial advertisement scene | |
| CN110300084A (en) | A kind of IP address-based portrait method and apparatus | |
| CN109977296A (en) | A kind of information-pushing method, device, equipment and storage medium | |
| JPWO2011043429A1 (en) | Information management apparatus, data processing method thereof, and computer program | |
| Xu et al. | Multi-modal description of public safety events using surveillance and social media | |
| CN111930927B (en) | Evaluation information display method and device, electronic equipment and readable storage medium | |
| CN115292330B (en) | Data collection method and device | |
| CN110990692A (en) | Data processing method and device based on portrait analysis | |
| CN115018473A (en) | Service processing method, device, storage medium and equipment | |
| Antunes et al. | Semantic-based publish/subscribe for M2M | |
| CN109815278A (en) | A kind of method for exhibiting data and its equipment, storage medium, electronic equipment | |
| CN112230902B (en) | Software development method and system based on nailing customization | |
| Kaufhold et al. | Cross-Media Usage of Social Big Data for Emergency Services and Volunteer Communities: Approaches, Development and Challenges of Multi-Platform Social Media Services | |
| CN114820011A (en) | User group clustering method and device, computer equipment and storage medium | |
| CN109614566A (en) | Traffic source analysis method, system and device | |
| CN111466102B (en) | Method, system and apparatus for providing a set of context keywords for a communication event in a multiple communication platform environment | |
| CN115718825B (en) | Method and device for determining duration label and electronic equipment | |
| WO2016199401A1 (en) | Information processing device, information analysis device, information processing method, information analysis method, information processing program, and information analysis program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |