CN115730171A - Data storage method, device, equipment and medium - Google Patents
Data storage method, device, equipment and medium Download PDFInfo
- Publication number
- CN115730171A CN115730171A CN202211530036.1A CN202211530036A CN115730171A CN 115730171 A CN115730171 A CN 115730171A CN 202211530036 A CN202211530036 A CN 202211530036A CN 115730171 A CN115730171 A CN 115730171A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- user
- information
- relationship
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a data storage method, a device, equipment and a medium, which relate to the field of computers, and the method comprises the following steps: acquiring target heterogeneous data from a target service system connected to a current system in advance; consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database; and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database. According to the invention, various heterogeneous data are integrated, the relationship data among users in the heterogeneous data is stored in the preset database, and the user behavior data is stored in the preset database, so that the user data of a service system side can be completely stored, the problem of incomplete user data storage in the prior art is solved, and the subsequent data query process is facilitated.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a data storage method, apparatus, device, and medium.
Background
With the increasing data, each business system generates own user behavior data. When a user wants to query complete user behavior data between subsequent business systems, the queried data is incomplete. Therefore, how to integrate different data sources to provide a complete user relationship system for a business system is a problem to be solved in the art.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a data storage method, apparatus, device and medium, which can consume various heterogeneous data in real time, store the heterogeneous data in real time, and obtain a complete user relationship system and user data when querying user data of a service system. The specific scheme is as follows:
in a first aspect, the present application discloses a data storage method, including:
acquiring target heterogeneous data from a target service system connected to a current system in advance;
consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database;
and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database.
Optionally, the consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data in a preset database, includes:
and consuming the target heterogeneous data by utilizing the kakfa, and storing the relationship data between the users determined from the target heterogeneous data into a Dgram database.
Optionally, the determining, by using a preset flow calculation tool, user behavior data from the target heterogeneous data, and storing the user behavior data in a preset database includes:
and determining user behavior data from the target heterogeneous data by using flink, and storing the user behavior data to starrocks.
Optionally, the obtaining target heterogeneous data from a target service system connected to the current system in advance includes:
acquiring target log information from a target service system which is connected to a current system in advance and starts a log tracking function;
and determining target heterogeneous data based on the target log information.
Optionally, the storing the relationship data between users determined from the target heterogeneous data into a preset database includes:
determining relationship data among users from the target heterogeneous data;
updating the current knowledge graph based on the relationship data among the users to determine an updated knowledge graph;
and storing the updated knowledge graph to a preset graph database.
Optionally, the determining relationship data between users from the target heterogeneous data includes:
determining node information for representing user nodes and/or relationship information for representing relationships among users from the target heterogeneous data;
correspondingly, the updating the current knowledge-graph based on the relationship data among the users comprises the following steps:
and adding a first target node in the current knowledge graph based on the node information, and/or adding a second target node with the target relation type in the current knowledge graph based on the target relation type in the relation information.
Optionally, the determining relationship data between users from the target heterogeneous data includes:
determining relationship data among users from the target heterogeneous data; the relationship data among the users comprises one or more of user identity information, source key value information, resource scene information, user telephone information, weChat information, visitor information, external contact information and user self-association information.
In a second aspect, the present application discloses a data storage device comprising:
the heterogeneous data acquisition module is used for acquiring target heterogeneous data from a target service system which is connected to the current system in advance;
the inter-user relationship data storage module is used for consuming the target heterogeneous data by using a preset distributed publish-subscribe message system and storing the inter-user relationship data determined from the target heterogeneous data into a preset database;
and the user behavior data storage module is used for determining user behavior data from the target heterogeneous data by using a preset flow calculation tool and storing the user behavior data into a preset database.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the aforementioned data storage method.
In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein the computer program realizes the steps of the data storage method disclosed in the foregoing when executed by a processor.
The method comprises the steps that target heterogeneous data are obtained from a target service system connected to a current system in advance; consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database; and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database. Therefore, the invention integrates various heterogeneous data, stores the relationship data among users in the heterogeneous data to the preset database, and stores the user behavior data to the preset database, thereby completely storing the user data of the service system side. In practical application, when user data of a business system is queried, the relationship data between users stored in the preset graph database can be used for acquiring complete user data of a target query user, and simultaneously user data of related users related to the target query user can be acquired, so that the problem of incomplete storage of user data in the prior art is solved, the subsequent data query process is facilitated, and the usability is high.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a data storage method provided herein;
FIG. 2 is a flow chart of a specific data storage method provided herein;
FIG. 3 is a diagram of a data model provided herein;
FIG. 4 is a data model diagram of relationship information provided herein;
FIG. 5 is a schematic illustration of a knowledge-graph as provided herein;
FIG. 6 is a flow chart of a data storage process provided herein;
FIG. 7 is a schematic diagram of a data storage device according to the present application;
fig. 8 is a block diagram of an electronic device provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the prior art, when a business system wants to query complete user behavior data, the queried data is incomplete. According to the method and the device, various heterogeneous data can be consumed in real time, the heterogeneous data can be stored in real time, and a complete user relationship system and user data can be obtained when the user data of the service system is inquired.
The embodiment of the invention discloses a data storage method, which comprises the following steps of referring to fig. 1:
step S11: and acquiring target heterogeneous data from a target service system connected to the current system in advance.
In a specific implementation manner of this embodiment, the target business system may include a guest resource platform, an enterprise wechat platform, an official activity platform, an official website card portal, an applet public number, and the like, and data structures of data acquired from various target business systems may all be different, so that the present invention is applied to a server that serves a business end where the business system is located, and can complete integration and storage of user data by collecting heterogeneous data in the business system.
In this embodiment, the acquiring target heterogeneous data from a target service system connected to a current system in advance includes: acquiring target log information from a target service system which is connected to a current system in advance and starts a log tracking function; and determining target heterogeneous data based on the target log information.
In this embodiment, the service system needs to start a log tracking function, and in a preferred embodiment, the log tracking function includes, but is not limited to, a bin log. After the service system starts the bin, the server can determine the target heterogeneous data based on the log in the bin.
Step S12: and consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset graph database.
In this embodiment, the consuming the target heterogeneous data by using a preset distributed publish-subscribe messaging system, and storing the relationship data between users determined from the target heterogeneous data into a preset graph database may include: and consuming the target heterogeneous data by utilizing kakfa, and storing the relationship data between the users determined from the target heterogeneous data into a Dgram database.
In a specific embodiment, the preset distributed publish-subscribe message system is preferably kakfa, and the preset database is preferably a Dgrap database. In particular, the inter-user relationship data may be node information for characterizing user nodes and/or relationship information for characterizing inter-user relationships.
In this embodiment, after the relationship data between users is determined, a data relationship may be created based on the relationship data between users, and the data relationship may be stored in a digrap database.
Step S13: and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database.
In this embodiment, the user behavior data may include time information, location information, user information, interaction types, interaction contents, and the like of user operations.
In this embodiment, the determining, by using a preset flow calculation tool, user behavior data from the target heterogeneous data and storing the user behavior data in a preset database may include: and determining user behavior data from the target heterogeneous data by using flink, and storing the user behavior data to starrocks. In this embodiment, the preset flow calculation tool is preferably a flink, and the preset database is preferably a starrocks. This step preferably generates user behavior by flink processing of real-time streaming data to be stored to starrocks.
In the embodiment, target heterogeneous data is obtained from a target service system connected to a current system in advance; consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database; and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database. Therefore, the invention integrates various heterogeneous data, stores the relationship data among users in the heterogeneous data to the preset database, and stores the user behavior data to the preset database, thereby completely storing the user data of the service system side. In practical application, when user data of a business system is queried, the relationship data between users stored in the preset graph database can be used for acquiring complete user data of a target query user, and simultaneously user data of related users related to the target query user can be acquired, so that the problem of incomplete storage of user data in the prior art is solved, the subsequent data query process is facilitated, and the usability is high.
Fig. 2 is a flowchart of a specific data storage method according to an embodiment of the present disclosure. Referring to fig. 2, the method includes:
step S21: and acquiring target heterogeneous data from a target service system connected to the current system in advance.
For a more specific processing procedure of step S21, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Step S22: and consuming the target heterogeneous data by utilizing kakfa, and determining the relationship data among users from the target heterogeneous data.
In this embodiment, the determining relationship data between users from the target heterogeneous data may include: determining node information for representing user nodes and/or relationship information for representing relationships among users from the target heterogeneous data; accordingly, the updating the current knowledge-graph based on the inter-user relationship data may include: and adding a first target node in the current knowledge graph based on the node information, and/or adding a second target node with the target relation type in the current knowledge graph based on the target relation type in the relation information.
In this embodiment, the inter-user relationship data may include node information used for characterizing the user node and/or relationship information used for characterizing the inter-user relationship. In a first specific implementation manner, if only node information representing a user node is acquired and relationship information representing a relationship between users is not acquired, for example, only identity data of a certain user is acquired and relationship data between the user and other users is not acquired, the server adds a node with the identity data of the user in the knowledge graph; in a second specific implementation manner, if only relationship information representing the relationship between users is obtained and node information representing a user node is not obtained, the server adds relationship data in the knowledge graph, that is, it is known that two nodes have a relationship but identity data of the two nodes is unknown; in a third specific embodiment, if relationship information representing a relationship between users and node information representing a user node are obtained at the same time, the server adds two nodes having relationship data to the knowledge graph. In addition, in many cases, when the relationship information representing the relationship between users and the node information representing the user nodes are acquired at the same time, one or two of the two nodes may have a relationship with a certain existing node in the currently established knowledge graph, and at this time, the node relationship needs to be integrated and displayed in the current knowledge graph.
It should be noted that, because the above-mentioned idea of "point first comes first to save point, relationship first to save relationship" is adopted to create the knowledge-graph, there may be a case that a single or multiple free nodes or free relationships are not connected to the backbone of the knowledge-graph and are free outside. In this case, if the missing information of the isolated nodes and the isolated relationships is subsequently obtained, the missing information is filled into the isolated nodes and the isolated relationships to establish a connection with the skeleton in the knowledge graph.
In this embodiment, the determining relationship data between users from the target heterogeneous data may include: determining relationship data among users from the target heterogeneous data; the relationship data among the users comprises one or more of user identity information, source key value information, resource scene information, user telephone information, weChat information, visitor information, external contact information and user self-association information. The user identification information, the source key value information and the resource scene information are node information representing user nodes, and the user telephone information, the WeChat information, the visitor information, the external contact information and the user self-association information are relationship information representing the relationship among users.
Fig. 3 is a data model diagram proposed in this embodiment, where the relationship data between users of each User may include uid (i.e., user Identification), sourcekey (i.e., source key value), resourcetype, tels, wechat, relationships (representing the associated information of the User), and Pid (enterprise identity). The sourcekey is used as a unique identifier of heterogeneous data corresponding to the database, and the resourcetype represents a resource scene or a resource type of the data, namely a target service system where a user is located during operation. In a specific embodiment, the resourcetype may be an enterprise wechat platform, an official activity platform, an official website card portal, an applet public number, or the like. The Pid is used for marking the enterprise identity, in the specific implementation manner of the present invention, the user data of a plurality of enterprises in different target business systems can be stored simultaneously, and the Pid is used for distinguishing different enterprises. The relationships are used for representing the self-associated information of the user, and when the same user logs in to operate in different resource scenes, the data of the user generates and stores the data of different users belonging to the recursion relationship. In addition, in the phone data associated with the user and the micro-message data associated with the user in the heterogeneous data, each phone number and micro-message code has its uid as a user identification, its PK (Primary Key) as a unique identifier in the database, and Pid is also used to distinguish different enterprises.
Fig. 4 is a data model diagram of relationship information according to the present invention, in which a left structure diagram may represent a model structure and a relationship of data after heterogeneous data fusion, each origin in the left structure diagram represents a different data type, a line represents a reference relationship between data, and an arrow represents a direction, and a pointing relationship between external contact information external _ user associated with a user, telephone information tel associated with the user, visitor information cuid associated with the user, wechat information associated with the user, and self-association information customer of the user in relationship data between users, which are information representing an association relationship between users, is interpreted. Wherein, the external contact information represents friend information (such as friend added in enterprise WeChat) of the user in the third-party contact platform, and the visitor information refers to information labeled by the user without leaving any information. The right-side summary in fig. 4 identifies that five node identifiers and five relationship types exist in the node map, where the five node identifiers are: external contact information, telephone information, visitor information, weChat information, user self-associated information. The five relationship types are: the associated information with the user is the associated type of the external contact information, the associated information with the user is the associated type of the associated information of the user, the associated information with the user is the associated type of the visitor information, the associated information with the user is the associated type of the telephone information, and the associated information with the user is the associated type of the WeChat information.
Step S23: updating the current knowledge-graph based on the inter-user relationship data to determine an updated knowledge-graph.
As shown in fig. 5, when creating a knowledge graph, nodes generated by heterogeneous data of a certain user in different resource scenes may be labeled with the same color, when the user operates in different resource scenes, different nodes may be generated due to different resource scene data in the heterogeneous data, and each node in the knowledge graph connects automatically generated relationship information with an associated node. The relationship information between users is generally displayed in a form of connection.
Step S24: and storing the updated knowledge graph to a Dgrap database.
In the embodiment, the knowledge graph constructed according to the real-time heterogeneous data is stored in the digrap database, so that the relationship data between the users can be stored better and visually and completely.
Step S25: and determining user behavior data from the target heterogeneous data by using flink, and storing the user behavior data to starrocks.
In this embodiment, the user behavior data determined according to the real-time heterogeneous data is stored in a starclocks database, so that the user behavior data can be completely stored.
In the embodiment, complete user behaviors and tracks can be obtained when certain user data is inquired, and when the user data is inquired, the data of the associated user of the target user can be inquired while the target user data is inquired, so that the inquiry of the user behavior depth among users is improved, the inquiry efficiency is greatly improved, a cyclic inquiry mode adopted in the prior art is abandoned when the data is inquired, the inquiry can be carried out according to the predefined unique identifier in the database, and the inquiry efficiency is greatly improved.
As shown in fig. 6, in a data storage flow chart provided by the present invention, at a data real-time processing stage, after target heterogeneous data is obtained in a preset trigger scenario, since the method provided by the present invention can be used to provide heterogeneous data storage services for multiple enterprises at the same time when the present invention is implemented specifically, enterprises which open heterogeneous data storage service plug-ins can be filtered first, and corresponding filtering mechanisms are set in multiple trigger scenarios, so that it is ensured that heterogeneous data which is sent backward is data in target enterprises which open heterogeneous data storage services. And respectively processing the relationship data and the user behavior data during data processing. In a specific implementation manner, each user relation data is cached in a preset cache table, when the relation data between users to be processed received in real time is processed, a starting point of the data is firstly inquired from the cache table, the inquiry can also be understood as a node corresponding to the relation data between the users to be processed, if the node exists, the associated data of the associated user is inquired according to the unique uid corresponding to the node, and then the relation data between the users to be processed and the existing associated data of the associated user are integrated and stored in a digrap database; if the node does not exist, the current knowledge graph can be subjected to condition query according to data in the relation data among the users to be processed, whether the relation information corresponding to the relation data among the users to be processed exists in the current knowledge graph is queried, and if the relation information exists, the relation data among the users to be processed and the corresponding relation information are integrated and stored in a Dgram database. Note that, when storing the relationship data, the above-described "point-first-save-point-first-relationship-first-save-relationship" is also used to store the relationship data, and then the received data fills the set of the previously saved data. In addition, the user behavior data is written to the starrocks in batches through the flink. In the application of the invention, an offline task which is completed at regular time can be set, and specifically, all data of the target enterprise which opens the heterogeneous data storage service plug-in can be synchronized to the data warehouse tool hive at regular time and written into the starrings so as to record the target enterprise which opens the heterogeneous data storage service.
Referring to fig. 7, an embodiment of the present application discloses a data storage device, which may specifically include:
a heterogeneous data obtaining module 11, configured to obtain target heterogeneous data from a target service system that is connected to a current system in advance;
the inter-user relationship data storage module 12 is configured to consume the target heterogeneous data by using a preset distributed publish-subscribe message system, and store inter-user relationship data determined from the target heterogeneous data in a preset database;
and the user behavior data storage module 13 is configured to determine user behavior data from the target heterogeneous data by using a preset stream calculation tool, and store the user behavior data in a preset database.
The method comprises the steps that target heterogeneous data are obtained from a target service system connected to a current system in advance; consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database; and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database. Therefore, the invention integrates various heterogeneous data, stores the relationship data among users in the heterogeneous data to the preset database, and stores the user behavior data to the preset database, thereby completely storing the user data of the service system side. In practical application, when user data of a business system is queried, the relationship data between users stored in the preset graph database can be used for acquiring complete user data of a target query user, and simultaneously user data of related users related to the target query user can be acquired, so that the problem of incomplete storage of user data in the prior art is solved, the subsequent data query process is facilitated, and the usability is high.
Further, an electronic device is also disclosed in the embodiments of the present application, fig. 8 is a block diagram of the electronic device 20 shown in the exemplary embodiment, and the content in the diagram cannot be considered as any limitation to the scope of the application.
Fig. 8 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a display 24, an input-output interface 25, a communication interface 26, and a communication bus 27. Wherein, the memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the data storage method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 26 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like, the resources stored thereon may include an operating system 221, a computer program 222, virtual machine data 223, and the like, and the virtual machine data 223 may include various data. The storage means may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, and may be Windows Server, netware, unix, linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the data storage method disclosed in any of the foregoing embodiments and executed by the electronic device 20.
Further, the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium includes a Random Access Memory (RAM), a Memory, a Read-Only Memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk, or an optical disk or any other form of storage medium known in the art. Wherein the computer program, when executed by a processor, implements the data storage method disclosed above. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed in the embodiment corresponds to the method disclosed in the embodiment, so that the description is simple, and the relevant points can be referred to the description of the method part. Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The data storage method, apparatus, device and storage medium provided by the present invention are described in detail above, and a specific example is applied in the present disclosure to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understand the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method of storing data, comprising:
acquiring target heterogeneous data from a target service system connected to a current system in advance;
consuming the target heterogeneous data by using a preset distributed publish-subscribe message system, and storing the relationship data between users determined from the target heterogeneous data into a preset database;
and determining user behavior data from the target heterogeneous data by using a preset flow calculation tool, and storing the user behavior data into a preset database.
2. The data storage method according to claim 1, wherein the consuming the target heterogeneous data by using a preset distributed publish-subscribe message system and storing the relationship data between users determined from the target heterogeneous data into a preset database comprises:
and consuming the target heterogeneous data by utilizing kakfa, and storing the relationship data between the users determined from the target heterogeneous data into a Dgram database.
3. The data storage method according to claim 1, wherein the determining user behavior data from the target heterogeneous data by using a preset flow calculation tool and storing the user behavior data in a preset database comprises:
and determining user behavior data from the target heterogeneous data by using flink, and storing the user behavior data to starrocks.
4. The data storage method according to claim 1, wherein the obtaining target heterogeneous data from a target business system pre-connected to a current system comprises:
acquiring target log information from a target service system which is connected to a current system in advance and starts a log tracking function;
and determining target heterogeneous data based on the target log information.
5. The data storage method according to any one of claims 1 to 4, wherein the storing the relationship data between users determined from the target heterogeneous data to a preset database comprises:
determining relationship data among users from the target heterogeneous data;
updating the current knowledge graph based on the relationship data among the users to determine an updated knowledge graph;
and storing the updated knowledge graph to a preset graph database.
6. The data storage method of claim 5, wherein the determining the inter-user relationship data from the target heterogeneous data comprises:
determining node information for representing user nodes and/or relationship information for representing relationships among users from the target heterogeneous data;
correspondingly, the updating the current knowledge-graph based on the inter-user relationship data comprises:
and adding a first target node in the current knowledge graph based on the node information, and/or adding a second target node with the target relation type in the current knowledge graph based on the target relation type in the relation information.
7. The data storage method of claim 5, wherein the determining the inter-user relationship data from the target heterogeneous data comprises:
determining relationship data among users from the target heterogeneous data; the relationship data among the users comprises one or more of user identity information, source key value information, resource scene information, user telephone information, weChat information, visitor information, external contact information and user self-association information.
8. A data storage device, comprising:
the heterogeneous data acquisition module is used for acquiring target heterogeneous data from a target service system which is connected to the current system in advance;
the inter-user relationship data storage module is used for consuming the target heterogeneous data by using a preset distributed publish-subscribe message system and storing the inter-user relationship data determined from the target heterogeneous data into a preset database;
and the user behavior data storage module is used for determining user behavior data from the target heterogeneous data by using a preset flow calculation tool and storing the user behavior data into a preset database.
9. An electronic device comprising a processor and a memory; wherein the processor, when executing the computer program stored in the memory, implements the data storage method of any of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements a data storage method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211530036.1A CN115730171A (en) | 2022-11-30 | 2022-11-30 | Data storage method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211530036.1A CN115730171A (en) | 2022-11-30 | 2022-11-30 | Data storage method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115730171A true CN115730171A (en) | 2023-03-03 |
Family
ID=85299731
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211530036.1A Pending CN115730171A (en) | 2022-11-30 | 2022-11-30 | Data storage method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115730171A (en) |
-
2022
- 2022-11-30 CN CN202211530036.1A patent/CN115730171A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8838679B2 (en) | Providing state service for online application users | |
| CN111068328B (en) | Game advertisement configuration form generation method, terminal equipment and medium | |
| CN108390933B (en) | Message distribution method, device, server and storage medium | |
| CN109618176B (en) | Processing method, equipment and storage medium for live broadcast service | |
| CN114124861A (en) | Message group sending method and device, computer equipment and storage medium | |
| CN111177081A (en) | Game log content query method and device, computer equipment and storage medium | |
| CN113014623B (en) | Method and device for processing real-time streaming data of embedded point, computer equipment and storage medium | |
| CN113806305A (en) | Data export method and device, computer readable storage medium and electronic equipment | |
| CN113391901A (en) | RPA robot management method, device, equipment and storage medium | |
| CN109213955B (en) | Data processing method and related equipment | |
| CN108804711B (en) | Data processing method and device and computer readable storage medium | |
| CN113660359A (en) | Management method and device for domain name resolution record, storage medium and electronic equipment | |
| CN111949719B (en) | Method and device for realizing configuration tool of power system and related components | |
| CN104135503B (en) | Method, Apparatus and system that data are downloaded | |
| CN111159142B (en) | Data processing method and device | |
| CN111767481A (en) | Access processing method, device, equipment and storage medium | |
| CN115730171A (en) | Data storage method, device, equipment and medium | |
| CN108628702B (en) | Memorandum method and device for chat records, electronic equipment and storage medium | |
| CN116028479A (en) | Data reconciliation system, method, equipment and storage medium | |
| CN115455477A (en) | Data authority management method, device, equipment, medium and program product | |
| CN113377682A (en) | Service testing method and device | |
| CN111953722A (en) | Terminal address book synchronization method and device | |
| CN110166505B (en) | Session record storage method, device, equipment and storage medium | |
| CN114629942B (en) | Fraud early warning task generation method, device, equipment and medium | |
| CN114785847B (en) | Network control software development configuration method, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |