CN116166873A

CN116166873A - User portrait generation method and device, electronic equipment and storage medium

Info

Publication number: CN116166873A
Application number: CN202210974291.9A
Authority: CN
Inventors: 方平; 吴鹏; 吴海英; 罗展松; 胡伟; 宋瑞鹏
Original assignee: Mashang Xiaofei Finance Co Ltd
Current assignee: Mashang Xiaofei Finance Co Ltd
Priority date: 2022-08-15
Filing date: 2022-08-15
Publication date: 2023-05-26

Abstract

The present disclosure provides a user portrait data generating method and apparatus, an electronic device, and a storage medium, where the method includes: the method comprises the steps of obtaining target user data to be processed in a streaming mode, wherein the target user data comprises user data generated by service application in real time; acquiring preconfigured tag information and crowd-sourced information, wherein the tag information comprises information of at least one tag, each tag represents a user characteristic of a user, the crowd-sourced information comprises information of at least one crowd-sourced, and each crowd-sourced comprises a plurality of different tags; portrait is carried out on the target user data according to the label information and the crowd pack information, and user portrait data is obtained; and in the process of portraying the target user data, acquiring the running state of the back-end node of the database cluster, and carrying out operation and maintenance management on the back-end node according to the running state. According to the embodiment of the disclosure, the user data generated in real time by the business application can be timely portrayed, so that the generation speed of the user portrayed data is improved.

Description

User portrait generation method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a user portrait generating method and apparatus, an electronic device, and a storage medium.

Background

User portraits (Persona), also known as user roles, are widely used in various fields as an effective tool for outlining target users and contacting user appeal and design directions, and can provide various convenient services for users through the user portraits.

At present, when user portrayal is performed, online user data generated by online business application within a period of time and offline user data uploaded offline are generally obtained in batches according to preset time intervals, and analysis processing is performed on the data obtained in batches to obtain user portrayal data.

As can be seen from this, the conventional user portrait creation method may have a problem of processing delay.

Disclosure of Invention

The present disclosure provides a user portrait generation method and apparatus, an electronic device, and a storage medium.

In a first aspect, the present disclosure provides a user portrait generation method applied to an electronic device deployed with a database cluster, where the method includes:

the method comprises the steps of obtaining target user data to be processed in a streaming mode, wherein the target user data comprise user data generated by service application in real time;

obtaining preconfigured tag information and crowd-sourced information, wherein the tag information comprises information of at least one tag, each tag represents a user characteristic of a user, the crowd-sourced information comprises information of at least one crowd-sourced, and each crowd-sourced comprises a plurality of different tags;

According to the label information and the crowd pack information, portrait is carried out on the target user data, and user portrait data are obtained; the method comprises the steps of,

and in the process of portraying the target user data, acquiring the running state of the back-end node of the database cluster, and carrying out operation and maintenance management on the back-end node according to the running state.

In a second aspect, the present disclosure provides a user portrayal generating apparatus applied to an electronic device deployed with a database cluster, the apparatus comprising:

the system comprises a user data acquisition unit, a service application and a processing unit, wherein the user data acquisition unit is used for acquiring target user data to be processed in a streaming mode, and the target user data comprises user data generated by the service application in real time;

a configuration information obtaining unit, configured to obtain preconfigured tag information and crowd-sourced information, where the tag information includes information of at least one tag, each tag represents a user feature of a user, the crowd-sourced information includes information of at least one crowd-sourced, and each crowd-sourced includes a plurality of different tags;

the portrait unit is used for portrait the target user data according to the label information and the crowd pack information to obtain user portrait data; the method comprises the steps of,

And the daemon unit is used for acquiring the running state of the back-end node of the database cluster in the process of portraying the target user data, and carrying out operation and maintenance management on the back-end node according to the running state.

In a third aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, one or more of the computer programs being executable by the at least one processor to enable the at least one processor to perform the user representation generation method described above.

In a fourth aspect, the present disclosure provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the user portrait generation method described above.

According to the embodiment provided by the disclosure, under the condition that user authorization is obtained, target user data to be processed is obtained through streaming, and preconfigured tag information and crowd pack information are obtained, and because the tag information comprises at least one tag information representing one user characteristic of a user and the crowd pack information comprises at least one crowd pack information respectively composed of a plurality of different tags, user portrait data can be timely generated by portraying the target user data obtained through streaming based on the tag information and the crowd pack information; meanwhile, in the process of portraying target user data, the running state of the back end node of the database cluster deployed in the electronic equipment is obtained, and the back end node is subjected to operation and maintenance management according to the running state, so that the stable and reliable running of the back end node can be ensured, and the health running of portraying processing is ensured. Because the stream processing mode has the characteristics of continuity and no interval, the target user data to be processed is acquired based on the stream processing mode, and the processing delay when the user portrayal is carried out can be reduced; meanwhile, operation and maintenance management is carried out on the back-end node based on the operation state of the back-end node, stable and reliable operation of the back-end node can be guaranteed, downtime of the back-end node is avoided, and therefore the electronic equipment can be guaranteed to provide real-time user portrait data for users to check.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure, without limitation to the disclosure. The above and other features and advantages will become more readily apparent to those skilled in the art by describing in detail exemplary embodiments with reference to the attached drawings, in which:

FIG. 1 is a flowchart of a user portrait generation method provided in an embodiment of the present disclosure;

FIG. 2 is a flowchart for obtaining target user data according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of a method for obtaining target portrait rules provided by an embodiment of the present disclosure;

FIG. 4 is a first frame processing diagram for detecting a backend node provided in an embodiment of the present disclosure;

FIG. 5 is a second frame processing diagram for detecting a backend node provided in an embodiment of the present disclosure;

FIG. 6 is a block diagram of a user representation generating apparatus provided by an embodiment of the present disclosure;

Fig. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

For a better understanding of the technical solutions of the present disclosure, exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Embodiments of the disclosure and features of embodiments may be combined with each other without conflict.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

It should be noted that, all actions for acquiring signals, information or data in the present application are performed under the condition of conforming to the corresponding data protection rule policy of the country of the location and obtaining the authorization given by the owner of the corresponding device.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In the related art, when the electronic device performs user portrait, generally, firstly, information of a label and crowd pack for performing user portrait and configuration information of a data source configured by a user based on a front-end configuration interface are acquired, wherein the data source can be a database for storing online data generated by service application or offline data uploaded by the user; then, the electronic equipment acquires newly generated user data to be processed from a data source according to a preset time interval, for example, 0 points per day; and performing a batch user profile analysis on the data to obtain user profile data. Because the user portrait generation method is based on a batch processing mechanism, the generated user portrait data often has delay, and can not reflect the latest dynamic state and the requirements of users in business application, so that other systems depending on the user portrait data, such as a member system, a marketing system and the like, can not respond to the user requirements in time.

To solve the above-mentioned problems, an embodiment of the present disclosure provides a user portrait generating method, please refer to fig. 1, which is a flowchart of a user portrait generating method provided by an embodiment of the present disclosure. The method can be applied to the electronic equipment deployed with the database cluster (such as a Starlock cluster), wherein the electronic equipment can be a server, and the server can be a physical server or a virtual server; of course, with the continuous progress of the technology, the electronic device may also be a terminal device, that is, the method may also be applied to the terminal device alone, for example, may be applied to an edge terminal device in an edge computing scenario, which is not limited in particular herein.

As shown in fig. 1, the user portrait creation method provided in the embodiment of the present disclosure includes the following steps S101 to S104, which are described in detail below.

Step S101, streaming obtains target user data to be processed, wherein the target user data comprises user data generated by service application in real time.

The target user data can be user data which is generated in real time by a business application and is to be subjected to user portrait under the condition of obtaining user authorization; of course, the target user data may also be user data generated off-line and uploaded to an on-line database, such as the Hive library of the data warehouse. In addition, in the embodiment of the present disclosure, the user data may be an individual user or an enterprise user, that is, the user data may be specific to an individual or an enterprise, that is, specific to a client, and is not limited herein.

For example, in the e-commerce application, the target user data may be at least one of user attribute data, user browsing data, click data, order data, merchandise interest data, and the like generated in real time by the business application.

For another example, in the financial application, the target user data may be at least one of user attribute data, user borrowing data, user repayment data, and the like generated in real time by the business application.

In the embodiment of the present disclosure, the service application for generating the service data may be one or more applications in any application scenario, which is not particularly limited herein.

Aiming at the problem of processing delay possibly existing when user portrait is performed by acquiring user data to be processed in a batch processing manner in the related art, in the embodiment of the present disclosure, the target user data to be processed may be acquired in a streaming manner when the user data is acquired, that is, the user data generated in real time by the service application is acquired in a streaming manner as the target user data, and the user data generated in real time by the service application is portrayed in time, so as to generate the latest user portrait data, thereby providing the latest user portrait data in time for display or for other service systems, so that an operation subject of the service application can respond to the user demand in time.

The streaming process is a process of writing the generated data records into the ordered data stream by the data producer, and the data producer continuously and uninterruptedly obtains the data records from the ordered data stream according to the same sequence for use or processing, in the streaming process, no preset starting or ending is usually performed, and the data records generated by the data producer are subjected to real-time response processing through a series of event nodes, wherein the data producer can be, for example, a business application for generating user data in the embodiment of the present disclosure, and the data consumer can be, for example, a user portrait application in an electronic device for executing the method in the embodiment of the present disclosure.

Step S102, obtaining preconfigured label information and crowd-sourced information, wherein the label information comprises information of at least one label, each label represents a user characteristic of a user, the crowd-sourced information comprises information of at least one crowd-sourced, and each crowd-sourced comprises a plurality of different labels.

The tag information is information of at least one tag for performing a user portrait, and each tag may correspond to one user feature of the user. For example, the tag information may be information of tags such as "gender", "age", "hobbies of interest", and the like; wherein, a Tag (Tag) is an abstract classification and generalization of a certain feature of a certain class of a specific group or object, and a Tag Value (Tag Value) is usually provided with a separable type.

For example, for "natural person", features such as "male" and "female" may be abstracted and summarized to obtain the tag "gender".

The tag value refers to the specific content contained in a certain tag, and its characteristics are in accordance with MECE (Mutually Exclusive Collectively Exhaustive) principle, that is, mutually independent and completely exhaustive.

For example, for the tag "gender", the tag values corresponding thereto may be classified into "male", "female" and "unknown".

Crowd-pack information, which is information of at least one crowd pack for carrying out user portraits; crowd-sourcing is commonly used to categorize users. For example, for individual users, the corresponding crowd-packs may be "student crowd-packs", "beauty crowd-packs", "e-commerce crowd-packs", and the like.

In the embodiment of the disclosure, the tag information and the crowd-sourced information may be obtained in advance by a user in a manner configured in a front-end interface. Of course, the tag information and crowd-sourced information may also be automatically generated by the electronic device, i.e., in some embodiments, the tag information and crowd-sourced information may also be obtained by: performing cluster analysis on the obtained target user data to obtain at least one cluster; performing feature extraction processing on target user data in each cluster to obtain at least one user feature corresponding to each cluster; obtaining at least one label according to the at least one user characteristic corresponding to each cluster, and obtaining at least one crowd pack according to the at least one label; and obtaining the label information and the crowd-sourced information according to the at least one label and the at least one crowd-sourced.

In the actual implementation process, after target user data is obtained, the electronic device performs real-time cluster analysis on the target user data to be processed to obtain at least one cluster, extracts one or more user features corresponding to the user data in each cluster, performs semantic analysis on the extracted user features to obtain corresponding labels, classifies the obtained labels to obtain at least one crowd pack, and then automatically generates label information and crowd pack information according to the labels and crowd packs obtained through analysis.

When the target user data is subjected to cluster analysis, any cluster analysis algorithm, such as a k-means algorithm, a Clara algorithm, a Clarans algorithm and the like, can be used, and the method is not particularly limited herein; in addition, when extracting the user feature of each cluster, the user feature may be extracted by a feature extraction model obtained by training in advance, where the feature extraction model may be, for example, a convolutional neural network model (CNN, convolutional Neural Networks) or may also be another network model, and the related model structure and model training method are not described herein again; in addition, after the electronic equipment automatically generates the tag information and the crowd-sourced information based on the steps, the tag information and the crowd-sourced information can be displayed for a user to check; and after the user is supplemented and confirmed, user portrait is carried out on the target user data based on the label information and crowd pack information after the user is supplemented and confirmed, and the method is not particularly limited.

And step S103, portrait is carried out on the target user data according to the label information and the crowd pack information, and the user portrait data is obtained.

After the target user data to be processed is obtained in a streaming manner according to the above step S101 and the pre-configured tag information and crowd pack information are obtained in step S102, the electronic device may perform portrait on the target user data based on the tag information and the crowd pack information, so as to obtain the latest user portrait data. For example, the latest user portrait data may be generated by screening and matching target user data according to the tag information and crowd-sourced information, and in the embodiment of the present disclosure, specific portrait processing is not particularly limited.

Step S104, in the process of portraying the target user data, the running state of the back-end node of the database cluster is obtained, and the back-end node is operated and managed according to the running state.

In the embodiment of the disclosure, the database cluster may be a starblocks cluster, where starblocks is a full-scene massive parallel processing (MPP, massively Parallel Processing) database, and is dedicated to building a very fast unified analysis experience, supporting multiple data models, such as a detail model, an aggregation model, an update model, and the like, and supporting multiple data importing modes, and is used for data analysis by building a cluster based on the starblocks, so that the processing speed can be greatly improved; of course, the database cluster may be other database clusters other than the starblocks cluster, for example, a doris cluster; in the embodiment of the present disclosure, the database cluster is exemplified as a starblocks cluster unless otherwise specified.

Considering that a back end node (BE) in a Starlock cluster, namely, the BE may BE abnormally down in the process of executing a user portrait or the processing performance is poor due to improper resource parameter configuration, the state of the back end node needs to BE detected by a user at any time, and when the down or the processing performance is found to BE poor, the back end node is manually restarted or the performance is optimized, so that the method is inconvenient for the user to operate; in addition, if restarting or performance optimization is not performed on the down back-end node in time, the electronic device may not generate user portrait data in time, so in the embodiment of the disclosure, the running state of the back-end node can be automatically detected in the process that the electronic device portrays the acquired target user data based on the deployed starblocks cluster, and the back-end node is automatically subjected to operation and maintenance management according to the running state, so that healthy running of the back-end node is ensured.

As can be seen from the above description, the user portrait generating method provided by the embodiments of the present disclosure, because the streaming processing method has the characteristics of continuous and non-interval, the processing delay when the user portrait is performed can be reduced by acquiring the target user data to be processed based on the streaming processing method and performing the processing method of the portrait; meanwhile, operation and maintenance management is carried out on the back-end node based on the operation state of the back-end node, stable and reliable operation of the back-end node can be guaranteed, downtime of the back-end node is avoided, and therefore the electronic equipment can be guaranteed to provide real-time user portrait data for users to check.

Referring to fig. 2, a flowchart of acquiring target user data is provided in an embodiment of the disclosure. As shown in fig. 2, in the above step S101, the streaming acquisition of target user data to be processed may include the following steps S201 to S204.

In step S201, data change information of user data in a target database is detected, where the target database is used to store user data generated by a service application.

The target database may be, for example, a mysql database or other relational database, and is not particularly limited herein.

In the embodiment, the target database can be detected through a Canal component or a flink-cdc component and other components to obtain the data change information, wherein the Canal component is a component developed by using Java language and used for providing incremental data subscription and consumption based on database incremental log analysis; the link-cdc component detects and captures change information of a database, for example, information such as insertion, update, and deletion of data or a data table, and records the change information in order of occurrence.

The data change information of the user data in the detection target database may be: and obtaining the data change information by detecting the change information of the preset log file corresponding to the target database.

The preset log file corresponding to the target database is used for recording the data change information of the target database. For example, where the target database is a mysql database, the pre-set log file may be a binlog file of the mysql database.

Step S202, according to the data change information, obtaining changed data records in the target database.

In step S203, the changed data record is written into the target message queue.

And a target message queue for caching the changed data record obtained based on the data change information, so that the electronic device can consume the changed data record in a streaming manner and process the changed data record piece by piece, wherein the target message queue can be a kafka message queue.

Step S204, the target user data is obtained in a streaming mode according to the changed data record in the target message queue.

That is, in view of the processing delay problem that may exist when user data to be processed is obtained in a batch manner in the related art to perform user portrait, in the embodiment of the present disclosure, through using a Canal component, a flink-cdc component, and the like, incremental data change of a target database may be detected, so that after a service application generates user data in real time and writes the user data into the target database, by detecting the incremental data change in the target database, a changed data record in the target database, that is, changed user data, is obtained, and the changed data record is written into a target message queue, so that when portrait is performed, the changed data record in the target message queue may be obtained through real-time and streaming consumption to obtain the target user data.

In this embodiment, the above-mentioned data record changed according to the target message queue may be the following data record, and the streaming target user data may be obtained: and importing the changed data record in the target message queue into a corresponding data table of the database cluster by a streaming data import (route Load) mode to obtain target user data.

In the embodiment of the present disclosure, the step S103 of portraying the target user data according to the tag information and the crowd-sourced information to obtain the user portrayed data may be: obtaining a target portrait rule according to the label information and the crowd-sourced information, wherein the target portrait rule is a structured query language (SQL, structured Query Language) statement executed by a back-end node supporting a database cluster; and drawing the target user drawing data in the data table imported into the Starlock cluster by the rear end node of the database cluster according to the target drawing rule to obtain the target user drawing data.

The target user data may include personal portrayal data and guest group portrayal data, the personal portrayal data may be portrayal data for a personal user, and the guest group portrayal data may be portrayal data for an enterprise client, for example.

Referring to fig. 3, a flowchart of a method for obtaining a target portrait rule according to an embodiment of the present disclosure is provided. As shown in fig. 3, in an embodiment of the present disclosure, the target portrait rules may be obtained by: step S301, generating rule conditions corresponding to the label information and the crowd pack information, wherein the rule conditions are used for screening user data; and step S302, converting the rule condition based on the preset rule of the database cluster to obtain the target portrait rule.

That is, in the embodiment of the present disclosure, after the tag information and the crowd-sourced information are provided to the electronic device, the electronic device may sort the tag information and the crowd-sourced information into the json (JavaScript Object Notation) form of rule conditions, and then, in order to facilitate the execution of the database cluster, for example, the starblocks cluster, convert the json form of rule conditions into sql statements in advance, so as to improve the image speed.

In an embodiment of the present disclosure, the acquiring the operation state of the backend node of the database cluster in step S104, and performing operation and maintenance management on the backend node according to the operation state may include: detecting the running state of a rear end node of the database cluster according to a preset time interval; and restarting the back-end node under the condition that the back-end node is determined to be abnormally operated.

In the embodiment, the daemon process tool, for example, a supervisor daemon process realized based on the python language, detects each back-end node in the Starlock cluster, and restarts the back-end node with abnormality under the condition that the abnormality of the running state is detected, so that the stable and reliable running of the back-end node is ensured, and the delay of the timely generation of user portrait data caused by the downtime of the back-end node is avoided.

For ease of understanding, please refer to fig. 4, which is a first frame processing diagram for detecting a backend node provided in an embodiment of the present disclosure. As shown in fig. 4, for the backend nodes in the starlocks cluster, namely BE1, BE2 and BE3, the state detection of each backend node can BE realized by configuring the configuration files, namely the corresponding configuration files, of the daemon 1, the corresponding configuration files, namely the corresponding configuration files, of the daemon 2 and the corresponding configuration files, namely the corresponding configuration files, of the daemon 3, so that the backend nodes can BE guaranteed to BE started in time after abnormality, such as downtime, wherein FE shown in fig. 4 represents the front-end node of the starlocks cluster, and Broker represents a process for executing Broker import in the starlocks cluster; in addition, the related method for setting the super is not described in detail herein because of the detailed description in the related art.

It should be noted that in the embodiment shown in fig. 4, the super is not used to detect the front end node and the Broker process of the starblocks cluster, and in actual implementation, the two processes may be detected at the same time to further improve the system stability, which is not limited in particular herein.

It should be further noted that, in the embodiment of the present disclosure, the step S104 of obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state may also be: detecting the running state of a rear end node of the database cluster according to a preset time interval; under the condition that the operation of the back-end node is abnormal, acquiring an operation index corresponding to the database cluster; generating an abnormality analysis report according to the operation index; and sending the anomaly analysis report to target terminal equipment, wherein the anomaly analysis report is used for performing performance optimization processing on the database cluster by a service user, the service user corresponds to the database cluster, and the target terminal equipment comprises terminal equipment used by the service user.

In the embodiment of the present disclosure, the operation indexes corresponding to the database cluster, for example, the starblocks cluster, may be: the central processing unit (CPU, central Processing Unit) uses the rate, the memory usage information, the running environment, such as JDK environment, running log of each node of the database cluster, network environment information, storage read-write information, call volume per second, whether there is slow sql, etc.

That is, considering that if an abnormality occurs in a back-end node, for example, when a crash occurs, there may be an abnormality or improper setting in a task resource, an operation environment, or an internal operation state of the back-end node, so that in a case where an abnormality occurs in a certain back-end node of a starblocks cluster, in order to facilitate problem investigation of a service user corresponding to the starblocks cluster, for example, a DBA, an operation maintenance person, a developer, or a cluster manufacturer, etc., to improve cluster stability, the method may further facilitate the performance optimization processing of the starblocks cluster by the service personnel in time by acquiring an operation index of the starblocks cluster when an abnormality occurs in the back-end node of the starblocks cluster is detected, according to the operation index, and pushing the abnormality analysis report to a terminal device used by the service personnel, for example, a mobile phone, a tablet computer, etc.

For ease of understanding, please refer to fig. 5, which is a second frame processing diagram for detecting a backend node according to an embodiment of the present disclosure. As shown in fig. 5, for any backend node in the starlocks cluster, for example, BE1 shown in fig. 5, when it detects that a downtime occurs in the backend node, a script for acquiring an operation index of the starlocks cluster is operated while restarting the backend node based on a daemon, for example, auto_check.sh shown in fig. 5 acquires the operation index of the starlocks cluster, generates an anomaly analysis report according to the operation index, and sends the anomaly analysis report to a service user shown in fig. 5 for viewing in a mail form.

In addition, in some embodiments, the step S104 of obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state may also be: detecting the running state of the back-end node according to a preset time interval; under the condition that the back-end node is determined to be abnormal in operation, acquiring an operation index corresponding to a database cluster and operation resource parameter information corresponding to the database cluster, wherein the operation resource parameter information comprises information of memory resources and CPU resources which are configured for the back-end node of the database cluster in advance; and under the condition that the operation resource parameter information meets the preset condition according to the operation index, adjusting the operation resource parameters of the database cluster to perform performance optimization processing on the database cluster.

The operation index may include, for example, a CPU usage rate, memory usage information, and the like, where the preset condition may be that the CPU usage rate is greater than or equal to a first preset threshold value within a preset duration, and/or the memory usage information indicates that the memory usage rate is greater than or equal to a second preset threshold value within the preset duration, where the first preset threshold value and the second preset threshold value may be set as needed. That is, in this embodiment, if an abnormality of the database cluster, for example, a starblocks cluster is detected, it may be determined whether the CPU utilization rate, the memory utilization rate, etc. are high for a long time, if so, it may be because the CPU resources currently configured for the database cluster, for example, the number of CPU cores and the memory resources are inappropriate, and at this time, it may be considered that some CPU resources and memory resources are allocated more appropriately; of course, this is merely illustrative, and in an actual implementation, when adjusting the operating resource parameters of the database cluster, the adjustment may also be performed based on a pre-configured resource adjustment policy, which is not limited herein.

It should be noted that, one or more embodiments of performing operation and maintenance management on the backend node according to the operation state of the backend node of the database cluster may be used alone or in combination, or may also be used in combination with other operation and maintenance management manners, which are not limited herein.

After obtaining the user portrait data through the above method, for the convenience of viewing by the user, in an embodiment of the present disclosure, the method further includes: generating target presentation data corresponding to the user portrait data, wherein the target presentation data comprises image data and/or form data; and displaying the target display data.

That is, considering that the obtained user portrait data may be only some numbers, the visibility is poor, in order to facilitate the user to view, after the user portrait data is obtained, visual analysis may be performed on the user portrait data based on a visual analysis component, for example, fineBI, so as to obtain target display data with better visibility, which is formed by image data, table data, and the like, for the user to view, thereby improving the user experience.

In summary, according to the method provided by the embodiment of the disclosure, the target user data to be processed is obtained through streaming, and the target user data is imaged based on the obtained tag information and crowd package information, so that the user data generated in real time by the service application can be imaged timely, the timeliness of the user image data is improved, and a user or a service system which relies on the user image data to make a service decision or execute service processing can respond to a user demand based on the user image data timely; in addition, the new generation database cluster, such as the Starlock cluster, is used for portraying, so that the generation speed of user portrayal data can be further improved, and meanwhile, in the process of acquiring the user portrayal data, the operation state of the back end node in the database cluster is detected to carry out operation and maintenance management on the back end node, so that the rapid generation of the user portrayal data can be ensured; in addition, after the user scribing data is obtained, the target display data comprising the image data and/or the form data can be generated for the user to view, and the user experience can be improved.

It will be appreciated that the above-mentioned method embodiments of the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the description of the present disclosure. It will be appreciated by those skilled in the art that in the above-described methods of the embodiments, the particular order of execution of the steps should be determined by their function and possible inherent logic.

In addition, the disclosure further provides a user portrait generating device, an electronic device and a computer readable storage medium, and the above may be used to implement any user portrait generating method provided in the disclosure, and corresponding technical schemes and descriptions and corresponding descriptions referring to method parts are not repeated.

Fig. 6 is a block diagram of a user portrait generating device according to an embodiment of the present disclosure.

Referring to fig. 6, an embodiment of the present disclosure provides a user portrait generating apparatus including: a user data acquisition unit 601, a configuration information acquisition unit 602, a portrait unit 603, and a daemon unit 604.

The user data obtaining unit 601 is configured to obtain, in a streaming manner, target user data to be processed, where the target user data includes user data generated in real time by a service application.

In some embodiments, the user data obtaining unit 601 may be configured to, when obtaining target user data to be processed in a streaming manner: detecting data change information of user data in a target database, wherein the target database is used for storing the user data generated by service application; obtaining changed data records in the target database according to the data change information; writing the changed data record into a target message queue; and obtaining the target user data in a streaming mode according to the changed data record in the target message queue.

In some embodiments, the user data obtaining unit 601, when detecting data change information of user data in the target database, may be configured to: and obtaining data change information by detecting change information of a preset log file corresponding to the target database.

In some embodiments, the user data obtaining unit 601 may be configured to, when obtaining the target user data in a streaming manner according to the changed data record in the target message queue: and importing the changed data record in the target message queue into a corresponding data table of the database cluster in a streaming data importing mode so as to obtain target user data.

The configuration information obtaining unit 602 is configured to obtain preconfigured tag information and crowd-pack information, where the tag information includes information of at least one tag, each tag represents a user feature of a user, the crowd-pack information includes information of at least one crowd-pack, and each crowd-pack includes a plurality of different tags.

The portrait unit 603 is used for portrait the target user data according to the label information and crowd pack information to obtain user portrait data.

The daemon unit 604 is configured to obtain an operation state of a backend node of the database cluster during a process of portraying target user data, and perform operation and maintenance management on the backend node according to the operation state.

In some embodiments, the portrayal unit 603 may be configured to, when portraying the target user data based on the tag information and the crowd-sourced information, obtain the user portrayal data: obtaining a target portrait rule according to the label information and the crowd pack information, wherein the target portrait rule is a structured query language statement executed by a back end node supporting a database cluster; portraying, by a backend node of the database cluster, target user data imported into the data table according to target portrayal rules to obtain user portrayal data, wherein the user portrayal data comprises personal portrayal data and/or guest group portrayal data.

In some embodiments, the portrayal unit 603, when obtaining the target portrayal rules based on the tag information and the crowd-sourced information, may be configured to: generating rule conditions corresponding to the label information and the crowd pack information, wherein the rule conditions are used for screening user data; and converting rule conditions based on preset rules of the database cluster to obtain target portrait rules.

In some embodiments, the daemon unit 604, when acquiring the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state, may be configured to: detecting the running state of a rear end node of the database cluster according to a preset time interval; and restarting the back-end node under the condition that the back-end node is determined to be abnormal in operation.

In some embodiments, the daemon unit 604, when acquiring the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state, may be configured to: detecting the running state of a rear end node of the database cluster according to a preset time interval; under the condition that the operation of the back-end node is abnormal according to the operation state, acquiring an operation index corresponding to the database cluster; generating an anomaly analysis report according to the operation index; and sending an anomaly analysis report to target terminal equipment, wherein the anomaly analysis report is used for performing performance optimization processing on the database cluster by a service user, the service user corresponds to the database cluster, and the target terminal equipment comprises terminal equipment used by the service user.

In some embodiments, the user portrait creation apparatus 600 further includes a presentation unit configured to: after obtaining the user portrait data, generating target display data corresponding to the user portrait data, wherein the target display data comprises image data and/or table data; and displaying the target display data.

Referring to fig. 7, an embodiment of the present disclosure provides an electronic device including: at least one processor 701; at least one memory 702, and one or more I/O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs executable by the at least one processor 701, the one or more computer programs being executable by the at least one processor 701 to enable the at least one processor 701 to perform the user portrayal generation method described above.

The disclosed embodiments also provide a computer readable storage medium having a computer program stored thereon, wherein the computer program when executed by a processor implements the user portrait generation method described above. The computer readable storage medium may be a volatile or nonvolatile computer readable storage medium.

Embodiments of the present disclosure also provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when executed in a processor of an electronic device, performs the user portrayal generation method described above.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable storage media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).

The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, random Access Memory (RAM), read Only Memory (ROM), erasable Programmable Read Only Memory (EPROM), static Random Access Memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and may include any information delivery media.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.

The computer program product described herein may be embodied in hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable user portrayal generation device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable user portrayal generation device, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable user image generation apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable user portrayal generation apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable user portrayal generation apparatus, or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable user portrayal generation apparatus, or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, it will be apparent to one skilled in the art that features, characteristics, and/or elements described in connection with a particular embodiment may be used alone or in combination with other embodiments unless explicitly stated otherwise. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A user portrayal generation method, applied to an electronic device deployed with a database cluster, the method comprising:

2. The method of claim 1, wherein the streaming acquisition of target user data to be processed comprises:

detecting data change information of user data in a target database, wherein the target database is used for storing the user data generated by service application;

obtaining a changed data record in the target database according to the data change information;

writing the changed data record into a target message queue;

and obtaining the target user data in a streaming mode according to the changed data record in the target message queue.

3. The method of claim 2, wherein said streaming said target user data based on said changed data record in said target message queue comprises:

And importing the changed data record in the target message queue into a corresponding data table of the database cluster in a streaming data importing mode to obtain the target user data.

4. The method of claim 3, wherein the portraying the target user data based on the tag information and the crowd-sourced information to obtain user portrayal data comprises:

obtaining a target portrait rule according to the label information and the crowd-sourced information, wherein the target portrait rule is a structured query language statement which supports the back-end node execution of the database cluster;

and carrying out portrayal on target user data imported into the data table by a rear end node of the database cluster according to the target portrayal rule to obtain the user portrayal data, wherein the user portrayal data comprises personal portrayal data and/or guest group portrayal data.

5. The method of claim 4, wherein the obtaining the target representation rule based on the tag information and the crowd-sourced information comprises:

generating rule conditions corresponding to the tag information and the crowd pack information, wherein the rule conditions are used for screening user data;

And converting the rule conditions based on the preset rules of the database cluster to obtain the target portrait rule.

6. The method of claim 1, wherein the obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state comprises:

detecting the running state of the back-end node of the database cluster according to a preset time interval;

and restarting the back-end node under the condition that the back-end node is determined to be abnormally operated.

7. The method of claim 1, wherein the obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state comprises:

acquiring an operation index corresponding to the database cluster under the condition that the operation of the back-end node is abnormal;

generating an anomaly analysis report according to the operation index;

and sending the anomaly analysis report to target terminal equipment, wherein the anomaly analysis report is used for performing performance optimization processing on the database cluster by a service user, the service user corresponds to the database cluster, and the target terminal equipment comprises terminal equipment used by the service user.

8. A user portrayal generation apparatus for use in an electronic device having a database cluster deployed therein, the apparatus comprising:

9. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,,

the memory stores one or more computer programs executable by the at least one processor to enable the at least one processor to perform the user representation generation method of any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the user portrayal generation method according to any one of claims 1-7.