CN114265856A

CN114265856A - Method and device for providing data service and electronic equipment

Info

Publication number: CN114265856A
Application number: CN202111348710.XA
Authority: CN
Inventors: 林书翰; 柏正权; 周赛玉
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2021-11-15
Filing date: 2021-11-15
Publication date: 2022-04-01

Abstract

The embodiment of the application discloses a method, a device and electronic equipment for providing data service, wherein the method comprises the following steps: determining a plurality of selectable dimensions according to received dimension configuration information, wherein the dimension configuration information comprises an identifier of the created dimension and a physical table implementation mode corresponding to the dimension; determining a plurality of selectable data indexes according to received data index configuration information, wherein the data index configuration information comprises an identifier of the created data index, a physical table implementation mode corresponding to the data index and a dimensionality supported by the data index; and providing data service for the data demand party according to the plurality of selectable data indexes, the dimension supported by each data index and the corresponding physical table implementation mode. Through the embodiment of the application, the capability of externally transmitting the service can be provided while the definition of the data index is realized, and the use threshold of the data index is reduced.

Description

Method and device for providing data service and electronic equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for providing data services, and an electronic device.

Background

With the "explosive" growth of various types of data, their potentially enormous value is being explored. However, in the face of large data, if the data cannot be organized and stored in an orderly and structured manner, a data cost disaster may come before the value is discovered. Therefore, it becomes a very critical issue to construct a data architecture of an enterprise big data system to ensure that data quickly supports applications and drives application development.

In the construction process of a big data system data architecture, data index management is one of important contents. The data index system is constructed because the most primitive data is usually detail data directly generated in the system, which is often a strip of data records, for example, in an e-commerce system, a trade order record, etc. may be included. However, the raw data has no measurable concept, and is data which has no statistical significance, and cannot be directly used in the application. Therefore, it is necessary to build a data index system on the top layer of the most original detail data, in order to define some data indexes with statistical significance, such as GMV (total volume of transaction), etc. Then, the upper layer application can perform data analysis and the like through the data index with statistical significance, and further can provide better service for users such as merchants or consumers and the like.

In the initial development stage of an enterprise, because a data research and development mode generally evolves along with the development of application, the data index system is also vertically established based on application units, and different vertical applications bring different chimney-type data index systems. However, with the development of enterprises, on one hand, the data scale is rapidly expanding, and the number of vertical application units is increasing, on the other hand, data required by large data-based applications is not only of a certain vertical unit, but also has core competitiveness only by using data with various data types, and therefore, data construction across vertical units is followed. However, the data use threshold is higher and higher due to disordered data calling and copying, resource waste caused by repeated construction of data indexes, ambiguity caused by different definitions of the data indexes and the like.

For example, in an e-commerce system, there may be tens of thousands of data indicators, where there may be some data indicators named the same but with inconsistent defined apertures. For example, there may be more than ten definitions of such an index uv alone, and thus the problem to the data application side of the upper layer is: are uv, which should be used? Are uv, why are the data results returned different? And so on.

In the prior art, some schemes for performing standardized definition on data indexes exist, so that the occurrence of the condition that the same data index has a plurality of defined calibers can be effectively reduced. However, after the registration is completed, the specific data index cannot provide the capability of exporting services to the outside, except for displaying. This leads to a high threshold for the use of such data indicators, and since the defined indicators are difficult to apply widely, it eventually leads to the developers of the indicators losing the motivation to continue to maintain the indicators.

Therefore, how to reduce the threshold for using the data index while implementing the definition of the data index becomes a technical problem to be solved by those skilled in the art.

Disclosure of Invention

The application provides a method and a device for providing data service and electronic equipment, which can realize data index definition, provide the capability of externally exporting service and reduce the use threshold of data indexes.

The application provides the following scheme:

a method of providing data services, comprising:

determining a plurality of selectable dimensions according to received dimension configuration information, wherein the dimension configuration information comprises an identifier of the created dimension and a physical table implementation mode corresponding to the dimension;

determining a plurality of selectable data indexes according to received data index configuration information, wherein the data index configuration information comprises an identifier of the created data index, a physical table implementation mode corresponding to the data index and a dimensionality supported by the data index; the physical table implementation mode of the same data index on the same dimension has uniqueness;

and providing data service for the data demand party according to the plurality of selectable data indexes, the dimension supported by each data index and the corresponding physical table implementation mode.

Wherein the determining a plurality of selectable data metrics according to the received data metric configuration information includes:

and providing a first configuration interface, wherein the first configuration interface comprises an operation option for defining a data index according to a preset specification, an operation option for configuring a physical table implementation mode for the currently defined data index, and an operation option for performing dimension configuration for the currently defined data index.

and providing a second configuration interface, wherein the second configuration interface comprises an operation option for selecting a physical table, an operation option for designating a target field in the selected physical table as a currently defined data index, and an operation option for performing dimension configuration on the data index.

The dimensionality supported by the data index comprises a basic dimensionality and an extended dimensionality, wherein the basic dimensionality is a self-owned dimensionality in a physical table associated with the data index, and the extended dimensionality is a higher dimensionality obtained by aggregation based on the self-owned dimensionality.

Wherein, still include:

and generating an expanded data index based on the received plurality of data indexes, and adding the expanded data index into an optional data index set.

Wherein, according to the multiple selectable data indexes, the dimension supported by each data index, and the corresponding physical table implementation manner, providing data service for the data demander comprises:

providing identifiers of the plurality of selectable data indexes and identifiers of dimensions supported by each data index to a data demander;

and determining a target data index and a target dimension required by the data demander, and generating a target query statement according to a physical table implementation mode corresponding to the target data index and the target dimension respectively and a preset query statement generation template so as to provide a data aggregation calculation result of the target data index on the target dimension for the data demander through the target query statement.

Wherein the providing the identifiers of the plurality of selectable data indicators to the data demander and the identifier of the dimension supported by each data indicator includes:

after an ad hoc query request submitted by the data demander is received, providing the identifiers of the multiple selectable data indexes and the identifier of the dimension supported by each data index to the data demander through a target interface;

the providing of the data aggregation calculation result of the target data index on the target dimension for the data demander includes:

and querying the data in the physical table corresponding to the target data index by using the target query statement, performing aggregate calculation on the queried data in the target dimension, and returning a data aggregate calculation result through the target interface.

Wherein the providing of the data aggregation calculation result of the target data index on the target dimension for the data demander includes:

and generating a service interface and a corresponding data scheduling task, and providing the service interface for the data demand side so as to return a corresponding data aggregation calculation result by executing the query statement in the process of executing the data scheduling task according to the calling condition of the service interface.

providing the identifiers of the multiple selectable data indexes, the identifier of the dimensionality supported by each data index and a corresponding physical table implementation mode to a data demander, so that the data demander can generate a target query statement according to the physical table implementation mode after selecting the target data index and the target dimensionality;

and after receiving a query request submitted by the data demander, returning a data aggregation calculation result of the target data index on the target dimension by executing a target query statement carried in the query request.

An apparatus for providing data services, comprising:

the dimension configuration unit is used for determining a plurality of selectable dimensions according to received dimension configuration information, wherein the dimension configuration information comprises an identifier of the created dimension and a physical table implementation mode corresponding to the dimension;

the index configuration unit is used for determining a plurality of selectable data indexes according to received data index configuration information, wherein the data index configuration information comprises the identification of the created data indexes, the physical table implementation mode corresponding to the data indexes and the dimensionality supported by the data indexes; the physical table implementation mode of the same data index on the same dimension has uniqueness;

and the service providing unit is used for providing data services for the data demand party according to the plurality of selectable data indexes, the dimension supported by each data index and the corresponding physical table implementation mode.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any of the preceding claims.

An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of the preceding claims.

According to the specific embodiments provided herein, the present application discloses the following technical effects:

according to the embodiment of the application, a plurality of selectable dimensions can be determined according to the received dimension configuration information, wherein the specific dimension configuration information can further comprise a physical table implementation mode corresponding to the dimension. In addition, a plurality of optional data indexes can be determined according to the received data index configuration information, wherein the data index configuration information may include a physical table implementation manner corresponding to the data index and a dimensionality supported by the data index. Therefore, the data index configuration information specifies the specific support for aggregation in which dimensions, the specific corresponding physical table implementation mode, and the specific dimension configuration information configures the physical table implementation mode of the specific dimension, so that the specific index management system can not only display the registered index, but also provide the capability of exporting services to the outside. Moreover, the same data index can be realized only in one physical way in the same dimension, so that data service can be better provided for a data demand party, and the situation that the same index returns various different aggregation results in the same dimension is avoided.

Of course, it is not necessary for any product to achieve all of the above-described advantages at the same time for the practice of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 2 is a flow chart of a method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a dimension configuration interface provided by an embodiment of the present application;

4-1, 4-2 are schematic diagrams of interfaces for configuring the index based on the index definition specification provided in the embodiments of the present application;

FIG. 5 is a schematic diagram of an interface for index configuration based on a physical table according to an embodiment of the present application;

fig. 6 is a schematic diagram illustrating index information provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of an interface for obtaining ad hoc query services according to an embodiment of the present application;

FIG. 8 is a schematic view of an apparatus provided by an embodiment of the present application;

fig. 9 is a schematic diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived from the embodiments given herein by a person of ordinary skill in the art are intended to be within the scope of the present disclosure.

In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, some concepts related to the embodiments are first described below.

Entity: the subject object described by the data, e.g., item, user, etc.

Data indexes are as follows: the index is a quantitative measurement standard for data and is used for measuring the numerical definition of a certain characteristic of a thing entity.

And (3) measurement: the metric is a value of a quantifiable measure in the application process node and is used for describing a value which cannot be split in a fact.

The application process comprises the following steps: an application event in the application occurrence, or an application state, such as a payment process for placing an order, is described.

Data field: a collection of application processes that make up a data field, such as a transaction field, a log field, etc.

Dimension: a dimension is an environment that describes a metric, represents a class of attributes of an entity object, and is used to describe an entity object, such as a good, a category, a user, a city, etc.

Dimension attribute: the series of related attributes for describing the dimension are attached to a certain dimension, and for example, for the product dimension, the attributes may include the name of the product, the storage condition, and the like.

In practical applications, the definition of the data index can be performed by a technician on the database side, and then, the data index can be provided for the application on the upper layer. For example, if a certain data index is defined as "GMV for order payment in 7 days", the definition information about the index may be directly provided to the application, for example, the data field, the application process, the caliber specification, etc. of the specific index may be included, but if the specific application wants to obtain the data corresponding to the index, the following problems may be faced: first, the data demander needs to have the ability to write query statements such as SQL (standard computer language for accessing and processing databases); second, because the specific index definition only implements the definition of the logical layer, but does not define the physical implementation thereof, in practical applications, the number of physical tables at the bottom is large, and there are often cases where different teams develop different physical tables, which may have the same or similar names (for example, different teams may develop "list of payment for placing orders in a trade", etc.). Therefore, even if the application side has the capacity of writing query statements such as SQL and the like, the situation that different query results are obtained aiming at the same index due to the fact that the physical tables are not uniform and the like can occur.

In view of the above situation, the embodiments of the present application provide corresponding solutions. In the scheme, entries for registering the dimension and the data index respectively can be provided for users at the database side, and a physical table implementation manner of a specific dimension can be specified when dimension registration is performed, and a physical table implementation of a specific data index (that is, mapping to a specific physical table, a field, and the like) can also be specified when data index definition is performed, and specific indexes can be specified to support aggregation calculation in which dimensions, so that only one physical implementation of one data index in the same dimension is ensured. Therefore, for a data demand side, only a specific data index and one or more dimensions need to be selected, the database side can automatically generate corresponding query statements such as SQL (structured query language) according to the specific data index and the physical table implementation mode corresponding to the dimensions, and further can provide a data aggregation calculation result corresponding to the specific data index on the specific dimension for the application side based on the query statements.

The defining (or accessing) manner of the data indicator may include defining directly based on preset specifications, including specifying a specific data field, an application process, binding a physical table, then registering a related metric field in the application process, then registering a related data indicator, and so on. That is, in the process of defining the indexes according to the specific specification, the physical table and the related fields may be bound to configure the physical table implementation manner of the specific indexes, so as to ensure that one data index has only one physical implementation in one dimension.

In addition, the embodiment of the application also provides a mode for realizing data index access more quickly. That is, for some physical tables that already have some statistical data fields, a field in the physical table may be directly designated as the currently defined data index. Therefore, a series of complex configurations such as a data domain, an application process and the like are not needed, and the data index can be accessed quickly. Certainly, in specific implementation, in order to implement unification of the defined apertures of the data indexes, some management may be performed on a data index access manner based on a physical table, for example, a data index "GMV" is defined for a certain physical table before, and it is stated that the index supports aggregation calculation on a commodity dimension to obtain GMV data of a commodity; later, if a GMV index is registered based on another physical table, and the aggregation calculation can be carried out in the commodity dimension in the index definition configuration, an error can be reported at the moment, so that the condition that the same data index has only one physical realization in the same dimension is ensured.

In addition, there may be various ways to provide the data aggregation calculation result to the application side. In one way, ad hoc query services can be provided for some users who do not have the ability to write SQL, and in particular, a target interface can be provided for such users, in which a plurality of selectable data metrics can be presented, and in which dimensions the aggregate calculations are supported by the data metrics. The user can select specific data indexes and corresponding dimensions according to the own requirements, so that the database side can determine the implementation modes of the respective bound physical tables according to the indexes and the dimensions selected by the user, further query statements such as SQL (structured query language) can be automatically generated according to a preconfigured SQL template, data in the specific physical tables can be queried based on the SQL statements, then aggregation calculation is carried out based on information such as a specific associated aggregation mode, and data aggregation calculation results are displayed through the interface.

In another mode, after the application side selects the specific required data index and the corresponding dimension, a specific service interface can be provided for the application side. In this way, the application side can obtain the data aggregation result of the specific data index on the corresponding dimension in the application by calling the service interface. In the process of calling the service interface, data query can be performed in the corresponding bound physical table on the data warehouse side based on the automatically generated SQL statement, and aggregation calculation is performed based on information such as an aggregation mode corresponding to a specific data index, so that an aggregation calculation result is obtained.

Certainly, in other manners, if the application side has the capability of writing the SQL, the application side may also provide the specific data index, the dimension, and the corresponding physical table implementation manner information to the application side, so that the application side may write the SQL statement by itself, and perform the query of the data and the corresponding aggregation calculation. At this time, in the application embodiment, a specific physical table implementation manner may be provided, so that when an application side queries the same data index in the same dimension, a uniform aggregation calculation result may be obtained.

From the perspective of system architecture, referring to fig. 1, embodiments of the present application may provide a data index access and management system, which may include a storage and computation layer and a core capability layer. The storage computing layer can comprise an index management module, through which a user at the database side can access data indexes and dimensions, and can bind specific physical table implementation modes, so that the same data index has only one physical implementation mode in the same dimension. On the basis, the capability of automatically generating query statements such as SQL and the like can be provided. At the core capability layer, the ad hoc query service can be provided for the data demand side, or a service interface can be provided for the data demand side. For example, when the ad hoc query service is improved, the data demander may select data indexes and dimensions, and accordingly, may generate SQL automatically to implement data query and aggregate calculation processing from the corresponding physical table, and feed the aggregate calculation result back to the data demander in real time for viewing. When the service interface is provided, the data demander can also select the data index and the dimensionality in advance, at the moment, the core capability layer can generate the service interface and schedule a task for the data demander, and the service interface can be provided for the data demander, so that the application side can call the service interface in own application to obtain a data aggregation calculation result of the specific data index under the corresponding dimensionality. In this process, in the process of calling the service interface, the scheduling task may be executed, and the statement executed by the scheduling task may specifically be an SQL statement automatically generated by the storage computation layer. Based on the capabilities of the ad hoc query, the service interface provision and the like, data can be exposed to an upper application scene. For example, specific application scenarios may include self-service analysis, AB experiments, logistics centers, retail centers, and so on.

The following describes in detail a specific technical solution provided in an embodiment of the present application.

First, an embodiment of the present application provides a method for providing a data service, and referring to fig. 2, the method may specifically include:

s201: determining a plurality of selectable dimensions according to received dimension configuration information, wherein the dimension configuration information comprises an identification of the created dimension and a physical table implementation mode corresponding to the dimension.

In particular implementation, as mentioned above, a dimension is an environment describing a metric, representing a class of attributes of an entity object, and used to describe an entity object, such as a commodity, a category, a user, a city, etc. In the case where a data pointer is defined, if one wants to view data about the data pointer, in addition to needing to know from which field of which physical table data reading is performed, what is the aggregation manner, the following information is generally needed to be known: the data index needs to be viewed in what dimension, what the dimension is, and so on.

For example, a certain index is GMV, the data of the index may be aggregated from multiple dimensions, for example, including a user dimension, a product dimension, a category dimension, and the like, and the final aggregated calculation result may be the GMV data of the user, the GMV data of a certain product, the GMV data of a certain category, and the like. Obviously, even for the same data index, when the aggregation calculation is performed in different dimensions, the obtained results are different.

In order to embody two layers of information, namely, a data index and a dimension, one way is to define the data index for different dimensions respectively when defining a specific data index, but this may cause a transient expansion of the data index. For example, the indexes are GMV-related indexes, but in consideration of the dimension, the user GMV, the commodity GMV, the category GMV, and the like need to be defined separately, and it is obvious that the number of the indexes is multiplied, which is not beneficial to subsequent management.

For this reason, in the embodiment of the present application, a configuration-based implementation may be adopted. Specifically, the dimensions and the data indexes may be defined and configured separately, and since the number of dimensions is usually small and the dimension table is usually slowly changed with respect to some detail tables, summary tables, and the like, the dimensions may be defined first. After some dimensions are defined, defining the data indexes, and configuring the dimensions on which the specific data indexes support aggregation and summarization when the specific data indexes are defined. Thus, assuming that there are m dimensions and n data indicators, a total of m + n configuration operations are required, and if the data indicators are defined for each dimension, m × n data indicators are required to be defined.

When defining dimensions, the method mainly defines specific identifiers such as dimension names and the like and associated physical table implementation manners. The physical table implementation manner is mainly used for mapping the specifically defined dimension to a certain field in a certain specific dimension table. For example, as shown in fig. 3, which is a specific embodiment of an interface for creating a dimension, an area shown at 31 therein may be used to edit basic information of the newly created dimension, including a dimension name, a dimension key, a dimension attribute, and the like, where the basic information may be customized by a creator. The area shown in 32 may be an area for configuring a physical table implementation, where the area includes options for filling out a table name, a dimension key, a dimension attribute, and the like, and each item of information in the area needs to be information of a physical table that actually exists in the data warehouse. For example, if a dimension named "user" is currently defined, it is possible to configure which dimension table and which field specifically correspond to the dimension in the area shown in fig. 32. In this way, the index management system can be made aware of the physical meaning of the currently defined dimension.

In this manner, a plurality of dimensions may be defined, which may include, for example, users, goods, categories, parties, and so forth, as previously described. The specifically defined dimension name, dimension key, and the like cannot be repeated with the defined dimension, so that in specific implementation, when the dimension name input by the user is received, whether the dimension name exists or not can be determined, if so, the user can be prompted, and the dimension key can be similarly processed.

S202: determining a plurality of selectable data indexes according to received data index configuration information, wherein the data index configuration information comprises an identifier of the created data index, a physical table implementation mode corresponding to the data index and a dimensionality supported by the data index.

After the configuration of the dimensions is completed, the data index may be defined, and in the embodiment of the present application, when the data index is specifically defined, the data index may also be defined in a configured manner. That is, an interface for configuring the data index may be provided, and a user on the database side may complete access to the data index by inputting or selecting various pieces of information in the interface.

In the embodiment of the application, a plurality of data index access modes can be provided. For example, in the first mode, the access may be performed based on a mode defined by a normalized index. At this time, a first configuration interface may be provided, where the first configuration interface may include an operation option for performing data index definition according to a preset specification, an operation option for configuring a physical table implementation manner for the currently defined data index, and an operation option for performing dimension configuration for the currently defined data index.

The specific data index definition specification may be an existing specification as long as the definition caliber of the specific data index is unified. For example, in one of the canonicalized definition standards, concrete metrics can be abstracted as: atom index, time period, other modifiers, and the like. For example, the indexes that have been defined in the past are: a merchant has committed for the last 7 days. Whereas in the above specification definition, the index can be structurally decomposed into: atomic index (amount of payment order) + time period (last 7 days) + modifier-seller type.

In a specific implementation manner, regarding a first configuration interface provided for a user, a first portion of the first configuration interface may be as shown in fig. 4-1, in which a type (which may include an atomic index, a derivative index, and the like), a name, a field, a description, and the like of a data index that needs to be defined currently may be input, and a specific name and the like may be customized by the user. Of course, there is also a deduplication issue here, and if a name already exists, no duplicate definitions are allowed. In addition, a data domain, an application process and the like suitable for specific indexes can be configured, so that the application of the same data index in various scenes is realized. In addition, the second part of the first configuration interface may be, as shown in fig. 4-2, mainly used for configuring the physical table implementation manner and the supported dimensions of the currently defined data index. As to the physical table implementation manner, specifically as shown at 41, the table name, the field, the aggregation manner, the time period, the time type, the accumulation manner, and the like of the physical table may be configured. The physical table is a fact table already existing in the data warehouse, and may include, for example, a most basic list, a summary table having some statistical meaning fields, and the like. The detail list main user records some most basic detail data, provides long-term precipitation of application system detail data, and provides historical data support for expansion of analysis requirements. The summary table is a fact table obtained by performing high-granularity summary aggregation according to specific dimensions according to data in the detail table, for example, the summary table is performed according to commodities, categories and the like. By configuring the table name, the field, the aggregation mode and the like of a specific physical table, it can be determined which field of which table needs to be accessed for a certain data index, and according to which mode, the aggregation calculation is performed. The time types may be divided into real-time indicators, offline indicators and other types, and different time types mainly affect the execution period of the subsequent scheduling task, for example, for a real-time indicator, a specific scheduling task may be executed once in five minutes, for an offline indicator, a scheduling task may be executed once a day, and the like.

Additionally, the portion shown at 42 in FIG. 4-2 may be used to configure which dimensions the current data metrics support to aggregate in. In particular, the dimensions created in the foregoing manner may be selected. For example, the currently defined data index is "GMV", and the selected dimension may include a user, a commodity, a category, and the like, that is, the index may support aggregation in the dimensions of the user, the commodity, the type, and the like, to obtain a GMV value of a certain user, a GMV value of a certain commodity, a GMV value of a certain category, and the like.

It should be noted here that, particularly when configuring the dimensions supported by the currently defined data index, the specific dimensions may include both a base dimension and an extended dimension, where the base dimension may be an owned dimension in a physical table associated with the data index, and the extended dimension may be a higher dimension obtainable by aggregating based on the owned dimension. In which the so-called owned dimension, i.e. the dimension field that exists in the particular physical table itself, is used. For example, as described above, the physical table specifically bound to the data index may be a summary table, and the summary table is a data table obtained by performing some summary processing on the basis of the detail table, so that there are some fields that have statistical significance, for example, a certain summary table may be summarized in a commodity dimension based on a certain detail table, and then the "commodity" dimension belongs to the owned dimension of the summary table. In addition, although there may be no field in the detail table itself that has been processed such as summary, the detail table may have some foreign keys, and the nature of such foreign keys is also of the dimension of the detail table itself. For example, a trade order payment schedule may include a series of foreign keys for items, users, time, channels, etc. that describe who the order was purchased, what was purchased, when it was purchased, what channels were purchased, etc. When a specific data index supports aggregation in which dimensions, if the current index can support aggregation in more extension dimensions in addition to the owned dimension, the owned dimension and the extension dimension can be configured as supported dimensions. This may enable aggregation from a low dimension to a higher dimension, e.g., aggregating metrics from a commodity dimension to a brand, category, etc. dimension, etc. Of course, for the data demander, it is not necessary to distinguish whether the specific dimension is a self dimension or an extended dimension, and the selection is performed according to the requirement of the data demander.

In addition, in the above-described manner, in addition to the definition of the atomic index, the definition of the derivative index may be performed. In practical applications, it is usually more practical to derive a specific derived index, for example, the atomic index is GMV, but actually, when the data is fetched, some time period or other modification condition needs to be provided for the atomic index to be able to aggregate specific data, for example, GMV of the last 7 days, and so on. Such indicators with time periods or other modification conditions are statistically significant. Therefore, specifically, the atomic index may be defined first, and then more derived indexes may be defined on the basis of the atomic index. For example, as shown in fig. 4-1, the interface is an interface for defining a derivative indicator, and after a user performing an indicator definition operation selects an indicator type as "derivative indicator", the user may provide options for inputting information such as an atomic indicator and a modification condition in the interface, and the user may configure a specific derivative indicator through the options. In addition, the physical table implementation manner of the derived index can be the same as that of the corresponding atomic index, and therefore, repeated configuration is not required.

In the above access mode based on the index normalized definition, the duplicate removal judgment can be performed on the index name, and the unique physical table implementation mode can be bound for the specific index name in the configuration information, so that the uniqueness of the physical table implementation mode of the same index on the same dimension can be ensured. This approach is suitable for data fields, such as transactions, logs, etc., for which the application process is already mature and versatile. Although the configuration flow is relatively complex, after the access is completed, a plurality of derivative indexes can be configured according to different combinations of the atomic indexes and the modifiers, and data aggregation of various complex dimensions can be realized.

Of course, in practical applications, there may be some situations that some data indexes may have relatively strong specialization and are not common in multiple data fields, for example, in a supply chain system, a certain index is used to count how many goods are delivered from a warehouse, other data fields may not be concerned about the index, and a certain physical table may have been summarized based on the index, that is, there is a field corresponding to the index in a direct field. At this time, in order to meet the requirement of performing fast access on specific indexes, an implementation scheme for performing data index access based on a physical table may also be provided. At this time, a second configuration interface may be provided, where the second configuration interface includes an operation option for selecting a physical table, an operation option for designating a target field in the selected physical table as a currently defined data index, and an operation option for performing dimension configuration for the data index.

That is, in this method, it is only necessary to directly designate and register a certain field in a certain physical table as a currently defined data index without configuring information such as a data field, an application process, and an aggregation method. Wherein the data index accessed in this way can be generally determined as a derivative index by default. Of course, after the index access is performed in this way, dimension expansion and time period expansion of the index can also be supported, for example, a user accesses a "GMV of near 1 day of a commodity dimension" derived index through a certain physical table, and based on the derived index, aggregation from a low dimension to a higher dimension can be realized, for example, the index is aggregated to dimensions such as a brand, a category, and the like from the commodity dimension. In addition, the expansion of the time period can be realized, for example, the aforementioned "GMV of near 1 day", and other derived indexes can be configured, for example, "GMV of near 7 days", and the like.

Thus, when a second configuration interface for metric access based on a physical table is specifically provided, options for configuring which higher dimensions the metrics support to aggregate in may also be provided in the interface. For example, as shown in fig. 5, an option for inputting information such as a table name, a field, an aggregation manner, an accumulation manner, and the like of a specific physical table may be provided in the interface, and in addition, an "available dimension" option may be provided for configuring which dimensions the specifically defined index supports aggregation.

Since the specific index is configured based on a specific field in the physical table, the information naturally includes the information of the physical table implementation manner of the index, and an option for configuring the physical implementation manner does not need to be additionally provided in the interface.

The data index access mode based on the physical table can realize quick access to the data index, and of course, the mode is generally applicable to a data field which is not clear in the application process and only has a data application layer (for example, a coarse-grained fact table after further aggregation and summary statistics based on the summary layer).

Of course, for the way of accessing the data index based on the physical table, some control can be performed on the unification of the index definition apertures. That is, in the process of receiving the configuration information of the data index, uniqueness judgment may be performed on the physical table implementation manner of the same data index in the same dimension, and if the uniqueness condition is not satisfied, an error notification may be performed. For example, when configuration information for a currently defined data index is received, it may be determined whether there is a situation in the defined data index that has the same identifier as the currently defined data index, supports the same dimension, and corresponds to a different physical table, and if there is a situation, an error notification is performed. For example, if a user accesses a certain index "GMV of a commodity" based on a certain physical table before, and then another user needs to register an index called "GMV of a commodity" based on another physical table, an error may be reported, so that only one physical implementation of the same index in the same dimension is realized.

In specific implementation, through the access port with the derived index based on the index normalized definition or based on the physical table and other manners, a specific index management system can also generate more extended indexes based on different derived indexes. For example, there are two derived indicators: the number of the successful users in the near 1 day and the exposure UV of the users in the near 1 day can be calculated, and more expansion indexes can be obtained by calculating the addition, subtraction, multiplication and division results of the indexes. For example, the conversion rate of the closing event in the near 1 day, which is the number of closing events in the near 1 day/the exposure UV of the near 1 day user, can be calculated by the above two derived indexes.

Whether index access is carried out in a standardized index definition mode or index quick access is carried out based on a physical table, specific indexes can be described in a metadata mode and stored in an index management system, and data services can be exposed to the outside based on the specific indexes subsequently.

In addition, after the indexes are accessed, the indexes can be managed through the index management page, and a data user can check the names, the caliber definition description and the calculation logic configuration of the indexes. For example, as shown in fig. 6, after selecting the "near 1 day helper sharing bring back UV" index, basic information of the index, including name, type, data field, definition description, and the like, may be exhibited, and development information, including associated atomic index, time period, time type, modification condition, and the like, may also be exhibited. In addition, if the definer needs to adjust the defined aperture of the index, the configuration can be modified through the page.

Besides index management, management of dimensions and dimension attributes can be supported, so that after indexes are defined, query statement generation can be performed on different dimensions or even different dimension combinations to complete data query.

S203: and providing data service for the data demand party according to the plurality of selectable data indexes, the dimension supported by each data index and the corresponding physical table implementation mode.

After the definition of the multiple dimensions and the multiple data indexes is completed, data services can be provided for the data demander according to the multiple optional data indexes, the dimensions supported by the data indexes and the corresponding physical table implementation modes, that is, specific services can be disclosed for the data demander.

Specifically, when providing data services, in one manner, the identifiers of the multiple selectable data indexes and the identifier of the dimension supported by each data index may be provided to a data demander (a specific data demander, etc.), so that the data demander may select the identifiers. After the target data index and the target dimension required by the data demander are determined, as the specific target data index and the specific target dimension are respectively corresponding to the physical table implementation manners, a target query statement (for example, an SQL statement) can be automatically generated according to the physical table implementation manners respectively corresponding to the target data index and the target dimension and a preset query statement generation template, so that a data aggregation calculation result of the target data index on the target dimension can be provided for the data demander through the target query statement.

The data demander can select the required indexes and dimensions based on the interface, and then can submit a specific reading request. For example, as shown in fig. 7, selectable indicators may be displayed at a position shown in 71, and during the displaying, the indicators may be displayed in a classified manner, for example, multiple classes such as "pull-up, Tuo-kuan" may be included, and a specific identifier such as a data indicator name may be displayed under each class, for example, indicators such as "pull-up-day-long pull-up", "near-1 day-long pull-up" may be included in the "pull-up" class. After selecting one of the metrics, it can be demonstrated at the location shown at 72 in fig. 7 which dimensions the metric supports aggregation. For example, a war zone, subsidiaries, parcel zone, etc. may be included. Of course, in addition to displaying the selectable indexes and dimensions in the interface, the identifiers of the selectable data indexes and the identifiers of the dimensions supported by the data indexes can be provided to the data demander in other ways, so long as the data demander can know which indexes and dimensions are selectable, and can submit the selection result to the index management system.

When the data aggregation calculation result of the target data index required by the data demand side on the target dimension is provided for the data demand side, various modes can be provided. For example, in one mode, the target query statement may be directly used to query data in a physical table corresponding to the target data index, and after performing aggregation calculation on the queried data in the target dimension, a data aggregation calculation result may be returned through a target interface. That is to say, in this way, after the data demander completes the selection of the target index and the target dimension through a certain target interface, the SQL statement generation is automatically generated according to the target index and the physical table implementation mode corresponding to the dimension, aggregation calculation is performed according to the corresponding aggregation mode after querying the corresponding physical table to obtain data, and the obtained aggregation calculation result is returned to the data demander through the target interface for display, so as to meet the demand of the user for instant reading.

In another mode, the data demander can also submit the required data indexes and dimensions in advance through multiple modes, so that the SQL statements can be automatically generated, and a service interface and a corresponding data scheduling task can be generated, so that the service interface can be provided for the data demander, and the data demander can call the service interface in the application of the data demander. Scheduling tasks may be performed periodically according to the time type of the specific metric, e.g., if the time type of the current target data metric is a real-time metric, it may be performed once in five minutes, the offline metric is performed once a day, etc. The statements scheduling the execution of the tasks may be the automatically generated SQL statements described above.

Certainly, under the condition that the data demander has the capability of writing SQL, information such as specific data index identifiers, supported dimension identifiers, and the like, and information of respective physical table implementation manners may also be provided to the data demander, so that the data demander may write a target query statement by itself according to the physical table implementation manner, for example, write SQL, and the like, after selecting a target data index and a target dimension. And then, the data demand party can initiate a data query request by utilizing the written SQL, and the index management system can return a data aggregation calculation result of the target data index on the target dimension by executing a target query statement carried in the query request.

In summary, according to the embodiment of the present application, a plurality of selectable dimensions may be determined according to the received dimension configuration information, where a specific dimension configuration information may further include a physical table implementation manner corresponding to the dimension. In addition, a plurality of optional data indexes can be determined according to the received data index configuration information, wherein the data index configuration information may include a physical table implementation manner corresponding to the data index and a dimensionality supported by the data index. Therefore, the data index configuration information specifies the specific support for aggregation in which dimensions, the specific corresponding physical table implementation mode, and the specific dimension configuration information configures the physical table implementation mode of the specific dimension, so that the specific index management system can not only display the registered index, but also provide the capability of exporting services to the outside. Moreover, the same data index can be realized only in one physical way in the same dimension, so that data service can be better provided for a data demand party, and the situation that the same index returns various different aggregation results in the same dimension is avoided.

It should be noted that, in the embodiments of the present application, the user data may be used, and in practical applications, the user-specific personal data may be used in the scheme described herein within the scope permitted by the applicable law, under the condition of meeting the requirements of the applicable law and regulations in the country (for example, the user explicitly agrees, the user is informed, etc.).

Corresponding to the foregoing method embodiment, an embodiment of the present application further provides an apparatus for providing a data service, and referring to fig. 8, the apparatus may include:

a dimension configuration unit 801, configured to determine multiple selectable dimensions according to received dimension configuration information, where the dimension configuration information includes an identifier of the created dimension and a physical table implementation manner corresponding to the dimension;

an index configuration unit 802, configured to determine a plurality of selectable data indexes according to received data index configuration information, where the data index configuration information includes an identifier of a created data index, a physical table implementation manner corresponding to the data index, and a dimension supported by the data index; the physical table implementation mode of the same data index on the same dimension has uniqueness;

a service providing unit 803, configured to provide a data service for the data demander according to the multiple optional data indexes, the dimensions supported by each data index, and the corresponding physical table implementation manner.

The index configuration unit may specifically be configured to:

Or, the index configuration unit may specifically be configured to:

In a specific implementation, the index configuration unit may further be configured to:

Specifically, the service providing unit may be specifically configured to:

the index providing subunit is used for providing the identifiers of the multiple selectable data indexes and the identifiers of the dimensions supported by the data indexes to the data demander;

and the aggregation calculation subunit is configured to determine a target data index and a target dimension required by the data demander, generate a target query statement according to a physical table implementation manner corresponding to the target data index and the target dimension, and a preset query statement generation template, so as to provide a data aggregation calculation result of the target data index on the target dimension for the data demander through the target query statement.

Specifically, the index providing subunit may specifically be configured to:

the aggregation calculation subunit may be specifically configured to:

Alternatively, the aggregation calculation subunit may be specifically configured to:

Alternatively, the service providing unit may be further configured to:

In addition, the present application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the method described in any of the preceding method embodiments.

And an electronic device comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of the preceding method embodiments.

Fig. 9 illustrates an architecture of an electronic device, which may specifically include a processor 910, a video display adapter 911, a disk drive 912, an input/output interface 913, a network interface 914, and a memory 920. The processor 910, the video display adapter 911, the disk drive 912, the input/output interface 913, and the network interface 914 may be communicatively connected to the memory 920 via a communication bus 930.

The processor 910 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solution provided in the present Application.

The Memory 920 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 920 may store an operating system 921 for controlling the operation of the electronic device 900, a Basic Input Output System (BIOS) for controlling low-level operations of the electronic device 900. In addition, web browser 923, data storage management system 924, data service processing system 925, and the like may also be stored. The data service processing system 925 may be an application program that implements the operations of the foregoing steps in this embodiment of the present application. In summary, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 920 and invoked by the processor 910 for execution.

The input/output interface 913 is used to connect the input/output module to realize information input and output. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.

The network interface 914 is used for connecting a communication module (not shown in the figure) to implement communication interaction between the present device and other devices. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).

The bus 930 includes a path to transfer information between the various components of the device, such as the processor 910, the video display adapter 911, the disk drive 912, the input/output interface 913, the network interface 914, and the memory 920.

It should be noted that although the above-mentioned devices only show the processor 910, the video display adapter 911, the disk drive 912, the input/output interface 913, the network interface 914, the memory 920, the bus 930 and so on, in a specific implementation, the device may also include other components necessary for normal operation. Furthermore, it will be understood by those skilled in the art that the apparatus described above may also include only the components necessary to implement the solution of the present application, and not necessarily all of the components shown in the figures.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The method, the apparatus and the electronic device for providing data services provided by the present application are introduced in detail, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific embodiments and the application range may be changed. In view of the above, the description should not be taken as limiting the application.

Claims

1. A method of providing data services, comprising:

2. The method of claim 1,

the determining a plurality of selectable data metrics according to the received data metric configuration information includes:

3. The method of claim 1,

4. The method of claim 1,

the providing data service for the data demand party according to the multiple selectable data indexes, the dimension supported by each data index and the corresponding physical table implementation mode comprises:

5. The method of claim 4,

the providing the identifiers of the multiple selectable data indexes to the data demander, the identifier of the dimension supported by each data index, includes:

6. The method of claim 4,

7. The method of claim 1,

8. An apparatus for providing data services, comprising:

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.

10. An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of claims 1 to 7.