CN111291230B - Feature processing method, device, electronic equipment and computer readable storage medium - Google Patents
Feature processing method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111291230B CN111291230B CN202010081786.XA CN202010081786A CN111291230B CN 111291230 B CN111291230 B CN 111291230B CN 202010081786 A CN202010081786 A CN 202010081786A CN 111291230 B CN111291230 B CN 111291230B
- Authority
- CN
- China
- Prior art keywords
- feature
- group
- features
- stored
- feature group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a feature processing method, a feature processing device, electronic equipment and a computer readable storage medium, and belongs to the technical field of computers. According to the method, the features corresponding to the object to be acquired are acquired from the preset feature library through the first service according to the object ID of the object to be acquired, wherein the sparse features and the dense features corresponding to the plurality of objects are stored in the preset feature library, and the storage space can be saved to a certain extent. And then, carrying out first processing on the features corresponding to the object to be acquired to obtain first features, then, sending the first features to a second service, carrying out second processing on the received first features through the second service to obtain second features, and carrying out feature processing on the second features by utilizing a model to be processed. Thus, the I/O amount occupied by transmission and the time length required by the transmission can be reduced, and the transmission efficiency is further improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a feature processing method, a feature processing device, an electronic device, and a computer readable storage medium.
Background
In order to provide a better service to a user, the network platform often extracts features of the object according to relevant information of the object in the network, for example, extracting features from usage data of the user and relevant information of the network object provided in the network platform, so as to use the features of the object as recommendation basis or optimization basis of a model of the network platform.
In the prior art, each model often directly stores the corresponding features of the model in the form of key value pairs according to the feature names and feature values of the features, namely, directly stores the corresponding features in a sparse type. When the feature is required to be processed, the first service often acquires the feature of the corresponding object and then directly transmits the feature to the second service. Because of the large number of models in the network platform, the number of features that need to be stored is correspondingly often large. This results in a larger space and a lower transfer efficiency when transferring features between the first service and the second service.
Disclosure of Invention
The invention provides a feature processing method, a device, electronic equipment and a computer readable storage medium, so as to solve the problems of large occupied space and low feature transfer efficiency.
In a first aspect of the present invention, there is provided a feature processing method, the method comprising:
acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode;
performing first processing on the features corresponding to the object to be acquired through the first service, and sending the first processed features to a second service;
performing second processing on the first feature through the second service to obtain a second feature, and performing feature processing on the second feature by using the model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process.
In a second aspect of the implementation of the present invention, there is also provided a feature processing apparatus, including:
the first acquisition module is used for acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode;
the first processing module is used for carrying out first processing on the characteristics corresponding to the object to be acquired through the first service, obtaining first characteristics after the first processing, and sending the first characteristics to the second service;
the second processing module is used for performing second processing on the first features through the second service to obtain second features, and performing feature processing on the second features by utilizing the model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process.
In yet another aspect of the present invention, there is also provided a computer readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform any of the above-described feature processing methods.
In yet another aspect of the invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the feature processing methods described above.
According to the feature processing method provided by the embodiment of the invention, the feature corresponding to the object to be acquired can be acquired from the preset feature library through the first service according to the object ID of the object to be acquired, wherein the sparse features and the dense features corresponding to a plurality of objects are stored in the preset feature library, and the sparse features and the dense features corresponding to the plurality of objects can be obtained by storing the feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode. In this way, storage space can be saved to some extent by combining the two storage modes. And then, carrying out first processing on the features corresponding to the object to be acquired to obtain first features, then, sending the first features to a second service, carrying out second processing on the received first features through the second service to obtain second features, and finally, carrying out feature processing on the second features by utilizing a model to be processed. Therefore, by performing the first processing before transmission, the data volume required to be transmitted can be reduced, the transmission convenience can be improved, the I/O volume required to be occupied by transmission and the time required by transmission can be further reduced, and the transmission efficiency can be further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flow chart of steps of a feature processing method according to an embodiment of the present invention;
FIG. 2-1 is a flow chart of steps of another feature processing method provided by an embodiment of the present invention;
2-2 are schematic diagrams of one embodiment provided by embodiments of the present invention;
FIG. 3 is a block diagram of a feature processing apparatus provided by an embodiment of the present invention;
fig. 4 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.
Fig. 1 is a flowchart of steps of a feature processing method according to an embodiment of the present invention, where, as shown in fig. 1, the method may include:
step 101, acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode.
In the embodiment of the invention, the first service may be a feature pulling service for acquiring features from a preset database, and the first service may be implemented by a recommendation engine. The second service may be a feature processing service for processing the acquired feature itself, that is, the first service performs the processing of: acquiring the features and transmitting the acquired features to a second service, wherein the second service performs the following processing: the feature itself, which is passed on by the first service, is operated on, and the processing performed by the two is often different. Further, the two services may be deployed on the same electronic device or may be deployed on different electronic devices. For example, to ensure proper operation of the two services, the first service and the second service may be deployed on different servers. The preset feature library may be a database for storing all features. The feature data corresponding to an object may include a plurality of feature pairs, one feature pair often consisting of a feature name and a corresponding feature value. Accordingly, the sparse feature refers to a feature stored in a key-value pair manner by taking a feature name and an object identifier (Identity, ID) as keys according to a sparse feature storage manner, that is, the stored sparse feature may include a feature name, the dense feature refers to a feature stored in a key-value pair manner by taking an object ID as a key according to a dense sparse feature storage manner, and the feature values in a plurality of feature pairs of the object as values. Because the sparse features of the sparse type and the dense features of the dense type of the object are stored in the preset feature library, the feature names do not need to be stored when the dense features are stored, and therefore, the storage space can be saved to a certain extent.
Further, the object to be acquired may be an object requiring acquisition of a feature, which may be a user in a network platform or a network entity in a network platform, e.g. video, audio, etc. When the feature is acquired, the feature corresponding to the ID matched with the object ID can be searched from the preset feature library according to the object ID of the object to be acquired, so that the feature corresponding to the object to be acquired is obtained.
Step 102, performing first processing on the features corresponding to the object to be acquired through the first service, and sending the first features obtained after the first processing to a second service.
In the embodiment of the present invention, the first processing is at least used for reducing the data size of the feature corresponding to the object to be acquired and/or changing the data arrangement mode of the feature corresponding to the object to be acquired. For example, when the first process is used to reduce the data amount of the feature corresponding to the object to be acquired, the first process may be a compression process, a deletion process of the useless data in the feature corresponding to the object to be acquired, or the like. When the first process is used to change the data arrangement of the features corresponding to the object to be acquired, the first process may be a serialization process, or the like. The serialization processing may be to fluidize the content of the feature corresponding to the object to be obtained, the feature after fluidization may be convenient for performing read-write operation, and be transmitted between networks, and the serialization processing may be Kryo serialization or crc serialization. The compression process may be an operation of converting the feature corresponding to the object to be acquired into a more compact form, and the sequence is processed as an Lz4 sequence. Specifically, redundant portions in the feature data corresponding to the object to be acquired may be removed. Therefore, the first service performs the first processing on the features corresponding to the object to be acquired before transmitting the features to the second service, and converts the features corresponding to the object to be acquired into the first features, so that the data size of the first features to be transmitted can be reduced to a certain extent, and the transmission convenience is improved.
In this step, the first service may send the first feature to the second service through a network connection established in advance with the second service. Since the first feature is a feature after the first processing, that is, a feature in which the data amount is small, transmission is facilitated. Therefore, compared with the mode that the first service directly transmits the characteristics corresponding to the object to be acquired to the second service, the method and the device can reduce the I/O amount occupied by transmission and the time required by the transmission to a certain extent by performing the first processing and then transmitting.
And 103, performing second processing on the first features through the second service to obtain second features, and performing feature processing on the second features by using the model to be processed.
In the embodiment of the present invention, the second process may be a reverse operation of the first process. Specifically, when the first process is used for reducing the data amount of the feature corresponding to the object to be acquired, the second process may be used for recovering the data amount of the first feature; the first processing is used for changing the data arrangement mode of the features corresponding to the object to be acquired, and the second processing can be used for recovering the data arrangement mode of the first features; the first processing is used for reducing the data amount of the features corresponding to the object to be acquired and changing the data arrangement mode of the features corresponding to the object to be acquired, and the second processing can be used for recovering the data amount of the first features and recovering the data arrangement mode of the first features. For example, in the case where the first process is a serialization process, the second process may be an inverse serialization process, in the case where the first process is a compression process, the second process may be a decompression process, and in the case where the first process is a serialization process and a compression process, the second process may be an inverse serialization process and a decompression process. In this way, the second service converts the first feature into the second feature by performing the second processing on the first feature, that is, converting the first feature into the original feature, so that the second service is convenient for processing the feature subsequently.
In the embodiment of the present invention, the model to be processed may be a recommendation model or a ranking model, and the feature processing may be model training or model prediction according to the input features, which is not limited in the embodiment of the present invention. Since the second feature is a feature converted by the second processing, the second feature is directly input as the model, so that the model can be ensured to be processed normally.
In summary, in the feature processing method provided by the embodiment of the present invention, the feature corresponding to the object to be acquired may be acquired from the preset feature library through the first service according to the object ID of the object to be acquired, where the preset feature library stores sparse features and dense features corresponding to a plurality of objects, and the sparse features and dense features corresponding to the plurality of objects may be obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage manner and a dense feature storage manner. In this way, storage space can be saved to some extent by combining the two storage modes. And then, carrying out first processing on the features corresponding to the object to be acquired to obtain first features, then, sending the first features to a second service, and carrying out second processing on the received first features through the second service to obtain second features. The first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired, and the second processing is the reverse operation of the first processing. Therefore, by performing the first processing before transmission, the data volume required to be transmitted can be reduced, the transmission convenience can be improved, the I/O volume required to be occupied by transmission and the time required by transmission can be further reduced, and the transmission efficiency can be further improved.
Fig. 2-1 is a flowchart illustrating steps of another feature processing method according to an embodiment of the present invention, where the method may be applied to a first service and a second service, as shown in fig. 2-1, and the method may include:
step 201, obtaining feature data corresponding to a plurality of objects to be stored.
In this step, the object to be stored may be an object that needs to store a corresponding feature, and feature data corresponding to the object to be stored may be feature pairs extracted according to related information of the object to be stored, and the feature data are stored according to a subsequent step, so that a feature corresponding to the object may be obtained. These feature data may be obtained by: and acquiring the related information of the object to be stored from the background data of the network platform, and then extracting the feature names and the corresponding values thereof from the related information to form feature pairs, thereby realizing the acquisition of the feature data. Therefore, the characteristic data is expressed in the form of characteristic pairs, so that the characteristic can be conveniently checked and processed by staff. The feature pairs may be key-value pairs with feature names as keys and feature values corresponding to the feature names as values, and the feature pairs may be stored in the data storage table in the form of Hive. The feature data corresponding to each object to be stored can be stored corresponding to the object ID of the object to be stored, so that the feature data corresponding to the object to be stored can be conveniently and accurately obtained from the data storage table according to the object ID. Accordingly, when the feature data is acquired, the feature data corresponding to the object to be stored can be read from the data storage table according to the object ID of the object to be stored. It should be noted that, in the embodiment of the present invention, the object to be stored is an object whose corresponding feature is not yet stored in the preset feature library, and the object to be obtained is an object whose corresponding feature is required to be obtained from the preset feature library, and the object to be obtained may be an object whose corresponding feature is already stored in the preset feature library, or may be an object whose corresponding feature is not yet stored in the preset feature library, that is, the feature corresponding to the object to be obtained may be already stored in the preset feature library, or may not be stored in the preset feature library. The object in the preset feature library is an object to be stored after the corresponding feature data is stored in the preset database.
Step 202, for any object to be stored, combining feature data corresponding to the object to be stored according to a preset feature combination format to obtain a feature group conforming to the preset feature combination format.
In this step, the preset feature combination format may be set according to actual requirements. By way of example, the preset feature combination format may be: common features-crossing features, wherein the common features can be a plurality of feature pairs connected by preset separators, the crossing features can be a plurality of single crossing features connected by preset separators, the single crossing features can be formed by a common feature and a single crossing feature identifier, and the single crossing feature identifier can be distributed according to actual requirements.
When generating the feature group, a plurality of feature pairs in the feature data can be utilized to generate common features. The number of the selected feature pairs may be selected according to practical requirements, which is not limited in the embodiment of the present invention. Then generating single cross features according to the common features, then generating cross features by utilizing the single cross features, and finally combining the common features and the cross features into a feature group.
For example, feature_name is used to represent feature name, feature_value is used to represent feature value, and the representation feature pair single_feature can be represented as: single_feature=feature_name + '002' +feature_value; wherein the feature name may be a character string type, i.e., feature_name=string_value; the feature value may include a plurality of values, that is, feature_value=value 1, value2, …, value n, + '\002' represents a separator, so that the feature name and the feature value can be conveniently distinguished by inserting the separator between them.
Further, the generated generic features may be expressed as: basic_feature=single_feature + '001' +single_feature+ … '\001' +single_feature;
the single crossover feature can be expressed as: single_symbols_feature=symbols_id+ '\003' +basic_feature; where corss_id represents a single cross feature identification. The cross-over feature can be expressed as: the core_feature=single_core_feature + '004' + single_core_feature + & gt. Wherein + '\001', + '\003' and + '\004' each represent a separator. The feature set may be expressed as: basic_feature|components_feature. Where "|" denotes a separator between a normal feature and a cross feature. In the embodiment of the invention, the number of the feature pairs can be reduced by combining the feature data corresponding to the object to be stored into the feature group conforming to the preset feature combination format, so that the object to be stored is convenient to store. Meanwhile, in generating the feature group, by separating by the separator, it is possible to ensure that the features are not confused while achieving the combination. Of course, other combinations are possible, and the embodiment of the present invention is not limited thereto.
And 203, performing feature group aggregation on all feature groups corresponding to the object to be stored to obtain an aggregated feature group string corresponding to the object to be stored.
Specifically, the present step may be implemented by the following substeps (1) to (2):
substep (1): and allocating a feature group ID for each feature group corresponding to the object to be stored.
In this step, the feature group ID may be an ID capable of uniquely representing the feature group, and when the feature group ID is assigned, a preset candidate ID may be assigned to each feature group in turn. Wherein, each alternative ID in the preset alternative IDs is different, so that the uniqueness of the feature group ID of each feature group can be ensured. Of course, ID allocation may be implemented in other manners, for example, a random generation algorithm may be used to randomly generate a feature group ID for the feature groups, so long as it is ensured that the feature group IDs of each feature group are different, which is not limited by the embodiment of the present invention.
Substep (2): and according to the feature group and the feature group ID of the feature group, aggregating the feature group according to a preset aggregation format to obtain an aggregate feature group string corresponding to the object to be stored.
In this step, the preset aggregation format may be set according to actual requirements. By way of example, the preset feature aggregation format may be: feature group-feature group ID-separator symbol-feature group ID.
When the aggregation is carried out, a plurality of feature groups can be extracted from the feature groups, and then the feature groups are connected according to a preset feature aggregation format, so that the aggregation is realized, and an aggregation feature group string is obtained. The number of the selected feature groups may be selected according to actual requirements, which is not limited in the embodiment of the present invention.
By way of example, the aggregate feature group string may be represented as: feature set+_005 ' +group_id+_006 ' +feature set ' \005' +group_id+ … + ' 006' +feature set+_005 ' +group_id. Where + '\005' and + '\006' denote separators, and group_id denotes feature set IDs. In the embodiment of the invention, the quantity of the feature groups can be reduced by further aggregating the feature groups, so that the scattered degree of the data is reduced, and further, the subsequent processing is facilitated. Further, in order to facilitate searching the aggregation feature group string, in the embodiment of the present invention, an ID may also be allocated to the aggregation feature group string.
And 204, storing the aggregation feature group strings corresponding to the objects to be stored into the preset feature library according to the sparse feature storage mode and the dense feature storage mode.
Specifically, the preset feature library may be a couchbase database or a redis database. The step can be realized by the following substep (3) to substep (5):
Substep (3): and analyzing the aggregation feature group string corresponding to the object to be stored to obtain a feature group ID and a feature group contained in the aggregation feature group string.
In the embodiment of the invention, the characteristic data is combined by the characteristic pairs, and the characteristic group is obtained by combining the characteristic data. Accordingly, a plurality of feature pairs may be included in one feature group, and one feature pair includes a feature name and a feature value, that is, the feature group includes a feature name.
In this step, for an object to be stored, which needs to store a feature in a preset feature library, an aggregate feature group string corresponding to the object to be stored may be obtained according to an ID of the object to be stored, then each separator in the aggregate feature group string is detected, then the content between each separator is extracted, and further the parsing operation is completed, so as to obtain a feature group ID and a feature group included in the aggregate feature group string.
Substep (4): generating a feature group version number according to a feature group with the contained feature name matched with a preset specific feature name; the feature group version number includes a group name of the feature group.
In this step, the preset specific feature name may be preset according to actual requirements, the feature pair corresponding to the preset specific feature name may be a feature pair that needs to be stored as dense features according to a dense feature format, a feature group version number may be used to represent the dense feature format, and a feature group version number may be represented by a field "schema". When a specific feature name is preset to be changed, the dense feature format, namely, the feature values of the feature names which are contained in the subsequently stored dense features, are correspondingly changed.
Specifically, when the feature group version number is generated, for a feature group including feature names that match a preset specific feature name, the feature names may be extracted from the feature group, and then the feature names are stored in an array form as a feature name array, which is the feature group name of the feature group. And then combining all feature name groups corresponding to the feature groups with the feature names matched with the preset specific feature names to obtain the feature group version numbers. By generating the feature group version number, the format of the stored dense features can be defined, namely, the sequence of storing the feature values is defined, and the method plays a guiding role for the subsequent storing process. Of course, in practical application, the feature set version number may be generated in combination with more information, for example, the feature set version number may be generated in combination with metadata information of the feature set. In this way, the generated feature group version number can be enabled to contain more information, and therefore the generation effect is improved. Wherein, the metadata information of the feature group can be searched according to the ID of the feature group.
Substep (5): and for the feature groups with group names in the feature group version numbers, storing the IDs of the aggregation feature group strings where the feature groups are located, the IDs of the objects corresponding to the feature groups and the feature values in the feature groups into the preset feature library in the form of key value pairs according to the dense feature storage mode, and storing the IDs of the objects corresponding to the feature groups, the feature names and the feature values in the feature groups into the preset feature library in the form of key value pairs according to the sparse feature storage mode for the feature groups with group names not in the feature group version numbers.
In this step, the group name may represent a feature name in the feature group, and if the group name of the feature group exists in the feature group version number, it is indicated that the feature group is a feature that needs to be stored as a dense feature, so that, according to a dense feature storage manner, an ID of an aggregate feature group string in which the feature group exists, an ID of an object corresponding to the feature group, and a feature value in the feature group may be stored in a preset feature library in a key value pair form, thereby obtaining a dense feature. Specifically, the dense feature storage manner may be: the ID of the aggregation feature group string where the feature group is located and the ID of the object corresponding to the feature group are used as keys, the feature value in the feature group is used as a value (value), and key-value pairs (key-value) formed by the two are stored. The key value pairs can be stored in dense feature groups of a preset feature library, so that dense features and sparse features can be conveniently and quickly regional. Meanwhile, the ID of the aggregation feature group string and the object ID are used as keys, so that the subsequent steps can be conveniently called. Accordingly, if the group name of the feature group does not exist in the feature group version number, it is indicated that the feature group is not a feature that needs to be stored as a dense feature, so that the ID of the object corresponding to the feature group, the feature name and the feature value in the feature group can be stored in a preset feature library in the form of key value pairs according to a sparse feature storage mode. Specifically, the sparse feature storage mode may be: and taking the feature name in the feature group as a key, taking the feature value corresponding to the feature name as a value, storing the key-valu e formed by the feature name and the value corresponding to the ID of the object, namely splitting the feature group into feature pairs formed by the feature name and the feature value, and storing the feature pairs. Specifically, the sparse features can be stored in a sparse feature group of a preset feature library, so that dense features and sparse features of a fast region can be conveniently and rapidly obtained. Further, by traversing all the aggregated feature group strings, the entire storage operation may be completed.
Furthermore, since the feature names corresponding to the feature values contained in the dense features are defined in the feature group version numbers, the feature names can be directly stored according to the dense features, so that the feature names corresponding to the feature values are not required to be stored while the feature values are stored once, the problem of data redundancy in a preset feature library can be reduced to a certain extent, and the storage space of the preset feature library is saved. Further, since feature storage requirements may be constantly changing, the format of dense features may be redefined. In the embodiment of the invention, a feature group version number is generated according to a specific feature when a warehousing operation is executed each time, namely, when an aggregated feature group string is stored in a preset feature library, and the feature is stored according to the feature group version number. Because the newly generated feature group version number can represent the latest format of dense features, the storage efficiency can be improved to a certain extent based on the storage mode of the newly generated feature group version number, and after the format of defining the dense features is changed, the subsequently stored dense features can be kept consistent according to the newly defined format, so that the storage confusion is avoided. And meanwhile, the format change of dense features can be conveniently determined based on the feature group version numbers of different stages.
Compared with the mode that each model stores the respective characteristics, in the embodiment of the invention, the characteristics of all objects to be stored are stored into the preset characteristic library together, so that each model can be pulled from the preset characteristic library when the characteristics need to be used, and the universality of the characteristics and the reusability between the characteristics and the models are improved to a certain extent.
In the embodiment of the present invention, after the obtained aggregated feature group strings, each aggregated feature group string and the object ID of the object to be stored may be stored in a feature library intermediate table, so that when the aggregated feature group strings are stored in a preset feature library, feature storage may be implemented by performing an adding operation based on the feature library intermediate table. Compared with a mode of taking the aggregation feature group strings as operation objects and adding the aggregation feature group strings into the preset feature library one by one, in the embodiment of the invention, the mode of adding the feature library intermediate tables for recording the aggregation feature group strings as the operation objects can be adopted, the connection access times with the preset feature library can be reduced to a certain extent, and the adding efficiency is improved.
Expressed in the form of PB, the structure of features stored in the preset feature library can be expressed as:
In the actual application scenario, because the number of feature pairs included in the feature data is often large, if a single dense feature storage mode is adopted for storage, a large feature name version number is often required to be maintained, and therefore storage difficulty is high. In the embodiment of the invention, the sparse feature storage mode and the dense feature storage mode are adopted at the same time, so that certain storage space is saved, and meanwhile, the storage difficulty is prevented from being too high.
Step 205, obtaining, by a first service, a feature corresponding to an object to be obtained from a preset feature library according to an object identification ID of the object to be obtained; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library.
In this step, the first service may further obtain, according to the object ID and the ID of the aggregate feature group string, a dense feature generated from the feature group in the aggregate feature group string represented by the ID of the aggregate feature group string, where the feature corresponds to the object to be obtained. Thereby realizing the acquisition of the appointed characteristics. Compared with the method for directly acquiring all the features corresponding to the object according to the object ID, in the embodiment of the invention, the features of the part can be acquired in a targeted manner according to the actual requirements by combining the object ID and the ID of the aggregation feature group string. The obtaining operation can be realized through the following statement:
And 206, performing first processing on the features corresponding to the object to be acquired through the first service, and sending the first processed features to a second service.
Specifically, the implementation manner of this step may refer to the foregoing step 102, and the embodiments of the present invention are not described herein in detail.
The transmission may be a transmission of the first feature as a message body (body) of the message. Accordingly, the second service may perform the second process after receiving the body.
And step 207, performing second processing on the first feature through the second service to obtain a second feature, and performing feature processing on the second feature by using the model to be processed.
In this step, for dense features in the second feature, the dense features may be first converted into sparse features according to the feature set version number. Specifically, the feature names corresponding to the feature values in the dense features are searched from the feature group version number, then the feature names and the feature values corresponding to the feature names are combined into a feature pair form, so that conversion is realized, and the conversion operation can be expressed as a sentence: < entity_id, list < single_feature > > is indicated. And then, performing feature processing on the sparse features corresponding to the object to be acquired and the sparse features obtained by conversion by utilizing the feature operators corresponding to the model to be processed. For example, a convolution process may be performed using a feature operator. The feature operators corresponding to the model to be processed can be preset according to the characteristics of the model, so that the flexibility and the efficiency of the model to process the features can be improved to a certain extent by setting the corresponding feature operators for different models. In the embodiment of the invention, the I/O amount occupied by transmission and the time length required by the transmission can be reduced to a certain extent by carrying out the first processing and then transmitting, and when the characteristics are transmitted in the form of a table, the maintenance of a larger Idmap table can be avoided by serialization and compression. Meanwhile, compared with the mode that the first service provides the object ID for the second service and the second service pulls the features, in the embodiment of the invention, the second service does not need to bear the pulling service by the second service pulling the features, and the universality of the online service of the model can be further ensured. By way of example, through experimentation, corresponding features of the video corresponding to 100 video prediction requests, where the video prediction requests may be for models in multiple scenes. Compared with the traditional characteristic storage mode, the method provided by the embodiment of the invention can save 1-1.5 times of storage space. And the data size in the transmission is reduced from 300k to 30k, the network I/O required in the transmission is reduced, and the time consumption is reduced from 10ms to 3ms.
Further, the second service may also combine the feature operator processing results to score the predictions for each feature, and then return the combined prediction scores as the prediction results.
In summary, according to the feature processing method provided by the embodiment of the invention, the aggregated feature group string can be generated according to the feature data corresponding to the plurality of objects to be stored, and the aggregated feature group string corresponding to the objects to be stored is stored in the preset feature library according to the sparse feature storage mode and the dense feature storage mode, so that the sparse features and the dense features corresponding to the plurality of objects are stored in the preset feature library, and the storage space can be saved to a certain extent. Then, the first service can acquire the feature corresponding to the object to be acquired from the preset feature library according to the object ID of the object to be acquired, the feature corresponding to the object to be acquired is subjected to first processing to obtain the first feature, then the first feature is sent to the second service, the second service performs second processing on the received first feature to obtain the second feature, and finally the second feature is subjected to feature processing by using the model to be processed. Therefore, by performing the first processing before transmission, the data volume required to be transmitted can be reduced, the transmission convenience can be improved, the I/O volume required to be occupied by transmission and the time required by transmission can be further reduced, and the transmission efficiency can be further improved.
In the following, an embodiment of the present invention will be described with reference to a specific example, and by way of example, fig. 2-2 are schematic diagrams of a specific example provided in the embodiment of the present invention, and as shown in fig. 2-2, features related to feature groups may be generated, specifically, multiple feature groups may be aggregated to form a feature library, and then encoded and put into storage. That is, the aggregation feature group strings are obtained in the steps 202 to 203, and stored according to the sparse storage method and the dense storage method. In the stage of using the coding features, engine splicing, compression and sorting can be performed on the obtained features, namely, a first operation is performed through a first service, then, in the stage of performing object sorting according to the features, decompression can be performed through a sorting service, namely, a second operation is performed through a second service, and finally, the features are analyzed and predicted by utilizing a feature operator. The feature operators can be feature operators corresponding to a plurality of models, and different models can correspond to different application scenes. In the embodiment of the invention, the feature aggregation of a plurality of scenes can be realized by designing the feature library and the feature group, the on-line feature storage of the basic feature group and the feature model consistency are ensured by using the sparse feature structure and the dense feature structure based on the version number, the engine service pulls up and splices the user and video features, compression, serialization, decompression and deserialization are carried out in the transmission process, and corresponding feature operators are applied to process specific features, so that the network IO can be reduced and the performance can be improved.
Fig. 3 is a block diagram of a feature processing apparatus according to an embodiment of the present invention, and as shown in fig. 3, the apparatus 30 may include:
the first obtaining module 301 is configured to obtain, by using a first service, a feature corresponding to an object to be obtained from a preset feature library according to an object identifier ID of the object to be obtained; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode.
The first processing module 302 is configured to perform a first process on the feature corresponding to the object to be acquired through the first service, and send the first feature obtained after the first process to a second service.
A second processing module 303, configured to perform a second process on the first feature through the second service to obtain a second feature, and perform feature processing on the second feature by using the model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process.
Optionally, the apparatus 30 further includes:
and the second acquisition module is used for acquiring the characteristic data corresponding to the plurality of objects to be stored.
And the combination module is used for combining the feature data corresponding to any object to be stored according to a preset feature combination format to obtain a feature group conforming to the feature combination format.
And the aggregation module is used for carrying out feature group aggregation on all feature groups corresponding to the object to be stored to obtain an aggregation feature group string corresponding to the object to be stored.
And the storage module is used for storing the aggregation feature group strings corresponding to the objects to be stored into the preset feature library according to the sparse feature storage mode and the dense feature storage mode.
Optionally, the aggregation module is specifically configured to:
and allocating a feature group ID for each feature group corresponding to the object to be stored.
And according to the feature group and the feature group ID of the feature group, aggregating the feature group according to a preset aggregation format to obtain an aggregate feature group string corresponding to the object to be stored.
Optionally, the storage module is specifically configured to:
analyzing the aggregation feature group strings corresponding to the objects to be stored to obtain feature group IDs and feature groups contained in the aggregation feature group strings; the feature group comprises a plurality of feature pairs, and the feature pairs comprise feature names and feature values.
Generating a feature group version number according to a feature group with the contained feature name matched with a preset specific feature name; the feature group version number includes a group name of the feature group.
And for the feature groups with group names existing in the feature group version numbers, storing the IDs of the aggregation feature group strings where the feature groups are located, the IDs of the objects corresponding to the feature groups and the feature values in the feature groups into the preset feature library in a key value pair mode according to the dense feature storage mode.
And for the feature group with the group name not in the feature group version number, storing the ID of the object corresponding to the feature group, the feature name and the feature value in the feature group into the preset feature library in the form of key value pairs according to the sparse feature storage mode.
Optionally, the second features include sparse features and dense features corresponding to the object to be acquired; the second processing module 303 is specifically configured to:
and for dense features in the second features, converting the dense features into sparse features according to the feature set version number.
And performing feature processing on the sparse features corresponding to the object to be obtained and the sparse features obtained through conversion by utilizing the feature operators corresponding to the model to be processed.
According to the feature processing device provided by the embodiment of the invention, the feature corresponding to the object to be acquired can be acquired from the preset feature library through the first service according to the object ID of the object to be acquired, wherein the sparse features and the dense features corresponding to a plurality of objects are stored in the preset feature library, and the sparse features and the dense features corresponding to the plurality of objects can be obtained by storing the feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode. In this way, storage space can be saved to some extent by combining the two storage modes. And then, carrying out first processing on the features corresponding to the object to be acquired to obtain first features, then, sending the first features to a second service, carrying out second processing on the received first features through the second service to obtain second features, and finally, carrying out feature processing on the second features by utilizing a model to be processed. Therefore, by performing the first processing before transmission, the data volume required to be transmitted can be reduced, the transmission convenience can be improved, the I/O volume required to be occupied by transmission and the time required by transmission can be further reduced, and the transmission efficiency can be further improved.
For the above-described device embodiments, the description is relatively simple, as it is substantially similar to the method embodiments, with reference to the description of the method embodiments in part.
The embodiment of the invention also provides an electronic device, as shown in fig. 4, which comprises a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 complete communication with each other through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401, when executing the program stored in the memory 403, implements the following steps:
acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode;
performing first processing on the features corresponding to the object to be acquired through the first service, and sending the first processed features to a second service;
performing second processing on the first feature through the second service to obtain a second feature, and performing feature processing on the second feature by using the model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer readable storage medium is provided, in which instructions are stored, which when run on a computer, cause the computer to perform the feature processing method of any of the above embodiments.
In a further embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the feature processing method of any of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It should be noted that relational terms such as first and second, and the like are used solely to distinguish one object or operation from another object or operation without necessarily requiring or implying any actual such relationship or order between such objects or operations. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (7)
1. A method of feature processing, the method comprising:
acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode;
performing first processing on the features corresponding to the object to be acquired through the first service, and sending the first processed features to a second service;
performing second processing on the first feature through the second service to obtain a second feature, and performing feature processing on the second feature by using a model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process;
Before the feature corresponding to the object to be obtained is obtained from the preset feature library through the first service according to the object ID of the object to be obtained, the method further comprises:
acquiring characteristic data corresponding to a plurality of objects to be stored;
for any object to be stored, combining the feature data corresponding to the object to be stored according to a preset feature combination format to obtain a feature group conforming to the feature combination format;
performing feature group aggregation on all feature groups corresponding to the object to be stored to obtain an aggregated feature group string corresponding to the object to be stored;
according to the sparse feature storage mode and the dense feature storage mode, storing the aggregation feature group strings corresponding to the objects to be stored into the preset feature library;
the storing the aggregate feature group strings corresponding to the objects to be stored in the preset feature library according to the sparse feature storage mode and the dense feature storage mode includes:
analyzing the aggregation feature group strings corresponding to the objects to be stored to obtain feature group IDs and feature groups contained in the aggregation feature group strings; the feature group comprises a plurality of feature pairs, wherein the feature pairs comprise feature names and feature values;
Generating a feature group version number according to a feature group with the contained feature name matched with a preset specific feature name; the feature group version number contains a group name of the feature group;
for the feature group with the group name in the feature group version number, storing the ID of the aggregation feature group string where the feature group is located, the ID of the object corresponding to the feature group and the feature value in the feature group into the preset feature library in a key value pair mode according to the dense feature storage mode;
and for the feature group with the group name not in the feature group version number, storing the ID of the object corresponding to the feature group, the feature name and the feature value in the feature group into the preset feature library in the form of key value pairs according to the sparse feature storage mode.
2. The method of claim 1, wherein the performing feature set aggregation on all feature sets corresponding to the object to be stored to obtain an aggregate feature set string corresponding to the object to be stored includes:
distributing a feature group ID for each feature group corresponding to the object to be stored;
and according to the feature group and the feature group ID of the feature group, aggregating the feature group according to a preset aggregation format to obtain an aggregate feature group string corresponding to the object to be stored.
3. The method of claim 1, wherein the second features comprise sparse and dense features corresponding to the object to be acquired; the feature processing of the second feature by using the model to be processed includes:
for dense features in the second features, converting the dense features into sparse features according to the feature set version number;
and performing feature processing on the sparse features corresponding to the object to be obtained and the sparse features obtained through conversion by utilizing the feature operators corresponding to the model to be processed.
4. A feature processing apparatus, the apparatus comprising:
the first acquisition module is used for acquiring the characteristics corresponding to the object to be acquired from a preset characteristic library through a first service according to the object identification ID of the object to be acquired; sparse features and dense features corresponding to a plurality of objects are stored in the preset feature library; the sparse features and the dense features corresponding to the plurality of objects are obtained by storing feature data corresponding to the plurality of objects according to a sparse feature storage mode and a dense feature storage mode;
the first processing module is used for carrying out first processing on the characteristics corresponding to the object to be acquired through the first service, obtaining first characteristics after the first processing, and sending the first characteristics to the second service;
The second processing module is used for performing second processing on the first features through the second service to obtain second features, and performing feature processing on the second features by utilizing a model to be processed; the first processing is at least used for reducing the data quantity of the features corresponding to the object to be acquired and/or changing the data arrangement mode of the features corresponding to the object to be acquired; the second process is a reverse operation of the first process;
the second acquisition module is used for acquiring characteristic data corresponding to a plurality of objects to be stored;
the combination module is used for combining the feature data corresponding to any object to be stored according to a preset feature combination format to obtain a feature group conforming to the feature combination format;
the aggregation module is used for carrying out feature group aggregation on all feature groups corresponding to the object to be stored to obtain an aggregation feature group string corresponding to the object to be stored;
the storage module is used for storing the aggregation feature group strings corresponding to the objects to be stored into the preset feature library according to the sparse feature storage mode and the dense feature storage mode;
the storage module is specifically configured to:
Analyzing the aggregation feature group strings corresponding to the objects to be stored to obtain feature group IDs and feature groups contained in the aggregation feature group strings; the feature group comprises a plurality of feature pairs, wherein the feature pairs comprise feature names and feature values;
generating a feature group version number according to a feature group with the contained feature name matched with a preset specific feature name; the feature group version number contains a group name of the feature group;
for the feature group with the group name in the feature group version number, storing the ID of the aggregation feature group string where the feature group is located, the ID of the object corresponding to the feature group and the feature value in the feature group into the preset feature library in a key value pair mode according to the dense feature storage mode;
and for the feature group with the group name not in the feature group version number, storing the ID of the object corresponding to the feature group, the feature name and the feature value in the feature group into the preset feature library in the form of key value pairs according to the sparse feature storage mode.
5. The apparatus of claim 4, wherein the aggregation module is specifically configured to:
distributing a feature group ID for each feature group corresponding to the object to be stored;
And according to the feature group and the feature group ID of the feature group, aggregating the feature group according to a preset aggregation format to obtain an aggregate feature group string corresponding to the object to be stored.
6. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the method of any of claims 1-3 when executing a program stored on a memory.
7. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081786.XA CN111291230B (en) | 2020-02-06 | 2020-02-06 | Feature processing method, device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081786.XA CN111291230B (en) | 2020-02-06 | 2020-02-06 | Feature processing method, device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111291230A CN111291230A (en) | 2020-06-16 |
| CN111291230B true CN111291230B (en) | 2023-09-15 |
Family
ID=71025543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010081786.XA Active CN111291230B (en) | 2020-02-06 | 2020-02-06 | Feature processing method, device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111291230B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282814A (en) * | 2021-05-19 | 2021-08-20 | 北京三快在线科技有限公司 | Characteristic data processing method, device, server and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8965855B1 (en) * | 2010-08-16 | 2015-02-24 | Symantec Corporation | Systems and methods for hotspot mitigation in object-based file systems |
| CN108510066A (en) * | 2018-04-08 | 2018-09-07 | 清华大学 | A kind of processor applied to convolutional neural networks |
| CN108510063A (en) * | 2018-04-08 | 2018-09-07 | 清华大学 | A kind of accelerated method and accelerator applied to convolutional neural networks |
| JP2019029003A (en) * | 2017-07-31 | 2019-02-21 | 株式会社リコー | Feature analysis method, feature analysis device, electronic equipment, program, and storage medium |
| CN110427222A (en) * | 2019-06-24 | 2019-11-08 | 北京达佳互联信息技术有限公司 | Data load method, device, electronic equipment and storage medium |
| CN110647508A (en) * | 2019-08-30 | 2020-01-03 | 北京达佳互联信息技术有限公司 | Data compression method, data decompression method, device and electronic equipment |
-
2020
- 2020-02-06 CN CN202010081786.XA patent/CN111291230B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8965855B1 (en) * | 2010-08-16 | 2015-02-24 | Symantec Corporation | Systems and methods for hotspot mitigation in object-based file systems |
| JP2019029003A (en) * | 2017-07-31 | 2019-02-21 | 株式会社リコー | Feature analysis method, feature analysis device, electronic equipment, program, and storage medium |
| CN108510066A (en) * | 2018-04-08 | 2018-09-07 | 清华大学 | A kind of processor applied to convolutional neural networks |
| CN108510063A (en) * | 2018-04-08 | 2018-09-07 | 清华大学 | A kind of accelerated method and accelerator applied to convolutional neural networks |
| CN110427222A (en) * | 2019-06-24 | 2019-11-08 | 北京达佳互联信息技术有限公司 | Data load method, device, electronic equipment and storage medium |
| CN110647508A (en) * | 2019-08-30 | 2020-01-03 | 北京达佳互联信息技术有限公司 | Data compression method, data decompression method, device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111291230A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106657213B (en) | File transmission method and device | |
| JP6734946B2 (en) | Method and apparatus for generating information | |
| CN112749326B (en) | Information processing method, information processing device, computer equipment and storage medium | |
| CN101477527B (en) | Multimedia resource retrieval method and apparatus | |
| US9357007B2 (en) | Controlling storing of data | |
| CN110149529B (en) | Media information processing method, server and storage medium | |
| CN111339078A (en) | Data real-time storage method, data query method, device, equipment and medium | |
| CN103440243A (en) | Teaching resource recommendation method and device thereof | |
| CN111611622A (en) | Block chain-based file storage method and electronic equipment | |
| WO2013175611A1 (en) | Data distributed search system, data distributed search method, and administrative computer | |
| CN111291230B (en) | Feature processing method, device, electronic equipment and computer readable storage medium | |
| CN113434506B (en) | Data management and retrieval method, device, computer equipment and readable storage medium | |
| CN110825706B (en) | Data compression method and related equipment | |
| CN114139040A (en) | Data storage and query method, device, equipment and readable storage medium | |
| CN111488464B (en) | Entity attribute processing method, device, equipment and medium | |
| CN117093619A (en) | Rule engine processing method and device, electronic equipment and storage medium | |
| CN109101595B (en) | Information query method, device, equipment and computer readable storage medium | |
| US10083121B2 (en) | Storage system and storage method | |
| KR102375511B1 (en) | Document storage management server for performing storage processing of document files received from a client terminal in conjunction with a plurality of document storage and operating method thereof | |
| US8788483B2 (en) | Method and apparatus for searching in a memory-efficient manner for at least one query data element | |
| US9996621B2 (en) | System and method for retrieving internet pages using page partitions | |
| CN115129899B (en) | Document tag information generation method, apparatus, device, medium, and program product | |
| CN113722537B (en) | Short video ordering and model training method and device, electronic equipment and storage medium | |
| KR100594180B1 (en) | Integrated search method online | |
| CN117874006A (en) | Data modeling method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |