CN113918532A - Portrait label aggregation method, electronic device and storage medium - Google Patents

Portrait label aggregation method, electronic device and storage medium Download PDF

Info

Publication number
CN113918532A
CN113918532A CN202111038502.XA CN202111038502A CN113918532A CN 113918532 A CN113918532 A CN 113918532A CN 202111038502 A CN202111038502 A CN 202111038502A CN 113918532 A CN113918532 A CN 113918532A
Authority
CN
China
Prior art keywords
portrait
tag
label
aggregation method
portrait label
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111038502.XA
Other languages
Chinese (zh)
Inventor
房英明
张晓栋
方磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Hujin Xinrong Technology Co ltd
Original Assignee
Beijing Hujin Xinrong Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Hujin Xinrong Technology Co ltd filed Critical Beijing Hujin Xinrong Technology Co ltd
Priority to CN202111038502.XA priority Critical patent/CN113918532A/en
Publication of CN113918532A publication Critical patent/CN113918532A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • G06F16/244Grouping and aggregation

Abstract

The invention discloses an image label aggregation method, electronic equipment and a storage medium, wherein the image label aggregation method comprises the following steps: determining a label table corresponding to the portrait label according to the metadata information of the portrait label; parsing the tag table into a key and value form; performing first sequencing on the pictorial labels according to the keys, and performing first aggregation on the pictorial labels of the same keys; and performing second sorting on the portrait tags according to the values, and performing second aggregation on the portrait tags with the same values. The invention reads the portrait label through the metadata information and performs distributed processing on the portrait label, thereby avoiding a large amount of low-efficiency code development work caused by the task dependence of configuration when executing the task of portrait label aggregation, optimizing the resource use, greatly reducing the time and improving the efficiency.

Description

Portrait label aggregation method, electronic device and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to an image tag aggregation method, an electronic device, and a storage medium.
Background
The portrait label of user is based on user attribute, consumption information carries out the various kinds of characteristics that the analysis abstracted, unify various scenes such as user portrait label can support marketing, recommendation, analysis, present various kinds of portrait labels distribute in different data tables, portrait label split is favorable to the management and control that becomes more meticulous of authority, but because user's portrait label data bulk is big, the output time is different, lead to once only gathering portrait label, thereby lead to gathering portrait label the time spent long, inefficiency, and when having new label data to enter, need redevelop code again, it is not flexible enough.
Disclosure of Invention
The invention provides an image tag aggregation method, an electronic device and a storage medium, aiming at overcoming the defects of long time spent on aggregating image tags, low efficiency and insufficient flexibility in the prior art.
The invention solves the technical problems through the following technical scheme:
according to a first aspect of the present invention, there is provided a portrait label aggregation method, comprising:
determining a label table corresponding to the portrait label according to the metadata information of the portrait label;
parsing the tag table into a key and value form;
performing first sequencing on the pictorial labels according to the keys, and performing first aggregation on the pictorial labels of the same keys;
and performing second sorting on the portrait tags according to the values, and performing second aggregation on the portrait tags with the same values.
Preferably, before the step of determining the tag table corresponding to the portrait tag according to the metadata information of the portrait tag, the portrait tag aggregation method further includes:
creating a metadata table, wherein the metadata table is used for storing the metadata information, and the metadata information comprises a path of a label table where the portrait label is located;
and acquiring metadata information of the portrait label from the metadata table.
Preferably, the portrait label aggregation method further includes:
creating the tag table for storing the portrait tag, the portrait tag including a user ID (identification number) and a tag value;
in the step of resolving the tag table into a form of a key and a value, the key corresponding to the user ID and the value corresponding to the tag value.
Preferably, the portrait label aggregation method further includes:
and preprocessing the portrait label.
Preferably, the pre-processing comprises washing dirty data in the tag table.
Preferably, the portrait label aggregation method further includes:
and partitioning the label table according to the date, and loading the portrait label of the date to the corresponding partition.
Preferably, the partition is a first partition, and before the step of parsing the tag table into a form of a key and a value, the portrait tag aggregation method further includes:
and splitting or combining the first fragments to obtain second fragments.
Preferably, after the step of performing the second aggregation on the portrait tags with the same value, the portrait tag aggregation method further includes:
and newly building a first image width table, and outputting the second aggregated image label to the first image width table.
Preferably, the first image width table is used for processing the image labels in a columnar storage manner.
Preferably, the portrait label aggregation method further includes:
and establishing a second image width table, and associating the first image width table to the second image width table.
According to a second aspect of the present invention, there is provided an electronic device comprising a memory and a processor connected to the memory, the processor implementing the image tag aggregation method as described above when executing a computer program stored on the memory.
According to a third aspect of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a portrait label aggregation method as described above.
The positive progress effects of the invention are as follows:
the method directly reads the data in the tag table where the portrait tags are located through the metadata information of the portrait tags, converts the portrait tag data of different types and different data sources into the same standard according to the form of key values, then performs distributed sorting and aggregation on the portrait tag data, and finally synthesizes a wide table, thereby avoiding a large amount of inefficient code development work caused by task dependence configuration when executing the task of portrait tag aggregation, optimizing resource utilization, greatly reducing time and improving efficiency.
Drawings
FIG. 1 is a flowchart of an image tag aggregation method according to embodiment 1 of the present invention.
Fig. 2 is a schematic structural diagram of an electronic device according to embodiment 2 of the present invention.
Detailed Description
The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
Example 1
The present embodiment provides a method for aggregating image tags, as shown in fig. 1, the method for aggregating image tags includes:
step 101, a tag table corresponding to the portrait tag is determined according to metadata information of the portrait tag.
The metadata information of the portrait tags is stored in a metadata table and includes hdfs (distributed file system) paths for representing the portrait tags.
As an alternative embodiment, the metadata information may also include a database characterizing the representation tag, a table of representation tags, the ID of the representation tag, and the data type of the representation tag.
A metadata table defining portrait tags, as an alternative embodiment, the main fields of the form are designed as follows:
field(s) Description of the invention
Db Database of portrait labels
Table Figure label place table
LabelId Portrait label unique ID
LabelType Portrait tag data type
Path hdfs path
Creating a tag table from hive (a data warehouse tool), as an alternative embodiment, the main fields of the form defining the tag table are designed as follows:
Figure BDA0003248278220000041
and (3) respectively establishing corresponding offline label tables by different producers of portrait labels, and importing the portrait labels into the label tables, wherein the portrait labels comprise a label main key, namely a user ID, a label number and a label value. The hdfs path is the path of the table we create in hive.
Thus, stored portrait tags are stored under different hdfs data paths, and metadata information records these hdfs data paths. When reading the image tag, the corresponding hdfs data path is read first, and then the image tag in the hdfs data path is called.
In an alternative embodiment, the producer is configured to schedule tasks daily, dynamically partition the tag table according to dates, and automatically partition the imported image tags according to different dates, so that a latest image tag table is generated every day.
As an optional implementation mode, ORC (columnar storage) is used in Hive as a file storage format of a tag table, and each portrait tag corresponds to one column, so that hdfs storage resources are saved, and the structure example is as follows:
struct<uid:string,labelidx1:int,labelidx2:float,labelidx3:decimal(10,2)>
as for portrait tag data directly imported into hdfs, there are often some error data, missing data, or duplicated data, which are collectively referred to as dirty data, so that data needs to be preprocessed, as an optional implementation manner, through metadata information of the portrait tag, including information describing the portrait tag, such as a unique ID of the portrait tag and a data type of the portrait tag, and extracting corresponding portrait tag data from an offline tag table for viewing, there is a preliminary identification on the portrait tag data itself, and preparation is made for subsequent data cleaning.
Before cleaning the missing data of the image tag data, the missing value proportion of each field needs to be calculated, and a cleaning strategy is respectively formulated according to the importance of each field and the missing proportion of each field. For example, if the missing proportion of a field with low importance is high, the field can be directly deleted; and if the missing proportion is low, no treatment or simple filling can be carried out.
And fields of higher importance, such as the user ID, tag number, and tag value in the portrait tag data, need to be complemented. If the field is a field with a low missing proportion, the missing value can be completed by calculation, usually by using methods such as mean, mode, median and the like, and the missing field can also be used as a target variable to predict according to other existing fields, so that the most possible completing data can be obtained, for example, the age can be obtained through the identity card number; if the field with higher missing proportion is the field with higher missing proportion, the data can be communicated with the producer, and the data can be fetched again and supplemented by other channels.
As an optional implementation manner, when data cleaning is performed on the image tag, a small part of data is extracted first for data cleaning, and after confirming that there is no problem in the cleaned data, the whole image tag is processed.
Before cleaning error data and repeated data of image tag data, the data needs to be processed into a consistent display format, which mainly includes data with inconsistent display formats such as time, date and numerical value. For specific fields under different portrait labels, problems can be found in a mode of automatically checking data, such as Chinese characters appearing in a mobile phone number, and then unnecessary characters in the portrait label data are removed. If more problems are found, it is necessary to analyze whether the row and column of the partial portrait label data are not aligned during import, and then further check and analyze, for example, check abnormal data by using a boxed chart. For completely unreasonable data, the data can be directly deleted or processed according to missing data, for example, the data of the field can be verified according to the data of other fields and then modified through logical calculation and speculation. For data records representing portrait labels, duplicate data records may be eliminated if there are multiple identical label values under the label primary key.
As an optional implementation manner, after performing operations such as removing an exception, correcting an error, and complementing an absence on data in the offline tag table, data integration, data transformation, data reduction, and the like are further included. In actual operation, no processing may be performed for the case where it is unclear whether the data field is reserved.
After preprocessing the portrait label data in hdfs, the integrity, accuracy and consistency of the data are further examined, and finally portrait label data with higher quality are obtained.
Step 102, the tag table is resolved into a key and value form.
Before step 102, as an optional implementation manner, an input value of an input file is set, a file under the hdfs directory of the latest partition of each portrait label is extracted, whether the file under the directory is smaller than the set input value is judged, the file size under the directory is sequentially compared with the set input value, and a virtual storage file after segmentation is obtained.
As an optional implementation manner, if the file in the directory is less than or equal to the set input value, no segmentation is performed, and the file is logically divided into a virtual storage file; if the file under the directory is larger than the set input value and is larger than twice of the input value, cutting out a virtual storage file according to the size of the input value; and if the files in the directory are larger than the set input value and smaller than twice of the input value, the files are averagely divided into two virtual storage files. For example, the input value is set to be 5M (a computer storage metering unit), the file sizes under the directory are respectively 1M, 2M, 4M, 12M, and 9M, and the virtual storage files obtained after splitting are 1M, 2M, and 4M, (5M, 3.5M), (4.5M, and 4.5M).
In this embodiment, the virtual storage file is not a true split, but rather belongs to a logical file.
And then judging whether the virtual storage file is equal to the set input value or not, if so, independently forming a fragment, then combining the virtual storage file smaller than the input value with the virtual storage files behind the virtual storage file, and if the last single virtual storage file is smaller than the input value, combining the virtual storage file with the independently formed fragment to ensure that the size of the finally formed fragment is larger than or equal to the set input value and is smaller than twice of the input value. For example, seven files of 1M, 2M, 4M, 5M, 3.5M, 4.5M, and 4.5M are merged to finally form four fragments of (1+2+4) M, 5M, (3.5+3.5) M, and (4.5+4.5) M, i.e., 7M, 5M, 7M, and 9M. It should be noted that the above is only an example, and the size of the input value specifically set is subject to the actual situation.
As an optional embodiment, a MapReduce (a computing framework) framework is called, the number of map (a mapping) tasks is set according to the number of fragments, then the map tasks are started, hdfs files in corresponding fragments are respectively read, each line of portrait tags in the files are parsed into a form of < key, value > (a data format), each key value pair calls a map function once, finally, the key (key) output by the map tasks is uid (user ID), and the value is labelvalue. The map tasks can be calculated in parallel, almost no dependency relationship exists between the map tasks, the time for processing data is greatly reduced, and the efficiency is improved.
As an alternative embodiment, a Spark (a computing engine) framework may also be called, and a map operation may also be performed, where each fragment runs a map task, and parses the data into a form of < key, value >, and an example of a final output data structure is < user ID, tag value >, where the user ID corresponds to a key of the map and the tag value corresponds to a value of the map.
Step 103, image labels are subjected to first sequencing according to the keys, and image labels of the same keys are subjected to first aggregation.
For the data of < user ID, tag value > after different fragment parsing, sorting according to the user ID, either ascending order or descending order, thereby ensuring that the data of the same user ID are arranged together.
As an optional implementation manner, a sort operation method of the MapReduce framework may be called, and a filter sort operation method of the Spark framework may also be called.
After sorting, the tag values of the same user ID are placed in a set, such as < zhang san, man >, < zhang san, 18>, and combined to get < zhang san, { man, 18} >.
As an alternative embodiment, a combine operation method of the MapReduce framework may be called, and a group operation method of the Spark framework may also be called.
And 104, performing second sorting on the portrait tags according to the values, and performing second aggregation on the portrait tags with the same values.
As an optional implementation manner, a reduce operator of a MapReduce framework is called, data is read from a map end, and then metadata information of a portrait tag is used to modify the read data in a value set into data in a < key, value > format, where the key stores a tag type and the value stores a tag value, and in a process of traversing an iterator of the reduce operator, the tag value is converted into a type suitable for hadoop (a distributed system open source framework) serialization based on the tag type. As another optional embodiment, a spare frame may also be called to perform a reduce operation, so as to obtain a new key-value pair < tag type, tag value >, and based on the tag type, the tag value is converted into a type suitable for hadoop serialization.
Sorting according to the tag values, merging the data with the same tag value, namely removing the repeated tag value, and finally storing the merged tag value into the original < user ID, tag value >.
As an optional implementation mode, the number of reduce operators is set based on the number of files, and the writing efficiency of portrait label data is improved.
Creating an image width table, wherein each image label corresponds to one column, writing < user ID, label value > output by reduce operator into a multi-column image width table, putting data at a corresponding position according to an ORC storage format, thereby completing the process of combining labels and outputting the labels to the image width table, and finally generating a combined hdfs file. If the images need to be merged into a plurality of wide tables, a multiple output (output multiple file format) method can be adopted to set the input path and the output path, so that the portrait labels are output into a plurality of image wide tables, and the merged hdfs files are respectively generated.
Creating a full-size image width table, and associating the finally merged hdfs file to the full-size image width table, wherein as an optional implementation mode, the merged image label data can be stored in the full-size image width table in a full-size overlay load (loading) mode.
As an alternative embodiment, the merged hdfs file directory may be associated with a full portrait width table and mounted using the msck repair (repair table constant) command.
The embodiment directly reads the data in the tag table where the portrait tags are located through the metadata information of the portrait tags, converts the portrait tag data of different types and different data sources into the same standard according to the form of < user ID, tag value >, sorts and merges the portrait tag data according to the user ID, sorts and merges the portrait tag data according to the tag value, outputs the merged data to the corresponding position of the portrait width table ORC, finally associates the portrait width tables to synthesize a full portrait width table, avoids a large amount of inefficient code development work caused by task dependence when executing the task of portrait tag aggregation, optimizes resource use, greatly reduces time and improves efficiency, and the invention can flexibly support more portrait access and flexibly configures programs without changing core logic, has stronger expansibility.
Example 2
This embodiment provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the image tag aggregation method of embodiment 1 is implemented.
The electronic device 30 shown in fig. 2 is only an example and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
The electronic device 30 may be embodied in the form of a general purpose computing device, which may be, for example, a server device. The components of the electronic device 30 may include, but are not limited to: the at least one processor 31, the at least one memory 32, and a bus 33 connecting the various system components (including the memory 32 and the processor 31).
The bus 33 includes a data bus, an address bus, and a control bus.
The memory 32 may include volatile memory, such as Random Access Memory (RAM)321 and cache memory 322, and may further include Read Only Memory (ROM) 323.
Memory 32 may also include a program tool 325 having a set (at least one) of program modules 324, such program modules 324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The processor 31 executes various functional applications and data processing, such as the portrait label aggregation method of embodiment 1 of the present invention, by running the computer program stored in the memory 32.
The electronic device 30 may also communicate with one or more external devices 34. Such communication may be through input/output (I/O) interfaces 35. Also, the model-generating device 30 may also communicate with one or more networks through a network adapter 36. As shown in FIG. 2, network adapter 36 communicates with the other modules of model-generating device 30 via bus 33. It should be understood that although not labeled in FIG. 2, other hardware and/or software modules may be used in conjunction with the model-generating device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
Example 3
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the portrait label aggregation method of embodiment 1.
More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In an alternative embodiment, the present invention may also be implemented in the form of a program product, which includes program code for causing a terminal device to execute the portrait label aggregation method implementing embodiment 1 when the program product runs on the terminal device.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims (12)

1. An image tag aggregation method is characterized by comprising the following steps:
determining a label table corresponding to the portrait label according to the metadata information of the portrait label;
resolving the data of the tag table into a key and value form;
performing first sequencing on the pictorial labels according to the keys, and performing first aggregation on the pictorial labels of the same keys;
and performing second sorting on the portrait tags according to the values, and performing second aggregation on the portrait tags with the same values.
2. The representation tag aggregation method of claim 1, wherein prior to the step of determining the tag table corresponding to the representation tag based on the metadata information of the representation tag, the representation tag aggregation method further comprises:
creating a metadata table, wherein the metadata table is used for storing the metadata information, and the metadata information comprises a path of a label table where the portrait label is located;
and acquiring metadata information of the portrait label from the metadata table.
3. The portrait label aggregation method of claim 1, further comprising:
creating the tag table for storing the portrait tag, the portrait tag including a user ID and a tag value;
in the step of resolving the tag table into a form of a key and a value, the key corresponding to the user ID and the value corresponding to the tag value.
4. The portrait label aggregation method of claim 1, further comprising:
and preprocessing the portrait label.
5. The portrait label aggregation method of claim 4, wherein the preprocessing includes washing dirty data in the label table.
6. The portrait label aggregation method of claim 1, further comprising:
and partitioning the label table according to the date, and loading the portrait label of the date to the corresponding partition.
7. The representation tag aggregation method of claim 6, wherein the partition is a first partition, and wherein the step of parsing the tag table into key and value forms is preceded by the representation tag aggregation method further comprising:
and splitting or combining the first fragments to obtain second fragments.
8. The portrait label aggregation method of claim 1, wherein after the step of performing a second aggregation of portrait labels of the same value, the portrait label aggregation method further comprises:
and newly building a first image width table, and outputting the second aggregated image label to the first image width table.
9. The portrait label aggregation method of claim 8, wherein the first portrait width table is used to process the portrait label in a columnar storage manner.
10. The portrait label aggregation method of claim 8, further comprising:
and establishing a second image width table, and associating the first image width table to the second image width table.
11. An electronic device comprising a memory and a processor coupled to the memory, the processor implementing the portrait label aggregation method of any of claims 1-10 when executing a computer program stored on the memory.
12. A computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the portrait label aggregation method of any of claims 1-10.
CN202111038502.XA 2021-09-06 2021-09-06 Portrait label aggregation method, electronic device and storage medium Pending CN113918532A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111038502.XA CN113918532A (en) 2021-09-06 2021-09-06 Portrait label aggregation method, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111038502.XA CN113918532A (en) 2021-09-06 2021-09-06 Portrait label aggregation method, electronic device and storage medium

Publications (1)

Publication Number Publication Date
CN113918532A true CN113918532A (en) 2022-01-11

Family

ID=79233822

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111038502.XA Pending CN113918532A (en) 2021-09-06 2021-09-06 Portrait label aggregation method, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN113918532A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114090590A (en) * 2022-01-20 2022-02-25 北京华品博睿网络技术有限公司 Multi-object label data extraction method and system
CN114528452A (en) * 2022-02-18 2022-05-24 浪潮卓数大数据产业发展有限公司 Data tag implementation method and system based on tobacco and wine sales

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114090590A (en) * 2022-01-20 2022-02-25 北京华品博睿网络技术有限公司 Multi-object label data extraction method and system
CN114528452A (en) * 2022-02-18 2022-05-24 浪潮卓数大数据产业发展有限公司 Data tag implementation method and system based on tobacco and wine sales
CN114528452B (en) * 2022-02-18 2023-07-18 浪潮卓数大数据产业发展有限公司 Method and system for realizing data tag based on cigarette and wine sales

Similar Documents

Publication Publication Date Title
CN111339073A (en) Real-time data processing method and device, electronic equipment and readable storage medium
CN110781231B (en) Database-based batch import method, device, equipment and storage medium
CN113918532A (en) Portrait label aggregation method, electronic device and storage medium
CN105787058B (en) A kind of user tag system and the data delivery system based on user tag system
CN111400011B (en) Real-time task scheduling method, system, equipment and readable storage medium
CN110956269A (en) Data model generation method, device, equipment and computer storage medium
CN103595571A (en) Preprocessing method, device and system for website access logs
CN111611276A (en) Data query method, device and storage medium
CN111639077A (en) Data management method and device, electronic equipment and storage medium
US10552419B2 (en) Method and system for performing an operation using map reduce
CN113010542A (en) Service data processing method and device, computer equipment and storage medium
CN111143390A (en) Method and device for updating metadata
CN116303427A (en) Data processing method and device, electronic equipment and storage medium
CN114168624B (en) Data analysis method, computing device and storage medium
CN115757481A (en) Data migration method, device, equipment and storage medium
CN110851437A (en) Storage method, device and equipment
CN111159213A (en) Data query method, device, system and storage medium
CN115168509A (en) Processing method and device of wind control data, storage medium and computer equipment
CN115203339A (en) Multi-data source integration method and device, computer equipment and storage medium
CN114691768A (en) Data processing method, accounting system and related equipment
CN112668285A (en) Method and device for generating fund daily report by combining RPA and AI and electronic equipment
CN112800091A (en) Flow-batch integrated calculation control system and method
CN113868267A (en) Method for injecting time sequence data, method for inquiring time sequence data and database system
CN112559641B (en) Pull chain table processing method and device, readable storage medium and electronic equipment
CN112347095B (en) Data table processing method, device and server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination