CN115168413A - Data processing method and device - Google Patents

Abstract

The embodiments of this specification provide a data processing method and device, wherein the data processing method is applied to a processing terminal in a data storage system, and the data storage system includes a processing terminal and a storage area; the method includes: receiving a target data access instruction sent by a user , wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; based on the data identifier of the target data, it is judged whether the target data exists in the storage area; , using the data identification of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate the target data, and process the data based on the processing end in the data storage system without the need for An additional maintenance data processing service is built to reduce the waste of data processing resources and further improve the data processing efficiency.

Description

Data processing method and device

technical field

The embodiments of this specification relate to the field of computer technology, and in particular, to a data processing method.

Background technique

With the continuous development of computer technology, more and more computer technology is applied in the field of data processing. Data is an expression of facts, concepts or instructions that can be processed by human or automated means. The data can be in the form of numbers, words, pictures or sounds. The basic purpose of data processing is to extract and derive valuable and meaningful data for some specific people from a large amount of data that may be disorganized and difficult to understand. Data processing runs through the entire social production and society. All areas of life, the development of data processing technology, and the breadth and depth of applications have greatly affected the development of human society.

At present, the original data is usually processed into result data in advance, and the obtained result data is saved, but processing the original data in advance requires a lot of computing resources, which affects the efficiency of data processing. Therefore, an efficient data processing solution is urgently needed.

SUMMARY OF THE INVENTION

In view of this, the embodiments of this specification provide a data processing method. One or more embodiments of this specification simultaneously relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program, so as to solve the technical defects existing in the prior art.

According to a first aspect of the embodiments of this specification, a data processing method is provided, which is applied to a processing terminal in a data storage system, where the data storage system includes a processing terminal and a storage area; the method includes:

Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

Based on the data identifier of the target data, determine whether the target data exists in the storage area;

If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding original data;

Use data processing rules to process raw data to generate target data.

Optionally, using the data identifier of the target data to determine the data processing rules corresponding to the target data and the steps of the corresponding original data, including:

Obtain user access feature information for target data;

According to the access feature information and the data identifier of the target data, the data processing rules corresponding to the target data and the corresponding original data are obtained.

Optionally, according to the access feature information and the data identifier of the target data, the steps of acquiring the data processing rules corresponding to the target data and the corresponding original data include:

Matching the access feature information with a preset processing level to obtain a target processing level corresponding to the access feature information;

According to the target processing level and the data identification of the target data, the data processing rule corresponding to the target data and the corresponding original data are determined.

Optionally, the target data includes target video data, and the data processing rules include data compression rules;

According to the access feature information and the data identifier of the target data, the steps of obtaining the data processing rules corresponding to the target data and the corresponding original data include:

According to the access feature information and the data identifier of the target video data, the data compression rule corresponding to the target video data and the corresponding original data are obtained.

Optionally, the method further includes:

If there is no data processing rule corresponding to the target data, send a message to the user that the acquisition of the data processing rule corresponding to the target data fails;

Receive data processing rules for target data sent by users.

Optionally, the method further includes:

In the case that the original data corresponding to the target data does not exist, send a message to the user that the acquisition of the original data corresponding to the target data fails;

Receive the raw data sent by the user for the target data.

Optionally, after using the data processing rules to process the original data to generate the target data, it also includes:

Compare the access feature information of the target data with the preset storage conditions, and store the target data in the storage area if the access feature information satisfies the preset storage conditions.

According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, which is applied to a processing terminal in a data storage system, and the data storage system includes a processing terminal and a storage area; the apparatus includes:

The receiving module is configured to receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

The judgment module is configured to judge whether the target data exists in the storage area based on the data identification of the target data;

The determining module is configured to, if not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding raw data;

The generating module is configured to process the original data by using the data processing rules to generate target data.

Optionally, the determining module is further configured to obtain user access feature information for the target data; and obtain data processing rules corresponding to the target data and corresponding raw data according to the access feature information and the data identifier of the target data.

Optionally, the determination module is further configured to match the access feature information with a preset processing level to obtain a target processing level corresponding to the access feature information; according to the target processing level and the data identifier of the target data, determine the corresponding target data. Data processing rules and corresponding raw data.

Optionally, the target data includes target video data, and the data processing rules include data compression rules; the determination module is further configured to obtain data compression rules corresponding to the target video data and corresponding data according to the access feature information and the data identification of the target video data. Raw data.

Optionally, the device further includes: a first receiving module configured to send a message to the user that the acquisition of the data processing rule corresponding to the target data fails to obtain the data processing rule corresponding to the target data in the absence of a data processing rule corresponding to the target data; receive the message sent by the user. Data processing rules for target data.

Optionally, the device further includes: a second receiving module, configured to send a message to the user that the original data corresponding to the target data fails to obtain the original data when there is no original data corresponding to the target data; receive a message sent by the user for the target data The raw data of the data.

Optionally, the device further includes: a storage module configured to compare the access feature information of the target data with a preset storage condition, and store the target data in the storage area when the access feature information satisfies the preset storage condition.

According to a third aspect of the embodiments of the present specification, a computing device is provided, including:

memory and processor;

The memory is used for storing computer-executable instructions, and the processor is used for executing the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the following steps are implemented:

Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

Based on the data identifier of the target data, determine whether the target data exists in the storage area;

If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding original data;

Use data processing rules to process raw data to generate target data.

According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, implements the steps of the above data processing method.

According to a fifth aspect of the embodiments of the present specification, a computer program is provided, wherein, when the computer program is executed in a computer, the computer is caused to execute the steps of the above data processing method.

A data processing method provided by an embodiment of this specification is applied to a processing terminal in a data storage system, and the data storage system includes a processing terminal and a storage area; the method includes: receiving a target data access instruction sent by a user, wherein the target data access The data identifier of the target data is carried in the instruction, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; based on the data identifier of the target data, it is judged whether the target data exists in the storage area; if not, the data of the target data is used. Identify, determine the data processing rules corresponding to the target data and the corresponding raw data; use the data processing rules to process the raw data, generate target data, and perform data processing based on the processing end in the data storage system, without the need to build additional maintenance data processing services , reducing the waste of data processing resources and further improving the efficiency of data processing.

Description of drawings

1 is a flowchart of a data processing method provided by an embodiment of the present specification;

Fig. 2 is a processing process flow chart of a data processing method provided by an embodiment of this specification;

3 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present specification;

FIG. 4 is a structural block diagram of a computing device provided by an embodiment of the present specification.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar promotions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementation disclosed below.

The terminology used in one or more embodiments of this specification is for the purpose of describing a particular embodiment only and is not intended to limit the one or more embodiments of this specification. As used in the specification or embodiments and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used in this specification in one or more embodiments refers to and includes any and all possible combinations of one or more of the associated listed items.

It will be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, a first could be termed a second, and similarly, a second could be termed a first, without departing from the scope of one or more embodiments of this specification. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

First, the terminology involved in one or more embodiments of the present specification is explained.

Storage system: a system or service used to save user data, users can save, modify, and read their own stored data through the storage system.

Raw data: The initial data obtained in the user project scenario, which is usually processed according to the needs of the user project for further use.

Target data: It can also be understood as result data. Target data is the data required in the user's project scenario, which is usually obtained from the original data through some kind of processing.

Processing: A collection of certain calculation methods that convert the user's original data into target data through processing

Processing rules: It is used to define the processing process and describe which raw data needs to be converted into the required target data through which processing process for specific result data.

In this specification, a data processing method is provided, and this specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

With the continuous development of computer technology, more and more computer technology is applied in the field of data processing. Data is an expression of facts, concepts or instructions that can be processed by human or automated means. The data can be in the form of numbers, words, pictures or sounds. The basic purpose of data processing is to extract and derive valuable and meaningful data for some specific people from a large amount of data that may be disorganized and difficult to understand. Data processing runs through the entire social production and society. All areas of life, the development of data processing technology, and the breadth and depth of applications have greatly affected the development of human society.

In practical applications, for users, the original data is usually stored in the storage system, and then the original data is processed according to their own business needs, and the processing results are also stored in the storage system. When you need to use the result data, you need to access the storage system and read and use the processing result. However, in many industries, not all raw data needs to be used. The traditional structure will lead to a large amount of raw data that does not need to be calculated as target data in advance, resulting in a waste of computing resources. Therefore, an efficient data processing solution is urgently needed.

In order to improve data processing efficiency, this specification provides a data processing method, which is applied to a processing end in a data storage system. The data storage system includes a processing end and a storage area; by receiving a target data access instruction sent by a user, the target data The data identifier of the target data is carried in the access instruction; based on the data identifier of the target data, it is judged whether the target data exists in the storage area; if not, the data processing rule corresponding to the target data and the corresponding original data are determined by using the data identifier of the target data; Use data processing rules to process raw data to generate target data. There is no need to change the user's usage habits, the user does not need to process the original data in advance, and directly reads the result data in the original way. When the result data does not exist, the data storage system automatically stores the data in real time according to the pre-stored data processing rules. The original data is processed into target data and returned to the user, which simplifies the process of using the data for the user, and at the same time processes the data according to the needs of the user, which can also reduce the calculation cost of the user processing data and the storage cost of the result data. The processing end in the system performs data processing without the need to build and maintain data processing services, reducing the waste of data processing resources and further improving data processing efficiency.

Referring to FIG. 1, FIG. 1 shows a flowchart of a data processing method provided by an embodiment of this specification. The data processing method is applied to a processing terminal in a data storage system. The data storage system includes a processing terminal and a storage area. Specifically Include the following steps:

Step 102: Receive a target data access instruction sent by the user, wherein the target data access instruction carries a data identifier of the target data.

In the embodiment of this specification, in order to process data according to the needs of users and reduce the calculation cost of user processing data and the storage cost of result data, the processing end in the data storage system can receive the target data access instruction sent by the user, wherein the target data access instruction The data access instruction carries the data identifier of the target data, and the processing end processes the data actually required by the user based on the target data access instruction sent by the user.

Specifically, the target data refers to the data that the user needs to use in the work scene, and the access command of the target data refers to the command to access the target data, which can be an input command or a click command, which is selected according to the actual situation. The embodiments of the specification do not make any limitation on this. The data identifier of the target data refers to the unique identifier of the data, and the target data can be located by the data identifier of the target data, wherein the data identifier of the target data includes but is not limited to the data number of the target data, which is selected according to the actual situation. The embodiment does not make any limitation on this.

In practical applications, there are many ways for the user to send the target data access instruction, which is specifically selected according to the actual situation, which is not limited in the embodiment of this specification.

In a possible implementation, the data storage system can provide the user with a target data input interface, and the user can input the data identifier of the target data in the target data input interface, so that the processing end in the data storage system can obtain the target data based on the data identifier. target data.

In another possible implementation manner, the data storage system can display the data identifier of each data stored in the storage area to the user, and the user can select the data identifier of the target data in the display interface of the data storage system, so that the data storage system can The processing end of the target data obtains the target data based on the data identifier of the target data.

Step 104: Based on the data identifier of the target data, determine whether the target data exists in the storage area.

In the embodiment of this specification, after receiving the data identifier of the target data, the processing end in the data storage system can determine whether the target data exists in the storage area based on the data identifier of the target data, and in the case that the target data exists in the storage area, the processing The terminal can directly obtain the target data and feed the target data back to the user.

In practical applications, the target data can be directly searched in the storage area according to the data identifier of the target data, and it is further judged whether the target data exists in the storage area.

Exemplarily, as shown in Table 1 below, Table 1 shows the correspondence between data stored in the storage area and data identifiers. If the data identification of the target data obtained by the processing end in the data storage system is "003", according to the data identification of the target data, the data identification "003" of the target data is searched in the storage area, and the obtained target data is "Data C"; The data identification of the target data obtained by the processing end in the data storage system is "006", and according to the data identification of the target data, the data identification "006" of the target data is searched in the storage area. The target data with the data ID "006" does not exist.

Table 1

data identification Data stored in the storage area 001 data A 002 dataB 003 data C 004 data D

Step 106: If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding raw data.

In the embodiment of this specification, the processing end in the data storage system may receive the target data access instruction that carries the data identifier of the target data sent by the user, and based on the data identifier of the target data, determine whether the target data exists in the storage area, and if the storage area contains the target data When the target data does not exist, the data processing rules corresponding to the target data and the original data for generating the target data can be obtained based on the data identifier of the target data, and the target data can be further generated.

In practical applications, after judging that the target data does not exist in the storage area, the processing end in the data storage system can judge whether there is a data processing rule corresponding to the target data in the data storage system based on the data identifier of the target data, and further process the data.

In an embodiment of the present specification, if there is no data processing rule corresponding to the target data in the data storage system, a message that the acquisition of the data processing rule corresponding to the target data fails is sent to the user.

Exemplarily, as shown in Table 2 below, Table 2 shows the correspondence between data processing rules and data identifiers. If the data identifier of the target data obtained by the processing end in the data storage system is "006", according to the data identifier "006" of the target data, the data processing rule corresponding to the data identifier "006" is searched in the data storage system. In this case, it is determined that there is no data processing rule with the data identifier "006", and a message that the acquisition of the data processing rule corresponding to the target data has failed is sent to the user.

Table 2

data identification data processing rules 001 Data Processing Rule A 002 Data Processing Rule B 003 Data Processing Rules C 004 Data Processing Rules D 005 Data Processing Rule E

Further, in the case of failure to obtain the data processing rules corresponding to the target data sent to the user, the data processing rules for the target data sent by the user can also be received, so that the processing end can obtain the target data according to the data processing rules sent by the user, and also That is, the above method may also include the following steps:

If there is no data processing rule corresponding to the target data, send a message to the user that the acquisition of the data processing rule corresponding to the target data fails;

Receive data processing rules for target data sent by users.

In the embodiment of this specification, the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data. Based on the data identifier of the target data, after determining whether the target data exists in the storage area, if the target In the case of the data processing rule corresponding to the data, a message of failure to obtain the data processing rule corresponding to the target data is sent to the user, and the data processing rule for the target data sent by the user is received.

It should be noted that there are various ways to send the message that the data processing rule corresponding to the target data fails to be obtained to the user, which is selected according to the actual situation, which is not limited in the embodiment of this specification.

In a possible implementation manner, a message of failure to acquire the data processing rule corresponding to the target data may be sent to the user in the form of a pop-up window on the display interface of the data processing system.

Exemplarily, when there is no data processing rule corresponding to the target data, a 10cm*15cm pop-up window is displayed in the center of the interface of the data processing system, and in the pop-up window, "The data processing rule corresponding to the target data is not exist".

In another possible implementation manner, a voice prompt may be used to prompt the user that the acquisition of the data processing rule corresponding to the target data fails.

In practical applications, after the user is prompted that the acquisition of the data processing rule corresponding to the target data fails, the user may be actively prompted to input the data processing rule corresponding to the target data.

Exemplarily, when there is no data processing rule corresponding to the target data, a 10cm*15cm pop-up window is displayed in the center of the interface of the data processing system, and in the pop-up window, "The data processing rule corresponding to the target data is not exists, please enter the data processing rule corresponding to the target data".

Applying the scheme of the embodiment of this specification, in the absence of the data processing rule corresponding to the target data, sending a message to the user that the acquisition of the data processing rule corresponding to the target data failed, and receiving the data processing rule for the target data sent by the user, so that It can be processed based on the data processing rules sent by the user to obtain target data and further improve the user experience.

In another embodiment of this specification, if there is a data processing rule corresponding to the target data in the data storage system, it is judged whether there is original data corresponding to the target data in the data storage system, and further data processing is performed.

If the original data corresponding to the target data does not exist in the data storage system, a message of failure to obtain the original data corresponding to the target data is sent to the user.

Exemplarily, as shown in Table 3 below, Table 3 shows the corresponding relationship between the original data and the data identifier. If the data identifier of the target data obtained by the processing end in the data storage system is "006", according to the data identifier "006" of the target data, the original data corresponding to the data identifier "006" is searched in the data storage system. Next, it is determined that there is no original data with the data identifier "006", and a message that the original data corresponding to the target data has failed to be obtained is sent to the user.

table 3

data identification Raw data 001 original data A 002 original data B 003 raw data C 004 original data D. 005 raw data E

Further, in the case of failure to obtain the raw data corresponding to the target data sent to the user, the raw data for the target data sent by the user can also be received, so that the processing end can obtain the target data according to the raw data sent by the user, that is, the above The method may also include the following steps:

In the case that the original data corresponding to the target data does not exist, send a message to the user that the acquisition of the original data corresponding to the target data fails;

Receive the raw data sent by the user for the target data.

In the embodiment of this specification, the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data. Based on the data identifier of the target data, after determining whether the target data exists in the storage area, if the target In the case of the original data corresponding to the data, a message that the acquisition of the original data corresponding to the target data has failed is sent to the user, and the original data for the target data sent by the user is received.

It should be noted that, there are many ways to send the message that the original data corresponding to the target data failed to obtain the original data to the user, which is specifically selected according to the actual situation, which is not limited in the embodiment of this specification.

In a possible implementation manner, a message of failure to obtain the original data corresponding to the target data may be sent to the user in the form of a pop-up window on the display interface of the data processing system.

Exemplarily, in the case where there is no raw data corresponding to the target data, a 10cm*15cm pop-up window is displayed in the center of the interface of the data processing system, and "the raw data corresponding to the target data does not exist" is displayed in the pop-up window. .

In another possible implementation manner, a voice prompt may be used to prompt the user that the acquisition of the original data corresponding to the target data fails.

In practical applications, after the user is prompted that the acquisition of the original data corresponding to the target data fails, the user may be actively prompted to input the original data corresponding to the target data.

Exemplarily, in the case where the original data corresponding to the target data does not exist, a 10 cm*15 cm pop-up window is displayed in the center of the interface of the data processing system, and the pop-up window displays "The original data corresponding to the target data does not exist, Please enter the original data corresponding to the target data".

It is worth noting that if there are data processing rules corresponding to the target data and raw data corresponding to the target data in the data storage system, the data processing rules corresponding to the target data and the raw data corresponding to the target data are directly extracted.

Applying the solutions of the embodiments of this specification, in the absence of the original data corresponding to the target data, a message that the acquisition of the original data corresponding to the target data failed to be sent to the user, and the original data for the target data sent by the user is received, so that it can be based on the user. The sent raw data is processed to obtain the target data, which further improves the user experience.

In practical applications, when using the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding raw data, the data corresponding to the target data can be obtained according to the user's access feature information for the target data and the data identifier of the target data. The processing rules and the corresponding original data, that is, the above-mentioned steps of using the data identification of the target data to determine the data processing rules corresponding to the target data and the corresponding original data may include the following steps:

Obtain user access feature information for target data;

According to the access feature information and the data identifier of the target data, the data processing rules corresponding to the target data and the corresponding original data are obtained.

In the embodiment of this specification, the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and based on the data identifier of the target data, it is judged whether the target data exists in the storage area, and if not, the user is obtained. For the access feature information of the target data, according to the access feature information and the data identifier of the target data, the data processing rules corresponding to the target data and the corresponding original data are obtained.

Specifically, the access feature information of the target data refers to the relevant information of the target data being accessed in the data storage system, which can reflect the characteristics such as the way and status of the target data being accessed. The access times, access time, access method, and access status of the data are selected according to actual conditions, which are not limited in the embodiments of this specification.

Exemplarily, the number of user accesses to the target data is obtained, and data processing rules corresponding to the target data and corresponding raw data are obtained according to the user's access times to the target data and the data identifier of the target data.

Applying the solution of the embodiment of this specification, by obtaining the user's access feature information for the target data, according to the access feature information and the data identifier of the target data, the data processing rules corresponding to the target data and the corresponding original data are obtained, without processing all the data in advance, The data processing cost is reduced, and the user experience is further improved.

It should be noted that, after the access feature information of the target data is obtained, the access feature information can be matched with the preset data processing level, the target processing level corresponding to the access feature can be obtained, and the data processing rules corresponding to the target data and the corresponding data processing rules can be further determined. The original data, that is, the above-mentioned steps of obtaining the data processing rules corresponding to the target data and the corresponding original data according to the access feature information and the data identifier of the target data, may include the following steps:

Matching the access feature information with a preset processing level to obtain a target processing level corresponding to the access feature information;

According to the target processing level and the data identification of the target data, the data processing rule corresponding to the target data and the corresponding original data are determined.

In the embodiment of this specification, the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and based on the data identifier of the target data, it is judged whether the target data exists in the storage area, and if not, the user is obtained. For the access feature information of the target data, match the access feature information with the preset processing level to obtain the target processing level corresponding to the access feature information, and determine the data processing rule corresponding to the target data according to the target processing level and the data identifier of the target data. and the corresponding raw data.

Specifically, the target processing level refers to the processing level corresponding to the target data, and the processing level refers to a preset level obtained by dividing data processing rules, which is specifically selected according to the actual situation, which is not limited in the embodiments of this specification. .

Exemplarily, the pre-set processing level table is obtained as shown in Table 4 below. Table 4 shows the corresponding relationship between the number of visits of the target data and the processing level, and the number of visits of the user for the target data is 2 times, in Table 2. The processing level corresponding to the number of query visits 2 is "primary compression". Further, the data processing rule corresponding to "primary compression" is obtained as "data processing rule A1", and the original data corresponding to the target data is obtained according to the data identifier of the target data.

Table 4

Applying the solution of the embodiment of this specification, by matching the access feature information with the preset processing level, the target processing level corresponding to the access feature information is obtained, and the data processing corresponding to the target data is determined according to the target processing level and the data identifier of the target data. Rules and corresponding raw data improve the accuracy of data processing rules and raw data, reduce data processing costs, and further improve user experience.

In practical applications, the target data can be target video data, and the data processing rules can be data compression rules. After obtaining the user's access feature information for the target video data, it can be based on the access feature information of the target video data and the data identifier of the target video data. , obtain the data compression rules corresponding to the target video data and the corresponding raw data, that is, the above-mentioned steps of obtaining the data processing rules corresponding to the target data and the corresponding raw data according to the access feature information and the data identification of the target data, may include the following step:

According to the access feature information and the data identifier of the target video data, the data compression rule corresponding to the target video data and the corresponding original data are obtained.

Specifically, the target video data refers to the video data that the user wants to view, and the data compression rule refers to the compression rule for each video data, which is selected according to the actual situation, which is not limited in the embodiments of this specification.

It should be noted that compressing the target video data can save user bandwidth. Since primary compression consumes less computing resources, but the compression result is larger, it consumes more network bandwidth. Advanced compression consumes more computing resources, but the compression result is smaller and consumes less network bandwidth. Therefore, it is possible to The target video data is compressed according to the access feature information of the target video data.

Exemplarily, the number of visits of the user to the target video data 1 is 20, and the data identifier of the target video data 1 is 003. Based on the above Table 4, according to the number of visits of the target video data 1, the processing level of obtaining the target video data 1 is: "Primary compression", the data processing rule of the target video data 1 is "data processing rule A1"; the number of visits to the target video data 2 by the user is 200,000, and the data identification of the target video data 2 is 005. Based on the above Table 4, according to The number of accesses of the target video data 2, the processing level of the obtained target video data 2 is "high-level compression", and the data processing rule of the target video data 2 is "data processing rule B1".

By applying the solutions of the embodiments of this specification, the data compression rules corresponding to the target video data and the corresponding original data are obtained according to the access feature information and the data identifier of the target video data, which reduces the user's cost and further improves the user experience.

Step 108: Process the original data by using the data processing rules to generate target data.

In the embodiment of this specification, after receiving the target data access instruction sent by the user, the target data access instruction carries the data identifier of the target data, and based on the data identifier of the target data, it is judged whether the target data exists in the storage area, if not, use After determining the data identification of the target data, the data processing rules corresponding to the target data and the corresponding raw data, the data processing rules can be used to process the raw data, generate target data, and feed the target data back to the user.

The solution of the embodiment of this specification is applied, and the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; The data identifier of the data is used to determine whether the target data exists in the storage area; if not, use the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate a target Data, through data processing based on the processing end in the data storage system, does not need to build and maintain data processing services, reduces the waste of data processing resources, and further improves data processing efficiency, and users do not need to change the behavior of accessing the target data, directly access For the target data, there is no need to modify the logic to request data on-demand processing services or batch processing services, which further improves the user experience.

In practical applications, after the original data is processed by the data processing rules and the target data is generated, the storage method of the target data can be determined according to the access feature information of the target data, and the target data can be stored based on the storage method of the target data. That is, after the above-mentioned steps of using data processing rules to process original data to generate target data, the following steps may also be included:

Compare the access feature information of the target data with the preset storage conditions, and store the target data in the storage area if the access feature information satisfies the preset storage conditions.

Specifically, the preset storage condition refers to a preset condition for storing data in the data storage system, which is specifically selected according to the actual situation, which is not limited in the embodiment of this specification.

Exemplarily, obtaining the preset storage condition is that the number of visits is greater than 5000. The number of visits to obtain the target data 1 is 2000. Since the number of visits of the target data 1 is less than the preset storage condition, the processing end does not store the target data 1 to the storage area in the data storage system; the access to obtain the target data 2 The number of times is 9000. Since the number of accesses of the target data 2 is greater than the preset storage condition, the processing end stores the target data 2 in the storage area in the data storage system.

By applying the solutions of the embodiments of this specification, by comparing the access feature information of the target data with the preset storage conditions, when the access feature information satisfies the preset storage conditions, the target data is stored in the storage area, which reduces the amount of storage in the data storage system. The amount of data stored in the area further improves the efficiency of data processing.

It should be noted that the data processing methods provided in this specification are applied to data processing processes in various scenarios, such as communication scenarios and transaction scenarios, and of course can also be applied to other scenarios. The application scenarios of the data processing methods in this specification Not limited.

Below in conjunction with Figure 2, the data processing method is further described by taking the application of the data processing method provided in this specification in a video data processing scene as an example, and the data processing method is applied to the processing end in the data storage system, The data storage system includes a processing end and a storage area. 2 shows a processing flow chart of a data processing method provided by an embodiment of this specification, which specifically includes the following steps:

Step 202: Receive a target video data access instruction sent by a user, wherein the target video data access instruction carries a data identifier of the target video data.

Step 204: Based on the data identifier of the target video data, determine whether the target video data exists in the storage area.

Step 206: If yes, feedback the target video data to the user.

Step 208: If no, use the data identifier of the target video data to determine whether there is a data compression rule corresponding to the target video data in the storage area.

Step 210: If not, feedback to the user that the acquisition of the data compression rule corresponding to the target video data fails.

Step 212: If yes, use the data identifier of the target video data to determine whether there is original video data corresponding to the target video data in the storage area.

Step 214: If no, feedback to the user that the acquisition of the original video data corresponding to the target video data fails.

Step 216: If yes, use the data compression rules to process the original video data to generate target video data.

Step 218: Feed back the target video data to the user.

Applying the scheme of the embodiment of this specification, receiving the target video data access instruction sent by the user, wherein the target video data access instruction carries the data identifier of the target video data, and based on the data identifier of the target video data, it is judged whether there is a target video in the storage area. data, if yes, feed back the target video data to the user, if not, use the data identifier of the target video data to determine whether there is a data compression rule corresponding to the target video data in the storage area, if not, feed back the data corresponding to the target video data to the user If the acquisition of the compression rule fails, if so, use the data identifier of the target video data to determine whether the original video data corresponding to the target video data exists in the storage area; The data compression rules process the original video data, generate target video data, feed the target video data to the user, and process the data based on the processing end in the data storage system, without the need to build additional maintenance data processing services, reducing the waste of data processing resources , which further improves the data processing efficiency.

Corresponding to the above method embodiments, this specification also provides an embodiment of a data processing apparatus. FIG. 3 shows a schematic structural diagram of a data processing apparatus provided by an embodiment of this specification, and the data processing apparatus is applied in a data storage system. The processing end of the data storage system includes a processing end and a storage area. As shown in Figure 3, the device includes:

The receiving module 302 is configured to receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

The judgment module 304 is configured to judge whether the target data exists in the storage area based on the data identification of the target data;

The determining module 306 is configured to, if not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding raw data;

The generating module 308 is configured to process the original data by using the data processing rules to generate target data.

Optionally, the determining module 306 is further configured to obtain user access feature information for the target data; obtain data processing rules corresponding to the target data and corresponding raw data according to the access feature information and the data identifier of the target data.

Optionally, the determining module 306 is further configured to match the access feature information with a preset processing level to obtain a target processing level corresponding to the access feature information; according to the target processing level and the data identifier of the target data, determine that the target data corresponds to The data processing rules and the corresponding raw data.

Optionally, the target data includes target video data, and the data processing rules include data compression rules; the determination module 306 is further configured to obtain data compression rules corresponding to the target video data and corresponding data according to the access feature information and the data identifier of the target video data. the original data.

Optionally, the device further includes: a first receiving module configured to send a message to the user that the acquisition of the data processing rule corresponding to the target data fails to obtain the data processing rule corresponding to the target data in the absence of a data processing rule corresponding to the target data; receive the message sent by the user. Data processing rules for target data.

Optionally, the device further includes: a second receiving module, configured to send a message to the user that the original data corresponding to the target data fails to obtain the original data when there is no original data corresponding to the target data; receive a message sent by the user for the target data The raw data of the data.

Optionally, the device further includes: a storage module configured to compare the access feature information of the target data with a preset storage condition, and store the target data in the storage area when the access feature information satisfies the preset storage condition.

The solution of the embodiment of this specification is applied, and the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; The data identifier of the data is used to determine whether the target data exists in the storage area; if not, use the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate a target Data, through data processing based on the processing end in the data storage system, does not need to build and maintain data processing services, reduces the waste of data processing resources, and further improves data processing efficiency.

The above is a schematic solution of a data processing apparatus according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method belong to the same concept, and the details that are not described in detail in the technical solution of the data processing device can be referred to the description of the technical solution of the above-mentioned data processing method. .

FIG. 4 shows a structural block diagram of a computing device 400 provided according to an embodiment of the present specification. Components of the computing device 400 include, but are not limited to, memory 410 and processor 420 . The processor 420 is connected with the memory 410 through the bus 430, and the database 450 is used for saving data.

Computing device 400 also includes access device 440 that enables computing device 400 to communicate via one or more networks 460 . Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN) or a communication network such as the Internet The combination. Access device 440 may include one or more of any type of network interface (eg, Network Interface Card (NIC)), wired or wireless, such as an IEEE 802.11 Wireless Local Area Networks (WLAN) wireless interface , World Interoperability for Microwave Access (Wi-MAX, World Interoperability for Microwave Access) interface, Ethernet interface, Universal Serial Bus (USB, Universal Serial Bus) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC, Near Field Communication) interface, and so on.

In one embodiment of the present specification, the above-described components of computing device 400 and other components not shown in FIG. 4 may also be connected to each other, such as through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 4 is only for the purpose of example, rather than limiting the scope of this specification. Those skilled in the art can add or replace other components as required.

Computing device 400 may be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (eg, tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (eg, smart phones) ), wearable computing devices (eg, smart watches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. Computing device 400 may also be a mobile or stationary server.

The processor 420 is configured to execute the following computer-executable instructions, the processor is configured to execute the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the following steps are implemented:

Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

Based on the data identifier of the target data, determine whether the target data exists in the storage area;

If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding original data;

Use data processing rules to process raw data to generate target data.

The solution of the embodiment of this specification is applied, and the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; The data identifier of the data is used to determine whether the target data exists in the storage area; if not, use the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate a target Data, through data processing based on the processing end in the data storage system, does not need to build and maintain data processing services, reduces the waste of data processing resources, and further improves data processing efficiency.

The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned data processing method belong to the same concept. For details not described in detail in the technical solution of the computing device, refer to the description of the technical solution of the above-mentioned data processing method.

An embodiment of the present specification further provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, implements the following steps:

Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

Based on the data identifier of the target data, determine whether the target data exists in the storage area;

If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding original data;

Use data processing rules to process raw data to generate target data.

The solution of the embodiment of this specification is applied, and the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; The data identifier of the data is used to determine whether the target data exists in the storage area; if not, use the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate a target Data, through data processing based on the processing end in the data storage system, does not need to build and maintain data processing services, reduces the waste of data processing resources, and further improves data processing efficiency.

The above is a schematic solution of a computer-readable storage medium of this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned data processing method belong to the same concept, and the details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above-mentioned data processing method.

An embodiment of the present specification also provides a computer program, wherein, when the computer program is executed in a computer, the computer is caused to perform the following steps:

Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data;

Based on the data identifier of the target data, determine whether the target data exists in the storage area;

If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding original data;

Use data processing rules to process raw data to generate target data.

The solution of the embodiment of this specification is applied, and the target data access instruction sent by the user is received, wherein the target data access instruction carries the data identifier of the target data, and the data processing efficiency is improved by performing data processing according to the actual data access situation of the user; The data identifier of the data is used to determine whether the target data exists in the storage area; if not, use the data identifier of the target data to determine the data processing rules corresponding to the target data and the corresponding original data; use the data processing rules to process the original data to generate a target Data, through data processing based on the processing end in the data storage system, does not need to build and maintain data processing services, reduces the waste of data processing resources, and further improves data processing efficiency.

The above is a schematic solution of a computer program of this embodiment. It should be noted that the technical solution of the computer program and the technical solution of the above-mentioned data processing method belong to the same concept, and the details that are not described in detail in the technical solution of the computer program can be referred to the description of the technical solution of the above-mentioned data processing method.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The computer instructions include computer program code, which may be in source code form, object code form, an executable file, some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

It should be noted that, for the convenience of description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of this specification are not limited by the described action sequences. Limitation, because certain steps may be performed in other orders or simultaneously according to embodiments of the present specification. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all necessary for the embodiments of the specification.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are provided only to aid in the elaboration of the present specification. Alternative embodiments are not intended to exhaust all details, nor do they limit the invention to only the described embodiments. Obviously, many modifications and changes can be made in accordance with the contents of the embodiments of the present specification. These embodiments are selected and described in this specification to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims (10)

1. A data processing method, applied to a processing terminal in a data storage system, the data storage system comprising a processing terminal and a storage area; the method comprises: Receive the target data access instruction sent by the user, wherein the target data access instruction carries the data identifier of the target data; Based on the data identifier of the target data, determine whether the target data exists in the storage area; If not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding raw data; The original data is processed by using the data processing rule to generate the target data. 2. The method according to claim 1, wherein the step of determining the data processing rule corresponding to the target data and the corresponding original data using the data identification of the target data, comprises: obtaining access feature information of the user for the target data; According to the access feature information and the data identifier of the target data, the data processing rule corresponding to the target data and the corresponding raw data are acquired. 3. The method according to claim 2, wherein the step of obtaining the corresponding data processing rules of the target data and the corresponding raw data according to the access feature information and the data identification of the target data, comprises: Matching the access feature information with a preset processing level to obtain a target processing level corresponding to the access feature information; According to the target processing level and the data identifier of the target data, the data processing rule corresponding to the target data and the corresponding raw data are determined. 4. The method according to claim 2, wherein the target data comprises target video data, and the data processing rules comprise data compression rules; The step of obtaining the data processing rules corresponding to the target data and the corresponding original data according to the access feature information and the data identifier of the target data, including: According to the access feature information and the data identifier of the target video data, a data compression rule corresponding to the target video data and corresponding original data are obtained. 5. The method of claim 1, further comprising: If there is no data processing rule corresponding to the target data, send a message to the user that the acquisition of the data processing rule corresponding to the target data fails; A data processing rule for the target data sent by the user is received. 6. The method of claim 1, further comprising: In the case that the original data corresponding to the target data does not exist, sending a message that the acquisition of the original data corresponding to the target data fails to the user; The original data for the target data sent by the user is received. 7. The method according to claim 2, after the step of using the data processing rules to process the original data to generate the target data, further comprising: Comparing the access feature information of the target data with a preset storage condition, in the case that the access feature information satisfies the preset storage condition, the target data is stored in the storage area. 8. A data processing device, applied to a processing terminal in a data storage system, the data storage system comprising a processing terminal and a storage area; the device comprising: a receiving module, configured to receive a target data access instruction sent by a user, wherein the target data access instruction carries a data identifier of the target data; a judgment module, configured to judge whether the target data exists in the storage area based on the data identifier of the target data; A determination module, configured to, if not, use the data identifier of the target data to determine the data processing rule corresponding to the target data and the corresponding raw data; The generating module is configured to process the original data by using the data processing rule to generate the target data. 9. A computing device comprising: memory and processor; The memory is used for storing computer-executable instructions, the processor is used for executing the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the data processing method of any one of claims 1 to 7 are implemented . 10. A computer-readable storage medium storing computer-executable instructions, the computer-executable instructions implementing the steps of the data processing method according to any one of claims 1 to 7 when the computer-executable instructions are executed by a processor.

Images