CN114493847A

CN114493847A - Data processing method, device and system based on big data environment

Info

Publication number: CN114493847A
Application number: CN202210120272.XA
Authority: CN
Inventors: 妥鑫
Original assignee: Jingdong Technology Holding Co Ltd
Current assignee: Jingdong Technology Holding Co Ltd
Priority date: 2022-02-07
Filing date: 2022-02-07
Publication date: 2022-05-13

Abstract

The application provides a data processing method, a device and a system based on a big data environment, wherein the method comprises the following steps: acquiring planning time and data income configuration information, determining expected data volume required to be acquired on each date in a data cycle acquisition period, adjusting the expected data volume required to be acquired on the current date according to the expected data volume and actual data income volume generated by data entering a pool to obtain target data volume, acquiring data holding volume ending to the current date, and acquiring data on the current date according to the target data volume and the data holding volume. According to the technical scheme, the expected data volume required to be acquired on each date in the data cycle acquisition period is dynamically adjusted in real time to obtain the target data volume, and then the data acquisition volume of the current date is determined through the target data volume and the current data holding volume, so that the data acquisition volume of each date can be accurately calculated, and the calculation accuracy is improved.

Description

Data processing method, device and system based on big data environment

Technical Field

The present application relates to the field of big data technologies, and in particular, to a data processing method, apparatus, and system based on a big data environment.

Background

During daily shopping, a user may conduct a transaction by credit, i.e., a merchant holds a customer's credit voucher as a collateral asset, and the user needs to pay back the asset and the generated interest in a specified period. The merchant can issue the credit assets to other groups for purchase, and the interest paid by the user can be used as the income of other groups after the other groups purchase the credit assets.

In the related art, when purchasing credit assets issued by merchants, the related data usually needs to invest a lot of labor and time cost for analysis and calculation, and the purchasing quantity data is determined by combining with the expected income of the buyers, so as to obtain the expected income through the purchased credit assets.

However, in the process of analyzing and calculating the purchase amount data by using manpower in the prior art, the process of analyzing and calculating the purchase amount data is very complicated, and situations such as omission and miscalculation are easily caused, so that the accuracy of the calculated result is poor, and the daily purchase amount data cannot be accurately calculated.

Disclosure of Invention

The application provides a data processing method, a data processing device and a data processing system based on a big data environment, which are used for solving the problem that omission and miscalculation are easy to occur in the existing data analysis and calculation, so that the accuracy of the calculation result of data is poor.

In a first aspect, an embodiment of the present application provides a data processing method based on a big data environment, which is applied to a server, and the method includes:

acquiring planning time and data benefit configuration information in a preset target plan, wherein the planning time at least comprises starting time and ending time of a data cycle acquisition period, and the data benefit configuration information at least comprises expected data benefit amount in the data cycle acquisition period;

determining the expected data volume required to be acquired by each date in the data cycle acquisition period according to the planning time and the data profit configuration information;

according to the expected data volume and the actual data yield generated by the data which enters the pool, the expected data volume required to be acquired on the current date is adjusted to obtain a target data volume, wherein the data which enters the pool is acquired before the current date;

and acquiring the data holding amount till the current date, and acquiring data at the current date according to the target data amount and the data holding amount, wherein the data is used for indicating the credit assets of the user.

In one possible design of the first aspect, the method further includes:

obtaining the return amount, the overdue unreturned amount and the return benefits of the data which are already put into the pool by the current date;

and acquiring the actual data yield of the data which has entered the pool according to the return amount, the overdue unreturned amount and the to-be-returned yield.

In another possible design of the first aspect, the adjusting the expected data volume required to be acquired at the current date according to the expected data volume and an actual data profit volume generated by the pooled data to obtain a target data volume includes:

accumulating the expected data volume of each date before the current date to obtain an accumulation result;

subtracting the actual data yield from the accumulation result to obtain a difference;

and taking the difference as the target data amount.

In yet another possible design of the first aspect, the method further includes:

acquiring data acquired by a current date;

according to data acquired on the current date, adjusting the entered pool data to obtain adjusted entered pool data;

and updating the return amount, the overdue unreturned amount and the adjusted to-be-returned income of the entered data acquired on the next date when the next date comes according to the adjusted entered data.

reading the credit assets of users in the upstream business system, and screening available data and information of the data in the credit assets, wherein the information of the data comprises the credit value and the pending reduction value of the users, and the pending reduction value is larger than the credit value.

and acquiring the data yield of the acquirable data according to the credit value and the pending reimbursement value of the user.

stopping data acquisition after the end time of the data cycle acquisition period.

and responding to an execution instruction of the target plan, creating a target plan task corresponding to the target plan and storing the target plan task.

and marking the target planning task after data acquisition is carried out on the current date, wherein the marking is used for triggering data acquisition of the next date when the next date in the data cycle acquisition period comes.

In yet another possible design of the first aspect, the performing data acquisition in the current date according to the target data amount and the data holding amount includes:

comparing the target data volume with the data holding volume;

if the target data volume is larger than the data holding volume, acquiring the data of the data holding volume on the current date;

and if the target data volume is less than or equal to the data holding volume, acquiring the data of the data holding volume on the current date.

In a second aspect, an embodiment of the present application provides a data processing apparatus based on a big data environment, including:

the system comprises an information acquisition module, a data processing module and a data processing module, wherein the information acquisition module is used for acquiring planning time and data profit configuration information in a preset target plan, the planning time at least comprises starting time and ending time of a data cycle acquisition period, and the data profit configuration information at least comprises expected data profit amount in the data cycle acquisition period;

the data volume determining module is used for determining the expected data volume required to be acquired by each date in the data cycle acquisition period according to the planning time and the data profit configuration information;

the data volume adjusting module is used for adjusting the expected data volume required to be acquired on the current date according to the expected data volume and the actual data yield generated by the data entered into the pool to obtain a target data volume, wherein the data entered into the pool is the data acquired before the current date;

and the data acquisition module is used for acquiring the data holding amount till the current date, and acquiring data at the current date according to the target data amount and the data holding amount, wherein the data is used for indicating the credit assets of the user.

In a third aspect, an embodiment of the present application provides a data processing system based on a big data environment, including: the system comprises a data updating module, a profit planning module, a data profit module and a data acquisition module, wherein the data updating module is connected with the data profit module, the data profit module is connected with the profit planning module, and the data acquisition module is connected with the profit planning module;

the data updating module is used for updating the amount, repayment amount and overdue unreturned amount of the acquired data of each date;

the revenue planning module is to determine an expected revenue for each date;

the data profit module is used for calculating the actual profit generated by acquiring data on each date;

and the data acquisition module is used for determining the target data volume of each date according to the actual income and the expected income and acquiring data.

In a fourth aspect, an embodiment of the present application provides a data processing apparatus, including: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes the computer-executable instructions stored by the memory to implement the methods described above.

In a fifth aspect, the present application provides a readable storage medium, in which computer instructions are stored, and when executed by a processor, the computer instructions are used to implement the method described above.

In a sixth aspect, the present application provides a program product including computer instructions, which when executed by a processor implement the method described above.

According to the data processing method, the data processing device and the data processing system based on the big data environment, the expected data volume required to be acquired on each date in the data cycle acquisition period is dynamically adjusted in real time to obtain the target data volume, and then the data purchase volume on each date is determined through the target data volume and the current data holding volume to perform fine acquisition of data, so that the situations of missing calculation, error calculation and the like caused by manual calculation can be reduced, the accuracy of data calculation is improved, and the accuracy of the data purchase volume on each date obtained through calculation is ensured.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application;

fig. 1 is a schematic scene diagram of a data processing method based on a big data environment according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of a data processing method based on a big data environment according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a target plan revenue growth trend;

FIG. 4 is a schematic structural diagram of a data processing apparatus based on a big data environment according to an embodiment of the present application;

FIG. 5 is a block diagram of a data processing system based on big data environment according to an embodiment of the present application;

FIG. 6 is a schematic workflow diagram of a data processing system according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms referred to in this application are explained first:

ABS：

asset-backed Securities (ABS) refers to the issuance of a financing form of negotiable Securities on the basis of credit enhancement by structured design with the future generation of Asset flows of the underlying assets as reimbursement support.

Big data:

big data refers to information that has a huge data size and cannot be captured, managed, processed and organized in a reasonable time through mainstream software tools to achieve the purpose of helping enterprises make business decisions more positive.

Fig. 1 is a scene schematic diagram of a data processing method based on a big data environment according to an embodiment of the present application. As shown in fig. 1, a customer can access a server 11 through a terminal device 10 to make a purchase of an item on an online shopping site. After the customer has selected the item for purchase, the web site may jump to a settlement page where the customer may choose to proceed with the payment, e.g., assuming the value of the item purchased by the customer is 100, the customer may agree on a credit mode for an installment, e.g., three payments in a three month time period, with each payment amount being 34. The final total value paid by the customer is 34 x 3 — 102, which is higher than the value of the item 100. That is, the credit method can bring the expected profit to the website operator as 2, but the website operator needs to fund the goods at the early stage.

In real-life applications, if the total amount of website customers is large and the customer who pays by credit is heavy, large credit assets are generated. The website operator may wish to securitize and distribute these credit assets for purchase by various other parties. When these groups purchase these credit assets, the expected revenue is captured when the customer completes the payment.

In the related technology, when purchasing credit funds issued by a website operator, subjective judgment of the group is usually relied on, an asset purchase plan is often made with time and labor, and if the assets are purchased for a long time, because the asset purchase process involves a complex analysis and calculation process, conditions of calculation omission, miscalculation and the like can exist, the accuracy of the data calculation process is poor, the income obtained by the purchased assets cannot be finely controlled, the fine management of the assets is inconvenient, and the cash flow distribution cannot be effectively predicted.

In order to solve the above problems, according to the data processing method, the data processing device and the data processing system based on the big data environment provided by the embodiment of the application, in the big data environment, the expected data volume required to be acquired on each date in the data cycle acquisition period is dynamically adjusted in real time to obtain the target data volume, then the data is finely acquired through the target data volume and the current data holding volume, the data purchase volume required on each date can be accurately calculated to purchase the data, the income generated by the acquired data is finely controlled, and the final income can reach the expected target income.

The technical solution of the present application will be described in detail below with reference to specific examples. It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 2 is a schematic flow chart of a data processing method based on a big data environment according to an embodiment of the present application. The method can be applied to a server in a big data environment. As shown in fig. 2, the method may specifically include the following steps:

s201, obtaining planning time and data benefit configuration information in a preset target plan.

The planning time at least comprises the starting time and the ending time of the data cycle acquisition period, and the data profit configuration information at least comprises the expected data profit amount in the data cycle acquisition period.

In this embodiment, the data is used to indicate the credit assets of the user, and the data may be issued by the website operator in general, and the data is obtained by obtaining the issued data from the website operator. Where the acquisition of data may refer to the purchase of a user's credit asset, the purchased asset may generate revenue (i.e., the acquired data may generate revenue), which in turn may be reused to support the recurring purchase of the asset during the data cycle acquisition period.

Specifically, the data cycle acquiring period refers to that the refund of the basic asset is not directly distributed to the target planning holder (i.e. the holder after the securitized product is issued) but is used for the original rights beneficiary (i.e. the initiator, which is the original holder of the securitized basic asset) to purchase a new same kind of basic asset, namely, after the acquired data generates revenue, the revenue is used for continuously acquiring the data which is equivalent to the revenue.

For example, the planning time of the target plan may further include a planning start time, a planning end time, a package date, a return period start date, and a return period end date of the target plan.

In this embodiment, after the basic asset (i.e. the customer's credit asset) is pooled, the principal, interest and related income generated by the basic asset are fixed, for example, the customer's credit asset is 100, and if a month is used as a repayment period for three periods, the customer needs to fix paying interest 3. If the holder holds the customer's credit asset, the asset will generate at least 3 revenue.

Illustratively, the data revenue configuration information may include a specific date, a target amount, a current amount, and a deficit. For example, from 1 st and 1 st, to 3 th and 1 st, the target amount of 100 ten thousand is required, and to 7 th and 1 st, the target amount of 300 ten thousand is required.

In this embodiment, the planning time and the data profit configuration information are preset for each target plan when the target plan is issued, and specifically, the target plan may be set by an operator according to actual situations.

S202, determining expected data volume required to be acquired on each date in the data cycle acquisition period according to the planning time and the data benefit configuration information.

In this embodiment, when issuing a target plan, a target plan task may be created and information of the target plan, such as the planning time and data revenue configuration information, may be saved, and a revenue schedule of the target plan, that is, the revenue amount that needs to be purchased on each date in the data cycle acquisition period, may be generated according to the information.

For example, the start time of the data loop acquisition period may be 1 month and 1 day, the end time of the data loop acquisition period may be 7 months and 1 day, and from 1 month and 1 day to 7 months and 1 day, each day may be used as a date, that is, each day has an expected data amount (i.e., an expected bought asset amount) to be acquired, for example, 10 assets are expected to be bought in 1 month and 1 day, and 10 assets are bought if the expected bought asset amount is not adjusted in 1 month and 1 day.

Wherein, the amount of the assets expected to be bought every day can be determined according to the target amount, the current amount and the data cycle acquisition period length. For example, the target amount is 1900, the current amount is 100, the data loop acquisition period is 180 days, and the amount of assets purchased per day may average to 100.

S203, according to the expected data volume and the actual data yield generated by the data which has entered the pool, the expected data volume required to be acquired on the current date is adjusted to obtain the target data volume.

The pooled data is data acquired before the current date, and the generated benefits comprise a return amount, a refund amount, a to-be-returned benefit amount and an overdue unreturned data amount which are up to the current date.

In this embodiment, the target data volume may be obtained by accumulating the expected data volume of each date before the current date to obtain the total data volume that should be obtained by the current date, then counting the actual profit generated by the data that has been obtained by the current date, and subtracting the actual profit from the total data volume.

S204, acquiring the data holding amount till the current date, and acquiring data at the current date according to the target data amount and the data holding amount.

Where the data is used to indicate the user's credit assets. In this embodiment, when data acquisition (i.e. asset purchase) is performed, a refined purchase may be performed according to a target data amount of a current date and a current data holding amount, where the data holding amount is used to indicate a maximum data acquisition amount that can be supported by the current date, and a main goal of the refined purchase is to satisfy an expected benefit of the current date when a cost amount of the current date is satisfied.

For example, the data holding amount may be 100, the target data amount may be 90, and the amount of data that can be obtained from the website operator at the current date may be 90. In other embodiments, if the target amount of data is greater than the data holding amount, for example, the target amount of data is 110, the amount of data that can be obtained from the website operator is 100 at maximum.

According to the embodiment of the application, the assets expected to be bought on each date in the data cycle acquisition period are adjusted in real time to obtain the target purchase amount, and then the assets are subjected to refined purchase through the target purchase amount and the principal, so that the refined control of the income can be realized, and the final income can reach the target income expected by the user.

In some embodiments, the actual data revenue generated by pooled data may be obtained by:

In this embodiment, the return amount may refer to the amount of the credit assets returned by the user, for example, the user returns 100 credit assets today, and the return amount may refer to the amount of the credit assets returned by the user, that is, the user may agree with the assets on credit 100 in advance with the website operator, but the user returns 50, that is, the user only generates 50 credit assets. The overdue unreturned amount is the property that the user did not return at the appointed time, for example, the user appointed the credit-sold property that needs to be returned by 50 today, but the user did not return the credit-sold property of 50 by now.

In this embodiment, when the actual data profit amount generated by the pooled data is counted, the amount profits can be counted according to the target plan dimension, and the effective profits are summarized to obtain the actual profits of the current date. For example, for data that is overdue and unreturned (i.e. the credit assets that are overdue and unreturned by the customer) are counted as 0 when the profit is counted, for example, the profit is over 30 days, even if the profit to be returned of the overdue assets is not 0, the profit to be returned is also counted as 0 when the actual profit is passed, so as to ensure that the counted actual profit can be smaller and the normal operation of buying the subsequent assets is ensured.

According to the method and the device, the actual income of the data which are already put into the pool in the current period is obtained and compared with the expected data volume which needs to be obtained in the current period, the expected data volume which needs to be obtained can be dynamically adjusted, the data can be obtained in a refined mode, and the purpose of purchasing the assets in a refined mode is achieved.

In some embodiments, the step S203 may be specifically implemented by the following steps:

subtracting the actual data yield from the accumulated result to obtain a difference;

the difference is taken as the target data amount.

In this embodiment, each date has a corresponding expected data amount to be acquired, and if each date performs data acquisition according to the expected data amount to be acquired, the data amount acquired on each date can generate a benefit. For example, the expected data amount required to be acquired on day 1/month and day 2 is 10, and the expected data amount required to be acquired on day 1/month and day 2 is 20. The expected amount of data that needs to be acquired for different dates may be the same or different.

Wherein, the sum of the expected data amount required to be acquired on each date can be regarded as the task target purchase amount of the target plan, after the data cycle acquisition period is finished, the sum of the data amount actually acquired on each date should be as close to the task target purchase amount as possible, and the income generated by the actually acquired sum of the data amount is ensured to be close to the target income of the target plan.

According to the method and the device, the sum of the expected data volume required to be acquired on each date before the current date is counted, the target data volume is obtained by subtracting the actual income of the data which enters the pool on the current date, the expected data volume required to be acquired on the current date can be dynamically adjusted, fine acquisition of the data is achieved, the income of purchasing assets can be controlled to be infinitely close to the task target income, and fine control of the income is achieved.

In some embodiments, the method may further include the steps of:

acquiring data acquired by a current date;

according to the data acquired by the current date, the entered pool data is adjusted to obtain the adjusted entered pool data;

In this embodiment, the data information needs to be updated on each date, and the data information to be updated specifically includes the latest data amount information, the overdue unreturned amount, the data returning information, and the like. By updating the data information for each date, the accuracy of the information during the execution of the target plan can be ensured.

When the next date comes, the data acquired in the current period are brought into the pool, and the data acquired in the current period influence the return amount, the overdue unreturned amount and the return income of the next date, so that the return amount, the overdue unreturned amount and the return income of the adjusted pooled data of the next date need to be updated.

Specifically, the repayment amount, the refund amount, the overdue amount, and the adjusted pending return income of the pooled data in the next period may be updated according to the expected return condition of the data acquired in the current period in the next period. For example, if the data acquired by buying in the current epoch is not paid for at the next epoch overdue, the amount of the next epoch overdue needs to be updated.

According to the embodiment of the application, the data information of the next date is updated, so that the data information of each date can be ensured to be accurate, and the fine control of benefits is realized.

In some embodiments, the above method further comprises the steps of:

and reading the credit assets of the users in the upstream business system, and screening available data and information of the data in the credit assets.

The information of the data comprises a credit value and a refund value of the user, and the refund value is larger than the credit value.

In this embodiment, the available data, namely securitized assets issued by the web site that are available for purchase by other groups. The property data details and repayment data details of the credit assets with the credit list as the dimension can be obtained by processing and integrating the credit assets issued by the business system. The repayment data details may include, among other things, the repayment method of the customer, e.g., the customer repays three months for his credit asset, one third of the credit asset being repayed each month, and the corresponding interest.

According to the method and the device for obtaining the credit assets of the users in the upstream business system, the credit assets of the users in the upstream business system are screened, the data amount which can be obtained by other groups in the current period can be determined, namely the credit assets issued by a website operator are determined, and therefore refined purchase can be achieved when data are obtained in the current period.

On the basis of the above embodiments, in some embodiments, the method further includes:

and acquiring the data yield of the acquirable data according to the credit value and the pending payment value of the user.

In this embodiment, the credit value refers to the amount actually credited by the user, and the pending payment value refers to the amount actually required to be returned by the user in the repayment process. The value to be returned typically consists of both the user's credit value and interest. The interest therein is the data yield of the acquirable data.

According to the data income amount acquiring the acquirable data, the expected income can be determined during data acquisition, so that fine control of the income during data acquisition is realized, and the income of the acquired data can be infinitely close to the income of a task target.

In some embodiments, the method may further include the steps of:

at the end of the end date of the data cycle acquisition period, data acquisition is stopped.

In this embodiment, each date of the data loop acquisition period dynamically adjusts the expected data amount it needs to acquire to perform data acquisition until the data loop acquisition period ends. After the last date of the data cycle acquisition period is over, the revenue bias for the entire target plan is the same as the revenue bid differential for the last date.

For example, FIG. 3 is a graph of the growth trend of the target plan gains, as shown in FIG. 3, the ideal gains are a smooth dashed line, and the actual gains are curved solid lines. And when the last date t is finished, the actual income is infinitely close to the ideal income, the target income S is reached, and the purpose of fine control of the income is realized.

In some embodiments, the method may further include the steps of:

and responding to the execution instruction of the target plan, and creating and saving a target plan task corresponding to the target plan.

Responding to an execution instruction of the target plan, and acquiring;

and creating and storing a target planning task according to the planning start time, the planning end time, the start time and the end time of the data cycle acquisition period, the package date, the amortization period start time and the amortization period end time of the target plan.

In this embodiment, the target plan includes time information, and the time information specifically includes a planned start time, a planned end time, a start time and an end time of the data cycle acquisition period, a package date, a contribution period start time, and a contribution period end time of the target plan. The planned starting time and the planned ending time determine a target planned duration, the package date refers to the date of the basic assets entering the pool, and from the day, principal, interest and related income generated by the basic assets belong to the issuing carrier together. The exhibition return period starting date and the exhibition return period ending date determine the exhibition return period duration, in the exhibition return period, the purchase of new assets is stopped, and cash generated by the basic assets is accumulated and then the cash is paid to investors (users) according to a plan.

In this embodiment, after the target planning task is created, the target planning will be performed according to the time information in the target planning task, for example, dynamically adjusting the expected amount of data that needs to be acquired per date within the data loop acquisition period.

Further, in some embodiments, the method may further include the steps of:

after data acquisition is performed on the current date, the target planning task is flagged. Wherein the flag is used to trigger data acquisition for a next date in the data loop acquisition period when the next date arrives.

In this embodiment, after the current date completes the purchase of the asset, a mark is marked on the target planning task, so that the next date can be continuously triggered when coming, namely, when the next date comes, the next date is automatically shifted to the current date, and the steps are continuously executed to complete the data acquisition of the next date until the whole data cycle acquisition period is finished.

By marking the target plan, the embodiment of the application can repeat the step of executing data acquisition when each date comes in the data cycle acquisition period until the data cycle acquisition period is finished. Meanwhile, due to the fact that the step of data acquisition is repeatedly executed on each date, the expected data volume required to be acquired on each date can be dynamically adjusted, fine acquisition of data on each date is achieved, the property purchase principal and the task target purchase amount can be consistent, and purchased property income is close to the task target property income.

In some embodiments, the method may further include the steps of:

comparing the target data quantity with the data holding quantity;

if the target data volume is larger than the data holding volume, acquiring data of the data holding volume at the current date;

if the target data volume is less than or equal to the data holding volume, the data of the data holding volume is acquired at the current date.

In this embodiment, after the target data volume in the current period is determined, refined data acquisition may be performed according to the data holding volume, so that the target data volume acquisition task in the current period is satisfied when the data holding volume sufficiency in each period is satisfied, and the benefit of the target data volume in the current period can reach the target benefit.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Fig. 4 is a schematic structural diagram of a data processing apparatus based on a big data environment according to an embodiment of the present application. The data processing device can be integrated on the server, and can also be independent of the server and realize the scheme in cooperation with the server. As shown in fig. 4, the data processing apparatus 40 includes an information acquisition module 41, a data amount determination module 42, a data amount adjustment module 43, and a data acquisition module 44.

The information obtaining module 41 is configured to obtain planning time and data revenue configuration information in a preset target plan. The data volume determination module 42 is configured to determine an expected data volume to be acquired for each date in the data cycle acquisition period according to the scheduled time and the data revenue configuration information. The data volume adjusting module 43 is configured to adjust the expected data volume required to be obtained at the current date according to the expected data volume and the actual data yield generated by the pooled data to obtain the target data volume. The data acquisition module 44 is configured to acquire the data holding amount up to the current date, and acquire data at the current date according to the target data amount and the data holding amount.

The planning time at least comprises the starting time and the ending time of the data cycle acquisition period, the data profit configuration information at least comprises expected data profit amount in the data cycle acquisition period, the data is used for indicating the credit assets of the user, and the pooled data is data acquired before the current date.

In some embodiments, the apparatus may further include an actual revenue acquisition module to:

In some embodiments, the data amount adjusting module may be specifically configured to:

the difference is taken as the target data amount.

In some embodiments, the apparatus further comprises an update module configured to:

acquiring data acquired by a current date;

according to the data acquired on the current date, the entered pool data is adjusted to obtain the adjusted entered pool data;

In some embodiments, the apparatus further comprises a screening module configured to:

In some embodiments, the apparatus further includes a profit computation module configured to obtain a data profit amount of the acquirable data according to the credit value and the pending-for-return value of the user.

In some embodiments, the apparatus further comprises a buy-in control module for stopping data acquisition after an end time of the data loop acquisition period.

In some embodiments, the shanghai apparatus further comprises a task creation module for creating and saving a target plan task corresponding to the target plan in response to the target plan execution instruction.

In some embodiments, the apparatus further comprises a marking module for marking the target planning task after the data acquisition is performed on the current date.

Wherein the flag is used to trigger data acquisition for a next date in the data loop acquisition period when the next date arrives.

In some embodiments, the data acquisition module may be specifically configured to:

comparing the target data volume with the data holding volume;

The apparatus provided in the embodiment of the present application may be used to execute the method in the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules may all be implemented in software invoked by a processing element. For example, the information obtaining module may be stored in a memory of the apparatus in the form of program code, and a certain processing element of the apparatus calls and executes the functions of the information obtaining module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. In the implementation process, each step of the above method or each module above may be completed by instructions in the form of software.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wirelessly. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium, an optical medium, a semiconductor medium, or the like.

Fig. 5 is a schematic structural diagram of a data processing system based on a big data environment according to an embodiment of the present application, and as shown in fig. 5, the data processing system 50 includes a data updating module 51, a data revenue module 52, a revenue planning module 53, and a data acquisition module 54. The data updating module 51 is connected with the data profit module 52, the data profit module 52 is connected with the profit planning module 53, and the data obtaining module 54 is connected with the profit planning module 53. Wherein, each module can be respectively integrated on different servers, and establishes connection relationship through communication link.

The data updating module 51 is configured to update the amount of the acquired data, the repayment amount, and the overdue unreturned amount for each date. The revenue planning module 53 is used to determine the expected revenue for each date. The data revenue module 52 is used to calculate the actual revenue generated by acquiring data on each date. The data obtaining module 54 is configured to determine a target data amount for each date according to the actual profit and the expected profit, and perform data obtaining.

Fig. 6 is a schematic workflow diagram of a data processing system according to an embodiment of the present disclosure, and as shown in fig. 6, a data updating module 51 in the data processing system 50 is configured to update asset data, refund data, and repayment data, and push an updated result to a data revenue module 52, and the data revenue module 52 is configured to count data revenue, determine a revenue situation of a current date, and transmit the revenue situation to a revenue planning module 53. The profit planning module 53 is configured to generate a profit plan according to the planning time and the profit configuration information in the target plan, and transmit the profit plan and the profit situation to the data obtaining module 54, where the data obtaining module 54 obtains the profit plan and the profit situation according to the profit plan and the profit situation. And generating an acquisition task to acquire data, and finally realizing fine purchasing of assets.

Fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application. As shown in fig. 7, the server 70 includes: at least one processor 71, a memory 72, a bus 73, and a communication interface 74.

Wherein: the processor 71, the communication interface 74 and the memory 72 communicate with each other via a bus 73.

A communication interface 74 for communicating with other devices, such as other servers. The communication interface 74 includes a communication interface for data transmission.

The processor 71 is configured to execute the computer instructions stored in the memory 72, and may specifically perform the relevant steps in the method described in the above embodiments.

The processor may be a central processing unit. The server may include one or more processors, which may be the same type of processor, such as one or more CPUs; or may be a different type of processor.

The memory 72 is used to store computer instructions. The memory may comprise high speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

The present embodiment also provides a readable storage medium, in which computer instructions are stored, and when at least one processor of the server executes the computer instructions, the server executes the data processing method based on the big data environment provided by the various embodiments described above.

The present embodiments also provide a program product comprising computer instructions stored in a readable storage medium. The computer instructions can be read by at least one processor of the server from a readable storage medium, and the computer instructions can be executed by the at least one processor to enable the server to implement the big data environment-based data processing method provided by the various embodiments.

In the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship; in the formula, the character "/" indicates that the preceding and following related objects are in a relationship of "division". "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.

It is to be understood that the various numerical references referred to in the embodiments of the present application are merely for convenience of description and distinction and are not intended to limit the scope of the embodiments of the present application. In the embodiment of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method based on big data environment is applied to a server, and the method comprises the following steps:

2. The method of claim 1, further comprising:

3. The method of claim 1, wherein adjusting the expected data volume required to be obtained at the current date to obtain the target data volume according to the expected data volume and the actual data revenue volume generated by the pooled data comprises:

and taking the difference as the target data amount.

4. The method of claim 1, further comprising:

acquiring data acquired by a current date;

5. The method of claim 1, further comprising:

6. The method of claim 5, further comprising:

7. The method of claim 1, further comprising:

8. The method of claim 1, further comprising:

9. The method of claim 8, further comprising:

10. The method of claim 1, wherein the performing data acquisition at a current date according to the target data volume and the data holding volume comprises:

comparing the target data volume with the data holding volume;

11. A data processing apparatus based on big data environment, comprising:

and the data acquisition module is used for acquiring the data holding amount till the current date and acquiring data at the current date according to the target data amount and the data holding amount, wherein the data is used for indicating the credit assets of the user.

12. A data processing system based on a big data environment, comprising: the system comprises a data updating module, a profit planning module, a data profit module and a data acquisition module, wherein the data updating module is connected with the data profit module, the data profit module is connected with the profit planning module, and the data acquisition module is connected with the profit planning module;

the revenue planning module is to determine an expected revenue for each date;

13. A data processing apparatus, characterized by comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1-10.

14. A readable storage medium having stored therein computer instructions, which when executed by a processor, are adapted to implement the method of any one of claims 1-10.

15. A program product comprising computer instructions, characterized in that the computer instructions, when executed by a processor, implement the method of any of claims 1-10.