CN114218268A

CN114218268A - Data processing method and device, electronic equipment and storage medium

Info

Publication number: CN114218268A
Application number: CN202111412376.XA
Authority: CN
Inventors: 古毅伟; 乔鹏宇; 吴德柱
Original assignee: Shanghai Pudong Development Bank Co Ltd
Current assignee: Shanghai Pudong Development Bank Co Ltd
Priority date: 2021-11-25
Filing date: 2021-11-25
Publication date: 2022-03-22

Abstract

The disclosure relates to a data processing method, a data processing device, an electronic device and a storage medium. The method comprises the following steps: receiving a data reading request of an application, and determining a first object to be read by the data reading request; and under the condition that the first object is stored in a cache region, acquiring the first object from the cache region, and sending the first object to the application. By the embodiment of the disclosure, the interaction times of the application and the data can be reduced, the data reading and writing efficiency of the application is improved, and the user experience is improved.

Description

Data processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.

Background

With the development of computer technology, Application technology appears, and people can realize different requirements such as shopping, entertainment, business handling and the like on line through Application, thereby greatly facilitating the production and life of people. However, in the process of data interaction between the application terminal and the application server, in the case of a large amount of data, the data is often stuck, the time consumed for program execution is long, and the user experience is poor.

Disclosure of Invention

The disclosure provides a data processing method, a data processing device, an electronic device and a storage medium. To solve at least the problem of long program execution time in the related art. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including:

receiving a data reading request of an application, and determining a first object to be read by the data reading request;

and under the condition that the first object is stored in a cache region, acquiring the first object from the cache region, and sending the first object to the application.

In a possible implementation manner, the method further comprises the following steps;

and under the condition that the first object is not stored in the cache region, acquiring the first object from an application database, sending the first object to the application, and storing the first object to the cache region.

In one possible implementation manner, the method further includes:

receiving a saving instruction of the application to the second object;

responding to the saving instruction, and acquiring a plurality of second objects stored at the last time from the cache region;

storing the plurality of second objects to an application database.

In a possible implementation manner, the manner in which the cache region stores the second object includes:

receiving a data writing request of an application, and determining a second object to be written in by the data writing request;

setting a version identification corresponding to the second object;

and storing the second object and the version identification to the cache region.

In one possible implementation, the storing the plurality of second objects to the application database includes:

and storing the plurality of second objects to an application database by using a plurality of second threads, wherein the second threads and the second objects have one-to-one correspondence incidence relation.

In a possible implementation manner, before the receiving a read data request of an application, determining a first object of the read data request, the method further includes:

acquiring a plurality of data reading requests of an application from a program code of the application, and determining a plurality of first objects to be read by the data reading requests;

the method comprises the steps of obtaining a plurality of first objects from an application database by utilizing a plurality of first threads, and storing the plurality of first objects into a cache region, wherein the first threads and the first objects have one-to-one correspondence association relationship.

In one possible implementation manner, the method further includes:

sequentially receiving a plurality of storage instructions of a plurality of application terminals to a second object and storage time corresponding to the storage instructions;

and storing the modified data of the application terminal with the earliest storage time to the second object in an application database, and sending a failure message to other application terminals except the application terminal in the plurality of application terminals.

According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for data processing, including:

the system comprises a first receiving module, a second receiving module and a control module, wherein the first receiving module is used for receiving a data reading request of an application and determining a first object to be read by the data reading request;

the first sending module is used for acquiring the first object from the cache region and sending the first object to the application under the condition that the first object is stored in the cache region.

In one possible implementation manner, the method further includes:

and the second sending module is used for acquiring the first object from an application database under the condition that the first object is not stored in the cache region, sending the first object to the application and storing the first object to the cache region.

In one possible implementation manner, the method further includes:

the second receiving module is used for receiving a storage instruction of the application to the second object;

the obtaining module is used for responding to the saving instruction and obtaining a plurality of second objects which are stored at the latest from the cache region;

and the first storage module is used for storing the plurality of second objects to an application database.

setting a version identification corresponding to the second object;

In one possible implementation, the storage module includes:

and the storage submodule is used for storing the plurality of second objects to an application database by utilizing a plurality of second threads, wherein the second threads and the second objects have one-to-one correspondence incidence relation.

In one possible implementation manner, the method further includes:

the system comprises a determining module, a judging module and a judging module, wherein the determining module is used for acquiring a plurality of data reading requests of an application from a program code of the application and determining a plurality of first objects to be read by the data reading requests;

the second storage module is used for acquiring the first objects from the application database by using a plurality of first threads and storing the first objects into a cache region, wherein the first threads and the first objects have one-to-one correspondence association relationship.

In one possible implementation manner, the method further includes:

the third receiving submodule is used for sequentially receiving a plurality of storage instructions of a plurality of application terminals to a second object and storage time corresponding to the storage instructions;

and the third storage submodule is used for storing the modified data of the application terminal with the earliest storage time to the second object in an application database and sending a failure message to other application terminals except the application terminal in the plurality of application terminals.

According to a third aspect of the embodiments of the present disclosure, there is provided a server, including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method of data processing according to any one of the embodiments of the present disclosure.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of a server, enable the server to perform a method of data processing according to any one of the embodiments of the present disclosure.

According to a third aspect of embodiments of the present disclosure, there is provided a computer program product comprising instructions which, when executed by a processor of a server, enable the server to perform the method of data processing according to any one of the embodiments of the present disclosure.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: frequent interaction with a database has great influence on a program, so that the running time of the program is increased by geometric multiples.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment.

FIG. 2 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment.

FIG. 3 is a block diagram illustrating an apparatus for data processing in accordance with an example embodiment.

FIG. 4 is a block diagram illustrating a server in accordance with an example embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are both information and data that are authorized by the user or sufficiently authorized by various parties.

FIG. 1 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment. As shown in fig. 1, the method is used in a server and includes the following steps.

Step S101, receiving a read data request of an application, and determining a first object to be read by the read data request.

In the disclosed embodiment, the application may include various terminal applications, such as business transaction applications, shopping applications, entertainment applications, and the like. The application may include an application developed in Java language, which accesses a Database of the application through a jdbc (Java Database connectivity) interface and supports functions of establishing a connection, querying SQL statements, processing results, and the like. The server receives a data reading request of the application and determines a first object read by the data reading request. The first object may comprise a data object to be read in a read data request, e.g. by reading price data of a certain product, the price data is the first object.

Step S103, under the condition that the first object is stored in the cache region, the first object is obtained from the cache region, and the first object is sent to the application.

In the embodiment of the present disclosure, the Cache region may include an auxiliary memory, such as a Cache, in the server except for the database. The buffer area is pre-buffered with some data, which may be left by historical access to the database, and the interaction with the database can be reduced by reading the data from the buffer.

TABLE 1

1, the time of each database IO is called Ts, R represents reading, W represents writing, and T1, T2 and T3 represent objects of different databases; 2. the check indicates that the operation can carry out database interaction; 3. the digital representation of the column is saved and several database interactions are required during the save phase.

Referring to table 1, in the conventional mode, each operation of T1, T2, and T3 is performed through a database interaction, which takes a long time compared to other programs of the application program, so that most of the time for the application to execute is spent interacting with the database. In the embodiment 1 of the present disclosure, in the 2 nd column in the table, the first reading of T1 requires one interaction with the database, and the read T1 is stored in the buffer. In subsequent reads of T1, such as column 6 and column 7 of the table, the contents of T1 can be read directly from the buffer without interaction with the database. Similarly, column 5 of the table, reading T2 for the first time, requires an interaction with the database and stores T2 in the buffer. In subsequent reads of T2, such as columns 9 and 10 of the table, the contents of T2 can be read directly from the cache without interaction with the database. In example 2, the database read operation was performed three times, corresponding to the first reads of T1, T2, and T3, respectively, and the total time consumed by the write operation (column 3, column 4, and column 8 in table 1) was 6 Ts.

Frequent interaction with a database has great influence on a program, so that the running time of the program is increased by geometric multiples.

In a possible implementation manner, when the first object is not stored in the cache region, the first object is obtained from an application database, the first object is sent to the application, and the first object is stored in the cache region.

In the disclosed embodiment, referring to columns 2 and 5 of table 1, T1 and T2 are not stored in the buffer at the time of the first reading of T1 and T2. First objects, such as T1, T2, may be retrieved from a database of an application and stored in a cache. In a subsequent read data request, the stored first object is read from the cache.

In one possible implementation, the data processing method further includes:

receiving a saving instruction of the application to the second object;

storing the plurality of second objects to an application database.

In the embodiment of the present disclosure, the second object may include a data object to which a write operation is applied to the database, for example, on APP for product purchase, a user performs an order placing operation, which results in a decrease in the number of product orders in stock, and then the number of products in the database is the second object. Since the write or modify operation on the second object may be performed many times, each write operation may be stored in the cache area, and after receiving the save instruction, the second object stored last may be stored in the database of the application. In one example, a version identification may be set for the second object, and the second object corresponding to the latest version identification is taken as the second object stored most recently. In one example, the retention time of the second object may also be marked, and the second object that was stored most recently may be determined based on the retention time. The saving instruction in the embodiment of the present disclosure may include an instruction for automatically saving the application program, and may also protect the user from saving the application program. Referring to table 1, embodiment 2 in one example, both column 3 and column 4 have written T1, column 8 has written T2, and during the save phase, only the 3 rd column to T1 and the 8 th column to T2 have been written to the database, so in column 12, 2 writes to the database have been made. In example 2, the database read operation was performed three times, corresponding to the first reads of T1, T2, and T3, respectively, and the total time consumed by the write operation was 5 Ts.

Compared with the method that the database interaction is performed every time when the write operation is performed, the method that the write operation is stored in the cache area first and the write operation performed on the second object last time is written into the database in the storage stage can further reduce the interaction on the database and improve the execution rate of the application program.

setting a version identification corresponding to the second object;

In the embodiment of the present disclosure, the version identifier may include a version number, a two-dimensional code, a barcode, a time tag, and the like. In one example, the version number of the second object is set in an optimistically locked manner, for example by adding a data version number version field to the second object indicating the number of times the data has been modified. When the data is modified, the version value is automatically incremented by 1.

According to the embodiment of the disclosure, the writing operation of the second object can be recorded in the form of the version identifier, which is beneficial to determining the modification time sequence of the second object and determining the second object which is stored only at most.

In the embodiment of the present disclosure, each pair of second objects, which performs a write operation, is stored in the buffer. In the case where there are a plurality of second objects, in the save phase, a thread process may be separately provided for each second object. Referring to table 1, embodiment 3, a second thread is opened for T1, a second thread is opened for T2, and in the save phase, the execution time is no longer the sum of the times of all write operations, but the one with the largest execution time among all write operations, for example, in table 1, the save phase in column 12, the execution time is the largest of the save T1 and the save T2. In example 3, the database read operation was performed three times, corresponding to the first reads of T1, T2, and T3, respectively, and the total time consumed by the write operation was 3Ts + max (T1-W, T2-W), which is close to 4 Ts.

According to the embodiment of the disclosure, the plurality of second threads are used for writing data to the plurality of second objects, so that the time consumed by the writing operation is the longest one of the write operation execution times of all the second objects, not the sum of the execution times of all the write operations. Therefore, the embodiment of the disclosure further reduces the interaction to the database and improves the execution rate of the program.

The embodiment of the present disclosure may adopt a static code analysis manner, and before an application program is executed, first analyze a data read operation to be executed by the program, for example, determine a first object to be read by searching for an interface of JDBC. All first objects required by the program are retrieved from the database in advance and stored in the buffer. In this process, a thread is opened for each read operation of the first object, and thus, the read operations of the plurality of first objects can be performed concurrently. In this way, the time consumption of the read operation is the largest time consumption of the read operations of the plurality of second objects, rather than the sum of the time consumption of all the read operations of the second objects. Referring to table 1, in embodiment 4, a plurality of read data requests in an application are pre-read, and a plurality of threads are used, each thread corresponds to a second object, and the consumed time is the maximum value of the read operations in the 2 nd, 5 th and 11 th columns in table 1: max (T1-R, T2-R, T3-R). The total time consumed, plus the time for the write operation, is Max (T1-R, T2-R, T3-R) + Max (T1-W, T2-W), approaching 2 Ts.

FIG. 2 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment. Referring to fig. 2, a plurality of saving instructions of a plurality of application terminals to a second object and saving times corresponding to the saving instructions are sequentially received;

In the embodiments of the data processing method, the read-write operation is performed at the same application terminal, and in the embodiment of the present disclosure, in consideration of a concurrency scenario, in different applications, if the application terminal S1 modifies the second object, the application terminal S2 also modifies the second object. In view of this situation, in the embodiment of the present disclosure, an optimistic lock is added to each second object in the database, each second object has a version number, and only the second object modified on the latest version can be successfully submitted and saved in the database. If a plurality of application terminals access the same second object at the same time, at most one application terminal can submit successfully in the storage stage. In one example, there may be one instance for each application terminal, and the instance rolls back if the delivery fails during the save phase. Referring to fig. 2, the second object records a piece of data: name Tom, age 20. When the application terminal S1 changes the age in the second subject to 21, simultaneously, the application terminal S2 changes the age in the second subject to 22. In one example, it is determined whether the application terminal S1 submitted earlier or the application terminal S2 submitted earlier, e.g., the application terminal S1 submitted earlier in the figure, the modification of the application terminal S1 was saved in the database, and the age in the second object was 21. A save failure message is sent to the application terminal S2, prompting the transaction failure.

According to the embodiment of the disclosure, the modified data of the second object, which is stored by the application terminal with the earliest storage time, is determined to be stored in the application database in the concurrent scene, so that the stability of the writing operation of the database is ensured.

It should be understood that, although the steps in the flowchart are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in the figures may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of execution of the steps or stages is not necessarily sequential, but may be performed alternately or in alternation with other steps or at least some of the other steps or stages.

It is understood that the same/similar parts between the embodiments of the method described above in this specification can be referred to each other, and each embodiment focuses on the differences from the other embodiments, and it is sufficient that the relevant points are referred to the descriptions of the other method embodiments.

FIG. 3 is a block diagram illustrating data processing according to an example embodiment. Referring to fig. 3, the apparatus includes: the system comprises a first receiving module, a second receiving module and a control module, wherein the first receiving module is used for receiving a data reading request of an application and determining a first object to be read by the data reading request;

a first receiving module 301, configured to receive a read data request of an application, and determine a first object to be read by the read data request;

a first sending module 302, configured to, in a case that the first object is stored in a cache, obtain the first object from the cache, and send the first object to the application.

In one possible implementation manner, the method further includes:

setting a version identification corresponding to the second object;

In one possible implementation, the storage module includes:

In one possible implementation manner, the method further includes:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 4 is a block diagram illustrating a server 400 according to an example embodiment. For example, the server 400 may be a server. Referring to fig. 4, server 400 includes a processing component 420 that further includes one or more processors and memory resources, represented by memory 422, for storing instructions, such as applications, that are executable by processing component 420. The application programs stored in memory 422 may include one or more modules that each correspond to a set of instructions. Further, the processing component 420 is configured to execute instructions to perform the above-described method.

The server 400 may further include: the power component 424 is configured to perform power management of the server 400, the wired or wireless network interface 426 is configured to connect the server 400 to a network, and the input output (I/O) interface 428. The Server 400 may operate based on an operating system stored in memory 422, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 422 comprising instructions, executable by the processor of the server 400 to perform the above-described method is also provided. The storage medium may be a computer-readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, which includes instructions executable by a processor of the server 400 to perform the above-described method.

It should be noted that the descriptions of the above-mentioned apparatus, the electronic device, the computer-readable storage medium, the computer program product, and the like according to the method embodiments may also include other embodiments, and specific implementations may refer to the descriptions of the related method embodiments, which are not described in detail herein.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of data processing, comprising:

2. The method of claim 1, further comprising;

3. The method of claim 1, further comprising:

receiving a saving instruction of the application to the second object;

storing the plurality of second objects to an application database.

4. The method of claim 3, wherein the manner in which the cache stores the second object comprises:

setting a version identification corresponding to the second object;

5. The method of claim 3, wherein storing the plurality of second objects to an application database comprises:

6. The method of claim 1, wherein before said receiving a read data request from an application, determining a first object of the read data request, further comprising:

7. The method of any one of claims 1 to 6, further comprising:

8. A data processing apparatus, comprising:

9. The apparatus of claim 8, further comprising:

10. The apparatus of claim 8, further comprising:

11. The apparatus of claim 10, wherein the cache stores the second object in a manner comprising:

setting a version identification corresponding to the second object;

12. The apparatus of claim 10, wherein the storage module comprises:

13. The apparatus of claim 8, further comprising:

14. The apparatus of any one of claims 8 to 13, further comprising:

15. A server, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement a method of data processing as claimed in any one of claims 1 to 7.

16. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of a server, enable the server to perform a method of data processing according to any one of claims 1 to 7.

17. A computer program product comprising instructions which, when executed by a processor of a server, enable the server to carry out a method of data processing according to any one of claims 1 to 7.