CN114064523A - A data processing method, apparatus, device and readable storage medium - Google Patents
A data processing method, apparatus, device and readable storage medium Download PDFInfo
- Publication number
- CN114064523A CN114064523A CN202111376588.7A CN202111376588A CN114064523A CN 114064523 A CN114064523 A CN 114064523A CN 202111376588 A CN202111376588 A CN 202111376588A CN 114064523 A CN114064523 A CN 114064523A
- Authority
- CN
- China
- Prior art keywords
- request
- data
- service
- annotation
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0877—Cache access modes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data processing method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute; if so, starting a service corresponding to the request, and calculating the service to obtain request data; and caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute. According to the technical scheme, when the received interface corresponding to the request is annotated, the service corresponding to the request is started and service calculation is carried out to obtain the latest data, the latest data is cached based on the data cache attribute in the annotation, the request data is output according to the data return attribute in the annotation, and the dynamic caching and obtaining of the service data are realized based on the annotation under the condition that the original code and logic are not changed.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.
Background
Data caching is widely used in data processing because it can greatly increase data access speed.
At present, data caching is generally performed on the basis of a Remote Dictionary service (Remote Dictionary Server), a redis expiration time is configured on the basis of a code during data caching, data cached in the redis remains unchanged within the expiration time, for example, the expiration time is configured to 1 day, then data presented within 24 hours are the same, and this may cause that the latest data cannot be cached in the redis, and may cause that a user cannot acquire the latest data.
In summary, how to implement caching and obtaining of the latest data is a technical problem to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of the above, an object of the present application is to provide a data processing method, apparatus, device and readable storage medium, which are used to implement caching and obtaining of latest data.
In order to achieve the above purpose, the present application provides the following technical solutions:
a method of data processing, comprising:
acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute;
if so, starting a service corresponding to the request, and calculating the service to obtain request data;
caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
Preferably, after the calculating the service to obtain the request data, the method further includes:
storing the request data and the user information corresponding to the request, and judging whether the extended parameters in the annotation are empty or not;
if not, the stored request data and the user information corresponding to the request are sent to a message queue, and when the expander monitors that the message queue receives the information, the expander calls a corresponding service implementation method according to the expansion parameters to execute so as to expand the service function of the interface corresponding to the request, and pushes the request data by adopting the expanded service function according to the user information.
Preferably, before initiating the service corresponding to the request, the method further includes:
obtaining an available thread from a thread manager;
wherein the thread manager acquiring the available threads comprises:
judging whether an available thread exists in a thread pool, if so, acquiring the available thread from the available thread in the thread pool, if not, judging whether thread resources in the thread pool are sufficient, if so, establishing the available thread, and if not, waiting for the occupied thread to finish execution and then acquiring the available thread.
Preferably, the calculating the service includes:
and acquiring a corresponding lock from a concurrency manager according to the lock type in the annotation, and calculating the service according to the lock and the locking time in the annotation.
Preferably, before initiating the service corresponding to the request, the method further includes:
and generating a key value according to the request parameters contained in the request, inquiring a corresponding value from the redis according to the key value, and outputting the value according to the data return attribute in the annotation if the value is not null.
Preferably, the determining whether the interface corresponding to the request is annotated includes:
acquiring an identifier of an interface corresponding to the request, and judging whether a record corresponding to the identifier exists in a database table according to the identifier; the records in the database table comprise interface identifiers and corresponding annotations, and the records in the database table are formed by collecting all interfaces with annotated functions by an annotation scanning component in an annotation manager, acquiring the interface identifiers and the corresponding annotations of all the interfaces and storing the interface identifiers and the corresponding annotations in the database table;
and if the database table has a record corresponding to the identifier, determining that the interface corresponding to the request is annotated.
A data processing apparatus comprising:
the acquisition module is used for acquiring a request and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute;
the computing module is used for starting a service corresponding to the request and computing the service to obtain request data if the interface corresponding to the request is annotated;
and the caching and outputting module is used for caching the request data according to the data caching attribute and outputting the request data according to the data return attribute.
Preferably, the method further comprises the following steps:
the storage module is used for storing the request data and the user information corresponding to the request after calculating the service to obtain the request data and judging whether the extended parameters in the annotation are empty or not;
and the sending module is used for sending the stored request data and the user information corresponding to the request to a message queue if the extension parameter in the annotation is not empty, calling a corresponding service implementation method according to the extension parameter to execute when the extender monitors that the message queue receives the information, so as to expand the service function of the interface corresponding to the request, and pushing the request data by adopting the expanded service function according to the user information.
A data processing apparatus comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method as claimed in any one of the above when executing said computer program.
A readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the data processing method according to any one of the preceding claims.
The application provides a data processing method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation may include a data cache attribute and a data return attribute; if so, starting a service corresponding to the request, and calculating the service to obtain request data; and caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
According to the technical scheme, when the received interface corresponding to the request is annotated, the service corresponding to the request is started and service calculation is carried out to obtain the request data, namely the latest data is obtained, then the obtained request data is cached based on the data cache attribute in the annotation to realize caching of the latest data, so that the cached data can be dynamically updated, meanwhile, the request data is output according to the data return attribute in the annotation, so that a user can obtain the latest request data, namely, the service data is dynamically cached and obtained under the condition that original codes and logic are not changed based on the annotation, and the real-time performance, effectiveness and accuracy of data caching and data obtaining are ensured.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
FIG. 2 is a block diagram of a data processing system according to an embodiment of the present application;
fig. 3 is a flowchart of another data processing method provided in the embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
At present, a client requires fast access to data of a deployed platform, and for this reason, a data presentation mode is based on redis for data caching so as to achieve fast access, but data accessed for the first time is always presented in the data access, whether data is updated or not is based on redis expiration time configured in a code, if the expiration time is configured for 1 day, data presented in 24 hours are the same, and the expectation that the client obtains the latest data is not achieved.
Therefore, the application provides a data processing method, a data processing device, a data processing apparatus and a readable storage medium, which are used for caching and acquiring the latest data.
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1 and fig. 2, where fig. 1 shows a flowchart of a data processing method provided in an embodiment of the present application, and fig. 2 shows a schematic structural diagram of a data processing system provided in an embodiment of the present application, a data processing method provided in an embodiment of the present application may include:
s11: acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute; if yes, go to step S12.
It should be noted that the execution main body of the data processing method provided by the present application may be an annotation interceptor, which is specifically a plug-in the data processing system (the plug-in is a pluggable service function device, has extremely strong service extensibility, does not need to be coupled with the original service, and does not sense the extended service function), and the annotation interceptor is a core processor of the data processing method, and controls the whole execution flow.
When a request comes, it can be intercepted by the annotation interceptor, specifically, the annotation interceptor will intercept all the requests of nginx (which is a high-performance HTTP and reverse proxy web server), that is, the annotation interceptor will obtain the request. Then, the annotation interceptor may determine the interface corresponding to the request, and determine whether the interface corresponding to the request is annotated (the annotation identifier may be a ScreenCache), specifically, may obtain a unique identifier corresponding to the interface, and determine whether the interface corresponding to the request is annotated based on the unique identifier corresponding to the interface, if the interface is not annotated, the request will be passed, and if the interface is annotated, step S12 is executed.
Where Annotations (Annotation), also called Metadata (Metadata), are a new feature of Java5, JDK5 introduces Metadata that can easily call Annotations. The annotation is in the same level with class, interface and enumeration, and can be applied to declarations of packages, types, construction methods, member variables, parameters and local variables, and is used for explanation and annotation of the elements.
The annotation in the present application may include a data cache attribute and a data return attribute, where the data cache attribute may specifically include a cache data type (cacheDataType) and a cache time (cacheTime), and the data return attribute may specifically include a cache return data type (responeDataType). The cache data type represents a data type cached by a corresponding interface, and for example, if the cache is read, five data types are supported: string, hash, list, set and zset, and only one can be selected, the default value is string; the cache time represents the failure time from the data to be cached to the redis, the unit is second, the default value is-1 (representing infinite time), the cache value is 0 to represent that no cache is performed, the cache value is-2 to perform real-time cache, and the value is of a digital type; the cache return data type represents the data type returned by the corresponding interface, and the self return type of the interface is defaulted according to the interface. In addition, the annotation ScreenCache may further include a function name (functional name), where the function name is a corresponding interface name, and the default value is a method name.
S12: and starting the service corresponding to the request, and calculating the service to obtain the request data.
If it is determined in step S11 that the interface corresponding to the request acts on the annotation, a corresponding service is found in the service processor, and the service corresponding to the request is started. Specifically, refer to fig. 3, which shows a flowchart of another data processing method provided in the embodiment of the present application, where a service processor may also be regarded as a plug-in, which is a request master service executor and acquires a corresponding service according to a request. And on the basis of acquiring the corresponding service, calculating the service to obtain the request data, namely obtaining the latest data.
S13: and caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
After the request data is obtained, the request data may be cached according to the data cache attribute in the annotation, specifically, the request data may be used as a value, a key value may be generated according to a request parameter included in the request, and the key value, the value, cacheDataType, cacheTime may be sent to redis, so as to implement caching of the obtained latest data according to the data cache attribute. Meanwhile, the obtained request data can be analyzed according to the data return attribute in the annotation, and the object obtained through analysis according to the data return attribute is output to nginx so as to be output to the user corresponding to the request through the nginx, so that the user can obtain the required latest data.
Through the process, the dynamic caching of the business data can be realized based on the annotation and the configuration of the pluggable plug-in, and the user can acquire the latest data, so that the effectiveness and the reliability of the data are improved, and the user experience degree is improved.
According to the technical scheme, when the received interface corresponding to the request is annotated, the service corresponding to the request is started and service calculation is carried out to obtain the request data, namely the latest data is obtained, then the obtained request data is cached based on the data cache attribute in the annotation to realize caching of the latest data, so that the cached data can be dynamically updated, meanwhile, the request data is output according to the data return attribute in the annotation, so that a user can obtain the latest request data, namely, the service data is dynamically cached and obtained under the condition that original codes and logic are not changed based on the annotation, and the real-time performance, effectiveness and accuracy of data caching and data obtaining are ensured.
The data processing method provided in the embodiment of the present application, after calculating the service to obtain the request data, may further include:
storing the request data and the user information corresponding to the request, and judging whether the extension parameters in the annotation are empty or not;
if not, the stored request data and the user information corresponding to the request are sent to a message queue, and when the expander monitors that the message queue receives the information, the expander calls a corresponding service implementation method according to the expansion parameters to execute so as to expand the service function of the interface corresponding to the request, and pushes the request data by adopting the expanded service function according to the user information.
In the present application, after the service is calculated to obtain the request data, all the parameters (including the request parameters, etc.), the request data, and the user information corresponding to the request may also be stored, and specifically, may be stored in a map (which is a key-value pair set). Then, it may be determined whether the extension parameter in the annotation is empty, that is, the annotation may further include an extension parameter in addition to the data cache attribute and the data return attribute mentioned above, where the extension parameter includes a service extension type (extended), which is a service extension function and may be selected more, separated by commas, and an optional value is defined in an extender, for example, the extender currently has a short message reminder (SmsReminder), a nail reminder (NailReminder), a data push (DataPush), and the like.
If the extension parameter in the annotation is not empty, the stored request data and the user information corresponding to the request can be sent to a message queue, and the message queue is monitored by an extender in the data processing system. The expander can also be regarded as a plug-in, and the setting is to expand the interface function and realize the function expected by the client quickly and without perception. When the extender monitors that the information is received in the message queue, the extender calls a corresponding service implementation method to execute according to the extension parameters in the annotation so as to perform service function extension on the interface corresponding to the request. The extender has a parent interface TaskService, and a method void task (Map < String, Object >) is arranged inside the extender. And when the expander monitors that the message queue has information receiving, the expander takes the corresponding service implementation method for execution according to the service expansion type extend. Wherein, the naming of the implementation classes is specified: service name + TaskService:
class names are implemented as follows: SmsReminderTaskService, NailReminderTaskService and DataPushTaskService, the extender finds the corresponding implementation type according to the extended, and then directly calls the implementation type task () method, for example, the extended is "SmsReminderTaskService", the task () method in the SmsReminderTaskService is called, and a plurality of values circularly find the implementation class and call the implementation class.
For example:
parent interface
public interface TaskService{
void void task(Map<String,Object>)；
}
Short message reminding extension implementation class
public class SmsReminderTaskService implement TaskService{
void void task(Map<String,Object>){
...
Specific service implementation
}
}
Nail reminding extension implementation class
public class NailReminderTaskService implement TaskService{
void void task(Map<String,Object>){
...
Specific service implementation
}
}
Data push extension implementation class
public class DataPushTaskService implement TaskService{
void void task(Map<String,Object>map){
...
Specific service implementation
}
}
After the expander expands the service of the interface corresponding to the request in the above manner, the expander can push the request data by adopting the service function obtained by expansion according to the user information in the information received by the message queue.
Note that if the extension parameter in the annotation contains multiple values, a loop call execution will be performed.
Through the process, the expansion of the interface service function can be realized based on the annotation and the plug-in, and the service expansion is realized without coupling and sensing with the original service by means of the message queue, so that the interface can better push data based on the expanded service function, the user requirement is met, and the user experience is improved.
Before starting a service corresponding to a request, a data processing method provided in an embodiment of the present application may further include:
obtaining an available thread from a thread manager;
the acquiring of the available thread by the thread manager may include:
judging whether an available thread exists in the thread pool, if so, acquiring the available thread from the available threads in the thread pool, if not, judging whether thread resources in the thread pool are sufficient, if so, creating the available thread, and if not, waiting for the occupied thread to finish execution and then acquiring the available thread.
In the application, before the service corresponding to the request is started, an available thread can be acquired from the thread manager, so that the business is started and business calculation is performed on the enterprise by using the acquired available thread.
The thread manager may also be considered as a plug-in the data processing system, and is configured according to the service being processed. When a request comes, the thread manager firstly judges whether an available thread exists in the thread pool, if the available thread exists in the thread pool, the thread can be started, namely the thread manager can obtain an available thread from the available threads in the thread pool and send the available thread to the annotation interceptor. If no available thread exists in the thread pool, the thread manager can judge whether thread resources in the thread pool are sufficient, if the thread resources in the thread pool are sufficient, the thread manager creates a new thread for the annotation interceptor, if the thread resources in the thread pool are insufficient, the thread manager waits for the occupied thread to be executed and then acquires the available thread, and the acquired available thread is sent to the annotation interceptor for use.
Through the process, the waste of server resources can be avoided, and the waste of time for starting and releasing the threads is avoided, so that the efficiency is improved. In addition, too many threads can bring burden to the server, and the process can be controllable and manageable, and adaptation is performed according to server resources.
The data processing method provided by the embodiment of the application, which is used for calculating the service, may include:
and acquiring a corresponding lock from the concurrency manager according to the lock type in the annotation, and calculating the service according to the lock and the locking time in the annotation.
In this application, the annotation may further include a business process attribute, which may specifically include a lock type (lockType) and a lock time (lockTime), where the lock type supports: synchronous locks (synchronization), asynchronous locks (asynchronous), locks without (noLock), distributed locks (distributedLock), and only one of them can be selected, and the default value is synchronous lock; the locking time is the locking time of the lock type service, and has a default value of 5 seconds and a unit of second, and is of a digital type.
On the basis that the annotation contains a lock type (lockType) and a locking time (lockTime), the annotation interceptor can acquire the lock type and the locking time according to the interface corresponding to the request and acquire the corresponding lock from a concurrency manager contained in the data processing system according to the lock type. The concurrency manager is a controller for managing lock resources, acquires a lock corresponding to the lock type according to the lock type contained in the annotation, and returns the acquired lock to the annotation interceptor. Among other things, better business calculations may be facilitated by including lock types and lock times in the annotations.
After receiving the lock, the annotation interceptor will calculate the traffic according to the lock time in the lock and annotation. Specifically, if the request is a synchronous lock, the annotation interceptor performs synchronous processing when performing business computation, and other requests may enter other requests after the request is executed, that is, other requests need to be queued for execution. If asynchronous locking is achieved, the locking time is effective, if the service execution is finished, the lock is automatically released, and if the service execution exceeds the locking time, the lock is also released, so that the phenomenon that the service execution always occupies lock resources to cause deadlock is avoided. If the distributed lock is the distributed lock, the distributed lock processing is carried out, the lock introduction can solve the problem that the service is the cluster deployment concurrency, the use mode is similar to that of an asynchronous lock, only the asynchronous lock is used in a single machine environment, and the distributed lock is used in a cluster environment. If the service is not locked, the service does not need to be locked, and the service can be carried out naturally.
It should be noted that, on the basis of the above, after the requested data is cached according to the data caching attribute, the lock may be released, so as to avoid occupying the resources for a long time. In addition, if an abnormal condition (such as calculation failure) occurs in the request, the lock is released according to the locking time so as to avoid occupying the resources for a long time.
Combining the foregoing processes, the annotation may specifically include a function name, a cache data type, a cache time, a lock type, a lock time, and a service extension type, so as to better meet the requirements of the user on data caching and data acquisition. For example: annotate the sample: @ screen cache (functionName ═ attack TOP5 ″, cacheDataType ═ string ═ canetime ═ 100, responeDataType ═ java. A complete example of the attack on the TOP5 attackRankTopFive method by the attack control layer AttackController:
@Controllerpublic class AttackController{
@ ScreenCache (funcationname ═ attack TOP5 ═ String ═ CapchDataType ═ String ═ 100, responeDataType ═ java
public String attackRankTopFive(Date startTime,Date endTime,StringregionCode){
..
work (); (ii) a // service handling
..
}
}
Before starting a service corresponding to a request, a data processing method provided in an embodiment of the present application may further include:
and generating a key value according to the request parameters contained in the request, inquiring a corresponding value from the redis according to the key value, and outputting the value according to the data return attribute in the annotation if the value is not null.
In the application, before the annotation interceptor starts the service corresponding to the request, a key value can be generated according to the request parameter contained in the request, then a value corresponding to the key value is inquired from redis according to the key value, if the value is not null, the value is analyzed according to the data return attribute in the annotation, and the analyzed result is output to nginx so as to be output to the user by the nginx.
Through the process, the quick response to the request of the user can be realized, and the data can be returned to the user. Therefore, the data can be returned to the user corresponding to the request twice through the method, the data can be directly and quickly returned once based on the cache of the redis, the latest data is calculated once based on the annotation, and the latest data is returned to the user, so that the user can quickly acquire the data and can acquire the latest data, and the user experience is improved.
The data processing method provided in the embodiment of the present application, which determines whether an interface corresponding to the request is annotated, may include:
acquiring an identifier of an interface corresponding to the request, and judging whether a record corresponding to the identifier exists in a database table according to the identifier; the record in the database table comprises interface identifiers and corresponding annotations, and the record in the database table is formed by collecting all annotated interfaces through an annotation scanning component in an annotation manager, acquiring the interface identifiers and the corresponding annotations of all the interfaces and storing the interface identifiers and the corresponding annotations in the database table;
and if the database table has a record corresponding to the identifier, determining that the interface corresponding to the request is annotated.
In the application, when judging whether the request is annotated, an identifier of the interface corresponding to the request can be specifically obtained, and the identifier is the only identifier of the interface, so that the accuracy of annotation judgment is improved. The identifier may specifically include a class name and a method name. For example: the aforementioned unique identifiers are: attackrankttopfive.
Then, whether a record corresponding to the identifier exists in the database table is judged according to the identifier of the interface acquired by the office, if the record corresponding to the identifier exists in the database table, the annotation effect of the interface corresponding to the request is determined, and convenience and accuracy of annotation effect judgment can be improved through the process.
The records in the data table include interface identifiers and corresponding annotations, and the records in the data table are formed by collecting all interfaces with annotated functions by an annotation scanning component in an annotation manager, acquiring the interface identifiers and the corresponding annotations of all the interfaces and storing the interface identifiers and the corresponding annotations in a database table.
Specifically, the annotation manager is also a plug-in the data processing system, and is composed of an annotation scanning component and an annotation configuration component, where the annotation scanning component collects all interfaces with annotations in the interface control layer (specifically, when the item program is started, the annotation scanning component is triggered to scan all control layer interfaces globally), and records an interface identifier (i.e., the above-mentioned unique identifier) corresponding to each interface with annotations and each attribute included in the corresponding annotations. Then, the interface identifier corresponding to each interface and each attribute included in the corresponding annotation are respectively used as a record (one record corresponds to one interface, and one record includes the interface identifier of the interface and each attribute included in the corresponding annotation) and stored in a database table (specifically, the record may be stored in a t _ screen _ cache _ management table). In order to prevent the program from covering the value of the annotation configuration component every time, during scanning, the annotation scanning component firstly inquires whether a corresponding record exists in a database table once according to an interface identifier of an interface, if so, the record is not processed, and if not, a record is newly added to avoid covering the record in the database table, so that the requirement of a user is met. The annotation configuration management component is used for managing data in a database table, and in order to meet a component set by a user for expecting a service value, each record in the database table can be edited, for example, if the user wants to change the cache time to 20 seconds, the user can directly modify the corresponding cacheTime in the database to 20 seconds, and if an extended service nailing reminder is added, a NailReminder and the like can be directly added behind the corresponding extended field in the database, so that the interface can meet the user requirements.
An embodiment of the present application further provides a data processing apparatus, see fig. 4, which shows a schematic structural diagram of the data processing apparatus provided in the embodiment of the present application, and the data processing apparatus may include:
an obtaining module 41, configured to obtain a request, and determine whether an interface corresponding to the request is annotated; the annotation may include a data cache attribute and a data return attribute;
the calculation module 42 is configured to start a service corresponding to the request and calculate the service to obtain request data if the interface corresponding to the request is annotated;
and a cache and output module 43, configured to cache the request data according to the data cache attribute, and output the request data according to the data return attribute.
The data processing apparatus provided in the embodiment of the present application may further include:
the storage module is used for storing the request data and the user information corresponding to the request after the business is calculated to obtain the request data, and judging whether the extension parameters in the annotation are empty or not;
and the sending module is used for sending the stored request data and the user information corresponding to the request to a message queue if the extension parameter in the annotation is not empty, calling a corresponding service implementation method according to the extension parameter to execute when the extender monitors that the message queue receives the information, so as to expand the service function of the interface corresponding to the request, and pushing the request data by adopting the expanded service function according to the user information.
The data processing apparatus provided in the embodiment of the present application may further include:
the acquisition module is used for acquiring available threads from the thread manager before starting the service corresponding to the request;
the acquiring of the available thread by the thread manager may include:
judging whether an available thread exists in the thread pool, if so, acquiring the available thread from the available threads in the thread pool, if not, judging whether thread resources in the thread pool are sufficient, if so, creating the available thread, and if not, waiting for the occupied thread to finish execution and then acquiring the available thread.
In an embodiment of the data processing apparatus, the calculation module 42 may include:
and the computing unit is used for acquiring the corresponding lock from the concurrency manager according to the lock type in the annotation and computing the service according to the lock and the locking time in the annotation.
The data processing apparatus provided in the embodiment of the present application may further include:
and the query module is used for generating a key value according to the request parameter contained in the request before the service corresponding to the request is started, querying a corresponding value from the redis according to the key value, and outputting the value according to the data return attribute in the annotation if the value is not null.
In an embodiment of the data processing apparatus, the obtaining module 41 may include:
the acquisition unit is used for acquiring the identifier of the interface corresponding to the request and judging whether a record corresponding to the identifier exists in the database table or not according to the identifier; the record in the database table comprises interface identifiers and corresponding annotations, and the record in the database table is formed by collecting all annotated interfaces through an annotation scanning component in an annotation manager, acquiring the interface identifiers and the corresponding annotations of all the interfaces and storing the interface identifiers and the corresponding annotations in the database table;
and the determining unit is used for determining that the interface corresponding to the request is annotated if the database table has a record corresponding to the identifier.
An embodiment of the present application further provides a data processing apparatus, see fig. 5, which shows a schematic structural diagram of the data processing apparatus provided in the embodiment of the present application, and the data processing apparatus may include:
a memory 51 for storing a computer program;
the processor 52, when executing the computer program stored in the memory 51, may implement the following steps:
acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation may include a data cache attribute and a data return attribute; if so, starting a service corresponding to the request, and calculating the service to obtain request data; and caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
An embodiment of the present application further provides a readable storage medium, in which a computer program is stored, and when executed by a processor, the computer program can implement the following steps:
acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation may include a data cache attribute and a data return attribute; if so, starting a service corresponding to the request, and calculating the service to obtain request data; and caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
The readable storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For a description of a relevant part in a data processing apparatus, a device, and a readable storage medium provided by the present application, reference may be made to a detailed description of a corresponding part in a data processing method provided by an embodiment of the present application, which is not described herein again.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data processing method, comprising:
acquiring a request, and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute;
if so, starting a service corresponding to the request, and calculating the service to obtain request data;
caching the request data according to the data caching attribute, and outputting the request data according to the data return attribute.
2. The data processing method of claim 1, wherein after the calculating the service to obtain the request data, further comprising:
storing the request data and the user information corresponding to the request, and judging whether the extended parameters in the annotation are empty or not;
if not, the stored request data and the user information corresponding to the request are sent to a message queue, and when the expander monitors that the message queue receives the information, the expander calls a corresponding service implementation method according to the expansion parameters to execute so as to expand the service function of the interface corresponding to the request, and pushes the request data by adopting the expanded service function according to the user information.
3. The data processing method of claim 1, further comprising, before initiating the service corresponding to the request:
obtaining an available thread from a thread manager;
wherein the thread manager acquiring the available threads comprises:
judging whether an available thread exists in a thread pool, if so, acquiring the available thread from the available thread in the thread pool, if not, judging whether thread resources in the thread pool are sufficient, if so, establishing the available thread, and if not, waiting for the occupied thread to finish execution and then acquiring the available thread.
4. The data processing method of claim 3, wherein computing the traffic comprises:
and acquiring a corresponding lock from a concurrency manager according to the lock type in the annotation, and calculating the service according to the lock and the locking time in the annotation.
5. The data processing method according to any one of claims 1 to 4, further comprising, before initiating the service corresponding to the request:
and generating a key value according to the request parameters contained in the request, inquiring a corresponding value from the redis according to the key value, and outputting the value according to the data return attribute in the annotation if the value is not null.
6. The data processing method of claim 5, wherein determining whether the interface corresponding to the request is annotated comprises:
acquiring an identifier of an interface corresponding to the request, and judging whether a record corresponding to the identifier exists in a database table according to the identifier; the records in the database table comprise interface identifiers and corresponding annotations, and the records in the database table are formed by collecting all interfaces with annotated functions by an annotation scanning component in an annotation manager, acquiring the interface identifiers and the corresponding annotations of all the interfaces and storing the interface identifiers and the corresponding annotations in the database table;
and if the database table has a record corresponding to the identifier, determining that the interface corresponding to the request is annotated.
7. A data processing apparatus, comprising:
the acquisition module is used for acquiring a request and judging whether an interface corresponding to the request is annotated; the annotation comprises a data cache attribute and a data return attribute;
the computing module is used for starting a service corresponding to the request and computing the service to obtain request data if the interface corresponding to the request is annotated;
and the caching and outputting module is used for caching the request data according to the data caching attribute and outputting the request data according to the data return attribute.
8. The data processing apparatus of claim 7, further comprising:
the storage module is used for storing the request data and the user information corresponding to the request after calculating the service to obtain the request data and judging whether the extended parameters in the annotation are empty or not;
and the sending module is used for sending the stored request data and the user information corresponding to the request to a message queue if the extension parameter in the annotation is not empty, calling a corresponding service implementation method according to the extension parameter to execute when the extender monitors that the message queue receives the information, so as to expand the service function of the interface corresponding to the request, and pushing the request data by adopting the expanded service function according to the user information.
9. A data processing apparatus, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method according to any one of claims 1 to 6 when executing the computer program.
10. A readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111376588.7A CN114064523A (en) | 2021-11-19 | 2021-11-19 | A data processing method, apparatus, device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111376588.7A CN114064523A (en) | 2021-11-19 | 2021-11-19 | A data processing method, apparatus, device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114064523A true CN114064523A (en) | 2022-02-18 |
Family
ID=80278520
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111376588.7A Pending CN114064523A (en) | 2021-11-19 | 2021-11-19 | A data processing method, apparatus, device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114064523A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114996036A (en) * | 2022-06-01 | 2022-09-02 | 深圳我家云网络科技有限公司 | Information calling method, system and related device |
| CN115118569A (en) * | 2022-06-29 | 2022-09-27 | 迈普通信技术股份有限公司 | Request processing method and device, network management equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793432A (en) * | 2012-11-01 | 2014-05-14 | 深圳中兴网信科技有限公司 | Method and device for splitting database reading and writing |
| CN109684086A (en) * | 2018-12-14 | 2019-04-26 | 广东亿迅科技有限公司 | A kind of distributed caching automatic loading method and device based on AOP |
| CN110990153A (en) * | 2019-11-27 | 2020-04-10 | 南京甄视智能科技有限公司 | Dubbo interface caching method and device under distributed environment based on SpringCache |
| CN112052263A (en) * | 2020-07-13 | 2020-12-08 | 浙江大搜车软件技术有限公司 | Method, system, computer device, and readable storage medium for requesting instruction processing |
| WO2021164462A1 (en) * | 2020-02-21 | 2021-08-26 | 深圳壹账通智能科技有限公司 | Data encryption method, data decryption method, computer device, and medium |
-
2021
- 2021-11-19 CN CN202111376588.7A patent/CN114064523A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793432A (en) * | 2012-11-01 | 2014-05-14 | 深圳中兴网信科技有限公司 | Method and device for splitting database reading and writing |
| CN109684086A (en) * | 2018-12-14 | 2019-04-26 | 广东亿迅科技有限公司 | A kind of distributed caching automatic loading method and device based on AOP |
| CN110990153A (en) * | 2019-11-27 | 2020-04-10 | 南京甄视智能科技有限公司 | Dubbo interface caching method and device under distributed environment based on SpringCache |
| WO2021164462A1 (en) * | 2020-02-21 | 2021-08-26 | 深圳壹账通智能科技有限公司 | Data encryption method, data decryption method, computer device, and medium |
| CN112052263A (en) * | 2020-07-13 | 2020-12-08 | 浙江大搜车软件技术有限公司 | Method, system, computer device, and readable storage medium for requesting instruction processing |
Non-Patent Citations (2)
| Title |
|---|
| 朱虎平;艾迪;: "一种基于Java注解和反射机制的Excel文件读写操作方法", 九江职业技术学院学报, no. 01, 15 March 2020 (2020-03-15) * |
| 陈军峰: "《JAVA语言程序设计》", 30 June 2019, 国家开放大学出版社, pages: 115 - 119 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114996036A (en) * | 2022-06-01 | 2022-09-02 | 深圳我家云网络科技有限公司 | Information calling method, system and related device |
| CN115118569A (en) * | 2022-06-29 | 2022-09-27 | 迈普通信技术股份有限公司 | Request processing method and device, network management equipment and readable storage medium |
| CN115118569B (en) * | 2022-06-29 | 2024-03-15 | 迈普通信技术股份有限公司 | Request processing method, request processing device, network management equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7483981B2 (en) | Scalable virtual partitioning of resources | |
| US6028605A (en) | Multi-dimensional analysis of objects by manipulating discovered semantic properties | |
| JP4795945B2 (en) | User interface for access control to computer objects | |
| CN109739815B (en) | File processing method, system, device, equipment and storage medium | |
| KR101150146B1 (en) | System and method for managing cached objects using notification bonds | |
| CN112364110A (en) | Metadata management method, device and equipment and computer storage medium | |
| US8572161B2 (en) | Simplifying synchronization of copies of same data used by multiple applications | |
| JP2010134963A (en) | Method for activating service using property attached to document | |
| JP2003150594A (en) | Data warehouse system | |
| CN114064523A (en) | A data processing method, apparatus, device and readable storage medium | |
| JP2005018754A (en) | Sharing computer objects with associations | |
| US7698280B2 (en) | Active cache offline sharing of project files | |
| CN112988777A (en) | Object processing method and device, computer equipment and storage medium | |
| CN114205354B (en) | Event management system, event management method, server, and storage medium | |
| JP2004288067A (en) | Coordination processing device, method and program | |
| JP6242087B2 (en) | Document management server, document management method, computer program | |
| JP5224839B2 (en) | Document management system, document management apparatus, document management method, and program | |
| CN117234467A (en) | Unified method, device, equipment and storage medium for flow node interfaces | |
| US20140280347A1 (en) | Managing Digital Files with Shared Locks | |
| CN117493271B (en) | Container node management device, method, equipment and medium of cloud operating system | |
| JP5018536B2 (en) | Information service search system, patrol collection method, and information service patrol collection program | |
| CN117171419B (en) | Content processing method, device, electronic apparatus, storage medium, and program product | |
| JP5715940B2 (en) | Virtual client provisioning system and method | |
| JPH10240603A (en) | File management device and file transfer method | |
| US20070289016A1 (en) | Bi-modular system and method for detecting and removing harmful files using signature scanning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |