CN116414871A - Service data processing method, device, computer equipment and storage medium - Google Patents
Service data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116414871A CN116414871A CN202310161793.4A CN202310161793A CN116414871A CN 116414871 A CN116414871 A CN 116414871A CN 202310161793 A CN202310161793 A CN 202310161793A CN 116414871 A CN116414871 A CN 116414871A
- Authority
- CN
- China
- Prior art keywords
- service
- identifier
- data
- cache
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 57
- 239000000872 buffer Substances 0.000 claims abstract description 55
- 238000004590 computer program Methods 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000004048 modification Effects 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000007853 buffer solution Substances 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a business data processing method, a business data processing device, computer equipment, a storage medium and a computer program product, and relates to the technical field of big data. The method comprises the following steps: sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier; inquiring a cache system based on the service identifier, and storing the service identifier to the cache system when the cache identifier consistent with the service identifier does not exist in the cache system; and inquiring a physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and carrying out data processing on the first service data corresponding to the service identifier to obtain second service data. The method can improve the efficiency of data processing.
Description
Technical Field
The present invention relates to the field of big data technologies, and in particular, to a service data processing method, apparatus, computer device, storage medium, and computer program product.
Background
With the rapid development of the internet, the data volume of online services is greatly increased. The processing capacity of the system is limited, and when the data blocking occurs due to overlarge service data volume in the same period, the system is easy to crash.
In the conventional technology, buffering of service data to be processed is generally implemented through a message queue. After each service data is processed, the processing state of the service data needs to be fed back to the client, and the message queue records the processed state of the message. When the service data is read next time, only the unprocessed service data is read, thereby improving the service processing efficiency.
However, in practical applications, due to network jitter, abnormal logic processing, and the like, a message queue may appear in which no service data is received and is processed, so that the processing state of the service data is still unprocessed, and further the same service data is repeatedly processed.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a service data processing method, apparatus, computer device, computer readable storage medium, and computer program product that can avoid the same service data from being repeatedly processed.
In a first aspect, the present application provides a service data processing method. The method comprises the following steps:
sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier;
inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system;
and inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, the service state of the first service data is to be processed, and after the data processing is performed on the first service data corresponding to the service identifier to obtain the second service data, the method further includes:
and modifying the service state of the first service data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the method further comprises the following steps:
when the cache system has the cache identification consistent with the service identification, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the method further comprises the following steps:
when the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the first service data comprises a service type; querying the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system, wherein the method comprises the following steps:
inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system;
the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in a cache system; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; inquiring the physical database based on the service identifier, and storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, wherein the method comprises the following steps:
Inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database;
the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in a physical database; wherein the service identification corresponds to a service type.
In a second aspect, the present application further provides a service data processing device. The device comprises:
the service data acquisition module is used for sequentially acquiring the first service data in the buffer queue; wherein, each first service data carries a corresponding service identifier;
the first query module is used for querying the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system;
and the second query module is used for querying the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and carrying out data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, the service state of the first service data is to be processed, and after the second query module, the apparatus further includes:
and the business state first modification module is used for modifying the business state of the first business data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the device further comprises:
and the second business state modifying module is used for modifying the business state of the first business data from to-be-processed to processed when the cache identifier consistent with the business identifier exists in the cache system.
In one embodiment, the service state of the first service data is to be processed; the device further comprises:
and the third modification module of the service state is used for modifying the service state of the first service data from to-be-processed to processed when the buffer identification consistent with the service identification exists in the physical database.
In one embodiment, the first service data comprises a service type; the first query module includes:
the first query sub-module of the business type is used for querying the cache system based on the business type to obtain a plurality of cache identifiers corresponding to the business type in the cache system;
The cache identifier matching sub-module is used for matching the service identifier with a plurality of cache identifiers respectively, and when the cache identifier consistent with the service identifier does not exist, the service identifier is stored into the cache system as a newly added cache identifier; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; the second query module includes:
the second query sub-module of the business type is used for querying the physical database based on the business type to obtain a plurality of database identifiers corresponding to the business type in the physical database;
the database identifier matching sub-module is used for matching the service identifier with a plurality of database identifiers respectively, and storing the service identifier as a newly added database identifier to a physical database when the database identifier consistent with the service identifier does not exist; wherein the service identification corresponds to a service type.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the method of any of the embodiments described above when the processor executes the computer program.
In a fourth aspect, the present application also provides a computer device readable storage medium. The computer device readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of the embodiments described above.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the method according to any of the embodiments described above.
The service data processing method, the device, the computer equipment, the storage medium and the computer program product comprise the steps of firstly, sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier. And then, inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system. And finally, inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data. The processed service identifiers are respectively stored in the cache system and the physical database, and double inquiry is carried out on the cache system and the physical database before data processing, so that the same service data can be prevented from being repeatedly processed, and only unprocessed service data is processed, so that the data processing efficiency can be improved.
Drawings
FIG. 1 is an application environment diagram of a business data processing method in one embodiment;
FIG. 2 is a flow chart of a business data processing method in one embodiment;
FIG. 3 is a flow chart of a business process for pulling first business data from a buffer queue in one embodiment;
FIG. 4 is a flow chart of a query performed on a cache system and a physical database in another embodiment;
FIG. 5 is a block diagram of a business data processing device in one embodiment;
fig. 6 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The service data processing method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the buffer queue 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The server 104 communicates with the buffer queue 102 to enter the context of business data processing. First, the server 104 may sequentially acquire the first service data in the buffer queue 102; wherein, each first service data carries a corresponding service identifier. Server 104 may then query the cache system based on the service identification and store the service identification to the cache system when there is no cache identification in the cache system that matches the service identification. Finally, the server 104 may query the physical database based on the service identifier, store the service identifier to the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and perform data processing on the first service data corresponding to the service identifier to obtain the second service data. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
The business data processing method provided by the embodiment of the application can be applied to a system comprising a buffer queue and a server, and is realized through interaction of the buffer queue and the server.
In one embodiment, as shown in fig. 2, a service data processing method is provided, and the method is described as being applied to a system implementation including a buffer queue and a server, and includes the following steps 202 to 206.
In this embodiment, when a user performs service handling at a terminal, a plurality of corresponding service data may be generated, and in order to avoid blocking caused by limited processing capacity of a server, the plurality of service data may be buffered by a buffer queue. As shown in fig. 3, the service data a, the service data B … …, and the service data N may be buffered in a buffer queue.
In this embodiment, each service data corresponds to a service status, and the service status may include, but is not limited to: pending, processed, etc.
In this embodiment, the server may pull, from the buffer queue, the service state as service data to be processed, as the first service data. Wherein the first service data may include, but is not limited to: service type, service identification. For example, as shown in fig. 3, the buffer queue may include service data a and service data B, and when the service state of the service data a is processed and the service state of the service data B is pending, the server may pull the service data B from the buffer queue as the first service data.
In this embodiment, the service identifier may include, but is not limited to: serial numbers, labels, etc.
In this embodiment, the plurality of service data may be stored in the buffer queue according to a preset arrangement sequence (such as time, service type, etc.).
In this embodiment, the server may sequentially obtain the first service data from the buffer queue according to the first-in first-out principle.
And 204, inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system.
In this embodiment, as shown in fig. 4, the server may perform traversal matching on a plurality of cache identifiers in the cache system based on the service identifier, and when there is no cache identifier consistent with the service identifier in the cache system, that is, the first service data corresponding to the service identifier is not processed by data, the server may store the service identifier in the cache system.
In this embodiment, as shown in fig. 4, when a cache identifier consistent with a service identifier exists in the cache system, that is, the first service data corresponding to the service identifier has been processed by data, in order to avoid that the same first service data is repeatedly processed, the server does not process the data for the first service data corresponding to the service identifier.
In this embodiment, the cache system may be a Redis (REmote DIctionary Server ) cache service system. The cache system is used for storing data by using the memory and can quickly search the data.
And 206, inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data.
In this embodiment, as shown in fig. 4, when a cache identifier consistent with a service identifier does not exist in the cache system, the server may perform traversal matching on a plurality of database identifiers in the physical database based on the service identifier, and when a database identifier consistent with the service identifier does not exist in the physical database, that is, the physical database performs recheck on the service identifier, it is determined that the first service data corresponding to the service identifier is not processed by data, and the server may store the service identifier to the physical database.
In this embodiment, when a cache identifier consistent with the service identifier does not exist in the cache system, the server may store the service identifier to the cache system.
In this embodiment, when there is no database identifier consistent with the service identifier in the physical database, the server may store the service identifier to the physical database.
In another embodiment, when there is no cache identifier consistent with the service identifier in the cache system and there is no database identifier consistent with the service identifier in the physical database, that is, there is no record in the cache system and in the physical database that the first service data corresponding to the service identifier is processed by data, the server may store the service identifier in the physical database and the cache system respectively.
In the service data processing method, first, sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier. And then, inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system. And finally, inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data. The processed service identifiers are respectively stored in the cache system and the physical database, and double inquiry is carried out on the cache system and the physical database before data processing, so that the same service data can be prevented from being repeatedly processed, and only unprocessed service data is processed, so that the data processing efficiency can be improved.
In some embodiments, the service state of the first service data is to be processed, and after the data processing is performed on the first service data corresponding to the service identifier to obtain the second service data, the method may further include: and modifying the service state of the first service data from to-be-processed to processed.
In this embodiment, after the server performs data processing on the first service data to obtain the second service data, the server may modify the service state of the first service data in the buffer queue from to be processed. Further, the server acquires the first service data with the service state to be processed from the buffer queue, and the same first service data can be prevented from being repeatedly pulled from the buffer queue by the server through modifying the service state of the first service data with the data processing completed in the buffer queue.
In some embodiments, the traffic state of the first traffic data is pending; the method may further include: when the cache system has the cache identification consistent with the service identification, the service state of the first service data is modified from to-be-processed to processed.
In this embodiment, when a cache identifier consistent with the service identifier exists in the cache system, that is, the first service data corresponding to the service identifier has been processed by data, in order to avoid that the same first service data is repeatedly processed, the server does not process the first service data corresponding to the service identifier, and modifies the service state corresponding to the first service data in the buffer queue from to-be-processed to processed. By modifying the service state of the first service data in the buffer queue after completing the data processing, the same first service data can be prevented from being repeatedly pulled from the buffer queue by the server.
In some embodiments, the traffic state of the first traffic data is pending; the method may further include: when the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is modified from to-be-processed to processed.
In this embodiment, when a buffer identifier consistent with a service identifier exists in the physical database, that is, the first service data corresponding to the service identifier has been processed by data, in order to avoid that the same first service data is repeatedly processed, the server does not process the first service data corresponding to the service identifier, and modifies the service state corresponding to the first service data in the buffer queue from to-be-processed to processed. By modifying the service state of the first service data in the buffer queue after completing the data processing, the same first service data can be prevented from being repeatedly pulled from the buffer queue by the server. By double inquiry of the buffer system and the physical database, the situation that the buffer queue does not accurately update the service state of the first service data and the same first service data is repeatedly pulled and the data is repeatedly processed due to network jitter or other conditions can be avoided.
In some embodiments, as shown in fig. 4, the first traffic data comprises a traffic type; querying the cache system based on the service identifier, and storing the service identifier to the cache system when the cache identifier consistent with the service identifier does not exist in the cache system, the method may include: inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system; the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in a cache system; wherein the service identification corresponds to a service type.
In this embodiment, the server may query the cache system based on the service type of the first service data to obtain a plurality of cache identifiers corresponding to the service type in the cache system, and screen the plurality of cache identifiers in the cache system through the service type, so that the number of data traversed in subsequent queries may be reduced, and the query efficiency of the cache system may be improved.
In this embodiment, after obtaining a plurality of cache identifiers corresponding to a service type in the cache system, the server may match the service identifier with the plurality of cache identifiers corresponding to the service type, and when there is no cache identifier consistent with the service identifier, store the service identifier as a newly added cache identifier to the cache system.
In some embodiments, as shown in fig. 4, the first traffic data comprises a traffic type; the physical database is queried based on the service identifier, and when the database identifier consistent with the service identifier does not exist in the physical database, the service identifier is stored in the physical database, which may include: inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database; the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in a physical database; wherein the service identification corresponds to a service type.
In this embodiment, the server may query the physical database based on the service type of the first service data to obtain a plurality of database identifiers corresponding to the service type in the physical database, and screen the plurality of database identifiers in the physical database through the service type, so that the number of data traversed in subsequent queries may be reduced, and the query efficiency of the physical database may be improved.
In this embodiment, after obtaining multiple database identifiers corresponding to service types in the physical database, the server may match the service identifier with the multiple database identifiers corresponding to the service types, and when there is no database identifier consistent with the service identifier, store the service identifier as a newly added database identifier to the physical database.
In this embodiment, when the server queries that the physical database has the database identifier consistent with the service identifier, but the cache system does not have the cache identifier consistent with the service identifier (i.e., the first service data corresponding to the service identifier has been subjected to data processing, but the cache system has no record in which the first service data is subjected to data processing, and the physical database has the record in which the first service data is subjected to data processing), the server may store the service identifier as a newly added database identifier to the physical database, and store the service identifier as a newly added cache identifier to the cache system.
In another embodiment, when a cache identifier consistent with the service identifier exists in the cache system, the server may also query the physical database based on the service identifier. By double inquiry of the buffer system and the physical database, the situation that the buffer queue does not accurately update the service state of the first service data and the same first service data is repeatedly pulled and the data is repeatedly processed due to network jitter or other conditions can be avoided.
In some embodiments, when a user performs service handling at a terminal, a plurality of corresponding service data may be generated, and in order to avoid blocking caused by limited processing capacity of a server, the plurality of service data may be cached in a buffer queue to realize data buffering.
In this embodiment, when the same user initiates one or more requests for the same operation, service data is correspondingly generated. For example, when the same user makes two requests for operation 1 in succession and then makes one request for operation 2, the service data 11 corresponding to the first request for operation 1, the service data 12 corresponding to the second request for operation 1, and the service data 21 corresponding to the first request for operation 2 will be generated.
In this embodiment, the service data corresponds to a service identifier, which may be a serial number. When the same user continuously initiates two requests to the operation 1 and generates the service data 11 corresponding to the first request of the operation 1 and the service data 12 corresponding to the second request of the operation 1, the service identifications corresponding to the service data 11 and the service data 12 are different.
In the embodiment, summarizing, a server sequentially acquires a plurality of service data with unprocessed service states in a buffer queue as first service data; wherein, each first service data carries a corresponding service identifier. When the service data 11, the service data 12 and the service data 21 are in the buffer queue at one time in the same direction as the time flow direction, and the service states of the service data 11, the service data 12 and the service data 21 are all unprocessed, the server may sequentially acquire the service data 11, the service data 12 and the service data 21 in the buffer queue as the first service data.
In this embodiment, the server may query the cache system based on the service identifier of the first service data, and store the service identifier to the cache system when there is no cache identifier in the cache system that is consistent with the service identifier; when a cache identifier consistent with the service identifier exists in the cache system, deleting (discarding processing) the service data corresponding to the service identifier from the first service data, and modifying the service state of the first service data in the buffer queue to be processed.
In this embodiment, after the server queries the cache system based on the service identifier of the first service data, the server may query the physical database based on the service identifier. For example, when the first service data 111, the first service data 112, and the first service data 211 exist, a cache identifier consistent with the service identifier of the first service data 111 exists in the cache system, and a cache identifier consistent with the service identifiers of the first service data 112 and the first service data 211 does not exist, the service identifiers of the first service data 112 and the first service data 211 are stored as new cache identifiers in the cache system, the first service data 112 and the first service data 211 are subjected to the next processing (the physical database is queried based on the service identifiers), and the first service data 111 is discarded (deleted).
In this embodiment, when a database identifier corresponding to a service identifier exists in the physical database, the service data corresponding to the service identifier is deleted from the first service data (discarding process), and the service state of the first service data in the buffer queue is modified to be processed. When the database identifier consistent with the service identifier does not exist in the physical database, the service identifier is stored in the physical database, the first service data corresponding to the service identifier is subjected to data processing to obtain second service data, and the service state of the first service data in the buffer queue is modified to be processed.
According to the method and the device, the processed service identifiers are respectively stored in the cache system and the physical database, and the cache system and the physical database are subjected to double inquiry before data processing, so that the same service data can be prevented from being repeatedly processed, only unprocessed service data are subjected to data processing, and the data processing efficiency can be improved.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a service data processing device for implementing the service data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of one or more service data processing devices provided below may refer to the limitation of the service data processing method in the above description, which is not repeated here.
In one embodiment, as shown in fig. 5, there is provided a service data processing apparatus, including: a service data acquisition module 502, a first query module 504, and a second query module 506, wherein:
a service data obtaining module 502, configured to sequentially obtain first service data in the buffer queue; wherein, each first service data carries a corresponding service identifier.
The first query module 504 is configured to query the cache system based on the service identifier, and store the service identifier to the cache system when the cache identifier consistent with the service identifier does not exist in the cache system.
And the second query module 506 is configured to query the physical database based on the service identifier, store the service identifier to the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and perform data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, after the second query module 506, the apparatus may further include:
and the business state first modification module is used for modifying the business state of the first business data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the apparatus may further include:
and the second business state modifying module is used for modifying the business state of the first business data from to-be-processed to processed when the cache identifier consistent with the business identifier exists in the cache system.
In one embodiment, the service state of the first service data is to be processed; the apparatus may further include:
and the third modification module of the service state is used for modifying the service state of the first service data from to-be-processed to processed when the buffer identification consistent with the service identification exists in the physical database.
In one embodiment, the first service data comprises a service type; the first query module 504 may include:
and the service type first query sub-module is used for querying the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system.
The cache identifier matching sub-module is used for matching the service identifier with a plurality of cache identifiers respectively, and when the cache identifier consistent with the service identifier does not exist, the service identifier is stored into the cache system as a newly added cache identifier; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; the second query module 506 may include:
and the second query sub-module of the service type is used for querying the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database.
The database identifier matching sub-module is used for matching the service identifier with a plurality of database identifiers respectively, and storing the service identifier as a newly added database identifier to a physical database when the database identifier consistent with the service identifier does not exist; wherein the service identification corresponds to a service type.
The various modules in the service data processing device described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data such as service identification, service state and the like. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a business data processing method.
It will be appreciated by those skilled in the art that the structure shown in fig. 6 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of: sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier; inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system; and inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, the service state of the first service data is to be processed, and when executing the computer program, the processor performs data processing on the first service data corresponding to the service identifier, and after obtaining the second service data, the following steps may be further implemented: and modifying the service state of the first service data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the processor may also implement the following steps when executing the computer program: when the cache system has the cache identification consistent with the service identification, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the processor may also implement the following steps when executing the computer program: when the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the first service data comprises a service type; the processor performs query on the cache system based on the service identifier when executing the computer program, and stores the service identifier to the cache system when the cache identifier consistent with the service identifier does not exist in the cache system, which may include: inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system; the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in a cache system; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; the processor performs a query of the physical database based on the service identifier when executing the computer program, and stores the service identifier to the physical database when there is no database identifier consistent with the service identifier in the physical database, which may include: inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database; the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in a physical database; wherein the service identification corresponds to a service type.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of: sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier; inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system; and inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, the service state of the first service data is to be processed; when the computer program is executed by the processor, the data processing is performed on the first service data corresponding to the service identifier, and after the second service data is obtained, the following steps can be further implemented: and modifying the service state of the first service data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the computer program may further implement the following steps when executed by a processor: when the cache system has the cache identification consistent with the service identification, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the computer program may further implement the following steps when executed by a processor: when the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the first service data comprises a service type; the computer program, when executed by the processor, performs querying on the cache system based on the service identifier, and when there is no cache identifier in the cache system, stores the service identifier into the cache system, which may include: inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system; the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in a cache system; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; the computer program, when executed by the processor, performs querying the physical database based on the service identifier, and when the physical database does not have a database identifier consistent with the service identifier, storing the service identifier into the physical database, may include: inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database; the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in a physical database; wherein the service identification corresponds to a service type.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of: sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier; inquiring the cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system; and inquiring the physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and performing data processing on the first service data corresponding to the service identifier to obtain second service data.
In one embodiment, the service state of the first service data is to be processed; when the computer program is executed by the processor, the data processing is performed on the first service data corresponding to the service identifier, and after the second service data is obtained, the following steps can be further implemented: and modifying the service state of the first service data from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the computer program may further implement the following steps when executed by a processor: when the cache system has the cache identification consistent with the service identification, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the service state of the first service data is to be processed; the computer program may further implement the following steps when executed by a processor: when the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is modified from to-be-processed to processed.
In one embodiment, the first service data comprises a service type; the computer program, when executed by the processor, performs querying on the cache system based on the service identifier, and when there is no cache identifier in the cache system, stores the service identifier into the cache system, which may include: inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system; the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in a cache system; wherein the service identification corresponds to a service type.
In one embodiment, the first service data comprises a service type; the computer program, when executed by the processor, performs querying the physical database based on the service identifier, and when the physical database does not have a database identifier consistent with the service identifier, storing the service identifier into the physical database, may include: inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database; the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in a physical database; wherein the service identification corresponds to a service type.
It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (15)
1. A method for processing service data, the method comprising:
sequentially acquiring first service data in a buffer queue; wherein, each first service data carries a corresponding service identifier;
inquiring a cache system based on the service identifier, and storing the service identifier to the cache system when the cache identifier consistent with the service identifier does not exist in the cache system;
And inquiring a physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and carrying out data processing on the first service data corresponding to the service identifier to obtain second service data.
2. The method of claim 1, wherein the service state of the first service data is to be processed, and the data processing is performed on the first service data corresponding to the service identifier to obtain second service data, and further comprising:
and modifying the service state of the first service data from to-be-processed to processed.
3. The method of claim 1, wherein the traffic state of the first traffic data is pending; the method further comprises the steps of:
when the cache system has the cache identifier consistent with the service identifier, the service state of the first service data is modified from to-be-processed to processed.
4. The method of claim 1, wherein the traffic state of the first traffic data is pending; the method further comprises the steps of:
When the buffer memory identification consistent with the service identification exists in the physical database, the service state of the first service data is changed from to be processed.
5. The method of claim 1, wherein the first service data comprises a service type; the query for the cache system based on the service identifier, when the cache identifier consistent with the service identifier does not exist in the cache system, the service identifier is stored in the cache system, including:
inquiring the cache system based on the service type to obtain a plurality of cache identifiers corresponding to the service type in the cache system;
the service identification is respectively matched with a plurality of cache identifications, and when the cache identifications consistent with the service identification do not exist, the service identification is used as a newly added cache identification to be stored in the cache system; wherein the service identifier corresponds to the service type.
6. The method of claim 1, wherein the first service data comprises a service type; the query of the physical database based on the service identifier, when the database identifier consistent with the service identifier does not exist in the physical database, the service identifier is stored in the physical database, including:
Inquiring the physical database based on the service type to obtain a plurality of database identifiers corresponding to the service type in the physical database;
the service identification is respectively matched with a plurality of database identifications, and when the database identifications consistent with the service identification do not exist, the service identification is used as a newly added database identification to be stored in the physical database; wherein the service identifier corresponds to the service type.
7. A traffic data processing apparatus, the apparatus comprising:
the service data acquisition module is used for sequentially acquiring the first service data in the buffer queue; wherein, each first service data carries a corresponding service identifier;
the first query module is used for querying a cache system based on the service identifier, and storing the service identifier into the cache system when the cache identifier consistent with the service identifier does not exist in the cache system;
and the second query module is used for querying a physical database based on the service identifier, storing the service identifier into the physical database when the database identifier consistent with the service identifier does not exist in the physical database, and carrying out data processing on the first service data corresponding to the service identifier to obtain second service data.
8. The apparatus of claim 7, wherein the traffic state of the first traffic data is pending, and wherein the second query module is followed by the apparatus further comprising:
and the business state first modification module is used for modifying the business state of the first business data from to-be-processed to processed.
9. The apparatus of claim 7, wherein the traffic state of the first traffic data is pending; the apparatus further comprises:
and the second business state modifying module is used for modifying the business state of the first business data from to-be-processed to processed when the cache identifier consistent with the business identifier exists in the cache system.
10. The apparatus of claim 7, wherein the traffic state of the first traffic data is pending; the apparatus further comprises:
and the third modification module of the service state is used for modifying the service state of the first service data from to-be-processed to processed when the buffer identification consistent with the service identification exists in the physical database.
11. The apparatus of claim 7, wherein the first traffic data comprises a traffic type; the first query module includes:
A service type first query sub-module, configured to query the cache system based on the service type, to obtain a plurality of cache identifiers corresponding to the service type in the cache system;
the cache identifier matching sub-module is used for respectively matching the service identifier with a plurality of cache identifiers, and storing the service identifier as a newly added cache identifier to the cache system when the cache identifier consistent with the service identifier does not exist; wherein the service identifier corresponds to the service type.
12. The apparatus of claim 7, wherein the first traffic data comprises a traffic type; the second query module includes:
a second query sub-module of service type, configured to query the physical database based on the service type, to obtain multiple database identifiers corresponding to the service type in the physical database;
a database identifier matching sub-module, configured to match the service identifier with a plurality of database identifiers, and store the service identifier as a newly added database identifier to the physical database when the database identifier consistent with the service identifier does not exist; wherein the service identifier corresponds to the service type.
13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
15. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310161793.4A CN116414871A (en) | 2023-02-23 | 2023-02-23 | Service data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310161793.4A CN116414871A (en) | 2023-02-23 | 2023-02-23 | Service data processing method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116414871A true CN116414871A (en) | 2023-07-11 |
Family
ID=87053980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310161793.4A Pending CN116414871A (en) | 2023-02-23 | 2023-02-23 | Service data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116414871A (en) |
-
2023
- 2023-02-23 CN CN202310161793.4A patent/CN116414871A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177302B (en) | Service bill processing method, device, computer equipment and storage medium | |
| CN110941598A (en) | Data deduplication method, device, terminal and storage medium | |
| CN106687955B (en) | Simplifying invocation of an import procedure to transfer data from a data source to a data target | |
| CN111309720A (en) | Time sequence data storage method, time sequence data reading method, time sequence data storage device, time sequence data reading device, electronic equipment and storage medium | |
| CN117033424A (en) | Query optimization method and device for slow SQL (structured query language) statement and computer equipment | |
| WO2020192663A1 (en) | Data management method and related device | |
| CN117151045A (en) | Report processing method and device based on block chain and computer equipment | |
| US11500889B1 (en) | Dynamic script generation for distributed query execution and aggregation | |
| CN116414871A (en) | Service data processing method, device, computer equipment and storage medium | |
| CN115809311A (en) | Data processing method and device of knowledge graph and computer equipment | |
| CN114205424B (en) | Bill file decompression method, device, computer equipment and storage medium | |
| CN116009985A (en) | Interface calling method, device, computer equipment and storage medium | |
| CN116932779B (en) | Knowledge graph data processing method and device | |
| CN116055589B (en) | Data management method and device and computer equipment | |
| US11954068B2 (en) | Data indexing for distributed query execution and aggregation | |
| CN115455031B (en) | Data query method, device, storage medium and equipment of Doris | |
| US11360983B2 (en) | Techniques for processing of data in motion | |
| CN116595033A (en) | Data storage method, device, computer equipment and storage medium | |
| CN117076476A (en) | Object information processing method, device, computer equipment and storage medium | |
| CN117076567A (en) | Data synchronization method, device, computer equipment and storage medium | |
| CN106227791B (en) | Data storage architecture | |
| CN117194729A (en) | Power data storage method, apparatus, device, storage medium, and program product | |
| CN115795134A (en) | Data acquisition method and device, computer equipment and storage medium | |
| CN117234562A (en) | Configuration parameter updating method and device and computer equipment | |
| CN117082076A (en) | Batch message processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |