CN117216114A - Data stream association method, device, equipment and storage medium thereof - Google Patents
Data stream association method, device, equipment and storage medium thereof Download PDFInfo
- Publication number
- CN117216114A CN117216114A CN202311348425.7A CN202311348425A CN117216114A CN 117216114 A CN117216114 A CN 117216114A CN 202311348425 A CN202311348425 A CN 202311348425A CN 117216114 A CN117216114 A CN 117216114A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- data stream
- target
- expected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 44
- 238000004458 analytical method Methods 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 17
- 230000004044 response Effects 0.000 claims description 9
- 238000012216 screening Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 16
- 238000013473 artificial intelligence Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000003780 insertion Methods 0.000 description 4
- 230000037431 insertion Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application belongs to the technical field of financial science and technology, is applied to a financial business data multi-data source acquisition scene, and relates to a data stream association method, a device, equipment and a storage medium thereof, wherein the method comprises the steps of pushing multi-source data streams in real time; obtaining an actual data attribute field and a desired data attribute field; acquiring data from a target data resource base in a request mode; constructing a virtual form; the data content in the virtual form is mapped into the actual association result table. In the multi-source data stream association, the query is not required according to the data attribute field, and only the form is acquired, then the processing and the multi-table splicing are required. The association speed of multi-source data flow in the financial business is improved, and the real-time performance of the financial business data processing is ensured. The accuracy of the correlation result of the multi-source data stream is ensured by acquiring the data content corresponding to the expected data attribute field, and when the correlation delay occurs, the virtual form can be directly used for supporting the financial service data, so that the delay of the financial service task is avoided.
Description
Technical Field
The application relates to the technical field of financial science and technology, and is applied to a financial business data multi-data source acquisition scene, in particular to a data stream association method, a data stream association device, data stream association equipment and a storage medium thereof.
Background
With the rapid development of the internet, various industries seek industry breakthrough points by relying on the internet, and in recent years, the financial industry is expanding online business around the internet. As the business volume and the data volume related to the financial industry are large, the aging requirement of data processing is increased along with the continuous increase of the product demand of users, and the data of a plurality of business scenes currently require real-time processing and pushing processing results to the users in real time; real-time processing technology in the big data field has been developed in recent years, but real-time streaming technology is very different in data processing details from traditional offline technology, and has many difficulties in migrating traditional offline processing logic to a data streaming real-time processing scene.
A major problem in migrating the traditional offline processing logic to a data stream real-time processing scene is multi-source data stream real-time association, and the problem that real-time data delay arrives on the multi-source data stream real-time association at present, and finally, the associated data is inconsistent with the actual service, so that financial service tasks are delayed to be completed or service processing fails due to associated data errors is solved.
Disclosure of Invention
The embodiment of the application aims to provide a data stream association method, a device, equipment and a storage medium thereof, which are used for solving the problems of delayed completion of financial business tasks or business processing failure caused by associated data errors in real-time association of multi-source data streams in the prior art.
In order to solve the above technical problems, the embodiment of the present application provides a data stream association method, which adopts the following technical scheme:
a method of data stream association comprising the steps of:
acquiring a multi-source data stream pushed by a target pushing component in real time, wherein the target pushing component sets source distinguishing information for the multi-source data stream in advance according to a data stream source;
analyzing the multi-source data stream based on a preset analysis component to obtain data attribute fields respectively contained in the multi-source data stream as actual data attribute fields;
obtaining a target association result table, wherein the target association result table is designed for a data attribute field finally required by a target association service in advance;
form analysis is carried out on the target association result table, and data attribute fields contained in the target association result table are identified according to the form analysis result and serve as expected data attribute fields;
Comparing the expected data attribute fields with the actual data attribute fields, and determining target data streams corresponding to all the expected data attribute fields respectively according to the comparison result;
acquiring service data finally required by the target associated service from a target data resource base according to the target data stream respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream;
and mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream.
Further, the target pushing component comprises a data pushing component based on distribution, and before the step of acquiring the multi-source data stream pushed by the target pushing component in real time is executed, the method further comprises:
based on a preset pushing record log, the pushing time when the data pushing component pushes each data stream in the multi-source data stream in real time is recorded;
after the step of acquiring the multi-source data stream pushed by the target pushing component in real time is executed, the method further comprises:
acquiring source distinguishing information corresponding to each data stream in the multi-source data stream according to source distinguishing information preset for the multi-source data stream by the target pushing assembly;
Analyzing the pushing record log through a preset log analysis component, and identifying pushing time corresponding to each data stream in the multi-source data stream.
Further, the preset parsing component is a data stream parsing component, and the step of parsing the multi-source data stream based on the preset parsing component to obtain data attribute fields respectively included in the multi-source data stream specifically includes:
analyzing each data stream in the multi-source data stream according to the data stream analyzing component to obtain the data content transmitted in each data stream;
identifying data attribute fields contained in each data stream based on the data content transmitted in each data stream;
after the step of executing the preset-based parsing component to parse the multi-source data stream and obtaining the data attribute fields respectively contained in the multi-source data stream, the method further includes:
marking data attribute fields contained in each data stream according to source distinguishing information corresponding to each data stream in the multi-source data stream respectively to obtain a marking result, wherein the marking processing mode is that the source distinguishing information corresponding to each data stream respectively is used as a marking field to be assigned to the data attribute fields contained in each data stream respectively;
And identifying source distinguishing information corresponding to all the actual data attribute fields respectively according to the marking result.
Further, the step of comparing the expected data attribute field with the actual data attribute field, and determining the target data stream corresponding to the expected data attribute field according to the comparison result specifically includes:
comparing the expected data attribute field with the actual data attribute field, and determining a one-to-one correspondence between the expected data attribute field and the actual data attribute field;
determining source distinguishing information corresponding to all the expected data attribute fields according to the one-to-one correspondence between the expected data attribute fields and the actual data attribute fields and the source distinguishing information corresponding to all the actual data attribute fields respectively;
and determining target data streams corresponding to all the expected data attribute fields according to the source distinguishing information corresponding to each data stream in the multi-source data streams and the source distinguishing information corresponding to all the expected data attribute fields.
Further, the step of obtaining service data finally required by the target associated service from a target data resource base according to the target data stream respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream specifically includes:
According to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams, a form obtaining request is sent to the target data resource library;
receiving a request response result returned by the target data resource base based on the form acquisition request;
and identifying the target data form related to the service data, the expected data attribute field contained in each target data form and all the data attribute fields contained in each target data form by analyzing the request response result.
Further, the step of mapping the service data to the association result table according to the expected data attribute field to complete association of the multi-source data stream specifically includes:
according to the expected data attribute fields contained in each target data form and all the data attribute fields contained in each target data form, carrying out NULL value processing on the data content corresponding to the unexpected data attribute fields in each target data form to obtain each processed target data form;
according to the target data forms related to the service data and the expected data attribute fields contained in each target data form, splicing all processed target data forms in a UN (UN) form splicing mode to obtain spliced forms;
Acquiring data content corresponding to expected data attribute fields according to the expected data attribute fields contained in each target data form;
according to the push time respectively corresponding to each data stream in the multi-source data streams, the target data streams respectively corresponding to all expected data attribute fields and the data contents corresponding to the expected data attribute fields, adding the data contents corresponding to all expected data attribute fields into the splicing form to obtain a splicing form filled with the data contents;
and mapping the data content in the spliced form into the association result table according to the data attribute field to complete the association of the multi-source data stream.
Further, the step of adding the data content corresponding to all the expected data attribute fields to the splicing form according to the push time corresponding to each data stream in the multi-source data stream, the target data stream corresponding to all the expected data attribute fields and the data content corresponding to the expected data attribute fields, so as to obtain the splicing form after filling the data content specifically includes:
according to the target data streams respectively corresponding to all the expected data attribute fields and the data contents corresponding to the expected data attribute fields, whether the data contents corresponding to the same expected data attribute field are pushed by two or more data streams is identified;
If the data content corresponding to the expected data attribute field is pushed only by one data stream, identifying a data form corresponding to the expected data attribute field according to source distinguishing information of the data stream, obtaining the data content corresponding to the expected data attribute field from the data form, and adding the data content into the spliced form in a common inserting mode, wherein the common inserting mode is specifically to directly add the data content corresponding to the expected data attribute field into the spliced form;
if there is a data content corresponding to the same expected data attribute field being pushed by two or more data streams successively, screening out a data stream which is pushed last according to a pushing time corresponding to each data stream in the multi-source data stream, identifying a data form corresponding to the expected data attribute field according to source distinguishing information of the data stream which is pushed last, obtaining the data content corresponding to the expected data attribute field from the data form as a content to be inserted, and adding the content to be inserted into the spliced form in an updating and inserting mode, wherein the updating and inserting mode is that, if the data content corresponding to the expected data attribute field pushed by a previous data stream is added into the spliced form, deleting the data content corresponding to the expected data attribute field in the spliced form, and adding the content to be inserted into the spliced form
If the data content corresponding to the expected data attribute field pushed by the previous data stream is not added in the splicing form, directly adding the content to be inserted into the splicing form;
and until the data contents corresponding to all the expected data attribute fields are added into the spliced form, obtaining the spliced form filled with the data contents.
In order to solve the above technical problems, the embodiment of the present application further provides a data stream association device, which adopts the following technical scheme:
a data stream association apparatus, comprising:
the multi-source data stream acquisition module is used for acquiring a multi-source data stream pushed by the target pushing component in real time, wherein the target pushing component sets source distinguishing information for the multi-source data stream in advance according to the source of the data stream;
the actual data attribute field acquisition module is used for analyzing the multi-source data stream based on a preset analysis component to acquire data attribute fields respectively contained in the multi-source data stream as actual data attribute fields;
the target association result table acquisition module is used for acquiring a target association result table, wherein the target association result table is designed for a data attribute field which is finally required according to a target association service in advance;
The expected data attribute field acquisition module is used for carrying out form analysis on the target association result table, and identifying the data attribute field contained in the target association result table according to the form analysis result as an expected data attribute field;
the comparison and determination module is used for comparing the expected data attribute field with the actual data attribute field and determining target data streams respectively corresponding to all the expected data attribute fields according to comparison results;
the service data acquisition module is used for acquiring service data finally required by the target associated service from a target data resource base according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams;
and the multi-source data stream association module is used for mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream.
In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical schemes:
a computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data stream correlation method described above.
In order to solve the above technical problems, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical schemes:
a computer readable storage medium having stored thereon computer readable instructions which when executed by a processor perform the steps of a data stream correlation method as described above.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
according to the data stream association method, the association of the multi-source data stream is completed through pushing the multi-source data stream, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data attribute field from the target data resource library in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured.
Drawings
In order to more clearly illustrate the solution of the present application, a brief description will be given below of the drawings required for the description of the embodiments of the present application, it being apparent that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a data stream association method according to the present application;
FIG. 3 is a flow chart of one embodiment of step 205 shown in FIG. 2;
FIG. 4 is a flow chart of one embodiment of step 206 shown in FIG. 2;
FIG. 5 is a flow chart of one embodiment of step 207 shown in FIG. 2;
FIG. 6 is a flow chart of one embodiment of step 504 shown in FIG. 5;
FIG. 7 is a schematic diagram illustrating the structure of one embodiment of a data stream correlation apparatus in accordance with the present application;
FIG. 8 is a schematic structural view of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to make the person skilled in the art better understand the solution of the present application, the technical solution of the embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture ExpertsGroup Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving PictureExperts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the data stream association method provided by the embodiment of the present application is generally executed by a server/terminal device, and accordingly, the data stream association apparatus is generally set in the server/terminal device.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to fig. 2, a flow chart of one embodiment of a data stream association method according to the present application is shown. The data stream association method comprises the following steps:
Step 201, a multi-source data stream pushed by a target pushing component in real time is obtained, wherein the target pushing component sets source distinguishing information for the multi-source data stream in advance according to a data stream source.
In this embodiment, the multi-source data stream is a data stream corresponding to financial service data of a plurality of data sources, that is, the financial service data pushed from a plurality of financial service databases in a data stream manner.
In this embodiment, the target pushing component includes a data pushing component based on a distributed manner, and before executing the step of obtaining the multi-source data stream pushed by the target pushing component in real time, the method further includes: based on a preset pushing record log, the pushing time when the data pushing component pushes each data stream in the multi-source data stream in real time is recorded.
Specifically, the distributed data pushing component comprises a distributed data pushing component formed by taking a Flink cluster real-time task and Kafka message pushing as a framework. Apache Flink is a distributed processing engine framework, which is used for data stream processing, and the Flink can run in all common cluster environments, can perform calculation at a memory speed and any scale, and can perform real-time stream data processing. Apache Kafka generally refers to a message system, and is biased to real-time data processing, and the application combines the Flink cluster real-time task and Kafka message pushing, so that the data stream can be processed in real time, the data pushed in the data stream can be processed in real time, and the processing speed of financial business data is improved.
In this embodiment, after the step of obtaining the multi-source data stream pushed by the target pushing component in real time is performed, the method further includes: acquiring source distinguishing information corresponding to each data stream in the multi-source data stream according to source distinguishing information preset for the multi-source data stream by the target pushing assembly; analyzing the pushing record log through a preset log analysis component, and identifying pushing time corresponding to each data stream in the multi-source data stream.
By identifying the push time corresponding to each data stream in the multi-source data streams, the associated data content is updated according to the push time when the data streams are associated later.
Step 202, analyzing the multi-source data stream based on a preset analyzing component, and obtaining data attribute fields respectively contained in the multi-source data stream as actual data attribute fields.
In this embodiment, the preset parsing component is a data stream parsing component, and because the Apache link may be directly used to perform stateful computation on the data stream, the Apache link may be directly used as the data stream parsing component.
In this embodiment, the step of analyzing the multi-source data stream based on the preset analysis component to obtain the data attribute fields respectively included in the multi-source data stream specifically includes: analyzing each data stream in the multi-source data stream according to the data stream analyzing component to obtain the data content transmitted in each data stream; based on the data content transmitted in each data stream, the data attribute fields contained in each data stream are identified.
And identifying the data attribute fields contained in each data stream respectively through analysis operation, wherein the data attribute fields comprise the primary key ID of each piece of financial service data in the data stream and the data attribute fields in each piece of financial service data related to the target service task, and when pushing, a complete piece of financial service data may not be needed to be pushed according to the financial service requirements, and only data contents corresponding to a plurality of data attribute fields in one piece of financial service data may be pushed. The data attribute fields contained in each data stream are identified, and the purpose of the data attribute fields is to support the subsequent data stream association operation.
In this embodiment, after the step of executing the preset-based parsing component to parse the multi-source data stream and obtain the data attribute fields respectively included in the multi-source data stream, the method further includes: marking data attribute fields contained in each data stream according to source distinguishing information corresponding to each data stream in the multi-source data stream respectively to obtain a marking result, wherein the marking processing mode is that the source distinguishing information corresponding to each data stream respectively is used as a marking field to be assigned to the data attribute fields contained in each data stream respectively; and identifying source distinguishing information corresponding to all the actual data attribute fields respectively according to the marking result.
The source distinguishing information corresponding to each data stream is assigned to the data attribute field contained in each data stream as a marking field, and the purpose of the method is to identify the source distinguishing information corresponding to all the actual data attribute fields, so that the subsequent program can conveniently carry out data stream association operation.
Step 203, a target association result table is obtained, wherein the target association result table is designed in advance according to the data attribute field finally required by the target association service.
And a target association result table is designed in advance according to the finally required data attribute field of the target association service, so that the association of the multi-source data stream is conveniently pointed out in association direction.
And 204, performing form analysis on the target association result table, and identifying the data attribute fields contained in the target association result table as expected data attribute fields according to the form analysis result.
And identifying the data attribute fields contained in the target association result table according to the form analysis result by carrying out form analysis on the target association result table, so that the association direction and the data attribute fields required in association can be clearly defined when the multi-source data streams are associated later.
Step 205, comparing the expected data attribute field with the actual data attribute field, and determining the target data streams corresponding to all the expected data attribute fields according to the comparison result.
With continued reference to fig. 3, fig. 3 is a flow chart of one embodiment of step 205 shown in fig. 2, comprising:
step 301, comparing the expected data attribute field with the actual data attribute field, and determining a one-to-one correspondence between the expected data attribute field and the actual data attribute field;
step 302, determining source distinguishing information corresponding to all the expected data attribute fields according to the one-to-one correspondence between the expected data attribute fields and the actual data attribute fields and the source distinguishing information corresponding to all the actual data attribute fields;
step 303, determining the target data streams corresponding to all the expected data attribute fields according to the source distinguishing information corresponding to each data stream in the multi-source data streams and the source distinguishing information corresponding to all the expected data attribute fields.
And comparing the expected data attribute field with the actual data attribute field, and finally determining source distinguishing information and target data streams corresponding to all the expected data attribute fields respectively because the actual data attribute fields correspond to the corresponding data streams and the source distinguishing information of the corresponding data streams respectively. And the subsequent association operation of the multi-source data stream is facilitated.
And step 206, obtaining the service data finally required by the target associated service from a target data resource base according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams.
With continued reference to fig. 4, fig. 4 is a flow chart of one embodiment of step 206 shown in fig. 2, comprising:
step 401, according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams, a form obtaining request is sent to the target data resource library;
step 402, receiving a request response result returned by the target data resource base based on the form acquisition request;
and step 403, identifying the target data form related to the service data, the expected data attribute field contained in each target data form and all the data attribute fields contained in each target data form by analyzing the request response result.
In this embodiment, a request response processing line is added in addition to the processing line of data stream push transmission, and the service data finally required by the target associated service is directly obtained from a target data resource base in a request response mode, so that the obtained service data and the result data associated with the subsequent data stream can be compared to identify whether the association is successful or not, and the service data finally required by the target associated service can be directly obtained from the target data resource base in the request response mode as the association result when the association is unsuccessful, thereby providing a reference value for the multi-source data stream association operation result.
And step 207, mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream.
With continued reference to fig. 5, fig. 5 is a flow chart of one embodiment of step 207 of fig. 2, comprising:
step 501, according to the expected data attribute field contained in each target data form and all the data attribute fields contained in each target data form, NULL value processing is performed on the data content corresponding to the unexpected data attribute field in each target data form, so as to obtain each processed target data form;
in essence, in this embodiment, the whole form in which the expected data attribute field is located is directly obtained directly according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and the form only containing the data content corresponding to the expected data attribute field is obtained by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later.
Step 502, splicing all processed target data forms by adopting a UN ION form splicing mode according to the target data forms related to the service data and the expected data attribute field contained in each target data form to obtain a spliced form;
The UN ION form splicing mode is a use mode in SQL sentences, and uses UN ION as a splicing character to splice a plurality of forms which are related to each other so as to generate a service comprehensive form, namely the spliced form, wherein the spliced form can be a virtual form cached in a memory.
Step 503, according to the expected data attribute field included in each target data form, obtaining the data content corresponding to the expected data attribute field;
step 504, adding the data content corresponding to all the expected data attribute fields to the splicing form according to the push time corresponding to each data stream in the multi-source data stream, the target data stream corresponding to all the expected data attribute fields and the data content corresponding to the expected data attribute fields, so as to obtain a splicing form filled with the data content;
with continued reference to fig. 6, fig. 6 is a flow chart of one embodiment of step 504 shown in fig. 5, comprising:
step 601, identifying whether data contents corresponding to the same expected data attribute field are pushed by two or more data streams successively according to the target data streams respectively corresponding to all the expected data attribute fields and the data contents corresponding to the expected data attribute fields;
Step 602, if the data content corresponding to the expected data attribute field is pushed only by one data stream, identifying a data form corresponding to the expected data attribute field according to the source distinguishing information of the data stream, obtaining the data content corresponding to the expected data attribute field from the data form, and adding the data content to the spliced form in a common insertion mode, wherein the common insertion mode is specifically to directly add the data content corresponding to the expected data attribute field to the spliced form;
step 603, if there is a situation that data contents corresponding to the same expected data attribute field are pushed by two or more data streams successively, filtering out a data stream which is pushed last according to a push time corresponding to each data stream in the multi-source data stream, identifying a data form corresponding to the expected data attribute field according to source distinguishing information of the data stream which is pushed last, obtaining the data contents corresponding to the expected data attribute field from the data form as contents to be inserted, and adding the contents to be inserted into the spliced form in an update insertion mode, wherein the update insertion mode is that if the data contents corresponding to the expected data attribute field pushed by a previous data stream are added into the spliced form, deleting the data contents corresponding to the expected data attribute field in the spliced form, adding the contents to be inserted into the spliced form, and
If the data content corresponding to the expected data attribute field pushed by the previous data stream is not added in the splicing form, directly adding the content to be inserted into the splicing form;
and step 604, until all the data contents corresponding to the expected data attribute fields are added to the spliced form, obtaining the spliced form filled with the data contents.
And 505, mapping the data content in the spliced form into the association result table according to the data attribute field to complete the association of the multi-source data stream.
In this embodiment, the data content in the spliced form is mapped to the association result table according to the data attribute field, and specifically, the location of the association result table may be in Apache Hudi, which is essentially that the spliced form, that is, the data content in the virtual form cached in the memory is mapped to the association result table in Apache Hudi.
In this embodiment, the association of the multi-source data stream is completed by pushing the multi-source data stream by using Apache Flink and Apache Kafka, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured. The accuracy of the correlation result of the multi-source data stream is guaranteed through the virtual form, and when the correlation delay occurs, the virtual form can be directly used for supporting financial business data, so that the delay of financial business tasks is avoided.
The application completes the association of the multi-source data flow by pushing the multi-source data flow, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured. The accuracy of the correlation result of the multi-source data stream is guaranteed through the virtual form, and when the correlation delay occurs, the virtual form can be directly used for supporting financial business data, so that the delay of financial business tasks is avoided.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data flow correlation technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In the embodiment of the application, the association of the multi-source data stream is completed by pushing the multi-source data stream, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured.
With further reference to fig. 7, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a data stream association apparatus, where an embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus is particularly applicable to various electronic devices.
As shown in fig. 7, the data stream associating apparatus 700 according to the present embodiment includes: a multi-source data stream acquisition module 701, an actual data attribute field acquisition module 702, a target association result table acquisition module 703, a desired data attribute field acquisition module 704, a contrast determination module 705, a traffic data acquisition module 706, and a multi-source data stream association module 707. Wherein:
the multi-source data stream obtaining module 701 is configured to obtain a multi-source data stream that is pushed by a target pushing component in real time, where the target pushing component sets source distinguishing information for the multi-source data stream in advance according to a data stream source;
the actual data attribute field obtaining module 702 is configured to parse the multi-source data stream based on a preset parsing component, and obtain data attribute fields respectively included in the multi-source data stream as actual data attribute fields;
a target association result table obtaining module 703, configured to obtain a target association result table, where the target association result table is designed in advance according to a data attribute field finally required by a target association service;
The expected data attribute field obtaining module 704 is configured to perform form parsing on the target association result table, and identify a data attribute field included in the target association result table according to a form parsing result, as an expected data attribute field;
the comparison determining module 705 is configured to compare the expected data attribute field with the actual data attribute field, and determine, according to a comparison result, target data flows corresponding to all the expected data attribute fields respectively;
a service data obtaining module 706, configured to obtain, from a target data resource base, service data finally required by the target associated service according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams;
and the multi-source data stream association module 707 is configured to map the service data to the association result table according to the expected data attribute field, so as to complete association of the multi-source data stream.
The application completes the association of the multi-source data flow by pushing the multi-source data flow, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured. The accuracy of the correlation result of the multi-source data stream is guaranteed through the virtual form, and when the correlation delay occurs, the virtual form can be directly used for supporting financial business data, so that the delay of financial business tasks is avoided.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by computer readable instructions, stored on a computer readable storage medium, that the program when executed may comprise the steps of embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 8, fig. 8 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 8 comprises a memory 8a, a processor 8b, a network interface 8c communicatively connected to each other via a system bus. It should be noted that only computer device 8 having components 8a-8c is shown in the figures, but it should be understood that not all of the illustrated components need be implemented, and that more or fewer components may alternatively be implemented. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 8a includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 8a may be an internal storage unit of the computer device 8, such as a hard disk or a memory of the computer device 8. In other embodiments, the memory 8a may also be an external storage device of the computer device 8, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 8. Of course, the memory 8a may also comprise both an internal memory unit of the computer device 8 and an external memory device. In this embodiment, the memory 8a is typically used to store an operating system and various application software installed on the computer device 8, such as computer readable instructions of a data stream association method. Further, the memory 8a may be used to temporarily store various types of data that have been output or are to be output.
The processor 8b may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data flow-related chip in some embodiments. The processor 8b is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is configured to execute computer readable instructions stored in the memory 8a or process data, such as computer readable instructions for executing the data stream association method.
The network interface 8c may comprise a wireless network interface or a wired network interface, which network interface 8c is typically used to establish a communication connection between the computer device 8 and other electronic devices.
The computer equipment provided by the embodiment belongs to the technical field of financial science and technology, and is applied to a financial business data multi-data source acquisition scene. The application completes the association of the multi-source data flow by pushing the multi-source data flow, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured.
The present application also provides another embodiment, namely, a computer readable storage medium storing computer readable instructions executable by a processor to cause the processor to perform the steps of the data stream correlation method as described above.
The computer readable storage medium provided by the embodiment belongs to the technical field of financial science and technology, and is applied to a financial business data multi-data source acquisition scene. The application completes the association of the multi-source data flow by pushing the multi-source data flow, determining the expected data attribute field, identifying the actual data attribute field, acquiring the data from the target data resource base in a request acquisition mode, constructing a virtual form, and mapping the data content in the virtual form into the actual association result table. Directly obtaining the whole form in which the expected data attribute field is located according to the target data stream corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream, and obtaining the form only containing the data content corresponding to the expected data attribute field by processing the data content in the form. When the form is acquired, the query is not required according to the data attribute field, the whole form is only required to be acquired, the method is faster, and the form is processed later. The association speed of multi-source data flow in the financial business is improved to a certain extent, and the real-time performance of the financial business data processing is ensured.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.
Claims (10)
1. A method of associating a data stream, comprising the steps of:
acquiring a multi-source data stream pushed by a target pushing component in real time, wherein the target pushing component sets source distinguishing information for the multi-source data stream in advance according to a data stream source;
analyzing the multi-source data stream based on a preset analysis component to obtain data attribute fields respectively contained in the multi-source data stream as actual data attribute fields;
obtaining a target association result table, wherein the target association result table is designed for a data attribute field finally required by a target association service in advance;
form analysis is carried out on the target association result table, and data attribute fields contained in the target association result table are identified according to the form analysis result and serve as expected data attribute fields;
comparing the expected data attribute fields with the actual data attribute fields, and determining target data streams corresponding to all the expected data attribute fields respectively according to the comparison result;
acquiring service data finally required by the target associated service from a target data resource base according to the target data stream respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data stream;
And mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream.
2. The data stream association method of claim 1, wherein the target pushing component comprises a distributed-based data pushing component, and wherein prior to performing the step of obtaining the multi-source data stream that the target pushing component pushes in real-time, the method further comprises:
based on a preset pushing record log, the pushing time when the data pushing component pushes each data stream in the multi-source data stream in real time is recorded;
after the step of acquiring the multi-source data stream pushed by the target pushing component in real time is executed, the method further comprises:
acquiring source distinguishing information corresponding to each data stream in the multi-source data stream according to source distinguishing information preset for the multi-source data stream by the target pushing assembly;
analyzing the pushing record log through a preset log analysis component, and identifying pushing time corresponding to each data stream in the multi-source data stream.
3. The method for associating data streams according to claim 1, wherein the preset parsing component is a data stream parsing component, and the step of parsing the multi-source data stream based on the preset parsing component to obtain data attribute fields respectively included in the multi-source data stream specifically includes:
Analyzing each data stream in the multi-source data stream according to the data stream analyzing component to obtain the data content transmitted in each data stream;
identifying data attribute fields contained in each data stream based on the data content transmitted in each data stream;
after the step of executing the preset-based parsing component to parse the multi-source data stream and obtaining the data attribute fields respectively contained in the multi-source data stream, the method further includes:
marking data attribute fields contained in each data stream according to source distinguishing information corresponding to each data stream in the multi-source data stream respectively to obtain a marking result, wherein the marking processing mode is that the source distinguishing information corresponding to each data stream respectively is used as a marking field to be assigned to the data attribute fields contained in each data stream respectively;
and identifying source distinguishing information corresponding to all the actual data attribute fields respectively according to the marking result.
4. The method for associating a data stream according to claim 3, wherein the step of comparing the expected data attribute field with the actual data attribute field and determining the target data stream to which the expected data attribute field corresponds respectively according to the comparison result specifically comprises:
Comparing the expected data attribute field with the actual data attribute field, and determining a one-to-one correspondence between the expected data attribute field and the actual data attribute field;
determining source distinguishing information corresponding to all the expected data attribute fields according to the one-to-one correspondence between the expected data attribute fields and the actual data attribute fields and the source distinguishing information corresponding to all the actual data attribute fields respectively;
and determining target data streams corresponding to all the expected data attribute fields according to the source distinguishing information corresponding to each data stream in the multi-source data streams and the source distinguishing information corresponding to all the expected data attribute fields.
5. The method for associating data streams according to claim 2, wherein the step of obtaining service data finally required by the target associated service from a target data resource base according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams specifically comprises:
according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams, a form obtaining request is sent to the target data resource library;
Receiving a request response result returned by the target data resource base based on the form acquisition request;
and identifying the target data form related to the service data, the expected data attribute field contained in each target data form and all the data attribute fields contained in each target data form by analyzing the request response result.
6. The method according to claim 5, wherein the step of mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream specifically comprises:
according to the expected data attribute fields contained in each target data form and all the data attribute fields contained in each target data form, carrying out NULL value processing on the data content corresponding to the unexpected data attribute fields in each target data form to obtain each processed target data form;
according to the target data forms related to the service data and the expected data attribute fields contained in each target data form, splicing all processed target data forms in a UNION form splicing mode to obtain spliced forms;
Acquiring data content corresponding to expected data attribute fields according to the expected data attribute fields contained in each target data form;
according to the push time respectively corresponding to each data stream in the multi-source data streams, the target data streams respectively corresponding to all expected data attribute fields and the data contents corresponding to the expected data attribute fields, adding the data contents corresponding to all expected data attribute fields into the splicing form to obtain a splicing form filled with the data contents;
and mapping the data content in the spliced form into the association result table according to the data attribute field to complete the association of the multi-source data stream.
7. The method for associating data streams according to claim 6, wherein the step of adding the data contents corresponding to all the expected data attribute fields to the splice form according to the push time corresponding to each data stream in the multi-source data stream, the target data stream corresponding to all the expected data attribute fields, and the data contents corresponding to the expected data attribute fields, and obtaining the splice form after filling the data contents, specifically includes:
According to the target data streams respectively corresponding to all the expected data attribute fields and the data contents corresponding to the expected data attribute fields, whether the data contents corresponding to the same expected data attribute field are pushed by two or more data streams is identified;
if the data content corresponding to the expected data attribute field is pushed only by one data stream, identifying a data form corresponding to the expected data attribute field according to source distinguishing information of the data stream, obtaining the data content corresponding to the expected data attribute field from the data form, and adding the data content into the spliced form in a common inserting mode, wherein the common inserting mode is specifically to directly add the data content corresponding to the expected data attribute field into the spliced form;
if there is a data content corresponding to the same expected data attribute field being pushed by two or more data streams successively, screening out a data stream which is pushed last according to a pushing time corresponding to each data stream in the multi-source data stream, identifying a data form corresponding to the expected data attribute field according to source distinguishing information of the data stream which is pushed last, obtaining the data content corresponding to the expected data attribute field from the data form as a content to be inserted, and adding the content to be inserted into the spliced form in an updating and inserting mode, wherein the updating and inserting mode is that, if the data content corresponding to the expected data attribute field pushed by a previous data stream is added into the spliced form, deleting the data content corresponding to the expected data attribute field in the spliced form, and then adding the content to be inserted into the spliced form,
If the data content corresponding to the expected data attribute field pushed by the previous data stream is not added in the splicing form, directly adding the content to be inserted into the splicing form;
and until the data contents corresponding to all the expected data attribute fields are added into the spliced form, obtaining the spliced form filled with the data contents.
8. A data stream associating apparatus, comprising:
the multi-source data stream acquisition module is used for acquiring a multi-source data stream pushed by the target pushing component in real time, wherein the target pushing component sets source distinguishing information for the multi-source data stream in advance according to the source of the data stream;
the actual data attribute field acquisition module is used for analyzing the multi-source data stream based on a preset analysis component to acquire data attribute fields respectively contained in the multi-source data stream as actual data attribute fields;
the target association result table acquisition module is used for acquiring a target association result table, wherein the target association result table is designed for a data attribute field which is finally required according to a target association service in advance;
the expected data attribute field acquisition module is used for carrying out form analysis on the target association result table, and identifying the data attribute field contained in the target association result table according to the form analysis result as an expected data attribute field;
The comparison and determination module is used for comparing the expected data attribute field with the actual data attribute field and determining target data streams respectively corresponding to all the expected data attribute fields according to comparison results;
the service data acquisition module is used for acquiring service data finally required by the target associated service from a target data resource base according to the target data streams respectively corresponding to all the expected data attribute fields and the source distinguishing information of the multi-source data streams;
and the multi-source data stream association module is used for mapping the service data into the association result table according to the expected data attribute field to complete the association of the multi-source data stream.
9. A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which when executed by the processor implement the steps of the data stream correlation method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data stream correlation method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311348425.7A CN117216114A (en) | 2023-10-18 | 2023-10-18 | Data stream association method, device, equipment and storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311348425.7A CN117216114A (en) | 2023-10-18 | 2023-10-18 | Data stream association method, device, equipment and storage medium thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117216114A true CN117216114A (en) | 2023-12-12 |
Family
ID=89048267
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311348425.7A Pending CN117216114A (en) | 2023-10-18 | 2023-10-18 | Data stream association method, device, equipment and storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117216114A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117851413A (en) * | 2024-03-07 | 2024-04-09 | 腾讯科技（深圳）有限公司 | Data operation method, device, electronic equipment, storage medium and program product |
| CN117851413B (en) * | 2024-03-07 | 2024-06-07 | 腾讯科技（深圳）有限公司 | Data operation method, device, electronic equipment, storage medium and program product |
-
2023
- 2023-10-18 CN CN202311348425.7A patent/CN117216114A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117851413A (en) * | 2024-03-07 | 2024-04-09 | 腾讯科技（深圳）有限公司 | Data operation method, device, electronic equipment, storage medium and program product |
| CN117851413B (en) * | 2024-03-07 | 2024-06-07 | 腾讯科技（深圳）有限公司 | Data operation method, device, electronic equipment, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021135455A1 (en) | Semantic recall method, apparatus, computer device, and storage medium | |
| CN112395390B (en) | Training corpus generation method of intention recognition model and related equipment thereof | |
| CN105302906A (en) | Information labeling method and apparatus | |
| CN116431878A (en) | Vector retrieval service method, device, equipment and storage medium thereof | |
| CN113010542B (en) | Service data processing method, device, computer equipment and storage medium | |
| CN112860662B (en) | Automatic production data blood relationship establishment method, device, computer equipment and storage medium | |
| CN112182157B (en) | Training method of online sequence labeling model, online labeling method and related equipment | |
| CN117033249A (en) | Test case generation method and device, computer equipment and storage medium | |
| CN115242684B (en) | Full-link pressure measurement method and device, computer equipment and storage medium | |
| CN116661936A (en) | Page data processing method and device, computer equipment and storage medium | |
| CN117216114A (en) | Data stream association method, device, equipment and storage medium thereof | |
| CN116737833A (en) | CDC data resource synchronization method based on partition mode and related equipment thereof | |
| CN115250200B (en) | Service authorization authentication method and related equipment thereof | |
| CN117874073A (en) | Search optimization method, device, equipment and storage medium thereof | |
| CN116756163A (en) | Data synchronization improvement method, device, equipment and storage medium thereof | |
| CN116701119A (en) | Batch running task data monitoring method, device, equipment and storage medium thereof | |
| CN116701512A (en) | Inter-server data call acceleration method, inter-server data call acceleration device, inter-server data call acceleration equipment and storage medium of inter-server data call acceleration equipment | |
| CN117850842A (en) | Plug-in updating method, device, equipment and storage medium thereof | |
| CN115578050A (en) | Approval progress identification method and device, computer equipment and storage medium | |
| CN117235260A (en) | Text labeling method, device, equipment and storage medium based on artificial intelligence | |
| CN116541417A (en) | Batch data processing method, device, computer equipment and storage medium | |
| CN116627416A (en) | Page configuration method, page configuration device, computer equipment and storage medium | |
| CN116028446A (en) | Time sequence data file management method, device, equipment and storage medium thereof | |
| CN117853241A (en) | Risk service provider identification method, apparatus, device and storage medium thereof | |
| CN116933733A (en) | Text input display method, device, equipment and storage medium thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |