CN117235400A - Unified multi-platform portal system based on Kafka technology - Google Patents
Unified multi-platform portal system based on Kafka technology Download PDFInfo
- Publication number
- CN117235400A CN117235400A CN202311133052.1A CN202311133052A CN117235400A CN 117235400 A CN117235400 A CN 117235400A CN 202311133052 A CN202311133052 A CN 202311133052A CN 117235400 A CN117235400 A CN 117235400A
- Authority
- CN
- China
- Prior art keywords
- file
- portal
- kafka
- portal system
- party system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004258 portal system Anatomy 0.000 title claims abstract description 87
- 238000005516 engineering process Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 14
- 230000001360 synchronised effect Effects 0.000 claims description 11
- 235000006719 Cassia obtusifolia Nutrition 0.000 claims description 7
- 235000014552 Cassia tora Nutrition 0.000 claims description 7
- 244000201986 Cassia tora Species 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 4
- 238000009877 rendering Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 3
- 238000013467 fragmentation Methods 0.000 abstract description 2
- 238000006062 fragmentation reaction Methods 0.000 abstract description 2
- 230000003993 interaction Effects 0.000 abstract description 2
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000003860 storage Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 238000005096 rolling process Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a unified multi-platform portal system based on Kafka technology, wherein the front end of the portal system comprises a plurality of front end pages of a third party system for display and interaction, the rear end of the portal system comprises a proxy server, a portal service platform and a corresponding message queue, the portal service platform acquires a service request from the front end pages of the third party system through the proxy server and returns data, and the message queue based on Kafka technology is adopted to serve registration and cancellation services from the third party system; and synchronizing the user information of the third-party system based on the incremental synchronization. The invention supports a plurality of third party systems as the front end of the portal system and realizes unification through the back end of the portal system; unified user by Kafka technology, intelligent registration and cancellation are realized, and the third party system is supported to display and arrange the front end of the portal system; and the large file is transmitted in a fragmentation way based on incremental synchronization, so that the effect and the speed of file transmission are ensured.
Description
Technical Field
The invention relates to the technical field of digital information transmission, in particular to a unified multi-platform portal system based on Kafka technology.
Background
Portal (Portal) originally points to doors and entrances and exits of houses, and is currently used for Portal sites of the Internet. Portal refers to an application system that leads to a comprehensive internet information resource of some kind and provides related information services. Along with the development of technology, concepts such as industry portals, enterprise information portals and the like appear.
With the development of society, on the basis of informatization construction of government departments, a cross-department comprehensive business application system, namely a government portal website is established, so that the public, enterprises and office staff can quickly and conveniently know business applications, organization contents and information of related government departments, and personalized services are obtained.
The government portal serves the masses, but a part of the government portal has special roles, namely, the staff needs to work cooperatively along with the difference of the level and the attribution, and also needs to obtain different operation rights, for example, a part of provincial departments with special requirements need to enter subsystems of various municipal levels through an interface to realize information viewing, and need to carry out personnel management and control through intelligent registration and cancellation through the rear end of the portal, and do not need to operate and manage through a third party system.
However, the current state is that the current government portal does not directly unify a plurality of third party systems, but unifies a plurality of sub-modules in the same system, and the current government portal does not have unified registration, logout and user association functions, and does not have the expandable functions of portal display arrangement, so that the current application requirements cannot be met.
Furthermore, for a special portal, such as a government portal, the interface of the user synchronization mechanism is not external, so that only equipment can be used for synchronization, and the encrypted file can be used for synchronization in a synchronization form, and the equipment has limitation on the size and the transmission rate of the file, so that the inconvenience of data migration is caused.
Disclosure of Invention
The invention solves the problems existing in the prior art and provides a unified multi-platform portal system based on Kafka technology.
The technical scheme adopted by the invention is that the unified multi-platform portal system based on Kafka technology comprises:
the front end of the portal system comprises a plurality of front end pages of a third party system for displaying and interacting;
the portal system back end comprises a proxy server, a portal service platform and a corresponding message queue, wherein the portal service platform acquires a service request from a front end page of a third party system through the proxy server and returns data, and adopts a message queue service based on Kafka technology to register and cancel the service third party system;
incremental synchronization of user information of the third-party system to the portal service platform is achieved.
Preferably, the synchronization of the user information of the third party system comprises the steps of:
step 1.1: creating an incremental file index table for recording uploading files of each business module in any third party system;
step 1.2: collecting and processing user information and attachment data of each business module in the project;
step 1.3: setting a timing task increment synchronization strategy, and pushing file increment to a target server by a data source side based on a preset data segmentation rule on the premise of evaluating a network environment; the evaluation of the network environment can be based on the judgment of the network bandwidth, wherein each sub-volume is larger as the bandwidth is higher;
step 1.4: setting a timed task loading file strategy, and when a target server detects a file pushing action, carrying out integrity judgment on the file, and carrying out file loading based on a judgment result until a complete file is obtained;
step 1.5: and decompressing the file into the corresponding directory according to the path in the file name.
Preferably, in step 1.2, obtaining new or modified interface or method return value parameters of the user, judging the type of the return value, analyzing by adopting different strategies according to different return value types to obtain unified user file information and generate corresponding path URL, and processing the URL based on a module rule strategy to generate a complete directory and a corresponding file path in the server; and generating an incremental file according to the file path, and recording incremental file information.
Preferably, the step 1.3 includes the steps of:
step 1.3.1: acquiring a file synchronization task and acquiring the last file synchronization completion time;
step 1.3.2: the method comprises the steps of obtaining an uploading file record to be pushed after the last file synchronization completion time, backing up data, and deleting the backed up data after the file synchronization is completed generally;
step 1.3.3: carrying out asymmetric encryption on an original file by using a public key to obtain an encrypted file and compressing the encrypted file;
step 1.3.4: if the compressed encrypted file exceeds the threshold value, taking a model according to the file size and the threshold value, determining the total number of to-be-split volumes, performing splitting treatment on the compressed encrypted file, and performing the next step, otherwise, directly pushing the compressed encrypted file to a target server and returning;
step 1.3.5: creating a temporary directory for the split volumes, splitting the compressed encrypted files and outputting the split volumes to obtain a split volume file set;
step 1.3.6: and finally, the partitioned file sets are pushed to the target server one by one, and push result information is recorded.
Preferably, the step 1.4 includes the steps of:
step 1.4.1: the target server monitors the file synchronization message, if the file synchronization message is monitored, synchronous file loading is carried out, all file volumes are obtained according to the prefix matching of the file name, otherwise, the step 1.4.1 is repeated;
step 1.4.2: creating a temporary directory, and generating an original compressed file by dividing all files into volumes;
step 1.4.3: decompressing the compressed file to obtain an encrypted file, decrypting the encrypted file by using an asymmetric algorithm private key, and placing the file in a corresponding directory;
step 1.4.4: the target server analyzes the user file, generates corresponding JavaBean information and stores the JavaBean information in a database.
Preferably, any one of the third party systems registers with the back end of the portal system through the registration code, and any one of the third party systems de-registers with the back end of the portal system through the registration code.
Preferably, any third party system carries an authentication parameter registration code, a user name and a password of Kafka and request parameters to request authentication to Kafka, the Kafka authenticates the user name and the password, an authentication result is returned after the authentication is passed, and the request is stored in the Kafka;
the portal service at the back end of the portal system monitors Kafka, when a registration message exists, a registration code is acquired to carry out registration code verification, request parameters are acquired after verification passes, a public key and a private key for synchronous authentication of the user information of the registered user are generated, the public key and the private key are stored at the back end of the portal system, the public key information is asynchronously transmitted back to a third party system through a public key callback interface URL, and the third party system stores the public key information.
Preferably, when the users are synchronous, the third party system inquires local user information, records the inquiring time as the next user inquiring time parameter, and encrypts the public key after obtaining the user information; sending the encrypted user information to a Kafka service with Kafka authentication, performing login authentication by Kafka, and storing the encrypted user information after the authentication is passed;
the back end of the portal system monitors whether the Kafka service has a message, and if so, the encrypted user information is decrypted by a private key to obtain the user information; and carrying out new addition judgment according to the unique identification number, carrying out automatic account number binding on the user which is not bound with the back end of the portal system, and carrying out association binding on the unique identification number of the user and the ID distributed by the back end of the portal system.
Preferably, any third party system carries CorpId, appKey, appSecret, a portal access URL, a public key callback interface URL, and Logo as request parameters.
Preferably, invoking the entire link through the portal system front end comprises the steps of:
step 2.1: performing login authentication of the front end of the current portal system;
step 2.2: after the front-end authentication of the current portal system is passed, portal list information is acquired from the back-end of the portal system, and front-end rendering display is carried out;
step 2.3: clicking any third party system, and acquiring an authentication code according to CorpID; the front end of the third party system initiates authentication to the rear end of the third party system according to the authentication code, the rear end of the third party system receives the request, firstly initiates the request to the third party system according to AppKey, appSecret, acquires the authentication accessToken, and then acquires the user information according to the accessToken and the authentication code;
step 2.4: and acquiring a unique identifier corresponding to the back-end user information of the third-party system according to the user information, acquiring the user information of the third-party system according to the mobile phone, and if the user information of the third-party system can be inquired, authenticating to pass, and finally entering a corresponding portal system to complete portal unification.
The invention relates to a unified multi-platform portal system based on Kafka technology, wherein the front end of the portal system comprises a plurality of front end pages of a third party system for display and interaction, the rear end of the portal system comprises a proxy server, a portal service platform and a corresponding message queue, the portal service platform acquires a service request from the front end pages of the third party system through the proxy server and returns data, and the message queue based on Kafka technology is adopted to serve registration and cancellation services from the third party system; and synchronizing the user information of the third-party system based on the incremental synchronization.
The invention has the beneficial effects that:
(1) Supporting a plurality of third party systems as the front end of the portal system, and realizing unification through the rear end of the portal system;
(2) Unified user by Kafka technology, intelligent registration and cancellation are realized, and the third party system is supported to display and arrange the front end of the portal system;
(3) And the large file is transmitted in a fragmentation way based on incremental synchronization, so that the effect and the speed of file transmission are ensured.
Drawings
FIG. 1 is a schematic diagram of a system architecture of the present invention;
FIG. 2 is a flow chart of a method for synchronizing user information of a third party system according to the present invention;
FIG. 3 is a flow chart of a method for invoking an entire link through a portal system front end in the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention relates to a unified multi-platform portal system based on Kafka technology, which unifies application inlets of platforms of a plurality of third party systems, opens each platform to enable the platforms to have access rights to each other, opens user information of each platform by adopting Kafka technology, completes portal registration and portal cancellation, and supports portal display position arrangement.
In the invention, kafka is applied to the system as a high-throughput distributed publish-subscribe message system, can process all action stream data of consumers in websites, including but not limited to web browsing, searching and actions of other users, unifies online and offline message processing through a Hadoop parallel loading mechanism, and provides real-time messages through clusters.
The system comprises:
the front end of the portal system comprises a plurality of front end pages of a third party system for displaying and interacting;
the portal service platform acquires a service request from a front page of a third party system through a proxy server and returns data, and adopts a message queue service based on Kafka technology to register and cancel the service third party system;
incremental synchronization of user information of the third-party system to the portal service platform is achieved.
In the invention, in order to ensure the applicability of the third party system, a timing heartbeat technology is generally adopted to monitor whether the front end of each portal system is online.
In the invention, in order to ensure the consistency of data and the real-time and reliability of synchronization in the synchronization process, user information including file synchronization is performed in an incremental mode, when the file is a large file, the large file is segmented and rolled, the segmented and rolled into an original file at a receiving party, the data are finally analyzed and synchronously stored, and after the data are synchronized to a target server, the segmented and rolled into the original file which is placed in a corresponding catalog.
In the implementation of the present invention, the synchronization of the user information of the third party system includes the following steps:
step 1.1: creating an incremental file index table for recording uploading files of each business module in any third party system;
step 1.2: collecting and processing user information and attachment data of each business module in the project;
here, the attachment data uploaded by each service module is collected by using a combination of a section (UploadFileAspect) and an annotation (UploadFileRecord), wherein the annotation only acts on the method for uploading the file, and the UploadFileAspect is a specific execution flow of the annotation.
Specifically, in step 1.2, obtaining new or modified interface or method return value parameters of the user, judging the type of the return value, analyzing by adopting different strategies according to different return value types, such as String, javabean types, and the like, obtaining unified user file information and generating corresponding path URL, processing the URL based on module rule strategies, and generating a complete directory and a corresponding file path in the server;
for example, when the return value is of the String type, then the value file path is returned directly, i.e. the return value is directly taken as the file path; when the return value is only one object, and the object attribute is id, file url, file name and the like, the strategy of acquiring the file path is to analyze the return value, an original object is generated through a java reflection mechanism, and finally the file url is acquired according to the object and is used as the file path.
And generating an incremental file according to the file path, wherein the incremental file is stored in the form of a compressed file, and the name of the compressed file is generally a file storage path, so that a target server conveniently loads the file and places the file in a corresponding directory, and incremental file information including, but not limited to, a service module, the file name, a file storage path, the compressed file path, file uploading time, creator information and the like is recorded for later file synchronization.
Step 1.3: setting a timing task increment synchronization strategy, and pushing file increment to a target server by a data source side based on a preset data segmentation rule on the premise of evaluating a network environment;
the step 1.3 comprises the following steps:
step 1.3.1: acquiring a file synchronization task and acquiring the last file synchronization completion time;
step 1.3.2: the method comprises the steps of obtaining an uploading file record to be pushed, which is created after the last file synchronization completion time (the creation time is later than the last synchronization completion time), from a t_uploading_file_record table, and preventing data backup from being lost;
step 1.3.3: carrying out asymmetric encryption on an original file by using a public key to obtain an encrypted file, and compressing the encrypted file to obtain a zip file package ASSET-fileName.
Step 1.3.4: if the compressed encrypted file exceeds the threshold value, taking a model according to the file size and the threshold value, determining the total number of to-be-split volumes, performing splitting treatment on the compressed encrypted file, and performing the next step, otherwise, directly pushing the compressed encrypted file to a target server and returning;
step 1.3.5: creating a temporary directory ASSET-fileName-X for the split volumes, splitting the compressed encrypted files and outputting the split volumes to obtain a split volume file set, wherein ASSET-fileName.z1, ASSET-fileName.z2 and the like;
step 1.3.6: finally, the partitioned file sets are pushed to a target server one by one, and push result information is recorded; typically, the number of retries for a push failure is 3.
Step 1.4: setting a timed task loading file strategy, and when a target server detects a file pushing action, carrying out integrity judgment on the file, and carrying out file loading based on a judgment result until a complete file is obtained; firstly judging whether the file is a segmented and rolled file, if so, carrying out file rolling and rolling processing to generate an original file, and then loading, otherwise, directly loading, and finally decompressing into a corresponding directory according to a path in a file name.
The step 1.4 comprises the following steps:
step 1.4.1: the target server monitors the file synchronization message, if the file synchronization message is monitored, synchronous file loading is carried out, all file sub-volumes are obtained according to the prefix matching of the file names, such as ASSET-fileName.z1 and ASSET-fileName.z2 sub-volume files, otherwise, the step 1.4.1 is repeated;
step 1.4.2: creating a temporary directory ASSET-fileName, and using a zip rolling command to generate an original compressed file by dividing all files into volumes; in practical application, after the file synchronization is completed, the user record can be marked in the incremental file index table to complete the file synchronization, at the moment, the incremental file index table is synchronized to the third party system side, and the third party system side only needs to process the marked user file synchronization until the third party system side receives all the sub-volumes based on the index table, namely, receives all the file sub-volumes, and does not need to judge any more;
step 1.4.3: decompressing the compressed file to obtain an encrypted file, decrypting the encrypted file by using an asymmetric algorithm private key, and placing the file in a corresponding directory;
step 1.4.4: the target server analyzes the user file, generates corresponding JavaBean information and stores the JavaBean information in a database.
Step 1.5: and decompressing the file into the corresponding directory according to the path in the file name.
In the invention, a portal service platform at the back end of the portal system provides the following services:
distributing portal registration codes, namely generating portal registration codes to be used when a third party system is registered before portal registration, and simultaneously giving out a user name and a secret key of Kafka;
portal registration, wherein a third party system carries a registration code to carry out portal registration, corpId, appKey, appSecret is required to be uploaded and stored during registration, and Logo, system name and the like can be uploaded and displayed;
portal cancellation, namely cancellation is carried out on registered portals, and authentication cancellation is carried out by carrying registration codes when cancellation is carried out;
user synchronization, integrating user information of all third party systems in a portal service platform;
portal management, which manages registered portals, such as modification, deletion, etc.;
portal arrangement, which manages the display position, style, logo and the like of the portal system.
It should be noted in advance that, in the system of the present invention, any third party system carries CorpId, appKey, appSecret, a portal access URL, a public key callback interface URL, and a Logo as request parameters, where the Logo is in a binary format or in a base64 string form.
Any one of the third party systems registers to the back end of the portal system through the registration code, and any one of the third party systems logs out to the back end of the portal system through the registration code.
Specifically, any third party system carries an authentication parameter registration code, a user name and a password of Kafka and request parameters to request authentication to Kafka, the Kafka authenticates the user name and the password, an authentication result is returned after the authentication is passed, and the request is stored in the Kafka so as to be convenient for other services to consume; the authentication parameter registration codes of each third party system are independently distributed and are different from each other;
the portal service at the back end of the portal system monitors Kafka, when a registration message exists, the message is processed in real time, registration codes are acquired to check the registration codes, request parameters are acquired after the check passes, corpId, appKey, appSecret and Logo are obtained, corpId, appKey, appSecret are stored, binary format or base64 character strings of the Logo are converted into texts, file paths are generated and stored, public keys and private keys for synchronous authentication of the user information of the registered user are generated, the public keys and the private keys are stored at the back end of the portal system, the public key information is asynchronously returned to a third party system through a public key callback interface URL, and the third party system stores the public key information.
Obviously, the third party system flow of cancellation is the same as portal registration, a cancellation request is initiated with a registration code and a public key, a private key can be obtained by inquiring according to the registration code and the public key, and portal deletion cancellation operation is performed.
When the users are synchronous, a third party system inquires local user information, here incremental inquiry, records the inquiry time as the next user inquiry time parameter, and encrypts a public key after obtaining the user information; sending the encrypted user information to a Kafka service with Kafka authentication, performing login authentication by Kafka, and storing the encrypted user information after the authentication is passed;
the back end of the portal system monitors whether the Kafka service has a message, if so, the message is processed in real time, and the encrypted user information is decrypted by a private key to obtain the user information; if decryption passes, carrying out new addition judgment according to a unique identification number, such as a mobile phone number, if the mobile phone number of the user exists, carrying out no processing, if the mobile phone number of the user does not exist, carrying out new addition processing, carrying out account number automatic binding on the user which is not bound with the back end of the portal system, initiating a token acquisition request to the back end of the portal system according to AppKey, appSecret information, returning the token through the request, then carrying the token and mobile phone number parameters to request a user interface at the back end of the portal system to acquire user information, and returning the user information after authentication passes, wherein the user information comprises an ID at the back end of the portal system; the unique user identification number is associated and bound with the ID distributed at the back end of the portal system, so that the user has access rights of all portals, all accounts are opened, and the accounts are enabled to be operable in all system logins.
The user can arrange the interface of the front end of the portal system, carry out portal position association binding according to the AppKey of the sub-portal system, and finally store the portal position association binding in a database, and when the front end of the portal system is inquired and rendered, the portal position is displayed according to the AppKey and the portal position, so that the arrangement effect is achieved.
Invoking the entire link through the portal system front end includes the steps of:
step 2.1: performing login authentication of the front end of the current portal system;
specifically, the front end of the portal system acquires an authentication code authCode from the rear end of the portal system according to CorpID, the front end of the portal system initiates authentication to the rear end of the portal system according to authCode, the rear end of the portal system receives a request, the request is initiated to a third party system according to AppKey and AppSecret to acquire authentication Access token, user information is acquired according to the Access token and authCode, the user information comprises a portal system rear end ID, finally the portal system rear end user information, such as a mobile phone number, is acquired according to the portal system rear end ID, portal user information is acquired according to the mobile phone number, if portal user information can be queried, authentication is passed, and finally the portal system is entered.
Step 2.2: after the front-end authentication of the current portal system passes, portal list information including but not limited to CorpID, portal name, portal URL and URL of portal logo is acquired from the back-end of the portal system, and front-end rendering display is carried out;
step 2.3: clicking any third party system, and acquiring an authentication code according to CorpID; the front end of the third party system initiates authentication to the rear end of the third party system according to the authentication code, the rear end of the third party system receives the request, firstly initiates the request to the third party system according to AppKey, appSecret, acquires the authentication accessToken, and then acquires the user information according to the accessToken and the authentication code;
step 2.4: and acquiring a unique identifier corresponding to the back-end user information of the third-party system according to the user information, acquiring the user information of the third-party system according to the mobile phone, and if the user information of the third-party system can be inquired, authenticating to pass, and finally entering a corresponding portal system to complete portal unification.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A unified multi-platform portal system based on Kafka technology is characterized in that: the system comprises:
the front end of the portal system comprises a plurality of front end pages of a third party system for displaying and interacting;
the portal system back end comprises a proxy server, a portal service platform and a corresponding message queue, wherein the portal service platform acquires a service request from a front end page of a third party system through the proxy server and returns data, and adopts a message queue service based on Kafka technology to register and cancel the service third party system;
incremental synchronization of user information of the third-party system to the portal service platform is achieved.
2. The unified multi-platform portal system based on Kafka technology as claimed in claim 1 wherein: the synchronization of the user information of the third party system comprises the following steps:
step 1.1: creating an incremental file index table for recording uploading files of each business module in any third party system;
step 1.2: collecting and processing user information and attachment data of each business module in the project;
step 1.3: setting a timing task increment synchronization strategy, and pushing file increment to a target server by a data source side based on a preset data segmentation rule on the premise of evaluating a network environment;
step 1.4: setting a timed task loading file strategy, and when a target server detects a file pushing action, carrying out integrity judgment on the file, and carrying out file loading based on a judgment result until a complete file is obtained;
step 1.5: and decompressing the file into the corresponding directory according to the path in the file name.
3. The unified multi-platform portal system based on Kafka technology as claimed in claim 2 wherein: in step 1.2, obtaining a new or modified interface or method return value parameter of a user, judging the type of a return value, analyzing by adopting different strategies according to different return value types to obtain unified user file information and generate a corresponding path URL, and processing the URL based on a module rule strategy to generate a complete directory and a corresponding file path in a server; and generating an incremental file according to the file path, and recording incremental file information.
4. The unified multi-platform portal system based on Kafka technology as claimed in claim 2 wherein: the step 1.3 comprises the following steps:
step 1.3.1: acquiring a file synchronization task and acquiring the last file synchronization completion time;
step 1.3.2: acquiring an uploading file record to be pushed after the last file synchronization completion time, and backing up data;
step 1.3.3: carrying out asymmetric encryption on an original file by using a public key to obtain an encrypted file and compressing the encrypted file;
step 1.3.4: if the compressed encrypted file exceeds the threshold value, taking a model according to the file size and the threshold value, determining the total number of to-be-split volumes, performing splitting treatment on the compressed encrypted file, and performing the next step, otherwise, directly pushing the compressed encrypted file to a target server and returning;
step 1.3.5: creating a temporary directory for the split volumes, splitting the compressed encrypted files and outputting the split volumes to obtain a split volume file set;
step 1.3.6: and finally, the partitioned file sets are pushed to the target server one by one, and push result information is recorded.
5. The unified multi-platform portal system based on Kafka technology as claimed in claim 2 wherein: the step 1.4 comprises the following steps:
step 1.4.1: the target server monitors the file synchronization message, if the file synchronization message is monitored, synchronous file loading is carried out, all file volumes are obtained according to the prefix matching of the file name, otherwise, the step 1.4.1 is repeated;
step 1.4.2: creating a temporary directory, and generating an original compressed file by dividing all files into volumes;
step 1.4.3: decompressing the compressed file to obtain an encrypted file, decrypting the encrypted file by using an asymmetric algorithm private key, and placing the file in a corresponding directory;
step 1.4.4: the target server analyzes the user file, generates corresponding JavaBean information and stores the JavaBean information in a database.
6. The unified multi-platform portal system based on Kafka technology as claimed in claim 1 wherein: any one of the third party systems registers to the back end of the portal system through the registration code, and any one of the third party systems logs out to the back end of the portal system through the registration code.
7. The unified multi-platform portal system based on Kafka technology of claim 6, wherein: any third party system carries an authentication parameter registration code, a user name and a password of Kafka and request parameters to request authentication to Kafka, the Kafka authenticates the user name and the password, an authentication result is returned after the authentication is passed, and the request is stored in the Kafka;
the portal service at the back end of the portal system monitors Kafka, when a registration message exists, a registration code is acquired to carry out registration code verification, request parameters are acquired after verification passes, a public key and a private key for synchronous authentication of the user information of the registered user are generated, the public key and the private key are stored at the back end of the portal system, the public key information is asynchronously transmitted back to a third party system through a public key callback interface URL, and the third party system stores the public key information.
8. The unified multi-platform portal system based on Kafka technology as claimed in claim 1 wherein: when the users are synchronous, the third party system inquires the local user information, records the inquiring time as the inquiring time parameter of the next time, and encrypts the public key after obtaining the user information; sending the encrypted user information to a Kafka service with Kafka authentication, performing login authentication by Kafka, and storing the encrypted user information after the authentication is passed;
the back end of the portal system monitors whether the Kafka service has a message, and if so, the encrypted user information is decrypted by a private key to obtain the user information; and carrying out new addition judgment according to the unique identification number, carrying out automatic account number binding on the user which is not bound with the back end of the portal system, and carrying out association binding on the unique identification number of the user and the ID distributed by the back end of the portal system.
9. The Kafka technology based unified multi-platform portal system according to claim 7 or 8, wherein: any third party system carries CorpId, appKey, appSecret, a portal access URL, a public key callback interface URL, and Logo as request parameters.
10. The unified multi-platform portal system based on Kafka technology as claimed in claim 9 wherein: invoking the entire link through the portal system front end includes the steps of:
step 2.1: performing login authentication of the front end of the current portal system;
step 2.2: after the front-end authentication of the current portal system is passed, portal list information is acquired from the back-end of the portal system, and front-end rendering display is carried out;
step 2.3: clicking any third party system, and acquiring an authentication code according to CorpID; the front end of the third party system initiates authentication to the rear end of the third party system according to the authentication code, the rear end of the third party system receives the request, firstly initiates the request to the third party system according to AppKey, appSecret, acquires the authentication accessToken, and then acquires the user information according to the accessToken and the authentication code;
step 2.4: and acquiring a unique identifier corresponding to the back-end user information of the third-party system according to the user information, acquiring the user information of the third-party system according to the mobile phone, and if the user information of the third-party system can be inquired, authenticating to pass, and finally entering a corresponding portal system to complete portal unification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311133052.1A CN117235400A (en) | 2023-09-04 | 2023-09-04 | Unified multi-platform portal system based on Kafka technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311133052.1A CN117235400A (en) | 2023-09-04 | 2023-09-04 | Unified multi-platform portal system based on Kafka technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117235400A true CN117235400A (en) | 2023-12-15 |
Family
ID=89081718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311133052.1A Pending CN117235400A (en) | 2023-09-04 | 2023-09-04 | Unified multi-platform portal system based on Kafka technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117235400A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117527678A (en) * | 2024-01-04 | 2024-02-06 | 贵州卓讯软件股份有限公司 | Data exchange center and method |
-
2023
- 2023-09-04 CN CN202311133052.1A patent/CN117235400A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117527678A (en) * | 2024-01-04 | 2024-02-06 | 贵州卓讯软件股份有限公司 | Data exchange center and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110971655B (en) | Offline client playback and synchronization | |
| US10447560B2 (en) | Data leakage protection in cloud applications | |
| US20160119429A1 (en) | Multi-service cloud storage decision optimization process | |
| US20170097973A1 (en) | System and method for duplicating an application state | |
| US20020002569A1 (en) | Systems, methods and computer program products for associating dynamically generated web page content with web site visitors | |
| CN103780700A (en) | Application system and method for achieving compatibility and sharing among multi-source heterogeneous systems | |
| US20140245025A1 (en) | System and method for storing data securely | |
| CN109660447B (en) | Chat data-based information directional capturing method and information management system | |
| US10942984B2 (en) | Portal connected to a social backend | |
| CN112311654A (en) | Instant messaging message processing method, device, medium and electronic equipment | |
| CN117235400A (en) | Unified multi-platform portal system based on Kafka technology | |
| US20220141227A1 (en) | Managing Metadata for External Content within a Computing Environment | |
| CN105335186B (en) | Data downloading management method and system | |
| CN111460394A (en) | Copyright file verification method and device and computer readable storage medium | |
| CN115185967A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN117035986A (en) | Remote double-record loan method and device, electronic equipment and storage medium | |
| Kao | Cybercrime investigation countermeasure using created-accessed-modified model in cloud computing environments | |
| CN114282210A (en) | Sandbox automatic construction method and system, computer equipment and readable storage medium | |
| CN112100534A (en) | Information processing method, device, medium and electronic equipment in page sharing | |
| KR20160132854A (en) | Asset collection service through capture of content | |
| CN116382596B (en) | Space-time big data storage method and system based on distributed technology | |
| CN105701110B (en) | A kind of method and apparatus that data are applied in inquiry | |
| US11803695B1 (en) | Dynamic sharable web clippings | |
| CN112148571B (en) | Webpage operation process recording and playback method and device | |
| US9607167B2 (en) | Self-service portal for tracking application data file dissemination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |