CN111090691A - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111090691A CN111090691A CN201911319414.XA CN201911319414A CN111090691A CN 111090691 A CN111090691 A CN 111090691A CN 201911319414 A CN201911319414 A CN 201911319414A CN 111090691 A CN111090691 A CN 111090691A
- Authority
- CN
- China
- Prior art keywords
- data source
- access
- management platform
- unified management
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
The application discloses a data processing method and device, electronic equipment and a storage medium, and relates to the field of data processing. The specific implementation scheme is as follows: the method comprises the steps that a unified management platform responds to a data source access request of a target content provider, and obtains an access type and an access parameter which are matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance; the unified management platform acquires a target data source access module matched with the access type from the data source access modules of a plurality of types; the unified management platform triggers a target data source access module according to the access parameters, and accesses a target data source matched with the data source access request; and the unified management platform acquires target data from a target data source and stores the target data in a storage space matched with the target content provider. According to the technical scheme, the data sources of different content providers can be managed and maintained by one unified management platform.
Description
Technical Field
The present disclosure relates to computer technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
With the development of computer technology, community-type internet products are more and more widely applied. In the community internet products, a large amount of high-quality timeliness content exists, and the high-quality and timeliness content of each content provider of the community internet products needs to be managed in a unified mode.
In the prior art, data sources of content providers of community-type internet products are relatively independent, so that high-quality and timeliness content of the content providers cannot be managed in a unified manner, flexible sharing of the high-quality and timeliness content among the content providers cannot be realized, the cost for maintaining and sharing the high-quality and timeliness content is high, and the user experience is poor.
Disclosure of Invention
The embodiment of the application discloses a data processing method, a data processing device, electronic equipment and a storage medium, which can realize that data sources of different content providers are managed and maintained by a unified management platform.
In a first aspect, an embodiment of the present application discloses a data processing method, including:
the method comprises the steps that a unified management platform responds to a data source access request of a target content provider, and obtains an access type and an access parameter which are matched with the data source access request, wherein the unified management platform is connected with a plurality of content providers in advance;
the unified management platform acquires a target data source access module matched with the access type from a plurality of types of data source access modules;
the unified management platform triggers the target data source access module according to the access parameters to access a target data source matched with the data source access request;
one embodiment of the above application has the following advantages or benefits:
the data sources of different content providers are accessed into the unified management platform, so that the data sources of different content providers can be managed and maintained by the unified management platform.
In addition, the data processing method according to the above-mentioned embodiment of the present application may further have the following additional technical features:
further, after the unified management platform triggers the target data source access module according to the access parameter and accesses the target data source matched with the data source access request, the method further includes:
and the unified management platform acquires target data from the target data source and stores the target data in a storage space matched with the target content provider.
One embodiment in the above application has the following advantages or benefits:
target data sources required by different content providers are accessed into the unified management platform for storage, so that unified management and flexible sharing of the target data sources can be realized.
Further, after the unified management platform accesses the target data source matched with the data source access request, the method further includes:
the unified management platform responds to the authority setting instruction of the target content provider and sets the viewing authority of the target data source as a private authority or a public authority;
the data source set to the private authority only allows the content provider accessing the data source to access, and the data source set to the public authority also allows other content providers with viewing authority to access.
One embodiment in the above application has the following advantages or benefits:
by setting the authority of the data source, flexible access to the data source among different data sources is realized according to the set authority.
Further, after the unified management platform responds to the right setting instruction of the target content provider and sets the viewing right of the target data source as a public right, the method further includes:
the unified management platform responds to data access requests of other content providers;
and if the unified management platform determines that the other content providers are other content providers with viewing rights, the stored target data is provided for the other content providers to access.
One embodiment in the above application has the following advantages or benefits:
when the viewing right of the target data source is set as the public right, the data sharing of the target data source to other content providers with open rights can be realized.
Further, the method further comprises:
the unified management platform counts the read-write state of each accessed data source;
the unified management platform carries out offline and/or online processing on each data source according to the read-write state;
the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.
One embodiment in the above application has the following advantages or benefits:
the online and offline management of the data source is realized, the intelligent switching of the state of the data source is realized, and the labor cost for maintaining the data source is reduced.
Further, the unified management platform performs offline and/or online processing on each data source according to the read-write state, including:
if the unified management platform determines that the number of reading and writing times of a first data source which is currently in an online state in a first time interval is smaller than a first threshold, the unified management platform changes the storage state of first data corresponding to the first data source from exposed storage to hidden storage so as to perform offline processing on the first data source;
and if the unified management platform determines that the number of reading and writing times of the second data source in the offline state in the second time interval is greater than or equal to a second threshold, the unified management platform changes the storage state of the second data corresponding to the second data source from hidden storage to exposed storage so as to perform online processing on the first data source.
One embodiment in the above application has the following advantages or benefits:
the intelligent switching of the on-line and off-line states of the data source is realized, the management efficiency of the data source is improved, and the cost for manually maintaining the data source is reduced.
Further, after the unified management platform counts the read-write states of the accessed data sources, the method further includes:
if the unified management platform determines that the read-write times of a third data source currently in an offline state in a third time interval are smaller than a third threshold, the unified management platform disconnects the access to the third data source;
and the unified management platform synchronously deletes all data corresponding to the third data source in the storage space.
One embodiment in the above application has the following advantages or benefits:
the method and the device realize timely recovery of the invalid data source and reduce resource waste caused by the fact that the invalid data source occupies a storage space for a long time.
Further, the access type includes at least one of:
the system comprises an application program interface push-pull access mode, a one-key mirror access mode, a file import access mode, a simple information aggregation access mode and a fifth generation hypertext markup language page access mode.
One embodiment in the above application has the following advantages or benefits:
the access time of the data source is reduced, and the quick access of the data source is realized.
Further, the content provider is a provider for setting vertical services in a website.
Other effects of the above-described alternative will be described below with reference to specific embodiments.
In a second aspect, an embodiment of the present application discloses a data processing apparatus, including:
the system comprises an access type and access parameter acquisition module, a data source access request acquisition module and a content source access request acquisition module, wherein the access type and access parameter acquisition module is used for enabling a unified management platform to respond to a data source access request of a target content provider and acquire an access type and an access parameter matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance;
the target data source access module acquisition module is used for enabling the unified management platform to acquire a target data source access module matched with the access type in a plurality of types of data source access modules;
the target data source access module is used for triggering the target data source access module by the unified management platform according to the access parameters and accessing a target data source matched with the data source access request;
in a third aspect, an embodiment of the present application discloses an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present application.
In a fourth aspect, the present application discloses a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in any one of the present application embodiments.
According to the technical scheme of the embodiment of the application, the data sources of the content providers are accessed into the unified management platform through the access module. The problem of the data source mutual independence between each content provider in the prior art to the administrative and maintenance cost that causes is solved. The unified management platform can be used for carrying out unified maintenance and management on different data sources.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a flowchart of a data processing method according to a second embodiment of the present application;
fig. 3 is a flowchart of a data processing method according to a third embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one
Fig. 1 is a flowchart of a data processing method provided in an embodiment of the present application, where the embodiment is applicable to a case where unified management and maintenance of data sources of different content providers are required, and the method may be performed by a data processing apparatus, which may be implemented by software and/or hardware and is generally integrated in a server.
As shown in fig. 1, the technical solution of the embodiment of the present application specifically includes the following steps:
s110, responding to a data source access request of a target content provider, a unified management platform acquires an access type and an access parameter matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance.
The unified management platform is used for enabling the data sources to be accessed in a unified mode and conducting unified management on data in the data sources. The content provider may be a main body providing a certain service, the content provider may establish a connection with the unified management platform in advance, the unified management platform may include a plurality of content providers, and the specific number of the content providers is not limited in the embodiment of the present application. The data source may be a device or raw media that provides some desired data in which all information establishing the database connection may be stored. The access type may be used to indicate how the data source is accessed into the unified management platform, and the access parameter may be a parameter that needs to be configured when the data source is accessed into the unified management module.
In the embodiment of the application, the unified management platform is connected with a plurality of content providers in advance, wherein a target content provider refers to a content provider which needs to access a new data source at present, and when the target content provider accesses the data source to the unified management platform, the target content provider sends a data source access request to the unified management platform, and the access type and the access parameters can be acquired according to the data source access request.
In an optional embodiment of the present application, the access type may include at least one of the following: the system comprises an application program interface push-pull access mode, a one-key mirror access mode, a file import access mode, a simple information aggregation access mode and a fifth generation hypertext markup language page access mode.
The push-pull access mode of the Application Programming Interface (API) may be to directly access the database through the API. The push-pull access mode of the application program interface can comprise a push mode and a pull mode, and the push mode can be that the database actively pushes data after the connection with the database is established; the pull mode may be to send a data acquisition request to the database after establishing a connection with the database. The content provider can flexibly select the access mode of the application program interface according to the service characteristics and risks of the content provider. The one-key mirror image access mode needs to be configured in advance by a user, and the data can be quickly synchronized. The file import access mode can generate a file according to the data, and the data is imported by selecting a corresponding file and analyzing the file. The RSS (Really Simple Syndication) access mode may refer to a Simple way for a site to share content with other sites. The access mode of a fifth generation hypertext markup Language page (Hyper text markup Language 5, H5) can be to establish connection with a database through an H5 interface, and to obtain data by analyzing the interface.
In the embodiment of the application, the access types of multiple data sources are set, so that the advantage that the access mode can be flexibly configured according to the actual needs of the data sources, the access time of the data sources is reduced, and the quick access of the data sources is realized.
In an alternative embodiment of the present application, the content provider may be a provider of a setting vertical service in a website.
The unified management platform can comprise a plurality of vertical classes, and different vertical classes correspond to different content providers. The vertical type can be a plate block contained by a unified management platform, in a specific example, a community type internet product can contain the unified management platform, the unified management platform can comprise a favorite game plate block, an entertainment plate block, a chat plate block and the like, the favorite game, the entertainment and the chat belong to different vertical types and correspond to different content providers. Accordingly, different content providers are providers that set services for different verticals.
And S120, the unified management platform acquires a target data source access module matched with the access type from the data source access modules of multiple types.
The data source access module is used for accessing a data source, the data source access module can correspond to the access type of the data source, and different data source access types can correspond to different data source access modules. In a specific example, the application program interface push-pull data source access module is used for accessing the data source into the unified management platform in an application program interface push-pull mode. In the unified management platform, access types of multiple data sources can be set, and correspondingly, multiple types of data source access modules can be set. Optionally, the data source access module may be configured in advance in the unified management platform.
In the embodiment of the application, among the multiple types of data source access modules arranged on the unified management platform, the data source access module corresponding to the access type is acquired and used as a target data source access module to access a target data source.
S130, the unified management platform triggers the target data source access module according to the access parameters, and accesses a target data source matched with the data source access request.
In the embodiment of the application, the access parameter in the target data source access request can trigger the target data source access module, and the target data source is accessed to the unified management platform after the target data source access module is triggered.
According to the technical scheme of the embodiment of the application, the data sources of the content providers are accessed into the unified management platform through the access module. The problem of the data source mutual independence between each content provider in the prior art to the administrative and maintenance cost that causes is solved. The unified management platform can be used for carrying out unified maintenance and management on different data sources.
Example two
Fig. 2 is a flowchart of a data processing method provided in the second embodiment of the present application, and in the second embodiment of the present application, after the step of accessing the target data source, the step of setting the viewing right of the target data source, and storing the target data in the target data source in the storage space are added, and the step of invoking the target data according to the viewing right of the target data source is further embodied.
Correspondingly, as shown in fig. 2, the technical solution of the embodiment of the present application specifically includes the following steps:
s210, responding to a data source access request of a target content provider, a unified management platform acquires an access type and an access parameter matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance.
S220, the unified management platform obtains a target data source access module matched with the access type from the data source access modules of multiple types.
And S230, the unified management platform triggers the target data source access module according to the access parameters, and accesses the target data source matched with the data source access request.
S240, the unified management platform responds to the authority setting instruction of the target content provider, and the viewing authority of the target data source is set to be a private authority or a public authority.
The permission setting instruction can be sent by a content provider and used for setting the data source viewing permission. The viewing permissions may be used to indicate whether data in the data source is accessible by other data sources.
In the embodiment of the present application, the data source set to the private right may only allow the content provider accessing the data source to access, and the data source set to the public right may also allow the content provider accessing the content provider issuing the right of the data source to access.
One server provider may issue rights to all server providers included in the unified management platform, or may issue rights to only one or a specified number of content providers included in the unified management platform, which is not limited in this embodiment.
S250, the unified management platform acquires target data from the target data source and stores the target data in a storage space matched with the target content provider.
The target data may be data that needs to be shared. In a specific example, the target data may be high-quality data in the target data source, and when a new content provider appears, the new content provider needs to refer to high-quality content in other content provider data sources, and at this time, the high-quality content in the other content provider data sources needs to be shared with the new content provider, and the high-quality content in the other content provider data sources may become the target data.
In the embodiment of the application, after the target data source is accessed to the unified management platform, the unified management platform stores the target data in the target data source into the storage space corresponding to the target content provider, and the data in the storage space corresponding to each content provider are relatively independent.
And S260, judging whether the viewing authority of the target data source is a public authority, if so, executing S270, and otherwise, executing S2100.
And S270, responding to data access requests of other content providers by the unified management platform.
The data access request can be sent by other content providers except the target content provider to request to access the target data in the target data source of the target content provider.
In one particular example, a content provider of the entertainment lobe class may send a data access request to a data source corresponding to a content provider of the lovely pet gameplay lobe class.
And S280, judging whether the other content providers are other content providers with viewing rights, if so, executing S290, otherwise, executing S2100.
In the embodiment of the application, if the other content providers are other content providers with viewing rights, the other content providers can access the target data source and call the target data.
And S290, providing the stored target data to the other content providers for access.
And S2100, ending.
In a specific example, when logging in the unified management platform for data management, the content provider a sends a data access request to the unified management platform, and the unified management platform may first obtain, according to the identification information of the content provider a, data stored in correspondence with the identification information from the storage space and provide the data to the content provider a, that is, provide all data obtained from the data source selected to be accessed by the content provider a to the content provider a; in addition, the unified management platform also screens out shared data of which the viewing right is public and the content provider A has the viewing right from data stored corresponding to the identification information of other content providers, and provides the shared data to the unified management platform. Further, the content provider a can perform data processing, or distribution based on the two types of data provided by the unified management platform.
According to the technical scheme, the data sources of the multiple content providers are accessed into the unified management platform through the access module, the data in the data sources are stored in the storage space corresponding to the content providers, different viewing permissions are set for the data sources, and whether the data corresponding to the data sources can be accessed by other content providers is determined according to the viewing permissions. The problem that data sharing cannot be performed among data sources of cross-content providers due to the fact that the data sources of the content providers are independent of one another in the prior art is solved. The data sharing among the data sources of the cross-content providers is realized, the cost of the content providers for acquiring the data in the data sources corresponding to other content providers is reduced, and the working efficiency of the content providers is improved.
EXAMPLE III
Fig. 3 is a flowchart of a data processing method provided in the third embodiment of the present application, where the third embodiment of the present application adds a step of counting read-write states of each accessed data source, and performing online and offline processing on each data source or disconnecting access to a data source according to the read-write states.
Correspondingly, as shown in fig. 3, the technical solution of the embodiment of the present application specifically includes the following steps:
s310, responding to a data source access request of a target content provider, a unified management platform acquires an access type and an access parameter which are matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance.
S320, the unified management platform obtains a target data source access module matched with the access type from the data source access modules of multiple types.
And S330, the unified management platform triggers the target data source access module according to the access parameters to access the target data source matched with the data source access request.
S340, the unified management platform acquires target data from the target data source and stores the target data in a storage space matched with the target content provider.
And S350, the unified management platform counts the read-write state of each accessed data source.
The read-write state may be the number of times that the data source is read or written in the data within a certain time. The read-write state can reflect the use state of the data source, and within a period of time, the higher the read-write times of the data source, the higher the activity or the use rate of the data source is, and otherwise, the lower the activity or the use rate of the data source is.
And S360, the unified management platform judges whether the current data source is in an online state, if so, S370 is executed, and otherwise, S390 is executed.
The online state may be a storage state of data in the data source, which is an exposed storage, and the exposed storage may mean that the data in the data source is displayed to a user in the unified management platform. The offline state may be a data storage state in the data source, which is a hidden storage state, and the hidden storage state may mean that data in the data source is hidden in the unified management platform and cannot be acquired by a user.
In the embodiment of the application, the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.
And S370, the unified management platform judges whether the read-write times of the first data source in the online state in the first time interval are smaller than a first threshold, if so, S380 is executed, and if not, S3140 is executed.
In this embodiment of the application, if the first data source currently belongs to the online state, but the number of times of reading and writing in the first time interval is less than the first threshold, it may be stated that the activity or the utilization rate of the first data source is low, and the state of the first data source may be modified to the offline state, so as to improve the resource utilization rate.
The number of read/write times may be a sum of the number of read times and the number of write times, and an average value or a maximum value of the number of read times and the number of write times is obtained. The number of writing times of the data source reflects the number of new data appearing in the data source, the number of reading times of the data source reflects the number of times that data in the data source is used by other downstream modules, and when the number of reading and writing times is lower, the activity and the utilization rate of one data source are lower.
Typically, the unified management platform may monitor the read/write status of each data source in the platform in a set time unit (for example, month, week, day, hour, or the like) through a preset offline task.
And 380, changing the storage state of the first data corresponding to the first data source from exposed storage to hidden storage, so as to perform offline processing on the first data source.
In the embodiment of the application, if the activity or the usage rate of the first data source is low, the state of the first data source can be modified to be the offline state by modifying the state of the first data storage in the first data source from the exposed storage to the hidden storage.
The method has the advantages that the state of the data source with low utilization rate is intelligently switched, the resource waste is reduced, and the maintenance cost of the data source is reduced.
And S390, the unified management platform judges whether the reading and writing times of the second data source in the offline state in the second time interval are larger than or equal to a second threshold, if so, S3100 is executed, and if not, S3110 is executed.
In this embodiment of the application, if the second data source currently belongs to the offline state, but the number of times of reading and writing in the second time interval is greater than or equal to the second threshold, it may be stated that the activity or the utilization rate of the second data source is higher, and the state of the second data source may be modified to the online state, so as to improve the utilization rate of the data source.
S3100, the unified management platform changes the storage state of second data corresponding to a second data source from hidden storage to exposed storage, and online processing is conducted on the first data source.
In this embodiment of the application, if the second data source is in the offline state, but the activity or the usage rate is high, the state of the second data source may be modified to the online state by modifying the second data storage state in the second data source from the hidden storage to the exposed storage.
The method has the advantages that the state of the data source with low utilization rate is intelligently switched, and the utilization efficiency of the data source is improved.
S3110, judging whether the number of reading and writing times of the third data source in the offline state in the third time interval is smaller than a third threshold, if so, executing S3120, otherwise, executing S3140.
In this embodiment of the application, if the third data source currently belongs to the offline state, but the number of times of reading and writing in the third time interval is less than the third threshold, it may be stated that the third data source has less data accessed in the third time interval, or has less times of being accessed, a low usage rate, and occupies a storage space for a long time, which may cause resource waste.
In this embodiment, values of the first threshold, the second threshold, and the third threshold may be the same or different, which is not limited in this embodiment, and optionally, the second threshold may be set to be greater than the first threshold, and the first threshold may be greater than the third threshold.
It should be noted that the first time interval, the second time interval, and the third time interval may be set to be the same or different, and the specific sizes of the first time interval, the second time interval, and the third time interval are not limited in this embodiment of the application.
And S3120, disconnecting the access to the third data source.
In this embodiment of the application, if the third data source is in an offline state and the activity or the usage rate is low, the third data source may be disconnected from the unified management platform, and all data corresponding to the third data source may be deleted from the unified management platform.
The method has the advantages that the data source with low activity, long-time non-use or low utilization rate is stopped using in time, resource waste can be avoided, and the utilization efficiency of the storage space of the unified management platform is improved.
S3130, the unified management platform deleting all data corresponding to the third data source in the storage space synchronously.
And S3140, ending.
According to the technical scheme of the embodiment, the data sources of the plurality of content providers are accessed into the unified management platform through the access module, data in the data sources are stored in the storage space corresponding to the content providers, the read-write states of the accessed data sources are counted, and the states of the data sources are modified or the access of the data sources is disconnected according to the read-write states. The problem of in the prior art, the long-term storage space that occupies of data source that is not quoted even empty data source for a long time, the wasting of resources that causes to and the problem that artifical management data source is with high costs, inefficiency are solved. The automatic management of the online and offline states of the data source is realized, the invalid data source is timely recovered, the management efficiency of the data source is improved, and the management cost of the data source is saved.
It should be noted that, in addition to the unified management platform determining the detection of the read-write status of all the data sources accessed in the platform in a unified manner and performing the online or offline processing on the data sources, the content provider may independently select to perform the online or offline processing on one or more specified accessed data sources, and set a processing policy for the online or offline processing, for example, values of the first time interval, the second time interval, the first threshold, and the second threshold, and the like. And the unified management platform selects the monitored data source according to the content provider and the corresponding processing strategy, and performs online or offline processing on the data source meeting the requirements.
Example four
Fig. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present application, where the apparatus includes: an access type and access parameter acquisition module 410, a target data source access module acquisition module 420, and a target data source access module 430. Wherein:
an access type and access parameter obtaining module 410, configured to enable a unified management platform to respond to a data source access request of a target content provider, and obtain an access type and an access parameter that are matched with the data source access request, where the unified management platform is connected to multiple content providers in advance;
a target data source access module obtaining module 420, configured to enable the unified management platform to obtain, in multiple types of data source access modules, a target data source access module matching the access type;
a target data source access module 430, configured to enable the unified management platform to trigger the target data source access module according to the access parameter, and access a target data source matched with the data source access request;
according to the technical scheme of the embodiment of the application, the data sources of the content providers are accessed into the unified management platform through the access module. The problem of the data source mutual independence between each content provider in the prior art to the administrative and maintenance cost that causes is solved. The unified management platform can be used for carrying out unified maintenance and management on different data sources.
On the basis of the above embodiment, the apparatus further includes:
and the target data storage module is used for enabling the unified management platform to acquire target data from the target data source and store the target data in a storage space matched with the target content provider.
On the basis of the above embodiment, the apparatus further includes:
the viewing authority setting module is used for enabling the unified management platform to respond to an authority setting instruction of the target content provider and setting the viewing authority of the target data source to be a private authority or a public authority;
the data source set to the private authority only allows the content provider accessing the data source to access, and the data source set to the public authority also allows other content providers with viewing authority to access.
On the basis of the above embodiment, the apparatus further includes:
the data access request response module is used for enabling the unified management platform to respond to data access requests of other content providers;
and the target data providing and accessing module is used for enabling the unified management platform to provide the stored target data for other content providers to access if the other content providers are determined to be other content providers with viewing rights.
On the basis of the above embodiment, the apparatus further includes:
the read-write state counting module is used for enabling the unified management platform to count the read-write state of each accessed data source;
the data source processing module is used for enabling the unified management platform to carry out offline and/or online processing on each data source according to the read-write state;
the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.
On the basis of the above embodiment, the data source processing module includes:
the offline processing unit is used for changing the storage state of the first data corresponding to the first data source from exposed storage to hidden storage if the unified management platform determines that the read-write times of the first data source currently in the online state in a first time interval are smaller than a first threshold value, so as to perform offline processing on the first data source;
and the online processing unit is used for changing the storage state of the second data corresponding to the second data source from hidden storage to exposed storage if the unified management platform determines that the read-write times of the second data source in the offline state in the second time interval are greater than or equal to a second threshold value, so as to perform online processing on the first data source.
On the basis of the above embodiment, the apparatus further includes:
a data source access disconnection module, configured to, if it is determined that the number of times of reading and writing of a third data source currently in an offline state in a third time interval is smaller than a third threshold, disconnect access to the third data source by the unified management platform;
and the data deleting module is used for synchronously deleting all data corresponding to the third data source in the storage space by the unified management platform.
On the basis of the above embodiment, the access type includes at least one of the following:
the system comprises an application program interface push-pull access mode, a one-key mirror access mode, a file import access mode, a simple information aggregation access mode and a fifth generation hypertext markup language page access mode.
On the basis of the above embodiment, the content provider is a provider of a vertical service set in a website.
The data processing device provided by the embodiment of the application can execute the data processing method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 5, the electronic apparatus includes: one or more processors 501, memory 502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 5, one processor 501 is taken as an example.
The memory 502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the modules corresponding to the data processing method in the embodiment of the present application (for example, the access type and access parameter acquisition module 410, the target data source access module acquisition module 420, and the target data source access module 430 shown in fig. 4). The processor 501 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 502, that is, implements the data processing method in the above-described method embodiments.
The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the data-processing electronic device, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 502 optionally includes memory located remotely from processor 501, which may be connected to data processing electronics via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the data processing method may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 5 illustrates the connection by a bus as an example.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the data processing electronic apparatus, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input device. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (12)
1. A method of data processing, comprising:
the method comprises the steps that a unified management platform responds to a data source access request of a target content provider, and obtains an access type and an access parameter which are matched with the data source access request, wherein the unified management platform is connected with a plurality of content providers in advance;
the unified management platform acquires a target data source access module matched with the access type from a plurality of types of data source access modules;
and the unified management platform triggers the target data source access module according to the access parameters to access the target data source matched with the data source access request.
2. The method according to claim 1, wherein after the unified management platform triggers the target data source access module to access the target data source matching the data source access request according to the access parameter, the method further comprises:
and the unified management platform acquires target data from the target data source and stores the target data in a storage space matched with the target content provider.
3. The method of claim 2, further comprising, after the unified management platform accesses the target data source matching the data source access request:
the unified management platform responds to the authority setting instruction of the target content provider and sets the viewing authority of the target data source as a private authority or a public authority;
the data source set to the private authority only allows the content provider accessing the data source to access, and the data source set to the public authority also allows other content providers with viewing authority to access.
4. The method as claimed in claim 3, wherein after the unified management platform responds to the permission setting instruction of the target content provider to set the viewing permission of the target data source to the public permission, the method further comprises:
the unified management platform responds to data access requests of other content providers;
and if the unified management platform determines that the other content providers are other content providers with viewing rights, the stored target data is provided for the other content providers to access.
5. The method of claim 1, further comprising:
the unified management platform counts the read-write state of each accessed data source;
the unified management platform carries out offline and/or online processing on each data source according to the read-write state;
the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.
6. The method according to claim 5, wherein the unified management platform performs offline and/or online processing on each data source according to the read-write status, including:
if the unified management platform determines that the number of reading and writing times of a first data source which is currently in an online state in a first time interval is smaller than a first threshold, the unified management platform changes the storage state of first data corresponding to the first data source from exposed storage to hidden storage so as to perform offline processing on the first data source; and/or
And if the unified management platform determines that the number of reading and writing times of the second data source in the offline state in the second time interval is greater than or equal to a second threshold, the unified management platform changes the storage state of the second data corresponding to the second data source from hidden storage to exposed storage so as to perform online processing on the first data source.
7. The method according to claim 5, wherein after the unified management platform counts the read-write status of each accessed data source, the method further comprises:
if the unified management platform determines that the reading and writing times of a third data source currently in the offline state in a third time interval are smaller than a third threshold, the unified management platform disconnects the access to the third data source;
and the unified management platform synchronously deletes all data corresponding to the third data source in the storage space.
8. The method according to any of claims 1-7, wherein the access type comprises at least one of:
the system comprises an application program interface push-pull access mode, a one-key mirror access mode, a file import access mode, a simple information aggregation access mode and a fifth generation hypertext markup language page access mode.
9. The method according to any of claims 1-7, wherein the content provider is a provider of a set vertical service in a web site.
10. An apparatus for data processing, comprising:
the system comprises an access type and access parameter acquisition module, a data source access request acquisition module and a content source access request acquisition module, wherein the access type and access parameter acquisition module is used for enabling a unified management platform to respond to a data source access request of a target content provider and acquire an access type and an access parameter matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance;
the target data source access module acquisition module is used for enabling the unified management platform to acquire a target data source access module matched with the access type in a plurality of types of data source access modules;
and the target data source access module is used for triggering the target data source access module by the unified management platform according to the access parameters and accessing a target data source content provider matched with the data source access request.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319414.XA CN111090691B (en) | 2019-12-19 | 2019-12-19 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319414.XA CN111090691B (en) | 2019-12-19 | 2019-12-19 | Data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111090691A true CN111090691A (en) | 2020-05-01 |
| CN111090691B CN111090691B (en) | 2021-02-09 |
Family
ID=70396478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911319414.XA Active CN111090691B (en) | 2019-12-19 | 2019-12-19 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111090691B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000694A (en) * | 2020-09-11 | 2020-11-27 | 支付宝(杭州)信息技术有限公司 | Data acquisition method, device, equipment and medium |
| CN112738567A (en) * | 2020-12-22 | 2021-04-30 | 北京百度网讯科技有限公司 | Platform content processing method and device, electronic equipment and storage medium |
| CN112966179A (en) * | 2021-03-05 | 2021-06-15 | 北京百度网讯科技有限公司 | Information unified management method, device, equipment and storage medium |
| CN113923256A (en) * | 2020-06-22 | 2022-01-11 | 北京金山云网络技术有限公司 | Data source management method, device, server and system |
| CN116560579A (en) * | 2023-05-18 | 2023-08-08 | 上海威固信息技术股份有限公司 | Simulation data acquisition system based on multisource data fusion |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271548A1 (en) * | 2005-05-25 | 2006-11-30 | Oracle International Corporation | Personalization and recommendations of aggregated data not owned by the aggregator |
| US20070061328A1 (en) * | 2005-09-14 | 2007-03-15 | Jorey Ramer | Managing sponsored content for delivery to mobile communication facilities |
| US20070061244A1 (en) * | 2005-09-14 | 2007-03-15 | Jorey Ramer | Increasing mobile interactivity |
| CN101080037A (en) * | 2006-05-26 | 2007-11-28 | 泰利双星科技有限公司 | Method and device for preparing mobile content |
| CN101102315A (en) * | 2007-07-31 | 2008-01-09 | 中兴通讯股份有限公司 | Content provision method based on content verification |
| CN101110079A (en) * | 2007-06-27 | 2008-01-23 | 中国科学院遥感应用研究所 | Digital globe antetype system |
| CN101115225A (en) * | 2006-07-28 | 2008-01-30 | 中兴通讯股份有限公司 | System and method for implementing RSS subscription using multimedia message of mobile terminal |
| CN101242299A (en) * | 2007-11-09 | 2008-08-13 | 中国移动通信集团江苏有限公司 | Networking method for data hinge-service supporting system |
| CN101247344A (en) * | 2008-03-28 | 2008-08-20 | 中国电信股份有限公司 | Access method supporting multiple IPTV services platform and IPTV terminal unit |
| CN101493825A (en) * | 2008-12-17 | 2009-07-29 | 武汉中地数码科技有限公司 | GIS middleware collocation and management method |
| CN101539931A (en) * | 2009-04-21 | 2009-09-23 | 中国民航大学 | Flight dynamic information data integrator and processing method |
| CN103095710A (en) * | 2013-01-17 | 2013-05-08 | 北京交通大学 | Broadcast encryption transmission method in network based on identification and centering on contents |
| CN104572794A (en) * | 2013-10-12 | 2015-04-29 | 李一石 | Method and system for showing network information in a user-friendly manner |
| CN109582718A (en) * | 2018-10-17 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | Data processing method, device and storage medium |
| CN110431848A (en) * | 2017-03-24 | 2019-11-08 | 索尼公司 | Content providing system, content providing and program |
| CN110474917A (en) * | 2019-08-16 | 2019-11-19 | 深圳前海微众银行股份有限公司 | The upper and lower line method of message-oriented middleware, device, equipment and readable storage medium storing program for executing |
| CN110502563A (en) * | 2019-08-26 | 2019-11-26 | 腾讯科技（深圳）有限公司 | A kind of processing method and processing device of the data of multi-data source, storage medium |
-
2019
- 2019-12-19 CN CN201911319414.XA patent/CN111090691B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271548A1 (en) * | 2005-05-25 | 2006-11-30 | Oracle International Corporation | Personalization and recommendations of aggregated data not owned by the aggregator |
| US20070061328A1 (en) * | 2005-09-14 | 2007-03-15 | Jorey Ramer | Managing sponsored content for delivery to mobile communication facilities |
| US20070061244A1 (en) * | 2005-09-14 | 2007-03-15 | Jorey Ramer | Increasing mobile interactivity |
| CN101080037A (en) * | 2006-05-26 | 2007-11-28 | 泰利双星科技有限公司 | Method and device for preparing mobile content |
| CN101115225A (en) * | 2006-07-28 | 2008-01-30 | 中兴通讯股份有限公司 | System and method for implementing RSS subscription using multimedia message of mobile terminal |
| CN101110079A (en) * | 2007-06-27 | 2008-01-23 | 中国科学院遥感应用研究所 | Digital globe antetype system |
| CN101102315A (en) * | 2007-07-31 | 2008-01-09 | 中兴通讯股份有限公司 | Content provision method based on content verification |
| CN101242299A (en) * | 2007-11-09 | 2008-08-13 | 中国移动通信集团江苏有限公司 | Networking method for data hinge-service supporting system |
| CN101247344A (en) * | 2008-03-28 | 2008-08-20 | 中国电信股份有限公司 | Access method supporting multiple IPTV services platform and IPTV terminal unit |
| CN101493825A (en) * | 2008-12-17 | 2009-07-29 | 武汉中地数码科技有限公司 | GIS middleware collocation and management method |
| CN101539931A (en) * | 2009-04-21 | 2009-09-23 | 中国民航大学 | Flight dynamic information data integrator and processing method |
| CN103095710A (en) * | 2013-01-17 | 2013-05-08 | 北京交通大学 | Broadcast encryption transmission method in network based on identification and centering on contents |
| CN104572794A (en) * | 2013-10-12 | 2015-04-29 | 李一石 | Method and system for showing network information in a user-friendly manner |
| CN110431848A (en) * | 2017-03-24 | 2019-11-08 | 索尼公司 | Content providing system, content providing and program |
| CN109582718A (en) * | 2018-10-17 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | Data processing method, device and storage medium |
| CN110474917A (en) * | 2019-08-16 | 2019-11-19 | 深圳前海微众银行股份有限公司 | The upper and lower line method of message-oriented middleware, device, equipment and readable storage medium storing program for executing |
| CN110502563A (en) * | 2019-08-26 | 2019-11-26 | 腾讯科技（深圳）有限公司 | A kind of processing method and processing device of the data of multi-data source, storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113923256A (en) * | 2020-06-22 | 2022-01-11 | 北京金山云网络技术有限公司 | Data source management method, device, server and system |
| CN112000694A (en) * | 2020-09-11 | 2020-11-27 | 支付宝(杭州)信息技术有限公司 | Data acquisition method, device, equipment and medium |
| CN112738567A (en) * | 2020-12-22 | 2021-04-30 | 北京百度网讯科技有限公司 | Platform content processing method and device, electronic equipment and storage medium |
| CN112966179A (en) * | 2021-03-05 | 2021-06-15 | 北京百度网讯科技有限公司 | Information unified management method, device, equipment and storage medium |
| CN112966179B (en) * | 2021-03-05 | 2024-03-12 | 北京百度网讯科技有限公司 | Unified management method, device, equipment and storage medium for information |
| CN116560579A (en) * | 2023-05-18 | 2023-08-08 | 上海威固信息技术股份有限公司 | Simulation data acquisition system based on multisource data fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111090691B (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111090691B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111741026B (en) | Cross-chain transaction request processing method, device, equipment and storage medium | |
| US20180020070A1 (en) | Onsite display in a location-based messaging platform | |
| CN111770161B (en) | https sniffing jump method and device | |
| CN112000394B (en) | Method, apparatus, device and storage medium for accessing applet | |
| JP7194214B2 (en) | Data processing method and device, electronic device, storage medium, and computer program for mini appli | |
| CN109032587B (en) | Data stream management framework, method, terminal equipment and storage medium | |
| JP2021149963A (en) | Information processing method, device, electronic apparatus, and storage medium | |
| US11831735B2 (en) | Method and device for processing mini program data | |
| US11734454B2 (en) | Method for providing applet service capability, electronic device, and storage medium | |
| CN111913884A (en) | Distributed test method, device, equipment, system and readable storage medium | |
| CN111610972A (en) | Page generation method, device, equipment and storage medium | |
| CN110545324B (en) | Data processing method, device, system, network equipment and storage medium | |
| CN111600790A (en) | Block chain based message processing method, device, equipment and storage medium | |
| CN112015468A (en) | Interface document processing method and device, electronic equipment and storage medium | |
| CN111246305A (en) | Video preview method, device, equipment and storage medium | |
| US11711285B2 (en) | Method, apparatus, electronic device and readable storage medium for adjusting instance number | |
| CN112561332B (en) | Model management method, device, electronic equipment, storage medium and program product | |
| CN111770176B (en) | Traffic scheduling method and device | |
| CN112346612A (en) | Page display method and device | |
| CN112069137A (en) | Method and device for generating information, electronic equipment and computer readable storage medium | |
| CN111638982B (en) | Flow guiding method, flow guiding device and electronic equipment | |
| CN112752323A (en) | Method and device for changing hotspot access state | |
| CN111177558A (en) | Channel service construction method and device | |
| CN111273950A (en) | Application awakening method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |