CN109582718A - Data processing method, device and storage medium - Google Patents
Data processing method, device and storage medium Download PDFInfo
- Publication number
- CN109582718A CN109582718A CN201811208728.8A CN201811208728A CN109582718A CN 109582718 A CN109582718 A CN 109582718A CN 201811208728 A CN201811208728 A CN 201811208728A CN 109582718 A CN109582718 A CN 109582718A
- Authority
- CN
- China
- Prior art keywords
- data
- domain
- storage equipment
- metadata
- logical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The present invention provides a kind of data processing method, device and storage medium, this method comprises: receiving the metadata information at least one logical data domain that storage equipment is sent, the data stored in the storage equipment are divided at least one described logical data domain, and each logical data domain can be read and write by least one data-storage system；According to the metadata information at least one logical data domain, Data View is generated；Show the Data View.Data processing method, device and storage medium provided by the invention, it can improve according to the data that can be read and write by least one data-storage system in storage equipment, automatically generated data view and obtain the efficiency that Data View obtains, and the accuracy of Data View.
Description
Technical field
The present invention relates to data processing technique more particularly to a kind of data processing methods, device and storage medium.
Background technique
As cloud computing technology is increasingly mature, traditional IT infrastructure is gradually turned to cloud computing architecture by enterprise, with
Meeting mass memory needs read group total demand.Currently, common cloud computing architecture has publicly-owned cloud infrastructure and private clound
Infrastructure.It is not stranghtforward due to changing from traditional IT infrastructure to cloud computing architecture, so, some enterprises
Two sets of infrastructure of publicly-owned cloud infrastructure and private clound may be disposed simultaneously, it is also possible to will continue to retain some tradition IT bases
Infrastructure, to meet the diversity of the business demand of enterprise.Wherein, above-mentioned each infrastructure may include at least one storage
Equipment.
Data-storage system kind when enterprise uses the mode of above-mentioned this mixed cloud, on mixed cloud for storing data
Class is various, such as Hadoop/Spark big data system, S3 object data storage system, NOSQL database and MySQL/
Oracle relevant database etc. causes the data of enterprise to will disperse in different data-storage systems.This storing data
Mode, exacerbate the dispersibility and isomerism of data.
It is dispersed in due to data in different data-storage systems, in the Data View for summarizing enterprise, enterprise
Data management staff needs successively to access the metadata information that each data-storage system obtains data-storage system storage, then
The metadata information of each data-storage system is summarized into merging again, forms the Data View of enterprise.Above-mentioned acquisition Data View
Mode causes the efficiency for obtaining Data View acquisition lower, and accuracy is low.
Summary of the invention
The present invention provides a kind of data processing method, device and storage medium, can be according to can be by extremely in storage equipment
The data of few data-storage system read-write, automatically generated data view improve and obtain the efficiency that Data View obtains, with
And the accuracy of Data View.
The first aspect of the present invention provides a kind of data processing method, which comprises
The metadata information at least one logical data domain that storage equipment is sent is received, is stored in the storage equipment
Data are divided at least one described logical data domain, and each logical data domain can be by least one data-storage system
Read-write；
According to the metadata information at least one logical data domain, Data View is generated；
Show the Data View.
Optionally, the storage equipment is multiple, the method also includes:
The metadata information update instruction that the first storage equipment is sent is received, the metadata updates instruction includes: described
Metadata information after first logical data area update of the first storage equipment, the first storage equipment is multiple storages
Any storage equipment in equipment, first logical data domain are any logical data domain in the first storage equipment；
According to the metadata information after the first logical data area update, the Data View is updated.
Optionally, the method also includes:
The privileges configuration request that user inputs by operating the Data View is received, the privileges configuration request is for asking
Configuration target user is asked to access the access authority in the second logical data domain；
According to the privileges configuration request, the access right that the target user accesses second logical data domain is configured
Limit.
Optionally, the storage equipment is multiple, the method also includes:
It receives user and is requested by operating the metadata updates of the Data View input, the metadata updates request is used
In update target metadata；
It is requested according to the metadata updates, sends the metadata updates request at least one second storage equipment,
The second storage equipment is the storage equipment in the presence of the logical data domain for including target metadata.
Optionally, the storage equipment is multiple, the method also includes:
The inquiry request of user's input is received, the inquiry request is for requesting inquiry target data；
According to the inquiry request, the target logic data field where the target data is determined；
The inquiry request is sent to the storage equipment where the target logic data field；
The inquiry response that storage equipment where receiving the target logic data field is sent, the inquiry response include:
The target data；
Show the target data.
Optionally, the inquiry request further include: the mark of user, the storage where Xiang Suoshu target logic data field are set
Before preparation send the inquiry request, the method also includes:
According to the mark of the user, determine that the user has the permission for inquiring the data.
Optionally, the data in the logical data domain are to belong to the data of same business.
The second aspect of the present invention provides a kind of data processing method, which comprises
Obtain the metadata information at least one logical data domain of storage equipment, the data stored in the storage equipment
It is divided at least one described logical data domain, each logical data domain can be read by least one data-storage system
It writes；
Send the metadata information at least one logical data domain.
Optionally, the method also includes:
Metadata information after obtaining the first logical data area update of the storage equipment, first logical data domain
For any logical data domain in the storage equipment；
Metadata information update instruction is sent, the metadata updates instruction includes: the first logical data area update
Metadata information afterwards.
Optionally, the method also includes:
Metadata updates request is received, the metadata updates request is for updating target metadata；
It is requested according to the metadata updates, updates the target in the logical data domain for including the target metadata
Metadata.
Optionally, the method also includes:
Inquiry request is received, the inquiry request is for requesting inquiry target data；
According to the inquiry request, the target data is obtained；
Inquiry response is sent, the inquiry response includes: the target data.
Optionally, the data in the logical data domain are to belong to the data of same business.
The third aspect of the present invention provides a kind of data processing equipment, and described device includes:
Receiving module, it is described to deposit for receiving the metadata information at least one logical data domain that storage equipment is sent
The data stored in storage equipment are divided at least one described logical data domain, and each logical data domain can be by least one
A data-storage system read-write；
Processing module generates Data View for the metadata information according at least one logical data domain；
Display module, for showing the Data View.
Optionally, the storage equipment is multiple, the receiving module, is also used to receive the member of the first storage equipment transmission
Data information update instruction, the metadata updates instruction include: the first logical data area update of the first storage equipment
Metadata information afterwards, the first storage equipment are any storage equipment in multiple storage equipment, and described first patrols
Collecting data field is any logical data domain in the first storage equipment；
The processing module is also used to according to the metadata information after the first logical data area update, described in update
Data View.
Optionally, the receiving module is also used to receive the authority configuration that user inputs by operating the Data View
Request, the access authority that the privileges configuration request is used to that configuration target user to be requested to access the second logical data domain；
Processing module is also used to configure the target user according to the privileges configuration request and access second logic
The access authority of data field.
Optionally, the storage equipment is multiple, described device further include: sending module,
The receiving module is also used to receive user and is requested by operating the metadata updates of the Data View input,
The metadata updates request is for updating target metadata；
The sending module sends institute at least one second storage equipment for requesting according to the metadata updates
Metadata updates request is stated, the second storage equipment is the storage equipment in the presence of the logical data domain for including target metadata.
Optionally, the storage equipment is multiple, described device further include: sending module；
The receiving module is also used to receive the inquiry request of user's input, and the inquiry request is for requesting inquiry mesh
Mark data；
The processing module is also used to determine the target logic number where the target data according to the inquiry request
According to domain；
The sending module, for sending the inquiry request to the storage equipment where the target logic data field；
The receiving module, the inquiry that the storage equipment where being also used to receive the target logic data field is sent are rung
It answers, the inquiry response includes: the target data；
The display module is also used to show the target data.
Optionally, the processing module is also used in depositing where the sending module to the target logic data field
Before storing up the equipment transmission inquiry request, according to the mark of the user, determine that the user has the power for inquiring the data
Limit.
Optionally, the data in the logical data domain are to belong to the data of same business.
The fourth aspect of the present invention provides a kind of data processing equipment；Described device includes:
Processing module, the metadata information at least one logical data domain for obtaining storage equipment, the storage are set
The data of standby upper storage are divided at least one described logical data domain, and each logical data domain can be counted by least one
It is read and write according to storage system；
Sending module, for sending the metadata information at least one logical data domain.
Optionally, the processing module, the member after being also used to obtain the first logical data area update of the storage equipment
Data information, first logical data domain are any logical data domain in the storage equipment；
The sending module is also used to send metadata information update instruction, and the metadata updates instruction includes: described
Metadata information after first logical data area update.
Optionally, described device further include:
Receiving module, for receiving metadata updates request, the metadata updates request is for updating target metadata；
The processing module is also used to be requested according to the metadata updates, in the logic including the target metadata
The target metadata is updated in data field.
Optionally, described device further include:
Receiving module, for receiving inquiry request, the inquiry request is for requesting inquiry target data；
The processing module is also used to obtain the target data according to the inquiry request；
The sending module, is also used to send inquiry response, and the inquiry response includes: the target data.
Optionally, the data in the logical data domain are to belong to the data of same business.
The fifth aspect of the present invention provides a kind of data processing equipment, comprising: at least one processor, memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of the memory storage, so that the data processing
The method that device executes any one of above-mentioned first aspect.
The sixth aspect of the present invention provides a kind of data processing equipment, comprising: at least one processor, memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of the memory storage, so that the data processing
The method that device executes any one of above-mentioned second aspect.
The seventh aspect of the present invention provides a kind of computer readable storage medium, deposits on the computer readable storage medium
Computer executed instructions are contained, when the computer executed instructions are executed by processor, are realized described in any one of first aspect
Method.
The eighth aspect of the present invention provides a kind of computer readable storage medium, deposits on the computer readable storage medium
Computer executed instructions are contained, when the computer executed instructions are executed by processor, are realized described in any one of second aspect
Method.
Data processing method, device and storage medium provided by the invention, by the way that the data quilt stored in equipment will be stored
It is divided into the mode at least one logical data domain, the data summarization that can will be read and write by different data storage system is abstracted
At a virtual database, and then number can be automatically generated by the metadata information at least one logical data domain of acquisition
According to view, no longer need to raw by way of accessing each data-storage system and obtaining the metadata information that the data-storage system stores
At Data View, the acquisition efficiency of Data View and the accuracy of Data View are improved.
Detailed description of the invention
Fig. 1 is the application scenario diagram of data processing method provided by the invention；
Fig. 2 is a kind of flow diagram of data processing method provided by the invention；
Fig. 3 is the flow diagram of another data processing method provided by the invention；
Fig. 4 is the flow diagram of another data processing method provided by the invention；
Fig. 5 is the flow diagram of another data processing method provided by the invention；
Fig. 6 is the flow diagram of another data processing method provided by the invention；
Fig. 7 is a kind of structural schematic diagram of data processing equipment provided by the invention；
Fig. 8 is the structural schematic diagram of another data processing equipment provided by the invention；
Fig. 9 is the structural schematic diagram of another data processing equipment provided by the invention；
Figure 10 is the structural schematic diagram of another data processing equipment provided by the invention；
Figure 11 is a kind of structural schematic diagram of data management system provided by the invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with the embodiment of the present invention, to this
Technical solution in inventive embodiments is clearly and completely described, it is clear that described embodiment is that a part of the invention is real
Example is applied, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creation
Property labour under the premise of every other embodiment obtained, shall fall within the protection scope of the present invention.
In the prior art, the number when enterprise uses the mode of above-mentioned this mixed cloud, on mixed cloud for storing data
It is many kinds of according to storage system, such as Hadoop/Spark big data system, S3 object data storage system, NOSQL database
With MySQL/Oracle relevant database etc., the data of enterprise is caused to will disperse in different data-storage systems.It is this
The mode of storing data exacerbates the dispersibility and isomerism of data.
It is dispersed in due to data in different data-storage systems, in the Data View for summarizing enterprise, data pipe
Reason personnel need successively to access the metadata information that each data-storage system obtains data-storage system storage, then again will be each
The metadata information of data-storage system summarizes merging, forms the Data View of enterprise.The mode of above-mentioned acquisition Data View, leads
Cause the efficiency for obtaining Data View acquisition lower, and accuracy is low.
It, can be according to can be by storage equipment the present invention provides a kind of data processing method in view of the above problem
The data of at least one data-storage system read-write, automatically generated data view improve and obtain the efficiency that Data View obtains, with
And the accuracy of Data View.It is appreciated that data processing method provided by the present invention, including but not limited to above-mentioned use
The scene of the mode storing data of mixed cloud, as long as any scene using multiple data-storage system storing datas, all may be used
To generate Data View using method provided by the present invention.
Fig. 1 is the application scenario diagram of data processing method provided by the invention.As shown in Figure 1, number provided by the present invention
At least one logical data domain is divided into according to the data stored in storage equipment involved in processing method, wherein Mei Gesuo
Stating logical data domain can be read and write by least one data-storage system.
Storage equipment mentioned here can be storage equipment included by a kind of infrastructure, such as traditional basis IT
Facility, publicly-owned cloud infrastructure, privately owned cloud infrastructure.At least one the storage equipment is also possible to different infrastructure and is wrapped
The storage equipment included, such as aforementioned described mixing cloud mode.The quantity that equipment is stored involved in this system can basis
Included storage equipment determines in infrastructure.
Wherein, the present invention does not limit the size in above-mentioned logical data domain, and, division mode.For example, can be by same class
The data of type are divided into a logical data domain, alternatively, the data for belonging to same business can be divided into a logical number
According in domain.Data in same logical data domain, may belong to same data-storage system, also may belong to different data and deposit
Storage system.Each logical data domain size is identical, can also be different.Above-mentioned described data type can be database where data
Type, such as: MySQL, SQLServer, Hadoop etc., or the storage class of data, for example, XML, Json etc.,
It can specifically be determined according to system configuration.
By way of above-mentioned division logical data domain, it can will be converged by the data that different data storage system is read and write
It is always abstracted into a virtual database, so as to handle according to identical operating habit and mode of operation data,
Without being handled according to the operating habit and mode of operation of different data-storage systems data, it is stored in shielding data
The isomerism of different data storage system.It is appreciated that heretofore described capable of being read and write by data-storage system can be managed
Solution is that the data belong to the data-storage system, it is understood that can pass through the data storage system administration etc. for the data.
Technical solution of the present invention is described in detail below with reference to specifically embodiment.These are specific real below
Applying example can be combined with each other, and the same or similar concept or process may be repeated no more in some embodiments.
Fig. 2 is a kind of flow diagram of data processing method provided by the invention.Executing subject of the invention can be
Data processing equipment, data processing equipment mentioned here can be any device with processing function, for example, terminal is set
Standby, server, computer etc..As shown in Fig. 2, this method may include:
S101, the metadata information for receiving at least one logical data domain that storage equipment is sent, store and store in equipment
Data be divided at least one logical data domain, each logical data domain can be read and write by least one data-storage system.
S102, according to the metadata information at least one logical data domain, generate Data View.
S103, display Data View.
Metadata information according to the present invention can be the information of description data attribute, can characterize corresponding logical number
According to the attribute information for the data that domain is stored.For example, above-mentioned metadata information may include: belonging to data in logical data domain
Service line, data type, the data physical location (position i.e. where storage equipment), the word included by data that are stored
Section etc..It is appreciated that content included by above-mentioned metadata information specifically can be added and be deleted according to the demand of user.
In the present embodiment, it is each storage equipment it is available oneself each logical data domain metadata information, and
It is sent to data processing equipment.Illustratively, storage equipment can traverse the data in each logical data domain of oneself, to obtain
It is derived from the metadata information in oneself each logical data domain.
In the present embodiment, metadata of the data processing equipment in each logical data domain for getting each storage equipment
After information, data processing equipment can summarize rule according to preset, carry out to the metadata information in all logical data domains
Summarize, generate and shows the Data View.In this way, data management staff can be known by the Data View and be dispersed in difference
Data-storage system in data global information, no longer need to obtain data storage system by accessing each data-storage system
The mode of metadata information of storage of uniting generates Data View, improves the acquisition efficiency and Data View of Data View
Accuracy.
The present embodiment does not limit the form of expression of above-mentioned Data View.For example, above-mentioned Data View can be form
Data View, can also be map view Data View, can also be bar graph form Data View etc..It can manage
Solution can more intuitively reflect the data that all data-storage systems are stored when using the Data View of map view
Status.
By above-mentioned data-storage system storage be business data, Data View for the Data View of form for,
It is assumed that the preset rule that summarizes is summarizes according to type of service, then data processing equipment is in the member for getting each logical data domain
After data information, Classifying Sum can be carried out to the metadata information in all logical data domains, generated as follows according to type of service
State Data View shown in table 1.In this way, data management staff can by the Data View, global every kind of business of perspective
The distribution of data, consequently facilitating data management staff can be analyzed and be handled to business data based on the Data View.
Table 1
For another example assuming the preset rule that summarizes to summarize according to Regional Distribution, then data processing equipment is being got often
After the metadata information in a logical data domain, the physical location that can be stored according to data, to the member in all logical data domains
Data information carries out Classifying Sum, generates Data View as described in Table 2.In this way, data management staff can be by this
Data View, the distribution of global perspective business data, consequently facilitating data management staff can be based on the Data View to enterprise
Industry data are analyzed and are handled.
Table 2
It is appreciated that Data View shown in above-mentioned Tables 1 and 2 is only a kind of signal, the present invention is not to Data View
Structure and layout be defined.
Data processing method provided by the invention is divided at least one by will store the data that store in equipment and patrols
The mode of data field is collected, the data summarization that can will be read and write by different data storage system is abstracted into a virtual data
Library, and then can no longer need to lead to by the metadata information automatically generated data view at least one logical data domain of acquisition
It crosses to access each data-storage system and obtain the mode of metadata information of data-storage system storage and generates Data View, improve
The acquisition efficiency of Data View and the accuracy of Data View.
Further, on the basis of the above embodiments, when a certain in a certain storage equipment in multiple storage equipment
When metadata information in logical data domain changes, the change of the metadata information can be synchronized to data by storage equipment
Processing unit, the Data View so that data processing equipment can timely update, to ensure the accuracy of Data View.Below with more
For the first storage equipment in a storage equipment, this method process is illustrated.
Fig. 3 is the flow diagram of another data processing method provided by the invention.As shown in figure 3, further,
On the basis of above-described embodiment, this method can also include:
S201, the first storage equipment obtain the metadata information after the first logical data area update.
Wherein, the first logical data domain is any logical data domain in the first storage equipment.
S202, the first storage equipment send metadata information update instruction to data processing equipment.
Metadata updates instruction includes: the metadata information after the first logical data area update of the first storage equipment.
S203, data processing equipment update Data View according to the metadata information after the first logical data area update.
Specifically, where the data in a certain logical data domain (i.e. the first logical data domain) of the first storage equipment
When the corresponding metadata information of data-storage system changes, the metadata information that will lead to the logical data domain becomes
Change.For example, being that the data that can be read and write by data-storage system A are with the data in the first logical data domain of the first storage equipment
Example, when the corresponding metadata information of data-storage system A changes, (such as user deletes a table or increases one
Open table or increase several fields etc.) when, the metadata information in the logical data domain also adjusts accordingly, to keep unified.
In the present embodiment, first storage equipment can track oneself each logical data domain metadata information whether
It changes.Illustratively, the first storage equipment can traverse the number in oneself each logical data domain periodically or in real time
According to obtain oneself current metadata information in each logical data domain.Then, data processing equipment can be by each logical number
The metadata information in the logical data domain stored according to the current metadata information in domain with oneself is compared, if two first numbers
It is believed that breath is different, then illustrate that the metadata information in the logical data domain is changed.
When the metadata information in (i.e. the first logical data domain) in a certain logical data domain changes, the first storage is set
It is standby to send metadata information update instruction to data processing equipment.Wherein, metadata updates instruction includes: the first storage equipment
The first logical data area update after metadata information.Correspondingly, data processing equipment, can after getting the update instruction
Data View is updated, to ensure the accuracy of Data View according to the metadata information after the logical data area update.
Data processing method provided by the invention, can be in the metadata letter in the first logical data domain of the first storage equipment
When breath changes, based on the metadata information after the first logical data area update, Data View is updated, it is ensured that data view
The accuracy of figure.
In the prior art, since the management mode and application method of each data-storage system are different from, therefore, it is necessary to
The permission that user is configured for each data-storage system, causes complicated for operation cumbersome.In addition, due to each data-storage system pipe
Reason mode is different, is difficult to carry out unified data control strategy, easily leads to and the case where access authority configuration disorder, serious feelings occur
Condition will lead to leaking data, and cause major accident.
Fig. 4 is the flow diagram of another data processing method provided by the invention.As shown in figure 4, further,
On the basis of above-described embodiment, this method can also include:
S301, the privileges configuration request that user is inputted by operation data view is received, privileges configuration request is for requesting
Configure the access authority that target user accesses the second logical data domain.
S302, according to privileges configuration request, configure the access authority that target user accesses the second logical data domain.
It in the present embodiment, can be each access user configuration access authority using logical data domain as granularity.That is, right
In same user, regardless of the data in logical data domain can be read and write by several data-storage systems, in access, this is patrolled the user
When collecting the data in data field, access authority all having the same.It is appreciated that above-mentioned described access authority for example can be
Configuration access strategy increases access authority or the permission that calls off a visit etc..
When specific implementation, data management staff can input privileges configuration request by operation data view.For example, data
Administrative staff can be by choosing a certain logical data domain (i.e. the second logical data domain) in Data View, and inputs or choose
The mark of user corresponding with the type of service, and, access authority corresponding with the type of service is triggered and is filled to data processing
Set input privileges configuration request.
Optionally, data management staff can also be by choosing a certain type (such as type of service, data type, region
Distribution etc.), for the logical data configuration of territory access authority where data corresponding to a certain type, to improve permission in batches and match
Set efficiency.It is appreciated that aforesaid way is only the signal of a kind of pair of batch configuration access authority.Data management staff can also adopt
In other ways, it accesses the configuration of permission to logical data domain in batches, to this without limiting.
Data processing method provided by the invention can be asked based on user by the authority configuration that operation data view inputs
It asks, configuration target user accesses the access authority of data in logical data domain.In this way, the system to data may be implemented
One management is effectively avoided without the dispersibility and isomerism of focused data because needing that use is respectively configured to each data-storage system
Family access authority, easily occur access authority configuration disorder caused by leaking data the problem of.
Fig. 5 is the flow diagram of another data processing method provided by the invention.As shown in figure 5, when storage equipment
When having multiple, further, on the basis of the above embodiments, this method can also include:
S401, the metadata updates request that user is inputted by operation data view is received, metadata updates request is used for
Update target metadata.
S402, it is requested according to metadata updates, sends metadata updates at least one second storage equipment and request, second
Storage equipment is the storage equipment in the presence of the logical data domain for including target metadata.
Specifically, when data management staff needs to modify to a certain metadata, for example, increasing a table, deleting
One table increases several fields etc., and data management staff can be by executing the operation, with to data processing in Data View
Device inputs metadata updates request.Data processing equipment, can be at least one after receiving metadata updates request
Second storage equipment sends metadata updates request.Wherein, the second storage equipment mentioned here is to exist including the target
The storage equipment in the logical data domain of metadata.
Correspondingly, the second storage equipment can request after receiving metadata updates request according to metadata updates,
The target metadata is updated in the logical data domain for including the target metadata.In this way, data management staff without
It need to go on each data-storage system where the metadata to carry out metadata updates again, improve the efficiency of metadata updates.
For example, when data management staff is added to a table to a certain metadata in Data View, can trigger to
Data processing equipment inputs metadata updates request, adds a table to the metadata with request.At this point, the metadata is mesh
Mark metadata.The metadata updates can be requested to be sent to by data processing equipment after receiving metadata updates request
In the presence of the storage equipment in the logical data domain for including the target metadata.It is accordingly, there are the logic including the target metadata
The storage equipment of data field is requested according to metadata updates, is the target element in the logical data domain for including the target metadata
Data increase a table.
Data processing method provided by the invention, the metadata updates that can be inputted based on user by operation data view
Request, updates the metadata in all logical data domains including the metadata.In this way, it may be implemented to data
Unified management improve the efficiency of metadata updates without the dispersibility and isomerism of focused data.
In the prior art, since the application method of each data-storage system is different, data management staff needs to learn
The application method for practising and grasping each data-storage system causes learning cost very high.Therefore, data pipe according to the present invention
Reason system can also provide unified access entrance for data management staff, without individually accessing a certain data-storage system
Data are accessed, learning cost is reduced.
Fig. 6 is the flow diagram of another data processing method provided by the invention.As shown in fig. 6, further,
On the basis of above-described embodiment, this method can also include:
S501, the inquiry request for receiving user's input, inquiry request is for requesting inquiry target data.
S502, according to inquiry request, determine the target logic data field where target data.
S503, inquiry request is sent to the storage equipment where target logic data field.
The inquiry response that storage equipment where S504, reception target logic data field is sent, which includes: mesh
Mark data.
S505, displaying target data.
Specifically, in the present embodiment, data processing equipment can provide unified access entrance (such as access interface
Deng), so that data management staff can request to data processing equipment input inquiry.The inquiry request can carry mesh
The mark for marking data, for example, the storage location of target data and/or the logical data domain where target data etc..Data processing
Device can carry out meaning of one's words understanding to the inquiry request, to determine target data after the inquiry request for receiving user's input
The target logic data field at place.
Then, data processing equipment can send inquiry request to the storage equipment where target logic data field.Accordingly
Ground, the storage equipment can be based on the inquiry request, search the mesh in the target logic domain after receiving the inquiry request
Data are marked, and the target data inquired carrying is sent to data processing equipment in inquiry response.
Optionally, in some embodiments, above-mentioned inquiry request further include: the mark of user, then data processing equipment to
Before storage equipment where target logic data field sends inquiry request, it can also determine that user has according to the mark of user
There is the permission for inquiring the data.In this way, it can be ensured that the safety of data avoids leaking data.
Data processing method provided by the invention, the inquiry request that can be inputted based on user, provides the inquiry for user
Request inquired target data.In this way, unified access entrance can be provided for data management staff, without single
Data can be accessed by solely accessing a certain data-storage system, reduce learning cost.
It should be noted that above-described embodiment is for storing and directly be communicated between equipment and data processing equipment
Data processing method provided by the invention is illustrated.But in some embodiments, some storage equipment and data
The data format supported between processing unit may be different, for example, the data format that some storages equipment is supported is .xml,
The data format that data processing equipment is supported is .xls.It therefore, can be between above-mentioned data processing equipment and storage equipment
It is provided with converter, the format that the data conversion for data processing equipment to be sent to storage equipment is supported at storage equipment,
Equipment will be stored and be sent to the format that the data conversion of data processing equipment is supported at data processing equipment, consequently facilitating place
Reason, improves efficiency.
Fig. 7 is a kind of structural schematic diagram of data processing equipment provided by the invention.As shown in fig. 7, the data processing fills
Set may include: receiving module 11, processing module 12 and display module 13.Optionally, which can also include
Sending module 14.Wherein,
Receiving module 11, it is described for receiving the metadata information at least one logical data domain that storage equipment is sent
The data stored in storage equipment are divided at least one described logical data domain, and each logical data domain can be by least
One data-storage system read-write；
Processing module 12 generates Data View for the metadata information according at least one logical data domain；
Display module 13, for showing the Data View.
Optionally, in some embodiments, the storage equipment is multiple, the receiving module 11, is also used to receive the
The metadata information update instruction that one storage equipment is sent, the metadata updates instruction include: the first storage equipment
Metadata information after first logical data area update, the first storage equipment are any depositing in multiple storage equipment
Equipment is stored up, first logical data domain is any logical data domain in the first storage equipment；The processing module 12,
It is also used to update the Data View according to the metadata information after the first logical data area update.
Optionally, in some embodiments, the receiving module 11 is also used to receive user by operating the data view
Scheme the privileges configuration request of input, the privileges configuration request is used to that configuration target user to be requested to access the second logical data domain
Access authority；Processing module 12 is also used to configure target user's access described second according to the privileges configuration request and patrol
Collect the access authority of data field.
Optionally, in some embodiments, the storage equipment is multiple, and the receiving module 11 is also used to receive use
Family is requested by operating the metadata updates of the Data View input, and the metadata updates request is for updating target element number
According to；The sending module 14 sends the member at least one second storage equipment for requesting according to the metadata updates
Data update request, and the second storage equipment is the storage equipment in the presence of the logical data domain for including target metadata.
Optionally, in some embodiments, the storage equipment is multiple, and the receiving module 11 is also used to receive use
The inquiry request of family input, the inquiry request is for requesting inquiry target data；The processing module 12, is also used to according to institute
Inquiry request is stated, determines the target logic data field where the target data；Sending module 14 is used for the target logic
Storage equipment where data field sends the inquiry request；The receiving module 11 is also used to receive the target logic number
According to the inquiry response that the storage equipment where domain is sent, the inquiry response includes: the target data；The display module
13, it is also used to show the target data.
Optionally, in some embodiments, the processing module 12 is also used in the sending module 14 to the target
Before storage equipment where logical data domain sends the inquiry request, according to the mark of the user, the user is determined
With the permission for inquiring the data.
Optionally, in some embodiments, the data in the logical data domain are to belong to the data of same business.
The movement of data processing equipment in above method embodiment may be implemented in data processing equipment provided by the invention,
The principle of realization is similar with technical effect, and therefore not to repeat here.
Fig. 8 is the structural schematic diagram of another data processing equipment provided by the invention.As shown in figure 8, the data processing
Device may include: processing module 21 and sending module 22.Optionally, which can also include receiving module 23.Its
In,
Processing module 21, the metadata information at least one logical data domain for obtaining storage equipment, the storage
The data stored in equipment are divided at least one described logical data domain, and each logical data domain can be by least one
Data-storage system read-write；
Sending module 22, for sending the metadata information at least one logical data domain.
Optionally, in some embodiments, the processing module 21 is also used to obtain the first logic of the storage equipment
The updated metadata information of data field, first logical data domain are any logical data domain in the storage equipment；
The sending module 22, is also used to send metadata information update instruction, and the metadata updates instruction includes: described first to patrol
Collect the updated metadata information of data field.
Optionally, in some embodiments, the receiving module 23, for receiving metadata updates request, the member number
It requests according to updating for updating target metadata；The processing module 21 is also used to be requested according to the metadata updates, wrap
It includes in the logical data domain of the target metadata and updates the target metadata.
Optionally, in some embodiments, the receiving module 23, for receiving inquiry request, the inquiry request is used
Target data is inquired in request；The processing module 21 is also used to obtain the target data according to the inquiry request；Institute
Sending module 22 is stated, is also used to send inquiry response to the data processing equipment, the inquiry response includes: the number of targets
According to.
Optionally, in some embodiments, the data in the logical data domain are to belong to the data of same business.
The movement that equipment side is stored in above method embodiment may be implemented in data processing equipment provided by the invention, in fact
Existing principle is similar with technical effect, and therefore not to repeat here.
Fig. 9 is the structural schematic diagram of another data processing equipment provided by the invention, which for example may be used
To be terminal device (such as smart phone, tablet computer, computer etc.), server can also be.As shown in figure 9, the data
Processing unit includes: memory 31, at least one processor 32 and communication interface 33.
Communication interface 33, for storage equipment communicated, execute preceding method embodiment involved in send or
Reception movement.
Memory 31, for storing program instruction.
Processor 32, for being performed the data processing method realized in the embodiment of the present invention in program instruction, specifically
Realization principle can be found in above-described embodiment, and details are not described herein again for the present embodiment.
The data processing equipment can also include and input/output interface 34.
Input/output interface 34 may include independent output interface and input interface, or integrated input and defeated
Integrated interface out.Wherein, output interface is used for output data, and input interface is used to obtain the data of input, above-mentioned output
Data are the general designation exported in above method embodiment, and the data of input are the general designation inputted in above method embodiment.
Figure 10 is the structural schematic diagram of another data processing equipment provided by the invention, which for example may be used
To be storage equipment.As shown in Figure 10, which may include: memory 41, at least one processor 42 and leads to
Believe interface 43.
Communication interface 43 executes in preceding method embodiment for being communicated with data processing equipment shown in Fig. 9
It is related to send or receive movement.
Memory 41, for storing program instruction.
Processor 42, for being performed the data processing method realized in the embodiment of the present invention in program instruction, specifically
Realization principle can be found in above-described embodiment, and details are not described herein again for the present embodiment.
The data processing equipment can also include and input/output interface 44.
Input/output interface 44 may include independent output interface and input interface, or integrated input and defeated
Integrated interface out.Wherein, output interface is used for output data, and input interface is used to obtain the data of input, above-mentioned output
Data are the general designation exported in above method embodiment, and the data of input are the general designation inputted in above method embodiment.
Figure 11 is a kind of structural schematic diagram of data management system provided by the invention, and as shown in figure 11, which includes
At least one storage equipment 51 and data processing equipment 52.Figure 11 is the data management system by taking one stores equipment as an example.
Wherein, data processing equipment 52, for executing the movement of data processing equipment in preceding method embodiment；Storage is set
Standby 51, for executing the movement for storing equipment in preceding method embodiment.
The present invention also provides a kind of readable storage medium storing program for executing, it is stored with computer execution on the computer readable storage medium and refers to
It enables, when computer executed instructions are executed by processor, realizes the movement of the data processing equipment side in above method embodiment.
The present invention also provides a kind of readable storage medium storing program for executing, it is stored with computer execution on the computer readable storage medium and refers to
It enables, when computer executed instructions are executed by processor, realizes the movement of the storage equipment side in above method embodiment.
The present invention also provides a kind of program product, the program product include execute instruction, this execute instruction be stored in it is readable
In storage medium.At least one processor of data processing equipment can read this from readable storage medium storing program for executing and execute instruction, at least
One processor executes this and executes instruction so that data processing equipment implements the data processing equipment side in above method embodiment
Movement.
The present invention also provides a kind of program product, the program product include execute instruction, this execute instruction be stored in it is readable
In storage medium.At least one processor of data processing equipment can read this from readable storage medium storing program for executing and execute instruction, at least
One processor executes this and executes instruction so that data processing equipment implements the dynamic of the storage equipment side in above method embodiment
Make.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or unit
Letter connection can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one
In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) or processor (English: processor) execute this hair
The part steps of bright each embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory
(English: Read-Only Memory, abbreviation: ROM), random access memory (English: Random Access Memory, letter
Claim: RAM), the various media that can store program code such as magnetic or disk.
In the embodiment of the above-mentioned network equipment or terminal device, it should be appreciated that processor can be central processing unit
(English: Central Processing Unit, referred to as: CPU), it can also be other general processors, digital signal processor
(English: Digital Signal Processor, abbreviation: DSP), specific integrated circuit (English: Application
Specific Integrated Circuit, referred to as: ASIC) etc..General processor can be microprocessor or the processor
It is also possible to any conventional processor etc..Hardware handles can be embodied directly in conjunction with the step of method disclosed in the present application
Device executes completion, or in processor hardware and software module combination execute completion.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (18)
1. a kind of data processing method, which is characterized in that the described method includes:
Receive the metadata information at least one logical data domain that storage equipment is sent, the data stored in the storage equipment
It is divided at least one described logical data domain, each logical data domain can be read by least one data-storage system
It writes；
According to the metadata information at least one logical data domain, Data View is generated；
Show the Data View.
2. the method according to claim 1, wherein the storage equipment be it is multiple, the method also includes:
The metadata information update instruction that the first storage equipment is sent is received, the metadata updates instruction includes: described first
Metadata information after storing the first logical data area update of equipment, the first storage equipment is multiple storage equipment
In any storage equipment, first logical data domain be it is described first storage equipment in any logical data domain；
According to the metadata information after the first logical data area update, the Data View is updated.
3. the method according to claim 1, wherein the method also includes:
The privileges configuration request that user inputs by operating the Data View is received, the privileges configuration request is matched for requesting
Set the access authority that target user accesses the second logical data domain；
According to the privileges configuration request, the access authority that the target user accesses second logical data domain is configured.
4. the method according to claim 1, wherein the storage equipment be it is multiple, the method also includes:
It receives user and is requested by operating the metadata updates of the Data View input, the metadata updates request is for more
Fresh target metadata；
It is requested according to the metadata updates, sends the metadata updates request at least one second storage equipment, it is described
Second storage equipment is the storage equipment in the presence of the logical data domain for including target metadata.
5. the method according to claim 1, wherein the storage equipment be it is multiple, the method also includes:
The inquiry request of user's input is received, the inquiry request is for requesting inquiry target data；
According to the inquiry request, the target logic data field where the target data is determined；
The inquiry request is sent to the storage equipment where the target logic data field；
The inquiry response that storage equipment where receiving the target logic data field is sent, the inquiry response include: described
Target data；
Show the target data.
6. according to the method described in claim 5, it is characterized in that, the inquiry request further include: the mark of user, Xiang Suoshu
Before storage equipment where target logic data field sends the inquiry request, the method also includes:
According to the mark of the user, determine that the user has the permission for inquiring the data.
7. method according to claim 1-6, which is characterized in that the data in the logical data domain are to belong to
The data of same business.
8. a kind of data processing method, which is characterized in that the described method includes:
The metadata information at least one logical data domain of storage equipment is obtained, the data stored in the storage equipment are drawn
It is divided at least one described logical data domain, each logical data domain can be read and write by least one data-storage system；
Send the metadata information at least one logical data domain.
9. according to the method described in claim 8, it is characterized in that, the method also includes:
Metadata information after obtaining the first logical data area update of the storage equipment, first logical data domain is institute
State any logical data domain in storage equipment；
Metadata information update instruction is sent, after metadata updates instruction includes: the first logical data area update
Metadata information.
10. according to the method described in claim 8, it is characterized in that, the method also includes:
Metadata updates request is received, the metadata updates request is for updating target metadata；
It is requested according to the metadata updates, updates the target element number in the logical data domain for including the target metadata
According to.
11. according to the method described in claim 8, it is characterized in that, the method also includes:
Inquiry request is received, the inquiry request is for requesting inquiry target data；
According to the inquiry request, the target data is obtained；
Inquiry response is sent, the inquiry response includes: the target data.
12. according to the described in any item methods of claim 8-11, which is characterized in that the data in the logical data domain are to belong to
In the data of same business.
13. a kind of data processing equipment, which is characterized in that described device includes:
Receiving module, for receiving the metadata information at least one logical data domain that storage equipment is sent, the storage is set
The data of standby upper storage are divided at least one described logical data domain, and each logical data domain can be counted by least one
It is read and write according to storage system；
Processing module generates Data View for the metadata information according at least one logical data domain；
Display module, for showing the Data View.
14. a kind of data processing equipment, which is characterized in that described device includes:
Processing module, the metadata information at least one logical data domain for obtaining storage equipment, in the storage equipment
The data of storage are divided at least one described logical data domain, and each logical data domain can be deposited by least one data
Storage system read-write；
Sending module, for sending the metadata information at least one logical data domain.
15. a kind of data processing equipment characterized by comprising at least one processor, memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of the memory storage, so that the data processing equipment
Execute any one of the claims 1-7 method.
16. a kind of data processing equipment characterized by comprising at least one processor, memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of the memory storage, so that the data processing equipment
Execute any one of the claims 8-12 method.
17. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
It executes instruction, when the computer executed instructions are executed by processor, realizes the described in any item methods of claim 1-7.
18. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
It executes instruction, when the computer executed instructions are executed by processor, realizes the described in any item methods of claim 8-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208728.8A CN109582718B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208728.8A CN109582718B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109582718A true CN109582718A (en) | 2019-04-05 |
| CN109582718B CN109582718B (en) | 2021-05-04 |
Family
ID=65920126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811208728.8A Active CN109582718B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109582718B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111090691A (en) * | 2019-12-19 | 2020-05-01 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060206507A1 (en) * | 2005-02-16 | 2006-09-14 | Dahbour Ziyad M | Hierarchal data management |
| CN101217571A (en) * | 2008-01-18 | 2008-07-09 | 清华大学 | Write/read document operation method applied in multi-copy data grid system |
| US20130179982A1 (en) * | 2012-01-09 | 2013-07-11 | Ezshield, Inc. | Data Processing Engine System And Method |
| CN103797770A (en) * | 2012-12-31 | 2014-05-14 | 华为技术有限公司 | Method and system for sharing storage resources |
| CN104866513A (en) * | 2014-02-26 | 2015-08-26 | 国际商业机器公司 | System and method for cross-tenant data access |
| CN105915588A (en) * | 2016-04-06 | 2016-08-31 | 易云捷讯科技（北京）股份有限公司 | Hybrid cloud computing management system based on data virtualization |
-
2018
- 2018-10-17 CN CN201811208728.8A patent/CN109582718B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060206507A1 (en) * | 2005-02-16 | 2006-09-14 | Dahbour Ziyad M | Hierarchal data management |
| CN101217571A (en) * | 2008-01-18 | 2008-07-09 | 清华大学 | Write/read document operation method applied in multi-copy data grid system |
| US20130179982A1 (en) * | 2012-01-09 | 2013-07-11 | Ezshield, Inc. | Data Processing Engine System And Method |
| CN103797770A (en) * | 2012-12-31 | 2014-05-14 | 华为技术有限公司 | Method and system for sharing storage resources |
| CN104866513A (en) * | 2014-02-26 | 2015-08-26 | 国际商业机器公司 | System and method for cross-tenant data access |
| CN105915588A (en) * | 2016-04-06 | 2016-08-31 | 易云捷讯科技（北京）股份有限公司 | Hybrid cloud computing management system based on data virtualization |
Non-Patent Citations (2)
| Title |
|---|
| KE Y: "Application of adaptive neuro-fuzzy inference system based on data field clustering in load forecasting", 《INTERNATIONAL CONFERENCE ON COMPUTER APPLICATION & SYSTEM MODELING》 * |
| 蒋秉川: "网络空间剖分机理与可视化方法研究", 《系统仿真学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111090691A (en) * | 2019-12-19 | 2020-05-01 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111090691B (en) * | 2019-12-19 | 2021-02-09 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109582718B (en) | 2021-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220188332A1 (en) | Distributed transaction database log with immediate reads and batched writes | |
| CN103246749B (en) | The matrix database system and its querying method that Based on Distributed calculates | |
| CN104113597B (en) | The HDFS data read-write method of a kind of many Data centres | |
| US10733172B2 (en) | Method and computing device for minimizing accesses to data storage in conjunction with maintaining a B-tree | |
| US20110022582A1 (en) | Indexing of Partitioned External Data Sources | |
| US10127272B2 (en) | Modifying a database query | |
| US10146814B1 (en) | Recommending provisioned throughput capacity for generating a secondary index for an online table | |
| DE102013111436A1 (en) | Sharing data between Nexus using different data-access-control classification schemes | |
| CN103605698A (en) | Cloud database system used for distributed heterogeneous data resource integration | |
| CN107026876B (en) | A kind of file data access system and method | |
| CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
| US10013449B1 (en) | Validating and non-validating secondary indexes for a table in a non-relational data store | |
| CN108509437A (en) | A kind of ElasticSearch inquiries accelerated method | |
| CN111221791A (en) | Method for importing multi-source heterogeneous data into data lake | |
| CN107832392A (en) | A kind of metadata management system | |
| CN107111720B (en) | Method, medium, device and apparatus for controlling access to an object | |
| CN103646051A (en) | Big-data parallel processing system and method based on column storage | |
| CN108363741B (en) | Big data unified interface method, device, equipment and storage medium | |
| CN113986830A (en) | Distributed CT-oriented cloud data management and task scheduling method and system | |
| US9898614B1 (en) | Implicit prioritization to rate-limit secondary index creation for an online table | |
| US20140172965A1 (en) | Unified social graph | |
| CN109582718A (en) | Data processing method, device and storage medium | |
| Chao et al. | Intelligent interaction design research based on block chain communication technology and fuzzy system | |
| Alaerjan et al. | Modeling functional behaviors of DDS | |
| US11789971B1 (en) | Adding replicas to a multi-leader replica group for a data set |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |