CN103246711A - Method and device generating snapshots of binary large object type data - Google Patents
Method and device generating snapshots of binary large object type data Download PDFInfo
- Publication number
- CN103246711A CN103246711A CN2013101409887A CN201310140988A CN103246711A CN 103246711 A CN103246711 A CN 103246711A CN 2013101409887 A CN2013101409887 A CN 2013101409887A CN 201310140988 A CN201310140988 A CN 201310140988A CN 103246711 A CN103246711 A CN 103246711A
- Authority
- CN
- China
- Prior art keywords
- snapshot
- blob
- categorical data
- correspondence
- generating algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
An embodiment of the invention provides a method and a device generating snapshots of binary large object type data. The method includes: recognizing file formats of BLOB (binary large object) type data in a database system; obtaining the snapshot generating algorithms corresponding to the file formats of the BLOB type data according to corresponding relationships between preset file formats and snapshot generating algorithms; generating snapshots corresponding to the BLOB type data according to the corresponding snapshot generating algorithms; and storing the snapshots corresponding to the BLOB type data in the database system, and storing mapping relationships between the BLOB type data and the corresponding snapshots. By automatically recognizing the file formats of the BLOB type data in a database, the snapshots corresponding to the BLOB type data are generated, and the relationships between the BLOB type data and the corresponding snapshots are stored inside the database, so that workload of developers and time delay of snapshot obtaining response are reduced.
Description
Technical field
The invention belongs to database field, relate in particular to a kind of snapshot creation method and device of binary large object categorical data.
Background technology
Binary large object BLOB(Binary Large Object), be the set of making as a whole binary data in the Database Systems, the typical application is to store multimedia object such as picture, sound, video etc.The generation of this data type of BLOB is later than relational database, and is main because this kind data type memory contents is too big, also do not have the demand of this respect when commercial relational database occurs.Now along with the development of infotech, the handled object of computing machine takes up space and also sharply increases, and also is becoming better and approaching perfection day by day for the processing of BLOB, and present main flow relational database platform is all supported the blob data type.
At present, the BLOB that generally will leave in the database by the query language interface retrieves to come out to use from database.The principle of work of database inside is during retrieval BLOB: because the volume of BLOB excessive (generally surpassing 2G) generally can not preserved actual BLOB in table, only can preserve an address, the position that the address is pointed to is BLOB actual storage zone.Application layer visits BLOB by calling the access of database interface, and Database Systems are returned the content of BLOB in the mode of stream, and application program is opened this object then.At present directly BLOB being passed back the problem that method that client opens exists is: the same time is transmitted the response delay that a collection of BLOB can increase client in the time of the BLOB super large, influence user's experience; Also can cause bigger pressure to the network bandwidth, reduce the utilization ratio of hardware resource.Such as each picture file 100M file, in the network of average transmission speed 100m/s, the time delay that passes to client surpasses 1s, if add client to the opening the time of big file, this time will be longer.Since network bottleneck, along with the increase of returning the BLOB number, the consumption meeting journey linear growth of time delay and the network bandwidth, even can cause network congestion.
Solution of the prior art all is to distinguish preview type access scenario and actual reading out type access scenario earlier basically, for preview type scene, generally be after database server side is taken out BLOB, calculate the snapshot of BLOB in real time by the handling procedure of prior association, then this snapshot is returned to client and show.Can reduce the time that preview scene lower network transmission bandwidth and client file are opened by this technology, yet, this technology equals to have increased the snapshot generation module between database and application program processing, and the exploitation of this module also is more complicated, the user need realize the snapshot extraction algorithm for multiple high efficiency BLOB, also to handle the relation of client-cache and snapshot in actual the use well, increase extra development amount; And can be subjected to the influence of actual type, size and the snapshot extraction algorithm of BLOB for processing time of preview snapshot, occasion at the BLOB super large, the extract real-time snapshot may spend a large amount of time, badly may also can prolong response delay if handle in actual applications.
Summary of the invention
The purpose of the embodiment of the invention is to provide a kind of snapshot creation method and device of binary large object BLOB categorical data, be intended to solve because prior art when obtaining the snapshot of BLOB categorical data, exists complexity height, snapshot response delay than problems such as height.
First aspect, the snapshot creation method of described binary large object BLOB categorical data comprises: the file layout of BLOB categorical data in the identification database system; According to default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of described BLOB categorical data; According to the snapshot generating algorithm of described correspondence, generate the snapshot corresponding with described BLOB categorical data; The storage snapshot corresponding with described BLOB categorical data in described Database Systems, and store mapping relations between described BLOB categorical data and the described corresponding snapshot.
In first kind of first aspect possible implementation, described method also comprises: the snapshot of the snapshot of the described BLOB categorical data of the acquisition request correspondence that the reception client sends obtains request, and the request of obtaining of described snapshot includes the sign of described BLOB categorical data; Obtain the sign of the categorical data of BLOB described in the request according to described snapshot, and the mapping relations between the described BLOB categorical data of storing in the described Database Systems and the described corresponding snapshot, inquiry obtains described BLOB categorical data and described corresponding snapshot in described Database Systems, and by default snapshot access interface, the snapshot of described BLOB categorical data correspondence is sent to described client.
In conjunction with first aspect or first kind of first aspect possible implementation, in second kind of possible implementation, before the described generation snapshot step corresponding with described BLOB categorical data, described method also comprises: obtain described Database Systems current system time, and judge whether the default snapshot rise time is identical with system of the institute time; If, then carry out described snapshot generating algorithm according to described correspondence, generate the step of the snapshot corresponding with described BLOB categorical data; If not, then repeat the described described Database Systems current system time of obtaining, and judge the step identical with described system time whether of described default snapshot rise time, identical with described Database Systems current system time until the described default snapshot rise time.
Second kind of possible implementation in conjunction with first kind of first aspect or first aspect possible implementation or first aspect, in the third possible implementation, before the step of the described generation snapshot corresponding with described BLOB categorical data, described method also comprises: the size of obtaining described BLOB categorical data; Whether the size of judging described BLOB categorical data surpasses predetermined threshold value; Described snapshot generating algorithm according to described correspondence, generate the snapshot corresponding with described BLOB categorical data, comprise: when the size of described BLOB categorical data surpasses predetermined threshold value, call the snapshot generating algorithm of described BLOB categorical data correspondence by described Database Systems backstage, generate the snapshot corresponding with described BLOB categorical data; When the size of described BLOB categorical data does not surpass predetermined threshold value, according to the snapshot generating algorithm of described BLOB categorical data correspondence, generate the snapshot corresponding with described BLOB categorical data.
Second kind of possible implementation in conjunction with first kind of first aspect or first aspect possible implementation or first aspect, in the 4th kind of possible implementation, before the step of the described generation snapshot corresponding with described BLOB categorical data, described method also comprises: the generation parameter of snapshot is set, and described generation parameter comprises ratio of compression, the size of snapshot; Described snapshot generating algorithm according to described correspondence, generate the snapshot corresponding with described BLOB categorical data, comprise: according to the snapshot generating algorithm of described correspondence, and the described generation parameter that arranges, generate the snapshot corresponding with described BLOB categorical data, and the ratio of compression of the snapshot of the described BLOB categorical data correspondence that generates and size with respectively with described generation parameter in ratio of compression and size coupling.
In conjunction with first kind of first aspect or first aspect possible implementation, in the 5th kind of possible implementation, described method also comprises: when detecting described BLOB categorical data when deleted, the snapshot corresponding with described BLOB categorical data that deletion is stored in described Database Systems deleted the mapping relations between described BLOB categorical data and the described corresponding snapshot simultaneously.
Second aspect, the snapshot generating apparatus of described BLOB categorical data comprises: the format identification unit, for the file layout of the BLOB of identification database system categorical data; The algorithm acquiring unit is used for obtaining the snapshot generating algorithm corresponding with the file layout of described BLOB categorical data according to default file layout and the corresponding relation of snapshot generating algorithm; The snapshot generation unit is used for the snapshot generating algorithm according to described correspondence, generates the snapshot corresponding with described BLOB categorical data; And storage unit, be used at the described Database Systems storage snapshot corresponding with described BLOB categorical data, and store mapping relations between described BLOB categorical data and the described corresponding snapshot.
In first kind of second aspect possible implementation, described device also comprises: the time judging unit is used for obtaining described Database Systems current system time, and judges whether the default snapshot rise time is identical with described system time; And the result treatment unit, be used for exporting the result when being when described time judging unit, trigger described snapshot generation unit immediately; Otherwise, the described time judging unit of repeated trigger is identical with described Database Systems current system time until the described default snapshot rise time.
In conjunction with first kind of second aspect or second aspect possible implementation, in second kind of possible implementation, described device also comprises: acquiring unit, for the size of obtaining described BLOB categorical data; Judging unit is used for judging whether the size of described BLOB categorical data surpasses predetermined threshold value; Described snapshot generation unit, when specifically surpassing predetermined threshold value for the size that goes out described BLOB categorical data when described judgment unit judges, trigger the snapshot generating algorithm that described BLOB categorical data correspondence is called on described Database Systems backstage, generate the snapshot corresponding with described BLOB categorical data; When the size that goes out described BLOB categorical data when described judgment unit judges surpasses predetermined threshold value, according to the snapshot generating algorithm of described BLOB categorical data correspondence, generate the snapshot corresponding with described BLOB categorical data.
In conjunction with first kind of second aspect or second aspect possible implementation, in the third possible implementation, described device also comprises: the parameter-definition unit, and for the generation parameter that snapshot is set, described generation parameter comprises ratio of compression, the size of snapshot; Described snapshot generation unit, the concrete snapshot generating algorithm that is used for according to described correspondence, and the described generation parameter that arranges, generate the snapshot corresponding with described BLOB categorical data, and the ratio of compression of the snapshot of the described BLOB categorical data correspondence that generates and size with respectively with described generation parameter in ratio of compression and size coupling.
In the 4th kind of possible implementation of second aspect, described device also comprises: edit cell, be used for when detecting described BLOB categorical data when deleted, the snapshot corresponding with described BLOB categorical data that deletion is stored in described Database Systems deleted the mapping relations between described BLOB categorical data and the described corresponding snapshot simultaneously.
The embodiment of the invention is by the file layout of BLOB categorical data in the automatic identification database system, according to default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of described BLOB categorical data, again according to the snapshot generating algorithm of described correspondence, automatically generate the snapshot corresponding with described BLOB categorical data, and at the described Database Systems storage inside snapshot corresponding with described BLOB categorical data, and store mapping relations between described BLOB categorical data and the described corresponding snapshot, transparent fully to the user to realize, solved prior art when obtaining the snapshot of BLOB categorical data, exist the development amount big, the snapshot response delay is than problems such as height, can realize the consistance of snapshot with corresponding BLOB categorical data in database inside, make exploitation complexity and snapshot response time-delay all be reduced.
Description of drawings
Fig. 1 is the realization flow figure of the snapshot creation method of the BLOB categorical data that provides of first embodiment of the invention;
Fig. 2 is the realization flow figure of the snapshot creation method of the BLOB categorical data that provides of second embodiment of the invention;
Fig. 3 is the realization flow figure of the snapshot creation method of the BLOB categorical data that provides of third embodiment of the invention;
Fig. 4 is the structural drawing of the snapshot generating apparatus of the BLOB categorical data that provides of fourth embodiment of the invention;
Fig. 5 is the structural drawing of the snapshot generating apparatus of the BLOB categorical data that provides of fifth embodiment of the invention.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explaining the present invention, and be not used in restriction the present invention.
The embodiment of the invention is by the file layout automatic identification algorithm according to default BLOB categorical data, the file layout of BLOB categorical data in the automatic identification database system, according to default file layout and the corresponding relation of snapshot generating algorithm, automatically obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data, again according to the snapshot generating algorithm of correspondence, automatically generate the snapshot corresponding with this BLOB categorical data, and at the Database Systems storage inside snapshot corresponding with described BLOB categorical data, and store mapping relations between this BLOB categorical data and the described corresponding snapshot, transparent fully to the user to realize, and self-defined file layout according to actual needs, recognizer and snapshot generating algorithm etc., the generation method is more flexible, and when client need be returned the snapshot of a certain BLOB categorical data, directly return the required snapshot that generates in advance by the snapshot access interface, reduced snapshot response time-delay.In addition, owing to do not need between database and application program processing, to increase complicated snapshot generation module as prior art, do not need to handle well the relation of client-cache and snapshot yet, reduced the development amount yet.
Below in conjunction with specific embodiment specific implementation of the present invention is described in detail:
Embodiment one:
Fig. 1 shows the realization flow of the snapshot creation method of the BLOB categorical data that first embodiment of the invention provides, and details are as follows:
In step S101, the file layout of BLOB categorical data in the identification database system.
Generally speaking, the BLOB categorical data is taken as binary data in Database Systems inside and handles, but can be identified as certain concrete file layout according to memory contents characteristics of binary data etc., such as the various file layouts under the file layout types such as picture format, audio format, video format, XML form, as, jpg, bmp, avi, rmvb, mp3, wav etc.The file layout of concrete BLOB categorical data and corresponding recognition methods can flexible expansion, and the user can increase self-defined file layout and recognition methods etc.In addition, after the file layout step by BLOB categorical data in the step S101 identification database system, can also judge further that the file layout of this BLOB categorical data is registered file layout, if, then continue execution in step S102,, then can this BLOB categorical data not carried out subsequent treatment.In embodiments of the present invention, when the file layout of judging this BLOB categorical data is not registered file layout, the file layout of this BLOB categorical data can also be converted to the file layout of the appointment under the file layout type of the same race, continue to carry out subsequent operation afterwards again.
In step S102, according to default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data.
In step S103, according to this corresponding snapshot generating algorithm, generate the snapshot corresponding with this BLOB categorical data.
Wherein, with each file layout can be according to the user to the type that generates snapshot, the requirement of parameter to the snapshot generating algorithm, by the user arrange voluntarily, modification etc.Such as, related algorithm is set, the generation of the BLOB categorical data of certain picture format is had the image snapshot of certain ratio of compression; Setting diminishes compression algorithm, the generation of the BLOB categorical data of audio format is had the audio frequency snapshot of certain ratio of compression; Relevant compression algorithm is set, video snapshot that the generation of the BLOB categorical data of video format is compressed etc., certainly the snapshot size that herein produces, ratio of compression, accuracy etc. and the algorithm etc. that generates corresponding snapshot can dispose voluntarily, also are that the user can revise according to acquisition algorithm etc. voluntarily.Wherein, after the generation parameter of snapshot is set, this generation parameter comprises ratio of compression, size of snapshot etc., then according to this corresponding snapshot generating algorithm, and this generation parameter that arranges, generate the snapshot corresponding with this BLOB categorical data, and the ratio of compression of the snapshot of this BLOB categorical data correspondence that generates and size with generate ratio of compression and size coupling in parameter with this respectively.Wherein, specifically the degree of coupling can be for identical, also certain error range can be set, as long as the ratio of compression of the snapshot of this BLOB categorical data correspondence that generates and size with generate error that ratio of compression in parameter and size exist in this error range with this respectively, think that then the match is successful etc., in addition, also can arrange the ratio of compression of snapshot of this BLOB categorical data correspondence of generation and size be no more than respectively with this generation parameter in ratio of compression and size etc., can arrange flexibly according to actual needs, not do restriction at this.In addition, the time that generates the snapshot corresponding with this BLOB categorical data can wait to trigger control etc. for the default time point of basis.
In step S104, in these Database Systems, store the snapshot corresponding with this BLOB categorical data, and store the mapping relations between this BLOB categorical data snapshot corresponding with this.
In specific implementation process, in identification database after the file layout of BLOB categorical data, corresponding relation according to the file layout that sets in advance and snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data, and then, according to this corresponding snapshot generating algorithm, generate the snapshot corresponding with this BLOB categorical data, at last, in these Database Systems, store the snapshot corresponding with this BLOB categorical data, and store the mapping relations between this BLOB categorical data snapshot corresponding with this.Need to prove, the memory location of snapshot is to user transparent, also namely when detecting certain BLOB categorical data and be modified or delete, automatically snapshot and the relation corresponding with this BLOB categorical data are revised or deleted to correspondence, the user does not need to be concerned about its actual storage locations, as long as can be by the mapping relations between this BLOB categorical data snapshot corresponding with this, the corresponding snapshot of inquiry gets final product, and snapshot and corresponding BLOB body can be stored in continuous storage space, also can be the storage space that disperses, not do restriction at this.
In addition, the snapshot creation method of this BLOB categorical data is further comprising the steps of:
The snapshot of the snapshot of this BLOB categorical data correspondence of acquisition request that the reception client sends obtains request, and the request of obtaining of this snapshot includes the sign of this BLOB categorical data;
Obtain this BLOB categorical data sign in the request according to this snapshot, and the mapping relations between this BLOB categorical data of storing in these Database Systems snapshot corresponding with this, inquiry obtains this BLOB categorical data snapshot corresponding with this in these Database Systems, and by default snapshot access interface, the snapshot of this BLOB categorical data correspondence is sent to this client.
In embodiments of the present invention, the snapshot creation method of this BLOB categorical data is applied in the data system storehouse of main flow, in Oracle11g, SQL Server2008, My SQL5.5 etc.Then under the actual scene that reads the blob data content, application layer visits BLOB by calling the access of database interface, and Database Systems are returned the content of BLOB, this object of application program layer open then in the mode of stream; And under the scene of preview snapshot, when needing the snapshot of this BLOB categorical data of preview, then application layer visits BLOB by the snapshot access interface that calls database, and then according to the relation between this BLOB categorical data of storage in advance snapshot corresponding with this, obtain this corresponding snapshot of storage in advance, Database Systems should correspondence snapshot be back to application layer, this snapshot of application program layer open.
In embodiments of the present invention, snapshot creation method by this BLOB categorical data obtains after the snapshot of each BLOB categorical data correspondence, the snapshot that then can receive the snapshot of this BLOB categorical data correspondence of acquisition request that client sends obtains request, in Database Systems, obtain according to this snapshot and comprise this BLOB categorical data sign in the request, and the mapping relations between this BLOB categorical data of storing in these Database Systems snapshot corresponding with this, inquiry obtains this BLOB categorical data snapshot corresponding with this, by default snapshot access interface, the snapshot of this BLOB categorical data correspondence is sent to this client, also namely can be quick, return the snapshot corresponding with this BLOB categorical data exactly to this client, improve the speed of inquiry snapshot, reduced the snapshot response delay; By this method, can also further carry out normalized etc. to snapshot, with the retrieval that realizes that realizations such as the user is content-based are carried out BLOB, be similar to existing image search method, can also generate the snapshot of different stage according to the level of detail of snapshot contents, so that the snapshot of different access rights visit different stage, thereby range of application is wider; In actual use, client can be obtained relevant snapshot immediately, does not need to be concerned about client-cache and the relation of snapshot rise time etc., has reduced extra development amount and complexity; Because this method does not need to read the BOLB categorical data in real time earlier from Database Systems, generates snapshot in real time, has reduced the bandwidth that takies owing to query transmission BLOB yet, has finally strengthened user's experience effect.
Embodiment two:
Fig. 2 shows the realization flow of the snapshot creation method of the BLOB categorical data that second embodiment of the invention provides, and details are as follows:
In step S201, the file layout of BLOB categorical data in the identification database.
In step S202, judge whether the file layout of this BLOB categorical data is registered, be, then execution in step S204; Deny, then execution in step S203.
In step S203, the file layout of this BLOB categorical data is converted to the file layout of appointment under the file layout type of the same race, and continues execution in step S204.
In step S204, according to default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data.
Particularly, owing to do not set in advance the corresponding snapshot generating algorithm of certain file layout, be the JPG form such as file layout, then after executing this step 202, judge the not registration of JPG form, the situation that can't continue to carry out then can appear, at this moment, the application forces to convert to specified file format under certain registered image file type by step S203 with this JPG form, such as the bmp form, thereby make method of operating more flexible, further can also provide this BLOB type does not have prompting of registration etc.Relation between the file layout that the user also can self-defined certain BLOB categorical data, snapshot generating algorithm etc. in addition.
In step S205, obtain this Database Systems current system time.
In step S206, judge whether the default snapshot rise time is identical with this system time, is, then execution in step S207; , then do not turn to and carry out this step S205.
In step S207, according to this corresponding snapshot generating algorithm, generate the snapshot corresponding with this BLOB categorical data.
In step S208, in these Database Systems, store the snapshot corresponding with this BLOB categorical data, and store the mapping relations between this BLOB categorical data snapshot corresponding with this.
Particularly, the snapshot creation method of this BLOB categorical data can be automatically or manual triggers generate, also namely can be set the rise time voluntarily, such as, can set in advance a snapshot rise time, when only identical with place Database Systems current system time in this snapshot rise time, just can trigger and generate the snapshot corresponding with this BLOB categorical data, wherein, this system time is the place Database Systems current time, concrete obtain manner can utilize the language codes of place Database Systems to obtain by prior art, does not repeat them here; If this snapshot rise time and place Database Systems current system time are inequality, then continue to obtain the Database Systems current system time, when identical with place Database Systems current system time in this snapshot rise time, trigger to generate the snapshot corresponding with this BLOB categorical data, specifically as described in the above-mentioned step S205 to 207.Wherein step S201, S204, S207, S208 are identical with above-described embodiment one step S101, S102, S103, S104 respectively, do not repeat them here.
Embodiment three:
Fig. 3 shows the realization flow of the snapshot creation method of the BLOB categorical data that third embodiment of the invention provides, and details are as follows:
In embodiments of the present invention, in based on above-described embodiment one after the step S102, before the step S103, perhaps based on step S206 judged result in above-described embodiment two for after being, before the step S207, the snapshot creation method of this BLOB categorical data also comprises the steps:
In step S301, obtain the size of this BLOB categorical data.
In step S302, judge that whether the size of this BLOB categorical data surpasses predetermined threshold value, be, execution in step S303 then, not, execution in step S204 then.
In step S303, call the snapshot generating algorithm of this BLOB categorical data correspondence by this Database Systems backstage, generate the snapshot corresponding with this BLOB categorical data.
In step S304, according to the snapshot generating algorithm of this BLOB categorical data correspondence, generate the snapshot corresponding with this BLOB categorical data.
In step S305, in Database Systems, store the snapshot corresponding with this BLOB categorical data, and store the mapping relations between this BLOB categorical data snapshot corresponding with this.
In specific implementation process, when detecting certain BLOB categorical data and be modified or delete, automatically the snapshot corresponding with this BLOB categorical data revised or deleted to correspondence, after replacing original BLOB with new BLOB, the snapshot of original BLOB correspondence ceases to be in force automatically, and can generate new snapshot according to the content of new BLOB, this new snapshot is replaced original snapshot in other words.
In embodiments of the present invention, after the size of obtaining the BLOB categorical data, whether surpass predetermined threshold value by the size of judging this BLOB categorical data, deciding is the snapshot generating algorithm of being called this BLOB categorical data correspondence by Database Systems backstage, place, automatically generate the snapshot corresponding with this BLOB categorical data, or direct snapshot generating algorithm according to this BLOB categorical data correspondence, automatically generate the snapshot corresponding with this BLOB categorical data, thereby the generating run of flexible processing snapshot more, also can not take the more time of system, and during the backstage generates snapshot, can not visit the snapshot of this BLOB categorical data correspondence, can also provide relevant backstage generating run this moment and be prompted to user etc., so that the user in time goes and finds out what's going on, other snapshots of preview etc.
One of ordinary skill in the art will appreciate that all or part of step that realizes in above-described embodiment method is to instruct relevant hardware to finish by program, described program can be stored in the computer read/write memory medium, described storage medium is as ROM/RAM, disk, CD etc.
Embodiment four:
Fig. 4 shows the structure of the snapshot generating apparatus of the BLOB categorical data that fourth embodiment of the invention provides, and for convenience of explanation, only shows the part relevant with the embodiment of the invention.
The snapshot generating apparatus of this BLOB categorical data can be used for existing relevant database or non-relational database, such as SQL Server2008, My SQL5.5 etc.As shown in Figure 4, the snapshot generating apparatus of this BLOB categorical data comprises: format identification unit 41, algorithm acquiring unit 42, snapshot generation unit 43 and storage unit 44, wherein:
Format identification unit 41 is for the file layout of the BLOB of identification database system categorical data.
Algorithm acquiring unit 42 is used for obtaining the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data according to default file layout and the corresponding relation of snapshot generating algorithm.
Wherein, after triggering this format identification unit 41, trigger before this algorithm acquiring unit 42, the snapshot generating apparatus of this BLOB categorical data also comprises:
The registration judging unit is used for judging whether the file layout of this BLOB categorical data is registered file layout; And
Format conversion unit, be used for when this registration judging unit output result when denying, with the file layout of this BLOB categorical data be converted to the file layout of appointment under the file layout type of the same race.
In embodiments of the present invention, can identify its file layout particularly by format identification unit 41 for each BLOB categorical data that inserts Database Systems.And being taken as binary data in Database Systems inside, handles the BLOB categorical data, but can be identified as certain file layout according to memory contents characteristics of binary data etc., such as the various file layouts under the file layout types such as picture format, audio format, video format, XML form.The file layout of concrete BLOB categorical data and corresponding recognition methods can flexible expansion, and the user can increase self-defined file layout and recognition methods etc.And then according to this algorithm acquiring unit 42 based on default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data.
In addition, after triggering this format identification unit 41, can also whether be registered file layout by the file layout of registration judgment unit judges this BLOB categorical data, export the result when being when this registration judging unit, directly trigger this algorithm acquiring unit 42; And when this registration judging unit output result for not the time, by this format conversion unit with after the file layout of this BLOB categorical data is converted to the file layout of appointment under the file layout type of the same race, carry out follow-up operation again, thereby can obtain the file layout of arbitrary BLOB categorical data more neatly.
Snapshot generation unit 43 is used for according to this corresponding snapshot generating algorithm, generates the snapshot corresponding with this BLOB categorical data.
Wherein, with each file layout can be according to the user to the type that generates snapshot, the requirement of parameter to the snapshot generating algorithm, by the user arrange voluntarily, modification etc.Such as, related algorithm is set, the generation of the BLOB categorical data of picture format is had the image snapshot of certain ratio of compression; Setting diminishes compression algorithm, the generation of the BLOB categorical data of audio format is had the audio frequency snapshot of certain ratio of compression; Relevant compression algorithm is set, video snapshot that the generation of the BLOB categorical data of video format is compressed etc.
Before generating the snapshot corresponding with this BLOB categorical data, the generation parameter of snapshot can also be set by the parameter-definition unit, this generation parameter comprises ratio of compression, size of snapshot etc., at this moment, this snapshot generation unit 43 specifically is used for according to this corresponding snapshot generating algorithm, and this generation parameter that arranges, generate the snapshot corresponding with this BLOB categorical data, and the ratio of compression of the snapshot of this BLOB categorical data correspondence that generates and size with generate ratio of compression and size coupling in parameter with this respectively.Wherein, specifically the degree of coupling can be for identical, also certain error range can be set, as long as the ratio of compression of the snapshot of this BLOB categorical data correspondence that generates and size with generate error that ratio of compression in parameter and size exist in this error range with this respectively, think that then the match is successful etc., in addition, also can arrange the ratio of compression of snapshot of this BLOB categorical data correspondence of generation and size be no more than respectively with this generation parameter in ratio of compression and size etc., can arrange flexibly according to actual needs, not do restriction at this.
In embodiments of the present invention, the time that generates the snapshot corresponding with this BLOB categorical data can also can generate immediately for sending out control according to default snapshot rise time etc. from movable contact, can arrange as required.Such as, after the snapshot generating algorithm of the file layout of obtaining arbitrary BLOB categorical data and this document form correspondence, before generating the snapshot corresponding with this BLOB categorical data, the snapshot generating apparatus of this BLOB categorical data can also comprise time judging unit and result treatment unit, realized the self-defined purpose that the snapshot rise time is set, wherein:
The time judging unit is used for obtaining this Database Systems current system time, and judges whether the default snapshot rise time is identical with this number system time; And
The result treatment unit is used for exporting the result when being when this time judging unit, triggers this snapshot generation unit immediately; Otherwise repeated trigger should the time judging unit, and is identical with described Database Systems current system time until this default snapshot rise time.
Storage unit 44 is used at this Database Systems storage snapshot corresponding with this BLOB categorical data, and stores the mapping relations between this BLOB categorical data snapshot corresponding with this.
In embodiments of the present invention, the snapshot creation method corresponding with the snapshot generating apparatus of this BLOB categorical data seen above-described embodiment one to three, do not repeat them here.
Snapshot generating apparatus by the BLOB categorical data that provides in the embodiment of the invention four, can generate and store the snapshot of BLOB categorical data in Database Systems inside automatically, and the BLOB categorical data is transparent fully to the user with the association store between the corresponding snapshot, solved the data super large in the BLOB type, the high delay problem that causes when needing its corresponding snapshot of preview, realized by independent snapshot access interface, the purpose of fast access snapshot, and, when detecting this BLOB categorical data when deleted, can also be by the corresponding deletion of the edit cell snapshot corresponding with this BLOB categorical data, and the mapping relations between this BLOB categorical data snapshot corresponding with this, then under the prerequisite that does not increase database user extra work amount, realized solving the consistency problem of snapshot and raw data by database inside.
Embodiment five:
Fig. 5 shows the structure of the snapshot generating apparatus of the BLOB categorical data that fifth embodiment of the invention provides, and for convenience of explanation, only shows the part relevant with the embodiment of the invention.
Snapshot generating apparatus structure based on BLOB categorical data given in above-described embodiment four, as shown in Figure 5, the snapshot generating apparatus of this BLOB categorical data comprises in the embodiment of the invention: format identification unit 51, algorithm acquiring unit 52, acquiring unit 53, judging unit 54, snapshot generation unit 55 and storage unit 56, this format identification unit 51, algorithm acquiring unit 52, storage unit 56 respectively with above-described embodiment four in format identification unit 41, algorithm acquiring unit 42, storage unit 44 corresponding one by one, identical.This moment, the snapshot generating apparatus of this BLOB categorical data can be used for existing system R or non-relational Database Systems, wherein:
Acquiring unit 53 is for the size of obtaining this BLOB categorical data.
Judging unit 54 is used for judging whether the size of this BLOB categorical data surpasses predetermined threshold value.
At this moment, when snapshot generation unit 55 specifically surpasses predetermined threshold value for the size of judging this BLOB categorical data when this judging unit 53, trigger the snapshot generating algorithm that this BLOB categorical data correspondence is called on this Database Systems backstage, generate the snapshot corresponding with this BLOB categorical data; When the size of judging this BLOB categorical data when this judging unit 53 surpasses predetermined threshold value, according to the snapshot generating algorithm of this BLOB categorical data correspondence, generate the snapshot corresponding with this BLOB categorical data.
By the device that provides in the embodiment of the invention five, after this acquiring unit 53 obtains the size of BLOB categorical data, judging unit 54 judges whether the size of this BLOB categorical data surpasses predetermined threshold value, it is to call this corresponding snapshot generating algorithm by the Database Systems backstage that triggering snapshot generation unit 55 decides, automatically generate the snapshot corresponding with this BLOB categorical data, still directly according to this corresponding snapshot generating algorithm, automatically generate the snapshot corresponding with this BLOB categorical data, the generating run of flexible processing snapshot more, also can not take the more time of system, and during the backstage generates snapshot, can not visit the snapshot of this BLOB categorical data correspondence, can also provide relevant backstage generating run this moment and be prompted to user etc., so that the user in time goes and finds out what's going on, other snapshots of preview etc.
The embodiment of the invention is by the file layout automatic identification algorithm according to default BLOB categorical data, the file layout of BLOB categorical data in the automatic identification database system, according to default file layout and the corresponding relation of snapshot generating algorithm, automatically obtain the snapshot generating algorithm corresponding with the file layout of this BLOB categorical data, again according to the snapshot generating algorithm of correspondence, automatically generate the snapshot corresponding with this BLOB categorical data, and store this BLOB categorical data snapshot corresponding with this in the Database Systems internal correlation, and store mapping relations between this BLOB categorical data snapshot corresponding with this, realized the user transparent fully, but also self-defined file layout according to actual needs, recognizer and snapshot generating algorithm etc., the generation method is more flexible, and when client need be returned the snapshot of a certain BLOB categorical data, directly return the required snapshot that generates in advance by the snapshot access interface, reduced snapshot response time-delay.In addition, owing to do not need between database and application program processing, to increase the higher snapshot generation module of complexity as prior art, do not need to handle well the relation of client-cache and snapshot yet, reduced complexity and development amount yet.
Need to prove, unit and the algorithm steps of each example of describing in conjunction with embodiment disclosed herein, can realize with electronic hardware, computer software or the combination of the two, for the interchangeability of hardware and software clearly is described, composition and the step of each example described in general manner according to function in the above description.These functions still are that software mode is carried out with hardware actually, depend on application-specific and the design constraint of technical scheme.The professional and technical personnel can specifically should be used for using distinct methods to realize described function to each, but this realization should not thought and exceeds scope of the present invention.
The method of describing in conjunction with embodiment disclosed herein or the step of algorithm can use the software module of hardware, processor execution, and perhaps the combination of the two is implemented.Software module can place the storage medium of any other form known in random access memory (RAM), internal memory, ROM (read-only memory) (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or the technical field.
Above-described embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the above only is the specific embodiment of the present invention; and be not intended to limit the scope of the invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (11)
1. the snapshot creation method of a binary large object BLOB categorical data is characterized in that, described method comprises:
The file layout of BLOB categorical data in the identification database system;
According to default file layout and the corresponding relation of snapshot generating algorithm, obtain the snapshot generating algorithm corresponding with the file layout of described BLOB categorical data;
According to the snapshot generating algorithm of described correspondence, generate the snapshot corresponding with described BLOB categorical data;
The storage snapshot corresponding with described BLOB categorical data in described Database Systems, and store mapping relations between described BLOB categorical data and the described corresponding snapshot.
2. the method for claim 1 is characterized in that, described method also comprises:
The snapshot of the snapshot of the described BLOB categorical data of the acquisition request correspondence that the reception client sends obtains request, and the request of obtaining of described snapshot includes the sign of described BLOB categorical data;
Obtain the sign of the categorical data of BLOB described in the request according to described snapshot, and the mapping relations between the described BLOB categorical data of storing in the described Database Systems and the described corresponding snapshot, inquiry obtains described BLOB categorical data and described corresponding snapshot in described Database Systems, and by default snapshot access interface, the snapshot of described BLOB categorical data correspondence is sent to described client.
3. method as claimed in claim 1 or 2 is characterized in that, before the described generation snapshot step corresponding with described BLOB categorical data, described method also comprises:
Obtain described Database Systems current system time, and judge whether the default snapshot rise time is identical with described system time;
If, then carry out described snapshot generating algorithm according to described correspondence, generate the step of the snapshot corresponding with described BLOB categorical data; If not, then repeat the described described Database Systems current system time of obtaining, and judge the step identical with described system time whether of described default snapshot rise time, identical with described Database Systems current system time until the described default snapshot rise time.
4. as each described method of claim 1-3, it is characterized in that before the step of the described generation snapshot corresponding with described BLOB categorical data, described method also comprises:
Obtain the size of described BLOB categorical data;
Whether the size of judging described BLOB categorical data surpasses predetermined threshold value;
Described snapshot generating algorithm according to described correspondence generates the snapshot corresponding with described BLOB categorical data, comprising:
When the size of described BLOB categorical data surpasses predetermined threshold value, call the snapshot generating algorithm of described BLOB categorical data correspondence by described Database Systems backstage, generate the snapshot corresponding with described BLOB categorical data;
When the size of described BLOB categorical data does not surpass predetermined threshold value, according to the snapshot generating algorithm of described BLOB categorical data correspondence, generate the snapshot corresponding with described BLOB categorical data.
5. as each described method of claim 1-3, it is characterized in that before the step of the described generation snapshot corresponding with described BLOB categorical data, described method also comprises:
The generation parameter of snapshot is set, and described generation parameter comprises ratio of compression, the size of snapshot;
Described snapshot generating algorithm according to described correspondence generates the snapshot corresponding with described BLOB categorical data, comprising:
Snapshot generating algorithm according to described correspondence, and the described generation parameter that arranges, generate the snapshot corresponding with described BLOB categorical data, and the ratio of compression of the snapshot of the described BLOB categorical data correspondence that generates and size with respectively with described generation parameter in ratio of compression and size coupling.
6. method as claimed in claim 1 or 2 is characterized in that, described method also comprises:
When detecting described BLOB categorical data when deleted, the snapshot corresponding with described BLOB categorical data that deletion is stored in described Database Systems deleted the mapping relations between described BLOB categorical data and the described corresponding snapshot simultaneously.
7. the snapshot generating apparatus of a binary large object BLOB categorical data is characterized in that, described device comprises:
The format identification unit is for the file layout of the BLOB of identification database system categorical data;
The algorithm acquiring unit is used for obtaining the snapshot generating algorithm corresponding with the file layout of described BLOB categorical data according to default file layout and the corresponding relation of snapshot generating algorithm;
The snapshot generation unit is used for the snapshot generating algorithm according to described correspondence, generates the snapshot corresponding with described BLOB categorical data; And
Storage unit is used at the described Database Systems storage snapshot corresponding with described BLOB categorical data, and stores the mapping relations between described BLOB categorical data and the described corresponding snapshot.
8. device as claimed in claim 7 is characterized in that, described device also comprises:
The time judging unit is used for obtaining described Database Systems current system time, and judges whether the default snapshot rise time is identical with described system time; And
The result treatment unit is used for exporting the result when being when described time judging unit, triggers described snapshot generation unit immediately; Otherwise, the described time judging unit of repeated trigger is identical with described Database Systems current system time until the described default snapshot rise time.
9. as claim 7 or 8 described devices, it is characterized in that described device also comprises:
Acquiring unit is for the size of obtaining described BLOB categorical data;
Judging unit is used for judging whether the size of described BLOB categorical data surpasses predetermined threshold value;
Described snapshot generation unit, when specifically surpassing predetermined threshold value for the size that goes out described BLOB categorical data when described judgment unit judges, trigger the snapshot generating algorithm that described BLOB categorical data correspondence is called on described Database Systems backstage, generate the snapshot corresponding with described BLOB categorical data; When the size that goes out described BLOB categorical data when described judgment unit judges surpasses predetermined threshold value, according to the snapshot generating algorithm of described BLOB categorical data correspondence, generate the snapshot corresponding with described BLOB categorical data.
10. as claim 7 or 8 described devices, it is characterized in that described device also comprises:
The parameter-definition unit, for the generation parameter that snapshot is set, described generation parameter comprises ratio of compression, the size of snapshot;
Described snapshot generation unit, the concrete snapshot generating algorithm that is used for according to described correspondence, and the described generation parameter that arranges, generate the snapshot corresponding with described BLOB categorical data, and the ratio of compression of the snapshot of the described BLOB categorical data correspondence that generates and size with respectively with described generation parameter in ratio of compression and size coupling.
11. device as claimed in claim 7 is characterized in that, described device also comprises:
Edit cell is used for deleting the snapshot of storing corresponding with described BLOB categorical data when detecting described BLOB categorical data when deleted in described Database Systems, deletes the mapping relations between described BLOB categorical data and the described corresponding snapshot simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310140988.7A CN103246711B (en) | 2013-04-22 | 2013-04-22 | The snapshot creation method of a kind of binary large object categorical data and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310140988.7A CN103246711B (en) | 2013-04-22 | 2013-04-22 | The snapshot creation method of a kind of binary large object categorical data and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103246711A true CN103246711A (en) | 2013-08-14 |
| CN103246711B CN103246711B (en) | 2016-12-07 |
Family
ID=48926231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310140988.7A Active CN103246711B (en) | 2013-04-22 | 2013-04-22 | The snapshot creation method of a kind of binary large object categorical data and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103246711B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066707A (en) * | 2017-03-27 | 2017-08-18 | 中国科学院计算技术研究所 | The adjustable design method for tracing and device of a kind of use snapshot |
| CN111199141A (en) * | 2018-10-31 | 2020-05-26 | 阿里巴巴集团控股有限公司 | Picture processing method and device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101080715A (en) * | 2004-12-15 | 2007-11-28 | 皇家飞利浦电子股份有限公司 | System and method for managing binary large objects |
| CN101344881A (en) * | 2007-07-09 | 2009-01-14 | 中国科学院大气物理研究所 | Index generation method and device and search system for mass file type data |
| US20090024578A1 (en) * | 2007-07-20 | 2009-01-22 | Oracle International Corporation | Delta Operations On A Large Object In A Database |
| US20090234880A1 (en) * | 2008-03-14 | 2009-09-17 | Microsoft Corporation | Remote storage and management of binary object data |
| CN102043830A (en) * | 2010-11-17 | 2011-05-04 | 北京握奇数据系统有限公司 | Method and device for processing BLOB (Binary Large Object) type data and intelligent card |
| CN102841918A (en) * | 2012-06-29 | 2012-12-26 | 用友软件股份有限公司 | Method and device for processing binary system long-object fluidized data |
-
2013
- 2013-04-22 CN CN201310140988.7A patent/CN103246711B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101080715A (en) * | 2004-12-15 | 2007-11-28 | 皇家飞利浦电子股份有限公司 | System and method for managing binary large objects |
| CN101344881A (en) * | 2007-07-09 | 2009-01-14 | 中国科学院大气物理研究所 | Index generation method and device and search system for mass file type data |
| US20090024578A1 (en) * | 2007-07-20 | 2009-01-22 | Oracle International Corporation | Delta Operations On A Large Object In A Database |
| US20090234880A1 (en) * | 2008-03-14 | 2009-09-17 | Microsoft Corporation | Remote storage and management of binary object data |
| CN102043830A (en) * | 2010-11-17 | 2011-05-04 | 北京握奇数据系统有限公司 | Method and device for processing BLOB (Binary Large Object) type data and intelligent card |
| CN102841918A (en) * | 2012-06-29 | 2012-12-26 | 用友软件股份有限公司 | Method and device for processing binary system long-object fluidized data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066707A (en) * | 2017-03-27 | 2017-08-18 | 中国科学院计算技术研究所 | The adjustable design method for tracing and device of a kind of use snapshot |
| CN107066707B (en) * | 2017-03-27 | 2019-07-30 | 中国科学院计算技术研究所 | A kind of adjustable design method for tracing and device using snapshot |
| CN111199141A (en) * | 2018-10-31 | 2020-05-26 | 阿里巴巴集团控股有限公司 | Picture processing method and device and electronic equipment |
| CN111199141B (en) * | 2018-10-31 | 2023-05-05 | 阿里巴巴集团控股有限公司 | Picture processing method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103246711B (en) | 2016-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107463635B (en) | Method for inquiring picture data and distributed NewSQL database system | |
| JP5376696B2 (en) | Document synchronization via stateless protocol | |
| KR102187127B1 (en) | Deduplication method using data association and system thereof | |
| CN103595797B (en) | Caching method for distributed storage system | |
| CN105677752A (en) | Streaming computing and batch computing combined processing system and method | |
| US20130067237A1 (en) | Providing random access to archives with block maps | |
| CN104253858B (en) | A kind of mobile data synchronization conflict pretreatment and digestion procedure based on packet | |
| CN101866359A (en) | Small file storage and visit method in avicade file system | |
| CN107302561A (en) | A kind of hot spot data Replica placement method in cloud storage system | |
| CN108415671B (en) | Method and system for deleting repeated data facing green cloud computing | |
| CN104881466A (en) | Method and device for processing data fragments and deleting garbage files | |
| CN104281414A (en) | Distributed file system and small file access method thereof | |
| CN106649676A (en) | Duplication eliminating method and device based on HDFS storage file | |
| CN104778229A (en) | Telecommunication service small file storage system and method based on Hadoop | |
| CN112235396B (en) | Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium | |
| CN104427341A (en) | Image processing method and apparatus, and terminal | |
| KR100946694B1 (en) | System and Method for managing and detecting duplicate moving picture files based on video contents | |
| CN105353988A (en) | Metadata reading and writing method and device | |
| CN112597348A (en) | Method and device for optimizing big data storage | |
| CN105407044A (en) | Method for implementing cloud storage gateway system based on network file system (NFS) | |
| CN104298695B (en) | Data caching method and device and server | |
| CN112035466B (en) | External index development framework for block chain query | |
| CN103246711A (en) | Method and device generating snapshots of binary large object type data | |
| US20180322179A1 (en) | Processing data between data stores | |
| CN114942908B (en) | Index system, data processing method, electronic device, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220215 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after: Huawei Cloud Computing Technology Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |