CN109614372A - A kind of object storage, read method, device and service server - Google Patents
A kind of object storage, read method, device and service server Download PDFInfo
- Publication number
- CN109614372A CN109614372A CN201811261886.XA CN201811261886A CN109614372A CN 109614372 A CN109614372 A CN 109614372A CN 201811261886 A CN201811261886 A CN 201811261886A CN 109614372 A CN109614372 A CN 109614372A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- read
- module
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
Disclosing a kind of object storage, read method, device and service server, the object storage method includes: to carry out serializing processing to target object, obtains the serialized data of target object；Deconsolidation process is carried out to serialized data；Based on deconsolidation process as a result, determining multiple data fragmentations of serialized data, multiple data fragmentations are stored into database, and store the index information for indicating multiple data fragmentations storage location in the database；The object read method includes: to determine target storage position in the database according to the index information pre-saved, and index information is for indicating the storage location of multiple data fragmentations in the database；Based on target storage position, data fragmentation is read from database；Processing is merged to the data fragmentation read；Unserializing processing is carried out to the obtained data of merging treatment, obtains target object.Using the object storage method and object read method, the access performance of object can be improved.
Description
Technical field
This specification embodiment is related to technical field of data processing more particularly to a kind of storage of object, read method, dress
It sets and service server.
Background technique
In operation system, the storage and reading of super large object are frequently involved.In the prior art, usually using document form
Store super large object, the memory capacity of document form be it is infinitely great, still, according to this form store super large object, rear
During continuous lookup super large object, each Documents Logical block can be searched first, and logical block is dispersed on physical block, physics
Block is dispersed in different sectors again, this also with regard to needing successively to be searched, can just find complete object, could finally complete
The reading of object.It can be seen that excessively poor using the access performance of document form storage super large object.
Summary of the invention
In view of the above technical problems, this specification embodiment provides a kind of object storage, read method, device and business
Server, technical solution are as follows:
According to this specification embodiment in a first aspect, providing a kind of object storage method, which comprises
Serializing processing is carried out to target object, obtains the serialized data of the target object；
Deconsolidation process is carried out to the serialized data；
Based on deconsolidation process as a result, multiple data fragmentations of the serialized data are determined, by the multiple data fragmentation
Store into database, and store for indicate the multiple data fragmentation in the database storage location index letter
Breath.
According to the second aspect of this specification embodiment, a kind of object read method is provided, which comprises
Target storage position is determined in the database according to the index information pre-saved, and the index information is for indicating
The storage location of multiple data fragmentations in the database；
Based on the target storage position, data fragmentation is read from the database；
Processing is merged to the data fragmentation read；
Unserializing processing is carried out to the obtained data of merging treatment, obtains target object.
According to the third aspect of this specification embodiment, a kind of object storage device is provided, described device includes:
Serialization module obtains the serialized data of the target object for carrying out serializing processing to target object；
Module is split, for carrying out deconsolidation process to the serialized data；
Memory module, for based on deconsolidation process as a result, determine multiple data fragmentations of the serialized data, will be described
Multiple data fragmentations are stored into database, and are stored for indicating that the multiple data fragmentation stores position in the database
The index information set.
According to the fourth aspect of this specification embodiment, a kind of object reading device is provided, described device includes:
Determining module, for determining target storage position, the rope in the database according to the index information pre-saved
Fuse is ceased for indicating the storage location of multiple data fragmentations in the database；
Read module reads data fragmentation for being based on the target storage position from the database；
Merging module, for merging processing to the data fragmentation read；
Unserializing module obtains target object for carrying out unserializing processing to the obtained data of merging treatment.
According to the 5th of this specification embodiment aspect, a kind of service server is provided, including memory, processor and deposit
Store up the computer program that can be run on a memory and on a processor, wherein the processor is realized when executing described program
The object storage method that this specification embodiment provides.
According to the 6th of this specification embodiment aspect, a kind of service server is provided, including memory, processor and deposit
Store up the computer program that can be run on a memory and on a processor, wherein the processor is realized when executing described program
The object read method that this specification embodiment provides.
Technical solution provided by this specification embodiment obtains target by carrying out serializing processing to target object
The serialized data of object carries out deconsolidation process to the serialized data, based on deconsolidation process as a result, determining serialized data
Multiple data fragmentations store multiple data fragmentations into database, and store for indicating multiple data fragmentation in data
The index information of storage location in library can break through the limitation of bottom storage, improve the storage performance of super large object.
Technical solution provided by this specification embodiment, by true in the database according to the index information pre-saved
Set the goal storage location, is based on the target storage position, data fragmentation is read from database, to the data fragmentation read
Merge processing；Unserializing processing is carried out to the obtained data of merging treatment, target object is obtained, super large can be improved
The reading performance of object.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
This specification embodiment can be limited.
In addition, any embodiment in this specification embodiment does not need to reach above-mentioned whole effects.
Detailed description of the invention
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
The some embodiments recorded in this specification embodiment for those of ordinary skill in the art can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is a kind of embodiment flow chart for object storage method that one exemplary embodiment of this specification provides；
Fig. 2 is a kind of embodiment flow chart for object read method that one exemplary embodiment of this specification provides；
Fig. 3 is a kind of embodiment block diagram for object storage device that one exemplary embodiment of this specification provides；
Fig. 4 is a kind of embodiment block diagram for object reading device that one exemplary embodiment of this specification provides；
Fig. 5 is the more specifically service server hardware structural diagram of one kind provided by this specification embodiment.
Specific embodiment
In order to make those skilled in the art more fully understand the technical solution in this specification embodiment, below in conjunction with this
Attached drawing in specification embodiment is described in detail the technical solution in this specification embodiment, it is clear that described
Embodiment is only a part of the embodiment of this specification, instead of all the embodiments.The embodiment of base in this manual,
Those of ordinary skill in the art's every other embodiment obtained, all should belong to the range of protection.
Since in the prior art, the access performance using document form storage super large object is excessively poor, therefore, this specification
Embodiment proposes a kind of object storage method and object read method, to improve the access performance of object.
Illustrate first, the realization of the object storage method and object read method is based on Hbase database, also i.e. by object
It stores into Hbase data, subsequent, the reading object from Hbase database.It is as follows, it is right to this respectively that following embodiments are shown
As storage method and the object read method are illustrated:
Firstly, being illustrated to object storage method:
Referring to Figure 1, a kind of embodiment process of the object storage method provided for one exemplary embodiment of this specification
Figure, this method may comprise steps of:
Step 102: serializing processing being carried out to target object, obtains the serialized data of target object.
Illustrate first, in this specification embodiment, for convenience, object to be processed is known as target object.
In order to enable target object can persist preservation, serializing processing can be carried out to target object, obtain one group
Byte, this group of byte are the serialized data of target object.
Wherein, when carrying out serializing processing to target object, json, kryo or jdk Serialize can be used
Etc. modes, this specification embodiment it is without limitation.
In addition, compression processing further can be done to the serialized data after obtaining the serialized data of target object,
It is subsequent, it is stored for the serialized data after compression processing.It is handled by this kind, number to be stored can be further decreased
According to data volume, improve the access performance of data.
Step 104: deconsolidation process is carried out to serialized data.
In one embodiment, physics deconsolidation process directly can be carried out to the serialized data that step 102 obtains, here institute
" the physics deconsolidation process " said refers in system memory, serialized data is split into multiple data fragmentations, for example, it is assumed that sequence
Columnization data include 100 bytes, then are 10 groupings by 100 byte splits, each grouping includes 10 bytes, each
Grouping is a data fragmentation, obtains 10 data fragmentations.
In one embodiment, logic deconsolidation process can be carried out to the serialized data that step 102 obtains, it is mentioned here
" logic deconsolidation process " refers in system memory, based on preset fractionation rule, obtains multiple fractionations in serialized data
Position, for example, it is assumed that serialized data includes 100 bytes, preset fractionation rule is that serialized data is split as 10
Data fragmentation, each data fragmentation include 10 bytes, then, accessed fractionation position includes: 10,20,30,40,50,
60,70,80,90, it will be appreciated by persons skilled in the art that being based on this 9 fractionation positions, serialized data can be split
At 10 data fragmentations, wherein the data on 1~10 byte location are a data fragmentation, the number on 11~20 byte locations
According to for a data fragmentation, and so on, totally 10 data fragmentations.
By foregoing description as it can be seen that carrying out physics deconsolidation process to serialized data in memory, can generate in memory more
A data fragmentation, thus committed memory, and logic deconsolidation process is carried out to serialized data in memory, relate only to multiple tear open
Quartile sets the occupancy to memory, thus, physics deconsolidation process is carried out compared to serialized data, serialized data is patrolled
Collecting deconsolidation process can be reduced in object storing process, to the occupancy of memory, lifting system performance.
Step 106: based on deconsolidation process as a result, determining multiple data fragmentations of serialized data, by multiple data point
Piece is stored into database, and stores the index information for indicating multiple data fragmentations storage location in the database.
In this specification embodiment, if using " physics deconsolidation process mode " at step 104, then, at fractionation
After reason, multiple data fragmentations can be directly obtained, multiple data fragmentation is temporarily stored in memory, is stored subsequent
Cheng Zhong can then store multiple data fragmentations in memory into database；If at step 104 using " at logic fractionation
Reason mode ", then, it, can be based on the fractionation position got after deconsolidation process, determination is more in the serialized data of script
A data fragmentation directly stores the data fragmentation determined into database in subsequent storing process.
In one embodiment, it is based on Hase database, different data fragmentations can be stored respectively to Hbase database
Record of not going together in, it will be appreciated by persons skilled in the art that in Hbase database, a row record has unique
Rowkey value.
, can also be in Hbase database in this specification embodiment based on this, such as one of Hbase database
Storage is for indicating multiple data fragmentations index information of storage location namely multiple numbers in Hbase database in row record
According to the rowkey value of fragment be expert at record in Hbase database.
In one embodiment, multiple data fragmentations be expert at record in Hbase database can directly be stored
Rowkey value, for example, storage rowkey value include: rowkey#01, rowkey#02, rowkey#03 ..., rowkey#
10。
In another embodiment, can be expert in Hbase database record with " indirect " the multiple data fragmentations of storage
Rowkey value, " indirect " storage mentioned here refers to that the quantity of storing data fragment, such as 10 stores rowkey value
Prefix, such as " # " stores the start sequence number of rowkey value, such as " 01 ".It can be seen that passing through " indirect " the multiple numbers of storage
According to the rowkey value of fragment be expert at record in Hbase database, data storage capacity can be further decreased.
It will be appreciated by persons skilled in the art that can be used for the reading of subsequent object by storing above-mentioned index information,
As for the reading process of object, it may refer to the description in following embodiment illustrated in fig. 2, be first not detailed herein.
Technical solution provided by this specification embodiment obtains target by carrying out serializing processing to target object
The serialized data of object carries out deconsolidation process to the serialized data, based on deconsolidation process as a result, determining serialized data
Multiple data fragmentations store multiple data fragmentations into database, and store for indicating multiple data fragmentation in data
The index information of storage location in library can break through the limitation of bottom storage, improve the storage performance of super large object.
Secondly, being illustrated to object read method:
Fig. 2 is referred to, for a kind of embodiment process for object read method that one exemplary embodiment of this specification provides
Figure, the object read method on the basis of object storage method, may comprise steps of exemplified by above-mentioned Fig. 1:
Step 202: target storage position being determined according to the index information pre-saved in the database, which uses
In the storage location of the multiple data fragmentations of expression respectively in the database.
By the description in above-mentioned embodiment illustrated in fig. 1 it is found that during storage object, it can store the more of the object
The index information of a data fragmentation storage location in the database, then, it, then can basis first during reading object
The index information pre-saved determines target storage position in the database, such as determines target rowkey value.
Step 204: being based on target storage position, data fragmentation is read from database.
In this step, then data can be read from database based on the target storage position got in step 202
Fragment.
It will be appreciated by persons skilled in the art that the data fragmentation read is temporarily stored in memory.
Step 206: processing is merged to the data fragmentation read.
In one embodiment, physics merging treatment can be carried out to the data fragmentation read in step 204, it is referred to herein
" physics merging treatment " i.e. refer in system memory, multiple data fragmentations are spliced, complete serialized data is obtained,
It will be appreciated by persons skilled in the art that the complete serialized data is also temporarily stored in memory.
In one embodiment, logic merging treatment can be carried out to the data fragmentation read in step 204, it is referred to herein
" logic merging treatment " can refer to, the data fragmentation read is packaged into inlet flow.Those skilled in the art can manage
Solution, inlet flow refer to read the object of a byte sequence, and when needing to read data, program then can be from inlet flow
Read data source, it can be seen that, it is handled by this kind, then can logically realize the merging of data fragmentation.
By foregoing description as it can be seen that carrying out physics merging treatment to multiple data fragmentations in memory, can generate in memory
One complete serialized data, thus committed memory, and logic merging treatment is carried out to serialized data in memory, it does not relate to
And the occupancy of memory, thus, physics merging treatment is carried out compared to multiple data fragmentations, logic is carried out to multiple data fragmentations
Merging treatment can be reduced in object reading process, to the occupancy of memory, lifting system performance.
Step 208: unserializing processing being carried out to the obtained data of merging treatment, obtains target object.
Due in object storing process, storage be object serialized data, thus, in this step, can be right
The obtained serialized data of merging treatment carries out unserializing processing, obtains target object.
In addition, if having carried out compression processing to serialized data in object storing process, then, in this step, then
Processing first can be unziped it to the obtained data of merging treatment, inverted sequence then is carried out to the data after decompression again
Columnization processing, to obtain target object.
In addition, when unziping it processing to the obtained data of merging treatment, preferably using stream nested mode into
Row decompression.
Technical solution provided by this specification embodiment, by true in the database according to the index information pre-saved
Set the goal storage location, is based on the target storage position, data fragmentation is read from database, to the data fragmentation read
Merge processing；Unserializing processing is carried out to the obtained data of merging treatment, target object is obtained, super large can be improved
The reading performance of object.
Corresponding to embodiment of the method exemplified by above-mentioned Fig. 1, this specification embodiment also provides a kind of object storage device,
It is shown in Figure 3, for a kind of embodiment block diagram for object storage device that one exemplary embodiment of this specification provides, the device
It may include: serialization module 31, split module 32 and memory module 33.
Wherein, serialization module 31 obtain the sequence of the target object for carrying out serializing processing to target object
Change data；
Module 32 is split, for carrying out deconsolidation process to the serialized data；
Memory module 33, for based on deconsolidation process as a result, multiple data fragmentations of the serialized data are determined, by institute
It states multiple data fragmentations to store into database, and stores for indicating that the multiple data fragmentation stores in the database
The index information of position.
In one embodiment, the fractionation module 32 can be specifically used for:
Based on preset fractionation rule, multiple fractionation positions in the serialized data are obtained.
In one embodiment, different data fragmentations is stored respectively into the record of not going together of the database；
The index information includes: the rowkey value of the be expert at record of data fragmentation.
It is understood that serialization module 31, splitting module 32 and memory module 33 as three kinds of functional independences
Module can both be configured in a device simultaneously as shown in Figure 3, can also individually be configured in a device, therefore shown in Fig. 3
Structure should not be construed as the restriction to this specification example scheme.
Corresponding to embodiment of the method exemplified by above-mentioned Fig. 2, this specification embodiment also provides a kind of object reading device,
It is shown in Figure 4, for a kind of embodiment block diagram for object reading device that one exemplary embodiment of this specification provides, the device
It may include: determining module 41, read module 42, merging module 43 and unserializing module 44.
Wherein it is determined that module 41, for determining target storage position in the database according to the index information pre-saved,
The index information is for indicating the storage location of multiple data fragmentations in the database；
Read module 42 reads data fragmentation for being based on the target storage position from the database；
Merging module 43, for merging processing to the data fragmentation read；
Unserializing module 44 obtains target pair for carrying out unserializing processing to the obtained data of merging treatment
As.
In one embodiment, the merging module 43 is specifically used for:
The data fragmentation read is packaged into inlet flow.
It is understood that determining module 41,44 conduct of read module 42, merging module 43 and unserializing module
The module of four kinds of functional independences can both be configured in a device simultaneously as shown in Figure 4, can also individually be configured in device
In, therefore structure shown in Fig. 4 should not be construed as the restriction to this specification example scheme.
In addition, the function of modules and the realization process of effect are specifically detailed in the above method corresponding step in above-mentioned apparatus
Rapid realization process, details are not described herein.
This specification embodiment also provides a kind of service server, includes at least memory, processor and is stored in
On reservoir and the computer program that can run on a processor, wherein processor realizes object above-mentioned when executing described program
Storage method.This method includes at least: carrying out serializing processing to target object, obtains the serializing number of the target object
According to；Deconsolidation process is carried out to the serialized data；Based on deconsolidation process as a result, determining multiple data of the serialized data
Fragment stores the multiple data fragmentation into database, and stores for indicating the multiple data fragmentation in the number
According to the index information of storage location in library.
This specification embodiment also provides another service server, includes at least memory, processor and is stored in
On memory and the computer program that can run on a processor, wherein it is above-mentioned right that processor is realized when executing described program
As read method.This method includes at least: determining target storage position, institute in the database according to the index information pre-saved
Index information is stated for indicating the storage location of multiple data fragmentations in the database；Based on the target storage position,
Data fragmentation is read from the database；Processing is merged to the data fragmentation read；To obtained by merging treatment
Data carry out unserializing processing, obtain target object.
Fig. 5 shows one kind provided by this specification embodiment, and more specifically service server hardware configuration is illustrated
Figure, which may include: processor 510, memory 520, input/output interface 530, communication interface 540 and bus 550.
Wherein processor 55, memory 520, input/output interface 530 and communication interface 540 by bus 550 realize between
Communication connection inside equipment.
Processor 510 can use general CPU (Central Processing Unit, central processing unit), micro process
Device, application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or one or
The modes such as multiple integrated circuits are realized, for executing relative program, to realize technical solution provided by this specification embodiment.
Memory 520 can use ROM (Read Only Memory, read-only memory), RAM (Random Access
Memory, random access memory), static storage device, the forms such as dynamic memory realize.Memory 520 can store
Operating system and other applications are realizing technical solution provided by this specification embodiment by software or firmware
When, relevant program code is stored in memory 520, and execution is called by processor 55.
Input/output interface 530 is for connecting input/output module, to realize information input and output.Input and output/
Module can be used as component Configuration (not shown) in a device, can also be external in equipment to provide corresponding function.Wherein
Input equipment may include keyboard, mouse, touch screen, microphone, various kinds of sensors etc., output equipment may include display,
Loudspeaker, vibrator, indicator light etc..
Communication interface 540 is used for connection communication module (not shown), to realize the communication of this equipment and other equipment
Interaction.Wherein communication module can be realized by wired mode (such as USB, cable etc.) and be communicated, can also be wirelessly
(such as mobile network, WIFI, bluetooth etc.) realizes communication.
Bus 550 includes an access, and in the various components of equipment, (such as processor 55, memory 520, input/output are connect
Mouthfuls 530 and communication interface 540) between transmit information.
It should be noted that although above equipment illustrates only processor 510, memory 520, input/output interface
530, communication interface 540 and bus 550, but in the specific implementation process, which can also include realizing to operate normally
Necessary other assemblies.In addition, it will be appreciated by those skilled in the art that, it can also be only comprising realizing in above equipment
Component necessary to this specification example scheme, without including all components shown in figure.
This specification embodiment also provides a kind of computer readable storage medium, is stored thereon with computer program, the journey
Object storage method above-mentioned is realized when sequence is executed by processor.This method includes at least: carrying out at serializing to target object
Reason, obtains the serialized data of the target object；Deconsolidation process is carried out to the serialized data；Based on deconsolidation process knot
Fruit determines multiple data fragmentations of the serialized data, the multiple data fragmentation is stored into database, and store use
In the index information for indicating the multiple data fragmentation storage location in the database.
This specification embodiment also provides another computer readable storage medium, is stored thereon with computer program, should
Object read method above-mentioned is realized when program is executed by processor.This method includes at least: being believed according to the index pre-saved
Breath determines target storage position in the database, and the index information is used to indicate multiple data fragmentations in the database
Storage location；Based on the target storage position, data fragmentation is read from the database；To the data fragmentation read
Merge processing；Unserializing processing is carried out to the obtained data of merging treatment, obtains target object.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
As seen through the above description of the embodiments, those skilled in the art can be understood that this specification
Embodiment can be realized by means of software and necessary general hardware platform.Based on this understanding, this specification is implemented
Substantially the part that contributes to existing technology can be embodied in the form of software products the technical solution of example in other words,
The computer software product can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are to make
It is each to obtain computer equipment (can be personal computer, server or the network equipment etc.) execution this specification embodiment
Method described in certain parts of a embodiment or embodiment.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.A kind of typically to realize that equipment is computer, the concrete form of computer can
To be personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
In device, navigation equipment, E-mail receiver/send equipment, game console, tablet computer, wearable device or these equipment
The combination of any several equipment.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device reality
For applying example, since it is substantially similar to the method embodiment, so describing fairly simple, related place is referring to embodiment of the method
Part explanation.The apparatus embodiments described above are merely exemplary, wherein described be used as separate part description
Module may or may not be physically separated, can be each module when implementing this specification example scheme
Function realize in the same or multiple software and or hardware.Can also select according to the actual needs part therein or
Person's whole module achieves the purpose of the solution of this embodiment.Those of ordinary skill in the art are not the case where making the creative labor
Under, it can it understands and implements.
The above is only the specific embodiment of this specification embodiment, it is noted that for the general of the art
For logical technical staff, under the premise of not departing from this specification embodiment principle, several improvements and modifications can also be made, this
A little improvements and modifications also should be regarded as the protection scope of this specification embodiment.
Claims (12)
1. a kind of object storage method, which comprises
Serializing processing is carried out to target object, obtains the serialized data of the target object；
Deconsolidation process is carried out to the serialized data；
Based on deconsolidation process as a result, determining multiple data fragmentations of the serialized data, the multiple data fragmentation is stored
Into database, and store the index information for indicating the multiple data fragmentation storage location in the database.
2. according to the method described in claim 1, described carry out deconsolidation process to the serialized data, comprising:
Based on preset fractionation rule, multiple fractionation positions in the serialized data are obtained.
3. according to the method described in claim 1, different data fragmentations is stored respectively to the record of not going together of the database
In；
The index information includes: the rowkey value of the be expert at record of data fragmentation.
4. a kind of object read method based on object storage method as described in any one of claims 1 to 3, the method packet
It includes:
Target storage position is determined in the database according to the index information pre-saved, and the index information is for indicating multiple
The storage location of data fragmentation in the database；
Based on the target storage position, data fragmentation is read from the database；
Processing is merged to the data fragmentation read；
Unserializing processing is carried out to the obtained data of merging treatment, obtains target object.
5. according to the method described in claim 4, described merge processing to the data fragmentation read, comprising:
The data fragmentation read is packaged into inlet flow.
6. a kind of object storage device, described device include:
Serialization module obtains the serialized data of the target object for carrying out serializing processing to target object；
Module is split, for carrying out deconsolidation process to the serialized data；
Memory module, for based on deconsolidation process as a result, determine multiple data fragmentations of the serialized data, will be the multiple
Data fragmentation is stored into database, and is stored for indicating the multiple data fragmentation storage location in the database
Index information.
7. device according to claim 6, the fractionation module is specifically used for:
Based on preset fractionation rule, multiple fractionation positions in the serialized data are obtained.
8. device according to claim 6, different data fragmentations is stored respectively to the record of not going together of the database
In；
The index information includes: the rowkey value of the be expert at record of data fragmentation.
9. a kind of object reading device based on the described in any item object storage devices of such as claim 6 to 8, described device packet
It includes:
Determining module, for determining target storage position, the index letter in the database according to the index information pre-saved
Breath is for indicating the storage location of multiple data fragmentations in the database；
Read module reads data fragmentation for being based on the target storage position from the database；
Merging module, for merging processing to the data fragmentation read；
Unserializing module obtains target object for carrying out unserializing processing to the obtained data of merging treatment.
10. device according to claim 9, the merging module is specifically used for:
The data fragmentation read is packaged into inlet flow.
11. a kind of service server including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, wherein the processor realizes the method according to claim 1 when executing described program.
12. a kind of service server including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, wherein the processor realizes such as the described in any item methods of claim 4-5 when executing described program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261886.XA CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261886.XA CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614372A true CN109614372A (en) | 2019-04-12 |
| CN109614372B CN109614372B (en) | 2023-06-02 |
Family
ID=66001648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811261886.XA Active CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614372B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287199A (en) * | 2019-07-01 | 2019-09-27 | 联想(北京)有限公司 | A kind of processing method and electronic equipment of database |
| CN110633283A (en) * | 2019-09-18 | 2019-12-31 | 达疆网络科技(上海)有限公司 | Method for changing data from horizontal expansion to vertical expansion |
| CN111049729A (en) * | 2019-11-29 | 2020-04-21 | 苏州浪潮智能科技有限公司 | Persistent message transmission method and device |
| CN113342824A (en) * | 2021-06-30 | 2021-09-03 | 平安资产管理有限责任公司 | Data storage method, device, equipment and medium based on target storage equipment |
| WO2022016865A1 (en) * | 2020-07-24 | 2022-01-27 | 苏州浪潮智能科技有限公司 | Service processing method and system, device, and medium |
| CN114063888A (en) * | 2020-07-31 | 2022-02-18 | 中移(苏州)软件技术有限公司 | Data storage system, data processing method, terminal and storage medium |
| CN114328501A (en) * | 2020-09-29 | 2022-04-12 | 华为技术有限公司 | Data processing method, device and equipment |
| WO2024032766A1 (en) * | 2022-08-12 | 2024-02-15 | 安徽新永拓新材料有限公司 | Method and system for storing and reading production data, and computer-readable medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101122885A (en) * | 2007-09-11 | 2008-02-13 | 腾讯科技（深圳）有限公司 | Data cache processing method, system and data cache device |
| CN101894238A (en) * | 2010-08-09 | 2010-11-24 | 中国人民解放军海军工程大学 | Double authentication-based word document electronic seal system and method |
| US20110148711A1 (en) * | 2009-12-21 | 2011-06-23 | Industrial Technology Research Institute | Positioning method and communication system using the same |
| US8553041B1 (en) * | 2008-09-10 | 2013-10-08 | Nvidia Corporation | System and method for structuring an A-buffer to support multi-sample anti-aliasing |
| CN104794231A (en) * | 2015-05-04 | 2015-07-22 | 罗敬宁 | Unstructured data processing method and device based on Map/Reduce |
| CN105007308A (en) * | 2015-06-24 | 2015-10-28 | 国家电网公司 | File transmission method for use in database isolation device environment |
| CN105138649A (en) * | 2015-08-26 | 2015-12-09 | 小米科技有限责任公司 | Data search method and device and terminal |
| CN106326377A (en) * | 2016-08-15 | 2017-01-11 | 东软集团股份有限公司 | Serialization method and device, deserialization method and device, and system |
| CN106933868A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method and data server for adjusting data fragmentation distribution |
-
2018
- 2018-10-26 CN CN201811261886.XA patent/CN109614372B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101122885A (en) * | 2007-09-11 | 2008-02-13 | 腾讯科技（深圳）有限公司 | Data cache processing method, system and data cache device |
| US8553041B1 (en) * | 2008-09-10 | 2013-10-08 | Nvidia Corporation | System and method for structuring an A-buffer to support multi-sample anti-aliasing |
| US20110148711A1 (en) * | 2009-12-21 | 2011-06-23 | Industrial Technology Research Institute | Positioning method and communication system using the same |
| CN101894238A (en) * | 2010-08-09 | 2010-11-24 | 中国人民解放军海军工程大学 | Double authentication-based word document electronic seal system and method |
| CN104794231A (en) * | 2015-05-04 | 2015-07-22 | 罗敬宁 | Unstructured data processing method and device based on Map/Reduce |
| CN105007308A (en) * | 2015-06-24 | 2015-10-28 | 国家电网公司 | File transmission method for use in database isolation device environment |
| CN105138649A (en) * | 2015-08-26 | 2015-12-09 | 小米科技有限责任公司 | Data search method and device and terminal |
| CN106933868A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method and data server for adjusting data fragmentation distribution |
| CN106326377A (en) * | 2016-08-15 | 2017-01-11 | 东软集团股份有限公司 | Serialization method and device, deserialization method and device, and system |
Non-Patent Citations (3)
| Title |
|---|
| MATIN HASHEMI 等: "Throughput-memory footprint trade-off in synthesis of streaming software on embedded multiprocessors" * |
| XUEWEN CUI 等: "Directive-Based Partitioning and Pipelining for Graphics Processing Units" * |
| 黄贵 等: "OceanBase分布式存储引擎" * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287199A (en) * | 2019-07-01 | 2019-09-27 | 联想(北京)有限公司 | A kind of processing method and electronic equipment of database |
| CN110287199B (en) * | 2019-07-01 | 2021-11-16 | 联想(北京)有限公司 | Database processing method and electronic equipment |
| CN110633283A (en) * | 2019-09-18 | 2019-12-31 | 达疆网络科技(上海)有限公司 | Method for changing data from horizontal expansion to vertical expansion |
| CN111049729A (en) * | 2019-11-29 | 2020-04-21 | 苏州浪潮智能科技有限公司 | Persistent message transmission method and device |
| WO2022016865A1 (en) * | 2020-07-24 | 2022-01-27 | 苏州浪潮智能科技有限公司 | Service processing method and system, device, and medium |
| CN114063888A (en) * | 2020-07-31 | 2022-02-18 | 中移(苏州)软件技术有限公司 | Data storage system, data processing method, terminal and storage medium |
| CN114328501A (en) * | 2020-09-29 | 2022-04-12 | 华为技术有限公司 | Data processing method, device and equipment |
| CN113342824A (en) * | 2021-06-30 | 2021-09-03 | 平安资产管理有限责任公司 | Data storage method, device, equipment and medium based on target storage equipment |
| WO2024032766A1 (en) * | 2022-08-12 | 2024-02-15 | 安徽新永拓新材料有限公司 | Method and system for storing and reading production data, and computer-readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614372B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109614372A (en) | A kind of object storage, read method, device and service server | |
| US11605087B2 (en) | Method and apparatus for identifying identity information | |
| TWI719557B (en) | Model training method and device based on gradient boosting decision tree | |
| CN110288082B (en) | Convolutional neural network model training method and device and computer readable storage medium | |
| CN107423085B (en) | Method and apparatus for deploying applications | |
| CN110391938B (en) | Method and apparatus for deploying services | |
| CN108334515A (en) | The method, apparatus and system of stack address in file are collapsed in a kind of processing | |
| US10223435B2 (en) | Data transfer between multiple databases | |
| CN109145238A (en) | Card display method and device and mobile device | |
| CN110046036A (en) | A kind of operation requests distribution method, device and equipment | |
| CN109359118A (en) | A kind of method for writing data and device | |
| KR20230014802A (en) | Method for recommending designated items | |
| JP2021111316A (en) | Method of processing information in parallel and apparatus | |
| CN109697452A (en) | Processing method, processing unit and the processing system of data object | |
| CN109101311B (en) | Display view compatibility method and electronic equipment | |
| CN110347645A (en) | A kind of electronic document deposits card method and device | |
| CN109165723A (en) | Method and apparatus for handling data | |
| US11323873B2 (en) | Method for wireless fidelity connection and related products | |
| CN107704104A (en) | List input item association method, system, equipment and storage medium | |
| CN103176843B (en) | The file migration method and apparatus of MapReduce distributed system | |
| CN106446272B (en) | Method and apparatus for sending data | |
| US20150066990A1 (en) | Systems and methods for discovering temporal patterns in time variant bipartite graphs | |
| CN108804625A (en) | A kind of optimization method, device and the computer equipment of LSM trees | |
| US20160103864A1 (en) | Structured Information Differentiation in Naming | |
| CN112486991B (en) | Database report processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |