CN102999395B - A kind of data read-write method and system - Google Patents
A kind of data read-write method and system Download PDFInfo
- Publication number
- CN102999395B CN102999395B CN201110268101.3A CN201110268101A CN102999395B CN 102999395 B CN102999395 B CN 102999395B CN 201110268101 A CN201110268101 A CN 201110268101A CN 102999395 B CN102999395 B CN 102999395B
- Authority
- CN
- China
- Prior art keywords
- data
- grouping
- write
- nearby
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
Abstract
The invention discloses a kind of data read-write method, comprising: divide multiple groupings, each grouping all can realize the service of reading and write service, when user need to read and write data, search grouping nearby; Whether the grouping nearby that judgement finds can be used, and judges that the grouping nearby finding is available, and grouping nearby described in access, carries out reading and writing data by corresponding packet interface; Judge that the grouping nearby finding is unavailable, access other groupings, carry out reading and writing data by corresponding packet interface. The present invention also correspondingly discloses a kind of data read-write system. The present invention can improve data and write reliability of service, improves service quality and user's experience, and is convenient to capacity expansion and upgrading.
Description
Technical field
The present invention relates to field of computer technology, relate in particular to a kind of data read-write method and system.
Background technology
Along with the development of network technology, web Related product is more and more, compete also more and more fiercely, is takingBusiness device is delayed in the situations such as emerging (IDC) fault in machine, internet data, network failure, and whether web productHigh-quality service still can be provided, and is user's one of top-priority factor in the time selecting web product, changes speechIt, provide the disaster tolerance service of high reliability, is a magic weapon that web product gets the upper hand of.
The data storage service of existing most of web products is all to adopt active/standby mode to carry out disaster tolerance, or passes throughMain writing for the mode of reading processed, and still, existing active/standby mode has only ensured number by multiple stand-by equipmentsAccording to reading disaster tolerance, still there is single-point hidden danger and write data to host apparatus, once host apparatus or its placeIDC breaks down, and will cause data to write failure, thereby can seriously reduce service quality and affect user's bodyTest.
Summary of the invention
In view of this, main purpose of the present invention is to provide a kind of data read-write method and system, Neng GoutiHigh data are write reliability of service, improve service quality and user's experience.
For achieving the above object, technical scheme of the present invention is achieved in that
A kind of data read-write method, divides multiple groupings, and each grouping all can realize the service of reading and write service,The method comprises:
When user need to read and write data, search grouping nearby;
Whether the grouping nearby that judgement finds can be used, and judges that the grouping nearby finding is available, accesses instituteState grouping nearby, carry out reading and writing data by corresponding packet interface; Judge that the grouping nearby finding can notWith, access other groupings, carry out reading and writing data by corresponding packet interface.
The method also comprises: data are after packet memory, and described grouping is selectively beyond described groupingAt least one packet synchronization described in the data of storing.
The method also comprises: in the process to grouping data writing, event occurs for packet interface or interface linkBarrier, described packet interface forwards the data that need to write to other packet interfaces, or, be switched to otherPacket interface data writing.
A kind of data read-write system, comprising: control module, multiple grouping and divide corresponding with groupingGroup interface; Wherein,
Described control module, in the time that user need to read and write data, search grouping nearby, and judgement is searchedTo grouping nearby whether can use, judge that the grouping nearby that finds is available, grouping nearby described in access,Carry out reading and writing data by corresponding packet interface; Judge that the grouping nearby finding is unavailable, access itHe divides into groups, and carries out reading and writing data by corresponding packet interface.
Described each grouping at least comprises reads serve and write service, and the method also comprises: data are by writing serviceAfter packet memory, the service of writing of described grouping is selectively to described grouping at least one grouping in additionWrite the data of storing described in service synchronization.
Described packet interface, also, in the time that packet interface or interface link break down, connects to other groupingsMouth forwards the data that need to write.
Described control module, also, for the process at grouping data writing, packet interface or interface link are sent outWhen raw fault, control and be switched to other packet interface data writings.
Described grouping comprises: write service, read service, master library, standby storehouse and buffer memory; Wherein, data are write fashionable,The service of writing is to master library data writing, and after data write, master library, to standby storehouse synchrodata, when data read, is readFirst reading out data from buffer memory of service, if buffer memory does not have,, from standby storehouse reading out data, returns resultsThese data are synchronized in buffer memory, for reading next time simultaneously.
Described grouping comprises: write service, read service, buffer memory and database; Wherein, application call is writeService, TTC data meeting automatic synchronization, in database, calls while reading to serve, if had in TTC buffer memory,Directly return, if not, to data base querying, then result is updated to buffer memory and returns again.
Described grouping comprises: write interface, read interface, database, at least one service of reading and with read to serve rightThe buffer memory of answering; Wherein, write operation is directly updated in database by writing interface, writes interface and will write and disappear simultaneouslyBreath is transmitted to and reads interface portion, reads interface this is write to message and is forwarded to the corresponding service of reading, to this is write and is disappearedBreath is updated in buffer memory.
Data read-write method and system that the present invention proposes, divide multiple groupings, and each grouping all can realize and readingServe and write service, when user need to read and write data, searching grouping nearby; The grouping nearby that judgement findsWhether can use, judge that the grouping nearby finding is available, grouping nearby described in access, by corresponding pointGroup interface carries out reading and writing data; Judge that the grouping nearby finding is unavailable, access other groupings, pass throughCorresponding packet interface carries out reading and writing data. The present invention can improve data and write reliability of service, improves clothesBusiness quality and user experience, and are convenient to capacity expansion and upgrading.
Brief description of the drawings
Fig. 1 is data read-write method schematic flow sheet of the present invention;
Fig. 2 is the principle schematic of Memchache cache way;
Fig. 3 is the principle schematic of TTC cache way;
Fig. 4 is the principle schematic of full dose disjoin buffer mode;
Fig. 5 is that the present invention is grouped into example with two, corresponding to the system architecture schematic diagram of Fig. 2;
Fig. 6 is that the present invention is grouped into example with two, corresponding to the system architecture schematic diagram of Fig. 3;
Fig. 7 is that the present invention is grouped into example with two, corresponding to the system architecture schematic diagram of Fig. 4.
Detailed description of the invention
Basic thought of the present invention is: divide multiple groupings, each grouping all can realize the service of reading and write service,When user need to read and write data, search grouping nearby; Whether the grouping nearby that judgement finds can be used, and judgesThe grouping nearby finding is available, and grouping nearby described in access, carries out data by corresponding packet interfaceRead-write; Judge that the grouping nearby finding is unavailable, access other groupings, by corresponding packet interfaceCarry out reading and writing data.
The present invention is in conjunction with read-write separation, grouping (set) model, distributed strategy, for magnanimity processing, heightThe features such as reliability provide a complete data storage scheme, and in the present invention, grouping is one group and is correlated withThe combination of resource, can provide user's service of relatively-stationary some, is also one and disposes unit.
Fig. 1 is data read-write method schematic flow sheet of the present invention, and as shown in Figure 1, the method comprises:
Step 101: user need to read and write data.
Step 102: search grouping nearby.
In the present invention, divide multiple groupings, each grouping all can realize the service of reading and write service, and each pointGroup has a unique number, and when storage data, the numbering (key) of each data item is taking its packet numbering as frontSew, can ensure like this global uniqueness of data.
Step 103: whether the grouping nearby that judgement finds can be used, if the grouping nearby finding is available,Execution step 104; If the grouping nearby finding is unavailable, perform step 105.
It should be noted that, the grouping nearby finding can be one and also can, for multiple, find multipleWhile grouping nearby, successively each is divided into groups to judge nearby, until find available grouping nearby or traversalAll groupings nearby that find, stop judgement.
Step 104: grouping nearby described in access, carry out reading and writing data.
Step 105: access other groupings, carry out reading and writing data.
It should be noted that, in the present invention, data are after certain packet memory, and this grouping can be selectivelyTo the data of storing described at least one packet synchronization beyond this grouping, and there is the mechanism of reforming of synchronization failure,For example, this grouping is recorded to the binary protocol bag of synchronization failure in file, and the form of file is: startMark (0xfd58646c)+unix timestamp+bag length+message bag+end mark (0x6c6458fd), filenameFor: IP address+port+method of synchronization (tcp be 1 or udp be 2)+date+hour (0-23)+sequence number (0-3,Because every 15 minutes files), then there is message synchronization instrument to enter according to the content in filename and fileRow real-time synchronization, every synchronous message bag, records the file line number of the next protocol package of this file, so thatWhen synchronization means is restarted, can directly start to process from this row; If treated to last column, per secondCheck whether this file has new bag to write, until there is next failed file to produce; If synchronous protocol bag is sameWalk unsuccessfully, alarm, and continuous retry is until success.
It should be noted that, if in the process to grouping data writing, packet interface or interface link are sent outRaw fault, this packet interface need to forward the data that need to write to other packet interfaces, or, systemBe switched to other packet interface data writings.
General, the read-write service of data is separated, make to read service or the service of writing and all facilitate expansion separately, readWrite to separate and mainly contain three kinds of modes below according to business difference:
Fig. 2 is the principle schematic of Memchache cache way, and as shown in Figure 2, data are write fashionable, writeService is to master library data writing, and after data write, master library, to standby storehouse synchrodata, when data read, is read clothesFirst reading out data from buffer memory of business, if buffer memory does not have,, from standby storehouse reading out data, what return results is sameIn time, is synchronized to these data in buffer memory, for reading next time.
Fig. 3 is the principle schematic of TTC (tencenttablecache) cache way, as shown in Figure 3, and TTCFor the database caches service of company, there are independently interface and api, service is write in application call, TTCData meeting automatic synchronization, in database, calls while reading to serve, if had in TTC buffer memory, directly returns,If not, to database (DB) inquiry, then result is updated to buffer memory and returns again.
Fig. 4 is the principle schematic of full dose disjoin buffer mode, and as shown in Figure 4, write operation is directly by leftThe interface of writing on limit is updated in database, writes interface and will write message simultaneously and be transmitted to and read interface portion, reads interfaceThis is write to message and is forwarded to the corresponding service of reading, to this is write to information updating in buffer memory (cache), asFruit is write interface and writes failure to reading interface, records synchronous documents, reforms, and read operation side is directly by readingInterface carries out reading of information to the corresponding service of reading.
The present invention also correspondingly proposes a kind of data read-write system, and this system comprises: control module, Duo GefenGroup and the packet interface corresponding with grouping; Wherein,
Described control module, in the time that user need to read and write data, search grouping nearby, and judgement is searchedTo grouping nearby whether can use, judge that the grouping nearby that finds is available, grouping nearby described in access,Carry out reading and writing data by corresponding packet interface; Judge that the grouping nearby finding is unavailable, access itHe divides into groups, and carries out reading and writing data by corresponding packet interface.
Described each grouping at least comprises reads serve and write service, and the method also comprises: data are by writing serviceAfter packet memory, the service of writing of described grouping is selectively to described grouping at least one grouping in additionWrite the data of storing described in service synchronization.
Described packet interface, also, in the time that packet interface or interface link break down, connects to other groupingsMouth forwards the data that need to write.
Described control module, also, for the process at grouping data writing, packet interface or interface link are sent outWhen raw fault, control and be switched to other packet interface data writings.
Described grouping comprises: write service, read service, master library, standby storehouse and buffer memory; Wherein, data are write fashionable,The service of writing is to master library data writing, and after data write, master library, to standby storehouse synchrodata, when data read, is readFirst reading out data from buffer memory of service, if buffer memory does not have,, from standby storehouse reading out data, returns resultsThese data are synchronized in buffer memory, for reading next time simultaneously.
Described grouping comprises: write service, read service, buffer memory and database; Wherein, application call is writeService, TTC data meeting automatic synchronization, in database, calls while reading to serve, if had in TTC buffer memory,Directly return, if not, to data base querying, then result is updated to buffer memory and returns again.
Described grouping comprises: write service, read service, database, at least one read interface and with read interface pairThe buffer memory of answering; Wherein, write operation is directly updated in database by the interface of writing on the left side, writes interface simultaneouslyTo write message and be transmitted to and read interface portion, and read interface this is write to message and is forwarded to the corresponding service of reading, so that willThis writes information updating in buffer memory, writes failure if write interface to reading interface, records synchronous documents, carries outReform, read operation side is directly carried out reading of information by reading interface to the corresponding service of reading.
Be grouped into example with two, corresponding to Fig. 2, the structure of data read-write system of the present invention as shown in Figure 5,The service of reading of packet interface 1 and grouping 1 and write between service to exist and link, packet interface 2 with divide into groups 2The service of reading and write between service and to have link, packet interface 1 and packet interface 2 also can be respectively and grouping 2And the service of reading of grouping 1 and write between service and to have link, and, the service of writing of grouping 1 and grouping 2Between there is link, for synchronous between group.
Be grouped into example with two, corresponding to Fig. 3, the structure of data read-write system of the present invention as shown in Figure 6,The service of reading of packet interface 3 and grouping 3 and write between service to exist and link, packet interface 4 with divide into groups 4The service of reading and write between service and to have link, packet interface 3 and packet interface 4 also can be respectively and grouping 4And the service of reading of grouping 3 and write between service and to have link, and, the service of writing of grouping 3 and grouping 4Between there is link, for synchronous between group.
Be grouped into example with two, corresponding to Fig. 4, the structure of data read-write system of the present invention as shown in Figure 7,The service of reading of packet interface 5 and grouping 5 and write between service to exist and link, packet interface 6 with divide into groups 6The service of reading and write between service and to have link, packet interface 5 and packet interface 6 also can be respectively and grouping 6And the service of reading of grouping 5 and write between service and to have link, and, the service of writing of grouping 5 and grouping 6Between there is link, for synchronous between group.
Can find out, the present invention is by having carried out distributed deployment by read-write service by grouping model, and user readsWhile writing data, preferentially access services grouping nearby, has improved performance; Divide into groups nearby when unavailable, Ke YifangAsk other groupings, improved service quality, thereby, the technic relization scheme providing in the present invention is provided, canTo realize the reading and writing data service of high-performance, high reliability, and can facilitate capacity expansion and upgrading.
The above, be only preferred embodiment of the present invention, is not intended to limit protection model of the present inventionEnclose.
Claims (8)
1. a data read-write method, is characterized in that, divides multiple groupings, and each grouping all can realize and readingServe and write service, the method comprises:
When user need to read and write data, search grouping nearby;
Find multiplely while dividing into groups nearby, successively each is divided into groups to judge nearby, until find availableDivide into groups nearby or travel through all groupings nearby that find, accessing described available grouping nearby, by correspondingPacket interface carry out reading and writing data; Judge that the grouping nearby finding is unavailable, access other groupings,Carry out reading and writing data by corresponding packet interface;
Described grouping comprises: write interface, read interface, database, at least one service of reading and with read to serve rightThe buffer memory of answering; Wherein, write operation is directly updated in database by writing interface, writes interface and will write and disappear simultaneouslyBreath is transmitted to and reads interface portion, reads interface this is write to message and is forwarded to the corresponding service of reading, to this is write and is disappearedBreath is updated in buffer memory.
2. data read-write method according to claim 1, is characterized in that, the method also comprises: numberAccording to after packet memory, described grouping is selectively to described grouping at least one packet synchronization institute in additionState the data of storage.
3. data read-write method according to claim 1, is characterized in that, the method also comprises:To grouping data writing process in, packet interface or interface link break down, described packet interface toOther packet interfaces forward the data that need to write, or, be switched to other packet interface data writings.
4. a data read-write system, is characterized in that, this system comprises: control module, multiple grouping,And the packet interface corresponding with grouping; Wherein,
Described control module, in the time that user need to read and write data, searches grouping nearby, finds multipleWhile grouping nearby, successively each is divided into groups to judge nearby, until find available grouping nearby or traversalAll groupings nearby that find, access described available grouping nearby, enter by corresponding packet interfaceRow reading and writing data; Judge that the grouping nearby finding is unavailable, access other groupings, by corresponding pointGroup interface carries out reading and writing data;
Described grouping comprises: write interface, read interface, database, at least one service of reading and with read to serve rightThe buffer memory of answering; Wherein, write operation is directly updated in database by writing interface, writes interface and will write and disappear simultaneouslyBreath is transmitted to and reads interface portion, reads interface this is write to message and is forwarded to the corresponding service of reading, to this is write and is disappearedBreath is updated in buffer memory.
5. data read-write system according to claim 4, is characterized in that,
Described packet interface, also, in the time that packet interface or interface link break down, connects to other groupingsMouth forwards the data that need to write.
6. data read-write system according to claim 4, is characterized in that,
Described control module, also, for the process at grouping data writing, packet interface or interface link are sent outWhen raw fault, control and be switched to other packet interface data writings.
7. a data read-write system, is characterized in that, this system comprises: control module, multiple grouping,And the packet interface corresponding with grouping; Wherein,
Described control module, in the time that user need to read and write data, searches grouping nearby, finds multipleWhile grouping nearby, successively each is divided into groups to judge nearby, until find available grouping nearby or traversalAll groupings nearby that find, access described available grouping nearby, enter by corresponding packet interfaceRow reading and writing data; Judge that the grouping nearby finding is unavailable, access other groupings, by corresponding pointGroup interface carries out reading and writing data;
Described grouping comprises: write service, read service, master library, standby storehouse and buffer memory; Wherein, data are write fashionable,The service of writing is to master library data writing, and after data write, master library, to standby storehouse synchrodata, when data read, is readFirst reading out data from buffer memory of service, if buffer memory does not have,, from standby storehouse reading out data, returns resultsThese data are synchronized in buffer memory, for reading next time simultaneously.
8. a data read-write system, is characterized in that, this system comprises: control module, multiple grouping,And the packet interface corresponding with grouping; Wherein,
Described control module, in the time that user need to read and write data, searches grouping nearby, finds multipleWhile grouping nearby, successively each is divided into groups to judge nearby, until find available grouping nearby or traversalAll groupings nearby that find, access described available grouping nearby, enter by corresponding packet interfaceRow reading and writing data; Judge that the grouping nearby finding is unavailable, access other groupings, by corresponding pointGroup interface carries out reading and writing data;
Described grouping comprises: write service, read service, buffer memory and database; Wherein, application call is writeService, TTC data meeting automatic synchronization, in database, calls while reading to serve, if had in TTC buffer memory,Directly return, if not, to data base querying, then result is updated to buffer memory and returns again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110268101.3A CN102999395B (en) | 2011-09-09 | 2011-09-09 | A kind of data read-write method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110268101.3A CN102999395B (en) | 2011-09-09 | 2011-09-09 | A kind of data read-write method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102999395A CN102999395A (en) | 2013-03-27 |
| CN102999395B true CN102999395B (en) | 2016-05-18 |
Family
ID=47927996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110268101.3A Active CN102999395B (en) | 2011-09-09 | 2011-09-09 | A kind of data read-write method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102999395B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103414545B (en) * | 2013-07-31 | 2016-09-14 | 东软集团股份有限公司 | Overtime determination methods between heterogeneous system and system |
| CN104572638B (en) * | 2013-10-09 | 2019-03-26 | 腾讯科技（深圳）有限公司 | Data read-write method and device |
| CN107704603A (en) * | 2017-10-16 | 2018-02-16 | 山东浪潮通软信息科技有限公司 | A kind of method and device for realizing read and write abruption |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102033912A (en) * | 2010-11-25 | 2011-04-27 | 北京北纬点易信息技术有限公司 | Distributed-type database access method and system |
| CN102143199A (en) * | 2010-10-19 | 2011-08-03 | 华为技术有限公司 | Content acquisition method, node and content network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4104281B2 (en) * | 2000-10-25 | 2008-06-18 | 株式会社日立製作所 | Database access method |
| CN101043310B (en) * | 2007-04-27 | 2010-09-08 | 北京佳讯飞鸿电气股份有限公司 | Image backup method for dual-core control of core controlled system |
-
2011
- 2011-09-09 CN CN201110268101.3A patent/CN102999395B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143199A (en) * | 2010-10-19 | 2011-08-03 | 华为技术有限公司 | Content acquisition method, node and content network |
| CN102033912A (en) * | 2010-11-25 | 2011-04-27 | 北京北纬点易信息技术有限公司 | Distributed-type database access method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102999395A (en) | 2013-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7090744B2 (en) | Distributed database cluster system and data synchronization method | |
| CN103699580B (en) | Database synchronization method and device | |
| CN102521269B (en) | Index-based computer continuous data protection method | |
| CN103116661B (en) | A kind of data processing method of database | |
| CN104111804B (en) | A kind of distributed file system | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| CN102012933B (en) | Distributed file system and method for storing data and providing services by utilizing same | |
| CN103530362B (en) | A kind of computer data reading/writing method for many copies distributed system | |
| CN102955720B (en) | A kind of method improving ext file system stability | |
| EP2369494A1 (en) | Web application based database system and data management method therof | |
| CN106557592A (en) | Method of data synchronization, device and server cluster | |
| WO2007115477A1 (en) | Data synchronization method und system | |
| US9195611B2 (en) | Efficiently updating and deleting data in a data storage system | |
| CN101997823A (en) | Distributed file system and data access method thereof | |
| CN103258036A (en) | Distributed real-time search engine based on p2p | |
| EP4213038A1 (en) | Data processing method and apparatus based on distributed storage, device, and medium | |
| CN102750326A (en) | Log management optimization method of cluster system based on downsizing strategy | |
| US7814178B2 (en) | Method and apparatus for data configuration in communication device | |
| CN111597160A (en) | Distributed database system, distributed data processing method and device | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| CN102999395B (en) | A kind of data read-write method and system | |
| CN101751470B (en) | System for storing and/or retrieving a data-set and method thereof | |
| US20150169623A1 (en) | Distributed File System, File Access Method and Client Device | |
| CN107370809A (en) | Method of data synchronization and data search system | |
| CN108958969B (en) | Database disaster recovery method, device and disaster recovery and backup systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |