CN107368438A - A kind of dirty data detection method and system - Google Patents
A kind of dirty data detection method and system Download PDFInfo
- Publication number
- CN107368438A CN107368438A CN201710823216.1A CN201710823216A CN107368438A CN 107368438 A CN107368438 A CN 107368438A CN 201710823216 A CN201710823216 A CN 201710823216A CN 107368438 A CN107368438 A CN 107368438A
- Authority
- CN
- China
- Prior art keywords
- data
- dirty
- dirty data
- host side
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0061—Error detection codes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/14—Protection against unauthorised use of memory or access to memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
This application discloses a kind of dirty data detection method and system, in the present invention, this method includes：When storage end receive host side transmission data, then using CRC detect data in whether there is dirty data；If dirty data in data be present, stop the transmission to dirty data.It can be seen that, when data are transferred to storage end by host side, whether storage end begins with CRC correct to detect data, if there is dirty data among data, then storage end can stop continuing to transmit by data immediately, so dirty data is just detected, and can prevent from writing dirty data on storage hard disk, so as to improve the storage efficiency of storage hard disk and its stability.
Description
Technical field
The present invention relates to computer communication technology field, more particularly to a kind of dirty data detection method and system.
Background technology
Data interaction between front end and server is one of core business of B/S systems, how to maintain system high efficiency, fast
Speed operating is always a study hotspot of the communications field.Dirty data (Dirty Read) refers to that the data in system are not giving
In the range of it is either meaningless to practical business, or data format is illegal, and exists in the system of source lack of standardization
Coding and ambiguous service logic.Study carefully the main reason for its dirty data is present be because source system design it is not tight enough, it is dirty
The main forms of data have：Data formatting error, service logic are unreasonable and violate business rule etc..
At present, the method for conventional process dirty data is to detect storage using Common business Oriented Language, C language either SQL statement
Whether there is in system has dirty data, but these methods need the substantial amounts of programing work of programmer, and the boundary of these language
Face processing function is not very strong, and execution efficiency is low, slow for the inquiry velocity of dirty data, particularly when the number that needs are inquired about
During according to measuring especially big, the inquiry velocities of data will dramatic decrease, greatly affected the operating efficiency of user, while also reduce
The Consumer's Experience of user.With the fast development of internet industry, traditional data query mode can not meet people
Needs, particularly dirty data presence, in place of bringing inconvenience to data query.Therefore, how quickly and accurately to examine
Dirty data is surveyed, is those skilled in the art's problem urgently to be resolved hurrily.
The content of the invention
In view of this, it is an object of the invention to provide a kind of dirty data detection method and system, its object is to quick
Ground detects dirty data, to improve the performance of whole storage system and stability.Its concrete scheme is as follows：
A kind of dirty data detection method, including：
When storage end receives the data of host side transmission, then using CRC detect in the data whether
Dirty data be present；
If the dirty data in the data be present, stop the transmission to the dirty data.
Preferably, before the process of the data that host side transmission is received when storage end, in addition to：
The data are sent to the storage end by the host side by network.
Preferably, before the process of the data that host side transmission is received when storage end, in addition to：
The data are sent to the storage end by the host side by optical fiber.
Preferably, if the dirty data in the data be present, the process of the transmission to the dirty data is stopped,
Including：
If the dirty data in the data be present, error message corresponding to the dirty data is reported into the main frame
End；
When the host side receives the error message, then stop the transmission to the dirty data.
Preferably, in addition to：
If the dirty data is not present in the data, continue to be transmitted the data.
Preferably, it is described when the dirty data is not present in the data, then continue the mistake being transmitted to the data
After journey, in addition to：
If the data are damaged when writing storage hard disk, forbid the data writing the storage hard disk.
Preferably, it is described to be damaged when the data write storage hard disk, then forbid described in data write-in
The process of storage hard disk, including：
If the data are damaged when writing the storage hard disk, Write Opcode corresponding to the data is fed back
To the host side；
When the host side receives the Write Opcode, then the error message of the data is recorded, and by the data
It is re-transmitted to the storage end；
When the storage end receives the data, then judge whether the data are damaged；
If the corrupted data, forbid the data writing the storage hard disk.
Preferably, it is described when the storage end receives the data, then judge process that whether data damage it
Afterwards, in addition to：
If the data are not damaged, the data are write into the storage hard disk.
Accordingly, the invention also discloses a kind of dirty data detecting system, including：
Dirty data detection module, for receiving the data of host side transmission when storage end, then utilize CRC
Code, which is detected in the data, whether there is dirty data；
Dirty data stops transport module, for when the dirty data in the data be present, then stopping to the dirty data
Transmission.
Preferably, in addition to：
Dirty data transport module, for when the dirty data is not present in the data, then continuing to carry out the data
Transmission.
In the present invention, a kind of dirty data detection method, including：It is when storage end receives the data of host side transmission, then sharp
Detected with CRC in data and whether there is dirty data；If dirty data in data be present, stop to dirty data
Transmission.It can be seen that when data are transferred to storage end by host side, storage end begins with CRC and carrys out testing number
Correctly whether according to, if there is dirty data among data, storage end can stop continuing to transmit by data immediately, by so
Method detect dirty data, can effectively prevent from writing dirty data on storage hard disk, so as to improving depositing for storage hard disk
Store up efficiency and its stability.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is a kind of dirty data detection method flow chart that the embodiment of the present invention one provides；
Fig. 2 is a kind of dirty data detection method flow chart that the embodiment of the present invention two provides；
Fig. 3 is a kind of dirty data detecting system structure chart that the embodiment of the present invention three provides.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
The embodiment of the present invention one discloses a kind of dirty data detection method, and shown in Figure 1, this method includes：
Step S11：When storage end receive host side transmission data, then using CRC detect data in
With the presence or absence of dirty data；
It is understood that it have passed through a variety of transmission parts, biography from during host side is sent to storage end in data
Defeated passage and processing software, if one link of any of which goes wrong, can all cause data when writing storage hard disk
Make a mistake.But among the application of reality, usually user can just have found data in the data in reading storage hard disk
It is wrong, so not only reduces the storage efficiency and stability of storage hard disk, and reduces the efficiency of the work of user.
CRC (Cyclic Redundancy Check, CRC) is a kind of debugging check code, and it is to utilize
The principle of division and remainder makees error detection, and in the present embodiment, host side is by CRC and needs to send
Data together send to storage end, when storage end receive host side transmission data after, cyclic redundancy school can be recalculated
Test code and compared with the CRC received, if two CRC differences, illustrate data logical
There is mistake during news transmission, by such method, to detect whether have dirty data among the data of transmission, and then
Ensure correctness and integrality of the data in transmitting procedure.
Step S12：If dirty data in data be present, stop the transmission to dirty data.
It is understood that when detecting to there are dirty data among data using CRC, then stop
It transmission to data, can thus avoid the data by mistake from being written on storage hard disk, and then improve depositing for storage hard disk
Store up efficiency and stability.
In the present invention, a kind of dirty data detection method, including：It is when storage end receives the data of host side transmission, then sharp
Detected with CRC in data and whether there is dirty data；If dirty data in data be present, stop to dirty data
Transmission.It can be seen that when data are transferred to storage end by host side, storage end begins with CRC and carrys out testing number
Correctly whether according to, if there is dirty data among data, storage end can stop continuing to transmit by data immediately, by so
Method detect dirty data, can effectively prevent from writing dirty data on storage hard disk, so as to improving depositing for storage hard disk
Store up efficiency and its stability.
The present invention implements two and discloses a kind of specific dirty data detection method, relative to a upper embodiment, the present embodiment
Further instruction and optimization have been made to technical scheme.It is as shown in Fig. 2 specific：
Step S21：When storage end receive host side transmission data, then using CRC detect data in
With the presence or absence of dirty data；
It is understood that host side together sends CRC and the data for needing to send to storage end,
After storage end receives the data of host side transmission, CRC and the cyclic redundancy with receiving can be recalculated
Check code compares, if two CRC differences, illustrate that mistake occur during communication transfer in data,
By such method, to detect whether have dirty data among the data of host side transmission, and then ensure data in transmitting procedure
In correctness and integrality.
Specifically, before above-mentioned steps S21, in addition to step S01 either step S02.
Step S01：Data are sent to storage end by host side by network.
Step S02：Data are sent to storage end by host side by optical fiber.
In the present embodiment, storage end can be transferred data to by network or is passed data by optical fiber
Defeated to arrive storage end, certainly, specifically used any mode is transmitted to data herein, and all are to reach practical application as mesh
, it is not construed as limiting herein.
Step S22：If dirty data in data be present, stop the transmission to dirty data.
Specifically, step S22 includes step S221 and step S222.
Step S221：If dirty data in data be present, error message corresponding to dirty data is reported into host side；
Step S222：When host side receives error message, then stop the transmission to dirty data.
It is understood that it can effectively prevent from the information of mistake writing storage by step S221 and step S222
Hard disk.
Step S23：If dirty data is not present in data, continue to be transmitted data.
In the present embodiment, host side has dirty number by using CRC to detect to whether there is among data
According to if during verification, not finding to contain dirty data among data, then continue to transmit by data.It can manage
Solution, passes through such mode, it is ensured that dirty data is will be free among the data of transmission, and then ensures transmission data
Accuracy.
Step S24：If data are damaged when writing storage hard disk, forbid writing data into storage hard disk.
Specifically, step S24 includes step S241, step S242, step S243 and step S244.
Step S241：If data are damaged when writing storage hard disk, Write Opcode corresponding to data is fed back to
Host side；
Step S242：When host side receives Write Opcode, then the error message of record data, and data are resend
To storage end；
Step S243：When storage end receives data, then judge whether data are damaged；
Step S244：If corrupted data, forbid writing data into storage hard disk.
In the present embodiment, when data are damaged during storage hard disk is write, then forbid writing data into and deposit
Store up hard disk, so it is prevented that by dirty data write storage hard disk in, so as to improve the storage efficiency of storage hard disk and surely
It is qualitative.
Specifically, above-mentioned when storage end receives data, then after judging the process whether data are damaged, in addition to step
Rapid S255.
Step S255：If data are not damaged, storage hard disk is write data into.
In the present embodiment, if data are during transmission, it is not damaged, then directly writes data into storage
In hard disk, the accuracy of data in write-in storage hard disk is so ensured that, and then improves the utilization rate of storage hard disk, and
User can be ensured during data are inquired about, be not in wrong data and dirty data, so as to improve the work of user effect
Rate.
Accordingly, the embodiment of the present invention three also discloses a kind of dirty data detecting system, as shown in figure 3, the system includes：
Dirty data detection module 31, for receiving the data of host side transmission when storage end, then utilize cyclic redundancy school
Test in code detection data and whether there is dirty data；
Dirty data stops transport module 32, for when dirty data in data be present, then stopping the transmission to dirty data.
Specifically, the dirty data detecting system also includes：
First data transmission module, for when storage end receive host side transmission data process before, by data
Storage end is sent to by network by host side.
Specifically, the dirty data detecting system also includes：
Second data transmission module, for when storage end receive host side transmission data process before, by data
Storage end is sent to by optical fiber by host side.
Specifically, dirty data stops transport module 32, including：
Error message reporting unit, for when dirty data in data be present, then reporting error message corresponding to dirty data
To host side；
Dirty data stops transmission unit, for receiving error message when host side, then stops the transmission to dirty data.
Specifically, the dirty data detecting system also includes：
Data transmission module, for when dirty data is not present in data, then continuing to be transmitted data.
Specifically, the dirty data detecting system also includes：
Data forbid writing module, for when dirty data is not present in data, then continuing the process being transmitted to data
Afterwards, if data are damaged in write-in storage hard disk, forbid writing data into storage hard disk.
Specifically, data forbid writing module to include：
Data feedback unit, for being damaged when data are writing storage hard disk, then by write operation corresponding to data
Code feeds back to host side；
Data transmission unit, for receiving Write Opcode when host side, then the error message of record data, and by data
It is re-transmitted to storage end；
Data determining unit, for receiving data when storage end, then judge whether data are damaged；
Data forbid writing unit, for working as corrupted data, then forbid writing data into storage hard disk.
Specifically, the dirty data detecting system also includes：
Data write unit, for receiving data when storage end, then after judging the process whether data are damaged, if number
According to not damaging, then storage hard disk is write data into.
Finally, it is to be noted that, herein, such as first and second or the like relational terms be used merely to by
One entity or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or operation
Between any this actual relation or order be present.Moreover, term " comprising ", "comprising" or its any other variant meaning
Covering including for nonexcludability, so that process, method, article or equipment including a series of elements not only include that
A little key elements, but also the other element including being not expressly set out, or also include for this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element limited by sentence "including a ...", is not arranged
Except other identical element in the process including the key element, method, article or equipment being also present.
A kind of dirty data detection method provided by the present invention and system are described in detail above, it is used herein
Specific case is set forth to the principle and embodiment of the present invention, and the explanation of above example is only intended to help and understands
The method and its core concept of the present invention；Meanwhile for those of ordinary skill in the art, according to the thought of the present invention, having
There will be changes in body embodiment and application, in summary, this specification content should not be construed as to the present invention
Limitation.
Claims (10)
- A kind of 1. dirty data detection method, it is characterised in that including：When storage end receives the data of host side transmission, then detected in the data and whether there is using CRC Dirty data；If the dirty data in the data be present, stop the transmission to the dirty data.
- 2. according to the method for claim 1, it is characterised in that the data that host side transmission is received when storage end Before process, in addition to：The data are sent to the storage end by the host side by network.
- 3. according to the method for claim 1, it is characterised in that the data that host side transmission is received when storage end Before process, in addition to：The data are sent to the storage end by the host side by optical fiber.
- 4. according to the method for claim 1, it is characterised in that if the dirty data be present in the data, stop The only process of the transmission to the dirty data, including：If the dirty data in the data be present, error message corresponding to the dirty data is reported into the host side；When the host side receives the error message, then stop the transmission to the dirty data.
- 5. according to the method described in any one of Claims 1-4, it is characterised in that also include：If the dirty data is not present in the data, continue to be transmitted the data.
- 6. according to the method for claim 5, it is characterised in that it is described when the dirty data is not present in the data, then After continuing the process that is transmitted to the data, in addition to：If the data are damaged when writing storage hard disk, forbid the data writing the storage hard disk.
- 7. according to the method for claim 6, it is characterised in that described to be damaged when the data write storage hard disk It is bad, then forbid writing the data process of the storage hard disk, including：If the data are damaged when writing the storage hard disk, Write Opcode corresponding to the data is fed back into institute State host side；When the host side receives the Write Opcode, then the error message of the data is recorded, and by the data again Send to the storage end；When the storage end receives the data, then judge whether the data are damaged；If the corrupted data, forbid the data writing the storage hard disk.
- 8. according to the method for claim 7, it is characterised in that it is described when the storage end receives the data, then sentence After the process whether data of breaking are damaged, in addition to：If the data are not damaged, the data are write into the storage hard disk.
- A kind of 9. dirty data detecting system, it is characterised in that including：Dirty data detection module, for receiving the data of host side transmission when storage end, then examined using CRC Survey and whether there is dirty data in the data；Dirty data stops transport module, for when the dirty data in the data be present, then stopping the biography to the dirty data It is defeated.
- 10. system according to claim 9, it is characterised in that also include：Dirty data transport module, for when the dirty data is not present in the data, then continuing to be transmitted the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710823216.1A CN107368438A (en) | 2017-09-13 | 2017-09-13 | A kind of dirty data detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710823216.1A CN107368438A (en) | 2017-09-13 | 2017-09-13 | A kind of dirty data detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107368438A true CN107368438A (en) | 2017-11-21 |
Family
ID=60302791
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710823216.1A Pending CN107368438A (en) | 2017-09-13 | 2017-09-13 | A kind of dirty data detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107368438A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112633206A (en) * | 2020-12-28 | 2021-04-09 | 上海眼控科技股份有限公司 | Dirty data processing method, device, equipment and storage medium |
| CN113837278A (en) * | 2021-09-24 | 2021-12-24 | 厦门市美亚柏科信息股份有限公司 | Method and device for detecting dirty data |
| CN115167767A (en) * | 2022-06-28 | 2022-10-11 | 慧之安信息技术股份有限公司 | Dirty data prevention method and system based on BBC exclusive or check |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101453478A (en) * | 2007-11-28 | 2009-06-10 | 新奥特（北京）视频技术有限公司 | Integrity checking method in document transmission |
| CN102136993A (en) * | 2010-07-29 | 2011-07-27 | 华为技术有限公司 | Data transfer method, device and system |
-
2017
- 2017-09-13 CN CN201710823216.1A patent/CN107368438A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101453478A (en) * | 2007-11-28 | 2009-06-10 | 新奥特（北京）视频技术有限公司 | Integrity checking method in document transmission |
| CN102136993A (en) * | 2010-07-29 | 2011-07-27 | 华为技术有限公司 | Data transfer method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 邱卫东: "<<英汉信息安全技术辞典>>", 30 November 2015, 上海交通大学出版社 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112633206A (en) * | 2020-12-28 | 2021-04-09 | 上海眼控科技股份有限公司 | Dirty data processing method, device, equipment and storage medium |
| CN113837278A (en) * | 2021-09-24 | 2021-12-24 | 厦门市美亚柏科信息股份有限公司 | Method and device for detecting dirty data |
| CN113837278B (en) * | 2021-09-24 | 2022-06-28 | 厦门市美亚柏科信息股份有限公司 | Method and device for detecting dirty data |
| CN115167767A (en) * | 2022-06-28 | 2022-10-11 | 慧之安信息技术股份有限公司 | Dirty data prevention method and system based on BBC exclusive or check |
| CN115167767B (en) * | 2022-06-28 | 2023-04-25 | 慧之安信息技术股份有限公司 | Dirty data prevention method and system based on BBC exclusive OR check |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7770073B2 (en) | Apparatus, system, and method for responsive acquisition of remote debug data | |
| US7472332B2 (en) | Method for the reliability of host data stored on fibre channel attached storage subsystems | |
| CN107368438A (en) | A kind of dirty data detection method and system | |
| CN103793479A (en) | Log management method and log management system | |
| CN102135925B (en) | Method and device for detecting error check and correcting memory | |
| KR101579493B1 (en) | Staging control method for source code, Computer program for the same, Recording medium storing computer program for the same | |
| CN103631721A (en) | Method and system for isolating bad blocks in internal storage | |
| CN108241676B (en) | Method and equipment for realizing data synchronization | |
| CN110058962B (en) | Method, apparatus and computer program product for determining consistency level of virtual machine snapshots | |
| CN107124335A (en) | One kind is based on cloud sea operating system platform test method and system | |
| CN106095678A (en) | Automatization's result inspection method of data bank service operation under windows platform | |
| CN110674161A (en) | Method, system, storage medium and SQL operation platform for online SQL | |
| CN105302924A (en) | File management method and device | |
| CN110008462B (en) | Command sequence detection method and command sequence processing method | |
| CN113485881A (en) | Testing method and device of temporary speed limiting server, electronic equipment and medium | |
| CN112732163B (en) | Data verification method and device | |
| CN104572917A (en) | Method and device for locking data and distributed storage system | |
| CN103645963A (en) | Storage system and data consistency verification method thereof | |
| CN113806332B (en) | Heterogeneous system integrated data processing method and device and computer equipment | |
| US8843472B2 (en) | Recovery of inconsistent data in databases | |
| US20050188279A1 (en) | Method, system, and program for data corruption detection and fault isolation | |
| JP2023501705A (en) | Method and System for Fast, Reliable, and Complete Verification of Memory States | |
| CN104780123A (en) | Network packet receiving and sending processing device and design method thereof | |
| CN106599221A (en) | Data processing method and device | |
| CN107368530A (en) | A kind of method and system based on custom properties checking data validity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171121 |