CN107479833A - Key value storage-oriented remote nonvolatile memory access and management method - Google Patents

Key value storage-oriented remote nonvolatile memory access and management method Download PDF

Info

Publication number
CN107479833A
CN107479833A CN201710716667.5A CN201710716667A CN107479833A CN 107479833 A CN107479833 A CN 107479833A CN 201710716667 A CN201710716667 A CN 201710716667A CN 107479833 A CN107479833 A CN 107479833A
Authority
CN
China
Prior art keywords
memory
memory block
data
area
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710716667.5A
Other languages
Chinese (zh)
Other versions
CN107479833B (en
Inventor
肖侬
余松平
邓明翥
邢玉轩
刘芳
陈薇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN201710716667.5A priority Critical patent/CN107479833B/en
Publication of CN107479833A publication Critical patent/CN107479833A/en
Application granted granted Critical
Publication of CN107479833B publication Critical patent/CN107479833B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Abstract

The invention relates to a remote nonvolatile memory access and management method facing key value storage, which is characterized in that a nonvolatile memory cache region is set for a key value storage system and is registered on a network card capable of supporting remote direct memory access technology; this non-volatile memory cache area is divided into two parts: the memory blocks in the sparse area are mainly used for receiving data of a remote client, different memory blocks are allocated for remote data writing each time, the memory block in each sparse area only belongs to one key value pair, the memory block in the compact area is a main cache area of the key value pair, each memory block comprises a plurality of key value pairs, and a read request of the client is served; compressing key value pairs in a plurality of allocated memory blocks in the sparse area into memory blocks in one or more compact areas by a compressor at regular intervals by the server; when the number of available memory blocks of the compact area is insufficient, data is replaced to the data storage area. The invention can relieve the fixed point write abrasion problem of the remote nonvolatile memory and improve the remote read-write performance.

Description

A kind of long-range nonvolatile memory towards key assignments storage accesses and management method
Technical field
It is particularly a kind of towards the remote of key assignments storage the present invention relates to a kind of long-range nonvolatile memory access and management method Journey nonvolatile memory accesses and management method.
Background technology
, it is necessary to face the storage demand of mass data under big data environment.On the one hand, the autgmentability of application system is carried Go out higher requirement, and the NoSQL systems based on key assignments storage architecture, the storage of data is using independent key-value pair to be substantially single Member, the relevance between data is reduced, there is good autgmentability;On the other hand, the meter based on convential memory and external memory framework Calculation machine storage system is faced with the challenge of mass data processing performance;Novel memory devices part, as PCM (phase-change random access memory), 3D Xpoint, possess the characteristics such as the access speed of the non-volatile of external memory, high integration and internal memory level, it is contemplated that existing The deficiency of DRAM (dynamic random access memory) volatile characteristic, high energy consumption and finite capacity, formed based on these devices Nonvolatile memory (NVM, Non-Volatile Memory) turn into Computer Design and researcher and improve performance of storage system Hot topic.Although NVM integrated level is higher than DRAM, the NVM capacity of unit or limited and current NVM skills The maturity of art is not high, reports and shows according to Intel:In the computer system in future, 4 times of convential memory capacity will be disposed It is commercialized NVM products (such as 3D Xpoint);Really, the NVM resources of unit are held far from the storage for meeting big data application system Amount demand, thus by network it is extending transversely become it is necessary.Legacy network based on ICP/IP protocol can turn into long-distance inner The performance bottleneck of extension, main cause are the data processing methods of ICP/IP protocol, buffer area of the application program in user's space The data being ready for, then called the kernel spacing of data copy by system, needed in kernel spacing to transmission Data add the header information needed for each layer protocol, and the machine of distal end is finally sent to by network interface card, and the machine of distal end passes through net Clamping receives data, according to reverse logical analysis packet is sent to, finally by the use of data copy to remote application The buffer area in family space;It is seen that carried out data transmission using conventional network techniques, it is necessary to undergo multiple data copy, The copy of data not only consumes CPU resource, also reduces the utilization rate of network bandwidth.Compared with TCP/IP, RDMA (Remote Direct Memory Access, long-range directly to access memory techniques) allows remotely directly to access long-range user Memory headroom, without distal end CPU participation, during data transfer, the CPU of distal end can run other journeys parallel Sequence.Current remote access internal memory be based on DRAM, due to NVM internal storage access characteristic, remote access NVM be it is rational, And NVM will play the part of important role in the RDMA technologies in future.
But, while NVM possesses the advantages characteristic of internal memory and external memory, NVM also there is read and write it is unbalanced, write abrasion The problems such as, because RDMA technologies can allow the nonvolatile memory of distal end directly to be accessed by client, and RDMA is accessed The mapping of virtual address and physical address be fixed, this has speeded up NVM and has write loss, so, how to design NVM access Mode and administrative mechanism become new challenge.
The content of the invention
In order to solve above-mentioned technical problem, it is an object of the invention to provide a kind of towards the long-range non-easy of key assignments storage Lose internal storage access and management method.
This method is key assignments storage one nonvolatile memory buffer area of setting, and is registered to and can support remote direct memory On the network interface card of access technique;It is two parts to divide this nonvolatile memory buffer area:Rarefaction and compact area, in rarefaction Memory block is mainly used in receiving the data of Terminal Server Client, and is write per the different memory block of sub-distribution for teledata, The memory block of each rarefaction is pertaining only to a key-value pair, and it is to improve the performance of remote write and extend non-volatile that it, which is acted on, The internal memory life-span.
And the memory block in compact area is the main buffer area of key-value pair, each memory block includes multiple key-value pairs, service visitor The read request at family end;Service end is every at regular intervals, by compression mechanism by the multiple allocated memory blocks in rarefaction Key-value pair is compressed in the memory block in one or more compact areas;When the lazy weight of the free memory block in compact area, can incite somebody to action Data are replaced to data storage area.It is the efficiency for improving caching that it, which is acted on,.
The technical solution adopted for the present invention to solve the technical problems is:
A kind of long-range nonvolatile memory based on RDMA accesses and management method, including:
The first step, service end division in save as three classes:Data storage area, nonvolatile memory buffer area and command buffer;Its In, the internal memory of data storage area and nonvolatile memory buffer area is NVM, and the internal memory of command buffer is DRAM;
Second step, the internal memory of nonvolatile memory buffer area and command buffer is registered to by service end supports remote direct memory to visit Ask on the network interface card of technology, and the registration for the parameter selectivity that data storage area is then set according to user, and nonvolatile memory is delayed Deposit zoning and be divided into compact area and rarefaction, the size of the memory block of rarefaction is uniformly set as the multiple of page byte, and the multiple is 1-n times, n is positive integer;
3rd step, service end receives the connection request of client, and distributes a rarefaction memory block and one for each client The memory block of individual command buffer;
4th step, client (Remote Direct Memory Access, are remotely directly visited when transmitting data by RDMA Ask memory techniques) the sparse memory block of write-in, while send control information in order buffer memory block, service end returns to next time The sparse block address memory write is as write acknowledgement (ACK) message;If one time data transfer needs multiple sparse memory blocks, root According to control information, multiple sparse memory blocks are cascaded and ensure the integrality of data;
5th step, when the quantity of the free memory block of sparse region drops to certain threshold values(The threshold value refers to:Free memory The quantity of block and the ratio of total internal memory number of blocks, scope 0.1~0.6), service end will will select by Memory Compress Mechanism Data compression in multiple sparse memory blocks reclaims the sparse internal memory for including junk data into the memory block in compact area Block;
6th step, when the quantity of the free memory block in compact area drops to certain threshold values(The threshold value refers to:Free memory block Quantity and the ratio of total internal memory number of blocks, scope 0.1~0.6), service end will will be selected multiple by Data Migration mechanism Data copy in the memory block in compact area is to data storage area.
It can reach following beneficial effect using the present invention:Firstly, since limited, division clothes are cached on piece on RDMA network interface cards The nonvolatile memory at business end be data storage area and data buffer area, log-on data buffer area and the storage of selective log-on data Area, reduce the performance for being not hit by ensure that data manipulation while expense cached on the piece of network interface card;Secondly, by data buffer storage Rarefaction and compact area are divided into, for client, when sending data each time, the memory block of rarefaction is different , it means that it is not fixed point that the data of client, which are write, can so extend nonvolatile memory writes the life-span;Furthermore Client distributes a sparse memory block as the confirmation message write complete after writing every time, ensure that the reliability of data transfer While improve the performance that continuous data is write;Finally, the valid data of sparse memory block will be taken by Memory Compress Mechanism It is compressed in compact area's memory block, improves the service efficiency of internal memory.
Brief description of the drawings
The invention will be further described with reference to the accompanying drawings and examples.
Fig. 1 is the access of long-range nonvolatile memory and management flow chart towards key assignments storage;
Fig. 2 is the sparse and compacting memory block organization chart in data buffer area.
Embodiment
Reference picture 1, the invention provides a kind of long-range nonvolatile memory access towards key assignments storage and management method, bag Include:
The first step, for service end to RDMA network interface card registering communication region of memorys, communication region of memory includes nonvolatile memory and volatile (tradition) internal memory, nonvolatile memory are used for the caching of key-value pair, and volatile memory receives the command information of user data transmission;
Second step, service end division nonvolatile memory is sparse and compact two regions, and acquiescently, the two regions are united by size One multiple memory blocks composition, the size of the memory block of rarefaction is in 4096 bytes between 16384 bytes;
3rd step, client are initiated to establish connection request to service end, and service end is that each client distributes a reception order Buffer memory block and a sparse region memory block, and these information are returned into client;
4th step, client are first write key-value pair in this sparse memory block by the long-range write operations of RDMA, and will control Message processed is sent into command buffer;
5th step, service end continue to distribute a sparse region by the control message in resolve command buffering area for client In memory block as the data that send of storage next time;If this is last time data block, then in upper once sparse The identification information of itself is write in memory block and represents this data transfer ends, otherwise, in upper sparse memory block end once It is written as the mark of the memory block for the sparse region that data transfer next time is distributed;
6th step, if more than one internal memory bound block size of the residue length of the key-value pair data of client, continue weight Multiple step 4;
7th step, at interval of a period of time, the quantity of the free memory block of service end statistics sparse region, if free memory block Data are reduced to certain threshold values, and service end selects a number of memory block, then will be therein by data compression scheme Data copy is into the memory block in one or more compact areas;
8th step, at interval of a period of time, the quantity of the free memory block of service end statistics compact area, if free memory block Data are reduced to certain threshold values, and service end selects a number of memory block, then will be therein by Data Migration mechanism Data copy increases available internal memory number of blocks to data storage areas;
Preferred embodiment is further used as, described step two, it is specially:
The internal memory of the buffer area of registration is that all clients are shared, the memory block in compact area and the memory block of rarefaction of acquiescence Size it is the same, for the memory block of rarefaction, its main effect is for caching more key-value pairs, and client is only There is the authority of reading, manage for convenience, the size of all memory blocks in this region is unified, but to allowing single Memory block cache more data, the size maximum of the memory block in rarefaction can be the several of compact area's internal memory block size Times;Because each client can bind the memory block in a compact area, if the internal memory block size setting in this region is too big, In the case where client increases, the resource consumption of internal memory is more, so, it is set as the memory block in compact area according to size Multiple grades, such as 4096 bytes, 8192 bytes, 16384 bytes, reference picture 2, connection or each key-value pair are established in client When transferring, a minimum memory block, that is, 4096 bytes are only bound;The distribution of other memory blocks, then according to client The demand dynamic at end adjusts.
Preferred embodiment is further used as, described step three, it is specially:
Due to the discontinuous nature of the data sending request of client, the binding memory block fixed for a long time if each connection, then Other can be caused to need the client of memory block to lack resource, so setting is each connected in the regular hour, if do not had The transmission of data, then can reclaim the memory block in the compact area of binding, and recovery has two ways:Client proactive notification abandons this The authority and service end of memory block force recovery memory block and notify client transmission next time data to need to apply for memory block again.

Claims (5)

1. a kind of long-range nonvolatile memory towards key assignments storage accesses and management method, non-volatile for key assignments storage setting one Internal memory cache region, and be registered on the network interface card that can support remote direct memory access technique;It is characterized in that, described non-volatile Internal memory cache region is divided into two parts:Rarefaction and compact area, the memory block in rarefaction are used to receive Terminal Server Client Data, and write per the different memory block of sub-distribution for teledata, the memory block of each rarefaction is pertaining only to a key Value pair, to improve the performance of remote write and extend the nonvolatile memory life-span;
The memory block in compact area is the main buffer area of key-value pair, and each memory block includes multiple key-value pairs, service client Read request;Service end is per at regular intervals, by compression mechanism by the key-value pair in the multiple allocated memory blocks in rarefaction It is compressed in the memory block in one or more compact areas;Improve the efficiency of caching;When compact area free memory block quantity not When sufficient, data are replaced to data storage area;
Specific steps include:
The first step, service end division in save as three classes:Data storage area, nonvolatile memory buffer area and command buffer;Its In, the internal memory of command buffer is DRAM;
Second step, the internal memory of nonvolatile memory buffer area and command buffer is registered to by service end supports remote direct memory to visit Ask on the network interface card of technology, and the registration for the parameter selectivity that data storage area is set according to user;By nonvolatile memory buffer area Compact area and rarefaction are divided into, the size of the memory block of rarefaction is uniformly set as the multiple of page byte, and the multiple is 1-n Times, n is positive integer;
3rd step, service end receives the connection request of client, and distributes a rarefaction memory block and one for each client The memory block of individual command buffer;
4th step, client enter sparse memory block, while send control information to order when transmitting data by RDMA Write In buffer memory block, service end returns to the sparse block address memory write next time as write acknowledgement message;An if data transfer Multiple sparse memory blocks are needed, then according to control information, multiple sparse memory blocks is cascaded and ensure the integrality of data;
5th step, when the quantity of the free memory block of sparse region drops to certain threshold values, service end will pass through internal memory pressure Contracting mechanism, the data compression in multiple sparse memory blocks will be selected into the memory block in compact area, and reclaim and include rubbish number According to sparse memory block;
6th step, when the quantity of the free memory block in compact area drops to certain threshold values, service end will pass through Data Migration Mechanism is by the data copy in the memory block for selecting multiple compact areas to data storage area.
2. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is that its effect of the memory block of the rarefaction is for caching more key-value pairs, and client only has the authority read, in order to Convenient management, it is unified in the size of all memory blocks of rarefaction, in order to allow single memory block to cache more data, The size maximum of memory block in rarefaction can be several times of compact area's internal memory block size;Because each client can be bound The memory block in one compact area, if the memory block setting in compact area is too big, in the case where client increases, the resource of internal memory Consume more, the memory block in compact area is set as several grades according to size.
3. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is, the 4th step,
If without the transmission of data, the memory block in the compact area of binding is reclaimed, reclaiming has two ways:Client proactive notification Abandon the authority of this memory block and service end forces recovery memory block and notifies client to send data needs Shen again next time Please memory block.
4. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is, the 4th step,
Client is it needs to be determined that the validity of the binding relationship of memory block before data is transmitted, if the relation of binding is solved Remove, then the memory block of the sparse memory field of first to file, be over if service end judges that a key-value pair is sent, can be this The transmission that a rarefaction memory block is used for key-value pair next time is bound in connection again.
5. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is that the threshold value in the 5th step, the 6th step refers to:The quantity of free memory block and the ratio of total internal memory number of blocks, Scope 0.1~0.6.
CN201710716667.5A 2017-08-21 2017-08-21 Key value storage-oriented remote nonvolatile memory access and management method Active CN107479833B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710716667.5A CN107479833B (en) 2017-08-21 2017-08-21 Key value storage-oriented remote nonvolatile memory access and management method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710716667.5A CN107479833B (en) 2017-08-21 2017-08-21 Key value storage-oriented remote nonvolatile memory access and management method

Publications (2)

Publication Number Publication Date
CN107479833A true CN107479833A (en) 2017-12-15
CN107479833B CN107479833B (en) 2020-04-17

Family

ID=60601682

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710716667.5A Active CN107479833B (en) 2017-08-21 2017-08-21 Key value storage-oriented remote nonvolatile memory access and management method

Country Status (1)

Country Link
CN (1) CN107479833B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109976947A (en) * 2019-03-11 2019-07-05 北京大学 A kind of method and system of the power loss recovery towards mixing memory
CN110535811A (en) * 2018-05-25 2019-12-03 中兴通讯股份有限公司 Remote memory management method and system, server-side, client, storage medium
CN111858418A (en) * 2019-04-30 2020-10-30 华为技术有限公司 Memory communication method and device based on remote direct memory access RDMA
CN111897784A (en) * 2020-07-13 2020-11-06 安徽大学 Key value storage-oriented near data computing cluster system
WO2022095685A1 (en) * 2020-11-04 2022-05-12 中兴通讯股份有限公司 Persistent memory key value system and operation method therefor

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060047771A1 (en) * 2004-08-30 2006-03-02 International Business Machines Corporation RDMA server (OSI) global TCE tables
CN102843435A (en) * 2012-09-10 2012-12-26 浪潮(北京)电子信息产业有限公司 Access and response method and access and response system of storing medium in cluster system
US20140325012A1 (en) * 2012-11-21 2014-10-30 International Business Machines Corporation Rdma-optimized high-performance distributed cache
CN106202138A (en) * 2015-06-01 2016-12-07 三星电子株式会社 Storage device and method for autonomous space compression
CN106469198A (en) * 2016-08-31 2017-03-01 华为技术有限公司 Key assignments storage method, apparatus and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060047771A1 (en) * 2004-08-30 2006-03-02 International Business Machines Corporation RDMA server (OSI) global TCE tables
CN102843435A (en) * 2012-09-10 2012-12-26 浪潮(北京)电子信息产业有限公司 Access and response method and access and response system of storing medium in cluster system
US20140325012A1 (en) * 2012-11-21 2014-10-30 International Business Machines Corporation Rdma-optimized high-performance distributed cache
CN106202138A (en) * 2015-06-01 2016-12-07 三星电子株式会社 Storage device and method for autonomous space compression
CN106469198A (en) * 2016-08-31 2017-03-01 华为技术有限公司 Key assignments storage method, apparatus and system

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110535811A (en) * 2018-05-25 2019-12-03 中兴通讯股份有限公司 Remote memory management method and system, server-side, client, storage medium
CN110535811B (en) * 2018-05-25 2022-03-04 中兴通讯股份有限公司 Remote memory management method and system, server, client and storage medium
CN109976947A (en) * 2019-03-11 2019-07-05 北京大学 A kind of method and system of the power loss recovery towards mixing memory
CN109976947B (en) * 2019-03-11 2020-11-27 北京大学 Power failure recovery method and system for hybrid memory
CN111858418A (en) * 2019-04-30 2020-10-30 华为技术有限公司 Memory communication method and device based on remote direct memory access RDMA
CN111858418B (en) * 2019-04-30 2023-04-07 华为技术有限公司 Memory communication method and device based on remote direct memory access RDMA
CN111897784A (en) * 2020-07-13 2020-11-06 安徽大学 Key value storage-oriented near data computing cluster system
CN111897784B (en) * 2020-07-13 2022-12-06 安徽大学 Key value storage-oriented near data computing cluster system
WO2022095685A1 (en) * 2020-11-04 2022-05-12 中兴通讯股份有限公司 Persistent memory key value system and operation method therefor

Also Published As

Publication number Publication date
CN107479833B (en) 2020-04-17

Similar Documents

Publication Publication Date Title
CN107479833A (en) Key value storage-oriented remote nonvolatile memory access and management method
CN103019970B (en) Memory storage and control method thereof
CN103946828B (en) Data processing system and method
CN102045772B (en) Data transmission method and device
CN101267361B (en) A high-speed network data packet capturing method based on zero duplication technology
CN101841545B (en) TCP stream restructuring and/or packetizing method and device
CN101626384B (en) Method and system for transmitting data packet based on TPC protocol
CN101751233B (en) Method and system for expanding capacity of memory device
CN102446139B (en) Method and device for data storage
US20080320155A1 (en) Aggregation and re-ordering of input/output requests for better performance in remote file systems
CN100535886C (en) Data-transmission system between nodes, and device and method
CN102118433A (en) Multiple-tier distributed cluster system
CN101150485A (en) A management method for network data transmission of zero copy buffer queue
WO2019127018A1 (en) Memory system access method and device
CN112948149A (en) Remote memory sharing method and device, electronic equipment and storage medium
CN106020731B (en) Store equipment, array of storage devices and network adapter
CN103517085B (en) Method for implementing remote server management based on video decoding design
CN112199309A (en) Data reading method and device based on DMA engine and data transmission system
CN110199512B (en) Management method and device for storage equipment in storage system
CN114629849A (en) Storage device and operation method thereof
US20090198699A1 (en) Remote space efficient repository
WO2023098050A1 (en) Remote data access method and apparatus
CN100459700C (en) Method for realizing video requesting system based on protocol buffer storage
Hines et al. Distributed anemone: Transparent low-latency access to remote memory
WO2023246843A1 (en) Data processing method, apparatus and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant