CN112053304A - Rapid focusing restoration method for single shooting of full-slice digital imaging - Google Patents
Rapid focusing restoration method for single shooting of full-slice digital imaging Download PDFInfo
- Publication number
- CN112053304A CN112053304A CN202010934440.XA CN202010934440A CN112053304A CN 112053304 A CN112053304 A CN 112053304A CN 202010934440 A CN202010934440 A CN 202010934440A CN 112053304 A CN112053304 A CN 112053304A
- Authority
- CN
- China
- Prior art keywords
- focus
- quasi
- image
- full
- digital imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003384 imaging method Methods 0.000 title claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000011156 evaluation Methods 0.000 claims abstract description 8
- 230000001575 pathological effect Effects 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000005259 measurement Methods 0.000 abstract description 6
- 238000013461 design Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 abstract description 4
- 230000007170 pathology Effects 0.000 abstract description 4
- 238000011084 recovery Methods 0.000 abstract 1
- 230000008569 process Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000012984 biological imaging Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010827 pathological analysis Methods 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention is based on the biomedical instrument field, particularly relate to a full-section digital imaging single shooting fast accurate focus recovery method, mainly design the virtual auto-focus network of single shooting; the method comprises the following steps that a pathology scanner pre-scans an image block in the middle of a pathology slice view field, an axial three-dimensional image stack is constructed through axial movement, and an initial in-focus image is found through an evaluation function; under the condition of predicting the quasi-focal distance, carrying out single photographing on all sub-images to obtain a tentative out-of-focus image for single photographing as network input; the output of the network is a restored quasi-focus image; through the end-to-end network design of a single image, the method realizes the quick restoration of the single-shot quasi-focus image under full-slice digital imaging, and replaces the traditional self-quasi-focus method of firstly compensating the distance and then shooting. Meanwhile, in the scanning strategy, a quasi-focus measurement diagram and quasi-focus scanning photographing are not required to be constructed, and the method has the advantages of high speed, low cost, offline processing and the like.
Description
Technical Field
The invention relates to a method for fast quasi-focus restoration by single shooting of full-section digital imaging based on the field of biomedical instruments and computational imaging and taking a deep learning technology as a core, and can be widely applied to the research in the fields of microscopic instruments, artificial intelligence, medical images, automation and the like.
Background
In recent years, advanced digital pathology imaging techniques have been widely studied and applied. The full-slice digital imaging technology (WSI, white Slide Images), i.e., virtual microscopy, can acquire traditional microscopic slices in the form of digitized Images, and can realize arbitrary computer access, easy storage, and remote transmission between researchers and doctors, etc. Full-slice digital imaging techniques are of critical importance in biological imaging research, such as in the fields of cancer analysis and disease prediction. The U.S. food and drug administration has currently adopted the full-slice digital imaging system of philips as the primary means of pathological analysis.
Full-slice digital imaging typically requires a two-step implementation: (1) scanning pathological images according to the sequence of the subareas, and then splicing the pathological images together to generate a complete pathological section image of the full view field; (2) purpose-built software is used to identify and analyze these digital images. Wherein the first step is crucial for the quality of the acquired image. At present, the challenge in full-slice digital imaging techniques is mainly how to quickly produce high quality in-focus images. Generally, full-slice scanning requires a high resolution and micron-length depth-of-field objective lens, which results in defocus due to the non-uniform three-dimensional distribution of the sample obtained during scanning of the system. This out-of-focus phenomenon is a major cause of degradation in full-slice digital imaging performance.
At present, a widely used method is to obtain high-quality full-slice digital images by adopting a method of quasi-focal image matching. The quasi-focus image matching method provides a quasi-focus priori, for the out-of-focus image at each position, a series of out-of-focus images (z-stack) at different quasi-focus distances can be obtained by moving the pathological sample along the optical axis, and finally the corresponding quasi-focus image is determined by maximizing the image contrast of each out-of-focus image or other image quality evaluation methods. This approach requires a one-by-one use for each sequentially scanned sub-region. However, this method results in a significant reduction in imaging speed due to repeated axial measurements. For other methods, for example, a double-camera device can be adopted to realize the self-focusing function, so that the axial layer-by-layer scanning of pathological images is avoided. However, this method is not suitable for adding a new imaging module in a conventional microscopy instrument due to problems of hardware incompatibility, high cost and the like. In the current relevant research, the full-slice digital imaging system mainly focuses on high-precision prediction of in-focus distance, and no relevant research currently performs in-focus restoration on out-of-focus images.
Quasi-focal restoration of out-of-focus images can change the scanning strategy of traditional pathological scanners. For a traditional pathological scanning method, firstly, when a scanner scans each image block, an axial image three-dimensional stack needs to be constructed, an in-focus image is found through evaluation functions such as maximized contrast and the like, and the in-focus position is recorded; secondly, for image blocks at other different positions, a block-by-block scanning method is adopted, and each image block executes a process of acquiring a quasi-focus position; then, integrating the quasi-focal distances corresponding to different image blocks of the whole pathological image into a quasi-focal measurement image according to positions; and finally, according to the coordinate position and the quasi-focus distance of the quasi-focus measurement image, moving a microscope objective lens to perform distance compensation and scanning and photographing to obtain a complete pathological digital image. The conventional scanning method inevitably consumes a lot of time cost because of the need to construct a large number of axial three-dimensional image stacks and the necessary axial mechanical displacement. Therefore, a new method is found for restoring the defocused image, a novel scanner scanning strategy is designed, and the method has great scientific research significance and clinical application value.
Disclosure of Invention
In consideration of the limitations of the conventional method, the invention utilizes an advanced machine learning algorithm to solve the problem of quasi-focus restoration of full-slice digital imaging. The invention discloses a full-slice digital imaging single-shot fast focusing restoration method based on deep learning. The method provides a full-slice digital imaging single-shot fast focusing restoration method based on deep learning, and mainly designs a virtual self-focusing network for single-shot; the pathological scanner pre-scans an image block in the middle of a pathological section view field, an axial three-dimensional image stack is constructed through axial movement, an initial quasi-focus image is found through an evaluation function, and a predicted quasi-focus distance of the quasi-focus image is obtained; under the condition of predicting the quasi-focal distance, carrying out single photographing on all sub-images to obtain a tentative out-of-focus image for single photographing as network input; the output of the network is a restored quasi-focus image; through the end-to-end network design of a single tentative out-of-focus image, the method realizes the quick restoration of the single-shot in-focus image under full-slice digital imaging, replaces the traditional self-in-focus method of firstly compensating the distance and then shooting, and performs software virtualization on the traditional full-slice digital pathological imaging hardware. Meanwhile, in the scanning strategy, a quasi-focus measurement diagram and quasi-focus scanning photographing are not required to be constructed, and the method has the advantages of high speed, low cost, offline processing and the like.
The purpose of the invention is realized as follows:
a quick quasi-focus restoration method for single shooting of full-slice digital imaging comprises the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
Further, in the step a, a sub-image position of the pathological section view field center is selected, the objective lens is moved in an axial scanning mode to obtain a z-stack, an axial image stack is constructed, a predicted quasi-focus image can be found by adopting evaluation functions such as contrast and the like, and the corresponding predicted quasi-focus position is recorded.
Further, the position of the selected first prediction quasi-focus image is the photographing position of all the sub-images, and other sub-images are photographed at the photographing position, so that a tentative out-of-focus image for photographing in a single time is obtained.
Further, the input to the network is a single shot tentative through-focus image.
Further, the output of the network is a restored quasi-in-focus image.
Further, after the tentative out-of-focus image for single photographing is processed through the neural network, an in-focus image is directly obtained.
Has the advantages that:
the invention realizes a quick focusing restoration method for single shooting of full-slice digital imaging, which is embodied in the following aspects:
firstly, the invention adopts a single-shot virtual self-focusing network, and realizes the function of restoring the focus-focusing image by carrying out end-to-end network processing on a single tentative out-of-focus image.
Secondly, the invention adopts a deep learning network algorithm, the method adopts a digital network structure to carry out simulation modeling, adopts a software method to realize the virtualization of a full-slice digital imaging system, directly realizes the restoration of a quasi-focus image, replaces the traditional method of firstly predicting the distance and then taking a picture by compensation, and effectively saves the cost of instrument experiment.
Thirdly, in the scanning strategy, the invention does not need to construct a quasi-focus measurement diagram and quasi-focus scanning photographing, and has the advantages of high speed, low cost, offline processing and the like.
Drawings
FIG. 1 is a flow chart of a single-shot fast in-focus restoration method for full-slice digital imaging according to the present invention;
fig. 2 is a schematic diagram of a scanning strategy of the pathology scanning system.
In the figure: microscope objective 1, single shot position 2, scan direction 3.
Detailed Description
The following further illustrates embodiments of the process of the present invention.
Referring to fig. 1 and fig. 2, a fast quasi-focus restoration method for full-slice digital imaging single-shot includes the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
Specifically, in the step a, a sub-image position of the pathological section view field center is selected, a z-stack is obtained through axial scanning movement of an objective lens, a predicted quasi-focus image can be found by adopting evaluation functions such as contrast and the like through constructing an axial image stack, and the corresponding predicted quasi-focus position is recorded.
Specifically, the position of the selected first prediction quasi-focus image is the photographing position of all sub-images, and other sub-images are photographed at the photographing position, so that a tentative out-of-focus image for single photographing is obtained.
Specifically, the input to the network is a single shot tentatively out-of-focus image.
Specifically, the output of the network is a restored quasi-in-focus image.
Specifically, a quasi-focus image is directly obtained after a tentative out-of-focus image for single photographing is processed through a neural network.
The flow chart of the quick focusing restoration method for single photographing of the full-slice digital imaging is known, and the algorithm comprises the following steps: predicting an in-focus image, photographing a tentative out-of-focus image for a single time, photographing a virtual self-in-focus network for a single time, and restoring an in-focus image.
In the training process, the predicted quasi-focus image is obtained through automatic quasi-focus, namely, in the central view field of the pathological section, a plurality of sub-images are continuously moved axially to obtain z-stack, and the predicted quasi-focus image is determined through maximizing contrast or other evaluation functions. (2) The tentative out-of-focus image for single photographing is obtained by photographing all image blocks with the position of the predicted in-focus image as a reference. (3) The single-shot virtual auto-focus network adopts a U-net mode. The input of the network is a tentative out-of-focus image taken once, and a restored in-focus image is finally output through network convolution. The method is only exemplified by U-net, including but not limited to various networks, and the quasi-focus image restoration is realized by using the single-shot tentative out-of-focus image.
In actual operation of the pathological scanning device, the scanning strategy is shown in fig. 2. The microscope objective 1 takes the position of the predicted in-focus image as a reference, and is used as a single photographing position 2, and exposure photographing is carried out to obtain a single photographing tentative out-of-focus image. The above procedure is then repeated for a second sub-image, at this position with x-y mechanical movements in the scanning direction 3.
In this embodiment, a method for fast focus restoration by single-shot full-slice digital imaging includes: the system comprises a prediction in-focus image module, a single-shot tentative out-of-focus image module, a single-shot virtual self-in-focus network module and a restoration in-focus image module. Wherein: the pathological scanner pre-scans an image block in the middle of a pathological section view field, an axial three-dimensional image stack is constructed through axial movement, an initial quasi-focus image is found through an evaluation function, and a predicted quasi-focus distance of the quasi-focus image is obtained; and under the condition of predicting the quasi-focal distance, photographing all the sub-images for a single time to obtain a tentative out-of-focus image for the single-time photographing, wherein the tentative out-of-focus image is used as an input end of a back-end neural network.
In the training process, the single-shot virtual self-focusing network performs network training and feature extraction according to the single-shot tentative out-of-focus image; the restored quasi-focus image is output of the neural network; and performing end-to-end network training through a single tentative out-of-focus image and the corresponding quasi-focus image to obtain a single-shot virtual self-quasi-focus network.
In the testing stage, a predicted quasi-focus sub-image is found at the center of a sample to obtain a corresponding predicted quasi-focus position; then, all other sub-images are photographed at the position to obtain a single photographing tentative out-of-focus image which is used as a test image of the rear-end single photographing virtual self-focusing network; and finally, processing through a single-time photographing virtual auto-focus network to obtain a restored auto-focus image.
In the scanning strategy, the specific process is as follows: firstly, selecting a sample center as an initial position to obtain a predicted in-focus position; secondly, taking the predicted quasi-focal distance as a reference position, and photographing all sub-images at the reference position to obtain a single tentative out-of-focus image; and finally, obtaining a plurality of sub-images of the quasi-focus, and splicing to obtain the pathological full-slice digital image of the quasi-focus.
Claims (6)
1. A quick quasi-focus restoration method for single shooting of full-slice digital imaging is characterized by comprising the following steps:
step a, predicting a quasi-focus image;
b, shooting a tentative out-of-focus image once;
step c, virtual auto-focus network for single photographing;
step d, restoring the in-focus image,
and directly obtaining pathological focus-aligning images at different positions by offline processing by adopting a neural network method.
2. The method for rapid quasi-focal restoration through single-shot full-slice digital imaging according to claim 1, wherein a sub-image position of the center of a pathological section field of view is selected in the step a, a z-stack is obtained through axial scanning movement of an objective lens, and by constructing an axial image stack, a predicted quasi-focal image can be found by using evaluation functions such as contrast and the like, and the corresponding predicted quasi-focal position is recorded.
3. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: and the position of the selected first prediction quasi-focus image is the photographing position of all sub-images, and other sub-images are photographed at the position to obtain a tentative out-of-focus image for single photographing.
4. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: the input to the network is a single shot tentative out-of-focus image.
5. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: the output of the network is a restored quasi-in-focus image.
6. The full-slice digital imaging single-shot fast quasi-focus restoration method according to claim 1, characterized in that: and processing the tentative defocused image obtained by single photographing through a neural network to directly obtain a quasi-focus image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010934440.XA CN112053304A (en) | 2020-09-08 | 2020-09-08 | Rapid focusing restoration method for single shooting of full-slice digital imaging |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010934440.XA CN112053304A (en) | 2020-09-08 | 2020-09-08 | Rapid focusing restoration method for single shooting of full-slice digital imaging |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112053304A true CN112053304A (en) | 2020-12-08 |
Family
ID=73610360
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010934440.XA Pending CN112053304A (en) | 2020-09-08 | 2020-09-08 | Rapid focusing restoration method for single shooting of full-slice digital imaging |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112053304A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112070660A (en) * | 2020-09-08 | 2020-12-11 | 哈尔滨工业大学 | Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180191948A1 (en) * | 2017-01-03 | 2018-07-05 | University Of Connecticut | Single-Frame Autofocusing Using Multi-LED Illumination |
CN108848302A (en) * | 2018-05-24 | 2018-11-20 | 杭州电子科技大学 | A kind of acquisition of micro-imaging, control and automated processing system and processing method |
CN109410130A (en) * | 2018-09-28 | 2019-03-01 | 华为技术有限公司 | Image processing method and image processing apparatus |
CN111462075A (en) * | 2020-03-31 | 2020-07-28 | 湖南国科智瞳科技有限公司 | Rapid refocusing method and system for full-slice digital pathological image fuzzy area |
-
2020
- 2020-09-08 CN CN202010934440.XA patent/CN112053304A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180191948A1 (en) * | 2017-01-03 | 2018-07-05 | University Of Connecticut | Single-Frame Autofocusing Using Multi-LED Illumination |
CN108848302A (en) * | 2018-05-24 | 2018-11-20 | 杭州电子科技大学 | A kind of acquisition of micro-imaging, control and automated processing system and processing method |
CN109410130A (en) * | 2018-09-28 | 2019-03-01 | 华为技术有限公司 | Image processing method and image processing apparatus |
CN111462075A (en) * | 2020-03-31 | 2020-07-28 | 湖南国科智瞳科技有限公司 | Rapid refocusing method and system for full-slice digital pathological image fuzzy area |
Non-Patent Citations (2)
Title |
---|
QIANG LI等: "Rapid Whole Slide Imaging via Learning-based Two-shot Virtual Autofocusing", 《ARXIV》 * |
葛云皓: "基于卷积神经网络的病理显微镜自动对焦与全局精准成像研究", 《中国优秀硕士学位论文全文数据库 (工程科技II辑)》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112070660A (en) * | 2020-09-08 | 2020-12-11 | 哈尔滨工业大学 | Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning |
CN112070660B (en) * | 2020-09-08 | 2022-08-12 | 哈尔滨工业大学 | Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6437947B2 (en) | Fully automatic rapid microscope slide scanner | |
CN109597195B (en) | Super-resolution three-dimensional light field microscopic imaging system and method based on lattice illumination | |
Yazdanfar et al. | Simple and robust image-based autofocusing for digital microscopy | |
EP2273302B1 (en) | Image acquiring apparatus, image acquiring method and image acquiring program | |
EP1336888B1 (en) | Microscopy imaging system and data acquisition method | |
US6816606B2 (en) | Method for maintaining high-quality focus during high-throughput, microscopic digital montage imaging | |
US9297995B2 (en) | Automatic stereological analysis of biological tissue including section thickness determination | |
CN109085695B (en) | Method for quickly focusing and photographing plane sample | |
RU2734447C2 (en) | System for forming a synthesized two-dimensional image of a biological sample with high depth of field | |
CN111429562B (en) | Wide-field color light slice microscopic imaging method based on deep learning | |
CN110411946B (en) | Method for focusing camera image in digital slice scanner | |
CN109272575B (en) | Method for improving modeling speed of digital slice scanner | |
He et al. | Microscope images automatic focus algorithm based on eight-neighborhood operator and least square planar fitting | |
CN111220615A (en) | Inclined three-dimensional scanning microscopic imaging system and method | |
CN112053304A (en) | Rapid focusing restoration method for single shooting of full-slice digital imaging | |
CN112070887A (en) | Depth learning-based full-slice digital imaging depth of field extension method | |
US11356593B2 (en) | Methods and systems for single frame autofocusing based on color- multiplexed illumination | |
CN113705298A (en) | Image acquisition method and device, computer equipment and storage medium | |
CN112070660B (en) | Full-slice digital imaging self-adaptive automatic focusing method based on transfer learning | |
CN112069735B (en) | Full-slice digital imaging high-precision automatic focusing method based on asymmetric aberration | |
CN110349237B (en) | Fast volume imaging method based on convolutional neural network | |
CN112070661A (en) | Full-slice digital imaging rapid automatic focusing method based on deep learning | |
CN112037154A (en) | High-precision quasi-focus restoration method for full-slice digital imaging and double photographing | |
CN112037153A (en) | Full-slice digital imaging quasi-focus restoration method based on quasi-focus distance prior | |
CN112037152A (en) | Full-slice digital imaging two-step quasi-focus restoration method based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20201208 |
|
WD01 | Invention patent application deemed withdrawn after publication |