CN105844690A - Rapid three-dimensional CT iteration reconstruction system based on GPU - Google Patents
Rapid three-dimensional CT iteration reconstruction system based on GPU Download PDFInfo
- Publication number
- CN105844690A CN105844690A CN201610216968.7A CN201610216968A CN105844690A CN 105844690 A CN105844690 A CN 105844690A CN 201610216968 A CN201610216968 A CN 201610216968A CN 105844690 A CN105844690 A CN 105844690A
- Authority
- CN
- China
- Prior art keywords
- projection
- module
- iteration
- ray
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2215/00—Indexing scheme for image rendering
- G06T2215/06—Curved planar reformation of 3D line structures
Abstract
The invention discloses a rapid three-dimensional CT iteration reconstruction system based on a GPU. The system comprises a data input module, a pre-processing module, a positive/back projection module, a variable update module, an iteration termination module and a result output module, wherein the data input module is mainly used for inputting the projection data, the pre-processing module is used for pre-processing the projection data and transmitting the processed data and reconstruction-related parameters to the GPU, the positive/back projection module is used for carrying out positive projection, information record and correction and back projection and obtaining a positive projection system matrix and a back projection system matrix through calculation, the variable update module is used for updating variable values in the present iteration process according to the back projection result, the iteration termination module is used for determining whether present iteration satisfies the iteration termination condition, and the result output module is used for outputting the iteration result. According to the system, only the intersection situation of primary rays and voxel is required to calculate, computational quantity required for system matrix calculation is reduced, and an iteration reconstruction speed is accelerated.
Description
Technical field
The invention belongs to X-ray fault imaging (CT) technical field, particularly to based on GPU
Quick three-dimensional CT iterative approximation system.
Background technology
Computed tomography (Computed Tomography, CT) has been widely used for people
The fields such as soma's imaging, industrial nondestructive testing.CT algorithm for reconstructing can be divided into iterative reconstruction algorithm and
The big class of analytic reconstruction algorithm two.The advantage of analytic reconstruction algorithm is that algorithm is simple, reconstruction speed fast, lacks
Point is that the requirement of the completeness to data is higher, and the quality that the noise in data for projection is to rebuilding image
Affect the biggest.Comparing analytic reconstruction algorithm, the advantage of iterative reconstruction algorithm is that reconstructed image quality is high,
And it is applicable to various forms of collection data, even in the case of limited angle data for projection, also
Can reconstruct preferable image, typical iterative algorithm has SART, ISRA, WLS and EM-ML
Deng.But owing to iterative reconstruction algorithm is computationally intensive, convergence rate slow, reconstruction time is long, the most
Become its wide variety of maximum bottleneck of restriction.
The iterative process of iterative reconstruction algorithm can be analyzed to four steps: orthographic projection, correction, back projection
Update with voxel.In iterative algorithm, compared with orthographic projection, back projection's same type of drive of employing,
Orthographic projection uses ray-driven mode, back projection to use the combination of voxel type of drive can effectively remove division ring
Shape artifact.Therefore, in iterative algorithm, orthographic projection uses type of drive based on ray to calculate system
Matrix；And back projection uses and calculates sytem matrix based on voxel driving method.The district of two kinds of type of drive
Not being that the former only considers the ray density through voxel, the latter also needs to consider that voxel around is to currently
The impact of the voxel that ray passes.During code realizes, generally orthographic projection and correction step are closed
And, process together；Back projection and voxel update step and merge, and process simultaneously.
At iterative reconstruction algorithm wherein, the amount of calculation of orthographic projection and back projection is very big, accounts for iterative approximation
The ratio of the amount of calculation that algorithm is total is the highest, therefore improves the speed of service of iterative reconstruction algorithm, and key exists
In the computational efficiency improving positive and negative projection.Wherein, solve ray to occupy with the situation of intersecting of voxel
The amount of calculation of the overwhelming majority of orthographic projection and back projection.For traditional method, orthographic projection and back projection
The orthographic projection carried out respectively it needs to be determined that ray and each voxel intersect situation；Carrying out back projection
Process, this operation must re-start once.
At present, GPU (graphic process unit, English: Graphics Processing Unit, abbreviation:
GPU) appearance of technology decreases the operation time of conventional iterative algorithm, because ray-driven mode
Orthographic projection and the back projection using voxel to drive support that GPU accelerates parallel.But rebuild efficiency still
Relatively low, it is difficult to meet and rebuild the application that efficiency requirements is high.It is therefore proposed that quick three-dimensional based on GPU
CT iterative reconstruction approach and system, all have higher value in theory and actual application.
Summary of the invention
The present invention proposes quick three-dimensional CT iterative approximation system based on GPU, is primarily directed to pass
Needed for system three dimensional CT iterative approximation, the problem of calculating time length, improves and rebuilds efficiency.
The present invention is achieved through the following technical solutions:
Quick three-dimensional CT iterative approximation system based on GPU, including data input module, pretreatment
Module, positive/negative projection module, variable update module, iteration termination module, result output module；
Described data input module includes inputting data for projection, motif position and size, center of rotation
Position, the distance of radiographic source to center of rotation, detector size, detector to center of rotation away from
From and projection angle；
Described pretreatment module includes making data for projection relevant pretreatment, and the number after processing
It is passed to GPU according to this and to rebuilding relevant parameter；
Described positive/negative projection module carries out including orthographic projection step, record information Step and revising step
Suddenly, back projection's step operation, this module realizes orthographic projection step and back projection's step on GPU, point
It is not calculated orthographic projection system matrix and back projection system's matrix, specifically includes: first, select one
These rays chosen are carried out orthographic projection by multithreading by the ray of part orthographic projection to be calculated respectively,
Record the result of every ray orthographic projection, and the information that ray intersects with voxel；Then, choosing
After the orthographic projection step of ray and information recording step complete, it is modified step according to the information of record
Rapid operation and back projection's step operation, wherein, makeover process is only with data for projection and every ray just
Project relevant, every can be obtained during current iteration according to the information of orthographic projection step record before
The projection value of ray, such that it is able to calculate projection value after the correction of every ray rapidly, in back projection
During, every ray can be quickly obtained through voxel according to the information of orthographic projection step record before
Position and length, such that it is able to intersect situation, rapidly by every according to every ray and voxel
After the correction of ray, projection value is assigned on 8 voxels adjacent at intersection location；Finally, continue from
Remaining ray is chosen the operation before a part of ray repeats, until all of ray is all calculated
Cross and terminate circulation；
Variate-value during current iteration is carried out by described variable update module according to back projection's result
Update；
Described iteration termination module includes judging the end condition whether current iteration meets iteration, as
Fruit meets the end condition of iteration, terminates iteration, and output iteration result is to result output module；Otherwise,
Positive/negative projection operation before continuation and variable update operation；
Described result output module includes the data after storage and display iterative approximation, by iteration result
Output.
Further, in described positive/negative projection module back projection step, back projection's operation is grasped by multithreading
Being accelerated, each thread is required for the value distributing a blocks of data to store each voxel.
Compared with prior art, advantage is as follows for the present invention:
1, in positive/negative projection process, only need to calculate ray and voxel intersects situation, decreases
Calculate the amount of calculation needed for sytem matrix, accelerate the speed of iterative approximation.
2, orthographic projection step and back projection's step are merged by positive/negative projection module, and minimizing is determined ray
Intersect the number of times of situation with voxel, accelerate the speed of iterative approximation.
Accompanying drawing illustrates:
Below in conjunction with the accompanying drawings the detailed description of the invention of the present invention is described in further detail.
Fig. 1 be present invention quick three-dimensional based on GPU CT iterative reconstruction approach be embodied as flow process
Figure；
Fig. 2 be the positive/negative projection of the present invention be embodied as schematic diagram；
Fig. 3 be orthographic projection of the present invention and record information be embodied as schematic diagram；
Fig. 4 be correction of the present invention and back projection be embodied as schematic diagram；
Fig. 5 is present invention quick three-dimensional based on GPU CT iterative approximation system block diagram.
Detailed description of the invention
Objects, features and advantages for enabling the present invention to implement become apparent from understandable, below in conjunction with attached
The specific embodiment of the present invention is described in detail by figure.
The invention provides quick three-dimensional CT iterative reconstruction approach based on GPU and system, such as Fig. 5
Shown in, quick three-dimensional CT iterative approximation system 100 based on GPU includes data input module 1,
Pretreatment module 2, positive/negative projection module 3, variable update module 4, iteration termination module 5, result
Output module 6；As shown in Fig. 1～Fig. 5, for the flow chart that is embodied as of this system:
Step S101 is data inputs, needs by CT equipment acquired projections data and is input to data
Input module 1, and motif position when obtaining this data for projection and size, the position of center of rotation,
Radiographic source is to distance, detector size, the distance of detector to center of rotation and the projection of center of rotation
Angle；
Step S102 is pretreatment, the pretreatment that pretreatment module 2 is relevant to data for projection work, such as,
In order to reduce the noise in data for projection, data for projection can be carried out nonlinear filtering, based on pattra leaves
The filtering of this theory of statistics；For the bar shaped artifact overcome after reconstruction in image, ring artifact, metal
Artifact and beam hardening artifact, can use at based on emerging images such as dictionary learning, form component analysis
Data for projection is processed by reason method, and data after processing and to rebuild relevant parameter biography
Entering to GPU, data and the parameter relevant to reconstruction after described process refer specifically to motif position and chi
Very little, the position of center of rotation, the distance of radiographic source to center of rotation, detector size, detector arrive
The distance of center of rotation and projection angle, data and parameter after processing due to these are on CPU,
These data also need to be passed on GPU to use GPU to be accelerated；
Step S103 is positive/negative projection, positive/negative projection module 3 carry out including orthographic projection step S301,
Record information Step S302 and correction step S401, back projection's step S402 operation (as shown in Figure 2),
This module realizes orthographic projection step S301 and back projection's step S301 on GPU, is calculated respectively
Orthographic projection system matrix and back projection system's matrix, as shown in Figure 2, Figure 3, Figure 4, this positive/negative throwing
Carry out orthographic projection step S301 and back projection's step S401 in shadow module 3, respectively obtain orthographic projection
Sytem matrix and back projection system's matrix, specifically include:
First, select the ray of a part of orthographic projection to be calculated, respectively these are chosen by multithreading
Ray carry out orthographic projection (such as the S301 of Fig. 3), record the result of every ray orthographic projection, and
The information (such as the S302 of Fig. 3) that ray intersects with voxel,
Then, after orthographic projection step S301 and information recording step S302 choosing ray completes,
Information according to record is modified step S401 and back projection's step S402 operation,
Wherein, makeover process is the most relevant with the orthographic projection of data for projection and every ray, according to the most just
The information of projection step record can obtain the projection value of every ray during current iteration, thus can
To calculate projection value after the correction of every ray rapidly, during back projection, according to the most just throwing
The information of shadow step record can be quickly obtained every ray and pass position and the length of voxel, thus can
To intersect situation according to every ray and voxel, rapidly projection value after the correction of every ray is divided
Being fitted on 8 voxels adjacent at intersection location, back projection's operation at this moment avoids to redefine penetrates
Line and the crossing situation of voxel, decrease the amount of calculation of algorithm, and back projection's operation can pass through multithreading
Operation is accelerated, but carries out processing ease based on voxel simultaneously and produce " write conflict ", each
Thread is required for the value distributing a blocks of data to store each voxel, and therefore back projection's operation can not enable
Many threads,
Finally, continue from remaining ray, choose the operation before a part of ray repeats, until institute
Some rays are all calculated and terminate circulation；
In described positive/negative projection module 3, correction step S401 of positive/negative projection process is by being used
Concrete iterative reconstruction algorithm determines, in order to become apparent, carries out specifically as a example by EM-ML algorithm
Bright, the following is the iterative formula of EM-ML algorithm:
Wherein, having N number of voxel in die body, in kth time iteration, the dose value at each voxel location is
Utilizing M bar roentgenization die body, data for projection is y, in formula,For orthographic projection
Journey,For modifying factor, to bijValue be modified, revised value is carried out back projectionBackprojected value is finally utilized to update currentValue, during orthographic projection, uses
Type of drive based on ray calculates sytem matrix aij, and back projection's process uses driving based on voxel
Mode calculates sytem matrix bij, the difference of two kinds of type of drive is: ray j is through the line length of voxel i
Degree is l, for type of drive based on ray, aijValue is the product of the density value of l and voxel i；Right
In type of drive based on voxel, bijValue is the product of the corresponding density value of l with l middle position, passes through
The density value of around middle position 8 voxels is carried out interpolation and obtains this density value, in positive/negative projection
During to use non-matching sytem matrix be to reduce the ring artifact rebuild in image, first calculating
Orthographic projectionRecord orthographic projection value and a of every rayijValue and position；Then, meter
Calculate the modifying factor of every ray, during above two, a plurality of penetrating can be calculated with multi-thread simultaneously
The orthographic projection value of line and modifying factor (as shown in Figure 3)；Finally, utilize interpolation algorithm, by modifying factor
Son and aijProduct be assigned to voxel i and around other 7 voxels, during back projection equally
Use multithreading is accelerated；
Step S104 is variable update, the variable update module 4 iteration as corresponding to concrete iterative algorithm
Formula updates currentValue；
Step S105 is iteration ends, and iteration termination module 5 judges whether current iteration meets iteration
End condition, if meeting the end condition of iteration, terminates iteration, and output iteration result is defeated to result
Go out module, otherwise, the positive/negative projection operation before continuation and variable update operation；
Step S106 be result output, result output module 6 by iteration result export, including storage and
Data after display iterative approximation.
The content not being described in detail in present specification belongs to existing known to professional and technical personnel in the field
Technology.
The above is only the preferred embodiment of the present invention, it is noted that general for the art
For logical technical staff, under the premise without departing from the principles of the invention, it is also possible to make some improvement
And retouching, these improvements and modifications also should be regarded as protection scope of the present invention.
Claims (2)
1. quick three-dimensional CT iterative approximation system based on GPU, it is characterised in that: include that data are defeated
Enter module, pretreatment module, positive/negative projection module, variable update module, iteration termination module, knot
Really output module；
Described data input module includes inputting data for projection, motif position and size, center of rotation
Position, the distance of radiographic source to center of rotation, detector size, detector to center of rotation away from
From and projection angle；
Described pretreatment module includes making data for projection relevant pretreatment, and the number after processing
It is passed to GPU according to this and to rebuilding relevant parameter；
Described positive/negative projection module carry out including orthographic projection step, record information Step and revise step,
Back projection's step operation, this module realizes orthographic projection step and back projection's step on GPU, counts respectively
Calculation obtains orthographic projection system matrix and back projection system's matrix, specifically includes: first, selects a part
These rays chosen are carried out orthographic projection by multithreading by the ray of orthographic projection to be calculated respectively, note
Record the result of every ray orthographic projection, and the information that ray intersects with voxel；Then, penetrate choosing
After the orthographic projection step of line and information recording step complete, it is modified step according to the information of record
Operation and back projection's step operation, wherein, makeover process and data for projection and the just throwing of every ray
Shadow is relevant, can obtain during current iteration every according to the information of orthographic projection step record before and penetrate
The projection value of line, such that it is able to calculate projection value after the correction of every ray rapidly, in back projection's mistake
Cheng Zhong, can be quickly obtained every ray through voxel according to the information of orthographic projection step record before
Position and length, such that it is able to intersect situation according to every ray and voxel, penetrate every rapidly
After the correction of line, projection value is assigned on 8 voxels adjacent at intersection location；Finally, continue from surplus
Remaining ray is chosen the operation before a part of ray repeats, until all of ray is all calculated
And terminate circulation；
Variate-value during current iteration is carried out by described variable update module according to back projection's result
Update；
Described iteration termination module includes judging the end condition whether current iteration meets iteration, as
Fruit meets the end condition of iteration, terminates iteration, and output iteration result is to result output module；Otherwise,
Positive/negative projection operation before continuation and variable update operation；
Described result output module includes the data after storage and display iterative approximation, by iteration result
Output.
2. quick three-dimensional CT iterative approximation system based on GPU as claimed in claim 1, it is special
Levy and be: in described positive/negative projection module back projection step, back projection's operation is entered by multithreading operation
Row accelerates, and each thread is required for the value distributing a blocks of data to store each voxel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610216968.7A CN105844690A (en) | 2016-03-29 | 2016-03-29 | Rapid three-dimensional CT iteration reconstruction system based on GPU |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610216968.7A CN105844690A (en) | 2016-03-29 | 2016-03-29 | Rapid three-dimensional CT iteration reconstruction system based on GPU |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105844690A true CN105844690A (en) | 2016-08-10 |
Family
ID=56597075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610216968.7A Pending CN105844690A (en) | 2016-03-29 | 2016-03-29 | Rapid three-dimensional CT iteration reconstruction system based on GPU |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105844690A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765514A (en) * | 2018-06-06 | 2018-11-06 | 上海交通大学 | A kind of accelerating method and device of CT image reconstructions |
| CN112051285A (en) * | 2020-08-18 | 2020-12-08 | 大连理工大学 | Intelligent nondestructive detection system integrating X-ray real-time imaging and CT (computed tomography) tomography |
| CN112669401A (en) * | 2020-12-22 | 2021-04-16 | 中北大学 | CT image reconstruction method and system based on convolutional neural network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080212860A1 (en) * | 2005-06-07 | 2008-09-04 | Koninklijke Philips Electronics, N.V. | Fast Reconstruction Algorithm for Cone-Beam Ct |
| CN101283913A (en) * | 2008-05-30 | 2008-10-15 | 首都师范大学 | GPU acceleration method of CT image reconstruction |
| CN103784158A (en) * | 2012-10-29 | 2014-05-14 | 株式会社日立医疗器械 | CT device and CT image generation method |
| CN104424625A (en) * | 2013-09-04 | 2015-03-18 | 中国科学院深圳先进技术研究院 | GPU (graphics processing unit) acceleration CBCT image reconstruction method and device |
-
2016
- 2016-03-29 CN CN201610216968.7A patent/CN105844690A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080212860A1 (en) * | 2005-06-07 | 2008-09-04 | Koninklijke Philips Electronics, N.V. | Fast Reconstruction Algorithm for Cone-Beam Ct |
| CN101283913A (en) * | 2008-05-30 | 2008-10-15 | 首都师范大学 | GPU acceleration method of CT image reconstruction |
| CN103784158A (en) * | 2012-10-29 | 2014-05-14 | 株式会社日立医疗器械 | CT device and CT image generation method |
| CN104424625A (en) * | 2013-09-04 | 2015-03-18 | 中国科学院深圳先进技术研究院 | GPU (graphics processing unit) acceleration CBCT image reconstruction method and device |
Non-Patent Citations (3)
| Title |
|---|
| ZENG G L等: "Unmatched Projector/Backprojector Pairs in an Iterative Reconstruction Algorithm", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 * |
| 莫会云: "三维锥束迭代算法的投影矩阵及去伪研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 雷德川: "基于CUDA的GPU加速迭代重建算法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765514A (en) * | 2018-06-06 | 2018-11-06 | 上海交通大学 | A kind of accelerating method and device of CT image reconstructions |
| CN108765514B (en) * | 2018-06-06 | 2020-12-11 | 上海交通大学 | Acceleration method and device for CT image reconstruction |
| CN112051285A (en) * | 2020-08-18 | 2020-12-08 | 大连理工大学 | Intelligent nondestructive detection system integrating X-ray real-time imaging and CT (computed tomography) tomography |
| CN112051285B (en) * | 2020-08-18 | 2021-08-20 | 大连理工大学 | Intelligent nondestructive detection system integrating X-ray real-time imaging and CT (computed tomography) tomography |
| CN112669401A (en) * | 2020-12-22 | 2021-04-16 | 中北大学 | CT image reconstruction method and system based on convolutional neural network |
| CN112669401B (en) * | 2020-12-22 | 2022-08-19 | 中北大学 | CT image reconstruction method and system based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105931280A (en) | GPU-based fast three-dimensional CT (computed tomography) iterative reconstruction method | |
| CN101404088B (en) | CT image reconstruction method and system | |
| CN109785334A (en) | Cardiac magnetic resonance images dividing method, device, terminal device and storage medium | |
| DE102007034006B9 (en) | System and method for direct volume playback | |
| DE102013106467A1 (en) | Method and device for iterative reconstruction | |
| CN106934766A (en) | A kind of infrared image super resolution ratio reconstruction method based on rarefaction representation | |
| DE102013020807A1 (en) | Handling Subordinate Z Cover Data in Raster Operations | |
| CN102984511A (en) | Method and apparatus for creating a multi-resolution framework for improving medical imaging workflow | |
| EP2469472A1 (en) | Computed tomography (ct) image reconstruction method and apparatus | |
| Bai et al. | Deep dose plugin: towards real-time Monte Carlo dose calculation through a deep learning-based denoising algorithm | |
| US8913710B2 (en) | Truncation correction imaging enhancement method and system | |
| CN105844690A (en) | Rapid three-dimensional CT iteration reconstruction system based on GPU | |
| CN108416821A (en) | A kind of CT Image Super-resolution Reconstruction methods of deep neural network | |
| CN107221013A (en) | One kind is based on variation light stream estimation lung 4D CT Image Super Resolution Processing methods | |
| Kandarpa et al. | DUG-RECON: a framework for direct image reconstruction using convolutional generative networks | |
| CN108520542A (en) | A kind of matched method for reconstructing of PET/CT data phase | |
| CN111862033B (en) | Medical image processing method, medical image processing device, medical image processing apparatus, and storage medium | |
| CN104408756A (en) | PET image reconstruction method and apparatus | |
| CN101520899B (en) | Method for parallel reconstruction of cone beam CT three-dimension images | |
| DE10255526B4 (en) | Recognition of vascular-associated pulmonary nodules by volume projection analysis | |
| CN109741254A (en) | Dictionary training and Image Super-resolution Reconstruction method, system, equipment and storage medium | |
| Nguyen et al. | Parallelizing a matched pair of ray-tracing projector and backprojector for iterative cone-beam CT reconstruction | |
| DE102005055922B4 (en) | Vessel reconstruction using bent planar reformation | |
| CN105809723B (en) | CBCT method for reconstructing and system | |
| US20230386036A1 (en) | Methods and systems for medical imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160810 |
|
| RJ01 | Rejection of invention patent application after publication |