CN117860289A - Calibration method and device of X-ray machine, electronic equipment and storage medium - Google Patents
Calibration method and device of X-ray machine, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117860289A CN117860289A CN202410044926.4A CN202410044926A CN117860289A CN 117860289 A CN117860289 A CN 117860289A CN 202410044926 A CN202410044926 A CN 202410044926A CN 117860289 A CN117860289 A CN 117860289A
- Authority
- CN
- China
- Prior art keywords
- calibration tool
- ray
- preoperative
- layer calibration
- ray machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 239000002356 single layer Substances 0.000 claims abstract description 107
- 239000010410 layer Substances 0.000 claims abstract description 60
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 239000002184 metal Substances 0.000 claims abstract description 22
- 230000002285 radioactive effect Effects 0.000 claims description 35
- 239000002355 dual-layer Substances 0.000 claims description 28
- 239000003550 marker Substances 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 15
- 238000003384 imaging method Methods 0.000 abstract description 14
- 230000008569 process Effects 0.000 abstract description 8
- 210000000988 bone and bone Anatomy 0.000 abstract description 4
- 210000000056 organ Anatomy 0.000 abstract description 4
- 230000005855 radiation Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/58—Testing, adjusting or calibrating thereof
- A61B6/582—Calibration
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/40—Arrangements for generating radiation specially adapted for radiation diagnosis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/42—Arrangements for detecting radiation specially adapted for radiation diagnosis
- A61B6/4208—Arrangements for detecting radiation specially adapted for radiation diagnosis characterised by using a particular type of detector
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/58—Testing, adjusting or calibrating thereof
- A61B6/582—Calibration
- A61B6/585—Calibration of detector units
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Veterinary Medicine (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- High Energy & Nuclear Physics (AREA)
- Optics & Photonics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Robotics (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention relates to a calibration method and device of an X-ray machine, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring N preoperative X-ray images; when shooting before operation, a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate; calculating preoperative internal parameters of the X-ray machine based on the N preoperative X-ray images; acquiring an intraoperative X-ray image; during preoperative shooting, removing a double-layer calibration tool between the X-ray machine and the detection plate, and reserving a single-layer calibration tool; and acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine and the intra-operative X-ray image. That is, the preoperative geometric calibration process avoids interference of human bones and organs to imaging of the calibration tool; meanwhile, only a single-layer calibration tool is used in the operation, the metal ball imaging in the single-layer calibration tool is positioned at the image edge, and the interference of the metal ball in the single-layer calibration tool on the imaging in the operation is partially or completely avoided.
Description
Technical Field
The invention relates to the technical field of X-ray machine calibration, in particular to a calibration method and device of an X-ray machine, electronic equipment and a storage medium.
Background
For an operation navigation system based on an X-ray machine, geometric calibration of the X-ray machine is an indispensable technology. Imaging parameters obtained through geometric calibration of an X-ray machine are data necessary for realizing real-time visualization of an operation. In order to ensure that the surgical navigation meets the established precision requirement, the geometric calibration quality of the X-ray machine must be ensured.
At present, the geometric calibration of the X-ray machine is realized through the following steps: when shooting an intraoperative image, a double-layer calibration tool is arranged on an X-Ray machine in advance, and the relative position relation between the radioactive source and the detection plate is calculated through the imaging of a metal ball in the calibration tool in an X-Ray (X-Ray) image and the design parameters of the calibration tool. In the method, imaging of the marking tool is interfered by imaging of a human body, and geometric marking precision is low.
Disclosure of Invention
In order to solve the above prior art problems, the present invention provides a calibration method, device, electronic device and storage medium for an X-ray machine, so as to improve the calibration accuracy of the X-ray machine and reduce interference.
In a first aspect, an embodiment of the present application provides a calibration method of an X-ray machine, including: acquiring N preoperative X-ray images; the N preoperative X-ray images are obtained by shooting through an X-ray machine before an operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the preoperative shooting; n is a positive integer greater than 1; calculating preoperative internal parameters of the X-ray machine based on the N preoperative X-ray images and the spatial position information of the markers of the double-layer calibration tool; acquiring an intraoperative X-ray image; the X-ray image in the operation is obtained by shooting through the X-ray machine in the operation, and the double-layer calibration tool is removed between the X-ray machine and the detection plate and the single-layer calibration tool is reserved when shooting is performed before the operation; and acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial position information of the marker in the single-layer calibration tool and the intra-operative X-ray image.
In an optional implementation manner of the first aspect, the dual-layer calibration tool and the single-layer calibration tool each include a plurality of markers, and the calculating the preoperative internal reference of the X-ray machine based on the N preoperative X-ray images and the spatial position information of the markers of the dual-layer calibration tool includes: calculating the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image; and calculating to obtain the position parameters of the double-layer calibration tool in each preoperative X-ray image under a radioactive source coordinate system and the preoperative internal parameters of the X-ray machine based on the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image and the spatial position information of the markers in the double-layer calibration tool.
In an optional implementation manner of the first aspect, a calculation formula of the preoperative internal parameter of the X-ray machine is:
wherein K_offly represents preoperative internal parameters of the X-ray machine; e_double_offset_i represents the position parameter of the double-layer calibration tool corresponding to the preoperative X-ray image i under a radioactive source coordinate system; c_double_calibration_tool represents the spatial position information of the markers in the dual-layer calibration tool; p_double_offset_i represents the projected center coordinates of the markers of the dual-layer calibration tool corresponding to the preoperative X-ray image i.
In an optional implementation manner of the first aspect, the acquiring the intra-operative reference of the X-ray apparatus based on the pre-operative reference of the X-ray apparatus, the spatial position information of the marker in the single layer calibration tool, and the intra-operative X-ray image includes: calculating the projection center two-dimensional pixel coordinates of the markers of the single-layer calibration tool in the intraoperative X-ray image; calculating the position parameters of a single-layer calibration tool in a target preoperative X-ray image under a radioactive source coordinate system based on preoperative internal parameters of the X-ray machine; the target preoperative X-ray image is one of the N preoperative X-ray images; determining the position parameter of the single-layer calibration tool in the X-ray image under a radioactive source coordinate system and the intra-operative reference of the X-ray machine based on the projection center two-dimensional pixel coordinates of the marker of the single-layer calibration tool in the X-ray image and the spatial position information of the marker in the single-layer calibration tool; wherein, the relative position relation between the single-layer calibration tool and the detection plate is unchanged before and during operation.
In an optional implementation manner of the first aspect, a calculation formula of the intra-operative parameter of the X-ray machine is:
min∑||K_online*E_single_online*C_single_calibration_tool-P_single_online|| 2 ；
wherein, K_online represents the intra-operative reference of the X-ray machine; e_single_line represents the position parameter of a single-layer calibration tool in the intraoperative X-ray image under a radioactive source coordinate system; c_single_calibration_tool represents the spatial position information of the markers in the single layer calibration tool; p_single_line represents the projected center coordinates two-dimensional pixel coordinates of the markers of the single layer calibration tool in the intra-operative X-ray image.
In an alternative embodiment of the first aspect, the markers in the dual layer calibration tool and the single layer calibration tool comprise any one of metal balls or metal holes.
In an alternative embodiment of the first aspect, the angles of rotation of the dual layer calibration tool about the rotation axis are different in the N pre-operative X-ray images.
In a second aspect, an embodiment of the present application provides a calibration device for an X-ray machine, including: the first acquisition unit is used for acquiring N preoperative X-ray images; the N preoperative X-ray images are obtained by shooting through an X-ray machine before an operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the preoperative shooting; n is a positive integer greater than 1; the preoperative internal reference calculation unit is used for calculating preoperative internal references of the X-ray machine based on the N preoperative X-ray images and the spatial position information of the markers in the double-layer calibration tool; a second acquisition unit for acquiring an intra-operative X-ray image; the X-ray image in the operation is obtained by shooting through an X-ray machine in the operation, and the double-layer calibration tool is removed between the X-ray machine and the detection plate and the single-layer calibration tool is reserved when shooting is performed before the operation; and the intra-operative reference calculation unit is used for acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial position information of the marker in the single-layer calibration tool and the intra-operative X-ray image.
In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor executes the computer program to implement a calibration method of an X-ray machine according to any one of the first aspect of the present invention.
In a fourth aspect, an embodiment of the present application provides a computer readable medium, on which a computer program is stored, which when executed by a processor, implements a method for calibrating an X-ray machine according to any of the first aspect of the invention.
The beneficial effects of the invention include: in summary, the embodiment of the application provides a calibration method of an X-ray machine. The preoperative internal parameters of the X-ray machine are calculated by acquiring N preoperative X-ray images passing through the double-layer calibration tool and the single-layer calibration tool before operation, removing the double-layer calibration tool, acquiring an intraoperative X-ray image during operation, and acquiring the intraoperative internal parameters of the X-ray machine based on the preoperative internal parameters of the X-ray machine and the intraoperative X-ray image. Therefore, in the embodiment of the application, the geometric calibration of the X-ray machine is divided into preoperative calibration and operative calibration processes, the preoperative geometric calibration is carried out before the intraoperative geometric calibration, and the interference of human bones and organs on imaging of calibration tools is avoided in the preoperative geometric calibration process; meanwhile, only a single-layer calibration tool is used in the operation, the marker imaging in the single-layer calibration tool is positioned at the image edge, and the interference of the marker in the single-layer calibration tool on the imaging in the operation is partially or completely avoided.
Drawings
FIG. 1 is a flowchart illustrating steps of a calibration method of an X-ray machine according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a two-layer calibration tool and a single-layer calibration tool according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a coordinate system of a test board according to an embodiment of the present invention;
FIG. 4 is a schematic view of a radioactive source coordinate system according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a positional relationship between a dual layer calibration tool and a single layer calibration tool according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating steps of another calibration method of an X-ray machine according to an embodiment of the present invention;
FIG. 7 is a block diagram of a calibration apparatus for an X-ray machine according to an embodiment of the present invention;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context.
In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
At present, the geometric calibration of the X-ray machine is realized through the following steps: when shooting an intraoperative image, a double-layer calibration tool is arranged on an X-Ray machine in advance, and the relative position relation between the radioactive source and the detection plate is calculated through the imaging of a metal ball in the calibration tool in an X-Ray (X-Ray) image and the design parameters of the calibration tool.
The inventor finds that in the mode, the number of metal balls in the used operation marking tool is large, and in the center area of the image, the shot X-ray image is blocked, so that the observation of the image in the operation is influenced; the identification of the calibration tool can be interfered by the human bones and organs of the image in the operation, the extraction precision of the metal ball projection center coordinate is affected, and the geometric calibration precision is finally affected.
In view of the above problems, the present application proposes the following embodiments to solve the above technical problems.
Referring to fig. 1, an embodiment of the present application provides a calibration method of an X-ray machine, including: steps 101 to 104.
Step 101: n preoperative X-ray images were acquired.
The X-ray images before the operation are obtained by shooting through an X-ray machine before the operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the shooting before the operation; n is a positive integer greater than 1.
It should be noted that the dual-layer calibration tool is composed of two parallel planes, and each plane distributes a plurality of markers according to a certain rule. The single layer calibration tool is composed of a plane which distributes a plurality of markers according to a certain rule.
In the embodiments of the present application, the marker may be any one of a metal ball or a metal hole.
That is, prior to step 101, a dual layer calibration tool and a single layer calibration tool need to be placed between the radiation source and the detector plate of the X-ray machine. Specifically, the single-layer calibration tool is fixed on the detection plate through a mechanical device, so that the relative positions of the single-layer calibration tool and the detection plate are not changed, and then the double-layer calibration tool is arranged between the single-layer calibration tool and the radioactive source. Referring to fig. 2, fig. 2 shows a schematic diagram of the positions of a dual layer calibration tool and a single layer calibration tool. As shown in fig. 2, a radiation source (preoperative light source position) of the X-ray machine, a double-layer calibration tool, a single-layer calibration tool, and a detection plate are sequentially provided.
Step 102: based on the N preoperative X-ray images and the spatial position information of the markers of the double-layer calibration tool, preoperative internal parameters of the X-ray machine are calculated.
Step 103: an intra-operative X-ray image is acquired.
The X-ray image in the operation is obtained by shooting the detection plate through an X-ray machine in the operation, and a double-layer calibration tool is removed between the X-ray machine and the detection plate and a single-layer calibration tool is reserved when shooting before the operation.
Specifically, before step 103, the dual-layer calibration tool is removed, and only a single-layer calibration tool is fixed on the detection plate. Then, an X-ray machine is used to take X-ray images in the case of operation, and an X-ray image in operation is obtained.
Step 104: and acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial position information of the marker in the single-layer calibration tool and the intra-operative X-ray image.
In summary, the embodiment of the application provides a calibration method of an X-ray machine. The preoperative internal parameters of the X-ray machine are calculated by acquiring N preoperative X-ray images passing through the double-layer calibration tool and the single-layer calibration tool before operation, removing the double-layer calibration tool, acquiring an intraoperative X-ray image during operation, and acquiring the intraoperative internal parameters of the X-ray machine based on the preoperative internal parameters of the X-ray machine and the intraoperative X-ray image. Therefore, in the embodiment of the application, the geometric calibration of the X-ray machine is divided into preoperative calibration and operative calibration processes, the preoperative geometric calibration is carried out before the intraoperative geometric calibration, and the interference of human bones and organs on imaging of calibration tools is avoided in the preoperative geometric calibration process; meanwhile, only a single-layer calibration tool is used in the operation, the marker imaging in the single-layer calibration tool is positioned at the image edge, and the interference of the marker in the single-layer calibration tool on the imaging in the operation is partially or completely avoided.
Optionally, the dual layer calibration tool and the single layer calibration tool each include a plurality of markers, and the computing of the preoperative internal parameters of the X-ray machine based on the N preoperative X-ray images includes: calculating the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image; based on the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image and the spatial position information of the markers in the double-layer calibration tool, the position parameters of the double-layer calibration tool in each preoperative X-ray image under the radioactive source coordinate system and the preoperative internal parameters of the X-ray machine are calculated.
Optionally, the calculation formula of the preoperative internal parameters of the X-ray machine is as follows:
wherein K_offly represents preoperative internal reference of the X-ray machine; e_double_offset_i represents the position parameter of the double-layer calibration tool corresponding to the preoperative X-ray image i under the radioactive source coordinate system; c_double_calibration_tool represents the spatial position information of the markers in the dual-layer calibration tool; p_double_offset_i represents the projected center coordinates of the marker of the dual layer calibration tool corresponding to the preoperative X-ray image i.
Referring to fig. 3, coordinates [ f, u, v ] of the radiation source in a coordinate system with the upper left corner of the detection plate as the origin are defined as preoperative internal parameters, and a matrix form of the preoperative internal parameters is expressed as:
K_offline＝[[f1，0，u1]，
[0，f1，v1]
[0，0，1]]；
referring to fig. 4, in the radioactive source coordinate system, the position parameters of the dual-layer calibration tool corresponding to the preoperative X-ray image i in the radioactive source coordinate system can be expressed as:
E_double_offline_i＝[[e11，e12，e13，e14],
[e21，e22，e23，e24],
[e31，e32，e33，e34],
[e31，e32，e33，1]]；
then, the K_offine can be obtained by a calculation formula of the preoperative internal reference of the X-ray machine.
In an optional implementation manner of the first aspect, the acquiring the intra-operative reference of the X-ray apparatus based on the pre-operative reference of the X-ray apparatus, the spatial position information of the marker in the single layer calibration tool, and the intra-operative X-ray image includes: calculating the projection center two-dimensional pixel coordinates of the markers of the single-layer calibration tool in the X-ray image in the arithmetic; calculating position parameters of a single-layer calibration tool in a target preoperative X-ray image under a radioactive source coordinate system based on preoperative internal parameters of an X-ray machine; the target preoperative X-ray image is one of N preoperative X-ray images; determining the position parameter of the single-layer calibration tool in the X-ray image in the operation and the intra-operation reference of the X-ray machine based on the projection center two-dimensional pixel coordinates of the marker of the single-layer calibration tool in the X-ray image in the operation and the spatial position information of the marker in the single-layer calibration tool; wherein, the relative position relation between the single-layer calibration tool and the detection plate is unchanged before and during operation.
Optionally, the calculation formula of the intra-operative reference of the X-ray machine is as follows:
min∑||K_online*E_single_online*C_single_calibration_tool-P_single_online|| 2
wherein, K_online represents the intraoperative reference of the X-ray machine; e_single_line represents the position parameter of a single-layer calibration tool in an X-ray image in operation under a radioactive source coordinate system; c_single_calibration_tool represents the spatial position information of the markers in the single layer calibration tool; p_single_line represents the projected center coordinates two-dimensional pixel coordinates of the markers of the single layer calibration tool in the intra-operative X-ray image.
Specifically, based on the preoperative internal parameters of the X-ray machine, a formula for calculating the position parameters of the single-layer calibration tool in the target preoperative X-ray image under the radioactive source coordinate system may be:
min∑||K_offline*E_single_offline_N*C_single_calibration_tool-P_single_offline_N|| 2 ；
wherein E_single_offset_N represents the position parameter of the single-layer calibration tool in the radioactive source coordinate system in the X-ray image before the target operation. P_single_offset_n represents the projected center two-dimensional pixel coordinates of the marker of the single layer calibration tool in the target preoperative X-ray image.
The target preoperative X-ray image may be the preoperative X-ray image 2 (i.e., the second preoperative X-ray image) or may be any of them, and the present application is not limited thereto.
The step of determining the intra-operative reference of the X-ray machine may specifically be:
step one: the preoperative reference k_offset and the intra-operative reference k_online are expressed as:
K_offline＝[[f，0，u]，
[0，f，v]
[0，0，1]],
K_online＝[[f+Δf，u1+Δu1],
[0，f，v+Δv]
[0，0，1]]；
Δf, Δu, and Δv are differences between corresponding parameters of the preoperative reference k_offset and the intra-operative reference k_online.
Step two: the position parameter of the single-layer calibration tool in the X-ray image before the target operation under the radioactive source coordinate system is E_single_offset_N, and the position parameter of the single-layer calibration tool in the X-ray image during the operation under the radioactive source coordinate system is E-single-online, which is expressed as:
E_single_offline_N＝[[e11，e12，e13，e14]，
[e21，e22，e23，e24],
[e31，e32，e33，e34],
[e31，e32，e33，1]]；
E_single_online＝[[e11，e12，e13，e14+Δf],
[e21，e22，e23，e24-Δu],
[e31，e32，e33，e34-Δv],
[e31，e32，e33，1]]；
step three: the intra-operative reference K_online of the X-ray machine is calculated through the formula.
Optionally, the angles of rotation of the dual layer calibration tool about the rotation axis are different in the N pre-operative X-ray images.
Referring to FIG. 5, the dual layer calibration tool is rotatable about the x, y, and z axes shown in phantom in FIG. 5; under the condition that the position of the single-layer calibration tool is kept unchanged, X-ray images before operation are shot after the double-layer calibration tool is rotated by a certain angle around the X, y and z rotation axes shown by dotted lines in fig. 5.
For example, after the single-layer calibration tool and the double-layer calibration tool are fixed as shown in fig. 2, an X-ray machine may be used for first photographing to obtain a pre-operative X-ray image 1, then, X, y, and z rotation axes around dotted lines in fig. 5 are respectively used for photographing the pre-operative X-ray image after rotating the double-layer calibration tool by a certain angle, to obtain a pre-operative X-ray image 2, and so on, to obtain a pre-operative X-ray image N.
In the embodiment of the application, by adopting a rotatable multi-layer calibration tool design, the calibration tools under different external parameters can be shot, and geometric calibration is performed based on images corresponding to the calibration tools under different external parameters for a plurality of times
The calibration method of the X-ray machine provided in the embodiment of the present application is described below with reference to a specific example, and referring to fig. 6, the method includes:
preoperative geometric calibration:
step 1: as shown in FIG. 2, the double-layer calibration tool and the single-layer calibration tool are fixed between a radiation source and a detection plate of an X-ray machine, specifically, the single-layer calibration tool is fixed on the detection plate through a mechanical device, so that the relative positions of the single-layer calibration tool and the detection plate are unchanged, and then the double-layer calibration tool is installed between the single-layer calibration tool and the radiation source. As shown in fig. 2, a radiation source (preoperative light source position) of the X-ray machine, a double-layer calibration tool, a single-layer calibration tool, and a detection plate are sequentially provided.
Step 2: the X-ray image is taken using an X-ray machine, and a pre-operative X-ray image 1 (i.e., a pre-operative X-ray image 1) is acquired. Under the condition that the position of the single-layer calibration tool is kept unchanged, X, y and z rotation axes are shown by dotted lines in fig. 5, and the double-layer calibration tool is rotated for a certain angle and then is subjected to one-time pre-operation X-ray image shooting, so that a pre-operation X-ray image 2 (namely, a pre-operation X-ray image 2) is obtained. The dual-layer calibration tool can be rotated continuously and X-ray images can be taken to obtain N preoperative X-ray images.
Step 3: the metal balls in the single-layer calibration tool and the double-layer calibration tool can show corresponding projection in the X-ray image obtained by shooting, and the projection center of the metal ball before operation is extracted, and the extraction result is as follows:
projection center coordinates p_double_offset_i= { p_uv1, p_uv2, p_uv3..p_ uvM } of the metal sphere of the double-layer calibration tool corresponding to the preoperative X-ray image i.
Projection center two-dimensional pixel coordinates P-single-offset_i= { p_uv1, p_uv2, p_uv3,..p_ uvM } of the metal sphere of the single-layer calibration tool corresponding to the preoperative X-ray image i.
Step 4: based on the projection center coordinates of the metal balls of the double-layer calibration tool in each preoperative X-ray image, the position parameters of the double-layer calibration tool in each preoperative X-ray image under the radioactive source coordinate system and the spatial position information of the metal balls in the double-layer calibration tool, the preoperative internal parameters of the X-ray machine are calculated (based on the internal parameters, the relative position relation between the radioactive source and the detection plate can be obtained). The calculation method comprises the following steps:
a: referring to fig. 3, coordinates [ f, u, v ] of the radiation source in a coordinate system with the upper left corner of the detection plate as the origin are defined as preoperative internal parameters, and a matrix form of the preoperative internal parameters is expressed as:
K_offline＝[[f1，0，u1]，
[0，f1，v1]
[0，0，1]]；
referring to fig. 4, in the radioactive source coordinate system, the position parameters of the dual-layer calibration tool corresponding to the preoperative X-ray image i in the radioactive source coordinate system can be expressed as:
E_double_offline_i＝[[e11，e12，e13，e14],
[e21，e22，e23，e24],
[e31，e32，e33，e34],
[e31，e32，e33，1]]；
b: k_offine is obtained by calculation of the formula:
the description of the parameters in the formula may refer to the description in the foregoing embodiments, and will not be repeated here.
c: E_single_offset_N is recalculated by the formula: comprising the following steps:
min∑||K_offline*E_single_offline_N*C_single_calibration_tool-P_single-offline_N|| 2 。
the description of the parameters in the formula may refer to the description in the foregoing embodiments, and will not be repeated here.
Geometric calibration in operation:
step 5: the double-layer calibration tool is removed from the X-ray machine detection plate, and only a single-layer calibration tool is fixed on the C-arm detection plate.
Step 6: in the case of an operation, an X-ray machine is used to take an X-ray image, and an intra-operation X-ray image (i.e., an intra-operation X-ray image) is obtained.
Step 7: the single-layer calibration tool presents corresponding projection in an X-ray image obtained through shooting, the projection center of the metal ball of the single-layer calibration tool in the operation image is extracted, and the projection center coordinate two-dimensional pixel coordinates P_single_line= { P_uv1, P_uv2, P_uv3, & gt, P_ uvN } of the metal ball of the single-layer calibration tool in the operation X-ray image are obtained.
Step 8: and determining the intra-operative reference of the X-ray machine (the relative position relation between the radioactive source and the detection plate can be obtained based on the reference) based on the projection center two-dimensional pixel coordinates of the metal ball of the single-layer calibration tool in the intra-operative X-ray image, the position parameters of the single-layer calibration tool in the intra-operative X-ray image under the radioactive source coordinate system and the spatial position information of the metal ball in the single-layer calibration tool.
The calculation method comprises the following steps:
a: the preoperative reference k_offset and the intra-operative reference k_online are expressed as:
K_offline＝[[f，0，u]，
[0，f，v]
[0，0，1]],
K_online＝[[f+Δf，u1+Δu1],
[0，f，v+Δv],
[0，0，1]]
Δf, Δu, and Δv are differences between corresponding parameters of the preoperative reference k_offset and the intra-operative reference k_online.
b: the position parameter of the single-layer calibration tool in the X-ray image before the target operation under the radioactive source coordinate system is E_single_offset_N, and the position parameter of the single-layer calibration tool in the X-ray image during the operation under the radioactive source coordinate system is E_single_online, which is expressed as:
E_single_offline_N＝[[e11，e12，e13，e14]，
[e21，e22，e23，e24],
[e31，e32，e33，e34],
[e31，e32，e33，1]]；
E_single_online＝[[e11，e12，e13，e14+Δf],
[e21，e22，e23，e24-Δu],
[e31，e32，e33，e34-Δv],
[e31，e32，e33，1]]；
c: the intra-operative reference K-online of the X-ray machine is calculated through the formula.
min∑||K_online*E_single_online*C_single_calibration_tool-P_single_online|| 2
Referring to fig. 7, based on the same inventive concept, an embodiment of the present application provides a calibration device 700 of an X-ray machine, including:
a first acquiring unit 701 for acquiring N preoperative X-ray images; the N preoperative X-ray images are obtained by shooting through an X-ray machine before an operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the preoperative shooting; n is a positive integer greater than 1.
The preoperative internal reference calculating unit 702 is configured to calculate preoperative internal references of the X-ray machine based on the N preoperative X-ray images and spatial position information of markers of the dual-layer calibration tool.
A second acquisition unit 703 for acquiring an intra-operative X-ray image; the X-ray image in the operation is obtained by shooting through an X-ray machine in the operation, and when shooting is performed before the operation, the double-layer calibration tool is removed between the X-ray machine and the detection plate, and the single-layer calibration tool is reserved.
An intra-operative reference calculation unit 704, configured to obtain an intra-operative reference of the X-ray apparatus based on the pre-operative reference of the X-ray apparatus, the spatial position information of the marker in the single-layer calibration tool, and the intra-operative X-ray image.
Optionally, in an embodiment, the dual-layer calibration tool and the single-layer calibration tool each include a plurality of markers, and the preoperative internal parameter calculating unit 702 is further specifically configured to calculate a projection center coordinate of the markers of the dual-layer calibration tool in each of the preoperative X-ray images; and calculating to obtain the position parameters of the double-layer calibration tool in each preoperative X-ray image under a radioactive source coordinate system and the preoperative internal parameters of the X-ray machine based on the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image and the spatial position information of the markers in the double-layer calibration tool.
Optionally, in an embodiment, the intra-operative reference calculation unit 704 is further specifically configured to calculate a projection center two-dimensional pixel coordinate of a marker of the single-layer calibration tool in the intra-operative X-ray image; calculating the position parameters of a single-layer calibration tool in a target preoperative X-ray image under a radioactive source coordinate system based on preoperative internal parameters of the X-ray machine; the target preoperative X-ray image is one of the N preoperative X-ray images; determining the position parameter of the single-layer calibration tool in the X-ray image under a radioactive source coordinate system and the intra-operative reference of the X-ray machine based on the projection center two-dimensional pixel coordinates of the marker of the single-layer calibration tool in the X-ray image and the spatial position information of the marker in the single-layer calibration tool; wherein, the relative position relation between the single-layer calibration tool and the detection plate is unchanged before and during operation.
Referring to fig. 8, based on the same concept, an electronic device 800 applying the calibration method of the X-ray machine is further provided in the embodiments of the present application. The electronic device includes: a memory 802, a processor 801 and a computer program 803 stored in the memory and executable on the processor.
The electronic device 800 may be a personal computer, notebook computer, or the like.
It will be appreciated by those skilled in the art that fig. 8 is merely an example of an electronic device 800 and is not intended to limit the electronic device 800, and may include more or fewer components than shown, or may combine certain components, or may be different components.
The processor 801 may be a central processing unit (Central Processing Unit, CPU), the processor 801 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or a memory of the electronic device 800, in some embodiments. The memory 802 may also be an external storage device of the electronic device 800 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device 800. Further, the memory 802 may also include both internal and external storage units of the electronic device 800.
It should be noted that, because the content of information interaction and execution process between the above devices/units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Embodiments of the present application also provide a computer readable storage medium storing a computer program which, when executed by a processor, implements steps that may implement the various method embodiments described above.
Embodiments of the present application provide a computer program product which, when run on a mobile terminal, causes the mobile terminal to perform steps that may be performed in the various method embodiments described above.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a camera device/electronic apparatus, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other manners. For example, the apparatus/network device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (10)
1. A method for calibrating an X-ray machine, comprising:
acquiring N preoperative X-ray images; the N preoperative X-ray images are obtained by shooting through an X-ray machine before an operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the preoperative shooting; n is a positive integer greater than 1;
calculating preoperative internal parameters of the X-ray machine based on the N preoperative X-ray images and the spatial position information of the markers of the double-layer calibration tool;
acquiring an intraoperative X-ray image; the X-ray image in the operation is obtained by shooting through the X-ray machine in the operation, and the double-layer calibration tool is removed between the X-ray machine and the detection plate and the single-layer calibration tool is reserved when shooting is performed before the operation;
and acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial position information of the marker in the single-layer calibration tool and the intra-operative X-ray image.
2. The method of claim 1, wherein the dual layer calibration tool and the single layer calibration tool each comprise a plurality of markers, and wherein calculating the pre-operative internal parameters of the X-ray machine based on the N pre-operative X-ray images and the spatial location information of the markers of the dual layer calibration tool comprises:
calculating the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image;
and calculating to obtain the position parameters of the double-layer calibration tool in each preoperative X-ray image under a radioactive source coordinate system and the preoperative internal parameters of the X-ray machine based on the projection center coordinates of the markers of the double-layer calibration tool in each preoperative X-ray image and the spatial position information of the markers in the double-layer calibration tool.
3. The method according to claim 2, wherein the calculation formula of the preoperative internal parameters of the X-ray machine is:
wherein K_offly represents preoperative internal parameters of the X-ray machine; e_double_offset_i represents the position parameter of the double-layer calibration tool corresponding to the preoperative X-ray image i under a radioactive source coordinate system; c_double_calibration_tool represents the spatial position information of the markers in the dual-layer calibration tool; p_double_offset_i represents the projected center coordinates of the markers of the dual-layer calibration tool corresponding to the preoperative X-ray image i.
4. The method of claim 2, wherein the acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial location information of the markers in the single layer calibration tool, and the intra-operative X-ray image comprises:
calculating the projection center two-dimensional pixel coordinates of the markers of the single-layer calibration tool in the intraoperative X-ray image;
calculating the position parameters of a single-layer calibration tool in a target preoperative X-ray image under a radioactive source coordinate system based on preoperative internal parameters of the X-ray machine; the target preoperative X-ray image is one of the N preoperative X-ray images;
determining the position parameter of the single-layer calibration tool in the X-ray image under a radioactive source coordinate system and the intra-operative reference of the X-ray machine based on the projection center two-dimensional pixel coordinates of the marker of the single-layer calibration tool in the X-ray image and the spatial position information of the marker in the single-layer calibration tool; wherein, the relative position relation between the single-layer calibration tool and the detection plate is unchanged before and during operation.
5. The method of claim 4, wherein the X-ray machine has an intra-operative reference formula:
min∑||K_online*E_single_online*C_single_calibration_tool-P_single_online|| 2 wherein, K_online represents the intra-operative reference of the X-ray machine; e_single_line represents the position parameter of a single-layer calibration tool in the intraoperative X-ray image under a radioactive source coordinate system; c_single_calibration_tool represents the spatial position information of the markers in the single layer calibration tool;p_single_line represents the projected center coordinates two-dimensional pixel coordinates of the markers of the single layer calibration tool in the intra-operative X-ray image.
6. The method of claim 2, wherein the markers in the dual layer calibration tool and the single layer calibration tool comprise any one of metal balls or metal holes.
7. The method of claim 1, wherein the angle by which the dual layer calibration tool rotates about the axis of rotation is different among the N pre-operative X-ray images.
8. A calibration device for an X-ray machine, comprising:
the first acquisition unit is used for acquiring N preoperative X-ray images; the N preoperative X-ray images are obtained by shooting through an X-ray machine before an operation, and a double-layer calibration tool and a single-layer calibration tool are sequentially arranged between the X-ray machine and the detection plate during the preoperative shooting; n is a positive integer greater than 1;
the preoperative internal reference calculation unit is used for calculating preoperative internal references of the X-ray machine based on the N preoperative X-ray images and the spatial position information of the markers in the double-layer calibration tool;
a second acquisition unit for acquiring an intra-operative X-ray image; the X-ray image in the operation is obtained by shooting through the X-ray machine in the operation, and the double-layer calibration tool is removed between the X-ray machine and the detection plate and the single-layer calibration tool is reserved when shooting is performed before the operation;
and the intra-operative reference calculation unit is used for acquiring the intra-operative reference of the X-ray machine based on the pre-operative reference of the X-ray machine, the spatial position information of the marker in the single-layer calibration tool and the intra-operative X-ray image.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 7 when the computer program is executed.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044926.4A CN117860289A (en) | 2024-01-11 | 2024-01-11 | Calibration method and device of X-ray machine, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044926.4A CN117860289A (en) | 2024-01-11 | 2024-01-11 | Calibration method and device of X-ray machine, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117860289A true CN117860289A (en) | 2024-04-12 |
Family
ID=90578852
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410044926.4A Pending CN117860289A (en) | 2024-01-11 | 2024-01-11 | Calibration method and device of X-ray machine, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117860289A (en) |
-
2024
- 2024-01-11 CN CN202410044926.4A patent/CN117860289A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109118545B (en) | Three-dimensional imaging system calibration method and system based on rotating shaft and binocular camera | |
| CN113556977B (en) | C-arm-based medical imaging system and method for matching 2D image with 3D space | |
| US9715739B2 (en) | Bone fragment tracking | |
| US20170103505A1 (en) | Identification and Registration of Multi-marker Jig | |
| JP2017524464A (en) | Method and system for determining the position of a probe at a surgical site | |
| US20090310832A1 (en) | Medical Image Processing Method | |
| WO2013164368A1 (en) | Image distortion correction and robust phantom detection | |
| CN113362446B (en) | Method and device for reconstructing object based on point cloud data | |
| US20210374452A1 (en) | Method and device for image processing, and elecrtonic equipment | |
| CN112085798B (en) | Camera calibration method and device, electronic equipment and storage medium | |
| EP3213259A1 (en) | Method and system for estimating a deflated lung shape for video assisted thoracic surgery | |
| US20220054103A1 (en) | X-ray ripple markers for x-ray calibration | |
| CN115661214A (en) | Registration precision verification method and device | |
| CN115294166A (en) | Image edge extraction method and device | |
| JPH1047920A (en) | Calibration method | |
| CN103841895B (en) | The apparatus and method shown for image | |
| CN117860289A (en) | Calibration method and device of X-ray machine, electronic equipment and storage medium | |
| Bertelsen et al. | Distortion correction and calibration of intra-operative spine X-ray images using a constrained DLT algorithm | |
| CN113143459A (en) | Navigation method and device for augmented reality operation of laparoscope and electronic equipment | |
| CN110176066B (en) | Skull defect structure reconstruction method and device and electronic equipment | |
| CN113870292B (en) | Edge detection method and device of depth image and electronic equipment | |
| CN116237995A (en) | Method and device for adjusting path track and readable storage medium | |
| CN113393475B (en) | Mammary gland molybdenum target image segmentation device, electronic equipment, medical equipment and medium | |
| EP3834883A1 (en) | Tumor positioning method and device | |
| Lin et al. | Automatic registration method using EM sensors in the IoT operating room |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |