CN116269496A - Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation - Google Patents
Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation Download PDFInfo
- Publication number
- CN116269496A CN116269496A CN202310179178.6A CN202310179178A CN116269496A CN 116269496 A CN116269496 A CN 116269496A CN 202310179178 A CN202310179178 A CN 202310179178A CN 116269496 A CN116269496 A CN 116269496A
- Authority
- CN
- China
- Prior art keywords
- heart
- dimensional
- implicit
- dimensional ultrasonic
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001537 neural effect Effects 0.000 title claims abstract description 34
- 238000003384 imaging method Methods 0.000 title claims abstract description 30
- 230000004217 heart function Effects 0.000 title claims abstract description 22
- 238000011156 evaluation Methods 0.000 title claims abstract description 14
- 210000005241 right ventricle Anatomy 0.000 claims abstract description 23
- 210000005240 left ventricle Anatomy 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 21
- 239000000523 sample Substances 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims abstract description 16
- 238000002604 ultrasonography Methods 0.000 claims description 23
- 230000000747 cardiac effect Effects 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 7
- 238000012285 ultrasound imaging Methods 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 claims description 2
- 210000005036 nerve Anatomy 0.000 abstract 1
- 238000003759 clinical diagnosis Methods 0.000 description 4
- 208000024172 Cardiovascular disease Diseases 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000002861 ventricular Effects 0.000 description 2
- 210000003484 anatomy Anatomy 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011976 chest X-ray Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
- A61B8/0883—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of the heart
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/46—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient
- A61B8/461—Displaying means of special interest
- A61B8/466—Displaying means of special interest adapted to display 3D data
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Cardiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention provides a heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation. The system comprises a two-dimensional ultrasonic image acquisition module, a three-dimensional ultrasonic imaging module based on implicit neural representation and a heart function evaluation module; the two-dimensional ultrasonic image acquisition module is used for acquiring a two-dimensional ultrasonic image of the heart and estimating the position parameters of the ultrasonic image; the three-dimensional ultrasonic imaging module based on implicit nerve representation is used for three-dimensional reconstruction of the heart, and the module represents the whole heart as an implicit function which is input into three-dimensional coordinates and output into voxel values of corresponding positions; the cardiac function assessment module is used for automatically calculating the volumes of the left ventricle and the right ventricle of the heart and the ejection fraction. The three-dimensional heart reconstructed by the method comprises more cavity interior detail information than the three-dimensional heart acquired by the traditional three-dimensional probe.
Description
Technical Field
The invention relates to the technical field of three-dimensional ultrasonic imaging, in particular to a heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation.
Background
Cardiovascular disease has affected over 2.5 million people worldwide over the past three decades, with over 650 ten thousand people losing life due to cardiovascular disease. Cardiac imaging is an important tool for cardiovascular disease screening and diagnosis, including cardiac ultrasound, chest X-ray, and magnetic resonance imaging. Cardiac ultrasound imaging is the most convenient imaging mode for clinical examination due to its characteristics of safety, real-time, noninvasive, low cost, easy operation, etc. However, the conventional two-dimensional ultrasound images can only provide two-dimensional cross-sectional views of the heart, but cannot provide a clear three-dimensional heart structure, and a doctor needs to continuously adjust the detection angle of the ultrasound probe according to years of clinical experience of the doctor, understand a plurality of two-dimensional ultrasound images and restore the real three-dimensional structure of the ultrasound images in the brain. The process has high clinical experience requirements on doctors, and limits the accuracy and speed of diagnosis results and popularization of ultrasonic imaging technology. Therefore, the three-dimensional ultrasonic imaging is carried out on the heart, the complex anatomical structure and morphology in the heart are visualized, the limitation of the two-dimensional ultrasonic imaging technology can be overcome, better tissue contrast is obtained, the ejection fraction of the left ventricle and the right ventricle of the heart can be accurately calculated by matching with the semantic segmentation algorithm of the left ventricle and the right ventricle of the heart, and the accurate evaluation of the heart function is realized.
To achieve three-dimensional ultrasound imaging, existing imaging schemes can be divided into two categories: imaging methods based on three-dimensional ultrasound probes and imaging methods based on positioning sensors and two-dimensional ultrasound probes. The three-dimensional heart acquired by the method has poor spatial resolution, and a sonographer with high experience is required to perform manual calibration to evaluate the heart function. The latter requires the acquisition of positional parameters of two-dimensional ultrasound images by calibrating complex and expensive positioning sensors (e.g., acoustic, optical, articulated arm, electromagnetic, etc.), further acquisition of three-dimensional structures using a stitching overlay method. This method of three-dimensional imaging based on a positioning sensor and a two-dimensional ultrasound probe has not been used for three-dimensional imaging of the heart and evaluation of heart function to date.
Implicit neural representations can learn a continuous mapping from coordinates to signal values and find wide application in many fields, including image representation, multi-view synthesis, and linear inverse problem solving.
Disclosure of Invention
With respect to the above prior art, the present invention aims to achieve high quality three-dimensional cardiac ultrasound imaging, by introducing an implicit neural representation into the three-dimensional ultrasound imaging problem, a high quality three-dimensional heart can be reconstructed only from a set of two-dimensional ultrasound images without a large number of data sets, and the reconstructed heart can be used for evaluation of cardiac function, assisting a doctor in better clinical diagnosis.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the heart three-dimensional ultrasonic imaging and heart function evaluation system based on the implicit neural representation comprises a two-dimensional ultrasonic image acquisition module, a three-dimensional ultrasonic imaging module based on the implicit neural representation and a heart function evaluation module, wherein the two-dimensional ultrasonic image acquisition module is used for acquiring a two-dimensional ultrasonic image of a heart and estimating position parameters of the ultrasonic image; the three-dimensional ultrasonic imaging module based on implicit neural representation is used for three-dimensional reconstruction of the heart, and the module represents the whole heart as an implicit function which is input into three-dimensional coordinates and output into voxel values of corresponding positions; the cardiac function assessment module is used for automatically calculating the volumes of the left ventricle and the right ventricle of the heart and the ejection fraction.
Further, the two-dimensional ultrasonic image acquisition module adopts a two-dimensional ultrasonic probe.
Further, the method comprises the steps of:
scanning a heart region for a plurality of periods by using a two-dimensional ultrasonic probe, collecting two-dimensional ultrasonic images of different visual angles of the heart, and estimating the position parameters of each two-dimensional ultrasonic image;
step two, constructing a heart implicit neural representation by using a neural network, wherein the implicit neural representation is used for representing a three-dimensional structure of a heart, the three-dimensional coordinate of a heart space is input, and the implicit function is output as an implicit function of a voxel value of the three-dimensional heart at a corresponding position;
thirdly, constructing a loss function inspired by a two-dimensional ultrasonic imaging physical process, and supervising training of a neural network model parameter and an ultrasonic image position parameter by using the loss function so as to carry out three-dimensional ultrasonic imaging of the heart;
generating a group of two-dimensional slices of the heart by using the implicit neural representation of the heart constructed in the second step, dividing left and right ventricle areas in the slices by using a semantic segmentation algorithm, and calculating the area of a cavity in each slice; and then, the volumes of the left ventricle and the right ventricle are obtained by accumulating the areas of the left ventricle and the right ventricle in all the slices, and then the ejection fraction of the left ventricle and the ejection fraction of the right ventricle are calculated.
Further, in step two, the implicit neural representation of the heart constructs a continuous mapping from three-dimensional coordinates to three-dimensional cardiac voxel values and does not require extensive data set training.
The innovation point and the advantage of the invention are that:
(1) The invention realizes three-dimensional ultrasonic imaging of the heart by utilizing the neural network, and solves the physical limitations of low resolution and more artifacts of the traditional three-dimensional ultrasonic probe. The invention reconstructs a high quality three-dimensional heart using only the two-dimensional ultrasound images acquired by the two-dimensional ultrasound probe most commonly used in clinic. Because the resolution of the two-dimensional ultrasonic probe is higher than that of the traditional three-dimensional probe, the three-dimensional heart reconstructed by the method comprises more cavity interior detail information than the three-dimensional heart acquired by the traditional three-dimensional probe.
(2) The invention only uses a group of two-dimensional ultrasonic images which are swept around the apex of the heart to carry out three-dimensional reconstruction of the heart, and does not depend on a large-scale data set to carry out training of a network model. Thus avoiding the problem of data bias in conventional data-driven artificial intelligence algorithms. The invention can be used as a non-data-driven artificial intelligent tool to be embedded into two-dimensional ultrasonic machines widely used in hospitals to carry out high-quality three-dimensional heart reconstruction so as to carry out accurate, reliable and automatic heart function assessment in clinical diagnosis.
(3) The heart function evaluation module disclosed by the invention does not need calibration of a doctor, can automatically divide the left ventricle and the right ventricle in the three-dimensional heart, further can accurately obtain the volumes and ejection fraction of the left ventricle and the right ventricle of the heart, and assists the doctor to perform clinical diagnosis better.
Drawings
FIG. 1 is a block diagram of a system of the present invention;
FIG. 2 is a flow chart of a two-dimensional ultrasound image acquisition module in an embodiment of the invention;
FIG. 3 is a flowchart of an algorithm of a three-dimensional ultrasound imaging module based on implicit neural representation in an embodiment of the present invention;
FIG. 4 is a flowchart of an algorithm of a central function evaluation module according to an embodiment of the present invention;
fig. 5 is a graph comparing a three-dimensional heart reconstructed in accordance with an embodiment of the present invention with a three-dimensional heart acquired by a conventional three-dimensional probe.
Detailed Description
Embodiments of the present invention are described in detail below with reference to the drawings, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
Referring to fig. 1 and 2, the two-dimensional ultrasound image acquisition module is used to acquire two-dimensional ultrasound images of the heart and to estimate the position parameters of the ultrasound images, wherein the acquisition device of the two-dimensional ultrasound images is the two-dimensional ultrasound probe most commonly used in clinic, and no additional positioning sensor is required. The acquisition mode is to perform circular scanning at the apex of the heart by using a two-dimensional ultrasonic probe.
Referring to fig. 3, a three-dimensional ultrasonic imaging module based on implicit neural representation is used for high-quality three-dimensional reconstruction of the heart, and the specific implementation method is as follows:
firstly, constructing an implicit neural representation of a three-dimensional heart, and unlike the traditional method of taking pictures as input, the method of the invention represents the whole heart as one input as three-dimensional coordinates
Output as corresponding position voxel value +.>
Implicit function F of (2) θ Where θ is a parameter in the implicit neural representation. Implicit neural representations of the heart construct a continuous mapping from three-dimensional coordinates to three-dimensional cardiac voxel values and do not require extensive data sets for training:
further, the invention models the physical process of two-dimensional ultrasound acquisition as a process of physically slicing the heart and taking a cross section, and builds a loss function based on the acquired physical process to supervise training of the implicit neural representation. Specifically, a two-dimensional ultrasonic image acquisition module is utilized to acquire an image and estimate the position parameters of the ultrasonic image
Then generating a corresponding two-dimensional ultrasound image according to the position parameters of the ultrasound image by using the constructed implicit neural representation:
finally, two-dimensional image I generated by implicit neural representation i With a truly acquired two-dimensional image P i Constructing an error loss function to jointly optimize network model parameters of the implicit neural representation and position parameters of the ultrasonic image:
where N is the number of acquired two-dimensional images.
Referring to fig. 4, after reconstructing a high-precision three-dimensional heart, in order to obtain accurate volumes of left and right ventricles at end diastole and end systole, and further accurately calculate ejection fraction, the present invention firstly uses the constructed implicit neural representation of the heart to generate a set of two-dimensional ultrasonic slices of the heart, then uses a semantic segmentation algorithm to automatically segment left and right ventricular regions respectively, and calculates the area of a cavity in each slice. After the segmentation is finished, the areas of the left ventricle and the right ventricle in all the slices are accumulated, and the volumes of the left ventricle and the right ventricle are obtained. Finally, the ejection fraction of the left ventricle and the ejection fraction of the right ventricle are obtained according to the change rates of the volumes of the left ventricle and the right ventricle at the end diastole and the end systole.
Referring to fig. 5, the reconstructed three-dimensional heart of the present invention contains more detailed information within the cavity than the three-dimensional heart acquired by a conventional three-dimensional probe. Wherein (1) is the end diastole heart and (2) is the end systole heart.
Referring to table 1, the accuracy of the left ventricular ejection fraction calculation of the present invention is higher than that of the Nature n-shonet method (Ouyang, d., he, b., guorbani, a, yuan, n., ebinger, j, langlotz, c.p., j.y. (2020), video-based AI for beat-to-beat assessment of cardiac function. Nature,580 (7802), 252-256), and the ejection fraction of the right ventricle that cannot be calculated by the echo method can also be calculated, where the accuracy of ejection fraction estimation is evaluated using Mean Absolute Error (MAE) and Root Mean Square Error (RMSE).
TABLE 1 quantitative comparison of the accuracy of cardiac function assessment of the present invention with the echo Net method
The invention introduces the implicit neural representation into the three-dimensional ultrasonic imaging scheme of the heart, and solves the physical limitations of low resolution and more artifacts of the traditional three-dimensional ultrasonic probe. Meanwhile, the invention does not depend on a large-scale data set to train a network model, and only uses a group of two-dimensional ultrasonic images which are swept around the apex of the heart to reconstruct the heart in three dimensions, thereby avoiding the problem of data deviation in the traditional data-driven artificial intelligence algorithm. After the reconstruction is finished, the invention does not need the calibration of doctors, can automatically divide the left ventricle and the right ventricle in the three-dimensional heart, further can accurately obtain the volumes and the ejection fraction of the left ventricle and the right ventricle of the heart, and assists the doctors to better carry out clinical diagnosis.
Claims (4)
1. The heart three-dimensional ultrasonic imaging and heart function evaluation system based on the implicit neural representation comprises a two-dimensional ultrasonic image acquisition module, a three-dimensional ultrasonic imaging module based on the implicit neural representation and a heart function evaluation module, and is characterized in that the two-dimensional ultrasonic image acquisition module is used for acquiring a two-dimensional ultrasonic image of a heart and estimating position parameters of the ultrasonic image; the three-dimensional ultrasonic imaging module based on implicit neural representation is used for three-dimensional reconstruction of the heart, and the module represents the whole heart as an implicit function which is input into three-dimensional coordinates and output into voxel values of corresponding positions; the cardiac function assessment module is used for automatically calculating the volumes of the left ventricle and the right ventricle of the heart and the ejection fraction.
2. The implicit neural representation-based heart three-dimensional ultrasound imaging and cardiac function assessment system of claim 1, wherein the two-dimensional ultrasound image acquisition module employs a two-dimensional ultrasound probe.
3. A method of using an implicit neural representation-based cardiac three-dimensional ultrasound imaging and cardiac function assessment system as claimed in claim 1, the method comprising the steps of:
scanning a heart region for a plurality of periods by using a two-dimensional ultrasonic probe, collecting two-dimensional ultrasonic images of different visual angles of the heart, and estimating the position parameters of each two-dimensional ultrasonic image;
step two, constructing a heart implicit neural representation by using a neural network, wherein the implicit neural representation is used for representing a three-dimensional structure of a heart, the three-dimensional coordinate of a heart space is input, and the implicit function is output as an implicit function of a voxel value of the three-dimensional heart at a corresponding position;
thirdly, constructing a loss function inspired by a two-dimensional ultrasonic imaging physical process, and supervising training of a neural network model parameter and an ultrasonic image position parameter by using the loss function so as to carry out three-dimensional ultrasonic imaging of the heart;
generating a group of two-dimensional slices of the heart by using the implicit neural representation of the heart constructed in the second step, dividing left and right ventricle areas in the slices by using a semantic segmentation algorithm, and calculating the area of a cavity in each slice; and then, the volumes of the left ventricle and the right ventricle are obtained by accumulating the areas of the left ventricle and the right ventricle in all the slices, and then the ejection fraction of the left ventricle and the ejection fraction of the right ventricle are calculated.
4. A method according to claim 3, wherein in step two, the implicit neural representation of the heart constructs a continuous mapping from three-dimensional coordinates to three-dimensional heart voxel values, and does not require extensive data set training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310179178.6A CN116269496A (en) | 2023-03-01 | 2023-03-01 | Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310179178.6A CN116269496A (en) | 2023-03-01 | 2023-03-01 | Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116269496A true CN116269496A (en) | 2023-06-23 |
Family
ID=86819927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310179178.6A Pending CN116269496A (en) | 2023-03-01 | 2023-03-01 | Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116269496A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116705307A (en) * | 2023-08-07 | 2023-09-05 | 天津云检医学检验所有限公司 | AI model-based heart function assessment method, system and storage medium for children |
-
2023
- 2023-03-01 CN CN202310179178.6A patent/CN116269496A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116705307A (en) * | 2023-08-07 | 2023-09-05 | 天津云检医学检验所有限公司 | AI model-based heart function assessment method, system and storage medium for children |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Suhling et al. | Myocardial motion analysis from B-mode echocardiograms | |
| US20200405269A1 (en) | Ultrasound system with a neural network for producing images from undersampled ultrasound data | |
| McCann et al. | Multidimensional ultrasonic imaging for cardiology | |
| CN111968222B (en) | Three-dimensional ultrasonic reconstruction method for human tissue in non-static state | |
| CN102423264B (en) | Image-based biological tissue elasticity measuring method and device | |
| DE102007020862A1 (en) | User interface and method for displaying information in an ultrasound system | |
| KR20090059048A (en) | Anatomical modeling from a 3-d image and a surface mapping | |
| DE102010000274A1 (en) | Method and device for the automatic identification of image views in a 3D data record | |
| CN107427279A (en) | Use the Ultrasonic Diagnosis of the cardiac function of the cardiac module chamber with user's control | |
| CN115861132B (en) | Blood vessel image correction method, device, medium and equipment | |
| US20210077062A1 (en) | Device and method for obtaining anatomical measurements from an ultrasound image | |
| Yeung et al. | Implicitvol: Sensorless 3d ultrasound reconstruction with deep implicit representation | |
| JP2023521466A (en) | Biplane and 3D ultrasound image acquisition and associated systems and devices for generating roadmap images | |
| CN116269496A (en) | Heart three-dimensional ultrasonic imaging and heart function evaluation system based on implicit neural representation | |
| CN116883322A (en) | Method and terminal for measuring and managing heart parameters by using three-dimensional ultrasonic model | |
| EP2498222B1 (en) | Method and system for regression-based 4D mitral valve segmentation from 2D+T magnetic resonance imaging slices | |
| CN115330681A (en) | Ultrasonic image measuring method, system, intelligent terminal and storage medium | |
| CN113689441B (en) | Left ventricle ultrasonic dynamic segmentation method based on DeepLabV network | |
| CN109741439A (en) | A kind of three-dimensional rebuilding method of two dimension MRI fetus image | |
| EP1772825A1 (en) | Method for registering images of a sequence of images, particularly ultrasound diagnostic images | |
| CN110310364B (en) | Heart 3D printing system and printing method thereof | |
| CN107427282A (en) | Ultrasonic Diagnosis to cardiac function is split by single-degree-of-freedom chamber | |
| US20230186477A1 (en) | System and methods for segmenting images | |
| EP2189812B1 (en) | Providing volume information on a periodically moving target object in an ultrasound system | |
| CN114359327A (en) | AI-based ultrasonic cardiac anchor point dynamic measurement model and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |