CN113951928B - System and method for measuring heart rate value by utilizing ultrasonic image - Google Patents
System and method for measuring heart rate value by utilizing ultrasonic image Download PDFInfo
- Publication number
- CN113951928B CN113951928B CN202111493062.7A CN202111493062A CN113951928B CN 113951928 B CN113951928 B CN 113951928B CN 202111493062 A CN202111493062 A CN 202111493062A CN 113951928 B CN113951928 B CN 113951928B
- Authority
- CN
- China
- Prior art keywords
- heart rate
- images
- ultrasound
- rate value
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/02—Measuring pulse or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Cardiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The present disclosure relates to a system and method for measuring heart rate values using ultrasound images, the system comprising: the acquisition module is used for acquiring continuous multi-frame ultrasonic images and presetting a plurality of estimated heart rate values, wherein the multi-frame ultrasonic images are generated at a preset frequency; the selection module is configured to respectively take each estimated heart rate value as a target heart rate value, acquire a preset frame number based on the target heart rate value and a preset frequency, and select images at corresponding positions from a plurality of frames of ultrasonic images based on the preset frame number to form a plurality of image sequences; the calculating module is configured to calculate the similarity of two adjacent frames of images in each image sequence, acquire a plurality of similarity values, calculate an average value of the plurality of similarity values as an effective similarity value corresponding to the target heart rate value, compare the effective similarity values corresponding to each estimated heart rate value, and acquire the heart rate value based on the estimated heart rate value corresponding to the maximum effective similarity value. In this case, the heart rate value can be obtained more accurately.
Description
Technical Field
The present disclosure relates in particular to a system and method for measuring heart rate values using ultrasound images.
Background
Cardiovascular disease has become a leading cause of mortality worldwide (e.g., coronary heart disease). Blood vessels are currently diagnosed and monitored by ultrasound imaging of the blood vessel by an ultrasound imaging system (e.g., an IVUS system). In performing ultrasound imaging, the ultrasound imaging system may be subject to artifacts due to movement of tissue or organs (e.g., respiration, heartbeat, etc.), resulting in reduced quality of the acquired ultrasound images.
To obtain higher quality ultrasound images, techniques such as electrocardiographic gating are typically used to correct the acquired ultrasound images. When an ultrasound image is corrected by using an electrocardiographic gating technology, a heart rate value or a cardiac cycle of an imaging object of the ultrasound image is often required to be acquired first, so that a corresponding cardiac cycle is determined to correct the ultrasound image.
In the prior art, the corresponding heart rate value or cardiac cycle is often estimated directly from the ultrasound image. However, directly estimated heart rate values or cardiac cycles tend to be difficult to match with actual heart rate values or cardiac cycles.
Disclosure of Invention
The present disclosure has been made in view of the above-mentioned conventional art, and an object thereof is to provide a system and a method for measuring a heart rate value using an ultrasound image, which can obtain a relatively accurate heart rate value.
To this end, a first aspect of the present disclosure provides a system for measuring heart rate values using ultrasound images, comprising: the device comprises an acquisition module, a selection module and a calculation module, wherein the acquisition module is used for acquiring continuous multi-frame ultrasonic images and presetting a plurality of estimated heart rate values, and the multi-frame ultrasonic images are generated at a preset frequency; the selection module is configured to respectively take each estimated heart rate value as a target heart rate value, acquire a preset frame number based on the target heart rate value and the preset frequency, and select images at corresponding positions from the multi-frame ultrasonic images based on the preset frame number to form a plurality of image sequences, wherein two ultrasonic images at intervals of the preset frame number in the multi-frame ultrasonic images are ultrasonic images at corresponding positions; the calculating module is configured to calculate the similarity of two adjacent frames of images in each image sequence, acquire a plurality of similarity values, calculate an average value of the plurality of similarity values as an effective similarity value corresponding to the target heart rate value, compare the effective similarity values corresponding to each estimated heart rate value, and acquire the heart rate value based on the estimated heart rate value corresponding to the maximum effective similarity value.
In the method, a plurality of ultrasonic images are acquired through an acquisition module, a plurality of estimated heart rate values are preset, the ultrasonic images are acquired in blood vessels at a preset frequency, and the selection module and the calculation module can process the ultrasonic images based on the preset frequency and the estimated heart rate values, so that heart rate values corresponding to the ultrasonic images can be obtained. In this case, the heart rate value corresponding to the multi-frame ultrasound image can be obtained more accurately.
In addition, in the system provided in the first aspect of the present disclosure, optionally, the calculation module is further configured to obtain the cardiac cycle based on the estimated cardiac rate value corresponding to the maximum effective similarity value. In this case, the cardiac cycle corresponding to the multi-frame ultrasound image can be obtained more accurately.
Further, in the system provided in the first aspect of the present disclosure, optionally, the plurality of frames of ultrasound images are sequentially arranged within the acquisition module to constitute an ultrasound image sequence. Thus, the subsequent division of the ultrasound image sequence into a plurality of image groups can be facilitated.
Additionally, in the system provided in the first aspect of the present disclosure, optionally, each of the plurality of estimated heart rate values is numerically continuous, the plurality of estimated heart rate values including at least a target range of values. In this case, it can be convenient to obtain a more accurate heart rate value therefrom later.
Further, in the system provided in the first aspect of the present disclosure, optionally, the number of ultrasound images within each image sequence is not less than 2 frames. Thus, the subsequent acquisition of the similarity value corresponding to the image sequence can be facilitated.
In addition, in the system provided in the first aspect of the present disclosure, optionally, the calculating module is configured to calculate a similarity of two adjacent frames of ultrasound images in the corresponding target area to obtain a corresponding similarity value. Under the condition, the similarity value corresponding to the corresponding target area of the two adjacent frames of ultrasonic images can be acquired in a targeted manner, so that the accurate heart rate value can be obtained conveniently.
Further, in the system provided in the first aspect of the present disclosure, optionally, the ultrasound images within each image sequence are arranged in order within the ultrasound image sequence. Therefore, the similarity value corresponding to the two adjacent frames of ultrasonic images can be conveniently and accurately obtained.
In addition, in the system provided in the first aspect of the present disclosure, optionally, the number of the plurality of image sequences is equal to the size of the predetermined frame number. Therefore, the heart rate value corresponding to the multi-frame ultrasonic image can be conveniently and accurately obtained later.
Additionally, in the system provided in the first aspect of the present disclosure, optionally, the multi-frame ultrasound image is a intravascular ultrasound image acquired intravascular. Thereby, the corresponding heart rate value can be measured by using a plurality of frames of ultrasonic images.
A second aspect of the present disclosure provides a method of measuring a heart rate value using an ultrasound image, comprising: acquiring continuous multi-frame ultrasonic images and presetting a plurality of estimated heart rate values, wherein the multi-frame ultrasonic images are generated at a preset frequency; respectively taking each estimated heart rate value as a target heart rate value, acquiring a preset frame number based on the target heart rate value and the preset frequency, and selecting images positioned at corresponding positions from the multi-frame ultrasonic images based on the preset frame number to form a plurality of image sequences, wherein two ultrasonic images at intervals of the preset frame number in the multi-frame ultrasonic images are ultrasonic images positioned at corresponding positions; and calculating the similarity of two adjacent frames of images in each image sequence, acquiring a plurality of similarity values, calculating the average value of the plurality of similarity values as an effective similarity value corresponding to the target heart rate value, comparing the effective similarity values corresponding to the estimated heart rate values, and acquiring the heart rate value based on the estimated heart rate value corresponding to the maximum effective similarity value.
In the method, a plurality of ultrasonic images are acquired, a plurality of estimated heart rate values are preset, the ultrasonic images are acquired in blood vessels at a preset frequency, and the ultrasonic images are processed based on the preset frequency and the estimated heart rate values, so that heart rate values corresponding to the ultrasonic images can be obtained. In this case, the heart rate value corresponding to the multi-frame ultrasound image can be obtained more accurately.
According to the present disclosure, a system and method for measuring a heart rate value using an ultrasound image that can obtain a more accurate heart rate value can be provided.
Drawings
Fig. 1 is a block diagram showing a structure of a system according to an embodiment of the present disclosure.
Fig. 2 is a schematic diagram illustrating a structure of an ultrasound imaging system according to an embodiment of the present disclosure.
Fig. 3 is a schematic diagram illustrating an application of an ultrasound imaging system according to an embodiment of the present disclosure.
Fig. 4 is a schematic diagram illustrating acquisition of multiple frames of ultrasound images in accordance with embodiments of the present disclosure.
Fig. 5 is a schematic diagram illustrating acquisition of multiple image sequences based on multiple frames of ultrasound images in accordance with embodiments of the present disclosure.
Fig. 6 is a flow chart illustrating a method of measuring a cardiac cycle using ultrasound images in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same members are denoted by the same reference numerals, and overlapping description thereof is omitted. In addition, the drawings are schematic, and the ratio of the sizes of the components to each other, the shapes of the components, and the like may be different from actual ones.
The system 10 (see fig. 1) for measuring cardiac cycles (or heart rate values) using ultrasound images, to which the present disclosure relates, may measure cardiac cycles or heart rate values of an imaging subject at the time of acquisition of the ultrasound images based on the ultrasound images. By the system 10 for measuring cardiac cycles (or heart rate values) using ultrasound images in accordance with the present disclosure, more accurate cardiac cycles or heart rate values can be obtained.
The system 10 to which the present disclosure relates is described in detail below with reference to the accompanying drawings.
Fig. 1 is a block diagram illustrating a system 10 according to an embodiment of the present disclosure.
In some examples, referring to fig. 1, system 10 may include an acquisition module 110, a selection module 120, and a calculation module 130. Wherein the acquisition module 110 may be configured to acquire a plurality of frames of ultrasound images. The selection module 120 and the calculation module 130 may process the ultrasound images acquired by the acquisition module 110 to acquire a cardiac cycle (or heart rate value) of the imaging subject when acquiring multiple frames of ultrasound images.
In some examples, the acquisition module 110 may acquire multiple frames of ultrasound images. Wherein the ultrasound image may be an ultrasound image acquired by imaging an imaging subject by the ultrasound imaging system 20 (see fig. 2-4). In some examples, the ultrasound imaging system 20 may acquire (or generate) ultrasound images at a predetermined frequency. In some examples, the multi-frame ultrasound image acquired by the acquisition module 110 may be screened from ultrasound images generated at a time from a start of imaging the imaging subject by the ultrasound imaging system 20 to an end of imaging. In some examples, the multi-frame ultrasound image may be continuous. Thus, it is possible to facilitate subsequent confirmation of the cardiac cycle (or heart rate value) for which the multi-frame ultrasound image corresponds. In some examples, the multiple frames of ultrasound images acquired by the acquisition module 110 may be arranged in a certain order to form the ultrasound image sequence L (see fig. 4). Thereby, the subsequent division of the ultrasound image sequence L into a plurality of image groups can be facilitated. In some examples, the plurality of frames of ultrasound images within the sequence of ultrasound images L may be arranged in the order in which the ultrasound imaging system 20 generated the ultrasound images (i.e., the temporal order in which the ultrasound images were generated). In the embodiments to which the present disclosure relates, the earlier the ultrasound images arranged within the ultrasound image sequence L may be generated. For example, the acquisition module 110 may acquire multiple frames of ultrasound images. The multi-frame ultrasound image may be a continuous ultrasound image acquired by the ultrasound imaging system 20 within the vessel at a predetermined frequency.
Fig. 2 is a schematic diagram illustrating the structure of an ultrasound imaging system 20 according to an embodiment of the present disclosure. Fig. 3 is a schematic diagram illustrating an application of the ultrasound imaging system 20 according to an embodiment of the present disclosure.
In some examples, the ultrasound imaging system 20 may be an intravascular ultrasound imaging system. For example, the ultrasound imaging system 20 may acquire ultrasound images of a blood vessel within the blood vessel (see fig. 3). In this case, the multi-frame ultrasound image may be an intravascular ultrasound image acquired within the blood vessel. Thus, a plurality of frames of ultrasound images can be utilized to measure their corresponding cardiac cycle or heart rate values. Examples of the present disclosure are not limited thereto and in some examples, the ultrasound imaging system 20 may be an in vitro imaging system. For example, the ultrasound imaging system 20 may ultrasonically image a blood vessel of an imaging subject using B-mode ultrasound or the like to acquire an ultrasound image.
In some examples, the ultrasound imaging system 20 may include an extracorporeal device 210 and an intravascular device 220 (see fig. 2). The extracorporeal device 210 may be connected to the intravascular device 220, and the intravascular device 220 may collect information within the blood vessel and transmit the collected information to the extracorporeal device 210 for processing to generate an ultrasound image. In some examples, the intravascular device 220 can have a distal side that is distal from the extracorporeal device 210 and a proximal side that is proximal to the extracorporeal device 210.
In some examples, the intravascular device 220 can include a catheter 221, a drive shaft 222, and a probe 223 (see fig. 2 and 3). Wherein the catheter 221 may have an interior cavity, the probe 223 may be coupled to the drive shaft 222 and may move within the catheter 221 with the drive shaft 222. In addition, the probe 223 may be used for intravascular information collection.
In some examples, the intravascular device 220 may be inserted into an imaging subject, such as the blood vessel 30, for information acquisition within the blood vessel 30 (see fig. 3) while the ultrasound imaging system 20 is in operation. Specifically, during an interventional procedure, a medical practitioner may first perform a puncture from a site on the imaging subject (e.g., at the radial or femoral artery) and advance a medical guidewire along a blood vessel to an area to be imaged (e.g., segment AB of blood vessel 30 in fig. 3) within blood vessel 30, then advance catheter 221 along the medical guidewire to the area to be imaged, then advance drive shaft 222 to move within catheter 221 and place probe 223 at target location a; the healthcare worker may then operate the retraction mechanism 212 to control retraction and rotation of the drive shaft 222, and the probe 223 may move with the drive shaft 222 and may perform information collection within the blood vessel during the movement, for example, by transmitting and receiving acoustic or optical signals, etc. In this case, the probe 223 may acquire intravascular information of the region to be imaged, such as intravascular lumen and wall profile structure information of the region to be imaged, and the like. In some examples, the target location a may be a side of the region to be imaged near the distal side.
In some examples, as shown in fig. 1, the conduit 221 may be elongate tubular. In some examples, the catheter 221 may have a catheter lumen for disposing the drive shaft 222 and the probe 223. In this case, the drive shaft 222 and the probe 223 may be disposed within the catheter lumen, and the drive shaft 222 and the probe 223 may be movable (e.g., translatable and rotatable) relative to the catheter 221. In some examples, the catheter 10 may be circular in cross-section. Thereby, friction between the catheter 10 and the blood vessel 40 can be reduced, thereby reducing the risk of damage to the blood vessel 40.
In some examples, the drive shaft 222 is flexible. In this case, the drive shaft 222 can be adapted to travel in a curved blood vessel with the catheter 221, whereby the possibility of damage to the drive shaft 222 can be reduced. In some examples, a drive shaft 222 near the proximal side may be coupled to the retraction mechanism 212. Thus, the retraction mechanism 212 is capable of controlling retraction and rotation of the drive shaft 222. In some examples, a drive shaft 222 near the distal end may be coupled to the probe 223, the retraction mechanism 212 may control the drive shaft 222 to retract the probe 223, and the drive shaft 222 may be configured to withstand the torsional forces exerted by the retraction mechanism 212 when the probe 223 is retracted, requiring high speed rotation. In some examples, the drive shaft 222 may rotate within the catheter 221 and move relative to the catheter 221, in which case the catheter 221 may remain stationary without rotation and movement, thus enabling reduced damage to the blood vessel.
In some examples, the probe 223 may perform information acquisition by transmitting and receiving acoustic or optical signals. For example, the probe 223 may be an ultrasound probe that includes an ultrasound transducer. The probe 223 may acquire vessel information of the region to be imaged by transmitting and receiving an ultrasonic sound beam within the vessel. In some examples, information acquired by the probe 223 may be transmitted to the extracorporeal device 210 for processing. In some examples, the probe 223 may be connected to the extracorporeal device 210 by a connecting wire. In some examples, the connection leads may be disposed along the drive shaft 222. Examples of the present disclosure are not limited thereto and in some examples, the probe 223 may wirelessly transmit the acquired information to the extracorporeal device 210 for processing.
In some examples, referring to fig. 2, an extracorporeal device 210 may include a host 211 and a retraction mechanism 212. Wherein the host 211 may be coupled to the retraction mechanism 212 to enable signal transmission between the host 211 and the retraction mechanism 212.
In some examples, the retraction mechanism 212 may be coupled to a drive shaft 222 in the intravascular device 220. In some examples, the retraction mechanism 212 may be mechanically coupled to the drive shaft 222. For example, the retraction mechanism 212 may be coupled to the drive shaft 222 by a snap fit or screw fit or the like. In this case, the retraction mechanism 212 can control retraction and rotation of the drive shaft 222, and the probe 223 can move with the drive shaft 222, thereby enabling an ultrasound image of the area to be imaged within the vessel to be obtained. For example, referring to fig. 3, the ultrasound imaging system 20 may acquire ultrasound images of the blood vessel 30 in section AB. In some examples, the rate at which the retraction mechanism 212 controls retraction of the drive shaft 222 may be between 0 and 2mm/s. For example, the rate at which the retraction mechanism 212 controls retraction of the drive shaft 222 may be 0.5, 1, 1.5, or 2mm/s, etc.
Fig. 4 is a schematic diagram illustrating acquisition of multiple frames of ultrasound images in accordance with embodiments of the present disclosure. Fig. 4 (a) is a schematic diagram showing selection of a plurality of frames of ultrasound images from among ultrasound images acquired by the ultrasound imaging system 20. Fig. 4 (b) is a schematic diagram showing a plurality of frames of ultrasound images (i.e., a sequence L of ultrasound images).
In some examples, the host 211 may receive information acquired by the intravascular device 220 within the vessel and process to acquire ultrasound images. In some examples, the ultrasound imaging system 20 may acquire ultrasound images intravascularly at a predetermined frequency. In some examples, the predetermined frequency at which the ultrasound imaging system 20 acquires ultrasound images may be between 0 and 100 frames/s. That is, the ultrasound imaging system 20 may acquire between 0 and 100 frames of ultrasound images per second. For example, the predetermined frequency of the ultrasound imaging system 20 may be 10, 20, 30, 40, 50, 60, 70, 80, 90, or 100 frames/s. For example, referring to fig. 4 (a), the ultrasound imaging system 20 may sequentially acquire an ultrasound image a, an ultrasound image b, an ultrasound image c, and the like at a predetermined frequency until an ultrasound image d is acquired. Where ultrasound image a may be the first frame of ultrasound image acquired by ultrasound imaging system 20 at a of blood vessel 30 and ultrasound image d may be the last frame of ultrasound image acquired by ultrasound imaging system 20 at B of blood vessel 30. In some examples, as described above, the multi-frame ultrasound image acquired by the acquisition module 110 may be screened from the ultrasound images acquired by the ultrasound imaging system 20. For example, referring to fig. 4 (a) and 4 (b), a plurality of consecutive ultrasound images are selected from within the image sequence from the ultrasound image a to the ultrasound image d to constitute the ultrasound image sequence L. In some examples, the acquisition module 110 may acquire a predetermined frequency corresponding to a plurality of frames of ultrasound images. Specifically, the acquisition module 110 may acquire a predetermined frequency for a plurality of frames of ultrasound images as they are generated (or acquired) by the ultrasound imaging system 20.
In some examples, the acquisition module 110 may preset a plurality of pre-estimated heart rate values. In some examples, each of the pre-determined plurality of pre-determined heart rate values may be continuous in value. In this case, the plurality of estimated heart rate values may comprise at least a target range of values. In this case, a more accurate cardiac cycle or heart rate value can be facilitated to be obtained therefrom later. In some examples, the target numerical range may be determined based on a normal heart rate range. In some examples, the lower limit of the target range of values may be selected from 0 to 60 times/min. In some examples, the lower limit of the target range of values may be selected from 100 to 200 times/min. For example, the target value range may be 50 to 150 times/min, 60 to 120 times/min, or 60 to 180 times/min.
In some examples, as described above, the system 10 may also include a selection module 120. In some examples, the selection module 120 may be coupled to the acquisition module 110. In some examples, the selection module 120 may obtain a plurality of pre-estimated heart rate values, a plurality of frames of ultrasound images, and a predetermined frequency from the acquisition module 110.
In some examples, the selection module 120 may obtain a plurality of estimated periods based on each estimated heart rate value. In some examples, the selection module 120 may obtain, based on each estimated heart rate value, an estimated cardiac cycle corresponding to each estimated heart rate value, and use the estimated cardiac cycle as the estimated cycle corresponding to the estimated heart rate value. For example, if the estimated heart rate value is 60 times/min, the estimated heart cycle is 1s, i.e. the estimated cycle is 1s; if the estimated heart rate value is 120 times/min, the estimated heart cycle is 0.5s, namely the estimated cycle is 0.5s.
Fig. 5 is a schematic diagram illustrating acquisition of multiple image sequences based on multiple frames of ultrasound images in accordance with embodiments of the present disclosure. Wherein fig. 5 (a) shows a schematic diagram of dividing the multi-frame ultrasound image shown in fig. 4 (b) into a plurality of image groups. Fig. 5 (b) shows a schematic diagram of acquiring a plurality of image sequences from the plurality of image groups shown in fig. 5 (a).
In some examples, the selection module 120 may each take the respective estimated period as the target period. In this case, it is possible to facilitate the subsequent acquisition of the effective similarity value corresponding to each estimated period, so that the cardiac cycle can be acquired.
In some examples, the selection module 120 may divide the multi-frame ultrasound image into a plurality of image groups based on the target period and the predetermined frequency. Thereby, it is possible to facilitate the subsequent acquisition of a plurality of image sequences.
In some examples, the selection module 120 may obtain a corresponding number of ultrasound images (i.e., a predetermined number of frames) within the target period based on the target period and the predetermined frequency. In some examples, the selection module 120 may multiply the target period by a predetermined frequency to obtain a corresponding number of ultrasound images within the target period. For example, if the target period is 1s and the predetermined frequency is 30 frames/s, the number of corresponding ultrasound images in the target period may be 30 frames; if the target period is 0.5s and the predetermined frequency is 30 frames/s, the number of corresponding ultrasound images in the target period may be 15 frames. However, examples of the present disclosure are not limited thereto, and in some examples, the selection module 120 may individually treat each of the estimated heart rate values as the target heart rate value. In some examples, the selection module 120 may obtain a corresponding number of ultrasound images (which may be equivalent to a predetermined number of frames) within the estimated cardiac cycle based on the target heart rate value and the predetermined frequency. In some examples, the selection module 120 may calculate a ratio of the predetermined frequency to the target heart rate value to obtain a corresponding number of ultrasound images over the estimated cardiac cycle. For example, if the target heart rate value is 60 times/min (corresponding to a target period of 1 s), and the predetermined frequency is 30 frames/s, the ratio of the predetermined frequency to the target heart rate value may be 30 frames/time, i.e., the number of corresponding ultrasound images in the estimated cardiac period may be 30 frames (the predetermined frame number is 30 frames).
In some examples, the selection module 120 may divide the multi-frame ultrasound image into a plurality of image groups based on a predetermined number of frames. Thus, the selection module 120 can be facilitated to divide the multi-frame ultrasound image into a plurality of image groups. In some examples, the selection module 120 may continue to select from a target frame of ultrasound images (e.g., a first frame of ultrasound images) within a plurality of frames of ultrasound images, each time a predetermined number of frames of ultrasound images are selected, as a group of images, until a plurality of groups of images are acquired. In this case, the number of the respective image groups may be a predetermined number of frames. For example, the ultrasound image sequence may be split into a plurality of image groups comprising a predetermined number of consecutive ultrasound images.
However, examples of the present disclosure are not limited thereto, and in some examples, the selection module 120 may divide each of the ultrasound images located after the target frame ultrasound image within the multi-frame ultrasound image into respective image groups. In this case, the number of ultrasound images within the last image group (i.e., the nth image group) may be not more than a predetermined number of frames. Wherein the plurality of image groups may be named according to the order of arrangement of the first frame of ultrasound images within the ultrasound image sequence within each image group. For example, the first group of images is arranged furthest forward, and the last group of images is arranged furthest rearward. In some examples, if the number of ultrasound images within the last image group is less than the predetermined number of frames, the selection module 120 may remove the image group and perform subsequent processing on the remaining other image groups. In other examples, the selection module 120 may also reserve the last group of images if the number of ultrasound images within the group is less than a predetermined number of frames, and then process each group of images.
In some examples, within two adjacent image groups, the last frame of ultrasound image within the previous image group and the first frame of ultrasound image within the subsequent image group may be consecutive two frames of ultrasound images within the ultrasound image sequence L. In some examples, the number of image groups may be not less than two. In some examples, multiple frames of ultrasound images within each image group may be arranged in the order of the sequence of ultrasound images L, respectively. For example, referring to fig. 5 (a), the predetermined number of frames may be 15 frames, the total number of frames of the multi-frame ultrasound image may be i, wherein the ith frame of ultrasound image may be represented as S i The selection module 120 may select from the 1 st frame of ultrasound images (i.e., S 1 ) Initially, the multi-frame ultrasound image is divided into n image groups, n may be not less than 2. For example, the first group of images (i.e., T 1 ) The ultrasound image in the image may be S 1 ～S 15 The second group of images (i.e.T 2 ) The ultrasound image in the image may be S 16 ～S 30 The nth group of images (i.e., T n ) The ultrasound image in the image may be S m ～S i Etc., where m may be no greater than i. In this case, the number of ultrasound images of the nth image group may be not more than a predetermined number of frames.
In some examples, the selection module 120 may select a plurality of image sequences from each image group separately. In some examples, the selection module 120 may select ultrasound images at respective locations in respective image groups to construct a plurality of image sequences. In some examples, the images of the respective locations in the respective image groups may refer to ultrasound images that are respectively located at the same arrangement location within the respective image groups. In some examples, the number of image sequences may be one or more. In some examples, the number of image sequences may be no greater than a predetermined number of frames. For example, the number of image sequences may be equal to the size of the predetermined number of frames. Therefore, the cardiac cycle or heart rate value corresponding to the multi-frame ultrasonic image can be conveniently and accurately obtained later. In this case, the number of ultrasound images within the sequence of ultrasound images may be not less than a maximum predetermined number of the predetermined number of frames corresponding to the plurality of estimated heart rate values. For example, the number of frames of ultrasound images within the sequence of ultrasound images may be at least twice the maximum predetermined number of frames. In some examples, the order of the ultrasound images within each image sequence may be ordered by the order within the ultrasound image sequence L. In some examples, the ultrasound images located within the same image sequence may be ultrasound images at corresponding locations at respective target periods. That is, the time interval between two adjacent frames of ultrasound images within the same image sequence may be the target period. In some examples, each image sequence may contain several frames of ultrasound images. In some examples, the number of ultrasound images within each image sequence may be no less than 2 frames. Therefore, the similarity value corresponding to the two adjacent frames of ultrasonic images can be conveniently and accurately obtained.
For example, referring to fig. 5 (b), the selection module 120 may select from T 1 、T 2 、···、T n Ultrasound images of corresponding positions are selected internally to form an image sequence L 1 Image sequence L 2 Sequence of images L x Etc. X may be the same size as the predetermined number of frames. Wherein the selection module 120 can select from T 1 、T 2 、···、T n Respectively selecting the first frame of ultrasonic images in each group and arranging according to the original sequence (namely the arrangement sequence in the ultrasonic image sequence L) to form an image sequence L 1 The method comprises the steps of carrying out a first treatment on the surface of the Image sequence L 1 Can be respectively S 1 、S 16 And S is 31 Etc. The selection modules 120 may be respectively selected from T 1 、T 2 、T 3 、···、T n Respectively selecting the second frame of ultrasonic images in each group and arranging according to the original sequence (namely the arrangement sequence in the ultrasonic image sequence L) to form an image sequence L 2 Image sequence L 2 Can be respectively S 2 、S 17 And S is 32 Etc. The selection modules 120 may be respectively selected from T 1 、T 2 、T 3 、···、T n Respectively selecting X-th frame ultrasonic images in each group and arranging according to the original sequence (namely the arrangement sequence in the ultrasonic image sequence L) to form an image sequence L x Image sequence L x Can be respectively S x 、S x+15 And S is x+30 Etc.
In other examples, the selection module 120 may select a plurality of image sequences from a plurality of frames of ultrasound images (i.e., ultrasound image sequences L) according to a predetermined number of frames. In some examples, as described above, the selection module 120 may obtain a predetermined number of frames. In some examples, the selection module 120 may select ultrasound images located at respective positions from within a plurality of frames of ultrasound images to obtain a plurality of image sequences. In this case, the frame distance of the ultrasound images in which two adjacent frames are located at the corresponding positions may be a predetermined number of frames within the ultrasound image sequence. That is, within the ultrasound image sequence, two frames per predetermined number of frames are exceededThe acoustic image may be an ultrasound image at a corresponding location. In some examples, several frames of ultrasound images may be contained within each image sequence. Adjacent two frames of ultrasound images within each image sequence may be separated by a predetermined number of frames in the ultrasound image sequence L. For example, the total number of frames of multiple frames of ultrasound images within the ultrasound image sequence L may be i, where the ith frame of ultrasound image may be represented as S i . If the predetermined frame number is 15 frames, the selection module 120 may sequentially select a plurality of image sequences from the ultrasound image sequences L, where the plurality of image sequences may be the image sequences L 1 Image sequence L 2 Sequence of images L 15 Etc., the number of multiple image sequences may be 15 sets. Wherein the image sequence L 1 The ultrasonic images in the ultrasonic imaging device can be S respectively 1 、S 16 And S is 31 Etc., image sequence L 2 The ultrasonic images in the ultrasonic imaging device can be S respectively 2 、S 17 And S is 32 Etc., image sequence L 15 The ultrasonic images in the ultrasonic imaging device can be S respectively 15 、S 30 And S is 45 Etc.
In some examples, as described above, system 10 may include a computing module 130. In some examples, the computing module 130 may be coupled to the selection module 120. The computing module 130 may obtain a plurality of image sequences corresponding to the target period from the selecting module 120.
In some examples, the computing module 130 may be configured to calculate a similarity of any two frames of ultrasound images to obtain a similarity value corresponding to the two frames of ultrasound images. In the embodiment of the disclosure, the greater the corresponding similarity value of the two frames of ultrasound images, the higher the similarity of the two frames of ultrasound images. For example, the computing module 130 may calculate the similarity of the two frames of ultrasound images using a histogram, a mean-hash algorithm, a mean-square error (Mean Squared Error, MSE) equation, or a structural similarity (structural similarity index, SSIM) algorithm, etc., and may obtain the similarity value of the two frames of ultrasound images.
In some examples, the calculation module 130 may calculate the overall similarity of the two frames of ultrasound images and obtain the corresponding similarity values. In other examples, the computing module 130 may also calculate the similarity of the respective target areas of the two frames of ultrasound images to obtain corresponding similarity values. Under the condition, the similar values corresponding to the adjacent two frames of ultrasonic images in the corresponding target areas can be acquired in a targeted manner, so that the accurate cardiac cycle or heart rate value can be acquired conveniently. For example, the computing module 130 may use a machine learning algorithm in advance and train with the relevant ultrasound image data, and the computing module 130 may perform feature matching on the ultrasound image based on the training result to confirm its corresponding respective target region from the ultrasound image, so as to perform a relevance calculation for the respective target region. In some examples, the computing module 130 may calculate the similarity of the corresponding sector areas (i.e., the respective target areas) of the two frames of ultrasound images to obtain corresponding similarity values. In some examples, the scalloped region may be at any angle. For example, the calculation module 130 may acquire sector areas of 0 ° to 30 ° corresponding to each of the two frames of ultrasound images, and calculate the similarity of the two sector areas and acquire the corresponding similarity value. In this case, the similarity value corresponding to the two sector areas may be regarded as the similarity value corresponding to the two frames of ultrasound images.
In some examples, the computing module 130 may separately compute the similarity of two adjacent frames of ultrasound images in each image sequence to obtain a plurality of similarity values. Specifically, the calculation module 130 may calculate the similarity of any two adjacent frames of ultrasound images in any one image sequence to obtain a plurality of similarity values corresponding to the image sequence. In this case, the calculation module 130 may obtain a plurality of similarity values corresponding to a plurality of image sequences. For example, the computing module 130 may compute the image sequence L 1 The calculation module 130 may calculate S respectively according to the similarity of two adjacent frames of ultrasound images 1 And S is equal to 16 To obtain S 1 And S is equal to 16 Corresponding similarity values; the calculation module 130 can also calculate S separately 16 And S is equal to 31 To obtain S 16 And S is equal to 31 Corresponding similarity values, etc. In other examples, the computing module 130 may randomly select a plurality of sets of ultrasound images of two adjacent frames from each set of image sequences and calculate and obtain corresponding similarity values. In this case, the calculation module130 may obtain a number of similarity values corresponding to each set of image sequences.
In some examples, the computing module 130 may obtain the valid similarity value based on a plurality of similarity values corresponding to a plurality of image sequences. In some examples, the calculation module 130 may calculate an average of a plurality of similarity values corresponding to a plurality of image sequences as the valid similarity value corresponding to the target period. Specifically, the computing module 130 may obtain a number of similarity values corresponding to each image sequence. The computing module 130 may average a plurality of similarity values corresponding to a plurality of image sequences to obtain an average value corresponding to the plurality of similarity values. The calculation module 130 may take the average value as the effective similarity value for the target period. In other examples, the calculation module 130 may obtain a median value, a minimum value, a maximum value, or the like from among a plurality of similarity values corresponding to a plurality of image sequences as the valid similarity value.
In some examples, as described above, the selection module 120 may each target a respective estimated period, or the selection module 120 may each target a respective estimated heart rate value. In this case, the calculation module 130 may obtain valid similarity values corresponding to each estimated period or each estimated heart rate value. In some examples, the calculation module 130 may compare the effective similarity values for each estimated period or each estimated heart rate value and may select the largest effective similarity value from among the effective similarity values. In some examples, the calculation module 130 may obtain the estimated period corresponding to the largest valid similarity value. In some examples, the calculation module 130 may obtain the cardiac cycle based on the estimated cycle corresponding to the largest valid similarity value. For example, the calculation module 130 may take the estimated period corresponding to the maximum effective similarity value as the cardiac period corresponding to the ultrasound image sequence L (i.e., the multi-frame ultrasound images). In this case, the cardiac cycle corresponding to the ultrasound image sequence L can be obtained more accurately.
In other examples, the calculation module 130 may obtain the estimated heart rate value corresponding to the largest valid similarity value. In some examples, the calculation module 130 may obtain the heart rate value based on the estimated heart rate value corresponding to the largest valid similarity value. For example, the calculation module 130 may use the estimated heart rate value corresponding to the maximum effective similarity value as the heart rate value corresponding to the ultrasound image sequence L (i.e., the multi-frame ultrasound image). In this case, the heart rate value corresponding to the ultrasound image sequence L can be obtained more accurately.
Hereinafter, a method of measuring a cardiac cycle using an ultrasound image according to an example of the present embodiment will be described in detail with reference to fig. 6. Fig. 6 is a flow chart illustrating a method of measuring a cardiac cycle using ultrasound images in accordance with an embodiment of the present disclosure.
A method of measuring a cardiac cycle using an ultrasound image according to an embodiment of the present disclosure, referring to fig. 6, may include the steps of: acquiring a plurality of frames of ultrasonic images and preset frequencies, and presetting a plurality of estimated heart rate values (step S10); acquiring a plurality of image sequences from a plurality of frames of ultrasonic images according to the preset frequency and the estimated heart rate value (step S20); and acquiring effective similarity values according to the plurality of image sequences, and further acquiring a cardiac cycle corresponding to the multi-frame ultrasonic image (step S30).
In this embodiment, the method may refer to the above description of the acquisition and processing of the multi-frame ultrasound image, the predetermined frequency, the estimated heart rate value, the image sequence, the effective similarity value, and the cardiac cycle, respectively.
In step S10, as described above, a plurality of frames of ultrasound images and a predetermined frequency may be acquired, and a plurality of estimated heart rate values may be set in advance.
In some examples, a plurality of frames of ultrasound images may be acquired in step S10. In some examples, the ultrasound image may be an ultrasound image generated by imaging an imaging subject by ultrasound imaging system 20. In some examples, multiple frames of ultrasound images may be screened from ultrasound images generated at a time from the beginning of imaging an imaging subject by the ultrasound imaging system 20 to the end of imaging. In some examples, the ultrasound imaging system 20 may acquire ultrasound images at a predetermined frequency. In some examples, the multi-frame ultrasound image may be continuous. In some examples, a predetermined frequency corresponding to a plurality of frames of ultrasound images may be acquired in step S10. In some examples, a plurality of pre-estimated heart rate values may be preset in step S10. In some examples, the predetermined plurality of pre-estimated heart rate values may be continuous in value. In some examples, the plurality of estimated heart rate values may include at least a target range of values. In this embodiment, the setting of the target value range in the method may refer to the above description of the target value range. In some examples, step S10 may be implemented with the acquisition module 110 in the system 10.
In step S20, a plurality of image sequences are acquired from a plurality of frames of ultrasound images according to a predetermined frequency and a predicted heart rate value, as described above.
In some examples, in step S20, each estimated heart rate value in step S10 may be obtained. In some examples, a plurality of estimated cycles may be obtained based on each estimated heart rate value. In some examples, in step S20, each estimated period may be taken as a target period, respectively. In some examples, multiple frames of ultrasound images may be divided into multiple image groups based on a target period and a predetermined frequency.
In some examples, the predetermined number of frames may be acquired based on the target period and the predetermined frequency. In other examples, each of the estimated heart rate values may be separately referred to as a target heart rate value. In some examples, a corresponding number of ultrasound images (which may be equivalent to a predetermined number of frames) within the estimated cardiac cycle may be acquired based on the target heart rate value and the predetermined frequency.
In some examples, the multi-frame ultrasound image may be divided into a plurality of image groups based on a predetermined number of frames. In some examples, ultrasound images (simply "images") at respective locations in each image group may be selected to construct a plurality of image sequences. In other examples, multiple image sequences may be selected directly from multiple frames of ultrasound images (i.e., ultrasound image sequences L) according to a predetermined number of frames.
In this embodiment, the method may refer to the above description of the estimated period, the target period, the predetermined frame number, the target heart rate value, and the image group. In some examples, step S20 may be implemented with selection module 120 in system 10.
In step S30, as described above, effective similarity values may be obtained from a plurality of image sequences, and thus a cardiac cycle corresponding to a plurality of frames of ultrasound images may be obtained.
In some examples, the similarity of two adjacent frames of ultrasound images in each image sequence may be calculated in step S30 to obtain a plurality of similarity values. In some examples, an average of a plurality of similarity values corresponding to a plurality of image sequences may be calculated as a valid similarity value corresponding to a target period. In some examples, as described above, each estimated period may be taken as the target period in step S20, or each estimated heart rate value may be taken as the target heart rate value in step S20, respectively. In this case, each estimated period or the effective similarity value corresponding to each estimated heart rate value may be obtained. The effective similarity values corresponding to each estimated period or each estimated heart rate value can be compared, and the largest effective similarity value can be selected from the effective similarity values. In some examples, the cardiac cycle may be obtained based on the estimated cycle corresponding to the largest effective similarity value, or the heart rate value may be obtained based on the estimated heart rate value corresponding to the largest effective similarity value. In this case, the cardiac cycle or heart rate value corresponding to the ultrasound image sequence can be obtained more accurately.
In this embodiment, the obtaining and processing of the similarity value, the effective similarity value, and the maximum effective similarity value in the method may refer to the above description of the similarity value, the effective similarity value, and the maximum effective similarity value. In some examples, step S30 may be implemented with the computing module 130 in the system 10.
While the disclosure has been described in detail in connection with the drawings and embodiments, it should be understood that the foregoing description is not intended to limit the disclosure in any way. Modifications and variations of the present disclosure may be made as desired by those skilled in the art without departing from the true spirit and scope of the disclosure, and such modifications and variations fall within the scope of the disclosure.
Claims (10)
1. A system for measuring heart rate values using ultrasound images, comprising: the device comprises an acquisition module, a selection module and a calculation module, wherein the acquisition module is used for acquiring continuous multi-frame ultrasonic images and presetting a plurality of estimated heart rate values, and the multi-frame ultrasonic images are generated at a preset frequency; the selection module is configured to respectively take each estimated heart rate value as a target heart rate value, acquire a preset frame number based on the target heart rate value and the preset frequency, and select images positioned at corresponding positions from the multi-frame ultrasonic images based on the preset frame number to form a plurality of image sequences, wherein two ultrasonic images at intervals of the preset frame number in the multi-frame ultrasonic images are ultrasonic images positioned at corresponding positions; the calculating module is configured to calculate the similarity of two adjacent frames of images in each image sequence, acquire a plurality of similarity values, calculate an average value of the plurality of similarity values as an effective similarity value corresponding to the target heart rate value, compare the effective similarity values corresponding to each estimated heart rate value, and acquire the heart rate value based on the estimated heart rate value corresponding to the maximum effective similarity value.
2. The system of claim 1, wherein:
the calculation module is also used for obtaining the cardiac cycle based on the estimated heart rate value corresponding to the maximum effective similarity value.
3. The system of claim 1, wherein:
the multi-frame ultrasound images are sequentially arranged within the acquisition module to form a sequence of ultrasound images.
4. The system of claim 1, wherein:
each of the plurality of estimated heart rate values is numerically continuous, the plurality of estimated heart rate values comprising at least a target range of values.
5. The system of claim 1, wherein:
the number of ultrasound images within each image sequence is not less than 2 frames.
6. The system of claim 1, wherein:
the calculation module is used for calculating the similarity of two adjacent frames of ultrasonic images in the corresponding target areas so as to obtain corresponding similarity values.
7. A system as claimed in claim 3, wherein:
the ultrasound images within each image sequence are arranged in order within the ultrasound image sequence.
8. The system of claim 1, wherein:
the number of the plurality of image sequences is equal to the size of the predetermined frame number.
9. The system of claim 1, wherein:
the multi-frame ultrasound image is a intravascular ultrasound image acquired within a blood vessel.
10. A method for measuring heart rate values using ultrasound images, comprising: acquiring continuous multi-frame ultrasonic images and presetting a plurality of estimated heart rate values, wherein the multi-frame ultrasonic images are generated at a preset frequency; respectively taking each estimated heart rate value as a target heart rate value, acquiring a preset frame number based on the target heart rate value and the preset frequency, and selecting images positioned at corresponding positions from the multi-frame ultrasonic images based on the preset frame number to form a plurality of image sequences, wherein two ultrasonic images at intervals of the preset frame number in the multi-frame ultrasonic images are ultrasonic images positioned at corresponding positions; and calculating the similarity of two adjacent frames of images in each image sequence, acquiring a plurality of similarity values, calculating the average value of the plurality of similarity values as an effective similarity value corresponding to the target heart rate value, comparing the effective similarity values corresponding to the estimated heart rate values, and acquiring the heart rate value based on the estimated heart rate value corresponding to the maximum effective similarity value.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202023341069 | 2020-12-31 | ||
| CN2020233410690 | 2020-12-31 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113951928A CN113951928A (en) | 2022-01-21 |
| CN113951928B true CN113951928B (en) | 2023-08-18 |
Family
ID=79473012
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111493062.7A Active CN113951928B (en) | 2020-12-31 | 2021-12-08 | System and method for measuring heart rate value by utilizing ultrasonic image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113951928B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110664435A (en) * | 2019-09-23 | 2020-01-10 | 东软医疗系统股份有限公司 | Method and device for acquiring cardiac data and ultrasonic imaging equipment |
| CN111374706A (en) * | 2018-12-28 | 2020-07-07 | 深圳迈瑞生物医疗电子股份有限公司 | Fetal heart rate display method, ultrasonic imaging device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012081523A1 (en) * | 2010-12-13 | 2012-06-21 | 株式会社東芝 | Ultrasonic diagnostic device, image processing device, and image processing method |
| US10646196B2 (en) * | 2017-05-16 | 2020-05-12 | Clarius Mobile Health Corp. | Systems and methods for determining a heart rate of an imaged heart in an ultrasound image feed |
-
2021
- 2021-12-08 CN CN202111493062.7A patent/CN113951928B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111374706A (en) * | 2018-12-28 | 2020-07-07 | 深圳迈瑞生物医疗电子股份有限公司 | Fetal heart rate display method, ultrasonic imaging device and storage medium |
| CN110664435A (en) * | 2019-09-23 | 2020-01-10 | 东软医疗系统股份有限公司 | Method and device for acquiring cardiac data and ultrasonic imaging equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113951928A (en) | 2022-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7927275B2 (en) | System and method of aquiring blood-vessel data | |
| JP5362663B2 (en) | System and method for capturing blood vessel data | |
| EP3092951B1 (en) | Method and apparatus for synthesizing medical images | |
| US7672706B2 (en) | Systems and methods for measuring pulse wave velocity with an intravascular device | |
| US20130231558A1 (en) | System and Method for Reducing Angular Geometric Distortion in an Imaging Device | |
| CN114504343B (en) | Method for calculating cardiac cycle based on intravascular ultrasound image | |
| US20230157672A1 (en) | Intravascular ultrasound imaging and calcium detection methods | |
| AU2014223201B2 (en) | Systems and methods for lumen border detection in intravascular ultrasound sequences | |
| US20150087986A1 (en) | Systems and methods for producing intravascular images | |
| EP3048982A1 (en) | Systems and methods for producing intravascular images | |
| JP2010269018A (en) | Ultrasonic diagnostic apparatus | |
| CN212365043U (en) | Data processing system based on blood vessel image | |
| CN113951928B (en) | System and method for measuring heart rate value by utilizing ultrasonic image | |
| CN111462117A (en) | Data processing system based on blood vessel image and data processing method thereof | |
| JP6100910B2 (en) | Information processing apparatus, photographing system, information processing method, and program | |
| JP5826984B2 (en) | Ultrasonic diagnostic apparatus, heart rate synchronization signal generation apparatus, and heart rate synchronization signal generation method | |
| JP7194501B2 (en) | Electrocardiographic waveform timing detection device and medical image diagnostic device | |
| JP7356229B2 (en) | Ultrasound diagnostic equipment | |
| US20230112017A1 (en) | Medical device systems for automatic lesion assessment | |
| WO2023052278A1 (en) | Intraluminal ultrasound vessel segment identification and associated devices, systems, and methods | |
| CN117224168A (en) | Systems and methods for intravascular visualization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |