CN114037775A - Bone structure growth method and device, electronic equipment and storage medium - Google Patents
Bone structure growth method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114037775A CN114037775A CN202210024152.XA CN202210024152A CN114037775A CN 114037775 A CN114037775 A CN 114037775A CN 202210024152 A CN202210024152 A CN 202210024152A CN 114037775 A CN114037775 A CN 114037775A
- Authority
- CN
- China
- Prior art keywords
- image frame
- determining
- current image
- original scanning
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000000988 bone and bone Anatomy 0.000 title claims abstract description 92
- 230000012010 growth Effects 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000037182 bone density Effects 0.000 claims abstract description 24
- 238000006073 displacement reaction Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 230000008468 bone growth Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 14
- 238000002591 computed tomography Methods 0.000 description 8
- 210000000689 upper leg Anatomy 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 210000002303 tibia Anatomy 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 210000003205 muscle Anatomy 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 210000003491 skin Anatomy 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- 208000029725 Metabolic bone disease Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 210000002082 fibula Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a bone structure growth method, a bone structure growth device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a sequence of original scanning image frames obtained by continuously scanning human bones; determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists; for each key image frame, respectively growing a skeleton structure in each target direction based on the seed points of the key image frame, and determining a backbone structure area in the skeleton structure growing process; the seed points are all pixel points in a foreground area of the key image frame. By the technical scheme of the embodiment of the invention, the convenience and the accuracy of the growth of the bone structure are improved, and the technical effect of determining the backbone structure is facilitated.
Description
Technical Field
The embodiment of the invention relates to the technical field of medical treatment, in particular to a bone structure growth method, a bone structure growth device, electronic equipment and a storage medium.
Background
CT (Computed Tomography) scanning, which is an X-ray Computed Tomography technique, can acquire images of tissue and organs inside a living body without loss and with accuracy. In clinical applications, accurate building of bone models is directly related to the subsequent analysis of bone conditions.
At present, in operations such as joint replacement, the CT scanning image line can be processed and judged by depending on experience observation of a doctor, so that on one hand, the workload of the doctor is increased, and the condition that the examination is not accurate enough exists. The methods for establishing the backbone structure through the CT scanning image comprise the steps of processing the image in the modes of filtering, automatic threshold value, morphological processing and the like, determining the backbone part, and then combining and establishing a skeleton model.
Disclosure of Invention
The embodiment of the invention provides a skeleton structure growth method, a skeleton structure growth device, electronic equipment and a storage medium, aiming at improving the convenience and accuracy of skeleton structure growth and facilitating determination of the technical effect of a skeleton structure.
In a first aspect, an embodiment of the present invention provides a method for growing a bone structure, including:
acquiring a sequence of original scanning image frames obtained by continuously scanning human bones;
determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists;
for each key image frame, respectively growing a skeleton structure in each target direction based on the seed points of the key image frame, and determining a backbone structure area in the skeleton structure growing process; the seed points are all pixel points in a foreground area of the key image frame.
In a second aspect, embodiments of the present invention also provide a bone structure growth apparatus, including:
the sequence acquisition module is used for acquiring a sequence of original scanning image frames obtained by continuously scanning human bones;
a key image frame determining module, configured to determine, according to each original scan image frame, at least one key image frame and at least one target direction corresponding to each key image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists;
the skeleton growth module is used for respectively growing skeleton structures in all target directions based on the seed points of the key image frames aiming at each key image frame and determining a backbone structure area in the skeleton structure growth process; the seed points are all pixel points in a foreground area of the key image frame.
In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a bone structure growth method according to any of the embodiments of the present invention.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements a bone structure growth method according to any one of the embodiments of the present invention.
According to the technical scheme of the embodiment, the sequence of original scanning image frames obtained by continuously scanning human bones is obtained, at least one key image frame and at least one target direction corresponding to each key image frame are determined according to each original scanning image frame, the bone structure growth is respectively carried out in each target direction based on the seed points of the key image frames for each key image frame, and the backbone structure area in the bone structure growth process is determined, so that the problems that the bones grow inaccurately and the backbone area is difficult to determine accurately are solved, the convenience and the accuracy of the bone structure growth are improved, and the technical effect of determining the backbone structure is facilitated.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, a brief description is given below of the drawings used in describing the embodiments. It should be clear that the described figures are only views of some of the embodiments of the invention to be described, not all, and that for a person skilled in the art, other figures can be derived from these figures without inventive effort.
FIG. 1 is a schematic flow chart illustrating a method for growing a bone structure according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a method for growing a bone structure according to a second embodiment of the present invention;
fig. 3 is a schematic view of a key image frame corresponding to a femur according to a second embodiment of the present invention;
fig. 4 is a schematic diagram of a key image frame corresponding to a tibia according to a second embodiment of the present invention;
fig. 5 is a schematic diagram of seed points on a key image frame corresponding to a femur according to a second embodiment of the present invention;
fig. 6 is a schematic diagram of a seed point on a key image frame corresponding to a tibia according to a second embodiment of the present invention;
FIG. 7 is a graph of predicted probabilities of a center and an edge according to a second embodiment of the present invention;
FIG. 8 is a schematic illustration of bone growth provided in accordance with a second embodiment of the present invention;
FIG. 9 is a schematic structural diagram of a bone structure growth device according to a third embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a schematic flow diagram of a bone structure growth method according to an embodiment of the present invention, which is applicable to determining a bone structure during bone growth according to a scanned image, and the method may be executed by a bone structure growth apparatus, and the system may be implemented in software and/or hardware, where the hardware may be an electronic device, and optionally, the electronic device may be a mobile terminal, and the like.
As shown in fig. 1, the method of this embodiment specifically includes the following steps:
and S110, acquiring a sequence of original scanning image frames obtained by continuously scanning human bones.
Wherein the sequence of raw scan image frames may be a plurality of successive slice images obtained by electronic computed tomography.
Specifically, continuous scanning of the computer tomography is performed on the bone part of the user, and a sequence composed of a plurality of continuous original scanning image frames for representing the bone condition of the human body can be obtained.
And S120, determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame.
The key image frame can be an initial image frame of subsequent bone growth, the key image frame is an original scanning image frame where an area with bone density meeting preset conditions exists, and the preset conditions can be pixel points with the maximum bone density value and the like. The target direction may be a bone extension direction, for example: there are two target directions for the femur, the cephalad direction and the plantar direction, and the target directions for different bones can be determined according to the growth characteristics of the bones.
Specifically, the original scan image frame in which the region with the bone density satisfying the preset condition is located is determined from each original scan image frame as a key image frame, for example: and taking the image frame with the local maximum bone density as a key image frame. Furthermore, at least one target direction may be determined according to the key image frame and the original scan image frame adjacent to the key image frame, or according to a human skeleton to which the key image frame belongs.
S130, for each key image frame, respectively growing the skeleton structure in each target direction based on the seed points of the key image frame, and determining the skeleton structure area in the skeleton structure growing process.
The seed points are all pixel points in a foreground region of the key image frame, and the foreground region is a region corresponding to the skeleton.
Specifically, bone growth is performed in each target direction according to the seed points of the key image frames, and when the bone growth reaches a region with a lower pixel value, the bone growth is stopped. And, the original scan image frame can be detected during the bone growth process to detect the backbone structure region.
According to the technical scheme of the embodiment, the sequence of original scanning image frames obtained by continuously scanning human bones is obtained, at least one key image frame and at least one target direction corresponding to each key image frame are determined according to each original scanning image frame, the bone structure growth is respectively carried out in each target direction based on the seed points of the key image frames for each key image frame, and the backbone structure area in the bone structure growth process is determined, so that the problems that the bones grow inaccurately and the backbone area is difficult to determine accurately are solved, the convenience and the accuracy of the bone structure growth are improved, and the technical effect of determining the backbone structure is facilitated.
Example two
Fig. 2 is a schematic flow chart of a bone structure growth method according to a second embodiment of the present invention, and in this embodiment, on the basis of the foregoing embodiments, reference may be made to the technical solution of this embodiment for a determination method of a key image frame and a bone growth method. The same or corresponding terms as those in the above embodiments are not explained in detail herein.
As shown in fig. 2, the method of this embodiment specifically includes the following steps:
s210, acquiring a sequence of original scanning image frames obtained by continuously scanning human bones.
And S220, determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame.
Specifically, the original scanning image frame where the area with the bone density meeting the preset condition is located is determined from each original scanning image frame as a key image frame, and at least one target direction corresponding to each key image frame is determined. Fig. 3 is a schematic diagram of a key image frame corresponding to a femur according to a second embodiment of the present invention, and fig. 4 is a schematic diagram of a key image frame corresponding to a tibia according to a second embodiment of the present invention.
Alternatively, the key image frame may be determined by two preset conditions:
determining the gray scale ratio of an original scanning image frame according to the maximum gray scale value and the minimum gray scale value in the gray scale values of the original scanning image frames aiming at each original scanning image frame; and if the number of the continuous original scanning image frames with the gray ratio value larger than the preset first ratio threshold value exceeds the preset first number, selecting the original scanning image frame corresponding to the maximum value of the gray ratio value as the key image frame.
Wherein, the gray scale ratio can be the ratio of the maximum gray scale value to the minimum gray scale value. The preset first number may be a number set according to the condition of the image frames, and may be, for example, 5, 10, or the like. The preset first ratio threshold may be a threshold used for judging the bone density.
Specifically, the gray scale ratio of each original scan image frame may be determined in the same manner, and next, an original scan image frame is taken as an example for description. Determining the maximum value and the minimum value of the gray scale according to the gray scale value of the pixel point at each position of the original scanning image frame, and taking the ratio of the maximum value and the minimum value of the gray scale as the gray scale ratio of the original scanning image frame. If the gray scale ratio of the consecutive original scanned image frames is greater than the preset first ratio threshold, and the number of the consecutive original scanned image frames exceeds the preset first number, it may be considered that the original scanned image frames include image frames with locally maximum bone density. And then, determining the maximum value of the gray ratio in the determined gray ratios of all the pixels in the continuous original scanning image frames, and determining the original scanning image frame to which the maximum value of the gray ratio belongs as the key image frame.
Determining the number of first pixels of each original scanning image frame based on an initial gray threshold and a first gray threshold, determining the number of second pixels of each original scanning image frame based on the first gray threshold and a second gray threshold, and determining the ratio of the number of the pixels of each original scanning image frame based on the number of the first pixels and the number of the second pixels; and if the number of the continuous original scanning image frames with the pixel point number ratio larger than the preset second ratio threshold exceeds the preset second number, selecting the original scanning image frame corresponding to the maximum value of the pixel point number ratio as the key image frame.
The initial gray threshold may be a gray value covering a non-bone region, the initial gray threshold may completely divide non-bone regions such as skin, muscle, blood vessels, etc. into a background region, and only a region with a higher bone density is reserved in a foreground region, the initial gray threshold may be between 450 and 500, and specific values may be set according to actual situations, for example: 500, etc. The first gray threshold may be a gray value for distinguishing low bone density from high bone density, and the first gray threshold may be between 950-. The second gray threshold may be an upper limit of the gray value of the high bone density region, the second gray threshold is between 1500-: 1500, etc. The first number of pixels may be the number of pixels in the original scanned image frame whose gray value is between the initial gray threshold and the first gray threshold. The second number of pixels may be the number of pixels in the original scanned image frame whose gray value is between the first gray threshold and the second gray threshold. The ratio of the number of pixels may be a ratio of the number of second pixels to the number of first pixels. The preset second number may be a number set according to the situation of the image frames, and may be, for example, 5, 10, or the like. The preset second ratio threshold may be a threshold used for judging the bone density.
Specifically, the ratio of the number of pixels can be determined in the same manner for each original scan image frame, and an original scan image frame is taken as an example for explanation: and taking the number of the pixel points with the gray values between the initial gray threshold value and the first gray threshold value as the number of the first pixel points, and taking the number of the pixel points with the gray values between the first gray threshold value and the second gray threshold value as the number of the second pixel points. And taking the ratio of the number of the second pixel points to the number of the first pixel points as the ratio of the number of the pixel points. If the ratio of the number of the pixels of the continuous original scanned image frames is greater than the preset second ratio threshold, and the number of the continuous original scanned image frames exceeds the preset second number, it can be considered that the original scanned image frames include image frames with local maximum bone density. And then, determining the maximum value of the ratio of the number of the pixels in the determined ratio of the number of the pixels in the continuous original scanning image frames, and determining the original scanning image frame to which the maximum value belongs as a key image frame.
When CT imaging is performed on a bone, the gray scale value of a region with high bone density such as a backbone in a CT image (original scan image frame) is high, for example: the gray value range exceeds 1000; whereas the grey values of non-osseous areas like skin, muscle, blood vessels are lower, for example: the grey scale values ranged from-100 to 250. In this embodiment, the range of the gray-level value may be between-1200 and 2400, or may be other range values, and is not particularly limited.
Illustratively, the number of pixels with gray values between the initial gray threshold and the first gray threshold (the number of first pixels) is counted and is denoted as T1. Further, the number of pixels with gray values between the first gray threshold and the second gray threshold (the number of second pixels) is counted and recorded as T2. If the initial gray threshold is 500, the first gray threshold is 1000, and the second gray threshold is 1500, counting the number of pixels with gray values in the range of 500-100, and recording as T1Counting the number of pixels with a gray value within the range of 1000-1500-2. And, can count the pixel quantity ratio as T according to the above-mentioned counted pixel quantity in each range2/T1And is denoted as T. If the T value of a plurality of continuous original scanning image frames is larger than a second preset value (for example, the preset value is 0.9-1), the original scanning image frames in the continuous range are considered to be in a region with higher bone density, and the image frame with the maximum T value is considered to be the image frame with local maximum bone density and is determined to be the key image frame.
And S230, regarding each key image frame, using the key image frame as a current image frame.
S240, determining a central point and an edge point set according to the seed points of the current image frame, determining a prediction probability of the central point and a prediction probability of the edge point set of the original scanning image frame adjacent to the current image frame in the target direction according to the central point and the edge point set, and determining the seed points in the original scanning image frame adjacent to the current image frame according to the prediction probabilities of the central point and the edge point set so as to grow the skeleton structure.
The original scan image frame adjacent to the current image frame refers to an original scan image frame subsequent to the current image frame, and optionally, a next original scan image frame. The center point may be a pixel point at the middle of the various sub-points. The set of edge points may be a set of pixels on a boundary formed among the various sub-points. Fig. 5 is a schematic diagram of a seed point on a key image frame corresponding to a femur according to a second embodiment of the present invention, and fig. 6 is a schematic diagram of a seed point on a key image frame corresponding to a tibia according to a second embodiment of the present invention.
Specifically, a center point and an edge point set in the current image frame may be determined according to the positions of various sub-points in the current image frame, and further, a center point and an edge point set of an original scan image frame adjacent to the current image frame in the target direction may be predicted according to the center point and the edge point set. And determining the center point as the middle position in the original scanning image frame adjacent to the current image frame according to the prediction result, and determining the pixel points in the region as seed points of the original scanning image frame adjacent to the current image frame by taking the pixel points in the edge point set as edges. Furthermore, the seed points are taken as foreground regions, i.e. parts of the bone region where bone is growing. In this way, the growth speed of the backbone region is faster while ensuring the accuracy of the growth of the backbone region.
Alternatively, the center point of the current image frame may be determined by:
step one, determining a zeroth order moment and a first order moment of a current image frame according to seed points of the current image frame.
The zeroth order moment can be understood as the sum of the gray values of the various sub-points. The first order moment can be understood as the sum of products of gray values of various sub-points and coordinate values of corresponding coordinate axes, wherein the coordinate values comprise a horizontal axis and a vertical axis, and the first order moment comprises a first order moment corresponding to the horizontal axis and a first order moment corresponding to the vertical axis.
Specifically, according to the gray value of the seed point of the current image frame, the zeroth order moment of the current image frame can be determined. According to the gray value of the seed point of the current image frame and the abscissa value or the ordinate value of each seed point, the first moment of the current image frame can be determined.
Optionally, the zeroth order moment and the first order moment of the current image frame are determined based on the following formulas:
wherein,
representing the zeroth order moment of the current image frame,
and
representing the first moment of the current image frame,V（i，j) Representing the coordinates located in the current image frame (i，j) The gray value of the seed point of (1), wherein,iis shown as the abscissa of the graph,jis a vertical coordinate of the main body of the device,Ia set of abscissas representing the seed points,Jrepresenting a set of seed point ordinates.
And step two, determining the central point of the current image frame according to the zero order moment and the first order moment.
Specifically, the abscissa of the central point is determined according to the ratio of the first moment to the zero moment corresponding to the abscissa, and the ordinate of the central point is determined according to the ratio of the first moment to the zero moment corresponding to the ordinate.
Optionally, the center point of the current image frame is determined based on the following formula:
wherein,
representing the zeroth order moment of the current image frame,
and
representing a first moment of the current image frame,
an abscissa representing the center point of the current image frame,
the ordinate representing the center point of the current image frame.
Fig. 7 is a prediction probability map of a center point and an edge according to a second embodiment of the present invention, and optionally, the prediction probability of the center point and the prediction probability of the edge point set of the original scan image frame adjacent to the current image frame in the target direction may be determined by the following steps:
step one, according to the center point of the current image frame and the center points of a preset number of original scanning image frames adjacent to the current image frame in the reverse direction of the target direction, the center point displacement direction and the center point displacement distance are determined.
The preset number may be a number set according to a requirement, for example: a value between 5 and 9. The center point displacement direction may be an average displacement direction from a preset number of original scanned image frames to the current image frame, and the center point displacement distance may be an average displacement distance from the preset number of original scanned image frames to the current image frame.
Specifically, the central points of a preset number of original scan image frames adjacent to the current image frame in the opposite direction of the target direction are counted, which may be understood as the central points of a preset number of original scan image frames before the original scan image frame. And then, calculating the direction and the distance of the central point displacement of each two adjacent frames, and calculating a mean value which is recorded as the central point displacement direction and the central point displacement distance.
Determining a predicted central point of the original scanning image frame adjacent to the current image frame in the target direction according to the central point, the central point displacement direction and the central point displacement distance of the current image frame, and determining the predicted probability of the central point of the original scanning image frame adjacent to the current image frame in the target direction based on the central point distribution probability with the predicted central point as the origin.
Wherein the predicted center point may be a predicted center point in a next original scan image frame of the current image frame. The center point distribution probability may be a probability distribution that predicts that each point around the center point is determined as the center point.
Specifically, the center point of the current image frame is used as a starting point, the center point displacement distance is displaced towards the center point displacement direction, and the displaced point is a predicted center point of an original scanning image frame adjacent to the current image frame in the target direction. And taking the prediction central point as an origin point to make a two-dimensional normal distribution probability, wherein the normal distribution probability is the central point prediction probability of the original scanning image frame adjacent to the current image frame in the target direction.
And step three, determining the distribution probability of the edge points with the edge points as the origin for each edge point in the edge point set of the current image frame.
The edge point distribution probability may be a probability that each point around the edge point is determined as an edge point.
Specifically, each point of the foreground edge points of the current image frame is taken as an origin to perform two-dimensional normal distribution, and the two-dimensional normal distribution is taken as the edge point distribution probability.
And fourthly, carrying out summation and normalization processing based on the distribution probability of each edge point, and determining the prediction probability of each edge point in the edge point set of the original scanning image frame adjacent to the current image frame in the target direction.
Specifically, the edge point distribution probabilities of the edge points are summed, and then normalized to obtain the prediction probability of the edge point set of the original scan image frame adjacent to the current image frame in the target direction.
S250, if the growth of the bone structure is finished, determining a backbone structure area in the growth process of the bone structure; and if not, taking the original scanning image frame adjacent to the current image frame in the target direction as the current image frame, and returning to execute the operation of determining the central point and the edge point set according to the seed points in the current image frame.
Specifically, if the growth of the bone structure is completed, the growth of the bone is indicated to be completed, and a backbone area in the bone growth process is determined. If the growth of the bone structure is not finished, the original scanning image frame adjacent to the current image frame in the target direction is used as the current image frame, and the operation of determining the central point and the edge point set according to the seed points in the current image frame is executed in a return mode so as to continue the bone growth.
For example, fig. 8 is a schematic diagram of bone growth provided by a second embodiment of the present invention, where the bone is a tibiofibula, a position indicated by a solid arrow is a position of a key image frame, directions indicated by dotted arrows on two sides of the key image frame are two target directions, a white region is two bone regions, an upper bone region is a tibiofibula bone region, and a lower bone region is a fibula bone region. .
Optionally, the backbone area may be determined by using a ratio of the number of pixels, for example: when the ratio of the number of the pixels is greater than or equal to a first preset value (for example, the first preset value is 0.4-0.6), it indicates that there are more gray-scale value high-brightness regions in the original scanned image frame, that is, there are more regions with higher bone density, and it can be determined as a backbone region; when the ratio of the number of the pixel points is smaller than a first preset value, it is indicated that the gray value highlight area in the original scanned image frame is less, namely the area with higher bone density is less, and the area can be determined as a bone sparse area.
Illustratively, taking the femur and the tibia as an example, starting from key image frames on the femur and the tibia, bone region growing is performed towards the proximal end and the distal end (target direction) respectively, so as to continuously add new bone voxel points in the three-dimensional voxel space. The region growth takes the seed points in the key image frame as a starting point, the growth condition is that according to the gray information in the 26 voxel points around the 3-dimensional space, if the difference of the gray values is less than a certain threshold value, the points are added into the seed set, the seed point set is continuously traversed to add new voxels, and the growth stop condition is as follows: (1) the gray value difference is larger than a threshold value, and (2) the bone sparse area grows to be the bone sparse area. The reason why the bone growth stops when the bone grows to the bone sparse region is that the gray scale ratio of the bone sparse region is small, wherein the bone high-density region is few, cannot be clearly distinguished from the surrounding muscle region or other tissue regions, and needs to be divided by other methods. In the process of three-dimensional region growing and segmenting, the center point and the edge (edge point set) of the two-dimensional bony structure of the current frame (current image frame) are identified and tracked, so that the center point of the next frame is calculated, and a prediction probability map of the center point and the edge is generated.
The technical scheme of the embodiment obtains the sequence of the original scanning image frames obtained by continuously scanning the human skeleton, determining at least one key image frame and at least one target direction corresponding to each key image frame from each original scan image frame, regarding the key image frame as a current image frame for each key image frame, determining a central point and an edge point set according to the seed points of the current image frame, determining a prediction probability of the central point and a prediction probability of the edge point set of an original scanning image frame adjacent to the current image frame in the target direction according to the central point and the edge point set, and determining seed points in the original scan image frame adjacent to the current image frame according to the prediction probability of the central point and the prediction probability of the edge point set, the skeletal structure growth is carried out, and if the skeletal structure growth is finished, a backbone structure area in the skeletal structure growth process is determined; otherwise, the original scanning image frame adjacent to the current image frame in the target direction is used as the current image frame, and the operation of determining the central point and the edge point set according to the seed points in the current image frame is executed in a return mode, so that the problems that the skeleton grows inaccurately and the backbone area is difficult to determine accurately are solved, the convenience and the accuracy of the skeleton structure growth are improved, and the technical effect of determining the backbone structure is facilitated.
EXAMPLE III
Fig. 9 is a schematic structural diagram of a bone structure growth device provided in a third embodiment of the present invention, the device including: a sequence acquisition module 310, a key image frame determination module 320, and a bone growth module 330.
The sequence acquisition module 310 is configured to acquire a sequence of original scan image frames obtained by continuously scanning human bones; a key image frame determining module 320, configured to determine, according to each original scan image frame, at least one key image frame and at least one target direction corresponding to each key image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists; a bone growth module 330, configured to perform, for each key image frame, bone structure growth in each target direction based on the seed points of the key image frame, and determine a backbone structure region in a bone structure growth process; the seed points are all pixel points in a foreground area of the key image frame.
Optionally, the key image frame determining module 320 is further configured to determine, for each original scan image frame, a gray scale ratio of the original scan image frame according to a maximum gray scale value and a minimum gray scale value in the gray scale values of the original scan image frame; and if the number of the continuous original scanning image frames with the gray ratio value larger than the preset first ratio threshold value exceeds the preset number, selecting the original scanning image frame corresponding to the maximum value of the gray ratio value as the key image frame.
Optionally, the key image frame determining module 320 is further configured to determine, for each original scanned image frame, a first number of pixels of the original scanned image frame based on an initial gray threshold and a first gray threshold, determine a second number of pixels of the original scanned image frame based on the first gray threshold and a second gray threshold, and determine a ratio of the numbers of pixels of the original scanned image frame based on the first number of pixels and the second number of pixels; and if the number of the continuous original scanning image frames with the pixel point number ratio larger than the preset second ratio threshold exceeds the preset second number, selecting the original scanning image frame corresponding to the maximum value of the pixel point number ratio as the key image frame.
Optionally, the initial grayscale threshold is between 450-.
Optionally, the bone growth module 330 is further configured to, for each key image frame, use the key image frame as a current image frame; determining a central point and an edge point set according to the seed points of the current image frame, determining a prediction probability of the central point and a prediction probability of the edge point set of an original scanning image frame adjacent to the current image frame in a target direction according to the central point and the edge point set, and determining the seed points in the original scanning image frame adjacent to the current image frame according to the prediction probabilities of the central point and the edge point set so as to grow a bone structure; if the growth of the bone structure is finished, determining a backbone structure area in the growth process of the bone structure; and if not, taking the original scanning image frame adjacent to the current image frame in the target direction as the current image frame, and returning to execute the operation of determining the central point and the edge point set according to the seed points in the current image frame.
Optionally, the bone growth module 330 is further configured to determine a zeroth order moment and a first order moment of the current image frame according to the seed point of the current image frame; and determining the central point of the current image frame according to the zero order moment and the first order moment.
Optionally, the bone growth module 330 is further configured to determine a zeroth order moment and a first order moment of the current image frame based on the following formulas:
wherein,
representing a zeroth order moment of the current image frame,
and
representing a first moment of the current image frame,V（i，j) Representing a position in the current image frame at coordinates (i，j) The gray value of the seed point of (1), wherein,iis shown as the abscissa of the graph,jis a vertical coordinate of the main body of the device,Ia set of abscissas representing the seed points,Jrepresenting a set of seed point ordinates;
accordingly, optionally, the bone growth module 330 is further configured to determine a center point of the current image frame based on the following formula:
wherein,
representing a zeroth order moment of the current image frame,
and
representing a first moment of the current image frame,
an abscissa representing a center point of the current image frame,
a vertical coordinate representing a center point of the current image frame.
Optionally, the bone growth module 330 is further configured to determine a central point displacement direction and a central point displacement distance according to a central point of the current image frame and central points of a preset number of original scan image frames adjacent to the current image frame in a direction opposite to the target direction; determining a predicted central point of an original scanning image frame adjacent to the current image frame in a target direction according to the central point of the current image frame, the central point displacement direction and the central point displacement distance, and determining the predicted probability of the central point of the original scanning image frame adjacent to the current image frame in the target direction based on the central point distribution probability with the predicted central point as an origin; determining the distribution probability of the edge points with the edge points as the origin for each edge point in the edge point set of the current image frame; and performing summation and normalization processing based on the distribution probability of each edge point, and determining the prediction probability of each edge point in the edge point set of the original scanning image frame adjacent to the current image frame in the target direction.
According to the technical scheme of the embodiment, the sequence of original scanning image frames obtained by continuously scanning human bones is obtained, at least one key image frame and at least one target direction corresponding to each key image frame are determined according to each original scanning image frame, the bone structure growth is respectively carried out in each target direction based on the seed points of the key image frames for each key image frame, and the backbone structure area in the bone structure growth process is determined, so that the problems that the bones grow inaccurately and the backbone area is difficult to determine accurately are solved, the convenience and the accuracy of the bone structure growth are improved, and the technical effect of determining the backbone structure is facilitated.
The bone structure growth device provided by the embodiment of the invention can execute the bone structure growth method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
It should be noted that, the units and modules included in the apparatus are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the embodiment of the invention.
Example four
Fig. 10 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. FIG. 10 illustrates a block diagram of an exemplary electronic device 40 suitable for use in implementing embodiments of the present invention. The electronic device 40 shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 10, electronic device 40 is embodied in the form of a general purpose computing device. The components of electronic device 40 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 that couples the various system components (including the system memory 402 and the processing unit 401).
The system memory 402 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 404 and/or cache 405. The electronic device 40 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 10, and commonly referred to as a "hard drive"). Although not shown in FIG. 10, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 403 by one or more data media interfaces. System memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 408 having a set (at least one) of program modules 407 may be stored, for example, in system memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 407 generally perform the functions and/or methods of the described embodiments of the invention.
The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), with one or more devices that enable a user to interact with the electronic device 40, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 40 to communicate with one or more other computing devices. Such communication may be performed through an I/O interface (input/output interface) 411. Also, the electronic device 40 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 412. As shown, the network adapter 412 communicates with the other modules of the electronic device 40 over the bus 403. It should be appreciated that although not shown in FIG. 10, other hardware and/or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 401 executes programs stored in the system memory 402 to execute various functional applications and data processing, for example, to implement the bone structure growth method provided by the embodiment of the present invention.
EXAMPLE five
An embodiment of the present invention also provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for bone structure growth, the method comprising:
acquiring a sequence of original scanning image frames obtained by continuously scanning human bones;
determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists;
for each key image frame, respectively growing a skeleton structure in each target direction based on the seed points of the key image frame, and determining a backbone structure area in the skeleton structure growing process; the seed points are all pixel points in a foreground area of the key image frame.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A method of growing a bone structure, comprising:
acquiring a sequence of original scanning image frames obtained by continuously scanning human bones;
determining at least one key image frame and at least one target direction corresponding to each key image frame according to each original scanning image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists;
for each key image frame, respectively growing a skeleton structure in each target direction based on the seed points of the key image frame, and determining a backbone structure area in the skeleton structure growing process; the seed points are all pixel points in a foreground area of the key image frame.
2. The method of claim 1, wherein determining at least one keyframe frame from each of the raw scan image frames comprises:
determining the gray ratio of each original scanning image frame according to the maximum gray value and the minimum gray value in the gray values of the original scanning image frames;
and if the number of the continuous original scanning image frames with the gray ratio value larger than the preset first ratio threshold value exceeds the preset number, selecting the original scanning image frame corresponding to the maximum value of the gray ratio value as the key image frame.
3. The method of claim 1, wherein determining at least one keyframe frame from each of the raw scan image frames comprises:
for each original scanning image frame, determining the number of first pixels of the original scanning image frame based on an initial gray threshold and a first gray threshold, determining the number of second pixels of the original scanning image frame based on the first gray threshold and a second gray threshold, and determining the ratio of the number of pixels of the original scanning image frame based on the number of the first pixels and the number of the second pixels;
and if the number of the continuous original scanning image frames with the pixel point number ratio larger than the preset second ratio threshold exceeds the preset second number, selecting the original scanning image frame corresponding to the maximum value of the pixel point number ratio as the key image frame.
4. The method as claimed in claim 3, wherein the initial gray threshold is between 450-500, the first gray threshold is between 950-1050, and/or the second gray threshold is between 1500-1800.
5. The method according to claim 1, wherein for each key image frame, performing bone structure growth in each target direction based on the seed points of the key image frame, and determining a skeletal structure region in the bone structure growth process comprises:
regarding each key image frame, taking the key image frame as a current image frame;
determining a central point and an edge point set according to the seed points of the current image frame, determining a prediction probability of the central point and a prediction probability of the edge point set of an original scanning image frame adjacent to the current image frame in a target direction according to the central point and the edge point set, and determining the seed points in the original scanning image frame adjacent to the current image frame according to the prediction probabilities of the central point and the edge point set so as to grow a bone structure;
if the growth of the bone structure is finished, determining a backbone structure area in the growth process of the bone structure; and if not, taking the original scanning image frame adjacent to the current image frame in the target direction as the current image frame, and returning to execute the operation of determining the central point and the edge point set according to the seed points in the current image frame.
6. The method of claim 5, wherein determining a center point from the seed point of the current image frame comprises:
determining a zeroth order moment and a first order moment of the current image frame according to the seed point of the current image frame;
and determining the central point of the current image frame according to the zero order moment and the first order moment.
7. The method of claim 6, wherein determining the zeroth order moment and the first order moment of the current image frame from the seed points of the current image frame comprises:
determining a zeroth order moment and a first order moment of the current image frame based on the following formulas:
wherein,
representing a zeroth order moment of the current image frame,
and
representing a first moment of the current image frame,V（i，j) Representing a position in the current image frame at coordinates (i，j) The gray value of the seed point of (1), wherein,iis shown as the abscissa of the graph,jis a vertical coordinate of the main body of the device,Ia set of abscissas representing the seed points,Jrepresenting a set of seed point ordinates;
correspondingly, the determining the center point of the current image frame according to the zero order moment and the first order moment includes:
determining a center point of the current image frame based on the following formula:
wherein,
representing a zeroth order moment of the current image frame,
and
representing a first moment of the current image frame,
an abscissa representing a center point of the current image frame,
a vertical coordinate representing a center point of the current image frame.
8. The method of claim 5, wherein determining the prediction probability of the center point and the prediction probability of the edge point set of the original scan image frame adjacent to the current image frame in the target direction according to the center point and the edge point set comprises:
determining a central point displacement direction and a central point displacement distance according to the central point of the current image frame and the central points of a preset number of original scanning image frames adjacent to the current image frame in the reverse direction of the target direction;
determining a predicted central point of an original scanning image frame adjacent to the current image frame in a target direction according to the central point of the current image frame, the central point displacement direction and the central point displacement distance, and determining the predicted probability of the central point of the original scanning image frame adjacent to the current image frame in the target direction based on the central point distribution probability with the predicted central point as an origin;
determining the distribution probability of the edge points with the edge points as the origin for each edge point in the edge point set of the current image frame;
and performing summation and normalization processing based on the distribution probability of each edge point, and determining the prediction probability of each edge point in the edge point set of the original scanning image frame adjacent to the current image frame in the target direction.
9. A bone structure growth device, comprising:
the sequence acquisition module is used for acquiring a sequence of original scanning image frames obtained by continuously scanning human bones;
a key image frame determining module, configured to determine, according to each original scan image frame, at least one key image frame and at least one target direction corresponding to each key image frame; the key image frame is an original scanning image frame in which an area with bone density meeting a preset condition exists;
the skeleton growth module is used for respectively growing skeleton structures in all target directions based on the seed points of the key image frames aiming at each key image frame and determining a backbone structure area in the skeleton structure growth process; the seed points are all pixel points in a foreground area of the key image frame.
10. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a bone structure growth method as recited in any of claims 1-8.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the bone structure growth method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210024152.XA CN114037775B (en) | 2022-01-11 | 2022-01-11 | Bone structure growth method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210024152.XA CN114037775B (en) | 2022-01-11 | 2022-01-11 | Bone structure growth method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114037775A true CN114037775A (en) | 2022-02-11 |
| CN114037775B CN114037775B (en) | 2022-04-01 |
Family
ID=80141534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210024152.XA Active CN114037775B (en) | 2022-01-11 | 2022-01-11 | Bone structure growth method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037775B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565606A (en) * | 2022-03-23 | 2022-05-31 | 推想医疗科技股份有限公司 | Coronary artery segmentation method, coronary artery segmentation device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201121509A (en) * | 2009-12-30 | 2011-07-01 | Pou Yuen Technology Co Ltd | Segmentation method of multiple bone regions. |
| CN107945278A (en) * | 2017-12-14 | 2018-04-20 | 桂林电子科技大学 | The three-dimensionalreconstruction and finite element method of a kind of organism structure |
| US20190336097A1 (en) * | 2014-07-21 | 2019-11-07 | Zebra Medical Vision Ltd. | Systems and methods for prediction of osteoporotic fracture risk |
| CN110570515A (en) * | 2019-09-03 | 2019-12-13 | 天津工业大学 | method for carrying out human skeleton three-dimensional modeling by utilizing CT (computed tomography) image |
-
2022
- 2022-01-11 CN CN202210024152.XA patent/CN114037775B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201121509A (en) * | 2009-12-30 | 2011-07-01 | Pou Yuen Technology Co Ltd | Segmentation method of multiple bone regions. |
| US20190336097A1 (en) * | 2014-07-21 | 2019-11-07 | Zebra Medical Vision Ltd. | Systems and methods for prediction of osteoporotic fracture risk |
| CN107945278A (en) * | 2017-12-14 | 2018-04-20 | 桂林电子科技大学 | The three-dimensionalreconstruction and finite element method of a kind of organism structure |
| CN110570515A (en) * | 2019-09-03 | 2019-12-13 | 天津工业大学 | method for carrying out human skeleton three-dimensional modeling by utilizing CT (computed tomography) image |
Non-Patent Citations (1)
| Title |
|---|
| 吴良武等: "用邻域运算从CT图像中分割骨骼", 《中国生物医学工程学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565606A (en) * | 2022-03-23 | 2022-05-31 | 推想医疗科技股份有限公司 | Coronary artery segmentation method, coronary artery segmentation device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114037775B (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458830B (en) | Image processing method, image processing apparatus, server, and storage medium | |
| CN110176010B (en) | Image detection method, device, equipment and storage medium | |
| US20120281904A1 (en) | System and method for automatic recognition and labeling of anatomical structures and vessels in medical imaging scans | |
| CN111340827A (en) | Lung CT image data processing and analyzing method and system | |
| CN111640100A (en) | Tumor image processing method and device, electronic equipment and storage medium | |
| CN114066886B (en) | Bone segmentation boundary determining method and device, electronic equipment and storage medium | |
| CN110544245A (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN111145160B (en) | Method, device, server and medium for determining coronary artery branches where calcified regions are located | |
| CN113989407B (en) | Training method and system for limb part recognition model in CT image | |
| CN110706241A (en) | Three-dimensional focus area extraction method and device | |
| CN111640124A (en) | Blood vessel extraction method, device, equipment and storage medium | |
| CN111260636B (en) | Model training method and device, image processing method and device, and medium | |
| CN114037775B (en) | Bone structure growth method and device, electronic equipment and storage medium | |
| CN115439453A (en) | Vertebral body positioning method and device, electronic equipment and storage medium | |
| CN113888566B (en) | Target contour curve determination method and device, electronic equipment and storage medium | |
| CN118485667A (en) | Auxiliary diagnosis method and system based on ultrasonic image intellectualization | |
| CN112308823B (en) | Method and device for positioning region of interest in medical image | |
| CN116433695B (en) | Mammary gland region extraction method and system of mammary gland molybdenum target image | |
| CN111539926A (en) | Image detection method and device | |
| CN114066885B (en) | Lower limb skeleton model construction method and device, electronic equipment and storage medium | |
| CN114037719B (en) | Bone region determination method and device, electronic equipment and storage medium | |
| CN113192099B (en) | Tissue extraction method, device, equipment and medium | |
| CN114708283A (en) | Image object segmentation method and device, electronic equipment and storage medium | |
| CN109447974B (en) | Volume data processing method, volume data processing apparatus, image processing workstation, and readable storage medium | |
| CN113222989A (en) | Image grading method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |