WO2022247173A1 - 图像识别及模型训练的方法、关节位置识别的方法 - Google Patents

图像识别及模型训练的方法、关节位置识别的方法 Download PDF

Info

Publication number
WO2022247173A1
WO2022247173A1 PCT/CN2021/131966 CN2021131966W WO2022247173A1 WO 2022247173 A1 WO2022247173 A1 WO 2022247173A1 CN 2021131966 W CN2021131966 W CN 2021131966W WO 2022247173 A1 WO2022247173 A1 WO 2022247173A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
feature
model
image data
result
Prior art date
Application number
PCT/CN2021/131966
Other languages
English (en)
French (fr)
Inventor
张逸凌
刘星宇
Original Assignee
北京长木谷医疗科技有限公司
张逸凌
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京长木谷医疗科技有限公司, 张逸凌 filed Critical 北京长木谷医疗科技有限公司
Publication of WO2022247173A1 publication Critical patent/WO2022247173A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the invention relates to the technical field of image recognition, in particular to a method for image recognition and model training, and a method and device for joint position recognition.
  • the preoperative planning of total hip replacement surgery mainly includes the calculation of the required prosthesis type and the position of the osteotomy line.
  • the preoperative planning of total hip joint replacement surgery plays a very important role in the success rate of the operation.
  • the main The method of preoperative planning is to manually measure through various tools to determine the specific location of the key parts that need to be operated on. This method is inefficient and the accuracy cannot be guaranteed. Therefore, how to quickly and accurately identify the key points in the image Location is a burning issue.
  • embodiments of the present invention provide a method for image recognition and model training, and a method and device for joint position recognition, so as to solve the problem in the prior art that key positions in an image cannot be accurately recognized.
  • an embodiment of the present invention provides a training method for an image recognition model, including: acquiring an image data set, the image data set includes marked positive sample images and unmarked negative sample images, the positive The sample image contains a mark used to characterize the target area; the image data set is input to the first neural network model, the image is segmented, and the first output result is obtained; based on the first output result and the image data set, the The first neural network model is trained, the first neural network model is updated, and the image segmentation model is determined; a feature image is generated based on the first output result; the feature image is input to the second neural network model for multi-stage Image scaling processing to obtain a second output result; based on the second output result and the feature image, train the second neural network model, update the second neural network model, and determine a key point recognition model; The image segmentation model and the key point recognition model construct an image recognition model.
  • inputting the image data set into the first neural network model, performing image segmentation, and obtaining the first output result include: inputting the image data set into the first neural network model
  • the first image processing sub-model of the first neural network model performs image sampling processing on the image data in the image data set, and extracts the image features of the image data; the image data after the image features are extracted is input to the first
  • the second image processing sub-model of a neural network model performs image segmentation on the image features, and identifies the category to which the image features belong.
  • image sampling processing is performed on the image data in the image data set, and image features of the image data are extracted, including: performing Downsampling, identifying deep features of the image data; upsampling the downsampled image data, and storing the deep features in the image data.
  • image segmentation is performed on the image features, and the category to which the image features belong is identified, including: screening the preset reliability from the image features performing bilinear interpolation calculation on the feature point data; identifying the category of the image feature based on the calculated feature point data.
  • the feature image is input to the second neural network model, and multi-level image scaling processing is performed to obtain a second output result, including: performing multi-level processing on the feature image downsampling at each level to obtain the first feature image conforming to the preset resolution; respectively upsampling the first feature image at each level of downsampling to obtain the second feature image; based on the first feature image at each level of downsampling and the generating a composite feature image from the up-sampled second feature image; determining a probability that a key point in the feature image is in the composite feature image based on the composite feature image, as the second output result.
  • the first neural network model is trained based on the first output result and the image data set, and the first neural network model is updated.
  • a neural network model, determining an image segmentation model including: calculating a first loss function based on the first output result and an image data set; updating parameters of the first neural network model based on the first loss function, and determining the Image segmentation model.
  • the second neural network model based on the second output result and the feature image, is trained, and the A second neural network model, determining a key point recognition model, including: calculating a second loss function based on the second output result and the first output result; updating the second neural network model based on the second loss function parameters to determine the key point recognition model.
  • an embodiment of the present invention provides an image recognition method, including: acquiring an image to be recognized; performing image segmentation on the image to be recognized to obtain an image segmentation result; performing multi-level scaling processing on the image segmentation result , to obtain an image scaling result; identifying a target object in the image to be identified based on the image scaling result.
  • performing image segmentation on the image to be recognized to obtain an image segmentation result includes: inputting the image to be recognized into a preset image segmentation model, and performing an image segmentation on the image to be recognized The image to be recognized is segmented to obtain an image segmentation result; the preset image segmentation model is trained based on an image data set, and the image data set includes marked positive sample images and unmarked negative sample images, so The above positive sample images contain labels for characterizing target regions.
  • the image segmentation model includes a first image processing sub-model and a second image processing sub-model, and the image to be recognized is input to a preset An image segmentation model that performs image segmentation on the image to be recognized to obtain an image segmentation result, including: inputting the image to be recognized into a first image processing sub-model of the image segmentation model, and performing image segmentation on the image to be recognized
  • the image data is subjected to image sampling processing, and the image features of the image data are extracted; the image data after the image features are extracted is input to the second image processing sub-model of the image segmentation model, and the image features are image-segmented, and the identified The category to which the above image features belong.
  • image sampling processing is performed on the image data in the image to be recognized, and image features of the image data are extracted, including: performing an image sampling process on the image data Down-sampling is performed to identify deep features of the image data; up-sampling is performed on the down-sampled image data, and the deep features are stored in the image data.
  • performing image segmentation on the image features and identifying the category to which the image features belong includes: screening the preset reliability from the image features The feature point data, performing bilinear interpolation calculation on the feature point data; identifying the category based on the calculated feature point data.
  • performing multi-level scaling processing on the image segmentation result to obtain the image scaling result includes: generating a feature image based on the image segmentation result;
  • the feature image is input to a preset key point recognition model, and multi-level image zoom processing is performed to obtain an image zoom result;
  • the preset key point recognition model is obtained based on the training of the image data set, and the image data set It includes a marked positive sample image and an unmarked negative sample image, and the positive sample image contains a mark used to characterize the target region.
  • the feature image is input to a preset key point recognition model, and multi-level image scaling processing is performed to obtain an image scaling result, including: Perform multi-level downsampling on the above feature images to obtain the first feature image that meets the preset resolution; respectively up-sample the feature images of each level of downsampling to obtain the second feature image; based on the first feature image of each level of downsampling and the second feature image upsampled at each level to generate a composite feature image; determine the probability that the key point in the feature image is in the composite feature image based on the composite feature image, as the image scaling result.
  • an embodiment of the present invention provides a joint position recognition method, comprising: acquiring medical image data; performing image segmentation on the medical image data to obtain an image segmentation result including the femur and pelvic region; performing image segmentation on the image
  • the segmentation result is subjected to multi-level scaling processing, and the key positions of the joints in the medical image data are determined based on the image scaling result.
  • determining the key position of the joint in the medical image data based on the image scaling result includes: determining the key position of the hip joint based on the image scaling result; determining the healthy side based on the image scaling result The central point of the femoral head and the position of the lower edge of the teardrop; the mirror image of the central point of the femoral head and the lower edge of the teardrop on the healthy side is flipped to the target area, and the mirror image position of the central point of the femoral head and the lower edge of the teardrop are obtained on the affected side; based on The position of the core point is determined by the mirror image position of the lower edge of the teardrop on the affected side, and the height of the pelvis is calculated; based on the position of the core point and the height of the pelvis, the area including the position of the true acetabulum is determined.
  • performing image segmentation on the medical image data to obtain an image segmentation result including the femur and pelvic region includes: inputting the medical image data into a preset image A segmentation model is used to perform image segmentation on the medical image data to obtain an image segmentation result including the femur and pelvic region; the preset image segmentation model is trained based on a medical image data set, and the medical image data set includes A labeled positive sample image and an unlabeled negative sample image, the positive sample image contains markers for characterizing femur and pelvic regions.
  • performing multi-level scaling processing on the image segmentation result to obtain the image scaling result including the key position of the hip joint including: based on the image segmentation result Generate a feature image; input the feature image to a preset key point recognition model, perform multi-level image scaling processing, and obtain an image scaling result; the preset key point recognition model is obtained based on the training of the image data set , the image data set includes marked positive sample images and unmarked negative sample images, and the positive sample images contain marks used to characterize target regions.
  • an embodiment of the present invention provides a training device for an image recognition model, including: an image acquisition module, configured to acquire an image data set, the image data set includes marked positive sample images and unmarked negative samples.
  • the sample image, the positive sample image contains a mark for characterizing the target area;
  • the first output module is used to input the image data set into the first neural network model, perform image segmentation, and obtain the first output result;
  • image The segmentation model determination module is used to train the first neural network model based on the first output result and the image data set, update the first neural network model, and determine the image segmentation model;
  • the feature image generation module uses Generate a feature image based on the first output result;
  • the second output module is used to input the feature image to the second neural network model, perform multi-level image scaling processing, and obtain the second output result;
  • the key point recognition model is determined
  • a module configured to train the second neural network model based on the second output result and the feature image, update the second neural network model, and determine a key point recognition model;
  • an embodiment of the present invention provides an image recognition device, including: an image acquisition module, configured to acquire an image to be recognized; an image segmentation result generation module, configured to perform image segmentation on the image to be recognized to obtain an image Segmentation results; image scaling result generation module, used to perform multi-level scaling processing on the image segmentation results to obtain image scaling results; target object recognition module, used to identify targets in the image to be recognized based on the image scaling results object.
  • an embodiment of the present invention provides a joint position recognition device, including: a medical image data acquisition module, used to acquire medical image data; an image segmentation result generation module, used to perform image segmentation on the medical image data to obtain an image segmentation result including the femur and pelvic region; the joint position determination module is configured to perform multi-level scaling processing on the image segmentation result, and determine the key joint position in the medical image data based on the image scaling result.
  • a joint position identification device configured to determine the hip joint position based on the image scaling result Key position; the real socket position determination module is configured to determine the center point of the femoral head and the position of the lower edge of the teardrop on the healthy side based on the image scaling results; mirror the position of the center point of the femoral head and the lower edge of the teardrop on the healthy side to the target area , obtain the image position of the center point of the femoral head of the affected side and the image position of the lower edge of the teardrop; determine the position of the core point based on the mirror image position of the lower edge of the teardrop on the affected side, and calculate the height of the pelvis; The area where the true socket is located.
  • an embodiment of the present invention provides a computer device, including: a memory and a processor, the memory and the processor are connected to each other in communication, the memory stores computer instructions, and the processor By executing the computer instructions, the image recognition model training method described in the first aspect or any implementation manner of the first aspect is executed, or the second aspect or any implementation manner of the second aspect is executed The image recognition method, or, execute the third aspect or the joint position recognition method described in any implementation manner of the third aspect.
  • an embodiment of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute any of the first aspect or the first aspect.
  • the training method of the image recognition model described in one embodiment, or, execute the image recognition method described in the second aspect or any one of the embodiments of the second aspect, or execute the third aspect or the third aspect The joint position recognition method described in any one of the implementations.
  • the beneficial effects of the embodiments of the present invention are: through the method of image recognition and model training, the method and device of joint position recognition in this embodiment, by acquiring medical image data; performing image segmentation on the medical image data, based on the image segmentation results Determine the femur and pelvic region; perform multi-level scaling processing on the image segmentation results, determine the center point of the femoral head and the lower edge of the teardrop on the healthy side based on the image scaling results; determine the center point of the femoral head and the lower edge of the teardrop based on the healthy side
  • the position determines the true socket position.
  • This application can quickly eliminate the false acetabular interference of DDH developmental dislocation of the hip, quickly identify the true acetabular position, facilitate the operator to place the acetabular cup prosthesis at the true acetabular position, and improve the efficiency of preoperative planning.
  • FIG. 1 shows a schematic flow chart of an image recognition method according to an embodiment of the present invention
  • Fig. 2 shows a schematic flow chart of an image recognition method according to another embodiment of the present invention
  • Fig. 3 shows a schematic flow chart of an image recognition method according to another embodiment of the present invention.
  • FIG. 4 shows a schematic flow chart of a method for identifying a key position of a hip joint according to an embodiment of the present invention
  • Fig. 5A shows a schematic flowchart of a method for determining the position of a real socket according to an embodiment of the present invention
  • Figure 5B shows a schematic diagram of determining the position of the true socket based on the center point of the femoral head and the position of the lower edge of the teardrop according to an embodiment of the present invention
  • FIG. 6A shows a schematic flowchart of a method for training an image recognition model according to an embodiment of the present invention
  • Fig. 6B shows a schematic structural diagram of an image recognition model according to an embodiment of the present invention.
  • FIG. 7 shows a schematic structural diagram of a training device for an image recognition model according to an embodiment of the present invention.
  • FIG. 8 shows a schematic structural diagram of an image recognition device according to an embodiment of the present invention.
  • Fig. 9 shows a schematic structural diagram of a hip joint key position recognition device according to an embodiment of the present invention.
  • Fig. 10 shows a schematic structural diagram of a real mortar position recognition device according to an embodiment of the present invention
  • FIG. 11 shows a schematic structural diagram of a computer device according to an embodiment of the present invention.
  • DDH Developmental Dysplasia of the Hip
  • CROWE type III and IV high dislocations
  • pelvic and acetabular deformity, soft tissue contracture, muscle dysplasia, and abnormal bone reserve When these patients undergo artificial total hip replacement, the acetabular cup needs to be implanted into the real socket to correct the abnormal bone shape of the patient, which further increases the difficulty of the operation and affects the long-term survival rate of the acetabular cup prosthesis.
  • determining the position of the operation process is also obtained by identifying the image of the joint position, that is to say, in this
  • it is mainly how to identify the target position and target object from the image more quickly and accurately.
  • this embodiment provides an image recognition method, which can be used in electronic equipment, such as computers, mobile phones, tablet computers, etc., as shown in Figure 1, the image recognition method mainly includes:
  • Step S11 Acquire the image to be recognized.
  • the identification object is generally image data, which can be obtained by general image acquisition equipment, such as video cameras, cameras, mobile phones, tablet computers, etc.
  • general image acquisition equipment such as video cameras, cameras, mobile phones, tablet computers, etc.
  • it may be more professional image acquisition equipment, such as X-ray projection equipment, CT projection equipment, etc., and the present invention is not limited thereto.
  • Step S12 performing image segmentation on the image to be recognized to obtain an image segmentation result.
  • the recognition process for the image to be recognized is to realize a more rapid and accurate recognition of the target area or target position in the image. Therefore, in this embodiment, it is proposed to carry out two main process, thereby improving the recognition effect. First, image segmentation is performed on the image to be recognized to obtain an image segmentation result. Through the image segmentation process, the target area in the image to be recognized can be accurately identified.
  • the target area can be a pre-set area before recognition, or it can be combined with a large amount of image data, by marking the area to be recognized, and through repeated learning, training and other processes, in the form of features
  • the characterized region to be identified is not limited by the present invention.
  • Step S13 Perform multi-level scaling processing on the image segmentation result to obtain the image scaling result. After the target area in the image is identified through the image segmentation process, the image segmentation result is further scaled, and the scaling process is performed successively and step by step until the image is scaled to the preset resolution, so that it can be achieved in A target object is identified in the target area.
  • Step S14 Identify the target object in the image to be identified based on the image scaling result.
  • the target object may refer to a person or object in the image to be recognized, or a part with certain specific features in the image to be recognized, and the present invention is not limited thereto.
  • image segmentation is performed on the image to be recognized, thereby extracting the image features of the image to be recognized, so that the target area in the image can be more accurately identified; then, based on the result of image segmentation and the target area , perform multi-level scaling processing on the image, and identify the target object in the image based on the target area by scaling the image segmentation result to a preset resolution.
  • This embodiment provides an image recognition method, which can be used in electronic devices, such as computers, mobile phones, and tablet computers.
  • the image recognition model is used as an example to process the image to be recognized.
  • the image recognition method mainly includes:
  • Step S21 Acquiring the image to be recognized.
  • Step S22 performing image segmentation on the image to be recognized to obtain an image segmentation result.
  • this step S22 may include:
  • the image to be recognized is input into a preset image segmentation model, and the image to be recognized is segmented to obtain an image segmentation result.
  • the preset image segmentation model can be obtained by training based on an image data set, the image data set includes marked positive sample images and unmarked negative sample images, and the positive sample images contain marks used to characterize target regions.
  • the image data set is used as the input of the image segmentation model.
  • the specific training process of the image segmentation model will be described in detail in the embodiment of the training method of the image segmentation model below.
  • Step S23 Perform multi-level scaling processing on the image segmentation result to obtain the image scaling result.
  • Step S24 Identify the target object in the image to be identified based on the image scaling result.
  • the image recognition method of this embodiment uses the method of deep learning to process the image to be recognized. Since the image segmentation model based on deep learning has the ability of self-learning, the target area and target object in the image learned by the image segmentation model Performing recognition processing on the image to be recognized can further improve the recognition accuracy.
  • the image segmentation model includes a first image processing sub-model and a second image processing sub-model
  • the image data set is input into a preset image segmentation model
  • the image to be recognized is segmented , to get the image segmentation results, including:
  • Step S221 Input the image to be recognized into the first image processing sub-model of the image segmentation model, perform image sampling processing on the image data in the image to be recognized, and extract the image features of the image data;
  • Step S222 Input the image data after extracting the image features into the second image processing sub-model of the image segmentation model, perform image segmentation on the image features, and identify the category to which the image features belong.
  • the first image processing sub-model is a neural network model used to roughly segment the image to be recognized, such as a unet network, and the present invention is not limited thereto.
  • step S221 the process of sampling the image to be recognized based on the first image processing sub-model, and extracting the image features of the image data includes:
  • the downsampled image data is upsampled, and the deep features are stored in the image data.
  • the second image processing sub-model is a neural network model for subdividing the image to be recognized, such as a pointrend network, and the present invention is not limited thereto.
  • step S222 the process of performing image segmentation based on the image features of the image to be recognized based on the second image processing sub-model, and identifying the category to which the image features belong includes:
  • the category to which it belongs is identified based on the calculated feature point data.
  • the image to be recognized is subjected to rough segmentation and fine segmentation respectively.
  • the image features of the image through the fine segmentation process, perform multi-class recognition of pixels from the image features, so as to identify the content in the image according to the characteristics of the pixels, so that the segmentation of the image to be recognized can be improved according to the overall process. Accuracy.
  • This embodiment provides an image recognition method, which can be used in electronic devices, such as computers, mobile phones, and tablet computers.
  • the image recognition model is used as an example to process the image to be recognized.
  • the image recognition method mainly includes:
  • Step S31 Acquire the image to be recognized.
  • Step S32 performing image segmentation on the image to be recognized to obtain an image segmentation result.
  • Step S33 Perform multi-level scaling processing on the image segmentation result to obtain the image scaling result.
  • this step S33 may include:
  • the preset key point recognition model can be obtained based on image data set training, the image data set includes marked positive sample images and unmarked negative sample images, and the positive sample images contain marks used to characterize the target area .
  • the image data set is used as the input of the key point recognition model.
  • the specific training process of the key point recognition model will be described in detail in the embodiment of the training method of the key point recognition model below.
  • Step S34 Identify the target object in the image to be identified based on the image scaling result.
  • the image recognition method of this embodiment uses the method of deep learning to process the image to be recognized. Since the key point recognition model based on deep learning has the ability of self-learning, the target object in the image learned by the key point recognition model is treated as Recognizing the image and performing recognition processing can further improve the accuracy of recognition.
  • the key point recognition model is a neural network model for key point recognition of the image to be recognized, such as an hourglass network, and the present invention is not limited thereto.
  • the process of inputting the characteristic image into the preset key point recognition model, performing multi-level image scaling processing, and obtaining the image scaling result mainly includes:
  • Step S331 Perform multi-level down-sampling on the feature image to obtain the first feature image that meets the preset resolution; in this embodiment, the preset resolution may be the lowest resolution set according to the needs of the actual application scene;
  • Step S332 respectively upsampling the feature images downsampled at each level to obtain a second feature image
  • Step S333 Generate a synthetic feature image based on the first feature image of each level of downsampling and the second feature image of each level of upsampling; Features are combined to obtain the feature image;
  • Step S334 Determine the probability that the key point in the feature image is in the synthetic feature image based on the synthetic feature image, as the image scaling result.
  • the key point recognition model adopted integrates the image features extracted at each level of sampling during the multi-level sampling process of the image. Since the image features of each scale are considered, the The overall image processing process runs faster, making the training process for the key point recognition model faster, and the key point recognition process for the image can be completed more quickly.
  • This embodiment provides a method for identifying the key position of the hip joint, which can be applied to electronic devices, such as computers, mobile phones, and tablet computers, and can also be applied to specific fields, such as the medical field.
  • the processing of the image to be recognized by the image recognition model is taken as an example for illustration.
  • the hip joint key position recognition method mainly includes:
  • Step S41 Obtain medical image data; in this embodiment, the medical image data may be, for example, image data collected by X-ray projection equipment, CT projection equipment, etc., and the present invention is not limited thereto.
  • the specific process of acquiring medical image data in this step please refer to the description of S11 in the embodiment shown in FIG. 1 , which will not be repeated here.
  • Step S42 Segment the medical image data to obtain an image segmentation result including the femur and pelvis; in this embodiment, the target area is the position of the hip joint, and optionally, the femur and the pelvis.
  • the target area is the position of the hip joint, and optionally, the femur and the pelvis.
  • Step S43 Perform multi-level scaling processing on the image segmentation result to obtain an image scaling result including the key position of the hip joint;
  • Step S44 Identify the key position of the hip joint in the medical image data based on the image scaling result.
  • image segmentation is performed on medical image data, thereby extracting the image features of the medical image data, so that the femur and pelvic region in the image can be more accurately identified; then, based on the image segmentation
  • multi-level zoom processing is performed on the image, and the target object in the image is identified based on the target area (for example, the femur and pelvis area) by scaling the image segmentation result to a preset resolution (in this paper In the embodiment, it refers to the central point of the femoral head and the position of the lower edge of the teardrop).
  • the process of performing image segmentation on the medical image data to obtain the image segmentation results including the femur and pelvic region can be realized by processing the image segmentation model, mainly include:
  • the dataset includes labeled positive images and unlabeled negative images, which contain markers for femur and pelvic regions.
  • the image to be recognized is subjected to rough segmentation and fine segmentation respectively.
  • the image features of the image, through the fine segmentation process, the loudness points are extracted from the image features, so that the content in the image can be identified according to the characteristics of the pixels, so that the segmentation of the image to be recognized can be improved according to the overall process the accuracy.
  • the process of performing multi-level scaling processing on the image segmentation result to obtain the image scaling result including the key position of the hip joint can be realized by processing the key point recognition model , mainly including:
  • the preset key point recognition model Input the feature image to the preset key point recognition model, perform multi-level image scaling processing, and obtain the image scaling result; the preset key point recognition model is trained based on the image data set, which includes marked positive samples image and the unlabeled negative image, the positive image contains the markers used to characterize the target region.
  • the key point recognition model adopted integrates the image features extracted at each level of sampling during the multi-level sampling process of the image. Since the image features of each scale are considered, the The overall image processing process runs faster, so that the training process for the key point recognition model is faster, and the key point recognition process for the image can be completed more quickly.
  • This embodiment provides a joint position recognition method, which can be applied to electronic devices, such as computers, mobile phones, and tablet computers, and can also be applied to specific fields, such as the medical field.
  • the joint position recognition method includes:
  • Step S51 acquiring medical image data
  • Step S52 performing image segmentation on the medical image data, and determining the femur and pelvis regions based on the image segmentation results;
  • Step S53 Perform multi-level scaling processing on the image segmentation results, and determine the center point of the femoral head and the position of the lower edge of the teardrop on the healthy side based on the image scaling results; in this embodiment, the target object is the center point of the femoral head on the healthy side and the position of the lower edge of the teardrop.
  • the central point of the femoral head and the position of the lower edge of the teardrop as an example, for details of this step, please refer to the description of S43 in the embodiment shown in FIG. 4 , which will not be repeated here.
  • Step S54 Determine the true acetabular position based on the center point of the femoral head of the healthy side and the lower edge of the teardrop.
  • the true socket position can be determined according to the specific positions.
  • the joint position identification method is specifically applied in a scene of preoperative planning of hip joint surgery.
  • Hip surgery is mainly performed on the position of the acetabulum on the patient's side, but in practice, due to the long-term wear and tear on the position of the acetabulum on the patient's side, it is impossible to accurately determine the actual acetabular position of the patient (that is, true mortar position). Therefore, in this embodiment, for this situation, the true acetabular position of the patient's diseased side is firstly determined according to the acetabular position of the patient's healthy side.
  • the process of determining the position of the true acetabular position based on the center point of the femoral head of the healthy side and the position of the lower edge of the teardrop mainly includes:
  • the area containing the position of the true socket is determined. From point M, make a vertical line L1 upwards (towards the teardrop position in the pelvis). The length of this line can be 20% of the pelvic height H, and then continue to make a vertical line to the outside of the pelvis (away from the teardrop position) The horizontal line L2, the length of L2 can also be 20% of the pelvic height H, then it can be determined at this time that the true acetabular position on the patient side is in the area surrounded by L1 and L2.
  • the joint position recognition method of this embodiment on the basis of the image recognition method and the hip joint key position recognition method of the foregoing embodiments, first determines the position of the acetabulum on the healthy side of the hip joint, and then determines the position of the acetabulum based on the position of the healthy side
  • the true acetabular position on the patient’s side can be identified based on deep learning and medical image data during the entire identification process, which not only improves the identification efficiency, but also improves the identification accuracy, and provides more accurate information for subsequent hip joint-related operations. technical support.
  • an embodiment of a training method for an image recognition model is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and , although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
  • This embodiment provides a training method for an image recognition model, which can be used in electronic devices, such as computers, mobile phones, tablet computers, etc., as shown in Figure 6A, the training method mainly includes:
  • Step S61 Obtain an image data set, which includes marked positive sample images and unmarked negative sample images, and the positive sample images contain marks used to characterize the target area; in this embodiment, the image data set can be It is collected by general image acquisition equipment, such as cameras, mobile phones, tablet computers, etc.; or it can also be acquired by relatively professional image acquisition equipment, such as X-ray projection equipment commonly used in the medical field, CT projection equipment, etc., the present invention is not limited thereto.
  • the image data set can be a CT medical image data set.
  • the femur and pelvic region are marked, so as to serve as a database for training the neural network model.
  • the image data set in the database can be divided into a training set and a test set according to a certain ratio (for example, 7:3). Convert the collected two-dimensional cross-sectional DICOM data into JPG format images, convert the annotation files into png format images, and save them as the input of the neural network model.
  • Step S62 Input the image data set into the first neural network model, perform image segmentation, and obtain the first output result;
  • the first neural network model is a neural network model for segmenting images, which may be composed of a first image processing sub-model and a second image processing sub-model.
  • the process of inputting the image data set into the first neural network model, performing image segmentation, and obtaining the first output result includes:
  • Step S621 Input the image data set into the first image processing sub-model of the image segmentation model, perform image sampling processing on the image data in the image data set, and extract the image features of the image data;
  • Step S622 Input the image data after extracting the image features into the second image processing sub-model of the image segmentation model, perform image segmentation on the image features, and identify the category to which the image features belong.
  • the first image processing sub-model is a neural network model used to roughly segment the image to be recognized, such as a unet network, and the present invention is not limited thereto.
  • step S621 the process of sampling the image to be recognized based on the first image processing sub-model, and extracting the image features of the image data includes:
  • the downsampled image data is upsampled, and the deep features are stored in the image data.
  • the first image processing sub-model is the unet network as an example for illustration.
  • the unet network is used as the backbone network to roughly segment the image data in the image data set.
  • 4 times of downsampling are used to learn the deep features of the image, and then 4 times of upsampling is performed to convert the feature map Save back to the image.
  • each downsampling layer includes 2 convolutional layers and 1 pooling layer, the convolution kernel size is 3*3, the convolution kernel size in the pooling layer is 2*2, and each convolution layer The number of convolution kernels is 128, 256, 512; each upsampling layer includes 1 upsampling layer and 2 convolution layers, where the convolution kernel size of the convolution layer is 3*2, and the upsampling layer The size of the convolution kernel in is 2*2, and the number of convolution kernels in each upsampling layer is 512, 256, 128. After the last upsampling, there is a dropout layer, and the dropout rate is set to 0.7.
  • the second image processing sub-model is a neural network model for subdividing the image to be recognized, such as a pointrend network, and the present invention is not limited thereto.
  • step S622 the process of performing image segmentation based on the image features of the image to be recognized based on the second image processing sub-model, and identifying the category to which the image features belong includes:
  • the category to which it belongs is identified based on the calculated feature point data.
  • the second image processing sub-model is a pointrend network as an example for illustration.
  • Step S63 Based on the first output result and the image data set, train the first neural network model, update the first neural network model, and determine the image segmentation model.
  • the parameters that can be set are: the background pixel value of the data label can be set to 0, the femur to 1, the pelvis to 2, the training batch_size to 6, the learning rate to 1e-4, and the optimizer to use Adam
  • the optimizer uses DICE loss as the loss function, inputs the training set into the first neural network for training, adjusts the size of the training batch according to the change of the loss function during the training process, and finally obtains the rough segmentation results of each part.
  • After entering the pointrend network first use bilinear interpolation to upsample the prediction results of the previous step, and then select the N most uncertain points in this denser feature map, such as points with a probability close to 0.5.
  • Step S64 Generate a feature image based on the first output result.
  • the first output result output by the first neural network model is reconstructed into an orthographic projection image as its corresponding feature image.
  • Step S65 Input the feature image into the second neural network model, perform multi-level image scaling processing, and obtain a second output result.
  • the key point recognition model is a neural network model for key point recognition of the image to be recognized, such as an hourglass network, and the present invention is not limited thereto.
  • the process of inputting the characteristic image into the preset key point recognition model, performing multi-level image scaling processing, and obtaining the image scaling result mainly includes:
  • Step S651 Perform multi-level down-sampling on the feature image to obtain the first feature image that meets the preset resolution; in this embodiment, the preset resolution may be the lowest resolution set according to the needs of the actual application scene;
  • Step S652 respectively upsampling the feature images downsampled at each level to obtain a second feature image
  • Step S653 Generate a synthetic feature image based on the first feature image of each level of downsampling and the second feature image of each level of upsampling; Features are combined to obtain the feature image;
  • Step S654 Based on the synthetic feature image, determine the probability that the key point in the feature image is in the synthetic feature image, as an image scaling result.
  • the second neural network model is an hourglass network as an example for illustration.
  • the Conv layer and Max Pooling layer of the hourglass network are used to scale the feature image to a preset standard resolution, and at each downsampling, the hourglass network simultaneously saves the feature image of the original size, and The pre-pooled resolution features are convolved, and after the lowest resolution features are obtained, the network starts upsampling (upsampling), and gradually combines feature information of different scales.
  • a nearest neighbor upsampling method is used for upsampling the feature image with a lower resolution, and two different feature sets are added element by element.
  • each network layer in the process of obtaining low-resolution features there will be a corresponding network layer in the process of upsampling.
  • two A continuous 1*1Conv layer is processed to obtain the final network output, and the output is a set of heatmaps, and each heatmap represents the probability of key points existing in each pixel.
  • the Hourglass network separates the upper half to retain the original scale information; after each upsampling, it is added to the data of the previous scale; between two downsamplings, three residual modules can be used Extract features; between two additions, use a residual module to extract features. Since the features of each scale are considered, the running speed is faster and the network training time is faster.
  • Step S66 Based on the second output result and the feature image, train the second neural network model, update the second neural network model, and determine the key point recognition model;
  • the parameters can be set to: the input pixel value is an orthographic image of 0-255 and label.txt, and the coordinates of the points corresponding to each other can be found by the name of each picture;
  • these points can be generated into a Gaussian map and supervised by heatmap, that is, the output of the network is a feature map of the same size as the input, with 1 at the position of the detection point and 0 at other positions.
  • feature maps of multiple channels can be output.
  • the network is optimized by Adam, the learning rate is 1e-5, the batch_size is 4, and the loss function is regularized by L2. According to the change of the loss function during the training process, the size of the training batch is adjusted, and finally the coordinate position of the target object is obtained.
  • Step S67 Construct an image recognition model based on the image segmentation model and the key point recognition model.
  • an image recognition model for the target object can be trained.
  • the image recognition model trained by the image recognition model training method of this embodiment can perform image segmentation on the image to be recognized during the actual image recognition process, thereby extracting the image features of the image to be recognized, so that it can be more accurate.
  • multi-level zoom processing is performed on the image, and the target in the image is identified based on the target area by scaling the image segmentation result to a preset resolution object.
  • This embodiment also provides a training device for an image recognition model, as shown in FIG. 7 , including:
  • the image acquisition module 101 is used to acquire an image data set, the image data set includes marked positive sample images and unmarked negative sample images, and the positive sample images contain marks for characterizing the target area; for details, please refer to Refer to the description of S61 in the foregoing method embodiment, and details are not repeated here.
  • the first output module 102 is configured to input the image data set into the first neural network model, perform image segmentation, and obtain the first output result; for details, please refer to the description of S62 in the above method embodiment, and will not be repeated here. .
  • the image segmentation model determination module 103 is used to train the first neural network model based on the first output result and the image data set, update the first neural network model, and determine the image segmentation model; for details, please refer to The description of S63 in the above method embodiment will not be repeated here.
  • the feature image generation module 104 is configured to generate a feature image based on the first output result; for details, please refer to the description of S64 in the above method embodiment, and details are not repeated here.
  • the second output module 105 is configured to input the feature image to the second neural network model, perform multi-level image scaling processing, and obtain the second output result; for details, please refer to the description of S65 in the above method embodiment, which will not be described here. Let me repeat.
  • a key point recognition model determination module 106 configured to train the second neural network model based on the second output result and the feature image, update the second neural network model, and determine a key point recognition model; detail
  • S66 the description of S66 in the foregoing method embodiment, and details are not repeated here.
  • An image recognition model building module 107 configured to build an image recognition model based on the image segmentation model and the key point recognition model. For details, refer to the description of S67 in the foregoing method embodiment, and details are not repeated here.
  • the image recognition model trained by the image recognition model training device of this embodiment can perform image segmentation on the image to be recognized during the actual image recognition process, so as to extract the image features of the image to be recognized, so that it can be more accurate Then, based on the result of image segmentation and the target area, multi-level zoom processing is performed on the image, and the target in the image is identified based on the target area by scaling the image segmentation result to a preset resolution object.
  • the features of the target area and the target object can be extracted more quickly and accurately, thereby obtaining a more accurate image recognition result.
  • This embodiment also provides an image recognition device, as shown in FIG. 8 , including:
  • the image acquisition module 201 is configured to acquire the image to be recognized; for details, please refer to the description of S11 in the above method embodiment, and details are not repeated here.
  • the image segmentation result generating module 202 is configured to perform image segmentation on the image to be recognized to obtain an image segmentation result; for details, please refer to the description of S12 in the above method embodiment, which will not be repeated here.
  • the image scaling result generation module 203 is configured to perform multi-level scaling processing on the image segmentation result to obtain an image scaling result; for details, please refer to the description of S13 in the above method embodiment, and details are not repeated here.
  • a target object identification module 204 configured to identify the target object in the image to be identified based on the image scaling result. For details, refer to the description of S14 in the foregoing method embodiment, and details are not repeated here.
  • image segmentation is performed on the image to be recognized, thereby extracting the image features of the image to be recognized, so that the target area in the image can be more accurately identified; then, based on the result of image segmentation and the target area , perform multi-level scaling processing on the image, and identify the target object in the image based on the target area by scaling the image segmentation result to a preset resolution.
  • This embodiment also provides a hip joint key position identification device, as shown in Figure 9, including:
  • the medical image data acquisition module 301 is configured to acquire medical image data; for details, please refer to the description of S41 in the above method embodiment, and details are not repeated here.
  • the image segmentation result generating module 302 is configured to perform image segmentation on the medical image data to obtain an image segmentation result including the femur and pelvic region; for details, please refer to the description of S42 in the above method embodiment, and details will not be repeated here.
  • the image scaling result generating module 303 is configured to perform multi-level scaling processing on the image segmentation result to obtain the image scaling result including the key position of the hip joint; for details, please refer to the description of S43 in the above method embodiment, and will not be repeated here .
  • the hip joint key position identification module 304 is configured to identify the hip joint key position in the medical image data based on the image scaling result. For details, refer to the description of S44 in the foregoing method embodiment, and details are not repeated here.
  • image segmentation is performed on medical image data, thereby extracting the image features of the medical image data, so that the femur and pelvic region in the image can be more accurately identified; then, based on the image segmentation
  • multi-level zoom processing is performed on the image, and the target object in the image is identified based on the target area by scaling the image segmentation result to a preset resolution (in this embodiment, it refers to the femoral head center point and lower edge of the teardrop).
  • This embodiment also provides a joint position recognition device, as shown in Figure 10, including:
  • the medical image data acquisition module 401 is configured to acquire medical image data; for details, please refer to the description of S51 in the above method embodiment, which will not be repeated here.
  • the femur and pelvis region determining module 402 is configured to perform image segmentation on the medical image data, and determine the femur and pelvis region based on the image segmentation results; for details, please refer to the description of S52 in the above method embodiment, and details will not be repeated here.
  • Hip key position identification module 403 configured to perform multi-level scaling processing on the image segmentation result, and determine the position of the center point of the femoral head and the lower edge of the teardrop on the healthy side based on the image scaling result; for details, please refer to S53 of the above method embodiment description and will not be repeated here.
  • the true acetabular position determination module 404 is configured to determine the true acetabular position based on the central point of the femoral head and the lower edge of the tear drop on the healthy side. For details, refer to the description of S54 in the foregoing method embodiment, and details are not repeated here.
  • the joint position recognition device of this embodiment on the basis of the image recognition method and the hip joint key position recognition method of the foregoing embodiments, first determines the position of the acetabulum on the healthy side of the hip joint, and then determines the position of the acetabulum based on the position of the healthy side by mirror image
  • the true acetabular position on the patient’s side can be identified based on deep learning and medical image data during the entire identification process, which not only improves the identification efficiency, but also improves the identification accuracy, and provides more accurate information for subsequent hip joint-related operations. technical support.
  • the embodiment of the present invention also provides a computer device.
  • the computer device may include a processor 111 and a memory 112, wherein the processor 111 and the memory 112 may be connected through a bus or in other ways.
  • FIG. Take the bus connection as an example.
  • the processor 111 may be a central processing unit (Central Processing Unit, CPU).
  • Processor 111 can also be other general processors, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or Other chips such as programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above-mentioned types of chips.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • Other chips such as programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above-mentioned types of chips.
  • the memory 112 can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the image recognition method in the embodiment of the present invention, or the hip joint key position recognition method, or a real mortar recognition method, or a program instruction/module corresponding to a training method of an image recognition model.
  • the processor 111 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 112, that is, realizes the image recognition method in the above method embodiment, or the hip joint key A position recognition method, or a real mortar recognition method, or a training method of an image recognition model.
  • the memory 112 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created by the processor 111 and the like.
  • the memory 112 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices.
  • the memory 112 may optionally include a memory that is remotely located relative to the processor 111, and these remote memories may be connected to the processor 111 through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the one or more modules are stored in the memory 112, and when executed by the processor 111, execute the image recognition method in the embodiment shown in Figures 1-6B, or the hip joint key position recognition method, Or a real mortar recognition method, or a training method for an image recognition model.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开一种图像识别及模型训练的方法、关节位置识别的方法及装置，该真臼位置识别方法包括：获取医学图像数据；对所述医学图像数据进行图像分割，基于图像分割结果确定股骨及骨盆区域；对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；基于所述健侧的股骨头中心点和泪滴下缘位置确定真臼位置。本申请可以快速排除DDH发育性髋关节脱位的假臼干扰，快速识别真臼位置，方便术者将髋臼杯假体安放在真臼位置，提高术前规划效率。

Description

图像识别及模型训练的方法、关节位置识别的方法
相关申请的交叉引用
本申请要求在2021年05月26日提交中国专利局、申请号为CN202110580663.5、发明名称为“图像识别及模型训练的方法、真臼位置识别的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及图像识别技术领域，具体涉及一种图像识别及模型训练的方法、关节位置识别的方法及装置。
背景技术
在医学领域中全髋关节置换手术的术前规划主要包括计算所需假体型号及截骨线位置，全髋关节置换手术的术前规划对于手术的成功率起着非常重要的作用，目前主要的术前规划方式为人工通过各种工具进行测量，来确定需要进行手术的关键部位的具体位置，这样的方式效率低而且准确性无法保证，因此，如何能够迅速并精确地识别图像中的关键位置是亟待解决的问题。
发明内容
有鉴于此，本发明实施例提供了一种图像识别及模型训练的方法、关节位置识别的方法及装置，以解决现有技术中无法精确识别图像中的关键位置的问题。
根据第一方面，本发明实施例提供了一种图像识别模型的训练方法，包括：获取图像数据集，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记；将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型；基于所述第一输出结果生成特征图像；将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果；基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型；基于所述图像分割模型及关键点识别模型构建图像识别模型。
结合第一方面，在第一方面第一实施方式中，将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果，包括：将所述图像数据集输入至所述第一神经网络模型的第一图像处理子模型，对所述图像数据集中的图像数据进行图像采样处理，提取所述图像数据的图像特征；将提取图像特征后的图像数据输入至所述第一神经网络模型的第二图像处理子模型，对所述图像特征进行图像分割，识别所述图像特征的所属类别。
结合第一方面第一实施方式，在第一方面第二实施方式中，对所述图像数据集中的图像数据进行图像采样处理，提取所述图像数据的图像特征，包括：对所述图像数据进行下采样，识别所述图像数据的深层特征；对进行下采样后的图像数据进行上采样，将所述深层特征存储到所述图像数据中。
结合第一方面第一实施方式，在第一方面第三实施方式中，对所述图像特征进行图像分割，识别所述图像特征的所属类别，包括：从所述图像特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算；基于计算后的特征点数据识别图像特征的所属类别。
结合第一方面，在第一方面第四实施方式中，将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果，包括：对所述特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；分别对各级下采样的第一特征图像进行上采样，得到第二特征图像；基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；基于所述合成特征图像确定所述特征图像中关键点处于所述合成特征图像中的概率，作为所述第二输出结果。
结合第一方面或第一方面任一实施方式，在第一方面第五实施方式中，基于所述第一输出结 果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型，包括：基于所述第一输出结果及图像数据集计算第一损失函数；基于所述第一损失函数更新所述第一神经网络模型的参数，确定所述图像分割模型。
结合第一方面或第一方面任一实施方式，在第一方面第六实施方式中，基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型，包括：基于所述第二输出结果及所述第一输出结果计算第二损失函数；基于所述第二损失函数更新所述第二神经网络模型的参数，确定所述关键点识别模型。
根据第二方面，本发明实施例提供了一种图像识别方法，包括：获取待识别图像；对所述待识别图像进行图像分割，得到图像分割结果；对所述图像分割结果进行多级缩放处理，得到图像缩放结果；基于所述图像缩放结果识别所述待识别图像中的目标对象。
结合第二方面，在第二方面第一实施方式中，对所述待识别图像进行图像分割，得到图像分割结果，包括：将所述待识别图像输入至预设的图像分割模型，对所述待识别图像进行图像分割，得到图像分割结果；所述预设的图像分割模型是基于图像数据集训练得到的，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记。
结合第二方面第一实施方式，在第二方面第二实施方式中，所述图像分割模型包括第一图像处理子模型及第二图像处理子模型，将所述待识别图像输入至预设的图像分割模型，对所述待识别图像进行图像分割，得到图像分割结果，包括：将所述待识别图像输入至所述图像分割模型的第一图像处理子模型，对所述待识别图像中的图像数据进行图像采样处理，提取所述图像数据的图像特征；将提取图像特征后的图像数据输入至所述图像分割模型的第二图像处理子模型，对所述图像特征进行图像分割，识别所述图像特征的所属类别。
结合第二方面第二实施方式，在第二方面第三实施方式中，对所述待识别图像中的图像数据进行图像采样处理，提取所述图像数据的图像特征，包括：对所述图像数据进行下采样，识别所述图像数据的深层特征；对进行下采样后的图像数据进行上采样，将所述深层特征存储到所述图像数据中。
结合第二方面第二实施方式，在第二方面第四实施方式中，对所述图像特征进行图像分割，识别所述图像特征的所属类别，包括：从所述图像特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算；基于计算后的特征点数据识别所属类别。
结合第二方面第一实施方式，在第二方面第五实施方式中，对所述图像分割结果进行多级缩放处理，得到图像缩放结果，包括：基于所述图像分割结果生成特征图像；将所述特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果；所述预设的关键点识别模型是基于所述图像数据集训练得到的，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记。
结合第二方面第五实施方式，在第二方面第六实施方式中，将所述特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果，包括：对所述特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；分别对各级下采样的特征图像进行上采样，得到第二特征图像；基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；基于所述合成特征图像确定所述特征图像中关键点处于所述合成特征图像中的概率，作为所述图像缩放结果。
根据第三方面，本发明实施例提供了一种关节位置识别方法，包括：获取医学图像数据；对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定所述医学图像数据中的关节关键位置。
结合第三方面，在第三方面第一实施方式中，基于图像缩放结果确定所述医学图像数据中的关节关键位置，包括：基于图像缩放结果确定髋关节关键位置；基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；将所述健侧的股骨头中心点和泪滴下缘位置镜像翻转至目标区域， 得到患侧的股骨头中心点镜像位置和泪滴下缘镜像位置；基于所述患侧的泪滴下缘镜像位置确定核心点位置，并计算骨盆高度；基于所述核心点位置及骨盆高度确定包含真臼位置的区域。
结合第三方面，在第三方面第二实施方式中，对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果，包括：将所述医学图像数据输入至预设的图像分割模型，对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；所述预设的图像分割模型是基于医学图像数据集训练得到的，所述医学图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征股骨及骨盆区域的标记。
结合第三方面第二实施方式，在第三方面第三实施方式中，对所述图像分割结果进行多级缩放处理，得到包含髋关节关键位置的图像缩放结果，包括：基于所述图像分割结果生成特征图像；将所述特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果；所述预设的关键点识别模型是基于所述图像数据集训练得到的，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记。
根据第四方面，本发明实施例提供了一种图像识别模型的训练装置，包括：图像获取模块，用于获取图像数据集，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记；第一输出模块，用于将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；图像分割模型确定模块，用于基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型；特征图像生成模块，用于基于所述第一输出结果生成特征图像；第二输出模块，用于将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果；关键点识别模型确定模块，用于基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型；图像识别模型构建模块，用于基于所述图像分割模型及关键点识别模型构建图像识别模型。
根据第五方面，本发明实施例提供了一种图像识别装置，包括：图像获取模块，用于获取待识别图像；图像分割结果生成模块，用于对所述待识别图像进行图像分割，得到图像分割结果；图像缩放结果生成模块，用于对所述图像分割结果进行多级缩放处理，得到图像缩放结果；目标对象识别模块，用于基于所述图像缩放结果识别所述待识别图像中的目标对象。
根据第六方面，本发明实施例提供了一种关节位置识别装置，包括：医学图像数据获取模块，用于获取医学图像数据；图像分割结果生成模块，用于对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；关节位置确定模块，用于对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定所述医学图像数据中的关节关键位置。
结合第六方面，在第六方面第一实施方式中，提供了一种关节位置识别装置，其中，关节位置确定模块，包括：髋关节关键位置识别模块，被配置为基于图像缩放结果确定髋关节关键位置；真臼位置确定模块，被配置为基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；将所述健侧的股骨头中心点和泪滴下缘位置镜像翻转至目标区域，得到患侧的股骨头中心点镜像位置和泪滴下缘镜像位置；基于所述患侧的泪滴下缘镜像位置确定核心点位置，并计算骨盆高度；基于所述核心点位置及骨盆高度确定包含所述真臼位置的区域。
根据第七方面，本发明实施例提供了一种计算机设备，包括：存储器和处理器，所述存储器和所述处理器之间互相通信连接，所述存储器中存储有计算机指令，所述处理器通过执行所述计算机指令，从而执行第一方面或者第一方面的任意一种实施方式中所述的图像识别模型的训练方法，或，执行第二方面或者第二方面的任意一种实施方式中所述的图像识别方法，或，执行第三方面或者第三方面的任意一种实施方式中所述的关节位置识别方法。
根据第八方面，本发明实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行第一方面或者第一方面的任意一种实施方式中所述的图像识别模型的训练方法，或，执行第二方面或者第二方面的任意一种实施方式中所述的图像识别方法，或，执行第三方面或者第三方面的任意一种实施方式中所述的关节位置识别 方法。
本发明实施例的有益效果在于：通过本实施例的图像识别及模型训练的方法、关节位置识别的方法及装置，通过获取医学图像数据；对所述医学图像数据进行图像分割，基于图像分割结果确定股骨及骨盆区域；对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；基于所述健侧的股骨头中心点和泪滴下缘位置确定真臼位置。本申请可以快速排除DDH发育性髋关节脱位的假臼干扰，快速识别真臼位置，方便术者将髋臼杯假体安放在真臼位置，提高术前规划效率。
针对待识别图像进行图像分割，从而提取该待识别图像的图像特征，从而能够更加准确地识别该图像中的目标区域；然后，基于图像分割的结果及目标区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分辨率，从而基于目标区域识别该图像中的目标对象。通过上述过程，对于待识别图像的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
附图说明
通过参考附图会更加清楚的理解本发明的特征和优点，附图是示意性的而不应理解为对本发明进行任何限制，在附图中：
图1示出了本发明实施例的图像识别方法的流程示意图；
图2示出了本发明另一实施例的图像识别方法的流程示意图；
图3示出了本发明另一实施例的图像识别方法的流程示意图；
图4示出了本发明实施例的髋关节关键位置识别方法的流程示意图；
图5A示出了本发明实施例的真臼位置确定方法的流程示意图；
图5B示出了本发明实施例的基于股骨头中心点和泪滴下缘位置确定真臼位置的示意图；
图6A示出了本发明实施例的图像识别模型的训练方法的流程示意图；
图6B示出了本发明实施例的图像识别模型的结构示意图；
图7示出了本发明实施例的图像识别模型的训练装置的结构示意图；
图8示出了本发明实施例的图像识别装置的结构示意图；
图9示出了本发明实施例的髋关节关键位置识别装置的结构示意图；
图10示出了本发明实施例的真臼位置识别装置的结构示意图；
图11示出了本发明实施例的计算机设备的结构示意图。
具体实施方式
发育性髋关节脱位(Developmental Dysplasia of the Hip，DDH),是由于遗传、臀位产等因素造成股骨头与髋臼对位不良的一种疾病,在医学上以往称为“先天性髋关节发育不良。对于DDH患者，行人工全髋关节置换技术要求高，尤其是CROWE分型III、IV型高位脱位患者，因其骨盆及髋臼畸形、软组织挛缩、肌肉发育不良、骨量储备异常，对此类患者行人工全髋关节置换术时需将髋臼杯植入真臼内，以纠正患者骨骼形态异常的问题，从而进一步增加了手术困难，影响髋臼杯假体远期生存率。因此，如何能够准确地对手术过程中各个位置进行识别、确认是亟待解决的问题。而针对该问题，确定手术过程的位置，也是针对关节位置的图像进行识别而获取的，也就是说，在本实施例中，为了解决上述问题，主要是如何能够实现更加快速、精准地从图像中识别目标位置、目标对象。从而实现可以快速排除DDH发育性髋关节发育异常的假臼干扰，快速识别真臼位置，方便术者将髋臼杯假体安放在真臼位置，提高术前规划效率。
为此，本实施例提供了一种图像识别方法，可用于电子设备，如电脑、手机、平板电脑等，如图1所示，该图像识别方法主要包括：
步骤S11：获取待识别图像。在本实施例中，所针对的识别对象一般是图像数据，该图像数据可以是通过一般的图像采集设备所获取的，例如是摄像机、照相机、手机、平板电脑等，此外，针对不同领域，也可以是较为专业的图像采集设备，例如是X射线投影设备、CT投影设备等，本发明并不以此为限。
步骤S12：对待识别图像进行图像分割，得到图像分割结果。在本实施例中，针对待识别图像进行的识别过程，是为了实现能够更加快速、精确地识别图像中的目标区域或目标位置，因此，在本实施例中，提出了对图像进行两个主要的处理过程，从而提高识别效果。首先，则是对待识别图像进行图像分割，从而得到图像分割结果，通过该图像分割过程，能够对待识别图像中的目标区域进行准确地识别。
在本实施例中，该目标区域可以是在识别之前，提前设定好的区域，也可以是结合大量的图像数据，通过标记要识别的区域，通过反复学习、训练等过程，以特征的形式所表征的要识别的区域，本发明并不以此为限。
步骤S13：对图像分割结果进行多级缩放处理，得到图像缩放结果。在通过图像分割处理识别到图像中的目标区域后，对图像分割结果做进一步的缩放处理，并且是逐次、逐级地进行缩放处理，直到缩放到预设的分辨率的图像，从而能够实现在目标区域中识别出目标对象。
步骤S14：基于图像缩放结果识别待识别图像中的目标对象。在本实施例中，该目标对象可以是指待识别图像中的人、物，或者是待识别图像中具有某些特定特征的部分等，本发明并不以此为限。通过上述步骤，在识别了待识别图像中的目标区域后，基于该目标区域，通过缩放处理，突出待识别图像中的目标对象的位置，从而实现准确地识别目标对象。
通过本实施例的图像识别方法，针对待识别图像进行图像分割，从而提取该待识别图像的图像特征，从而能够更加准确地识别该图像中的目标区域；然后，基于图像分割的结果及目标区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域识别该图像中的目标对象。通过上述过程，对于待识别图像的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
本实施例提供了一种图像识别方法，可用于电子设备，如电脑、手机、平板电脑等。在本实施例中，以图像识别模型对待识别图像进行处理为例进行说明，如图2所示，该图像识别方法主要包括：
步骤S21：获取待识别图像。
详细内容请参见图1所示实施例的S11的描述，在此不再赘述。
步骤S22：对待识别图像进行图像分割，得到图像分割结果。
具体地，该步骤S22可以包括：
将待识别图像输入至预设的图像分割模型，对待识别图像进行图像分割，得到图像分割结果。
其中，该预设的图像分割模型可以是基于图像数据集训练得到的，图像数据集中包括已标记的正样本图像及未标记的负样本图像，正样本图像中含有用于表征目标区域的标记。
可选地，在图像分割模型的训练过程中，利用图像数据集作为图像分割模型的输入进行。其中，关于图像分割模型的具体训练过程将在下文的图像分割模型的训练方法实施例中做详细描述。
步骤S23：对图像分割结果进行多级缩放处理，得到图像缩放结果。
详细内容请参见图1所示实施例的S13的描述，在此不再赘述。
步骤S24：基于图像缩放结果识别待识别图像中的目标对象。
详细内容请参见图1所示实施例的S14的描述，在此不再赘述。
本实施例的图像识别方法，利用深度学习的方法进行待识别图像的处理，由于基于深度学习的图像分割模型具有自学习的能力，通过该图像分割模型学习到的图像中的目标区域、目标对象对待识别图像进行识别处理，能够进一步提高识别的准确度。
在本实施例的一些可选实施方式中，该图像分割模型包括第一图像处理子模型及第二图像处理子模型，将图像数据集输入至预设的图像分割模型，对待识别图像进行图像分割，得到图像分割结果，包括：
步骤S221：将待识别图像输入至图像分割模型的第一图像处理子模型，对待识别图像中的图像数据进行图像采样处理，提取图像数据的图像特征；
步骤S222：将提取图像特征后的图像数据输入至图像分割模型的第二图像处理子模型，对图 像特征进行图像分割，识别图像特征的所属类别。
在本实施例的一些可选实施方式中，该第一图像处理子模型是用以对待识别图像进行粗分割的神经网络模型，例如unet网络等，本发明不以此为限。
在一些实施例中，步骤S221中，基于该第一图像处理子模型对待识别图像进行采样处理，提取图像数据的图像特征的过程，包括：
对图像数据进行下采样，识别图像数据的深层特征；
对进行下采样后的图像数据进行上采样，将深层特征存储到图像数据中。
在本实施例的一些可选实施方式中，该第二图像处理子模型是用以对待识别图像进行细分割的神经网络模型，例如pointrend网络等，本发明并不以此为限。
在一些实施例中，步骤S222中，基于该第二图像处理子模型对待识别图像的图像特征进行图像分割，识别图像特征的所属类别的过程，包括：
从图像特征中筛选预设置信度的特征点数据，对特征点数据进行双线性插值计算；
基于计算后的特征点数据识别所属类别。
在本实施例中，通过设定的两个图像处理子模型，分别对待识别图像进行粗分割以及细分割处理，分成两个不同的分割处理过程的原因在于，通过粗分割过程，提取该待识别图像的图像特征，通过细分割过程，从图像特征中进行像素点多类识别，从而根据像素点的特征来对图像中的内容进行类别的识别，从而能够依据整体过程提高对待识别图像的分割的精确度。
本实施例提供了一种图像识别方法，可用于电子设备，如电脑、手机、平板电脑等。在本实施例中，以图像识别模型对待识别图像进行处理为例进行说明，如图3所示，该图像识别方法主要包括：
步骤S31：获取待识别图像。
详细内容请参见图1所示实施例的S11的描述，在此不再赘述。
步骤S32：对待识别图像进行图像分割，得到图像分割结果。
详细内容请参见图1所示实施例的S12，或参见图2所示的实施例的S22的描述，在此不再赘述。
步骤S33：对图像分割结果进行多级缩放处理，得到图像缩放结果。
可选地，该步骤S33可以包括：
基于图像分割结果生成特征图像；
将待识别图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果。
其中，该预设的关键点识别模型可以是基于图像数据集训练得到的，图像数据集中包括已标记的正样本图像及未标记的负样本图像，正样本图像中含有用于表征目标区域的标记。
可选地，在关键点识别模型的训练过程中，利用图像数据集作为关键点识别模型的输入进行。其中，关于关键点识别模型的具体训练过程将在下文的关键点识别模型的训练方法实施例中做详细描述。
步骤S34：基于图像缩放结果识别待识别图像中的目标对象。
详细内容请参见图1所示实施例的S14的描述，在此不再赘述。
本实施例的图像识别方法，利用深度学习的方法进行待识别图像的处理，由于基于深度学习的关键点识别模型具有自学习的能力，通过该关键点识别模型学习到的图像中的目标对象对待识别图像进行识别处理，能够进一步提高识别的准确度。
在本实施例的一些可选实施方式中，该关键点识别模型是用以对待识别图像进行关键点识别的神经网络模型，例如hourglass网络等，本发明并不以此为限。
可选地，上述步骤S33中，将特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果的过程，主要包括：
步骤S331：对特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；在本实施例中，该预设分别率可以是根据实际应用场景的需要所设置的最低分辨率；
步骤S332：分别对各级下采样的特征图像进行上采样，得到第二特征图像；
步骤S333：基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；在进行各级下采样及上采样的过程中，对于各级采样得到的不同尺度的特征，进行结合，从而得到该特征图像；
步骤S334：基于合成特征图像确定特征图像中关键点处于合成特征图像中的概率，作为图像缩放结果。
在本实施例中，所采用的关键点识别模型，在对图像进行多级采样处理的过程中，针对各级采样时提取的图像特征，进行整合处理，由于考虑了各个尺度的图像特征，所以整体的图像处理的过程运行速度更快，使得针对该关键点识别模型的训练过程更快，能够更加迅速的完成针对该图像的关键点的识别过程。
本实施例提供了一种髋关节关键位置识别方法，可应用于电子设备，如电脑、手机、平板电脑等，也可应用在具体的领域，例如是医学领域等。在本实施例中，以图像识别模型对待识别图像进行处理为例进行说明，如图4所示，该髋关节关键位置识别方法主要包括：
步骤S41：获取医学图像数据；在本实施例中，该医学图像数据可以例如是通过X射线投影设备、CT投影设备等采集的图像数据，本发明并不以此为限。此步骤获取医学图像数据的具体过程请参见图1所示实施例的S11的描述，在此不再赘述。
步骤S42：对医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；在本实施例中，所针对的目标区域为髋关节关节位置，可选地，可以是股骨及骨盆区域。以股骨及骨盆区域为例，该步骤的详细内容请参见图1所示实施例的S12的描述或图2所示实施例的S22的描述或图3所示实施例的步骤S32的描述，在此不再赘述。
步骤S43：对图像分割结果进行多级缩放处理，得到包含髋关节关键位置的图像缩放结果；
详细内容请参见图1所示实施例的S13的描述或图2所示实施例的S23的描述或图3所示实施例的步骤S33的描述，在此不再赘述。
步骤S44：基于图像缩放结果识别医学图像数据中的髋关节关键位置。
详细内容请参见图1所示实施例的S14的描述或图2所示实施例的S24的描述或图3所示实施例的步骤S34的描述，在此不再赘述。
通过本实施例的髋关节关键位置识别方法，针对医学图像数据进行图像分割，从而提取该医学图像数据的图像特征，从而能够更加准确地识别该图像中的股骨及骨盆区域；然后，基于图像分割的结果及股骨及骨盆区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域(例如，股骨及骨盆区域)识别该图像中的目标对象(在本实施例中，是指股骨头中心点和泪滴下缘位置)。通过上述过程，对于医学图像数据的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
在本实施例的一些可选实施方式中，上述步骤S42中，对医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果的过程，可以是通过图像分割模型进行处理实现的，主要包括：
将医学图像数据输入至预设的图像分割模型，对医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；预设的图像分割模型是基于医学图像数据集训练得到的，医学图像数据集中包括已标记的正样本图像及未标记的负样本图像，正样本图像中含有用于表征股骨及骨盆区域的标记。
详细内容请参见图2所示实施例的S22的描述，在此不再赘述。
在本实施例中，通过设定的两个图像处理子模型，分别对待识别图像进行粗分割以及细分割处理，分成两个不同的分割处理过程的原因在于，通过粗分割过程，提取该待识别图像的图像特征，通过细分割过程，从图像特征中进行响度点多额是被，从而根据像素点的特征来对图像中的内容进行类别的识别，从而能够依据整体过程提高对待识别图像的分割的精确度。
在本实施例的一些可选实施方式中，上述步骤S43中，对图像分割结果进行多级缩放处理， 得到包含髋关节关键位置的图像缩放结果的过程，可以是通过关键点识别模型进行处理实现的，主要包括：
基于图像分割结果生成特征图像；
将特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果；预设的关键点识别模型是基于图像数据集训练得到的，图像数据集中包括已标记的正样本图像及未标记的负样本图像，正样本图像中含有用于表征目标区域的标记。
详细内容请参见图3所示实施例的S33的描述，在此不再赘述。
在本实施例中，所采用的关键点识别模型，在对图像进行多级采样处理的过程中，针对各级采样时提取的图像特征，进行整合处理，由于考虑了各个尺度的图像特征，所以整体的图像处理的过程运行速度更快，使得针对该关键点识别模型的训练过程更快，能够更加迅速的完成针对该图像的关键点的识别过程。
本实施例提供了一种关节位置识别方法，可应用于电子设备，如电脑、手机、平板电脑等，也可应用在具体的领域，例如是医学领域等。如图5A所示，该关节位置识别方法包括：
步骤S51：获取医学图像数据；
详细内容请参见图4所示实施例的S41的描述，在此不再赘述。
步骤S52：对医学图像数据进行图像分割，基于图像分割结果确定股骨及骨盆区域；
请参见图4所示实施例的S42的描述，在此不再赘述。
步骤S53：对图像分割结果进行多级缩放处理，基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；在本实施例中，所针对的目标对象为健侧的股骨头中心点和泪滴下缘位置。以股骨头中心点和泪滴下缘位置为例，该步骤的详细内容请参见图4所示实施例的S43的描述，在此不再赘述。
步骤S54：基于健侧的股骨头中心点和泪滴下缘位置确定真臼位置。
在本实施例中，通过步骤S51-S53确定了髋关节的关键位置(例如是股骨头中心点和泪滴下缘位置)的具体位置后，则可依据该具体位置来确定真臼位置。
在本实施例的一些可选实施方式中，该关节位置识别方法是具体应用于髋关节手术的术前规划场景中。髋关节手术主要是针对患者的病患侧的髋臼位置进行手术，而在实际应用中，由于病患侧的髋臼位置受到长期磨损等影响，无法准确确定患者的实际髋臼位置(也就是真臼位置)。因此，在本实施例中，是针对此种情况，先根据患者健康侧的髋臼位置来确定患者的病患侧的真臼位置。
因此，在本实施例中，基于健侧的股骨头中心点和泪滴下缘位置确定真臼位置的过程，主要包括：
首先，将健侧的股骨头中心点和泪滴下缘位置镜像翻转至目标区域，得到患侧的股骨头中心点镜像位置和泪滴下缘镜像位置；在本实施例中，由于是基于健康侧的髋臼位置来确定病患侧的髋臼位置，因此，通过上述方法所识别的股骨头中心点和泪滴下缘位置实际是健康侧的股骨头中心点和泪滴下缘位置，因此，首先将股骨头中心点和泪滴下缘位置镜像翻转至目标区域，此处的目标区域则是指患者的骨盆区域的病患侧。
如图5B所示，基于患侧的泪滴下缘镜像位置确定核心点位置，并计算骨盆高度；以泪滴下缘镜像位置做一条水平线，距离泪滴最低点往骨盆外的方向一定距离(例如是5mm)的设置一个核心点M，并计算整个骨盆的高度H：在图像中确定骨盆开始的层面b和骨盆结束的层面f，则对应地，计算出骨盆的高度H，H＝f-b；
基于核心点位置及骨盆高度确定包含真臼位置的区域。从M点开始向上(朝向骨盆中泪滴位置的方向)做一条垂线L1，这条线的长度可以为骨盆高度H的20％，然后继续向骨盆外(远离泪滴位置的方向)做一条水平线L2，L2的长度也可以为骨盆高度H的20％，则此时可确定，病患侧的真臼位置就在L1和L2所包围的区域内。
本实施例的关节位置识别方法，在前述实施例的图像识别方法、髋关节关键位置识别方法的 基础上，首先确定髋关节健康侧的髋臼位置，然后基于健康侧的髋臼位置，镜像确定病患侧的真臼位置，整个识别过程中，可采用深度学习为基础，基于医学图像数据进行识别，既提高了识别效率，也提高了识别精度，为后续进行髋关节相关手术提供了更加精确的技术支持。
根据本发明实施例，提供了一种图像识别模型的训练方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
本实施例提供了一种图像识别模型的训练方法，可用于电子设备，如电脑、手机、平板电脑等，如图6A所示，该训练方法主要包括：
步骤S61：获取图像数据集，图像数据集中包括已标记的正样本图像及未标记的负样本图像，正样本图像中含有用于表征目标区域的标记；在本实施例中，该图像数据集可以是通过一般的图像采集设备所采集的，例如是摄像机、手机、平板电脑等；或者，也可以是通过相对专业的图像采集设备所采集的，例如是医学领域常用的X射线投影设备、CT投影设备等，本发明并不以此为限。
在实际应用中，该图像数据集可以使CT医学图像数据集，在正样本图像中，则是标注股骨及骨盆区域，从而作为训练神经网络模型的数据库。并且可将该数据库中的图像数据集按照一定比例(例如是7:3)划分为训练集、测试集。将采集的二维横断面DICOM数据转换为JPG格式的图片，将标注文件转换成png格式的图片，保存后作为神经网络模型的输入。
步骤S62：将图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；
在本实施例的一些可选实施方式中，该第一神经网络模型是用以对图像进行分割的神经网络模型，其可以是由第一图像处理子模型及第二图像处理子模型组成。
可选地，将图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果的过程，包括：
步骤S621：将图像数据集输入至图像分割模型的第一图像处理子模型，对图像数据集中的图像数据进行图像采样处理，提取图像数据的图像特征；
步骤S622：将提取图像特征后的图像数据输入至图像分割模型的第二图像处理子模型，对图像特征进行图像分割，识别图像特征的所属类别。
在本实施例的一些可选实施方式中，该第一图像处理子模型是用以对待识别图像进行粗分割的神经网络模型，例如unet网络等，本发明并不以此为限。
可选地，步骤S621中，基于该第一图像处理子模型对待识别图像进行采样处理，提取图像数据的图像特征的过程，包括：
对图像数据进行下采样，识别图像数据的深层特征；
对进行下采样后的图像数据进行上采样，将深层特征存储到图像数据中。
实际应用中，以第一图像处理子模型为unet网络为例进行说明。
如图6B所示，首先利用unet网络作为主干网络，对图像数据集中的图像数据进行粗分割，第一阶段使用4次下采样，学习图像的深层特征，然后进行4次上采样以将特征图重新存储到图像中。其中，每个下采样层中包括2个卷积层和1个池化层，卷积核大小为3*3，池化层中的卷积核大小为2*2，每个卷积层中的卷积核的个数为128，256，512；每个上采样层中包括1个上采样层和2个卷积层，其中卷积层的卷积核大小为3*2，上采样层中的卷积核大小为2*2，每个上采样层中的卷积核个数为512，256，128。最后一次上采样结束后设有dropout层，droupout率设置为0.7，通过该dropout层减少中间特征的数量，从而避免不必要的冗余。所有的卷积层后面都设有激活函数为relu函数，从而增加神经网络各层之间的非线性关系，从而能够更加准确地表示每次采样提取的图像特征之间的关联关系。
在本实施例的一些可选实施方式中，该第二图像处理子模型是用以对待识别图像进行细分割的神经网络模型，例如pointrend网络等，本发明并不以此为限。
可选地，步骤S622中，基于该第二图像处理子模型对待识别图像的图像特征进行图像分割， 识别图像特征的所属类别的过程，包括：
从图像特征中筛选预设置信度的特征点数据，对特征点数据进行双线性插值计算；
基于计算后的特征点数据识别所属类别。
实际应用中，以第二图像处理子模型为pointrend网络为例进行说明。
如图6B所示，在通过unet网络对图像数据进行粗分割处理后，使用pointrend精确分割结果，选择一组预设置信度(例如可以为0.5)的特征点，提取被选择出来的点的特征，这些点的特征通过双线性插值Bilinear计算，使用分类器去判断这个点属于哪个类别。这个过程可以等价于用一个1*1的卷积来预测，但是对于置信度接近于1或者0的点并不计算。通过这样的过程，能够提高分割的精准度。
步骤S63：基于第一输出结果及图像数据集，对第一神经网络模型进行训练，更新第一神经网络模型，确定图像分割模型。
在上述的模型训练过程中，可设置参数为：数据标签的背景像素值可设置为0，股骨为1，骨盆为2，训练的batch_size为6，学习率设置为1e-4，优化器使用Adam优化器，使用的损失函数为DICE loss，将训练集输入第一神经网络进行训练，根据训练过程中损失函数的变化，调整训练批次的大小，最终得到各个部分的粗分割结果。进入pointrend网络后，先使用双线性插值上采样前一步分割预测结果，然后在这个更密集的特征图中选择N个最不确定的点，比如概率接近0.5的点。然后计算这N个点的特征表示，并且预测它们的labels。对于每个选定点的逐点特征表示，使用简单的多层感知器进行逐点预测，在本实施例中，也可以使用Unet粗分割任务中的损失函数来训练。
步骤S64：基于第一输出结果生成特征图像。本实施例中，通过第一神经网络模型输出的第一输出结果，将其重建为正投影图像作为其对应的特征图像。
步骤S65：将特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果。
在本实施例的一些可选实施方式中，该关键点识别模型是用以对待识别图像进行关键点识别的神经网络模型，例如hourglass网络等，本发明并不以此为限。
相应地，上述步骤S65中，将特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果的过程，主要包括：
步骤S651：对特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；在本实施例中，该预设分别率可以是根据实际应用场景的需要所设置的最低分辨率；
步骤S652：分别对各级下采样的特征图像进行上采样，得到第二特征图像；
步骤S653：基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；在进行各级下采样及上采样的过程中，对于各级采样得到的不同尺度的特征，进行结合，从而得到该特征图像；
步骤S654：基于合成特征图像确定特征图像中关键点处于合成特征图像中的概率，作为图像缩放结果。
实际应用中，以第二神经网络模型为hourglass网络为例进行说明。
如图6B所示，首先，hourglass网络的Conv层和Max Pooling层用于将特征图像缩放到预设标准的分辨率，每一个下采样处，hourglass网络同时保存原始尺寸的特征图像，并对原来的pre-pooled分辨率的特征进行卷积，得到最低分辨率特征后，网络开始进行upsampling(上采样)，并逐渐结合不同尺度的特征信息。本实施例中，对较低分辨率的特征图像的上采样采用的是最近邻上采样(nearest neighbor upsampling)方式，将两个不同的特征集进行逐元素相加。
由于整个hourglass网络结构是对称的，获取低分辨率特征过程中每有一个网络层，则在上采样的过程中相应的就会有一个对应网络层，得到hourglass网络模块的输出后，再采用两个连续的1*1Conv层进行处理，得到最终的网络输出，并输出为heatmaps的集合，每一个heatmap则表征了关键点在每个像素点存在的概率。
可选地，Hourglass网络在每次降采样之前，分出上半路保留原尺度信息；每次上采样之后，和上一个尺度的数据相加；两次降采样之间，可使用三个residual模块提取特征；两次相加之间，使用一个residual模块提取特征。由于考虑了各个尺度的特征，所以运行速度更快，网络训练时间更快。
步骤S66：基于第二输出结果及特征图像，对第二神经网络模型进行训练，更新第二神经网络模型，确定关键点识别模型；
在对该第二神经网络模型的训练过程中，可设置参数为：输入像素值为0-255的正投影图像和label.txt，可以通过每张图片的名称找到互相对应的点的坐标；在本实施例中，可以使将这些点生成高斯图，用heatmap去监督，即网络的输出是一个与输入大小相同尺寸的特征图，在检测点的位置为1，其他位置为0。对应于多个点的检测，可输出多个通道的特征图。网络使用Adam优化，学习率为1e-5，batch_size为4，损失函数使用L2正则化，根据训练过程中损失函数的变化，调整训练批次的大小，最终得到目标对象的坐标位置。
步骤S67：基于图像分割模型及关键点识别模型构建图像识别模型。
通过上述过程，则可训练得到针对目标对象的图像识别模型。通过本实施例的图像识别模型的训练方法所训练得到的图像识别模型，在进行实际图像识别过程中，可针对待识别图像进行图像分割，从而提取该待识别图像的图像特征，从而能够更加准确地识别该图像中的目标区域；然后，基于图像分割的结果及目标区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域识别该图像中的目标对象。通过上述过程，对于待识别图像的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
本实施例还提供一种图像识别模型的训练装置，如图7所示，包括：
图像获取模块101，用于获取图像数据集，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记；详细内容请参见上述方法实施例的S61的描述，在此不再赘述。
第一输出模块102，用于将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；详细内容请参见上述方法实施例的S62的描述，在此不再赘述。
图像分割模型确定模块103，用于基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型；详细内容请参见上述方法实施例的S63的描述，在此不再赘述。
特征图像生成模块104，用于基于所述第一输出结果生成特征图像；详细内容请参见上述方法实施例的S64的描述，在此不再赘述。
第二输出模块105，用于将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果；详细内容请参见上述方法实施例的S65的描述，在此不再赘述。
关键点识别模型确定模块106，用于基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型；详细内容请参见上述方法实施例的S66的描述，在此不再赘述。
图像识别模型构建模块107，用于基于所述图像分割模型及关键点识别模型构建图像识别模型。详细内容请参见上述方法实施例的S67的描述，在此不再赘述。
通过本实施例的图像识别模型的训练装置所训练得到的图像识别模型，在进行实际图像识别过程中，可针对待识别图像进行图像分割，从而提取该待识别图像的图像特征，从而能够更加准确地识别该图像中的目标区域；然后，基于图像分割的结果及目标区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域识别该图像中的目标对象。通过上述过程，对于待识别图像的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
本实施例还提供一种图像识别装置，如图8所示，包括：
图像获取模块201，用于获取待识别图像；详细内容请参见上述方法实施例的S11的描述，在此不再赘述。
图像分割结果生成模块202，用于对所述待识别图像进行图像分割，得到图像分割结果；详细内容请参见上述方法实施例的S12的描述，在此不再赘述。
图像缩放结果生成模块203，用于对所述图像分割结果进行多级缩放处理，得到图像缩放结果；详细内容请参见上述方法实施例的S13的描述，在此不再赘述。
目标对象识别模块204，用于基于所述图像缩放结果识别所述待识别图像中的目标对象。详细内容请参见上述方法实施例的S14的描述，在此不再赘述。
通过本实施例的图像识别装置，针对待识别图像进行图像分割，从而提取该待识别图像的图像特征，从而能够更加准确地识别该图像中的目标区域；然后，基于图像分割的结果及目标区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域识别该图像中的目标对象。通过上述过程，对于待识别图像的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
本实施例还提供一种髋关节关键位置识别装置，如图9所示，包括：
医学图像数据获取模块301，用于获取医学图像数据；详细内容请参见上述方法实施例的S41的描述，在此不再赘述。
图像分割结果生成模块302，用于对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；详细内容请参见上述方法实施例的S42的描述，在此不再赘述。
图像缩放结果生成模块303，用于对所述图像分割结果进行多级缩放处理，得到包含髋关节关键位置的图像缩放结果；详细内容请参见上述方法实施例的S43的描述，在此不再赘述。
髋关节关键位置识别模块304，用于基于所述图像缩放结果识别所述医学图像数据中的髋关节关键位置。详细内容请参见上述方法实施例的S44的描述，在此不再赘述。
通过本实施例的髋关节关键位置识别方法，针对医学图像数据进行图像分割，从而提取该医学图像数据的图像特征，从而能够更加准确地识别该图像中的股骨及骨盆区域；然后，基于图像分割的结果及股骨及骨盆区域，对图像做多级缩放处理，通过将图像分割结果缩放至预设分别率，从而基于目标区域识别该图像中的目标对象(在本实施例中，是指股骨头中心点和泪滴下缘位置)。通过上述过程，对于医学图像数据的分割以及缩放，能够更加快速、准确地从中提取目标区域及目标对象的特征，从而得到更加准确的图像识别结果。
本实施例还提供一种关节位置识别装置，如图10所示，包括：
医学图像数据获取模块401，用于获取医学图像数据；详细内容请参见上述方法实施例的S51的描述，在此不再赘述。
股骨及骨盆区域确定模块402，用于对所述医学图像数据进行图像分割，基于图像分割结果确定股骨及骨盆区域；详细内容请参见上述方法实施例的S52的描述，在此不再赘述。
髋关节关键位置识别模块403，用于对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；详细内容请参见上述方法实施例的S53的描述，在此不再赘述。
真臼位置确定模块404，用于基于所述健侧的股骨头中心点和泪滴下缘位置确定真臼位置。详细内容请参见上述方法实施例的S54的描述，在此不再赘述。
本实施例的关节位置识别装置，在前述实施例的图像识别方法、髋关节关键位置识别方法的基础上，首先确定髋关节健康侧的髋臼位置，然后基于健康侧的髋臼位置，镜像确定病患侧的真臼位置，整个识别过程中，可采用深度学习为基础，基于医学图像数据进行识别，既提高了识别效率，也提高了识别精度，为后续进行髋关节相关手术提供了更加精确的技术支持。
本发明实施例还提供了一种计算机设备，如图11所示，该计算机设备可以包括处理器111和存储器112，其中处理器111和存储器112可以通过总线或者其他方式连接，图11中以通过总线连接为例。
处理器111可以为中央处理器(Central Processing Unit，CPU)。处理器111还可以为其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等芯片，或者上述各类芯片的组合。
存储器112作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块，如本发明实施例中的图像识别方法，或髋关节关键位置识别方法，或真臼识别方法，或图像识别模型的训练方法对应的程序指令/模块。处理器111通过运行存储在存储器112中的非暂态软件程序、指令以及模块，从而执行处理器的各种功能应用以及数据处理，即实现上述方法实施例中的图像识别方法，或髋关节关键位置识别方法，或真臼识别方法，或图像识别模型的训练方法。
存储器112可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储处理器111所创建的数据等。此外，存储器112可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器112可选包括相对于处理器111远程设置的存储器，这些远程存储器可以通过网络连接至处理器111。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
所述一个或者多个模块存储在所述存储器112中，当被所述处理器111执行时，执行如图1-图6B所示实施例中的图像识别方法，或髋关节关键位置识别方法，或真臼识别方法，或图像识别模型的训练方法。
上述计算机设备具体细节可以对应参阅图1至图6B所示的实施例中对应的相关描述和效果进行理解，此处不再赘述。

Claims (20)

  1. 一种图像识别方法，包括：
    获取待识别图像；
    对所述待识别图像进行图像分割，得到图像分割结果；
    对所述图像分割结果进行多级缩放处理，得到图像缩放结果；
    基于所述图像缩放结果识别所述待识别图像中的目标对象。
  2. 根据权利要求1所述的图像识别方法，其中，对所述待识别图像进行图像分割，得到图像分割结果，包括：
    将所述待识别图像输入至预设的图像分割模型，对所述待识别图像进行图像分割，得到图像分割结果；所述预设的图像分割模型是基于图像数据集训练得到的，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记。
  3. 根据权利要求2所述的图像识别方法，所述图像分割模型包括第一图像处理子模型及第二图像处理子模型，将所述图像数据集输入至预设的图像分割模型，对所述待识别图像进行图像分割，得到图像分割结果，包括：
    将所述待识别图像输入至所述图像分割模型的第一图像处理子模型，对所述待识别图像中的图像数据进行图像采样处理，提取所述图像数据的图像特征；
    将提取图像特征后的图像数据输入至所述图像分割模型的第二图像处理子模型，对所述图像特征进行图像分割，识别所述图像特征的所属类别。
  4. 根据权利要求3所述的图像识别方法，其中，对所述图像数据集中的图像数据进行图像采样处理，提取所述图像数据的图像特征，包括：
    对所述图像数据进行下采样，识别所述图像数据的深层特征；
    对进行下采样后的图像数据进行上采样，将所述深层特征存储到所述图像数据中。
    其中，对所述图像特征进行图像分割，识别所述图像特征的所属类别，包括：
    从所述图像特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算；
    基于计算后的特征点数据识别所属类别。
  5. 根据权利要求1所述的图像识别方法，其中，对所述图像分割结果进行多级缩放处理，得到图像缩放结果，包括：
    基于所述图像分割结果生成特征图像；
    将所述特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果；所述预设的关键点识别模型是基于所述图像数据集训练得到的，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记。
  6. 根据权利要求5所述的图像识别方法，其中，将所述特征图像输入至预设的关键点识别模型，进行多级图像缩放处理，得到图像缩放结果，包括：
    对所述特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；
    分别对各级下采样的特征图像进行上采样，得到第二特征图像；
    基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；
    基于所述合成特征图像确定所述特征图像中关键点处于所述合成特征图像中的概率，作为所述图像缩放结果。
  7. 一种利用如权利要求1至6任一项所述的图像识别方法识别关节位置的方法，包括：
    获取医学图像数据；
    对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；
    对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定所述医学图像数据中的关节位置。
  8. 根据权利要求7所述的识别关节位置的方法，其中，基于图像缩放结果确定所述医学图像数据中的关节位置，包括：
    基于图像缩放结果确定髋关节关键位置；
    基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；
    将所述健侧的股骨头中心点和泪滴下缘位置镜像翻转至目标区域，得到患侧的股骨头中心点镜像位置和泪滴下缘镜像位置；
    基于所述患侧的泪滴下缘镜像位置确定核心点位置，并计算骨盆高度；
    基于所述核心点位置及骨盆高度确定包含真臼位置的区域。
  9. 一种图像识别模型的训练方法，包括：
    获取图像数据集，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记；
    将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；
    基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型；
    基于所述第一输出结果生成特征图像；
    将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果；
    基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型；
    基于所述图像分割模型及关键点识别模型构建图像识别模型。
  10. 根据权利要求9所述的图像识别模型的训练方法，其中，将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果，包括：
    将所述图像数据集输入至所述第一神经网络模型的第一图像处理子模型，对所述图像数据集中的图像数据进行图像采样处理，提取所述图像数据的图像特征；
    将提取图像特征后的图像数据输入至所述第一神经网络模型的第二图像处理子模型，对所述图像特征进行图像分割，识别所述图像特征的所属类别。
  11. 根据权利要求10所述的图像识别模型的训练方法，其中，对所述图像数据集中的图像数据进行图像采样处理，提取所述图像数据的图像特征，包括：
    对所述图像数据进行下采样，识别所述图像数据的深层特征；
    对进行下采样后的图像数据进行上采样，将所述深层特征存储到所述图像数据中。
  12. 根据权利要求10所述的图像识别模型的训练方法，其中，对所述图像特征进行图像分割，识别所述图像特征的所属类别，包括：
    从所述图像特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算；
    基于计算后的特征点数据识别所述图像特征的所属类别。
  13. 根据权利要求9所述的图像识别模型的训练方法，其中，将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果，包括：
    对所述特征图像进行多级下采样，得到符合预设分辨率的第一特征图像；
    分别对各级下采样的第一特征图像进行上采样，得到第二特征图像；
    基于各级下采样的第一特征图像及各级上采样的第二特征图像生成合成特征图像；
    基于所述合成特征图像确定所述特征图像中关键点处于所述合成特征图像中的概率，作为所述第二输出结果。
  14. 根据权利要求9至13中任一项所述的图像识别模型的训练方法，其中，基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型，包括：
    基于所述第一输出结果及图像数据集计算第一损失函数；
    基于所述第一损失函数更新所述第一神经网络模型的参数，确定所述图像分割模型；
    其中，基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型，包括：
    基于所述第二输出结果及所述特征图像计算第二损失函数；
    基于所述第二损失函数更新所述第二神经网络模型的参数，确定所述关键点识别模型。
  15. 一种图像识别模型的训练装置，包括：
    图像获取模块，用于获取图像数据集，所述图像数据集中包括已标记的正样本图像及未标记的负样本图像，所述正样本图像中含有用于表征目标区域的标记；
    第一输出模块，用于将所述图像数据集输入至第一神经网络模型，进行图像分割，得到第一输出结果；
    图像分割模型确定模块，用于基于所述第一输出结果及图像数据集，对所述第一神经网络模型进行训练，更新所述第一神经网络模型，确定图像分割模型；
    特征图像生成模块，用于基于所述第一输出结果生成特征图像；
    第二输出模块，用于将所述特征图像输入至第二神经网络模型，进行多级图像缩放处理，得到第二输出结果；
    关键点识别模型确定模块，用于基于所述第二输出结果及所述特征图像，对所述第二神经网络模型进行训练，更新所述第二神经网络模型，确定关键点识别模型；
    图像识别模型构建模块，用于基于所述图像分割模型及关键点识别模型构建图像识别模型。
  16. 一种图像识别装置，包括：
    图像获取模块，用于获取待识别图像；
    图像分割结果生成模块，用于对所述待识别图像进行图像分割，得到图像分割结果；
    图像缩放结果生成模块，用于对所述图像分割结果进行多级缩放处理，得到图像缩放结果；
    目标对象识别模块，用于基于所述图像缩放结果识别所述待识别图像中的目标对象。
  17. 一种关节位置识别装置，包括：
    医学图像数据获取模块，用于获取医学图像数据；
    图像分割结果生成模块，用于对所述医学图像数据进行图像分割，得到包含股骨及骨盆区域的图像分割结果；
    关节位置确定模块，用于对所述图像分割结果进行多级缩放处理，基于图像缩放结果确定所述医学图像数据中的关键位置；
  18. 根据权利要求17所述的关节位置识别装置，其中，关节位置确定模块，包括：
    髋关节关键位置识别模块，被配置为基于图像缩放结果确定髋关节关键位置；
    真臼位置确定模块，被配置为基于图像缩放结果确定健侧的股骨头中心点和泪滴下缘位置；
    将所述健侧的股骨头中心点和泪滴下缘位置镜像翻转至目标区域，得到患侧的股骨头中心点镜像位置和泪滴下缘镜像位置；
    基于所述患侧的泪滴下缘镜像位置确定核心点位置，并计算骨盆高度；
    基于所述核心点位置及骨盆高度确定包含所述真臼位置的区域。
  19. 一种计算机设备，包括：存储器和处理器，所述存储器和所述处理器之间互相通信连接，
    所述存储器中存储有计算机指令；
    所述处理器通过执行所述计算机指令，从而执行如权利要求1至6中任一项所述的图像识别方法，或，执行如权利要求7或8所述的关节位置识别方法，或，执行如权利要求9至14中任一项所述的图像识别模型的训练方法。
  20. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使所述计算机执行如权利要求1至6中任一项所述的图像识别方法，或，执行如权利要求7或8所述的关节位置识别方法，或，执行如权利要求9至14中任一项所述的图像识别模型的训练方法。
PCT/CN2021/131966 2021-05-26 2021-11-22 图像识别及模型训练的方法、关节位置识别的方法 WO2022247173A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110580663.5A CN113298786B (zh) 2021-05-26 2021-05-26 图像识别及模型训练的方法、真臼位置识别的方法及装置
CN202110580663.5 2021-05-26

Publications (1)

Publication Number Publication Date
WO2022247173A1 true WO2022247173A1 (zh) 2022-12-01

Family

ID=77325360

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131966 WO2022247173A1 (zh) 2021-05-26 2021-11-22 图像识别及模型训练的方法、关节位置识别的方法

Country Status (2)

Country Link
CN (1) CN113298786B (zh)
WO (1) WO2022247173A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298786B (zh) * 2021-05-26 2022-02-11 北京长木谷医疗科技有限公司 图像识别及模型训练的方法、真臼位置识别的方法及装置
CN113962927B (zh) * 2021-09-01 2022-07-12 北京长木谷医疗科技有限公司 基于强化学习的髋臼杯位置调整方法、装置及存储介质
CN113870261B (zh) * 2021-12-01 2022-05-13 杭州柳叶刀机器人有限公司 用神经网络识别力线的方法与系统、存储介质及电子设备
CN114663363B (zh) * 2022-03-03 2023-11-17 四川大学 一种基于深度学习的髋关节医学图像处理方法和装置
CN114972215B (zh) * 2022-05-11 2024-07-19 平安科技（深圳）有限公司 病灶图像的分割方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260712A (zh) * 2015-10-03 2016-01-20 上海大学 一种车辆前方行人检测方法及系统
CN110136147A (zh) * 2019-05-21 2019-08-16 湖北工业大学 一种基于U-Net模型的分割医学图像的方法、装置及存储介质
CN111369567A (zh) * 2018-12-26 2020-07-03 腾讯科技（深圳）有限公司 三维图像中目标对象的分割方法、装置及电子设备
CN111627098A (zh) * 2020-05-21 2020-09-04 广州光锥元信息科技有限公司 对图像中水流区域识别并生成动态水流视频的方法及装置
CN113298786A (zh) * 2021-05-26 2021-08-24 北京长木谷医疗科技有限公司 图像识别及模型训练的方法、真臼位置识别的方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190122104A1 (en) * 2017-10-19 2019-04-25 General Electric Company Building a binary neural network architecture
CN109858310A (zh) * 2017-11-30 2019-06-07 比亚迪股份有限公司 车辆和交通标志检测方法
CN110731777B (zh) * 2019-09-16 2023-07-25 平安科技（深圳）有限公司 基于图像识别的左心室测量方法、装置以及计算机设备
CN111429458B (zh) * 2020-03-20 2021-06-08 北京创世云科技股份有限公司 一种图像还原的方法、装置及电子设备
CN111563901B (zh) * 2020-04-15 2023-08-08 中国科学院苏州生物医学工程技术研究所 基于磁共振的髋关节图像处理方法及系统、存储介质、设备
CN111429452A (zh) * 2020-04-15 2020-07-17 深圳市嘉骏实业有限公司 基于UNet卷积神经网络的膀胱超声图像分割方法及装置
CN111968120B (zh) * 2020-07-15 2022-03-15 电子科技大学 一种3d多特征融合的牙齿ct图像分割方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260712A (zh) * 2015-10-03 2016-01-20 上海大学 一种车辆前方行人检测方法及系统
CN111369567A (zh) * 2018-12-26 2020-07-03 腾讯科技（深圳）有限公司 三维图像中目标对象的分割方法、装置及电子设备
CN110136147A (zh) * 2019-05-21 2019-08-16 湖北工业大学 一种基于U-Net模型的分割医学图像的方法、装置及存储介质
CN111627098A (zh) * 2020-05-21 2020-09-04 广州光锥元信息科技有限公司 对图像中水流区域识别并生成动态水流视频的方法及装置
CN113298786A (zh) * 2021-05-26 2021-08-24 北京长木谷医疗科技有限公司 图像识别及模型训练的方法、真臼位置识别的方法及装置

Also Published As

Publication number Publication date
CN113298786B (zh) 2022-02-11
CN113298786A (zh) 2021-08-24

Similar Documents

Publication Publication Date Title
WO2022247173A1 (zh) 图像识别及模型训练的方法、关节位置识别的方法
EP3961500A1 (en) Medical image detection method based on deep learning, and related device
WO2020133636A1 (zh) 前列腺手术中外包膜智能检测和预警方法及系统
WO2022257345A1 (zh) 医学图像融合方法及系统、模型训练方法及存储介质
WO2019140957A1 (zh) 一种骨折识别模型的构建方法及应用
WO2023029896A1 (zh) 基于强化学习的髋臼杯位置调整方法、装置及存储介质
WO2021017481A1 (zh) 图像处理方法及装置、电子设备和存储介质
CN113076987B (zh) 骨赘识别方法、装置、电子设备及存储介质
CN109558862B (zh) 基于空间感知的注意力细化框架的人群计数方法及系统
JP2023530023A (ja) 深層学習に基づく脊椎ｍｒｉ映像キーポイント検出方法
CN113223072B (zh) 一种脊柱Cobb角测量方法和系统
WO2023137914A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN112862824A (zh) 新型冠状病毒肺炎病灶检测方法、系统、装置和存储介质
CN112308822B (zh) 基于深度卷积神经网络的椎间盘ct图像检测方法
WO2023130648A1 (zh) 一种图像数据增强方法、装置、计算机设备和存储介质
CN110974306B (zh) 一种超声内镜下识别和定位胰腺神经内分泌肿瘤的系统
CN113488163B (zh) 一种基于机器视觉的肌少症识别方法、装置、设备及介质
CN106228198B (zh) 一种医疗ct图像的超分辨率识别方法
CN114787828A (zh) 利用具有有意受控畸变的成像器进行人工智能神经网络的推理或训练
CN114399799A (zh) 一种基于YOLOv5网络的口罩佩戴检测方法
CN116934859B (zh) 基于矢状位图的关键点识别方法及相关装置
CN116188435B (zh) 一种基于模糊逻辑的医学图像深度分割方法
CN117237304A (zh) 一种具有自主学习能力的儿童骨龄评估方法及系统
CN115252233B (zh) 基于深度学习的全髋关节置换术中髋臼杯的自动化规划方法
EP3852062A1 (en) Segmenting an object in an image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21942736

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21942736

Country of ref document: EP

Kind code of ref document: A1