WO2022088729A1 - 点定位方法及相关装置、设备、介质及计算机程序 - Google Patents

点定位方法及相关装置、设备、介质及计算机程序 Download PDF

Info

Publication number
WO2022088729A1
WO2022088729A1 PCT/CN2021/103150 CN2021103150W WO2022088729A1 WO 2022088729 A1 WO2022088729 A1 WO 2022088729A1 CN 2021103150 W CN2021103150 W CN 2021103150W WO 2022088729 A1 WO2022088729 A1 WO 2022088729A1
Authority
WO
WIPO (PCT)
Prior art keywords
target point
heat map
heat
target
positioning
Prior art date
Application number
PCT/CN2021/103150
Other languages
English (en)
French (fr)
Inventor
顾宇俊
袁璟
赵亮
黄宁
张少霆
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2022088729A1 publication Critical patent/WO2022088729A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present disclosure relates to the technical field of computer vision, and in particular, to a point positioning method and related apparatuses, devices, media and computer programs.
  • the embodiments of the present disclosure are expected to provide a point positioning method and related apparatus, equipment, medium and computer program.
  • a first aspect of the embodiments of the present disclosure provides a point positioning method, including: acquiring an image to be positioned; performing target point detection on the image to be positioned to obtain a coarse positioning heat map and a fine positioning heat map, wherein the coarse positioning heat map includes a target The heat value of the first area of the point is within the first heat value range; the heat value of the second area including the target point in the fine positioning heat map is within the second heat value range, where the first area is greater than the second area; The positioning heat map and the fine positioning heat map are used to obtain the position information of the target point.
  • the combined analysis of the coarse positioning heat map and the fine positioning heat map to obtain the position information of the target point includes: obtaining the position of the first target point in the coarse positioning heat map and the position of the first target point in the fine positioning heat map
  • the position information of the target point is obtained by processing the position of the first target point and the position of the second target point based on the confidence degree of the position of the second target point and the position of the second target point.
  • the confidence level of the position, the position information of the target point is obtained by processing the position of the first target point and the position of the second target point, including: determining the position of the first target point based on the heat value of the coarse positioning heat map; The heat value is used to determine the position of the second target point and the confidence of the position of the second target point; and based on the confidence of the position of the second target point, the position of the second target point or the position of the first target point is selected as the position information of the target point.
  • the second target point position or the first target point position is selected based on the confidence degree of the second target point position, and the position information as the target point includes: if the confidence degree of the second target point position If the position confidence condition is satisfied, the position of the second target point is used as the position information of the target point; if the confidence of the position of the second target point does not satisfy the position confidence condition, the position of the first target point is used as the position information of the target point.
  • the method before determining the position of the second target point and the confidence of the position of the second target point based on the heat value of the fine positioning heat map, the method further includes: determining the first target point based on the heat value of the coarse positioning heat map. A confidence level of the position of the target point. If the confidence level of the position of the first target point satisfies the confidence condition of the coarse position, the heat value in the heat map of the fine positioning which is located outside the preset distance range of the position of the first target point is adjusted to the preset heat value value; wherein, the preset heat value is outside the second heat value range.
  • the rough position confidence condition includes that the confidence level of the first target point position is greater than a first preset threshold
  • the fine position confidence condition includes at least one of the following: the confidence level of the second target point position is greater than the first target point position. Two preset thresholds, the confidence of the position of the second target point is greater than the confidence of the position of the first target point.
  • determining the position of the first target point based on the heat value of the coarse positioning heat map, or determining the position of the second target point based on the heat value of the fine positioning heat map includes: placing the positioning heat map The point with the largest heat value is used as the target point position, or the regional preset point in the positioning heat map is used as the target point position.
  • the confidence of the position of the first target point is determined based on the heat value of the coarse positioning heat map, or the confidence of the position of the second target point is determined based on the heat value of the fine positioning heat map, Including: obtaining at least one reference heat value; for each reference heat value, obtaining the size of the reference area whose heat value is greater than the reference heat value from the positioning heat map; and obtaining based on the size of each reference area and the heat value of the target point position The confidence of the target point position; or, based on the target point position of the positioning heat map, the target heat map is obtained; based on the similarity of the heat value distribution between the positioning heat map and the target heat map, the confidence of the target point position is obtained.
  • the acquiring at least one reference heat value includes: acquiring at least one magnification, and using the product between the at least one magnification and the heat value of the target point position as the at least one reference heat value; the reference area includes The location of the target point; the size of the reference area includes the perimeter and area of the reference area; the confidence of the target point location is obtained based on the size of each reference area and the heat value of the target point location, including: obtaining the area and circumference of each reference area The first ratio between the squares of the lengths is obtained by using the sum of the first ratios of the at least one reference area, the heat value of the target point position, and the preset heat value peak to obtain the confidence level of the target point position.
  • obtaining the target heat map based on the position of the target point in the positioning heat map includes: obtaining the heat value of each pixel in the target heat map by using a two-dimensional Gaussian function based on the position of the target point in the positioning heat map , where the exponent of the two-dimensional Gaussian function includes the range parameter, and the absolute value of the exponent is negatively correlated with the range parameter.
  • the range parameter in the dimensional Gaussian function based on the similarity of the heat value distribution between the positioning heat map and the target heat map, the confidence of the target point position is obtained, including: based on the heat value distribution between the positioning heat map and the target heat map, Obtain the correlation coefficient between the positioning heat map and the target heat map as the confidence of the target point location.
  • the method before performing target point detection on the image to be positioned to obtain the heat map of coarse positioning and the heat map of fine positioning, the method further includes: preprocessing the image to be positioned; wherein the preprocessing includes at least one of the following: The image to be positioned is normalized, and the image contrast of the image to be positioned is enhanced.
  • the method further includes: outputting the position information of the target point and the confidence level of the corresponding position information.
  • the normalizing the image to be positioned includes: setting a pixel value greater than a first pixel value in the image to be positioned as the first pixel value, and setting a pixel value of the image to be positioned smaller than a second pixel value as the first pixel value The pixel value of the pixel value is set as the second pixel value; wherein, among the sequentially arranged pixel values of the image to be positioned, the pixel value in the first numerical ranking is the first pixel value, and the pixel value in the second numerical ranking is the first pixel value. is the second pixel value.
  • performing target point detection on the image to be positioned to obtain a coarse positioning heatmap and a fine positioning heatmap includes: using a deep learning model to perform target point detection on the image to be positioned to obtain a coarse positioning heatmap and a fine positioning heatmap. Locate heatmaps.
  • the deep learning model is a fully convolutional neural network, and/or the deep learning model is trained by at least the following steps: acquiring a sample image, wherein the sample image is marked with the real image of the target point Location information; use the real location information of the target point to generate a coarse target heat map and a fine target heat map; wherein, the heat value of the third area containing the target point in the coarse target heat map is within the range of the third heat value; the fine target heat map The heat value of the fourth area including the target point is in the fourth heat value range, and the third area is larger than the fourth area; use the deep learning model to detect the target point of the sample image, and obtain the coarse positioning heat map and the fine positioning heat map ; Adjust the network parameters of the deep learning model based on the difference between the coarse target heatmap and the coarse localization heatmap and the difference between the fine target heatmap and the fine localization heatmap.
  • the to-be-located image is an X-ray image; in the first area and the second area, the closer to the target point the higher the heat value; the heat value outside the first area in the heat map is roughly located The value is lower than the lower limit value of the first heat value range, and the heat value outside the second area in the fine-location heat map is lower than the lower limit value of the second heat value range.
  • a second aspect of the embodiments of the present disclosure provides a point positioning device, including: an image acquisition module, a target detection module, and a position analysis module, an image acquisition module configured to acquire an image to be positioned; The target point is detected, and the coarse positioning heat map and the fine positioning heat map are obtained, wherein the heat value of the first area containing the target point in the coarse positioning heat map is within the first heat value range; the fine positioning heat map contains the target point.
  • the heat value of the area is in the second heat value range, wherein the first area is greater than the second area; the location analysis module is configured to combine and analyze the coarse positioning heat map and the fine positioning heat map to obtain the position information of the target point.
  • a third aspect of the embodiments of the present disclosure provides an electronic device, including a mutually coupled memory and a processor, where the processor is configured to execute program instructions stored in the memory, so as to implement the point positioning method in the first aspect.
  • a fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, the point positioning method in the above-mentioned first aspect is implemented.
  • a fifth aspect of the embodiments of the present disclosure provides a computer program, where the computer program includes computer-readable codes, and when the computer-readable codes are executed in an electronic device, a processor of the electronic device executes commands for The point positioning method in the first aspect above is implemented.
  • a coarse positioning heat map and a fine positioning heat map are obtained, and the heat value of the first area including the target point in the coarse positioning heat map is in The first heat value range, the heat value of the second area including the target point in the fine positioning heat map is within the second heat value range, the first area of the coarse positioning heat map is larger than the second area of the fine positioning heat map, so the coarse positioning heat map
  • the map can represent the target response in a large range near the target point, and the fine positioning heat map can represent the target response in a small range near the target point. Therefore, combining the analysis of the coarse positioning heat map and the fine positioning heat map can have both coarse positioning heat map and fine positioning heat map at the same time.
  • the positioning stability of the positioning heat map and the accuracy of the fine positioning heat map can improve the accuracy and stability of point positioning.
  • FIG. 1 is a schematic flowchart of a point positioning method according to an embodiment of the present disclosure
  • FIG. 2 is an optional schematic diagram of a positioning result of an image to be positioned in an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a coarse positioning heat map and a fine positioning heat map in an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of training a deep learning model in an embodiment of the present disclosure
  • step S13 is a schematic flowchart of step S13 in the point positioning method according to an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of a frame of a point positioning device according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram of a framework of an electronic device according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a framework of a computer-readable storage medium according to an embodiment of the present disclosure.
  • system and “network” are often used interchangeably herein.
  • the term “and/or” in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases.
  • the character "/” in this document generally indicates that the related objects are an “or” relationship.
  • “multiple” herein means two or more than two.
  • FIG. 1 is a schematic flowchart of a point positioning method according to an embodiment of the present disclosure. As shown in FIG. 1 , the point positioning method may include the following steps:
  • Step S11 Acquire an image to be positioned.
  • the image to be located may be an image including the facial features of a human face, so that target points such as the eyes, mouth, nose, etc. of the human face in the image can be located in the image by the point positioning method of this embodiment, so as to be subsequently used for the human face.
  • the image to be located may also be an image image including human tissue and organs, so that target points in human tissue and organs in the image image can be located by the point positioning method of this embodiment.
  • the image to be positioned may be an X-ray image, that is, a computed tomography image. In a specific implementation scenario, please refer to FIG.
  • the image to be located can be the X-ray image of the human lower limb (including the left lower limb and the right lower limb), and the target points obtained by positioning can include but are not limited to: the center of the femoral head, the tip of the greater trochanter, the medial malleolus of the femur, the lateral malleolus of the femur, and the medial endpoint of the tibial plateau , the outer end point of the tibial plateau, the inner end point of the ankle joint space, the outer end point of the ankle joint space, the target points shown in Figure 2 can be obtained through the point positioning method of this embodiment as the above-mentioned 8 target points on the left lower limb and the right lower limb, namely A total of 16 target points (black filled circles in Figure 2). In other application scenarios, it can be deduced by analogy, and no examples are given here.
  • the 16 target points in FIG. 2 are examples of target points to be positioned by using the point positioning method of this embodiment.
  • the above-mentioned 16 target points are not marked in the to-be-positioned image. It can be understood that the above 16 target points are removed from FIG. 2 .
  • the image behind the black-filled dots can be a to-be-located image.
  • Step S12 Perform target point detection on the image to be positioned to obtain a coarse positioning heat map and a fine positioning heat map.
  • the heat value of the first area including the target point in the coarse positioning heat map is within the first heat value range
  • the heat value of the second area including the target point in the fine positioning heat map is within the second heat value range
  • the first area is greater than the first area.
  • the positioning heat map can reflect the target response of each pixel in the image to be positioned. In some optional embodiments, in the first area and the second area, the heat value closer to the target point is higher, that is, the larger the target response value is.
  • the heat value outside the first area in the coarse positioning heat map is lower than the lower limit of the first heat value range
  • the heat value outside the second area in the fine positioning heat map is lower than the lower limit of the second heat value range
  • the coarse positioning heat map has a higher response value in a larger range than the fine positioning heat map, so the target points in the coarse positioning heat map and the fine positioning heat map can be easily determined, and the coarse positioning heat map can be located at subsequent points.
  • the stability of point positioning is ensured, and the fine positioning heat map can ensure the accuracy of point positioning in the subsequent point positioning process.
  • FIG. 3 is a schematic diagram of a coarse positioning heat map and a fine positioning heat map in an embodiment of the present disclosure.
  • (a) is a coarse positioning heat map
  • (b) is a fine positioning heat map.
  • Heat map for the convenience of description, the coarse positioning heat map and fine positioning heat map shown in Figure 3 represent the target response to the same target point, the solid circles in the coarse positioning heat map and the fine positioning heat map and their surrounding
  • the white filled areas respectively represent the first area and the second area containing the target points.
  • the heat value can also be represented in the order of spectral colors. For example, “red” can be used to represent the target with the largest heat value. Points, as the heat value decreases, the points gradually away from the target point are represented by "orange”, “yellow”, “green”, “blue”, etc. respectively.
  • a deep learning model in order to make full use of hardware parallel acceleration and reduce the complexity of target point detection, can be used to perform target point detection on the image to be positioned, thereby obtaining a coarse positioning heat map and a fine positioning heat map.
  • the deep learning model is a fully convolutional neural network.
  • a deep learning model can employ a Unet network with an encoder, decoder, and skip link structure. When using the deep learning model to detect the target point of the image to be positioned, the corresponding coarse positioning heat map and fine positioning heat map can be generated for each target point.
  • the to-be-located image may also be preprocessed before the target point detection is performed and the coarse positioning heat map and the fine positioning heat map are obtained;
  • the preprocessing may include normalizing the image to be positioned.
  • the normalizing the to-be-located image may include: setting a pixel value greater than a first pixel value in the to-be-located image as the first pixel value, and setting a pixel value of the to-be-located image smaller than a second pixel value as the first pixel value.
  • the pixel value of the value sets the second pixel value, wherein, among the sequentially arranged pixel values of the image to be positioned, the pixel value in the first numerical rank is the first pixel value, and the pixel value in the second numerical rank is the first pixel value.
  • sort the pixels of the image to be located according to the pixel value from small to large take the pixel value of the 99th percentile (that is, the 99th percentile of the total) as the first pixel value, and take the 3 percent (that is, the The pixel value ranked by the 3% bit) is the second pixel value, and the pixel value greater than the first pixel value is set as the first pixel value, and the pixel value smaller than the second pixel value is set as the second pixel value.
  • the first numerical ranking and the second numerical ranking may also be set according to specific applications, which are not limited herein.
  • the pixel value in the first numerical rank is the first pixel value
  • the pixel in the second numerical rank is the second pixel value
  • image contrast enhancement processing may be performed on the to-be-located image.
  • a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm may be used to process the image to be positioned, so as to enhance the local contrast of the image.
  • Step S13 Combine and analyze the coarse positioning heat map and the fine positioning heat map to obtain the position information of the target point.
  • the position of the first target point and the confidence level of the position of the first target point in the rough positioning heat map may be obtained, and when the confidence level of the position of the first target point is greater than the confidence threshold
  • the position of the target point is used as the position information of the target point, so that the position information of the target point can be quickly determined under the condition that the accuracy requirement is not high.
  • the point with the highest heat value in the rough positioning heat map may be used as the position of the first target point.
  • the confidence level of the first target point position is used to indicate the reliability of the located first target point position, and the higher the confidence level of the first target point position, the higher the reliability of the first target point position.
  • the first confidence level is used to represent the confidence level of the position of the first target point.
  • the position of the first target point in the heat map of coarse positioning may also be obtained, and the position of the second target point and the confidence of the position of the second target point in the heat map of fine positioning may be obtained, and based on the position of the second target point in the heat map of fine positioning
  • the confidence level of the second target point position, the first target point position and the second target point position are processed to obtain the position information of the target point, and then the confidence level of the target point can be further based on the coarse positioning heat map and the fine positioning heat map. Therefore, the accuracy and stability of point positioning can be taken into account, and the accuracy of point positioning can be further improved.
  • the position of the first target point or the position of the second target point can be selected as the position information of the target point according to the confidence of the position of the second target point; or, according to the confidence of the position of the second target point, the output contains the first target point
  • the position information of the point position and the second target point position is not limited here.
  • the point with the highest heat value in the coarse positioning heat map may be used as the first target point position
  • the point with the highest heat value in the fine positioning heat map may be used as the second target point position.
  • the confidence level of the second target point position is used to indicate the reliability of the located second target point position, and the higher the confidence level of the second target point position, the higher the confidence level of the second target point position.
  • the second confidence level is used to represent the confidence level of the position of the second target point.
  • the position information of the target point position and the corresponding position may also be output confidence in the information.
  • the confidence level of the first target point can be used as the confidence level of the corresponding position information; or, when the position of the second target point is used as the position information of the target point, The confidence level of the second target point can be used as the confidence level of the corresponding position information, which can help the user to evaluate the position information of the target point obtained by the positioning and improve the user's perception.
  • a coarse positioning heat map and a fine positioning heat map are obtained, and the heat value of the first area including the target point in the coarse positioning heat map is within the first heat value range.
  • the heat value of the second area containing the target point in the fine positioning heat map is within the second heat value range
  • the first area of the coarse positioning heat map is larger than the second area of the fine positioning heat map, so the coarse positioning heat map can represent the target point.
  • the target response in a relatively large range nearby, and the fine positioning heat map can represent the target response in a small range near the target point, so combining the analysis of the coarse positioning heat map and the fine positioning heat map, the positioning of the coarse positioning heat map can be combined at the same time.
  • the stability and accuracy of the heat map for fine positioning can improve the accuracy and stability of point positioning.
  • FIG. 4 is a schematic flowchart of training a deep learning model in an embodiment of the present disclosure. As shown in FIG. 4 , the following steps may be included:
  • Step S41 Obtain a sample image, wherein the sample image is marked with the real position information of the target point.
  • the sample image may be an image including the facial features of a human face, and the target point may include at least one of the eyes, the mouth, and the nose of the human face.
  • the sample image may also be a video image including human tissues and organs.
  • the sample image may be an X-ray image, that is, a computed tomography image.
  • the sample image may be an X-ray image of a human lower limb (including a left lower limb and a right lower limb), and the target point may include, but is not limited to, at least one of the following: Bone center, tip of the greater trochanter, medial malleolus of femur, lateral malleolus of femur, medial end point of tibial plateau, lateral end point of tibial plateau, medial end point of ankle joint space, outer end point of ankle joint space, for details, please refer to the relevant steps in the previous embodiment, and will not be repeated here. Repeat.
  • Step S42 Generate a coarse target heat map and a fine target heat map by using the real position information of the target point.
  • the heat value of the third area including the target point in the coarse target heat map is within the third heat value range; the heat value of the fourth area including the target point in the fine target heat map is within the fourth heat value range, where the third area is greater than Fourth area.
  • the heat value of each pixel in the coarse target heat map and the fine target heat map can be obtained by using a two-dimensional Gaussian function based on the real position information of the target point in the sample image, wherein the index of the two-dimensional Gaussian function includes the range parameter. , and there is a negative correlation between the absolute value of the index and the range parameter, and the range parameter in the two-dimensional Gaussian function corresponding to the coarse target heatmap is larger than the range parameter in the two-dimensional Gaussian function corresponding to the fine target heatmap.
  • the heat value of each pixel in the target heat map can be expressed as:
  • (x, y) represents the abscissa and ordinate of the pixel point
  • (x 0 , y 0 ) represents the abscissa and ordinate of the target point
  • e is a natural constant
  • f(x, y) Represents the heat value of the pixel point
  • ⁇ represents the range parameter, which is used to control the size of the response area on the coarse target heat map and the fine target heat map
  • M represents the preset heat peak value, which is used to control the heat map peak value.
  • Step S43 Use the deep learning model to detect target points on the sample image, and obtain a heat map of coarse positioning and a heat map of fine positioning.
  • the deep learning model may be a fully convolutional neural network (Fully Convolutional Neural Networks), and a fully convolutional neural network may also be referred to as a fully convolutional network (Fully Convolutional Networks, FCN).
  • FCN Fully Convolutional Networks
  • Step S44 Adjust the network parameters of the deep learning model based on the difference between the coarse target heat map and the coarse positioning heat map and the difference between the fine target heat map and the fine positioning heat map.
  • the difference between the coarse target heat map and the coarse positioning heat map may include: a position difference between the points with the largest heat value in the heat map, a lower heat value in the heat map greater than the third heat range
  • the size difference between the regions of the limit For example, a mean square error function and a cross-entropy function can be used to process the above differences to obtain a first loss value corresponding to the difference between the coarse target heat map and the coarse positioning heat map.
  • the difference between the fine target heat map and the fine positioning heat map may include: the position difference between the points with the largest heat value in the heat map, the size between the areas in the heat map where the heat value is greater than the lower limit value of the fourth heat range difference.
  • a mean square error function and a cross-entropy function can be used to process the above differences to obtain a second loss value corresponding to the difference between the fine target heat map and the fine location heat map.
  • the difference between the coarse target heat map and the coarse positioning heat map, and the difference between the fine target heat map and the fine positioning heat map may also be weighted to obtain the total difference.
  • the above-mentioned first loss value and second loss value may be weighted to obtain the loss value of the deep learning model.
  • methods such as Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), Mini-Batch Gradient Descent (MBGD), etc.
  • the network parameters of the learned model are adjusted.
  • batch gradient descent means that all samples are used to update parameters at each iteration; stochastic gradient descent means that one sample is used to update parameters at each iteration; mini-batch gradient descent means that at each iteration When , a batch of samples is used to update the parameters, which will not be repeated here.
  • the network parameters of the deep learning model may include: weights, biases and the like of hidden layer neurons.
  • a training end condition can also be set, and when the training end condition is satisfied, the training of the deep learning model can be ended.
  • the training end condition may include: the loss value of the deep learning model is smaller than the preset loss threshold, and the loss value is no longer reduced; and/or, the current training times reaches the preset times threshold (the preset times threshold is, for example, 500 times, 1000 times, etc.), which is not limited here.
  • the coarse target heat map and the fine target heat map are generated by using the real position information of the target point, and the heat value of the third area including the target point in the coarse target heat map is within the third heat value range, and the fine target heat map is The heat value of the fourth area including the target point in the target heat map is within the range of the fourth heat value, and the third area is larger than the fourth area. Therefore, the deep learning model is used to detect the target point of the sample image, and the heat map of coarse positioning and fine positioning are obtained. Heat map, based on the difference between the coarse target heat map and the coarse positioning heat map and the difference between the fine target heat map and the fine positioning heat map, adjusting the network parameters of the deep learning model can help the deep learning model to generate accurate heat maps. The coarse positioning heat map and the fine positioning heat map can help to improve the accuracy and stability of point positioning.
  • FIG. 5 is a schematic flowchart of step S13 in the point positioning method according to an embodiment of the present disclosure. As shown in Figure 5, the following steps may be included:
  • Step S131 Determine the position of the first target point based on the heat value of the rough positioning heat map.
  • the point with the largest heat value in the rough positioning heat map may be used as the position of the first target point.
  • a preset point in the first region (for example, the center of gravity of the first region) in the rough positioning heat map may also be used as the position of the first target point, which is not limited herein.
  • the difficulty of determining the position of the target point can be reduced and the speed of point positioning can be improved.
  • a first confidence level of the position of the first target point may also be determined.
  • at least one reference heat value may be obtained, for example, one reference heat value, two reference heat values, three reference heat values, etc. For each reference heat value, obtain the size of the reference area whose heat value is greater than the reference heat value from the rough positioning heat map, and obtain the first target point position based on the size of each reference area and the heat value of the first target point position. a confidence level.
  • the at least one reference heat value may be obtained by obtaining at least one magnification (eg, 0.2, 0.4, 0.6, 0.8, etc.), and comparing the at least one magnification (eg, 0.2, 0.4, 0.6, 0.8, etc.) with the first target, respectively
  • the product between the heat values of the point positions is used as at least one reference heat value.
  • the reference area includes the location of the target point; the size of the reference area includes the perimeter and area of the reference area, and the reference area is obtained based on the size of each reference area and the heat value of the location of the target point
  • the confidence of the target point position may include: obtaining a first ratio between the area of each reference area and the square of the perimeter, using the sum of the first ratios of at least one reference area, the heat value of the first target point position, and The preset heat peak value is obtained to obtain the first confidence level of the position of the first target point.
  • the first confidence level can be expressed as:
  • confidence represents the first confidence level
  • m represents the heat value of the position of the first target point
  • M represents the preset heat peak value
  • K represents the number of reference areas
  • s i represents the area of the ith reference area
  • a coarse target heat map may also be obtained based on the position of the first target point in the coarse positioning heat map, and based on the coarse positioning heat map The similarity of the heat value distribution between the image and the rough target heat map is obtained, and the first confidence level of the position of the first target point is obtained.
  • the heat value of each pixel in the coarse target heat map may be obtained by using a two-dimensional Gaussian function based on the position of the first target point in the coarse positioning heat map, wherein the exponent of the two-dimensional Gaussian function includes a range parameter, and the exponent There is a negative correlation between the absolute value of , and the range parameter; the range parameter in the two-dimensional Gaussian function corresponding to the coarse positioning heat map is greater than the range parameter in the two-dimensional Gaussian function corresponding to the fine positioning heat map .
  • the correlation coefficient between the coarse positioning heat map and the coarse target heat map may be obtained based on the heat value distribution between the coarse positioning heat map and the coarse target heat map, as the first target point position Confidence.
  • the confidence of the target point position or based on the similarity of the heat value distribution between the positioning heat map and the target heat map, the confidence of the target point position can be obtained, so it can improve the accuracy of the confidence, and at the same time, it can facilitate the subsequent biasing of the confidence.
  • the low area is repositioned and completed.
  • Step S132 Determine the position of the second target point and the confidence level of the position of the second target point based on the heat value of the fine positioning heat map.
  • the point with the largest heat value in the fine positioning heat map may be used as the second target point position.
  • a preset point in the second region (eg, the center of gravity of the second region) in the fine positioning heat map may also be used as the position of the second target point, which is not limited herein.
  • the difficulty of determining the position of the target point can be reduced and the speed of point positioning can be improved.
  • At least one reference heat value may be obtained, and for each reference heat value, from The size of the reference area whose heat value is greater than the reference heat value is obtained from the fine positioning heat map, so that the second confidence level of the position of the second target point is obtained based on the size of each reference area and the position of the second target point.
  • at least one reference heat value may be obtained by multiplying the at least one magnification and the heat value at the second target point respectively.
  • the reference area includes the location of the target point; the size of the reference area includes the perimeter and area of the reference area, and the reference area is obtained based on the size of each reference area and the heat value of the location of the target point
  • the confidence of the target point position may include: obtaining a first ratio between the area of each reference area and the square of the perimeter, using the sum of the first ratios of at least one reference area, the heat value of the second target point position, and Presetting the heat peak value to obtain the second confidence level of the position of the second target point, for details, please refer to the above-mentioned relevant steps, which will not be repeated here.
  • a fine target heat map may also be obtained based on the second target point position of the fine positioning heat map, and the fine target heat map may be obtained based on the fine positioning heat map and the fine target heat
  • the similarity of the heat value distribution between the graphs is obtained to obtain the confidence of the position of the second target point.
  • a two-dimensional Gaussian function may be used to obtain the heat value of each pixel in the fine-target heat map based on the position of the second target point in the fine-positioning heat map, wherein the exponent of the two-dimensional Gaussian function includes a range parameter, and the exponent of the exponent There is a negative correlation between the absolute value and the range parameter.
  • the range parameter in the two-dimensional Gaussian function corresponding to the coarse positioning heat map is larger than the range parameter in the two-dimensional Gaussian function corresponding to the fine positioning heat map.
  • the correlation coefficient between the fine positioning heat map and the fine target heat map may be obtained based on the similarity of the heat value distribution between the fine positioning heat map and the fine target heat map, as the second target point position
  • the second confidence level reference may be made to the foregoing related steps for details, and details are not repeated here.
  • the confidence of the target point position or based on the similarity of the heat value distribution between the positioning heat map and the target heat map, the confidence of the target point position can be obtained, so it can improve the accuracy of the confidence, and at the same time, it can facilitate the subsequent biasing of the confidence.
  • the low area is repositioned and completed.
  • the first target point in order to make the second target point position near the first target point position under the condition that the position of the first target point is relatively accurate, it is also possible to determine the first target point based on the heat value of the rough positioning heat map. a confidence level of the position of the target point, and when the confidence level of the position of the first target point satisfies the confidence condition of the coarse position, adjust the heat value in the heat map of the fine positioning outside the preset distance range of the position of the first target point to the preset heat value value, and the preset heat value is outside the second heat value range.
  • the preset heat value is set to 0, which is not limited here.
  • the position of the second target point and the confidence level of the position of the second target point may be determined.
  • the coarse position confidence condition may include that the confidence of the position of the first target point is greater than a first preset threshold (eg, 0.5, etc.).
  • the first target point position determined based on the coarse location heat map is basically accurate, so the second target point location and the second target point location can be obtained by combining the first target point location and the fine location heat map location.
  • the confidence level of the target point position is used for subsequent judgment, and the second target point position is located near the first target point position.
  • the position of the first target point and the first confidence level may also be directly output, which is not limited herein.
  • steps S131 and S132 may be performed in sequence, for example, step S131 is performed first and then step S132 is performed, or step S132 is performed first, and then step S131 is performed.
  • the foregoing step S131 and step S132 may also be performed simultaneously, which is not limited herein.
  • Step S133 Determine whether the confidence level of the position of the second target point satisfies the position confidence condition; if yes, execute step S134; otherwise, execute step S135.
  • the fine-tuned position confidence condition may include at least one of the following: the confidence level of the position of the second target point (ie, the second confidence level) is greater than a second preset threshold (eg, 0.5), the second target point The confidence of the position is greater than the confidence of the position of the first target point (ie, the first confidence), which can help to screen the position information of the target point with better confidence, which can help to improve the accuracy and stability of point positioning .
  • a second preset threshold eg, 0.5
  • step S134 can be performed, that is, the position of the second target point is taken as the position information of the target point; on the contrary, in order to ensure the stability of point positioning, step S135 can be performed, that is, the position of the first target point can be taken as the target point location information.
  • Step S134 Take the second target point position as the position information of the target point.
  • the position of the second target point may be used as the position information of the target point.
  • the confidence of the position of the second target point may be used as the confidence of the corresponding position information, and the position information of the target point and the corresponding position information can be output. Confidence of location information.
  • Step S135 Take the position of the first target point as the position information of the target point.
  • the position of the first target point can be used as the position information of the target point.
  • the confidence of the position of the first target point may be used as the confidence of the corresponding position information, and the position information of the position of the target point and The confidence level of the corresponding location information.
  • the point positioning process can be ended.
  • the first preset reliability threshold and the second preset reliability threshold may be set according to actual conditions, which are not limited herein.
  • the above objective reasons can also be output as a reminder, so as to avoid erroneous positioning with excessive deviation when it cannot be accurately positioned, and also facilitate subsequent completion.
  • the position of the first target point is determined by the coarse positioning heat map
  • the confidence of the position of the second target point and the position of the second target point is determined by fine positioning the heat value of the heat map, and when the second target point is obtained.
  • the position of the second target point is used as the position information of the target point
  • the position of the first target point is used as the position information of the target point.
  • the position information of the target point it can be beneficial to select the position information of the target point with better confidence, which can help to improve the accuracy and stability of the point positioning.
  • FIG. 6 is a schematic diagram of a frame of a point positioning device according to an embodiment of the present disclosure.
  • the positioning device 60 includes: an image acquisition module 61, a target detection module 62 and a position analysis module 63.
  • the image acquisition module 61 is configured to acquire an image to be positioned; the target detection module 62 is configured to perform target point detection on the image to be positioned.
  • the location analysis module 63 is configured to combine and analyze the coarse positioning heat map and the fine positioning heat map to obtain the position information of the target point.
  • a coarse positioning heat map and a fine positioning heat map are obtained, and the heat value of the first area including the target point in the coarse positioning heat map is within the first heat value range.
  • the heat value of the second area containing the target point in the fine positioning heat map is within the second heat value range
  • the first area of the coarse positioning heat map is larger than the second area of the fine positioning heat map, so the coarse positioning heat map can represent the target point.
  • the target response in a relatively large range nearby, and the fine positioning heat map can represent the target response in a small range near the target point, so combining the analysis of the coarse positioning heat map and the fine positioning heat map, the positioning of the coarse positioning heat map can be combined at the same time.
  • the stability and accuracy of the heat map for fine positioning can improve the accuracy and stability of point positioning.
  • the position analysis module 63 is configured to obtain the first target point position in the coarse positioning heat map and the confidence level of the second target point position and the second target point position in the fine positioning heat map, and based on the second target point position The confidence of the position of the target point, the position information of the target point is obtained by processing the position of the first target point and the position of the second target point.
  • the confidence of the second target point position is based on The position of the first target point and the position of the second target point are processed to obtain the position information of the target point, and then the confidence of the target point of the coarse positioning heat map and the fine positioning heat map can be further determined. position information, so as to further improve the accuracy of point positioning.
  • the location analysis module 63 includes a first analysis sub-module configured to determine the location of the first target point based on the heat value of the rough positioning heat map; the location analysis module 63 further includes a second analysis sub-module configured to be based on Finely locate the heat value of the heat map to determine the position of the second target point and the confidence of the position of the second target point; the position analysis module 63 also includes a position selection sub-module, configured to select the second target point based on the confidence of the position of the second target point. The target point position or the first target point position is used as the position information of the target point.
  • the position of the first target point is determined by the coarse positioning heat map
  • the confidence of the position of the second target point and the position of the second target point is determined by the heat map of fine positioning, and based on the position of the second target point. Confidence, select the position of the second target point or the position of the first target point as the position information of the target point.
  • the confidence of the position of the second target point can be determined based on the fine positioning heat map.
  • the position information of the selected target point in the first target point position determined by the heat map can be beneficial to improve the stability of point positioning.
  • the position selection sub-module includes a condition determination unit configured to determine whether the confidence of the position of the second target point satisfies the fine-tuned position confidence condition; the position selection sub-module further includes a position determination unit, configured to determine the condition determination unit in the When the confidence of the position of the second target point satisfies the position confidence condition, the position of the second target point is used as the position information of the target point, and the condition determination unit is further configured to determine that the confidence of the position of the second target point is not satisfied When the position information condition is finely determined, the position of the first target point is used as the position information of the target point.
  • the position of the second target point is used as the position information of the target point, and when the confidence of the position of the second target point does not meet the detailed position confidence condition.
  • the position of the first target point is used as the position information of the target point, so it can help to select the position information of the target point with better confidence, which can help to improve the accuracy and stability of point positioning.
  • the first analysis sub-module is further configured to determine the confidence level of the position of the first target point based on the heat value of the rough positioning heat map; the second analysis sub-module includes an adjustment unit, configured to be at the position of the first target point When the confidence level of the first target point satisfies the rough location confidence condition, the heat value in the fine positioning heat map that is outside the preset distance range of the first target point position is adjusted to the preset heat value; wherein, the preset heat value is within the second heat value range. outside.
  • the confidence of the position of the first target point satisfies the condition of coarse position confidence, directly adjust the heat value in the heat map of fine positioning outside the preset distance range of the position of the first target point to the preset heat value , to determine the confidence of the second target point position and the second target point position based on the heat value of the fine positioning heat map after the adjustment of the fine positioning heat map, so that the second target point position can be located at the first target point position nearby, further improving the accuracy of point localization.
  • the coarse position confidence condition includes that the confidence of the first target point position is greater than a first preset threshold
  • the fine position confidence condition includes at least one of the following: the confidence of the second target point position is greater than a second preset threshold , the confidence of the position of the second target point is greater than the confidence of the position of the first target point.
  • the coarse position confidence condition is set to include the confidence of the first target point position is greater than the first preset threshold
  • the fine position confidence condition is set to include the confidence of the second target point position is greater than the second preset. Threshold, the confidence of the position of the second target point is greater than at least one of the confidence of the position of the first target point, which can help to filter the position information of the target point with better confidence, which can help improve the accuracy of point positioning and stability.
  • the position analysis module 63 (specifically including the first analysis sub-module and the second analysis sub-module) is configured to: take the point with the largest heat value in the positioning heat map as the target point position, or use the position in the positioning heat map The area preset point is used as the target point position.
  • the difficulty of determining the target point position can be reduced, and the point positioning can be improved. speed.
  • the location analysis module 63 (specifically including the first analysis sub-module and the second analysis sub-module) is configured to obtain at least one reference heat value; for each reference heat value, obtain the heat value from the positioning heat map The size of the reference area larger than the reference heat value; the confidence of the target point position is obtained based on the size of each reference area and the heat value of the target point position; or, it is configured to obtain the target heat map based on the target point position of the positioning heat map; based on The similarity of the heat value distribution between the positioning heat map and the target heat map is obtained, and the confidence of the target point position is obtained.
  • the size of the reference area whose heat value is greater than the reference heat value is obtained from the positioning heat map, so as to be based on the size of each reference area and the target.
  • the heat value of the point position can obtain the confidence of the target point position, or based on the similarity of the heat value distribution between the positioning heat map and the target heat map, the confidence of the target point position can be obtained, so the accuracy of the confidence can be improved, and at the same time it can be It is convenient for subsequent relocation and completion of areas with low confidence.
  • the position analysis module 63 is configured to obtain at least one magnification, and use the product between the at least one magnification and the heat value of the target point position as at least one reference heat value, and the reference area includes the target point position;
  • the size includes the perimeter and the area of the reference region, is configured to obtain a first ratio between the area of each reference region and the square of the perimeter, using the sum of the first ratios of the at least one reference region, the heat value of the target point location, and Preset the heat peak value to get the confidence of the target point position.
  • the product between the at least one magnification and the heat value of the target point position is used as at least one reference heat value, so the reference heat value can be determined conveniently and quickly, which is beneficial to improve point positioning. and obtain the target by obtaining the first ratio between the area of each reference area and the square of the perimeter, using the sum of the first ratios of at least one reference area, the heat value of the target point location, and the preset heat value
  • the confidence of the position of the point can be accurately determined to obtain the confidence of the position of the target point.
  • the position analysis module 63 is configured to obtain the heat value of each pixel in the target heat map by using a two-dimensional Gaussian function based on the position of the target point in the positioning heat map, wherein the exponent of the two-dimensional Gaussian function includes a range parameter, And there is a negative correlation between the absolute value of the index and the range parameter, and the range parameter in the two-dimensional Gaussian function corresponding to the coarse positioning heat map is larger than the range parameter in the two-dimensional Gaussian function corresponding to the fine positioning heat map; it is also configured to be based on positioning The heat value distribution between the heat map and the target heat map, and the correlation coefficient between the positioning heat map and the target heat map is obtained as the confidence of the target point location.
  • the heat value of each pixel in the target heat map is obtained by using a two-dimensional Gaussian function, and the index of the two-dimensional Gaussian function includes a range parameter, and the absolute value of the index and range. There is a negative correlation between the parameters.
  • the range parameter in the two-dimensional Gaussian function corresponding to the coarse positioning heat map is larger than the range parameter in the two-dimensional Gaussian function corresponding to the fine positioning heat map, so the target heat map can be obtained conveniently and accurately, thereby Based on the heat value distribution between the positioning heat map and the target heat map, the correlation coefficient between the positioning heat map and the target heat map is obtained as the confidence of the target point position, so the confidence of the target point position can be easily and accurately obtained.
  • the point positioning device 60 further includes a preprocessing module, configured to pre-process the image to be positioned before the target detection module 62 performs target point detection on the image to be positioned to obtain a heat map of coarse positioning and a heat map of fine positioning. processing; wherein the preprocessing includes at least one of the following: normalizing the to-be-located image, and enhancing the image contrast of the to-be-located image.
  • the preprocessing includes normalizing the to-be-located image, and/or enhancing the image contrast of the to-be-located image, there can be a It is beneficial to improve the accuracy of subsequent target point detection.
  • the point positioning device 60 further includes an output module configured to, after the position analysis module 63 combines and analyzes the coarse positioning heat map and the fine positioning heat map to obtain the position information of the target point, The position information of the target point and the confidence level of the corresponding position information are output.
  • the preprocessing module includes a normalization sub-module configured to set a pixel value greater than the first pixel value in the image to be positioned as the first pixel value, and set a pixel value in the image to be positioned smaller than the second pixel value as the first pixel value The pixel value is set to the second pixel value; wherein, among the sequentially arranged pixel values of the image to be positioned, the pixel value in the first numerical ranking is the first pixel value, and the pixel value in the second numerical ranking is the second pixel value. Pixel values.
  • the pixel value in the first numerical ranking is the first pixel value
  • the pixel positioned in the second numerical ranking is the second pixel value
  • the target detection module 62 is configured to use a deep learning model to perform target point detection on the image to be positioned to obtain a coarse positioning heat map and a fine positioning heat map.
  • the hardware parallel acceleration can be fully utilized to reduce the complexity of target point detection.
  • the deep learning model is a fully convolutional neural network
  • the point positioning device 60 further includes a model training module, specifically including: a sample acquisition sub-module configured to acquire sample images, wherein the sample images are marked with The real position information of the target point; the heat map generation sub-module is configured to use the real position information of the target point to generate a coarse target heat map and a fine target heat map; wherein, the coarse target heat map includes the heat of the third area of the target point The value is in the third heat value range; the heat value of the fourth area including the target point in the fine target heat map is in the fourth heat value range, where the third area is larger than the fourth area; the target detection sub-module is configured to use deep learning
  • the model performs target point detection on the sample image, and obtains the coarse positioning heat map and the fine positioning heat map; the parameter adjustment sub-module is configured to be based on the difference between the coarse target heat map and the coarse positioning heat map and the fine target heat map and fine positioning heat map. The difference between the graph
  • the coarse target heat map and the fine target heat map are generated by using the real position information of the target point, and the heat value of the third area including the target point in the coarse target heat map is within the third heat value range, and the fine target heat map is The heat value of the fourth area including the target point in the target heat map is within the range of the fourth heat value, and the third area is larger than the fourth area. Therefore, the deep learning model is used to detect the target point of the sample image, and the heat map of coarse positioning and fine positioning are obtained. Heat map, based on the difference between the coarse target heat map and the coarse positioning heat map and the difference between the fine target heat map and the fine positioning heat map, adjusting the network parameters of the deep learning model can help the deep learning model to generate accurate heat maps. The coarse positioning heat map and the fine positioning heat map can help to improve the accuracy and stability of point positioning.
  • the image to be located is an X-ray image; in the first area and the second area, the heat value closer to the target point is higher; the heat value outside the first area in the rough positioning heat map is lower than the heat value in the first area and the second area.
  • a lower limit value of a range of heat values, and heat values located outside the second area in the fine positioning heat map are lower than the lower limit value of the second range of heat values.
  • FIG. 7 is a schematic frame diagram of an electronic device according to an embodiment of the present disclosure.
  • the electronic device 70 includes a memory 71 and a processor 72 coupled to each other, and the processor 72 is configured to execute program instructions stored in the memory 71 to implement the steps in any of the above-mentioned embodiments of the point positioning method.
  • the electronic device 70 may include, but is not limited to, a microcomputer and a server.
  • the electronic device 70 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
  • the processor 72 is configured to control itself and the memory 71 to implement the steps in any of the above-mentioned embodiments of the point positioning method.
  • the processor 72 may also be referred to as a central processing unit (Central Processing Unit, CPU).
  • the processor 72 may be an integrated circuit chip with signal processing capability.
  • the processor 72 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the processor 72 may be co-implemented by an integrated circuit chip.
  • the above solution can have both the positioning stability of the coarse positioning heat map and the accuracy of the fine positioning heat map at the same time, thereby improving the accuracy and stability of point positioning.
  • FIG. 8 is a schematic diagram of a framework of a computer-readable storage medium according to an embodiment of the present disclosure.
  • the computer-readable storage medium 80 stores program instructions 801 that can be executed by the processor, and the program instructions 801 are used to implement the steps in any of the foregoing embodiments of the point positioning method.
  • It can have both the positioning stability of the coarse positioning heat map and the accuracy of the fine positioning heat map at the same time, so that the accuracy and stability of point positioning can be improved.
  • Embodiments of the present disclosure also provide a computer program, the computer program includes computer-readable codes, and when the computer-readable codes are executed in an electronic device, the processor of the electronic device executes the code to implement the above point positioning method.
  • the disclosed method and apparatus may be implemented in other manners.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other divisions.
  • units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.
  • Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed over network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

一种点定位方法及相关装置、设备，其中，点定位方法包括：获取待定位图像（S11）；对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图（S12），其中，粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，其中，第一区域大于第二区域；结合分析粗定位热度图和细定位热度图，得到目标点的位置信息（S13）。

Description

点定位方法及相关装置、设备、介质及计算机程序
相关申请的交叉引用
本公开基于申请号为202011182566.2、申请日为2020年10月29日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本公开。
技术领域
本公开涉及计算机视觉技术领域，特别涉及一种点定位方法及相关装置、设备、介质及计算机程序。
背景技术
在计算机视觉领域中，对于图像中目标点的精确识别，具有重要意义。以医学影像为例，通过定位下肢影像中某些目标点，能够在极大程度上辅助医务人员，例如，依据膝关节中心点与下肢力线的相对位置，可以辅助分析患者有无膝外翻或膝内翻；或者，依据膝关节中心点与下肢力线，还可以辅助分析胫骨高位截骨、股骨高端截骨等手术的术前规划、术后评估。
发明内容
本公开实施例期望提供一种点定位方法及相关装置、设备、介质及计算机程序。
本公开实施例第一方面提供了一种点定位方法，包括：获取待定位图像；对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，其中，粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，其中，第一区域大于第二区域；结合分析粗定位热度图和细定位热度图，得到目标点的位置信息。
在一些可选实施例中，所述结合分析粗定位热度图和细定位热度图，得到目标点的位置信息，包括：获取粗定位热度图中的第一目标点位置以及细定位热度图中的第二目标点位置和第二目标点位置的置信度，并基于第二目标点位置的置信度，对第一目标点位置和第二目标点位置进行处理得到目标点的位置信息。
在一些可选实施例中，所述获取粗定位热度图中的第一目标点位置以及细定位热度图中的第二目标点位置和第二目标点位置的置信度，并基于第二目标点位置的置信度，对第一目标点位置和第二目标点位置进行处理得到目标点的位置信息，包括：基于粗定 位热度图的热度值，确定第一目标点位置；基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度；并基于第二目标点位置的置信度，选择第二目标点位置或第一目标点位置，作为目标点的位置信息。
在一些可选实施例中，所述基于第二目标点位置的置信度，选择第二目标点位置或第一目标点位置，作为目标点的位置信息包括：若第二目标点位置的置信度满足细定位置信条件，则将第二目标点位置作为目标点的位置信息；若第二目标点位置的置信度不满足细定位置信条件，则将第一目标点位置作为目标点的位置信息。
在一些可选实施例中，所述基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度之前，还包括：基于粗定位热度图的热度值，确定第一目标点位置的置信度，若第一目标点位置的置信度满足粗定位置信条件，则将细定位热度图中位于第一目标点位置的预设距离范围外的热度值调整为预设热度值；其中，预设热度值在第二热度值范围之外。
在一些可选实施例中，所述粗定位置信条件包括第一目标点位置的置信度大于第一预设阈值，细定位置信条件包括以下至少一者：第二目标点位置的置信度大于第二预设阈值，第二目标点位置的置信度大于第一目标点位置的置信度。
在一些可选实施例中，所述基于粗定位热度图的热度值，确定第一目标点位置，或者基于细定位热度图的热度值，确定第二目标点位置，包括：将定位热度图中热度值最大的点作为目标点位置，或者将定位热度图中的区域预设点作为目标点位置。
在一些可选实施例中，所述基于粗定位热度图的热度值，确定第一目标点位置的置信度，或者，基于细定位热度图的热度值，确定第二目标点位置的置信度，包括：获取至少一个参考热度值；针对每个参考热度值，从定位热度图中获取热度值大于参考热度值的参考区域的尺寸；并基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度；或者，基于定位热度图的目标点位置得到目标热度图；基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度。
在一些可选实施例中，所述获取至少一个参考热度值，包括：获取至少一个倍率，将至少一个倍率分别与目标点位置的热度值之间的乘积作为至少一个参考热度值；参考区域包含目标点位置；参考区域的尺寸包括参考区域的周长和面积；基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度，包括：获取每个参考区域的面积与周长的平方之间的第一比例，利用至少一个参考区域的第一比例之和、目标点位置的热度值以及预设热度峰值，得到目标点位置的置信度。
在一些可选实施例中，所述基于定位热度图的目标点位置得到目标热度图，包括：基于定位热度图的目标点位置，利用二维高斯函数获得目标热度图中各像素点的热度值，其中，二维高斯函数的指数包含范围参数，且指数的绝对值与范围参数之间为负相关关系，粗定位热度图对应的二维高斯函数中的范围参数大于细定位热度图对应的二维高斯函数中的范围参数；基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度，包括：基于定位热度图与目标热度图之间的热度值分布，获得定 位热度图和目标热度图之间的相关系数，以作为目标点位置的置信度。
在一些可选实施例中，所述在对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图之前，还包括：对待定位图像进行预处理；其中预处理包括以下至少一种：将待定位图像进行归一化，对待定位图像进行增强图像对比度。
在一些可选实施例中，所述在结合分析粗定位热度图和细定位热度图，得到目标点的位置信息之后，还包括：输出目标点的位置信息和对应位置信息的置信度。
在一些可选实施例中，所述将待定位图像进行归一化，包括：将待定位图像中大于第一像素值的像素值设置为第一像素值，并将待定位图像中小于第二像素值的像素值设置为第二像素值；其中，在待定位图像的顺序排列的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值。
在一些可选实施例中，所述对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，包括：利用深度学习模型对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图。
在一些可选实施例中，所述深度学习模型为全卷积神经网络，和/或，所述深度学习模型至少由以下步骤训练得到：获取样本图像，其中，样本图像标注有目标点的真实位置信息；利用目标点的真实位置信息，生成粗目标热度图和细目标热度图；其中，粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围；细目标热度图中包含目标点的第四区域的热度值在第四热度值范围，其中，第三区域大于第四区域；利用深度学习模型对样本图像进行目标点检测，得到粗定位热度图和细定位热度图；基于粗目标热度图与粗定位热度图之间的差异和细目标热度图和细定位热度图之间的差异，调整深度学习模型的网络参数。
在一些可选实施例中，所述待定位图像为X线图像；在第一区域和第二区域中，越靠近目标点的热度值越高；粗定位热度图中位于第一区域以外的热度值低于第一热度值范围的下限值，细定位热度图中位于第二区域以外的热度值低于第二热度值范围的下限值。
本公开实施例第二方面提供了一种点定位装置，包括：图像获取模块、目标检测模块和位置分析模块，图像获取模块，配置为获取待定位图像；目标检测模块，配置为对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，其中，粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，其中，第一区域大于第二区域；位置分析模块，配置为结合分析粗定位热度图和细定位热度图，得到目标点的位置信息。
本公开实施例第三方面提供了一种电子设备，包括相互耦接的存储器和处理器，处理器用于执行存储器中存储的程序指令，以实现上述第一方面中的点定位方法。
本公开实施例第四方面提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述第一方面中的点定位方法。
本公开实施例第五方面提供了一种计算机程序，所述计算机程序包括计算机可读代 码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现上述第一方面中的点定位方法。
采用本公开实施例的技术方案，通过对获取到的待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，且粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围，细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，粗定位热度图的第一区域大于细定位热度图的第二区域，故粗定位热度图能够表示目标点附近较大范围内的目标响应，而细定位热度图能够表示目标点附近较小范围内的目标响应，故结合分析粗定位热度图和细定位热度图，能够同时兼具粗定位热度图的定位稳定性以及细定位热度图的准确性，从而能够提高点定位的准确性和稳定性。
附图说明
图1是本公开实施例的点定位方法的流程示意图；
图2是本公开实施例中的待定位图像的定位结果的可选示意图；
图3是本公开实施例中的粗定位热度图和细定位热度图的示意图；
图4是本公开实施例中对深度学习模型进行训练的流程示意图；
图5是本公开实施例的点定位方法中步骤S13的流程示意图；
图6是本公开实施例的点定位装置的框架示意图；
图7是本公开实施例的电子设备的框架示意图；
图8是本公开实施例的计算机可读存储介质的框架示意图。
具体实施方式
下面结合说明书附图，对本申请实施例的方案进行详细说明。
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、接口、技术之类的具体细节，以便透彻理解本申请。
本文中术语“系统”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。此外，本文中的“多”表示两个或者多于两个。
图1是本公开实施例的点定位方法的流程示意图，如图1所示，点定位方法可以包括如下步骤：
步骤S11：获取待定位图像。
示例性的，待定位图像可以是包括人脸部五官的图像，以通过本实施例的点定位方法对图像中的人脸部双眼、口、鼻等目标点进行定位，以便后续用于人脸识别等应用场景；或者，待定位图像还可以是包括人体组织、器官的影像图像，以通过本实施例的点 定位方法对影像图像中的人体组织、器官中的目标点进行定位。例如，待定位图像可以为X线图像，即计算机断层扫描图像，在一个具体的实施场景中，请结合参阅图2，图2是本公开实施例中的待定位图像的定位结果的可选示意图，待定位图像可以为人体下肢(包括左下肢和右下肢)的X线图像，定位得到的目标点可以包括但不限于：股骨头中心、大转子尖、股骨内踝、股骨外踝、胫骨平台内侧端点、胫骨平台外侧端点、踝关节间隙内侧端点、踝关节间隙外侧端点，通过本实施例的点定位方法可得到图2所示的目标点为左下肢和右下肢上的上述8个目标点，即共计16个目标点(图2中黑色填充的圆点)。在其他应用场景中，可以以此类推，在此不再一一举例。
图2中的16个目标点是采用本实施例的点定位方法待进行定位的目标点的示例，待定位图像中并未标注有上述16个目标点，可以理解，图2中去除上述16个黑色填充的圆点后的图像可以是一种待定位图像。
步骤S12：对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图。
粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围，细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，且第一区域大于第二区域。定位热度图可以反映待定位图像中各个像素点的目标响应，在一些可选实施例中，在第一区域和第二区域中，越靠近目标点的热度值越高，即目标响应值越大，粗定位热度图中位于第一区域以外的热度值低于第一热度值范围的下限值，细定位热度图中位于第二区域以外的热度值低于第二热度值范围的下限值，粗定位热度图较细定位热度图在更大的范围上具有高响应值，故能够便捷地确定粗定位热度图和细定位热度图中的目标点，且粗定位热度图能够在后续点定位过程中确保点定位的稳定性，细定位热度图能够在后续点定位过程中确保点定位的准确性。
请结合参阅图3，图3是本公开实施例中的粗定位热度图和细定位热度图的示意图，如图3所示，(a)图为粗定位热度图，(b)图为细定位热度图，为了便于描述，图3所示的粗定位热度图和细定位热度图表示的是对同一目标点的目标响应，粗定位热度图和细定位热度图中的实心圆点及其外围的白色填充区域分别表示包含目标点的第一区域、第二区域，在一个实施场景中，为了便于区分，还可以以光谱颜色的顺序表示热度值，例如可以采用“赤色”表示热度值最大的目标点，随着热度值的降低，分别以“橙色”、“黄色”、“绿色”、“蓝色”等表示逐渐远离目标点的点。
在一些可选实施例中，为了能够充分利用硬件并行加速，降低目标点检测的复杂度，可以利用深度学习模型对待定位图像进行目标点检测，从而得到粗定位热度图和细定位热度图。在一个示例中，所述深度学习模型为全卷积神经网络。例如，深度学习模型可以采用一个含编码器、解码器以及跳跃链接结构的Unet网络。利用深度学习模型对待定位图像进行目标点检测时，可以对每一目标点生成相应的粗定位热度图和细定位热度图。
在一些可选实施例中，为了提高目标点检测的准确性，还可以在进行目标点检测、得到粗定位热度图和细定位热度图之前，对所述待定位图像进行预处理；其中，所述预 处理可包括对待定位图像进行归一化处理。示例性的，所述将所述待定位图像进行归一化，可以包括：将待定位图像中大于第一像素值的像素值设置为第一像素值，并将待定位图像中小于第二像素值的像素值设置第二像素值，其中，在待定位图像的顺序排列的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值。例如，将待定位图像的像素点按照像素值从小到大进行排序，取第99％(即位于总数的99％位)排位的像素值为第一像素值，取第3％(即位于总数的3％位)排位的像素值为第二像素值，并将大于第一像素值的像素值设置为第一像素值，将小于第二像素值的像素值设置为第二像素值。在其他实施例中，第一数值排位、第二数值排位还可以根据具体应用进行设置，在此不做限定。
通过将待定位图像中大于第一像素值的像素值设置为第一像素值，并将待定位图像中小于第二像素值的热度值设置为第二像素值，且在待定位图像的排序顺序的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值，从而能够有利于剔除待定位图像中诸如特亮或特暗的干扰，进而能够有利于提高后续目标点检测的准确性，提高点定位的准确性。
在一些可选实施例中，为了提高目标点检测的准确性，在所述对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图之前，还可以对待定位图像进行增强图像对比度处理。示例性的，可以采用限制对比度自适应直方图均衡化算法(Contrast Limited Adaptive Histogram Equalization，CLAHE)对待定位图像进行处理，以增强图像的局部对比度。
步骤S13：结合分析粗定位热度图和细定位热度图，得到目标点的位置信息。
在一些可选实施例中，可以获取粗定位热度图中的第一目标点位置及第一目标点位置的置信度，并在第一目标点位置的置信度大于置信度阈值时，将第一目标点位置作为目标点的位置信息，从而能够在精确度要求不高的情况下，快速确定目标点的位置信息。示例性的，可以将粗定位热度图中热度值最高的点作为第一目标点位置。第一目标点位置的置信度用于表示定位到的第一目标点位置的可信度，第一目标点位置的置信度越高，表示第一目标点位置的可信度越高。本公开实施例中，为了便于描述，如无其他特别说明，采用第一置信度表示第一目标点位置的置信度。
在另一些可选实施例中，还可以获取粗定位热度图中的第一目标点位置，并获取细定位热度图中的第二目标点位置及第二目标点位置的置信度，并基于第二目标点位置的置信度，对第一目标点位置和第二目标点位置进行处理，得到目标点的位置信息，进而能够进一步基于粗定位热度图和细定位热度图两者目标点的置信度情况，确定目标点的位置信息，从而能够兼顾点定位的准确性和稳定性，进一步提高点定位的准确性。例如，可以根据第二目标点位置的置信度，选择第一目标点位置或第二目标点位置，作为目标点的位置信息；或者，根据第二目标点位置的置信度，输出包含第一目标点位置和第二目标点位置的位置信息，在此不做限定。示例性的，可以将粗定位热度图中热度值最高的点作为第一目标点位置，将细定位热度图中热度值最高的点作为第二目标点位置。第 二目标点位置的置信度用于表示定位到的第二目标点位置的可信度，第二目标点位置的置信度越高，表示第二目标点位置的可信度越高。本公开实施例中，为了便于描述，如无其他特别说明，采用第二置信度表示第二目标点位置的置信度。
在又一些可选实施例中，在所述结合分析所述粗定位热度图和所述细定位热度图、得到所述目标点的位置信息之后，还可以输出目标点位置的位置信息和对应位置信息的置信度。例如，当将第一目标点位置作为目标点的位置信息时，可以将第一目标点的置信度作为对应位置信息的置信度；或者，当第二目标点位置作为目标点的位置信息时，可以将第二目标点的置信度作为对应位置信息的置信度，从而能够有利于用户评价定位得到的目标点的位置信息，提高用户感知。
采用上述方案，通过对获取到的待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，且粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围，细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，粗定位热度图的第一区域大于细定位热度图的第二区域，故粗定位热度图能够表示目标点附近较大范围内的目标响应，而细定位热度图能够表示目标点附近较小范围内的目标响应，故结合分析粗定位热度图和细定位热度图，能够同时兼具粗定位热度图的定位稳定性以及细定位热度图的准确性，从而能够提高点定位的准确性和稳定性。
图4是本公开实施例中对深度学习模型进行训练的流程示意图，如图4所示，可以包括如下步骤：
步骤S41：获取样本图像，其中，样本图像标注有目标点的真实位置信息。
示例性的，样本图像可以是包括人脸部五官的图像，目标点可以包括：人脸部双眼、口、鼻中的至少之一。或者，样本图像还可以是包括人体组织、器官的影像图像。例如，样本图像可以为X线图像，即计算机断层扫描图像，例如，样本图像可以为人体下肢(包括左下肢和右下肢)的X线图像，目标点可以包括但不限于以下至少之一：股骨头中心、大转子尖、股骨内踝、股骨外踝、胫骨平台内侧端点、胫骨平台外侧端点、踝关节间隙内侧端点、踝关节间隙外侧端点，具体可以参阅前述实施例中的相关步骤，在此不再赘述。
步骤S42：利用目标点的真实位置信息，生成粗目标热度图和细目标热度图。
粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围；细目标热度图中包含目标点的第四区域的热度值在第四热度值范围，其中，第三区域大于第四区域。
示例性的，可以基于样本图像中目标点的真实位置信息，利用二维高斯函数获得粗目标热度图和细目标热度图中各像素点的热度值，其中，二维高斯函数的指数包含范围参数，且指数的绝对与范围参数之间为负相关关系，粗目标热度图对应的二维高斯函数中的范围参数大于细目标热度图对应的二维高斯函数中的范围参数。在一些可选实施例中，目标热度图中各像素点的热度值可以表示为：
Figure PCTCN2021103150-appb-000001
上述公式(1)中，(x,y)表示像素点的横坐标和纵坐标，(x 0,y 0)表示目标点的横坐标和纵坐标，e为自然常数，f(x,y)表示像素点的热度值，σ表示范围参数，用于控制粗目标热度图和细目标热度图上的响应区域的大小，M表示预设热度峰值，用于控制热度图峰值。
步骤S43：利用深度学习模型对样本图像进行目标点检测，得到粗定位热度图和细定位热度图。
在一些可选实施例中，深度学习模型可以为全卷积神经网络(Fully Convolutional Neural Networks)，全卷积神经网络也可称为全卷积网络(Fully Convolutional Networks，FCN)。利用深度学习模型对样本图像进行目标检测，从而得到粗定位热度图和细定位热度图。
步骤S44：基于粗目标热度图与粗定位热度图之间的差异和细目标热度图和细定位热度图之间的差异，调整深度学习模型的网络参数。
在一些可选实施例中，粗目标热度图与粗定位热度图之间的差异可以包括：热度图中热度值最大的点之间的位置差异、热度图中热度值大于第三热度范围的下限值的区域之间的尺寸差异。例如，可以采用均方误差函数、交叉熵函数对上述差异进行处理，得到与粗目标热度图与粗定位热度图之间的差异对应的第一损失值。细目标热度图与细定位热度图之间的差异可以包括：热度图中热度值最大的点之间的位置差异、热度图中热度值大于第四热度范围的下限值的区域之间的尺寸差异。例如，可以采用均方误差函数、交叉熵函数对上述差异进行处理，得到与细目标热度图和细定位热度图之间的差异对应的第二损失值。
在一些可选实施例中，还可以对粗目标热度图与粗定位热度图之间的差异，以及细目标热度图和细定位热度图之间的差异进行加权处理，得到总的差异。例如，可以对上述第一损失值和第二损失值进行加权处理，得到深度学习模型的损失值。
在一些可选实施例中，可以采用随机梯度下降(Stochastic Gradient Descent，SGD)、批量梯度下降(Batch Gradient Descent，BGD)、小批量梯度下降(Mini-Batch Gradient Descent，MBGD)等方式，对深度学习模型的网络参数进行调整。其中，批量梯度下降是指在每一次迭代时，使用所有样本来进行参数更新；随机梯度下降是指在每一次迭代时，使用一个样本来进行参数更新；小批量梯度下降是指在每一次迭代时，使用一批样本来进行参数更新，在此不再赘述。在一些可选实施例中，深度学习模型的网络参数可以包括：隐层神经元的权重、偏置等等。
在一些可选实施例中，还可以设置训练结束条件，当满足训练结束条件时，可以结束对深度学习模型的训练。例如，训练结束条件可以包括：深度学习模型的损失值小于预设损失阈值，且损失值不再减小；和/或，当前训练次数达到预设次数阈值(预设次数阈值例如，500次、1000次等)，在此不做限定。
区别于前述实施例，通过利用目标点的真实位置信息，生成粗目标热度图和细目标 热度图，且粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围，细目标热度图中包含目标点的第四区域的热度值在第四热度值范围，第三区域大于第四区域，从而利用深度学习模型对样本图像进行目标点检测，得到粗定位热度图和细定位热度图，基于粗目标热度图与粗定位热度图之间的差异和细目标热度图和细定位热度图之间的差异，调整深度学习模型的网络参数，能够有利于使深度学习模型生成精确的粗定位热度图和细定位热度图，进而能够有利于提高点定位的准确性和稳定性。
图5是本公开实施例的点定位方法中步骤S13的流程示意图。如图5所示，可以包括如下步骤：
步骤S131：基于粗定位热度图的热度值，确定第一目标点位置。
在一些可选实施例中，可以将粗定位热度图中热度值最大的点作为第一目标点位置。在另一些可选实施例中，还可以将粗定位热度图中的第一区域中的预设点(例如，第一区域的重心)作为第一目标点位置，在此不做限定。
通过将定位热度图中热度值最大的点作为目标点位置，或者将定位热度图中的区域预设点作为目标点位置，能够降低确定目标点位置的难度，提高点定位的速度。
在一些可选实施例中，在确定第一目标点位置时，还可以确定第一目标点位置的第一置信度。示例性的，为了能够准确地确定得到目标点位置的置信度，可以获取至少一个参考热度值，例如，获取1个参考热度值、2个参考热度值、3个参考热度值等等，并针对每个参考热度值，从粗定位热度图中获取热度值大于参考热度值的参考区域的尺寸，并基于每个参考区域的尺寸以及第一目标点位置的热度值得到第一目标点位置的第一置信度。在一些示例中，至少一个参考热度值可以通过获取至少一个倍率(例如，0.2、0.4、0.6、0.8等)、将至少一个倍率(例如，0.2、0.4、0.6、0.8等)分别与第一目标点位置的热度值之间的乘积作为至少一个参考热度值。在另一些示例中，参考区域包含所述目标点位置；参考区域的尺寸包括参考区域的周长和面积，则基于每个所述参考区域的尺寸以及所述目标点位置的热度值得到所述目标点位置的置信度，可以包括：获取每个参考区域的面积与周长的平方之间的第一比例，利用至少一个参考区域的第一比例之和、第一目标点位置的热度值以及预设热度峰值，得到第一目标点位置的第一置信度。例如，第一置信度可以表示为：
Figure PCTCN2021103150-appb-000002
上述公式(2)中，confidence表示第一置信度，m表示第一目标点位置的热度值，M表示预设热度峰值，K表示参考区域的数量，s i表示第i个参考区域的面积，
Figure PCTCN2021103150-appb-000003
表示第i个参考区域周长的平方。
在另一些可选实施例中，为了方便、准确地确定第一目标点位置的第一置信度，还可以基于粗定位热度图的第一目标点位置得到粗目标热度图，并基于粗定位热度图和粗目标热度图之间的热度值分布相似度，得到第一目标点位置的第一置信度。在一些示例 中，可以基于粗定位热度图的第一目标点位置，利用二维高斯函数获得粗目标热度图中各像素点的热度值，其中，二维高斯函数的指数包含范围参数，且指数的绝对值与范围参数之间为负相关关系；所述粗定位热度图对应的二维高斯函数中的所述范围参数大于所述细定位热度图对应的二维高斯函数中的所述范围参数。目标热度图的获取过程，具体可以参阅前述实施例中的相关步骤，在此不再赘述。在另一些示例中，可以基于粗定位热度图与粗目标热度图之间的热度值分布，获得粗定位热度图和粗目标热度图之间的相关系数，以作为第一目标点位置的第一置信度。
通过获取至少一个参考热度值，并针对每个参考热度值，从定位热度图中获取热度值大于参考热度值的参考区域的尺寸，从而基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度，或者基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度，故能够提高置信度的准确性，同时能够便于后续对置信度偏低的区域进行重新定位、补全。
步骤S132：基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度。
在一些可选实施例中，可以将细定位热度图中热度值最大的点作为第二目标点位置。在另一些可选实施例中，还可以将细定位热度图中的第二区域中的预设点(例如，第二区域的重心)作为第二目标点位置，在此不做限定。
通过将定位热度图中热度值最大的点作为目标点位置，或者将定位热度图中的区域预设点作为目标点位置，能够降低确定目标点位置的难度，提高点定位的速度。
在一些可选实施例中，为了能够方便、准确的确定第二目标点位置的置信度(可记为第二置信度)，可以获取至少一个参考热度值，并针对每个参考热度值，从细定位热度图中获取热度值大于参考热度值的参考区域的尺寸，从而基于每个参考区域的尺寸以及第二目标点位置得到第二目标点位置的第二置信度。在一些示例中，可以通过至少一个倍率分别与第二目标点位置的热度值之间的乘积，得到至少一个参考热度值，具体可以参考前述相关步骤，在此不再赘述。在另一些示例中，参考区域包含所述目标点位置；参考区域的尺寸包括参考区域的周长和面积，则基于每个所述参考区域的尺寸以及所述目标点位置的热度值得到所述目标点位置的置信度，可以包括：获取每个参考区域的面积与周长的平方之间的第一比例，利用至少一个参考区域的第一比例之和、第二目标点位置的热度值以及预设热度峰值，得到第二目标点位置的第二置信度，具体可以参考前述相关步骤，在此不再赘述。
在另一些可选实施例中，为了确定第二目标点位置的第二置信度，还可以基于细定位热度图的第二目标点位置得到细目标热度图，基于细定位热度图与细目标热度图之间的热度值分布相似度，得到第二目标点位置的置信度。在一些示例中，可以基于细定位热度图的第二目标点位置，利用二维高斯函数获得细目标热度图中各个像素点的热度值，其中二维高斯函数的指数包含范围参数，且指数的绝对值与范围参数之间为负相关关系，粗定位热度图对应的二维高斯函数中的范围参数大于细定位热度图对应的二维高 斯函数中的范围参数，具体可以参考前述相关步骤，在此不再赘述。在另一些示例中，可以基于细定位热度图与细目标热度图之间的热度值分布相似度，获得细定位热度图和细目标热度图之间的相关系数，以作为第二目标点位置的第二置信度，具体可以参考前述相关步骤，在此不再赘述。
通过获取至少一个参考热度值，并针对每个参考热度值，从定位热度图中获取热度值大于参考热度值的参考区域的尺寸，从而基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度，或者基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度，故能够提高置信度的准确性，同时能够便于后续对置信度偏低的区域进行重新定位、补全。
在又一些可选实施例中，为了在第一目标点位置定位较为准确的情况下，使第二目标点位置位于第一目标点位置附近，还可以基于粗定位热度图的热度值，确定第一目标点位置的置信度，并在第一目标点位置的置信度满足粗定位置信条件时，将细定位热度图中位于第一目标点位置的预设距离范围外的热度值调整预设热度值，且预设热度值在第二热度值范围之外。例如，将预设热度值设置为0，在此不做限定。在细定位热度图进行上述调整之后，可以基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度。第一目标点位置的置信度、第二目标点位置和第二目标点位置的置信度的确定方式具体可以参阅前述步骤，在此不再赘述。在一个示例中，粗定位置信条件可以包括第一目标点位置的置信度大于第一预设阈值(如，0.5等)。故此，当满足粗定位置信条件时，可以认为基于粗定位热度图确定的第一目标点位置基本准确，故可以结合第一目标点位置和细定位热度图定位得到第二目标点位置和第二目标点位置的置信度，以进行后续判断，并使得第二目标点位置位于第一目标点位置附近。在另一个示例中，当点定位精度要求不高时，还可以直接输出第一目标点位置及第一置信度，在此不做限定。
上述步骤S131和步骤S132可以按照先后顺序执行，例如，先执行步骤S131后执行步骤S132，或者，先执行步骤S132，后执行步骤S131。在另一个实施场景中，上述步骤S131和步骤S132还可以同时执行，在此不做限定。
步骤S133：判断第二目标点位置的置信度是否满足细定位置信条件，若是，则执行步骤S134，否则执行步骤S135。
在一些可选实施例中，细定位置信条件可以包括以下至少一者：第二目标点位置的置信度(即第二置信度)大于第二预设阈值(如，0.5)、第二目标点位置的置信度大于第一目标点位置的置信度(即第一置信度)，这样能够有利于筛选置信度较优的目标点的位置信息，从而能够有利于提高点定位的准确性和稳定性。当第二置信度满足细定位置信条件时，可以认为第二目标点位置的可信度较高，或者第二目标点位置的可信度较第一目标点位置的可信度更高，故为了提高点定位的准确性，可以执行步骤S134，即将第二目标点位置作为目标点的位置信息；反之，为了确保点定位的稳定性，可以执行步骤S135，即将第一目标点位置作为目标点的位置信息。
步骤S134：将第二目标点位置作为目标点的位置信息。
当第二置信度满足细定位置信条件时，可以认为第二目标点位置的可信度较高，或者第二目标点位置的可信度较第一目标点位置的可信度更高，故为了提高点定位的准确性，可以将第二目标点位置作为目标点的位置信息。在一些可选实施例中，在将第二目标点位置作为目标点的位置信息时，可以将第二目标点位置的置信度作为对应位置信息的置信度，并输出目标点的位置信息和对应位置信息的置信度。
步骤S135：将第一目标点位置作为目标点的位置信息。
当第二置信度不满足细定位置信条件时，可以认为第二目标点位置的可信度较低，或者第一目标点位置的可信度较第二目标点位置的可信度更高，故为了确保点定位的稳定性，可以将第一目标点位置作为目标点的位置信息。在一些可选实施例中，在将第一目标点位置作为目标点的位置信息时，可以将第一目标点位置的置信度作为对应位置信息的置信度，并输出目标点位置的位置信息和对应位置信息的置信度。
此外，当粗定位热度图中的第一目标点位置的第一置信度低于第一预设置信度阈值，且第二目标点位置的第二置信度低于第二预设置信度阈值时，可以认为待定位图像存在图像质量过差、不存在目标点等客观原因，无法实现精确定位，故可以结束点定位流程。第一预设置信度阈值和第二预设置信度阈值可以根据实际情况进行设置，在此不做限定。在一个实施场景中，还可以将上述客观原因输出，以做提醒，从而能够避免无法被精确定位时而给出过大偏差的错误定位，同时也能够便于后续补全。
区别于前述实施例，通过粗定位热度图确定得到的第一目标点位置并通过细定位热度图的热度值，确定得到第二目标点位置和第二目标点位置的置信度，并当第二目标点位置的置信度满足细定位置信条件时，将第二目标点位置作为目标点的位置信息，而当第二目标点位置的置信度不满足细定位置信条件时，将第一目标点位置作为目标点的位置信息，故能够有利于选取置信度较优的作为目标点的位置信息，从而能够有利于提高点定位的准确性和稳定性。
图6是本公开实施例的点定位装置的框架示意图。如图6所示，定位装置60包括：图像获取模块61、目标检测模块62和位置分析模块63，图像获取模块61配置为获取待定位图像；目标检测模块62配置为对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，其中，粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，其中，第一区域大于第二区域；位置分析模块63配置为结合分析粗定位热度图和细定位热度图，得到目标点的位置信息。
采用上述方案，通过对获取到的待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，且粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围，细定位热度图中包含目标点的第二区域的热度值在第二热度值范围，粗定位热度图的第一区域大于细定位热度图的第二区域，故粗定位热度图能够表示目标点附近较大范围内的目标响应，而细定位热度图能够表示目标点附近较小范围内的目标响应，故结合分析 粗定位热度图和细定位热度图，能够同时兼具粗定位热度图的定位稳定性以及细定位热度图的准确性，从而能够提高点定位的准确性和稳定性。
在一些实施例中，位置分析模块63配置为获取粗定位热度图中的第一目标点位置以及细定位热度图中的第二目标点位置和第二目标点位置的置信度，并基于第二目标点位置的置信度，对第一目标点位置和第二目标点位置进行处理得到目标点的位置信息。
区别于前述实施例，通过获取粗定位热度图中的第一目标点位置和细定位热度图中的第二目标点位置和第二目标点位置的置信度，从而基于第二目标点位置的置信度，对第一目标点位置和第二目标点位置进行处理，得到目标点的位置信息，进而能够进一步基于粗定位热度图和细定位热度图两者目标点的置信度情况，确定目标点的位置信息，从而能够进一步提高点定位的准确性。
在一些实施例中，位置分析模块63包括第一分析子模块，配置为基于粗定位热度图的热度值，确定第一目标点位置；位置分析模块63还包括第二分析子模块，配置为基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度；位置分析模块63还包括位置选择子模块，配置为基于第二目标点位置的置信度，选择第二目标点位置或第一目标点位置，作为目标点的位置信息。
区别于前述实施例，通过粗定位热度图确定得到的第一目标点位置并通过细定位热度图确定得到第二目标点位置和第二目标点位置的置信度，并基于第二目标点位置的置信度，选择第二目标点位置或第一目标点位置，作为目标点的位置信息，能够基于细定位热度图确定得到的第二目标点位置的置信度，从第一目标点位置和粗定位热度图确定得到的第一目标点位置中选择目标点的位置信息，从而能够有利于提高点定位的稳定性。
其中，位置选择子模块包括条件判断单元，配置为判断第二目标点位置的置信度是否满足细定位置信条件；位置选择子模块还包括位置确定单元，配置为在所述条件判断单元判定所述第二目标点位置的置信度满足细定位置信条件时，将第二目标点位置作为目标点的位置信息，还配置为在所述条件判断单元判定所述第二目标点位置的置信度不满足细定位置信条件时，将第一目标点位置作为目标点的位置信息。
区别于前述实施例，当第二目标点位置的置信度满足细定位置信条件时，将第二目标点位置作为目标点的位置信息，而当第二目标点位置的置信度不满足细定位置信条件时，将第一目标点位置作为目标点的位置信息，故能够有利于选取置信度较优的作为目标点的位置信息，从而能够有利于提高点定位的准确性和稳定性。
在一些实施例中，第一分析子模块还配置为基于粗定位热度图的热度值，确定第一目标点位置的置信度；第二分析子模块包括调整单元，配置为在第一目标点位置的置信度满足粗定位置信条件时，将细定位热度图中位于第一目标点位置的预设距离范围外的热度值调整为预设热度值；其中，预设热度值在第二热度值范围之外。
区别于前述实施例，当第一目标点位置的置信度满足粗定位置信条件时，直接将细定位热度图中位于第一目标点位置的预设距离范围外的热度值调整为预设热度值，以在 细定位热度图进行调整后，基于细定位热度图的热度值，确定第二目标点位置和第二目标点位置的置信度，故能够使第二目标点位置处于第一目标点位置附近，进一步提高点定位的准确性。
在一些实施例中，粗定位置信条件包括第一目标点位置的置信度大于第一预设阈值，细定位置信条件包括以下至少一者：第二目标点位置的置信度大于第二预设阈值，第二目标点位置的置信度大于第一目标点位置的置信度。
区别于前述实施例，将粗定位置信条件设置为包括第一目标点位置的置信度大于第一预设阈值，将细定位置信条件设置为包括第二目标点位置的置信度大于第二预设阈值，第二目标点位置的置信度大于第一目标点位置的置信度中的至少一者，能够有利于筛选置信度较优的目标点的位置信息，从而能够有利于提高点定位的准确性和稳定性。
在一些实施例中，位置分析模块63(具体包括第一分析子模块、第二分析子模块)配置为：将定位热度图中热度值最大的点作为目标点位置，或者将定位热度图中的区域预设点作为目标点位置。
区别于前述实施例，通过将定位热度图中热度值最大的点作为目标点位置，或者将定位热度图中的区域预设点作为目标点位置，能够降低确定目标点位置的难度，提高点定位的速度。
在一些实施例中，位置分析模块63(具体包括第一分析子模块、第二分析子模块)配置为，获取至少一个参考热度值；针对每个参考热度值，从定位热度图中获取热度值大于参考热度值的参考区域的尺寸；基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度；或者，配置为基于定位热度图的目标点位置得到目标热度图；基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度。
区别于前述实施例，通过获取至少一个参考热度值，并针对每个参考热度值，从定位热度图中获取热度值大于参考热度值的参考区域的尺寸，从而基于每个参考区域的尺寸以及目标点位置的热度值得到目标点位置的置信度，或者基于定位热度图与目标热度图之间的热度值分布相似度，得到目标点位置的置信度，故能够提高置信度的准确性，同时能够便于后续对置信度偏低的区域进行重新定位、补全。
在一些实施例中，位置分析模块63配置为获取至少一个倍率，将至少一个倍率分别与目标点位置的热度值之间的乘积作为至少一个参考热度值，参考区域包含目标点位置；参考区域的尺寸包括参考区域的周长和面积，配置为获取每个参考区域的面积与周长的平方之间的第一比例，利用至少一个参考区域的第一比例之和、目标点位置的热度值以及预设热度峰值，得到目标点位置的置信度。
区别于前述实施例，通过获取至少一个倍率，将至少一个倍率分别与目标点位置的热度值之间的乘积作为至少一个参考热度值，故能够方便快捷地确定参考热度值，有利于提高点定位的速度，并通过获取每个参考区域的面积与周长的平方之间的第一比例，利用至少一个参考区域的第一比例之和、目标点位置的热度值以及预设热度峰值，得到目标点位置的置信度，故能够准确地确定得到目标点位置的置信度。
在一些实施例中，位置分析模块63配置为基于定位热度图的目标点位置，利用二维高斯函数获得目标热度图中各像素点的热度值，其中，二维高斯函数的指数包含范围参数，且指数的绝对值与范围参数之间为负相关关系，粗定位热度图对应的二维高斯函数中的范围参数大于细定位热度图对应的二维高斯函数中的范围参数；还配置为基于定位热度图与目标热度图之间的热度值分布，获得定位热度图和目标热度图之间的相关系数，以作为目标点位置的置信度。
区别于前述实施例，通过定位热度图的目标点位置，利用二维高斯函数获得目标热度图中各像素点的热度值，且二维高斯函数的指数包含范围参数，且指数的绝对值与范围参数之间为负相关关系，粗定位热度图对应的二维高斯函数中的范围参数大于细定位热度图对应的二维高斯函数中的范围参数，故能够方便、准确地获得目标热度图，从而基于定位热度图与目标热度图之间的热度值分布，获得定位热度图和目标热度图之间的相关系数，以作为目标点位置的置信度，故能够方便、准确地得到目标点位置的置信度。
在一些实施例中，点定位装置60还包括预处理模块，配置为在所述目标检测模块62对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图之前，对待定位图像进行预处理；其中预处理包括以下至少一种：将待定位图像进行归一化，对待定位图像进行增强图像对比度。
区别于前述实施例，通过在对待定位图像进行目标点检测之前，对待定位图像进行预处理，且预处理包括将待定位图像进行归一化，和/或对待定位图像进行增强图像对比度，能够有利于提高后续目标点检测的准确性。
在一些实施例中，点定位装置60还包括输出模块，配置为在所述位置分析模块63结合分析所述粗定位热度图和所述细定位热度图，得到所述目标点的位置信息之后，输出目标点的位置信息和对应位置信息的置信度。
区别于前述实施例，通过输出目标点的位置信息和对应位置信息的置信度，能够有利于用户评价定位得到的目标点的位置信息，提高用户感知。
在一些实施例中，预处理模块包括归一化子模块，配置为将待定位图像中大于第一像素值的像素值设置为第一像素值，并将待定位图像中小于第二像素值的像素值设置为第二像素值；其中，在待定位图像的顺序排列的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值。
区别于前述实施例，通过将待定位图像中大于第一像素值的像素值设置为第一像素值，并将待定位图像中小于第二像素值的热度值设置为第二像素值，且在待定位图像的排序顺序的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值，从而能够有利于剔除待定位图像中诸如特亮或特暗的干扰，进而能够有利于提高后续目标点检测的准确性，提高点定位的准确性。
在一些实施例中，目标检测模块62配置为利用深度学习模型对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图。
区别于前述实施例，通过利用深度学习模型对待定位图像进行目标点检测，能够充 分利用硬件并行加速，降低目标点检测的复杂度。
在一些实施例中，深度学习模型为全卷积神经网络，和/或，点定位装置60还包括模型训练模块，具体包括：样本获取子模块，配置为获取样本图像，其中，样本图像标注有目标点的真实位置信息；热度图生成子模块，配置为利用目标点的真实位置信息，生成粗目标热度图和细目标热度图；其中，粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围；细目标热度图中包含目标点的第四区域的热度值在第四热度值范围，其中，第三区域大于第四区域；目标检测子模块，配置为利用深度学习模型对样本图像进行目标点检测，得到粗定位热度图和细定位热度图；参数调整子模块，配置为基于粗目标热度图与粗定位热度图之间的差异和细目标热度图和细定位热度图之间的差异，调整深度学习模型的网络参数。
区别于前述实施例，通过利用目标点的真实位置信息，生成粗目标热度图和细目标热度图，且粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围，细目标热度图中包含目标点的第四区域的热度值在第四热度值范围，第三区域大于第四区域，从而利用深度学习模型对样本图像进行目标点检测，得到粗定位热度图和细定位热度图，基于粗目标热度图与粗定位热度图之间的差异和细目标热度图和细定位热度图之间的差异，调整深度学习模型的网络参数，能够有利于使深度学习模型生成精确的粗定位热度图和细定位热度图，进而能够有利于提高点定位的准确性和稳定性。
在一些实施例中，待定位图像为X线图像；在第一区域和第二区域中，越靠近目标点的热度值越高；粗定位热度图中位于第一区域以外的热度值低于第一热度值范围的下限值，细定位热度图中位于第二区域以外的热度值低于第二热度值范围的下限值。
区别于前述实施例，在第一区域和第二区域中，越靠近目标点的热度值越高，粗定位热度图中位于第一区域以外的热度值低于第一热度值范围的下限值，细定位热度图中位于第二区域以外的热度值低于第二热度值范围的下限值，故能够便捷地确定粗定位热度图和细定位热度图中的目标点。
图7是本公开实施例的电子设备的框架示意图。如图7所示，电子设备70包括相互耦接的存储器71和处理器72，处理器72用于执行存储器71中存储的程序指令，以实现上述任一点定位方法实施例中的步骤。在一个具体的实施场景中，电子设备70可以包括但不限于：微型计算机、服务器，此外，电子设备70还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。
具体而言，处理器72用于控制其自身以及存储器71以实现上述任一点定位方法实施例中的步骤。处理器72还可以称为中央处理单元(Central Processing Unit，CPU)。处理器72可能是一种集成电路芯片，具有信号的处理能力。处理器72还可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器72可以由集成 电路芯片共同实现。
上述方案，能够同时兼具粗定位热度图的定位稳定性以及细定位热度图的准确性，从而能够提高点定位的准确性和稳定性。
图8为本公开实施例的计算机可读存储介质的框架示意图。如图8所示，计算机可读存储介质80存储有能够被处理器运行的程序指令801，程序指令801用于实现上述任一点定位方法实施例中的步骤。
能够同时兼具粗定位热度图的定位稳定性以及细定位热度图的准确性，从而能够提高点定位的准确性和稳定性。
本公开实施例还提供了一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现上述点定位方法。
本公开所提供的几个方法或装置实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或装置实施例。
在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (18)

  1. 一种点定位方法，所述方法包括：
    获取待定位图像；
    对所述待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，其中，所述粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；所述细定位热度图中包含所述目标点的第二区域的热度值在第二热度值范围，其中，所述第一区域大于第二区域；
    结合分析所述粗定位热度图和所述细定位热度图，得到所述目标点的位置信息。
  2. 根据权利要求1所述的方法，其中，所述结合分析所述粗定位热度图和所述细定位热度图，得到所述目标点的位置信息，包括：
    获取所述粗定位热度图中的第一目标点位置以及所述细定位热度图中的第二目标点位置和所述第二目标点位置的置信度，并基于所述第二目标点位置的置信度，对所述第一目标点位置和第二目标点位置进行处理得到所述目标点的位置信息。
  3. 根据权利要求2所述的方法，其中，所述获取所述粗定位热度图中的第一目标点位置以及所述细定位热度图中的第二目标点位置和所述第二目标点位置的置信度，并基于所述第二目标点位置的置信度，对所述第一目标点位置和第二目标点位置进行处理得到所述目标点的位置信息，包括：
    基于所述粗定位热度图的热度值，确定第一目标点位置；以及，基于所述细定位热度图的热度值，确定第二目标点位置和所述第二目标点位置的置信度；
    基于所述第二目标点位置的置信度，选择所述第二目标点位置或所述第一目标点位置，作为所述目标点的位置信息。
  4. 根据权利要求3所述的方法，其中，所述基于所述第二目标点位置的置信度，选择所述第二目标点位置或所述第一目标点位置，作为所述目标点的位置信息，包括：
    若所述第二目标点位置的置信度满足细定位置信条件，则将所述第二目标点位置作为所述目标点的位置信息；
    若所述第二目标点位置的置信度不满足所述细定位置信条件，则将所述第一目标点位置作为所述目标点的位置信息。
  5. 根据权利要求3所述的方法，其中，在所述基于所述细定位热度图的热度值，确定第二目标点位置和所述第二目标点位置的置信度之前，所述方法包括：
    基于所述粗定位热度图的热度值，确定所述第一目标点位置的置信度；
    若所述第一目标点位置的置信度满足粗定位置信条件，则将所述细定位热度图中位于所述第一目标点位置的预设距离范围外的热度值调整为预设热度值；其中，所述预设热度值在所述第二热度值范围之外。
  6. 根据权利要求5所述的方法，其中，所述粗定位置信条件包括所述第一目标点 位置的置信度大于第一预设阈值，所述细定位置信条件包括以下至少一者：所述第二目标点位置的置信度大于第二预设阈值，所述第二目标点位置的置信度大于所述第一目标点位置的置信度。
  7. 根据权利要求5或6所述的方法，其中，所述基于所述粗定位热度图的热度值，确定第一目标点位置，或者所述基于所述细定位热度图的热度值，确定第二目标点位置，包括：
    将定位热度图中热度值最大的点作为目标点位置，或者将所述定位热度图中的区域预设点作为所述目标点位置；
    和/或，所述基于所述粗定位热度图的热度值，确定所述第一目标点位置的置信度，或者，基于所述细定位热度图的热度值，确定所述第二目标点位置的置信度，包括：
    获取至少一个参考热度值；针对每个所述参考热度值，从定位热度图中获取热度值大于所述参考热度值的参考区域的尺寸；并基于每个所述参考区域的尺寸以及所述目标点位置的热度值得到所述目标点位置的置信度；
    或者，基于定位热度图的目标点位置得到目标热度图；基于所述定位热度图与所述目标热度图之间的热度值分布相似度，得到所述目标点位置的置信度。
  8. 根据权利要求7所述的方法，其中，所述获取至少一个参考热度值，包括：
    获取至少一个倍率，将所述至少一个倍率分别与所述目标点位置的热度值之间的乘积作为至少一个参考热度值；
    所述参考区域包含所述目标点位置；所述参考区域的尺寸包括所述参考区域的周长和面积；所述基于每个所述参考区域的尺寸以及所述目标点位置的热度值得到所述目标点位置的置信度，包括：
    获取每个所述参考区域的面积与周长的平方之间的第一比例，利用所述至少一个参考区域的第一比例之和、所述目标点位置的热度值以及预设热度峰值，得到目标点位置的置信度。
  9. 根据权利要求7所述的方法，其中，所述基于定位热度图的目标点位置得到目标热度图，包括：
    基于定位热度图的目标点位置，利用二维高斯函数获得目标热度图中各像素点的热度值，其中，所述二维高斯函数的指数包含范围参数，且所述指数的绝对值与所述范围参数之间为负相关关系，所述粗定位热度图对应的二维高斯函数中的所述范围参数大于所述细定位热度图对应的二维高斯函数中的所述范围参数；
    所述基于所述定位热度图与所述目标热度图之间的热度值分布相似度，得到所述目标点位置的置信度，包括：
    基于所述定位热度图与所述目标热度图之间的热度值分布，获得所述定位热度图和所述目标热度图之间的相关系数，以作为所述目标点位置的置信度。
  10. 根据权利要求1至9任一项所述的方法，其中，在所述对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图之前，还包括：
    对所述待定位图像进行预处理；其中所述预处理包括以下至少一种：将所述待定位图像进行归一化，对所述待定位图像进行增强图像对比度；
    和/或，在所述结合分析所述粗定位热度图和所述细定位热度图，得到所述目标点的位置信息之后，还包括：
    输出所述目标点的位置信息和对应所述位置信息的置信度。
  11. 根据权利要求10所述的方法，其中，所述将所述待定位图像进行归一化，包括：
    将所述待定位图像中大于第一像素值的像素值设置为所述第一像素值，并将所述待定位图像中小于第二像素值的像素值设置为所述第二像素值；
    其中，在所述待定位图像的顺序排列的像素值中，位于第一数值排位的像素值为第一像素值，位于第二数值排位的像素值为第二像素值。
  12. 根据权利要求1至11任一项所述的方法，其中，所述对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，包括：
    利用深度学习模型对所述待定位图像进行目标点检测，得到粗定位热度图和细定位热度图。
  13. 根据权利要求11所述的方法，其中，所述深度学习模型为全卷积神经网络，和/或，所述深度学习模型至少由以下步骤训练得到：
    获取样本图像，其中，所述样本图像标注有目标点的真实位置信息；
    利用所述目标点的真实位置信息，生成粗目标热度图和细目标热度图；其中，所述粗目标热度图中包含目标点的第三区域的热度值在第三热度值范围；所述细目标热度图中包含所述目标点的第四区域的热度值在第四热度值范围，其中，所述第三区域大于第四区域；
    利用所述深度学习模型对所述样本图像进行目标点检测，得到粗定位热度图和细定位热度图；
    基于所述粗目标热度图与粗定位热度图之间的差异和所述细目标热度图和细定位热度图之间的差异，调整所述深度学习模型的网络参数。
  14. 根据权利要求1至13任一项所述的方法，其中，所述待定位图像为X线图像；
    在所述第一区域和所述第二区域中，越靠近所述目标点的热度值越高；所述粗定位热度图中位于所述第一区域以外的热度值低于所述第一热度值范围的下限值，所述细定位热度图中位于所述第二区域以外的热度值低于所述第二热度值范围的下限值。
  15. 一种点定位装置，包括：
    图像获取模块，配置为获取待定位图像；
    目标检测模块，配置为对待定位图像进行目标点检测，得到粗定位热度图和细定位热度图，其中，所述粗定位热度图中包含目标点的第一区域的热度值在第一热度值范围；所述细定位热度图中包含所述目标点的第二区域的热度值在第二热度值范围，其中，所述第一区域大于第二区域；
    位置分析模块，配置为结合分析所述粗定位热度图和所述细定位热度图，得到所述目标点的位置信息。
  16. 一种电子设备，包括相互耦接的存储器和处理器，所述处理器用于执行所述存储器中存储的程序指令，以实现权利要求1至14任一项所述的点定位方法。
  17. 一种计算机可读存储介质，其上存储有程序指令，所述程序指令被处理器执行时实现权利要求1至14任一项所述的点定位方法。
  18. 一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备的处理器执行用于实现如权利要求1至14任意一项所述的点定位方法。
PCT/CN2021/103150 2020-10-29 2021-06-29 点定位方法及相关装置、设备、介质及计算机程序 WO2022088729A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011182566.2A CN112348892A (zh) 2020-10-29 2020-10-29 点定位方法及相关装置、设备
CN202011182566.2 2020-10-29

Publications (1)

Publication Number Publication Date
WO2022088729A1 true WO2022088729A1 (zh) 2022-05-05

Family

ID=74355909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/103150 WO2022088729A1 (zh) 2020-10-29 2021-06-29 点定位方法及相关装置、设备、介质及计算机程序

Country Status (2)

Country Link
CN (1) CN112348892A (zh)
WO (1) WO2022088729A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173523A (zh) * 2023-08-04 2023-12-05 山东大学 一种基于频率感知的伪装目标检测方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112348892A (zh) * 2020-10-29 2021-02-09 上海商汤智能科技有限公司 点定位方法及相关装置、设备
CN114677503A (zh) * 2022-03-10 2022-06-28 上海联影医疗科技股份有限公司 感兴趣区域检测方法、装置、计算机设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106651955A (zh) * 2016-10-10 2017-05-10 北京小米移动软件有限公司 图片中目标物的定位方法及装置
CN109214245A (zh) * 2017-07-03 2019-01-15 株式会社理光 一种目标跟踪方法、装置、设备及计算机可读存储介质
CN110348412A (zh) * 2019-07-16 2019-10-18 广州图普网络科技有限公司 一种关键点定位方法、装置、电子设备及存储介质
CN111768399A (zh) * 2020-07-07 2020-10-13 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN112348892A (zh) * 2020-10-29 2021-02-09 上海商汤智能科技有限公司 点定位方法及相关装置、设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108229488B (zh) * 2016-12-27 2021-01-01 北京市商汤科技开发有限公司 用于检测物体关键点的方法、装置及电子设备
CN111316319A (zh) * 2019-03-15 2020-06-19 深圳市大疆创新科技有限公司 图像处理方法、电子设备及计算机可读存储介质
CN111274977B (zh) * 2020-01-22 2023-05-23 中能国际高新科技研究院有限公司 多任务卷积神经网络模型及使用方法、装置和存储介质
CN111508019A (zh) * 2020-03-11 2020-08-07 上海商汤智能科技有限公司 目标检测方法及其模型的训练方法及相关装置、设备
CN111768400A (zh) * 2020-07-07 2020-10-13 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106651955A (zh) * 2016-10-10 2017-05-10 北京小米移动软件有限公司 图片中目标物的定位方法及装置
CN109214245A (zh) * 2017-07-03 2019-01-15 株式会社理光 一种目标跟踪方法、装置、设备及计算机可读存储介质
CN110348412A (zh) * 2019-07-16 2019-10-18 广州图普网络科技有限公司 一种关键点定位方法、装置、电子设备及存储介质
CN111768399A (zh) * 2020-07-07 2020-10-13 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN112348892A (zh) * 2020-10-29 2021-02-09 上海商汤智能科技有限公司 点定位方法及相关装置、设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117173523A (zh) * 2023-08-04 2023-12-05 山东大学 一种基于频率感知的伪装目标检测方法及系统
CN117173523B (zh) * 2023-08-04 2024-04-09 山东大学 一种基于频率感知的伪装目标检测方法及系统

Also Published As

Publication number Publication date
CN112348892A (zh) 2021-02-09

Similar Documents

Publication Publication Date Title
WO2022088729A1 (zh) 点定位方法及相关装置、设备、介质及计算机程序
US11645748B2 (en) Three-dimensional automatic location system for epileptogenic focus based on deep learning
Wang et al. Evaluation and comparison of anatomical landmark detection methods for cephalometric x-ray images: a grand challenge
US9480439B2 (en) Segmentation and fracture detection in CT images
Lin et al. An effective classification and numbering system for dental bitewing radiographs using teeth region and contour information
CN106023151B (zh) 一种开放环境下中医舌象目标检测方法
US8285013B2 (en) Method and apparatus for detecting abnormal patterns within diagnosis target image utilizing the past positions of abnormal patterns
US20080260254A1 (en) Automatic 3-D Object Detection
US20120099771A1 (en) Computer aided detection of architectural distortion in mammography
CN112507963B (zh) 一种批量口罩人脸样本自动生成以及口罩人脸识别方法
US20080123929A1 (en) Apparatus, method and program for image type judgment
CN112613471B (zh) 人脸活体检测方法、装置及计算机可读存储介质
Lee et al. Deep learning based cephalometric landmark identification using landmark-dependent multi-scale patches
JP2008011901A (ja) 画像種類判別装置および方法並びにプログラム
TW201915947A (zh) 醫學影像之迭代式分析法
CN117237591A (zh) 一种心脏超声影像伪影智能去除方法
CN117788407A (zh) 基于人工神经网络的青光眼图像特征提取的训练方法
US7352888B2 (en) Method for computer recognition of projection views and orientation of chest radiographs
JP2008011900A (ja) 画像種類判別装置および方法並びにプログラム
CN112132788B (zh) 一种基于特征区域等级识别的骨龄评估方法
CN111862118A (zh) 压疮分期的训练方法、分期方法及分期系统
Banumathi et al. Diagnosis of dental deformities in cephalometry images using support vector machine
Fondon et al. Perceptually adapted method for optic disc detection on retinal fundus images
EP4075385A1 (en) Method and system for anonymizing facial images
CN115456974A (zh) 基于人脸关键点的斜视检测系统、方法、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21884468

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04.10.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21884468

Country of ref document: EP

Kind code of ref document: A1