WO2024087910A1 - 正畸治疗监测方法、装置、设备及存储介质 - Google Patents

正畸治疗监测方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2024087910A1
WO2024087910A1 PCT/CN2023/117857 CN2023117857W WO2024087910A1 WO 2024087910 A1 WO2024087910 A1 WO 2024087910A1 CN 2023117857 W CN2023117857 W CN 2023117857W WO 2024087910 A1 WO2024087910 A1 WO 2024087910A1
Authority
WO
WIPO (PCT)
Prior art keywords
tooth
point cloud
target
sub
image
Prior art date
Application number
PCT/CN2023/117857
Other languages
English (en)
French (fr)
Inventor
田彦
简国堂
江腾飞
赵晓波
Original Assignee
先临三维科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 先临三维科技股份有限公司 filed Critical 先临三维科技股份有限公司
Publication of WO2024087910A1 publication Critical patent/WO2024087910A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the embodiments of the present disclosure relate to the field of computer technology, and in particular to an orthodontic treatment monitoring method, device, equipment and storage medium.
  • Orthodontic treatment technology is to have patients wear dental braces periodically so that designated teeth can grow in the direction of the designed movement of the dental braces, thereby achieving the purpose of gradually changing the position and posture of the teeth.
  • the monitoring of orthodontic treatment during the correction process mainly relies on patients to go to the clinic for regular check-ups, and the doctor's naked eye observation is used to evaluate the orthodontic effect of the current treatment stage. Once the doctor finds that the orthodontic effect is not growing in the direction of the design plan, it is necessary to adjust the dental appliance in time to ensure the smooth progress of the treatment.
  • this orthodontic treatment monitoring method is inefficient and difficult to meet user needs.
  • the technical problem to be solved by the present disclosure is to solve the problem of low efficiency of existing orthodontic treatment monitoring methods.
  • the embodiments of the present disclosure provide an orthodontic treatment monitoring method, device, equipment and storage medium.
  • a first aspect of the present disclosure provides an orthodontic treatment monitoring method, the method comprising:
  • a first tooth point cloud of the target tooth and a second tooth point cloud of the target tooth are compared to obtain a deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • a second aspect of the embodiments of the present disclosure provides an orthodontic treatment monitoring device, the device comprising:
  • the first acquisition module is used to acquire multiple frames of the user's current intraoral images at the current moment;
  • a first determination module configured to determine a first tooth point cloud of each target tooth in the user's oral cavity based on multiple frames of current intraoral images
  • the comparison module is used to compare the first tooth point cloud of the target tooth with the second tooth point cloud of the target tooth for each target tooth to obtain a deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • a third aspect of an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method of the first aspect above.
  • the fourth aspect of the embodiments of the present disclosure provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may implement part or all of the steps in each implementation method of an orthodontic treatment monitoring method provided in the first aspect of the present disclosure.
  • the orthodontic treatment monitoring method can obtain multiple frames of current intraoral images of the user at the current moment, and determine the first tooth point cloud of each target tooth in the user's mouth based on the multiple frames of current intraoral images, so as to compare the first tooth point cloud of the target tooth with the second tooth point cloud of the target tooth for each target tooth, and obtain the deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • the above technical solution can realize automatic comparison of the first tooth point cloud and the second tooth point cloud, and compared with the traditional doctor's naked eye observation comparison, the deviation value between the first tooth point cloud and the second tooth point cloud can be obtained conveniently, quickly and accurately, thereby improving the efficiency of orthodontic treatment monitoring.
  • FIG1 is a flow chart of an orthodontic treatment monitoring method provided by an embodiment of the present disclosure.
  • FIG2 is a flow chart of another orthodontic treatment monitoring method provided by an embodiment of the present disclosure.
  • FIG3 is a logic diagram of an example segmentation provided by an embodiment of the present disclosure.
  • FIG4 is a schematic diagram of the structure of an orthodontic treatment monitoring device provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure.
  • FIG1 is a flow chart of an orthodontic treatment monitoring method provided by an embodiment of the present disclosure, which can be performed by an electronic device.
  • the electronic device can be exemplarily understood as a device with a page display function such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, etc.
  • the method provided by this embodiment includes the following steps:
  • the electronic device in order to perform orthodontic treatment monitoring on the user's orthodontic correction process, the electronic device needs to obtain multiple frames of the user's current intraoral images at the current moment.
  • the current intraoral image is an image obtained by capturing the intraoral image of the user using an image capture device at the current moment.
  • the image acquisition device may include an RGBD camera, and the current intraoral image includes an RGB image and a depth image, but is not limited thereto.
  • images of the user's oral cavity can be acquired from different angles and/or different distances, so that at least each target tooth in the oral cavity is imaged, for example, all teeth in the oral cavity are imaged, etc., wherein the target teeth are teeth that require orthodontic treatment monitoring.
  • the electronic device may directly receive multiple frames of current intraoral images sent by the image acquisition device, or read multiple frames of current intraoral images pre-stored locally or in a storage device, but is not limited thereto.
  • the electronic device can determine at least a first tooth point cloud of each target tooth based on multiple frames of current intra-oral images, so as to subsequently compare the first tooth point cloud and the second tooth point cloud of the target tooth.
  • the first tooth point cloud is a tooth point cloud obtained based on the current intra-oral image.
  • S120 may include: for each target tooth, segmenting a tooth image (or tooth region) of the target tooth from multiple frames of current intraoral images to obtain at least one tooth image, A first tooth point cloud of the target tooth is determined based on the at least one tooth image.
  • the tooth image is the image region where the teeth are located.
  • segmentation process can be performed manually or by using a two-dimensional object segmentation algorithm, but is not limited thereto.
  • three-dimensional reconstruction can be performed based on the at least one tooth image to obtain the first tooth point cloud of the target tooth; three-dimensional reconstruction can also be performed based on each tooth image to obtain the first tooth sub-point cloud of the target tooth, thereby obtaining at least one first tooth sub-point cloud corresponding to the target tooth, and the at least one first tooth sub-point cloud corresponding to the at least one tooth image is fused and then sampled to obtain the first tooth point cloud of the target tooth, etc., but it is not limited to this.
  • S120 may include: performing three-dimensional reconstruction based on multiple frames of current intraoral images to obtain a current jaw point cloud of the user's intraoral jaw; and segmenting a first tooth point cloud of the target tooth from the current jaw point cloud.
  • the current dental point cloud is the overall point cloud of all upper teeth and/or lower teeth in the user's oral cavity.
  • segmentation process can be performed manually or by using a three-dimensional point cloud segmentation algorithm, but is not limited thereto.
  • the electronic device may pre-store the second tooth point cloud of the target tooth, so that for each target tooth that needs orthodontic treatment monitoring, the first tooth point cloud of the target tooth may be compared with the second tooth point cloud to obtain the deviation value between the two.
  • the "before the current moment” mentioned here can be any moment before the current moment, for example, before treatment, or after the last treatment, etc., but is not limited thereto.
  • the deviation value is the difference between the first tooth point cloud and the second tooth point cloud of the same tooth.
  • the difference may include differences in position, posture, etc.
  • the deviation value may include parameters such as position deviation and posture deviation.
  • S130 may include: for each target tooth, aligning the first tooth point cloud and the second tooth point cloud of the target tooth to obtain a transformation matrix corresponding to the target tooth, and determining a deviation value based on the transformation matrix corresponding to the target tooth, so that a deviation value corresponding to each target tooth can be obtained.
  • the transformation matrix includes a translation matrix and a rotation matrix.
  • the posture deviation can be determined according to the rotation matrix, and the position parameter can be determined according to the translation matrix.
  • the embodiment of the present disclosure can realize the automatic comparison between the first tooth point cloud and the second tooth point cloud, and obtain the quantitative comparison result (i.e., the deviation value), compared with the traditional doctor's naked eye observation comparison, it can not only reduce the burden of the doctor, but also improve the comparison speed, and can also reduce the confusion and misjudgment problems, and find some small deviations that are not easy to find with the naked eye, thereby improving the accuracy of the comparison result. It can be seen that the embodiment of the present disclosure can conveniently, quickly and accurately obtain the deviation value between the first tooth point cloud and the second tooth point cloud, thereby improving the orthodontic treatment monitoring efficiency. In addition, the embodiment of the present disclosure realizes full digitization.
  • the user can also get the advice and guidance of expert doctors in different geographical locations during the process of dental diagnosis or follow-up, so that expert doctors can get rid of geographical constraints and provide professional services to a wider range of patients.
  • it also greatly facilitates the doctor's tracking and monitoring of the patient's treatment process, and is more conducive to the timely promotion, adjustment and improvement of the treatment plan.
  • the disclosed embodiment can obtain multiple frames of the current intraoral images of the user at the current moment, and determine the first tooth point cloud of each target tooth in the user's mouth based on the multiple frames of the current intraoral images, so as to compare the first tooth point cloud of the target tooth with the second tooth point cloud of the target tooth for each target tooth, and obtain the deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • the above technical solution can realize the automatic comparison of the first tooth point cloud and the second tooth point cloud, and compared with the traditional doctor's naked eye observation comparison, the deviation value between the first tooth point cloud and the second tooth point cloud can be obtained conveniently, quickly and accurately, thereby improving the efficiency of orthodontic treatment monitoring.
  • Fig. 2 is a flow chart of another orthodontic treatment monitoring method provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure is optimized on the basis of the above embodiment, and the embodiment of the present disclosure can be combined with various optional solutions in one or more of the above embodiments.
  • the orthodontic treatment monitoring method may include the following steps.
  • S210 is similar to S110 and will not be described in detail here.
  • each frame of the current intraoral image includes at least one tooth. Therefore, for each frame of the current intraoral image, the first tooth sub-point cloud and the tooth position of each tooth in the current intraoral image can be determined based on the current intraoral image. In this way, the first tooth sub-point cloud and the tooth position of each tooth in any frame of the current intraoral image can be obtained.
  • the first tooth sub-point cloud is based on the tooth image (or tooth area) in the current intraoral image.
  • the point cloud obtained by 3D reconstruction of the domain.
  • all teeth on the jaw are usually arranged in sequence, and the tooth position is used to indicate the arrangement position of the tooth on the jaw to which it belongs, for example, the nth tooth from left to right, or the mth tooth from right to left, etc.
  • the current intraoral image includes an RGB image and a depth image; wherein S220 may include:
  • any possible instance segmentation method may be used to perform instance segmentation on the RGB image, which is not limited here.
  • the tooth RGB image is the image region where the teeth are located in the RGB image
  • the tooth depth image is the image region where the teeth are located in the depth image.
  • the dental RGB image of each tooth can be segmented from the RGB image based on the instance segmentation result, and based on the dental RGB image of each tooth and the first association relationship, the dental RGB image corresponding to the dental RGB image can be segmented from the depth image.
  • the RGB image provides the color information of the point
  • the RGB image also provides the x, y coordinates of the point in the pixel coordinate system
  • the depth image directly provides the ⁇ coordinate in the camera coordinate system, that is, the distance between the camera and the point. Therefore, based on the camera's intrinsic parameters, the coordinates of any point in the camera coordinate system can be calculated, thereby realizing three-dimensional reconstruction and obtaining the first tooth sub-point cloud of the tooth.
  • those skilled in the art may also input the current intraoral image into a trained first neural network model to obtain a first tooth sub-point cloud of each tooth in the current intraoral image output by the first neural network model.
  • each tooth determines the tooth position of the tooth based on the first tooth sub-point cloud of the tooth, the tooth RGB image, and the second tooth point cloud of multiple teeth in the user's oral cavity.
  • the first tooth sub-point cloud, the tooth RGB image, and the second tooth point cloud of multiple teeth in the user's mouth can be input into a trained second neural network model to obtain the tooth position of the tooth corresponding to the first tooth sub-point cloud output by the second neural network model.
  • a trained second neural network model to obtain the tooth position of the tooth corresponding to the first tooth sub-point cloud output by the second neural network model.
  • obtaining the first tooth sub-point cloud of the tooth through instance segmentation and three-dimensional reconstruction can make the acquisition method of the first tooth sub-point cloud simple in logic and easy to implement, which is conducive to reducing the difficulty of obtaining the first tooth sub-point cloud.
  • the first tooth sub-point cloud of the same tooth is set to obtain at least one first tooth sub-point cloud corresponding to the target tooth.
  • each frame of the current intraoral image includes at least one tooth
  • different current intraoral images may include overlapping teeth, so the same tooth may appear in different current intraoral images, so one or more first tooth sub-point clouds of the same tooth can be obtained through S220. Since the tooth position of the tooth can be used to determine which tooth on the jaw the tooth is, for each target tooth, the tooth in the current intraoral image that has the same tooth position as the target tooth is the target tooth, and the first tooth sub-point cloud of the target tooth is selected from the multiple first tooth sub-point clouds corresponding to the current intraoral image containing the target tooth, so as to obtain at least one first tooth sub-point cloud corresponding to the target tooth.
  • the electronic device may determine a more complete first tooth point cloud of the target tooth based on at least one first tooth sub-point cloud corresponding to the target tooth.
  • three-dimensional reconstruction based on the tooth RGB image and RGB depth image of the target tooth segmented from the current intraoral image can obtain a complete point cloud of the target tooth, that is, the first tooth sub-point cloud obtained by three-dimensional reconstruction is the first tooth point cloud.
  • the tooth in the current intraoral image is usually incomplete, that is, it only includes a part of the tooth.
  • the first tooth sub-point cloud obtained by three-dimensional reconstruction only corresponds to a part of the tooth, and the image acquisition angles of different current intraoral images are usually different. Therefore, the multiple first tooth sub-point clouds of the tooth obtained based on multiple frames of previous intraoral images usually include different parts of the tooth. In this way, a more complete first tooth point cloud of the tooth can be obtained based on the multiple first tooth sub-point clouds of the tooth.
  • determining the first tooth point cloud of the target tooth includes: fusing at least one first tooth sub-point cloud corresponding to the target tooth and then performing sampling processing to obtain the first tooth point cloud of the target tooth.
  • any possible fusion processing method may be used to fuse at least one first tooth sub-point cloud corresponding to the target tooth, which is not limited here.
  • first tooth point cloud of the target tooth can be obtained through fusion processing and sampling processing.
  • first tooth point cloud is more complete, a more detailed comparison of the tooth conditions of the target tooth at different stages can be achieved by comparing the first tooth point cloud with the second tooth point cloud, so that the deviation value obtained by the comparison is more consistent with the actual growth change of the target tooth, that is, the accuracy of orthodontic monitoring is improved.
  • first tooth point cloud is more concise, that is, the amount of data is smaller, the amount of calculation when comparing the first tooth point cloud with the second tooth point cloud can be reduced, the comparison effect can be improved, and the overall efficiency of orthodontic monitoring can be improved.
  • S250 is similar to S130 and will not be described in detail here.
  • the implementation of the present disclosure can determine the first tooth sub-point cloud and the tooth position of each tooth in the current intra-oral image, and obtain at least one first tooth sub-point cloud corresponding to the target tooth according to the tooth position, so as to determine the first tooth point cloud of the target tooth based on the at least one first tooth sub-point cloud corresponding to the target tooth, which can make the acquisition method of the first tooth point cloud simple and easy to implement, and because the first tooth point cloud is more complete, it is possible to achieve a more detailed comparison of the tooth conditions of the target teeth at different stages, so that the deviation value obtained by comparison is more consistent with the actual growth changes of the target teeth, that is, to improve the accuracy of orthodontic monitoring of teeth.
  • the RGB image is instance segmented to obtain an instance segmentation result, including:
  • Swin transformer is a deep learning model that can be used as a general backbone network for computer vision tasks. It can be used for a series of visual downstream tasks such as image classification, image segmentation, and object detection.
  • S2212 Determine instance segmentation results based on the feature map.
  • FIG3 is a logic diagram of an instance segmentation provided by an embodiment of the present disclosure.
  • a feature extraction network 311 based on Swin Transformer is used to extract features from an input RGB image 321 to obtain a feature map 322; then, a region proposal network (RPN) 312 is used to extract a region of interest from the obtained feature map 322, and a positioning operation is performed to obtain a preliminary positive sample and its positioning information, which is recorded as a candidate box 323; then, the obtained feature map 322 and the candidate box 323 are input to the region of interest alignment layer (Rol Align) 313, and the positioning information, category information and a fixed-size feature map are obtained through regression.
  • RPN region proposal network
  • the detection box generation network 315 generates a detection box 324 based on the positioning information, and the classification network 316 generates a classification result 325 based on the category information.
  • the fixed-size feature map is used to perform semantic segmentation using an image segmentation network 314 based on a fully convolutional network (FCN), thereby obtaining a final instance segmentation result 326.
  • FCN fully convolutional network
  • MaskRCNN is derived from FasterRCNN. Specifically, a Mask branch is added to the output module of FasterRCNN to segment objects within each bounding box.
  • the feature extraction network used for feature map extraction is very critical, because MaskRCNN will search for regions of interest on feature maps at different levels of the feature extraction network for subsequent precise target positioning.
  • the receptive field of CNN-based feature extraction networks was local, and it was usually necessary to deepen the network to expand the receptive field, while the attention-based method can capture global regional relationships.
  • SwinTransformer When performing instance segmentation on the intraoral tooth image (i.e., the RGB image in the current intraoral image), considering that the appearance of the non-tooth area in the mouth is relatively messy, and the boundary between some teeth and the gums is not obvious, SwinTransformer can be used as the It is the feature extraction network of MaskRCNN.
  • MaskRCNN the feature extraction network of MaskRCNN.
  • it designs a sliding window mechanism similar to CNN, which only calculates the attention between patches within each window, thus solving the problem of large computational complexity of the first-generation Vision Transformer.
  • it realizes information interaction between different windows by circularly moving pixels, making the information flow between different regions smooth, thereby improving the final tooth instance segmentation effect.
  • determining the tooth position of a tooth based on a first tooth sub-point cloud of a tooth, a tooth RGB image, and a second tooth point cloud of a plurality of teeth in a user's oral cavity includes:
  • coarse registration refers to registration when the relative positions of the two point clouds are completely unknown.
  • the coarse registration can achieve a rough alignment of the first tooth sub-point cloud with the second tooth point cloud.
  • the multiple teeth in the user's mouth are all the teeth on the upper jaw; when the target teeth are teeth on the lower jaw, the multiple teeth in the user's mouth are all the teeth on the lower jaw; of course, the multiple teeth in the user's mouth can also be all the teeth in the user's mouth, and there is no limitation to this.
  • Any possible coarse registration method may be used to coarsely register the first tooth sub-point cloud with the second tooth point cloud, which is not limited here.
  • a feature-based coarse registration method a coarse registration method based on a random sample consensus algorithm (RANSAC) framework, etc. may be used, but it is not limited thereto.
  • RANSAC random sample consensus algorithm
  • the coordinate systems of the first tooth sub-point cloud and the second tooth point cloud are both the camera coordinate systems corresponding to the first tooth sub-point cloud. Therefore, based on the internal parameters of the camera, the coordinates of any point on the second tooth point cloud in the camera coordinate system can be calculated and projected to the pixel coordinate system to obtain the projection area. In this way, the overlapping area between the projection area and the tooth RGB image corresponding to the first tooth sub-point cloud can be calculated, and the union area after the projection area and the tooth RGB image corresponding to the first tooth sub-point cloud are taken can also be calculated, and then the ratio of the overlapping area to the union area is calculated to obtain the area intersection ratio of the overlapping area.
  • the first tooth sub-point cloud of the target tooth is traversed and roughly aligned with the second tooth point cloud of multiple teeth in the user's mouth, the overlapping area is projected, and the area intersection and union ratio of the overlapping area is calculated, and the tooth position of the tooth with the largest area intersection and union ratio is assigned to the target tooth.
  • the method of determining the tooth position can be simple and easy to implement, and it can avoid missing a match with a certain tooth in the mouth, so as to find the tooth that best matches the target tooth, which is conducive to improving the accuracy of the tooth position. Determine the accuracy.
  • a first tooth point cloud of a target tooth and a second tooth point cloud of a target tooth are compared to obtain a deviation value corresponding to the target tooth, including: coarsely aligning the first tooth point cloud and the second tooth point cloud to obtain a first transformation matrix; continuing to finely align the first tooth point cloud and the second tooth point cloud to obtain a second transformation matrix; and obtaining the deviation value corresponding to the target tooth based on the first transformation matrix and the second transformation matrix.
  • the coarse registration can provide a good initial value of the transformation matrix (ie, the first transformation matrix) for the fine registration.
  • the precise registration is to further calculate a more accurate transformation matrix when the initial value of the transformation matrix (ie, the first transformation matrix) is known (the initial value is probably correct).
  • IPC iterative closest point algorithm
  • NCP non-rigid iterative closest point
  • the first transformation matrix includes a first rotation matrix and a first translation matrix
  • the second transformation matrix includes a second rotation matrix and a second translation matrix
  • the product of the first rotation matrix and the second rotation matrix and the product of the first translation matrix and the second translation matrix can be used as the rotation matrix and the translation matrix of the first tooth point cloud and the second tooth point cloud of the target tooth, respectively.
  • the posture deviation of the target tooth can be determined based on the rotation matrix, and the position deviation of the target tooth can be determined based on the translation matrix.
  • the second tooth point cloud can also be rotated and translated according to the rotation matrix and the translation matrix, and the error between the second tooth point cloud and the first tooth point cloud after rotation and translation can be calculated to verify the accuracy of the rotation matrix and the translation matrix, so that when the error exceeds the threshold, the first tooth point cloud and the second tooth point cloud of the target tooth can be compared again, or relevant personnel can be notified to check whether there is any problem with orthodontic treatment monitoring.
  • FIG4 is a schematic diagram of the structure of an orthodontic treatment monitoring device provided by an embodiment of the present disclosure, and the orthodontic treatment monitoring device can be understood as the above-mentioned electronic device or a part of the functional modules in the above-mentioned electronic device.
  • the orthodontic treatment monitoring device 400 includes:
  • the first acquisition module 410 is used to acquire multiple frames of the current intraoral images of the user at the current moment;
  • a first determination module 420 is used to determine a first tooth point cloud of each target tooth in the user's oral cavity based on multiple frames of current intraoral images;
  • the comparison module 430 is used to compare the first tooth point cloud of the target tooth with the second tooth point cloud of the target tooth for each target tooth to obtain a deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • the first determining module 420 may include:
  • a first determination submodule is used to determine, for each frame of the current intraoral image, a first tooth sub-point cloud and a tooth position of each tooth in the current intraoral image, wherein the tooth position is used to indicate an arrangement position of the tooth on the jaw;
  • a first acquisition submodule is used to acquire, for each target tooth, a first tooth sub-point cloud of a tooth having the same tooth position as the target tooth in multiple frames of current intraoral images, to obtain at least one first tooth sub-point cloud corresponding to the target tooth;
  • the second determination submodule is used to determine, for each target tooth, a first tooth point cloud of the target tooth based on at least one first tooth sub-point cloud corresponding to the target tooth.
  • the current intraoral image includes an RGB image and a depth image
  • the first determining submodule may include:
  • An instance segmentation unit is used to perform instance segmentation on the RGB image to obtain an instance segmentation result
  • a region segmentation unit is used to perform region segmentation on the RGB image and the depth image based on the instance segmentation result, so as to obtain a tooth RGB image and a tooth depth image of each tooth in the current intraoral image;
  • a three-dimensional reconstruction unit used for performing three-dimensional reconstruction on each tooth based on a tooth RGB image and a tooth depth image of the tooth to obtain a first tooth sub-point cloud of the tooth;
  • the first determination unit is used to determine the tooth position of each tooth based on the first tooth sub-point cloud of the tooth, the tooth RGB image and the second tooth point cloud of multiple teeth in the user's mouth.
  • the instance segmentation unit may include:
  • the feature extraction subunit is used to input the RGB image into the feature extraction network based on Swin Transformer to extract features and obtain a feature map;
  • the first determination subunit is used to determine the instance segmentation result based on the feature map.
  • the first determining unit may include:
  • a coarse registration and projection subunit for determining, for each second tooth point cloud, an area intersection-over-union ratio of an overlapping area of a projection area and a tooth RGB image when the second tooth point cloud is projected onto a plane where a tooth RGB image corresponding to the first tooth sub-point cloud is located, after coarse registration of the first tooth sub-point cloud with the second tooth point cloud;
  • the second determining subunit is used to take the tooth position of the tooth corresponding to the second tooth point cloud with the largest area intersection-union ratio as the tooth position of the target tooth.
  • the second determining submodule may include:
  • the first determining unit is used to perform fusion processing and then sampling processing on at least one first tooth sub-point cloud corresponding to the target tooth to obtain a first tooth point cloud of the target tooth.
  • the comparison module 430 may include:
  • a coarse registration submodule used for coarsely registering the first tooth point cloud and the second tooth point cloud to obtain a first transformation matrix
  • a fine registration submodule used to continue fine registration of the first tooth point cloud and the second tooth point cloud to obtain a second transformation matrix
  • the third determination submodule is used to obtain the deviation value corresponding to the target tooth based on the first conversion matrix and the second conversion matrix.
  • the device provided in this embodiment can execute the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.
  • An embodiment of the present disclosure further provides an electronic device, which includes: a memory, in which a computer program is stored; and a processor, for executing the computer program.
  • a computer program is stored; and a processor, for executing the computer program.
  • FIG5 is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure.
  • the electronic device 500 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG5 is merely an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503.
  • a processing device e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 500 are also stored.
  • the processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504.
  • An input/output (I/O) interface 505 is also connected to the bus 504.
  • the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509.
  • the communication devices 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG. 5 shows an electronic device 500 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be transmitted from the communication device 509 to the computer readable medium.
  • the computer program is downloaded and installed on the network, or installed from the storage device 508, or installed from the ROM 502.
  • the processing device 501 the above functions defined in the method of the embodiment of the present disclosure are executed.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device When the one or more programs are executed by the electronic device, the electronic device:
  • a first tooth point cloud of the target tooth and a second tooth point cloud of the target tooth are compared to obtain a deviation value corresponding to the target tooth, wherein the second tooth point cloud is the tooth point cloud of the tooth before the current moment.
  • Computers used to perform operations of the present disclosure may be written in one or more programming languages, or a combination thereof.
  • Computer program code the above programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also conventional procedural programming languages such as "C" language or similar programming languages.
  • the program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
  • LAN local area network
  • WAN wide area network
  • Internet Internet service provider
  • each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chip
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • the present disclosure also provides a computer-readable storage medium, wherein the storage medium stores a computer-readable storage medium.
  • the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The execution method and beneficial effects are similar and will not be described in detail here.
  • the orthodontic treatment monitoring method provided by the present invention can realize automatic comparison between the first tooth point cloud and the second tooth point cloud. Compared with the traditional doctor's naked eye observation comparison, the deviation value between the first tooth point cloud and the second tooth point cloud can be obtained conveniently, quickly and accurately, thereby improving the efficiency of orthodontic treatment monitoring.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)

Abstract

本公开实施例涉及一种正畸治疗监测方法、装置、设备及存储介质，其中，正畸治疗监测方法包括：获取用户在当前时刻的多帧当前口腔内图像；基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云；针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。根据本公开实施例，能够实现第一牙齿点云和第二牙齿点云的自动对比，从而提高正畸治疗监测效率。

Description

正畸治疗监测方法、装置、设备及存储介质
本公开要求于2022年10月28日提交中国专利局、申请号为202211337209.8、发明名称为“正畸治疗监测方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开实施例涉及计算机技术领域，尤其涉及一种正畸治疗监测方法、装置、设备及存储介质。
背景技术
牙齿正畸矫治技术是通过给患者周期性佩戴牙齿矫治器，从而让指定的牙齿能够朝着牙齿矫治器设计移动量的方向去生长，进而达到逐步改变牙齿位置姿态等的目的。
现阶段，矫治过程中的正畸治疗监测主要靠患者定期到诊所进行检查，通过医生肉眼观察来评估当前治疗阶段的正畸效果，一旦医生发现正畸效果未按照设计方案制定的方向去生长，则需要及时调整牙齿矫治器以保证治疗的顺利进行。但是，这种正畸治疗监测方法效率较低，难以满足用户需求。
发明内容
(一)要解决的技术问题
本公开要解决的技术问题是解决现有的正畸治疗监测方法效率较低的问题。
(二)技术方案
为了解决上述技术问题，本公开实施例提供了一种正畸治疗监测方法、装置、设备及存储介质。
本公开实施例的第一方面提供了一种正畸治疗监测方法，该方法包括：
获取用户在当前时刻的多帧当前口腔内图像；
基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云；
针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。
本公开实施例的第二方面提供了一种正畸治疗监测装置，该装置包括：
第一获取模块，用于获取用户在当前时刻的多帧当前口腔内图像；
第一确定模块，用于基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云；
对比模块，用于针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。
本公开实施例的第三方面提供了一种电子设备，该电子设备包括：处理器和存储器，其中，所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行上述第一方面的方法。
本公开实施例的第四方面提供了一种计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可实现本公开第一方面提供的一种正畸治疗监测方法的各实现方式中的部分或全部步骤。
(三)有益效果
本公开实施例提供的上述技术方案与现有技术相比具有如下优点：
本公开实施例提供的该正畸治疗监测方法，能够获取用户在当前时刻的多帧当前口腔内图像，并基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云，从而针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。采用上述技术方案，能够实现第一牙齿点云和第二牙齿点云的自动对比，相比于传统的医生肉眼观察对比，可方便、快捷、准确地获取第一牙齿点云和第二牙齿点云之间的偏差值，从而提高正畸治疗监测效率。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本公开实施例提供的一种正畸治疗监测方法的流程图；
图2是本公开实施例提供的另一种正畸治疗监测方法的流程图；
图3是本公开实施例提供的一种实例分割的逻辑示意图；
图4是本公开实施例提供的一种正畸治疗监测装置的结构示意图；
图5是本公开实施例中的一种电子设备的结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
图1是本公开实施例提供的一种正畸治疗监测方法的流程图，该方法可以由一种电子设备来执行。该电子设备可以示例性的理解为诸如手机、平板电脑、笔记本电脑、台式机、智能电视等具有页面展示功能的设备。如图1所示，本实施例提供的方法包括如下步骤：
S110、获取用户在当前时刻的多帧当前口腔内图像。
在本公开实施例中，为对用户的正畸矫治过程进行正畸治疗监测，电子设备需要获取用户在当前时刻的多帧当前口腔内图像。
具体地，当前口腔内图像为在当前时刻采用图像采集设备对用户的口腔内进行图像采集得到的图像。
图像采集设备的具体类型，本领域技术人员可根据实际情况设置，此处不作限定。例如，图像采集设备可以包括RGBD相机，则当前口腔内图像包括RGB图像和深度图像，但并不限于此。
在进行图像采集的过程中，可以从不同的角度和/或不同的距离对用户的口腔内进行图像采集，以使至少对口腔内的每个目标牙齿均进行图像采集，例如，对口腔内的所有牙齿均进行图像采集等，其中，目标牙齿为需要进行正畸治疗监测的牙齿。
在一些实施例中，电子设备可以直接接收图像采集设备发送的多帧当前口腔内图像，也可以从本地、或存储设备中读取预先存储的多帧当前口腔内图像。但并不限于此。
S120、基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云。
在本公开实施例中，电子设备可以基于多帧当前口腔内图像，至少确定每个目标牙齿的第一牙齿点云，以便后续将目标牙齿的第一牙齿点云和第二牙齿点云进行对比。
具体地，第一牙齿点云为基于当前口腔内图像得到的牙齿的点云。
在一些实施例中，S120可以包括：针对每个目标牙齿，从多帧当前口腔内图像中分割出该目标牙齿的牙齿图像(或者说牙齿区域)，得到至少一个牙齿图像， 并基于该至少一个牙齿图像确定该目标牙齿的第一牙齿点云。
具体地，牙齿图像即牙齿所在图像区域。
具体地，分割出牙齿图像的具体实施方式有多种，此处不作限定。例如，该分割过程可以利用人工进行分割；也可以使用二维物体分割算法进行分割等，但并不限于此。
具体地，确定第一牙齿点云的具体实施方式有多种，此处也不作限定。例如，可以基于该至少一个牙齿图像进行三维重建，得到该目标牙齿的第一牙齿点云；也可以针对每个牙齿图像，基于该牙齿图像进行三维重建得到该目标牙齿的第一牙齿子点云，从而得到该目标牙齿对应的至少一个第一牙齿子点云，并将该至少一个牙齿图像对应的至少一个第一牙齿子点云进行融合处理后再进行采样处理，得到该目标牙齿的第一牙齿点云等，但并不限于此。
在另一些实施例中，S120可以包括：基于多帧当前口腔内图像进行三维重建，得到用户的口腔内牙颌的当前牙颌点云；从当前牙颌点云中分割出目标牙齿的第一牙齿点云。
具体地，当前牙颌点云为用户的口腔内所有上牙和/或下牙的整体点云。
具体地，分割出第一牙齿点云的具体实施方式有多种，此处不作限定。例如，该分割过程可以利用人工进行分割，也可以使用三维点云分割算法进行分割等，但并不限于此。
S130、针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。
在本公开实施例中，电子设备可以预先存储目标牙齿的第二牙齿点云，如此，针对每个需要进行正畸治疗监测的目标牙齿，可以将该目标牙齿的第一牙齿点云与第二牙齿点云进行对比，得到两者之间的偏差值。
具体地，这里所述的“在当前时刻之前”可以为当前时刻之前的任意某一时刻，例如可以包括治疗前、或上次治疗后等，但并不限于此。
具体地，偏差值为同一牙齿的第一牙齿点云和第二牙齿点云之间的差距，该差距可以包括位置、姿态等方面的差距，相应的，偏差值可以包括位置偏差、姿态偏差等参数。
具体地，偏差值的确定方式有多种，下面就典型示例进行说明，但并不限于此。
在一些实施例中S130可以包括：针对每个目标牙齿，将该目标牙齿的第一牙齿点云和第二牙齿点云进行配准，得到该目标牙齿对应的转换矩阵，并根据该目标牙齿对应的转换矩阵确定偏差值，如此，可得到每个目标牙齿对应的偏差值。
本领域技术人员可采用任意可能的配准算法对第一牙齿点云和第二牙齿点云 进行配准，此处不作限定。
具体地，转换矩阵包括平移矩阵和旋转矩阵，根据旋转矩阵可以确定姿态偏差，根据平移矩阵可以确定位置参数。
可以理解的是，由于本公开实施例能够实现第一牙齿点云和第二牙齿点云之间的自动对比，并得到定量的对比结果(即偏差值)，相比于传统的医生肉眼观察对比，既可以减轻医生的负担，又可以提高对比速度，还可以减少出现混淆和误判问题，发现一些肉眼不容易发现的微小的偏差，从而提高对比结果的准确性，可见，本公开实施例可以方便、快捷、准确地获取第一牙齿点云和第二牙齿点云之间的偏差值，从而提高正畸治疗监测效率。并且，本公开实施例实现了全程数字化，若后续将用户的目标牙齿的第一牙齿点云、第二牙齿点云、以及偏差值等数字化数据通过互联网等形式传送给位于不同地理空间位置的医生查阅，这样用户在进行牙齿诊断或复诊的过程也可以得到不同地理空间位置的专家级医生的建议与指导，实现了专家级医生摆脱地理空间上制约给更大范围的患者提供专业的服务，当然也大大方便了医生对患者治疗过程的追踪和监测，更有助于治疗方案的及时推进、调整和完善。
本公开实施例，能够获取用户在当前时刻的多帧当前口腔内图像，并基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云，从而针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。采用上述技术方案，能够实现第一牙齿点云和第二牙齿点云的自动对比，相比于传统的医生肉眼观察对比，可方便、快捷、准确地获取第一牙齿点云和第二牙齿点云之间的偏差值，从而提高正畸治疗监测效率。
图2是本公开实施例提供的另一种正畸治疗监测方法的流程图。本公开实施例在上述实施例的基础上进行优化，本公开实施例可以与上述一个或者多个实施例中各个可选方案结合。
如图2所示，该正畸治疗监测方法可以包括如下步骤。
S210、获取用户在当前时刻的多帧当前口腔内图像。
具体地，S210与S110类似，此处不再赘述。
S220、针对每帧当前口腔内图像，确定当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置，其中，牙齿位置用于指示牙齿在牙颌上的排列位置。
在本公开实施例中，每帧当前口腔内图像中包括至少一个牙齿，因此，针对每帧当前口腔内图像，可以根据该当前口腔内图像确定该当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置，如此，可得到任一帧当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置。
具体地，第一牙齿子点云为基于当前口腔内图像中牙齿图像(或者说牙齿区 域)进行三维重建得到的点云。
具体地，牙颌(上牙颌、下牙颌)上的所有牙齿通常按照顺序依次排列，牙齿位置用于指示牙齿在其所属的牙颌上的排列位置，例如，从左向右数的第n颗牙齿、或从右向左数的第m颗牙齿等。
在一些实施例中，当前口腔内图像中包括RGB图像和深度图像；其中，S220可以包括：
S221、将RGB图像进行实例分割，得到实例分割结果；
具体地，可采用任意可能的实例分割方式对RGB图像进行实例分割，此处不作限定。
S222、基于实例分割结果，将RGB图像和深度图像进行区域分割，得到当前口腔内图像中每个牙齿的牙齿RGB图像和牙齿深度图像；
具体地，牙齿RGB图像为RGB图像中牙齿所在图像区域，牙齿深度图像为深度图像中牙齿所在图像区域。
具体地，由于RGB图像和深度图像的像素点之间具有一一对应关系(称之为第一关联关系)，因此基于实例分割结果可以从RGB图像中分割出每个牙齿的牙齿RGB图像，并且基于每个牙齿的牙齿RGB图像、以及第一关联关系，可以从深度图像中分割出与牙齿RGB图像对应的牙齿RGB图像。
S223、针对每个牙齿，基于牙齿的牙齿RGB图像和牙齿深度图像进行三维重建，得到牙齿的第一牙齿子点云；
具体地，RGB图像提供了点的颜色信息，并且，RGB图像还提供了点在像素坐标系下的x，y坐标，而深度图像直接提供了相机坐标系下的□坐标，也就是相机与点的距离，因此，根据相机的内参，可以计算出任何一个点在相机坐标系下的坐标，从而实现三维重建，得到牙齿的第一牙齿子点云。
当然，本领域技术人员也可以将当前口腔内图像输入经过训练的第一神经网络模型，得到该第一神经网络模型输出的当前口腔内图像中每个牙齿的第一牙齿子点云。
S224、针对每个牙齿，基于牙齿的第一牙齿子点云、牙齿RGB图像以及用户的口腔内多个牙齿的第二牙齿点云，确定牙齿的牙齿位置。
具体地，可以将第一牙齿子点云、牙齿RGB图像以及用户的口腔内多个牙齿的第二牙齿点云输入经过训练的第二神经网络模型，得到该第二神经网络模型输出的第一牙齿子点云对应的牙齿的牙齿位置。但并不限于此。
可以理解的是，通过实例分割以及三维重建得到牙齿的第一牙齿子点云，可使第一牙齿子点云的获取方式逻辑简单、易于实现，有利于降低第一牙齿子点云的获取难度。
S230、针对每个目标牙齿，获取多帧当前口腔内图像中与目标牙齿的牙齿位 置相同的牙齿的第一牙齿子点云，得到目标牙齿对应的至少一个第一牙齿子点云。
在本公开实施例中，每帧当前口腔内图像中包括至少一个牙齿，而不同当前口腔内图像可能会包括重合的牙齿，因此，同一牙齿可能出现在不同当前口腔内图像中，如此，经过S220可以得到同一牙齿的一个甚至多个第一牙齿子点云。由于根据牙齿的牙齿位置可以确定该牙齿是牙颌上的哪颗牙齿，因此，针对每个目标牙齿，当前口腔内图像中与该目标牙齿的牙齿位置相同的牙齿即为该目标牙齿，从包含该目标牙齿的当前口腔内图像对应的多个第一牙齿子点云中选取出该目标牙齿的第一牙齿子点云，即可得到该目标牙齿对应的至少一个第一牙齿子点云。
S240、针对每个目标牙齿，基于目标牙齿对应的至少一个第一牙齿子点云，确定目标牙齿的第一牙齿点云。
在本公开实施例中，电子设备可以基于目标牙齿对应的至少一个第一牙齿子点云，确定目标牙齿更完整的第一牙齿点云。
具体地，若当前口腔内图像中包含完整的目标牙齿，则基于从该当前口腔内图像中分割出的目标牙齿的牙齿RGB图像和RGB深度图像进行三维重建可得到目标牙齿完整的点云，即三维重建得到的第一牙齿子点云即为第一牙齿点云。
通常情况下，针对某一牙齿，由于图像采集角度等原因，当前口腔内图像中的该牙齿通常是不完整的，即仅包括该牙齿的一部分，此时，通过三维重建得到的第一牙齿子点云仅对应于该牙齿的一部分，而不同当前口腔内图像的图像采集角度通常不同，因此，基于多帧前口腔内图像得到的该牙齿的多个第一牙齿子点云通常包括该牙齿的不同部分，如此，根据该牙齿的多个第一牙齿子点云可以得到该牙齿更完整的第一牙齿点云。
在一些实施例中，基于目标牙齿对应的至少一个第一牙齿子点云，确定目标牙齿的第一牙齿点云，包括：对目标牙齿对应的至少一个第一牙齿子点云进行融合处理后再进行采样处理，得到目标牙齿的第一牙齿点云。
具体地，可采用任意可能的融合处理方式对目标牙齿对应的至少一个第一牙齿子点云进行融合，此处不作限定。
可以理解的是，通过融合处理和采样处理，可得到目标牙齿的更完整、更精简的第一牙齿点云。一方面，由于第一牙齿点云更完整，因此，后续通过对第一牙齿点云和第二牙齿点云的对比，可实现对目标牙齿在不同阶段的牙齿状况进行更加细致的对比，使得对比得到的偏差值更符合目标牙齿的真实生长变化情况，即提高牙齿正畸监测的准确性，另一方面，由于第一牙齿点云更精简，即数据量更小，因此，可降低第一牙齿点云和第二牙齿点云进行对比时的计算量，提高对比效果，从而提高牙齿正畸监测的整体效率。
S250、针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当 前时刻之前的牙齿点云。
具体地，S250与S130类似，此处不再赘述。
本公开实施，能够确定当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置，并根据牙齿位置获取目标牙齿对应的至少一个第一牙齿子点云，从而基于目标牙齿对应的至少一个第一牙齿子点云，确定目标牙齿的第一牙齿点云，可使第一牙齿点云的获取方式简单、易于实现，并且由于第一牙齿点云更完整，因此可实现对目标牙齿在不同阶段的牙齿状况进行更加细致的对比，进而使得对比得到的偏差值更符合目标牙齿的真实生长变化情况，即提高牙齿正畸监测的准确性。
在本公开另一种实施方式中，将RGB图像进行实例分割，得到实例分割结果，包括：
S2211、将RGB图像输入基于Swin Transformer的特征提取网络中提取特征，得到特征图；
具体地，Swin transformer是一种深度学习模型，可以用于计算机视觉任务的通用主干网络，可以用于图像分类、图像分割、目标检测等一系列视觉下游任务。
S2212、基于特征图，确定实例分割结果。
示例性的，图3是本公开实施例提供的一种实例分割的逻辑示意图。参见图3，首先，使用基于Swin Transformer的特征提取网络311，对输入的RGB图像321进行特征提取得到特征图322；然后，使用区域候选网络(Region Proposal Network，RPN)312对得到的特征图322提取感兴趣的区域，并进行一次定位操作，得到初步的正样本及其定位信息，记为候选框323；接着，将得到的特征图322以及候选框323输入到感兴趣区域校准层(Region of interest Align，Rol Align)313，通过回归得到定位信息、类别信息以及固定尺寸的特征图，检出框生成网络315基于定位信息生成检出框324，分类网络316基于类别信息生成分类结果325，将该固定尺寸的特征图使用基于全卷积网络(Fully Convolution Nets，FCN)的图像分割网络314执行语义分割，从而得到最终的实例分割结果326。
可以理解的是，MaskRCNN由FasterRCNN衍生得到，具体来说，在FasterRCNN的输出模块添加一个Mask分支，用于对每个边界框内的物体进行分割。其中，用于特征图提取的特征提取网络十分关键，因为MaskRCNN会在特征提取网络不同层次的特征图上寻找感兴趣区域，用于后续的目标精定位。以往基于CNN的特征提取网络的感受野具有局部性，通常需要通过加深网络来实现感受野的扩大，而基于注意力的方法能够捕获全局的区域关系。在针对口腔内的牙齿图像(即当前口腔内图像中的RGB图像)进行实例分割时，考虑到口腔内的非牙齿区域外观较为杂乱，且部分牙齿与牙龈交界线并不明显，因此，可以选用SwinTransformer作 为MaskRCNN的特征提取网络，一方面，它设计了类似CNN的滑动窗口机制，只计算每个窗口内的patch之间的注意力，从而解决了初代Vision Transformer计算量较大的问题；另一方面，它通过像素循环移动的方式实现了不同窗口之间的信息交互，使得不同区域之间的信息流通顺畅，从而能够提升最终的牙齿实例分割效果。
在本公开又一种实施方式中，基于牙齿的第一牙齿子点云、牙齿RGB图像以及用户的口腔内多个牙齿的第二牙齿点云，确定牙齿的牙齿位置，包括：
S2241、针对每个第二牙齿点云，将第一牙齿子点云与第二牙齿点云进行粗配准后，确定第二牙齿点云投影到第一牙齿子点云对应的牙齿RGB图像所在平面上时投影区域与牙齿RGB图像的交叠区域的区域交并比；
具体地，粗配准是指在两个点云的相对位姿完全未知的情况下进行配准，通过粗配准可实现第一牙齿子点云与第二牙齿点云的粗略对齐。
具体地，当目标牙齿为上颌上的牙齿时，用户的口腔内多个牙齿为上颌上的所有牙齿，当目标牙齿为下颌上的牙齿时，用户的口腔内多个牙齿为下颌上的所有牙齿，当然，用户的口腔内多个牙齿为也可以为用户口腔内的所有牙齿，对此不作限定。
可采用任意可能的粗配准方法对第一牙齿子点云与第二牙齿点云进行粗配准，此处不作限定。例如，可以采用基于特征的粗配准方法、基于随机抽样一致算法(random sample consensus，RANSAC)框架的粗配准方法等，但并不限于此。
具体地，第一牙齿子点云与第二牙齿点云进行粗配准后，第一牙齿子点云与第二牙齿点云所在坐标系均为第一牙齿子点云对应的相机坐标系，因此，根据相机的内参，可以计算出第二牙齿点云上任何一个点在该相机坐标系下的坐标投影到像素坐标系下的坐标，从而得到投影区域，如此，可计算投影区域与第一牙齿子点云对应的牙齿RGB图像之间的交叠区域，还可以计算投影区域与第一牙齿子点云对应的牙齿RGB图像取并集之后的并集区域，进而计算交叠区域与并集区域的比值，得到交叠区域的区域交并比。
S2242、将区域交并比最大的第二牙齿点云对应的牙齿的牙齿位置作为目标牙齿的牙齿位置。
具体地，针对每个目标牙齿，将目标牙齿的第一牙齿子点云与用户的口腔内多个牙齿的第二牙齿点云进行遍历粗配准、投影得到交叠区域、以及计算交叠区域的区域交并比，并将区域交并比最大的牙齿的牙齿位置赋给该目标牙齿。
可以理解的是，通过遍历用户口腔内多个牙齿中的每个牙齿与目标牙齿进行匹配(即粗配准以及计算区域交并比)，并将与目标牙齿最匹配的牙齿的牙齿位置赋给目标牙齿，可使牙齿位置的确定方式简单、易于实现，并且可避免与口腔内的某一牙齿漏匹配，从而找到与目标牙齿最匹配的牙齿，有利于提高牙齿位置的 确定精度。
在本公开再一种实施方式中，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，包括：对第一牙齿点云和第二牙齿点云进行粗配准，得到第一转换矩阵；继续对第一牙齿点云和第二牙齿点云进行精配准，得到第二转换矩阵；基于第一转换矩阵和第二转换矩阵，得到目标牙齿对应的偏差值。
具体地，粗配准可以为精配准提供良好的转换矩阵初值(即第一转换矩阵)。
本领域技术人员可采用任意可能的粗配准方法对第一牙齿点云和第二牙齿点云继续进行粗配准，此处不作限定。例如，采用Lepard方法等，但并不限于此。
具体地，精配准是在已知转换矩阵的初值(即第一转换矩阵)的情况下(初值大概已经是正确的了)，进一步计算得到更加精确的转换矩阵。
本领域技术人员可采用任意可能的精配准算法对粗配准之后的第一牙齿点云和第二牙齿点云继续进行精配准，此处不作限定。例如，可以采用迭代最近点算法(Iterative Closest Point，IPC)、非刚性最近点迭代(Non-rigid Iterative Closest Point，NICP)等，但并不限于此。
具体地，第一转换矩阵包括第一旋转矩阵和第一平移矩阵，第二转换矩阵包括第二旋转矩阵和第二平移矩阵，第一旋转矩阵和第二旋转矩阵的乘积、以及第一平移矩阵和第二平移矩阵的乘积可以分别作为目标牙齿的第一牙齿点云和第二牙齿点云的旋转矩阵、以及平移矩阵。基于旋转矩阵可以确定目标牙齿的姿态偏差，基于平移矩阵可以确定目标牙齿的位置偏差。
可以理解的是，先对第一牙齿点云和第二牙齿点云进行粗配准，可实现二者的快速初步对齐，再在二者初步对齐的基础上，继续进行精配准以实现更加精细的对齐效果，如此，可提高第一牙齿点云和第二牙齿点云的对齐速度和对齐精度。
当然，在得到目标牙齿的第一牙齿点云和第二牙齿点云之间的旋转矩阵、以及平移矩阵后，还可以根据旋转矩阵、以及平移矩阵将第二牙齿点云进行旋转、平移，并计算旋转、平移后的第二牙齿点云与第一牙齿点云的误差，从而验证旋转矩阵、以及平移矩阵的准确度，以便在误差超过阈值时重新对目标牙齿的第一牙齿点云和第二牙齿点云进行对比、或通知相关人员检查正畸治疗监测是否出现问题。
图4是本公开实施例提供的一种正畸治疗监测装置的结构示意图，该正畸治疗监测装置可以被理解为上述电子设备或者上述电子设备中的部分功能模块。如图4所示，该正畸治疗监测装置400包括：
第一获取模块410，用于获取用户在当前时刻的多帧当前口腔内图像；
第一确定模块420，用于基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云；
对比模块430，用于针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。
在本公开另一种实施方式中，第一确定模块420可以包括：
第一确定子模块，用于针对每帧当前口腔内图像，确定当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置，其中，牙齿位置用于指示牙齿在牙颌上的排列位置；
第一获取子模块，用于针对每个目标牙齿，获取多帧当前口腔内图像中与目标牙齿的牙齿位置相同的牙齿的第一牙齿子点云，得到目标牙齿对应的至少一个第一牙齿子点云；
第二确定子模块，用于针对每个目标牙齿，基于目标牙齿对应的至少一个第一牙齿子点云，确定目标牙齿的第一牙齿点云。
在本公开又一种实施方式中，当前口腔内图像中包括RGB图像和深度图像；
其中，第一确定子模块可以包括：
实例分割单元，用于将RGB图像进行实例分割，得到实例分割结果；
区域分割单元，用于基于实例分割结果，将RGB图像和深度图像进行区域分割，得到当前口腔内图像中每个牙齿的牙齿RGB图像和牙齿深度图像；
三维重建单元，用于针对每个牙齿，基于牙齿的牙齿RGB图像和牙齿深度图像进行三维重建，得到牙齿的第一牙齿子点云；
第一确定单元，用于针对每个牙齿，基于牙齿的第一牙齿子点云、牙齿RGB图像以及用户的口腔内多个牙齿的第二牙齿点云，确定牙齿的牙齿位置。
在本公开再一种实施方式中，实例分割单元可以包括：
特征提取子单元，用于将RGB图像输入基于Swin Transformer的特征提取网络中提取特征，得到特征图；
第一确定子单元，用于基于特征图，确定实例分割结果。
在本公开再一种实施方式中，第一确定单元可以包括：
粗配准和投影子单元，用于针对每个第二牙齿点云，将第一牙齿子点云与第二牙齿点云进行粗配准后，确定第二牙齿点云投影到第一牙齿子点云对应的牙齿RGB图像所在平面上时投影区域与牙齿RGB图像的交叠区域的区域交并比；
第二确定子单元，用于将区域交并比最大的第二牙齿点云对应的牙齿的牙齿位置作为目标牙齿的牙齿位置。
在本公开再一种实施方式中，第二确定子模块，可以包括：
第一确定单元，用于对目标牙齿对应的至少一个第一牙齿子点云进行融合处理后再进行采样处理，得到目标牙齿的第一牙齿点云。
在本公开再一种实施方式中，对比模块430可以包括：
粗配准子模块，用于对第一牙齿点云和第二牙齿点云进行粗配准，得到第一转换矩阵；
精配准子模块，用于继续对第一牙齿点云和第二牙齿点云进行精配准，得到第二转换矩阵；
第三确定子模块，用于基于第一转换矩阵和第二转换矩阵，得到目标牙齿对应的偏差值。
本实施例提供的装置能够执行上述任一实施例的方法，其执行方式和有益效果类似，在这里不再赘述。
本公开实施例还提供了一种电子设备，该电子设备包括：存储器，存储器中存储有计算机程序；处理器，用于执行所述计算机程序，当所述计算机程序被所述处理器执行时可以实现上述任一实施例的方法。
示例的，图5是本公开实施例中的一种电子设备的结构示意图。下面具体参考图5，其示出了适于用来实现本公开实施例中的电子设备500的结构示意图。本公开实施例中的电子设备500可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(ROM)502中的程序或者从存储装置508加载到随机访问存储器(RAM)503中的程序而执行各种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的各种程序和数据。处理装置501、ROM 502以及RAM 503通过总线504彼此相连。输入/输出(I/O)接口505也连接至总线504。
通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从 网络上被下载和安装，或者从存储装置508被安装，或者从ROM 502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：
获取用户在当前时刻的多帧当前口腔内图像；
基于多帧当前口腔内图像，确定用户的口腔内每个目标牙齿的第一牙齿点云；
针对每个目标牙齿，将目标牙齿的第一牙齿点云和目标牙齿的第二牙齿点云进行对比，得到目标牙齿对应的偏差值，其中，第二牙齿点云为牙齿在当前时刻之前的牙齿点云。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计 算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
本公开实施例还提供一种计算机可读存储介质，所述存储介质中存储有计算 机程序，当所述计算机程序被处理器执行时可以实现上述任一实施例的方法，其执行方式和有益效果类似，在这里不再赘述。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所述的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。
工业实用性
本公开提供的正畸治疗监测方法，能够实现第一牙齿点云和第二牙齿点云的自动对比，相比于传统的医生肉眼观察对比，可方便、快捷、准确地获取第一牙齿点云和第二牙齿点云之间的偏差值，从而提高正畸治疗监测效率。

Claims (10)

  1. 一种正畸治疗监测方法，其特征在于，包括：
    获取用户在当前时刻的多帧当前口腔内图像；
    基于所述多帧当前口腔内图像，确定所述用户的口腔内每个目标牙齿的第一牙齿点云；
    针对每个所述目标牙齿，将所述目标牙齿的第一牙齿点云和所述目标牙齿的第二牙齿点云进行对比，得到所述目标牙齿对应的偏差值，其中，所述第二牙齿点云为牙齿在所述当前时刻之前的牙齿点云。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述多帧当前口腔内图像，确定所述用户的口腔内每个目标牙齿的第一牙齿点云，包括：
    针对每帧所述当前口腔内图像，确定所述当前口腔内图像中每个牙齿的第一牙齿子点云以及牙齿位置，其中，所述牙齿位置用于指示牙齿在牙颌上的排列位置；
    针对每个所述目标牙齿，获取所述多帧当前口腔内图像中与所述目标牙齿的牙齿位置相同的牙齿的第一牙齿子点云，得到所述目标牙齿对应的至少一个第一牙齿子点云；
    针对每个所述目标牙齿，基于所述目标牙齿对应的至少一个第一牙齿子点云，确定所述目标牙齿的第一牙齿点云。
  3. 根据权利要求2所述的方法，其特征在于，所述当前口腔内图像中包括RGB图像和深度图像；
    其中，所述确定所述当前口腔内图像中每个所述目标牙齿的第一牙齿子点云以及牙齿位置，包括：
    将所述RGB图像进行实例分割，得到实例分割结果；
    基于所述实例分割结果，将所述RGB图像和所述深度图像进行区域分割，得到所述当前口腔内图像中每个牙齿的牙齿RGB图像和牙齿深度图像；
    针对每个所述牙齿，基于所述牙齿的牙齿RGB图像和牙齿深度图像进行三维重建，得到所述牙齿的第一牙齿子点云；
    针对每个所述牙齿，基于所述牙齿的第一牙齿子点云、牙齿RGB图像以及所述用户的口腔内多个牙齿的第二牙齿点云，确定所述牙齿的牙齿位置。
  4. 根据权利要求3所述的方法，其特征在于，所述将所述RGB图像进行实例分割，得到实例分割结果，包括：
    将所述RGB图像输入基于Swin Transformer的特征提取网络中提取特征，得到特征图；
    基于所述特征图，确定所述实例分割结果。
  5. 根据权利要求3所述的方法，其特征在于，所述基于所述牙齿的第一牙齿子点云、牙齿RGB图像以及所述用户的口腔内多个牙齿的第二牙齿点云，确定所述牙齿的牙齿位置，包括：
    针对每个第二牙齿点云，将所述第一牙齿子点云与所述第二牙齿点云进行粗配准后，确定所述第二牙齿点云投影到所述第一牙齿子点云对应的牙齿RGB图像所在平面上时投影区域与所述牙齿RGB图像的交叠区域的区域交并比；
    将区域交并比最大的第二牙齿点云对应的牙齿的牙齿位置作为所述目标牙齿的牙齿位置。
  6. 根据权利要求2所述的方法，其特征在于，所述基于所述目标牙齿对应的至少一个第一牙齿子点云，确定所述目标牙齿的第一牙齿点云，包括：
    对所述目标牙齿对应的至少一个第一牙齿子点云进行融合处理后再进行采样处理，得到所述目标牙齿的第一牙齿点云。
  7. 根据权利要求1所述的方法，其特征在于，将所述目标牙齿的第一牙齿点云和所述目标牙齿的第二牙齿点云进行对比，得到所述目标牙齿对应的偏差值，包括：
    对所述第一牙齿点云和所述第二牙齿点云进行粗配准，得到第一转换矩阵；
    继续对所述第一牙齿点云和所述第二牙齿点云进行精配准，得到第二转换矩阵；
    基于所述第一转换矩阵和所述第二转换矩阵，得到所述目标牙齿对应的偏差值。
  8. 一种正畸治疗监测装置，其特征在于，包括：
    第一获取模块，用于获取用户在当前时刻的多帧当前口腔内图像；
    第一确定模块，用于基于所述多帧当前口腔内图像，确定所述用户的口腔内每个目标牙齿的第一牙齿点云；
    对比模块，用于针对每个所述目标牙齿，将所述目标牙齿的第一牙齿点云和所述目标牙齿的第二牙齿点云进行对比，得到所述目标牙齿对应的偏差值，其中，所述第二牙齿点云为牙齿在所述当前时刻之前的牙齿点云。
  9. 一种电子设备，其特征在于，包括：
    处理器和存储器，其中，所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行权利要求1-7中任一项所述的方法。
  10. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有计算机程序，当所述计算机程序被处理器执行时，实现如权利要求1-7中任一项所述的方法。
PCT/CN2023/117857 2022-10-28 2023-09-08 正畸治疗监测方法、装置、设备及存储介质 WO2024087910A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211337209.8 2022-10-28
CN202211337209.8A CN115797595A (zh) 2022-10-28 2022-10-28 正畸治疗监测方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024087910A1 true WO2024087910A1 (zh) 2024-05-02

Family

ID=85434283

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/117857 WO2024087910A1 (zh) 2022-10-28 2023-09-08 正畸治疗监测方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN115797595A (zh)
WO (1) WO2024087910A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115797595A (zh) * 2022-10-28 2023-03-14 先临三维科技股份有限公司 正畸治疗监测方法、装置、设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110276758A (zh) * 2019-06-28 2019-09-24 电子科技大学 基于点云空间特征的牙咬合分析系统
CN110960333A (zh) * 2019-11-21 2020-04-07 西安博恩生物科技有限公司 基于ai算法的错颌畸形矫治模型匹配方法
CN111784754A (zh) * 2020-07-06 2020-10-16 浙江得图网络有限公司 基于计算机视觉的牙齿正畸方法、装置、设备及存储介质
CN115797595A (zh) * 2022-10-28 2023-03-14 先临三维科技股份有限公司 正畸治疗监测方法、装置、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110276758A (zh) * 2019-06-28 2019-09-24 电子科技大学 基于点云空间特征的牙咬合分析系统
CN110960333A (zh) * 2019-11-21 2020-04-07 西安博恩生物科技有限公司 基于ai算法的错颌畸形矫治模型匹配方法
CN111784754A (zh) * 2020-07-06 2020-10-16 浙江得图网络有限公司 基于计算机视觉的牙齿正畸方法、装置、设备及存储介质
CN115797595A (zh) * 2022-10-28 2023-03-14 先临三维科技股份有限公司 正畸治疗监测方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN115797595A (zh) 2023-03-14

Similar Documents

Publication Publication Date Title
WO2024087910A1 (zh) 正畸治疗监测方法、装置、设备及存储介质
WO2020199694A1 (zh) 一种脊柱Cobb角测量方法、装置、可读存储介质及终端设备
WO2022012085A1 (zh) 人脸图像处理方法、装置、存储介质及电子设备
JP7150917B2 (ja) 地図作成のためのコンピュータ実施方法及び装置、電子機器、記憶媒体並びにコンピュータプログラム
WO2024109796A1 (zh) 一种扫描头位姿检测方法、装置、设备及介质
WO2023213254A1 (zh) 一种口内扫描处理方法、系统、电子设备及介质
WO2024001959A1 (zh) 一种扫描处理方法、装置、电子设备及存储介质
CN111783626A (zh) 图像识别方法、装置、电子设备及存储介质
WO2023241704A1 (zh) 牙齿模型的获取方法、装置、设备及介质
WO2023155581A1 (zh) 一种图像检测方法和装置
CN113537153A (zh) 仪表图像识别方法、装置、电子设备和计算机可读介质
CN114993328B (zh) 车辆定位评估方法、装置、设备和计算机可读介质
CN115457196A (zh) 一种咬合调整方法、装置、设备及介质
WO2024067033A1 (zh) 标志点识别方法、装置、设备及存储介质
WO2024109646A1 (zh) 图像渲染方法、装置、设备及存储介质
WO2023198101A1 (zh) 基于人工智能的口腔检测方法、装置、电子设备及介质
WO2024109268A1 (zh) 一种数字模型比对方法、装置、设备及介质
WO2023213252A1 (zh) 扫描数据处理方法、装置、设备及介质
WO2023138558A1 (zh) 一种图像场景分割方法、装置、设备及存储介质
CN115620264B (zh) 车辆定位方法、装置、电子设备和计算机可读介质
CN111281355A (zh) 一种用于确定脉搏采集位置的方法与设备
CN114748195A (zh) 扫描装置及其连接方法、装置、电子设备及介质
CN114140771A (zh) 一种图像深度数据集自动标注方法及系统
CN116492082B (zh) 基于三维模型的数据处理方法、装置、设备及介质
US20240193807A1 (en) Gaze tracking method, apparatus, device, and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23881491

Country of ref document: EP

Kind code of ref document: A1