WO2021104061A1

WO2021104061A1 - Liver deformation prediction method and system, and electronic device

Info

Publication number: WO2021104061A1
Application number: PCT/CN2020/128849
Authority: WO
Inventors: 贾富仓; 王宇
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2019-11-26
Filing date: 2020-11-13
Publication date: 2021-06-03
Also published as: CN111063441A

Abstract

A liver deformation prediction method and system, and an electronic device, comprising: step a. obtaining deformation data of a liver tetrahedron (100); step b. performing resampling interpolation and preprocessing on the deformation data of the liver tetrahedron to generate a liver data set (200); and step c. constructing a convolutional neural network according to the liver data set (300), the convolutional neural network predicting overall deformation of the liver after obtaining displacement of a liver surface (400). After displacement of a partial liver surface is obtained under a laparoscopic image, displacement information of the entire surface of the liver and an internal structure of the liver is obtained using the convolutional neural network; an effective prediction effect on the overall displacement of the liver can be realized, and the time used for prediction can be greatly reduced.

Description

Method, system and electronic equipment for predicting liver deformation

Technical field

This application belongs to the technical field of medical image processing, and in particular relates to a liver deformation prediction method, system and electronic equipment.

Background technique

Laparoscopic liver surgery has been widely used in abdominal surgery due to its characteristics of small surgical trauma and low harm to patients. Laparoscopic surgery navigation obtains the patient's preoperative data, and obtains information such as the location of the liver lesion area and nearby blood vessels. Combined with the intraoperative laparoscopic images, it can effectively obtain more information in the narrow field of view of the laparoscopic, and guide the doctor Complete the surgical planning and reduce the occurrence of accidents, and gradually become the doctor's right-hand man. Because the patient’s abdomen needs to be inflated during the operation, pneumoperitoneum pressure and squeezing between the abdominal organs will cause the abdominal organs to deform. The preoperative data and the intraoperative image are often not in an accurate linear correspondence. The rigid registration algorithm cannot be used. Effectively unify the pre-operative data and intra-operative information, which is easy to lead the doctors to the actual operation of the operation. It is also unrealistic to reacquire patient lesion information during the operation. Only part of the organ surface can be obtained through laparoscopic images. Most of the traditional non-rigid registration is also difficult to use in actual use.

The current mainstream simulation of liver deformation in laparoscopy is to use a biomechanical model. The biomechanical model can accurately represent the complex structure of the liver with smaller irregular unit bodies, and can calculate precise physical deformation of the liver. However, due to the huge parameters and calculations, the results are accurate but time-consuming and reduce the consumption. The problem of time but low accuracy often cannot achieve real-time results. At the same time, as the boundary conditions for solving the conditions of the biomechanical model, it is difficult to obtain and determine under the conditions in the operation.

Summary of the invention

The present application provides a liver deformation prediction method, system, and electronic equipment, which aim to solve at least one of the above technical problems in the prior art to a certain extent.

In order to solve the above problems, this application provides the following technical solutions:

A method for predicting liver deformation includes the following steps:

Step a: Obtain the deformation data of the liver tetrahedron;

Step b: performing re-sampling interpolation and preprocessing on the deformation data of the liver tetrahedron to generate a liver data set;

Step c: Construct a convolutional neural network based on the liver data set, and the convolutional neural network predicts the overall liver deformation after obtaining the liver surface displacement.

The technical solution adopted in the embodiment of the present application further includes: in the step a, the obtaining the deformation data of the liver tetrahedron specifically includes: obtaining liver data from the liver CT image of the preoperative patient, and reconstructing the liver data Obtain the liver surface triangle mesh, and generate the corresponding liver tetrahedral mesh; randomly select a set size area on the liver surface as the zero-displacement boundary condition, simulate the place where the liver is fixed to the adjacent organs, and apply a certain amount of force to Another randomly set size area gives the liver the power to deform; the Elmer algorithm is used to calculate the deformed result to obtain the displacement vector information of the corresponding point; and the area with the largest displacement greater than a certain size in the liver data is removed.

The technical solution adopted in the embodiment of the present application further includes: in the step b, the re-sampling, interpolation and pre-processing of the deformation data of the liver tetrahedron are specifically: first, re-sampling the liver to 64×64×64 In a regular grid, calculate the distance from each point in the regular grid to the nearest surface point of the liver, and set the distance from the point outside the liver surface to the nearest surface point of the liver to be positive, and the point on the liver surface to the nearest surface point of the liver The distance of is negative; then, the part of the liver zero displacement point is marked to distinguish it from the non-liver internal points; for the regular grid, each point contains five data of triple information, namely: the three directions of the vector, the distance The closest distance to the liver surface and whether it is the zero displacement point.

The technical solution adopted by the embodiment of the application further includes: in the step c, the convolutional neural network includes an encoder and a decoder, the encoder is used to reduce the resolution and learn data characteristics, and the jump connection is used to copy the front-end information To the decoder, the decoder uses average pooling to reduce the data resolution and increase the number of channels; the decoder has three up-sampling and uses the nearest interpolation to double the resolution so that the output resolution of the network is the same as the input resolution.

The technical solution adopted in the embodiment of the application further includes: after the step c, it also includes: constructing a loss function to obtain an optimized convolutional neural network; the loss function adds an additional down-sampling displacement estimation and the down-sampling phase error corresponding to the label Calculation; the error function is expressed as:

In the above formula, u _i is the extra output part of the decoder when the resolution is i, u _tar,i is the label data is the downsampling result of the corresponding resolution,

For the number of corresponding resolution points, if the point is outside the liver, then O(p)=0; resolution i∈(64,32,16,8), and the final loss function is used as the weighted sum of different resolutions:

In the above formula, select the weights _{_{_{λ 64 = λ 32 = λ 16}}} = λ 8 = 1.

Another technical solution adopted by the embodiment of the present application is: a liver deformation prediction system, including:

Data acquisition module: used to acquire deformation data of the liver tetrahedron;

Data processing module: used to resample, interpolate and preprocess the deformation data of the liver tetrahedron to generate a liver data set;

The network construction module is used to construct a convolutional neural network based on the liver data set, and the convolutional neural network predicts the overall liver deformation after obtaining the liver surface displacement.

The technical solution adopted in the embodiment of the present application further includes: the data acquisition module acquiring the deformation data of the liver tetrahedron is specifically: segmenting and acquiring liver data from the liver CT image of the preoperative patient, and reconstructing the liver data to obtain the liver surface triangle Grid, and generate the corresponding liver tetrahedral grid; randomly select a set size area on the liver surface as the zero displacement boundary condition, simulate the place where the liver is fixed to the neighboring organs, and apply a certain amount of force to another random Set the size of the area to give the liver deformation power; use the Elmer algorithm to calculate the deformed result to obtain the displacement vector information of the corresponding point; and remove the area with the largest displacement greater than a certain size in the liver data.

The technical solution adopted in the embodiment of the application further includes: the data processing module performs re-sampling, interpolation and pre-processing on the deformation data of the liver tetrahedron. Specifically: First, re-sampling the liver into a 64×64×64 regular grid , Calculate the distance from each point in the regular grid to the nearest surface point of the liver, and set the distance from the point outside the liver surface to the nearest surface point of the liver to be positive, and the distance from the point on the liver surface to the nearest surface point of the liver to be negative ; Then, mark the part of the liver zero displacement point to distinguish it from the non-liver internal points; for a regular grid, each point contains five data of triple information, namely: the three directions of the vector and the nearest to the liver surface The distance and whether it is the zero displacement point.

The technical solution adopted in the embodiment of the application also includes: the convolutional neural network includes an encoder and a decoder, the encoder is used to reduce the resolution and learn data characteristics, and the skip connection is used to copy the front-end information to the decoder, and the decoder uses Average pooling is used to reduce the data resolution and increase the number of channels; the decoder has three upsampling and uses nearest interpolation to double the resolution so that the output resolution of the network is the same as the input resolution.

The technical solution adopted in the embodiment of the present application also includes a network optimization module, which is used to construct a loss function to obtain an optimized convolutional neural network; the loss function adds additional downsampling displacement estimation and downsampling corresponding to the label Phase error calculation; the error function is expressed as:

Another technical solution adopted in the embodiments of the present application is: an electronic device, including:

At least one processor; and

A memory communicatively connected with the at least one processor; wherein,

The memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform the following operations of the aforementioned liver deformation prediction method:

Step a: Obtain the deformation data of the liver tetrahedron;

Compared with the prior art, the beneficial effects produced by the embodiments of the present application are: the liver deformation prediction method, system and electronic equipment of the embodiments of the present application obtain the displacement of part of the liver surface under the laparoscopic image, and then use the convolutional neural network to obtain the liver Displacement information of the entire surface and internal structure of the liver; this application has a very effective prediction effect on the overall displacement of the liver, and can greatly reduce the time used for prediction.

Description of the drawings

FIG. 1 is a flowchart of a method for predicting liver deformation according to an embodiment of the present application;

Figure 2 is a structure diagram of a convolutional neural network;

Fig. 3 is a schematic structural diagram of a liver deformation prediction system according to an embodiment of the present application;

Fig. 4 is a schematic diagram of the hardware device structure of the liver deformation prediction method provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer and clearer, the following further describes the application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and are not used to limit the application.

Please refer to FIG. 1, which is a flowchart of a method for predicting liver deformation according to an embodiment of the present application. The liver deformation prediction method of the embodiment of the present application includes the following steps:

Step 100: Obtain deformation data of the liver tetrahedron;

In step 100, the method for acquiring liver tetrahedral deformation data is specifically: segmenting to obtain liver data from a preoperative patient's liver CT image, reconstructing the liver data to obtain a triangular mesh of the liver surface, and generating a corresponding liver tetrahedral mesh. Randomly select an area of set size on the liver surface (this application sets the area to be 2.5-5.5cm, which can be set according to actual operation) as the zero displacement boundary condition to simulate the place where the liver and adjacent organs are fixed, and apply A certain amount of force (this application is set to 0.5-1.5N, which can be set according to actual operation) to another randomly set size area (this application sets the area to be 2.5-3.5cm, which can be specified according to Set by actual operation) to give the liver the power to deform. Using Elmer software to calculate the deformed result, the displacement vector information of the corresponding point can be obtained. Remove the area where the maximum displacement is larger than a certain size in the liver data (this area is set to 15cm in this application, which can be set according to actual operation), because it does not meet the actual situation.

Step 200: Perform resampling, interpolation and preprocessing on the deformation data of the liver tetrahedron to generate a liver data set;

In step 200, the re-sampling interpolation and pre-processing specifically include: first, the liver is re-sampled into a 64×64×64 regular grid with a side length of 0.3 cm, and the distance from each point in the regular grid to the nearest surface point of the liver is calculated. The distance, and suppose that the distance from a point outside the liver surface to the nearest surface point of the liver is positive, and the distance from a point inside the liver surface to the nearest surface point of the liver is negative. Then, mark the part of the liver zero displacement point to distinguish it from the non-liver internal point (because both displacement vectors are zero vectors). For a regular grid, each point contains five data of triple information, namely: the three directions of the vector, the closest distance to the liver surface, and whether it is a zero displacement point.

Step 300: Construct a convolutional neural network according to the liver data set;

In step 300, the structure of the convolutional neural network is shown in FIG. 2, where the upper number represents the number of channels, and the lower number represents the resolution. During the learning process of the displacement field, the deformation of a part of the liver will not only affect the displacement of the surrounding points, but may also affect the entire liver. Therefore, each input point in the network may act on the output point, that is, each output point must have a receptive domain that spans the entire input. The structure of the convolutional neural network in the embodiment of this application is similar to the U-net structure. The input data size of the network is 64×64×64×5, and the output of the network is the displacement vector of the liver, which is 64×64×64×3 . The convolutional neural network includes two parts: encoder and decoder. The encoder is used to reduce the resolution and learn data characteristics, and the jump connection is used to copy the front-end information to the decoder part. The decoder uses average pooling to reduce the data resolution Increase the number of channels. Unlike U-Net, this network uses three-dimensional input data, and all convolutions are calculated in three-dimensional space. The decoder has three upsamplings, and uses nearest interpolation to double the resolution to ensure that the output resolution of the network is the same as the input resolution. The size of the convolution kernel is 3, padding=1, and the SoftSign function is used as the activation function after convolution.

Step 400: After obtaining the liver surface displacement, the convolutional neural network predicts the overall liver deformation;

In step 400, the convolutional neural network can have a very effective prediction effect on the overall liver displacement after acquiring part of the liver surface displacement, and greatly reduce the time used for prediction.

Step 500: Construct a loss function to obtain an optimized convolutional neural network;

In step 500, the loss function adds an additional down-sampling displacement estimation and the down-sampling phase error calculation corresponding to the label, so that the network can focus more on reducing errors at the encoder and the bottom layer, and the decoder can focus more on improving resolution. The error function is expressed as:

In formula (1), u _i is the extra output part of the decoder when the resolution is i, u _tar,i is the label data which is the downsampling result of the corresponding resolution,

For the number of corresponding resolution points, if the points are outside the liver, then O(p)=0. Resolution i∈(64,32,16,8), and the final loss function is used as the weighted sum of different resolutions:

In formula (2), the weight λ ₆₄ =λ ₃₂ =λ ₁₆ =λ ₈ =1 is selected.

Please refer to FIG. 3, which is a schematic structural diagram of a liver deformation prediction system according to an embodiment of the present application. The liver deformation prediction system of the embodiment of the present application includes a data acquisition module, a data processing module, a network construction module, and a network optimization module.

Data acquisition module: used to acquire the deformation data of the liver tetrahedron; the method of acquiring the liver tetrahedron deformation data is specifically: segmenting the liver data from the liver CT images of the preoperative patient, and reconstructing the liver data to obtain the liver surface triangle network Grid and generate the corresponding liver tetrahedral mesh. Randomly select an area of set size on the liver surface (this application sets the area to be 2.5-5.5cm, which can be set according to actual operation) as the zero displacement boundary condition to simulate the place where the liver and adjacent organs are fixed, and apply The force of 0.5-1.5N to another randomly set size area (this application sets the area to be 2.5-3.5cm, which can be set according to actual operation) to give the liver the power to deform. Using Elmer software to calculate the deformed result, the displacement vector information of the corresponding point can be obtained. Remove the area where the maximum displacement is greater than 15cm in the liver data, because it does not meet the actual situation.

Data processing module: used to resample, interpolate and preprocess the deformation data of the liver tetrahedron to generate a liver data set; among them, the resample interpolation and preprocessing specifically include: First, resample the liver to 64 with a side length of 0.3cm In a ×64×64 regular grid, calculate the distance from each point in the regular grid to the nearest surface point of the liver, and set the distance from the point outside the liver surface to the nearest surface point of the liver to be positive, and the point inside the liver surface The distance to the nearest surface point of the liver is negative. Then, mark the part of the liver zero displacement point to distinguish it from the non-liver internal point (because both displacement vectors are zero vectors). For a regular grid, each point contains five data of triple information, namely: the three directions of the vector, the closest distance to the liver surface, and whether it is a zero displacement point.

Network building module: used to construct a convolutional neural network based on the liver data set. The convolutional neural network predicts the overall liver deformation after obtaining part of the liver surface displacement; among them, the convolutional neural network structure is shown in Figure 2, where the above The number indicates the number of channels, and the number below indicates the resolution. During the learning process of the displacement field, the deformation of a part of the liver will not only affect the displacement of the surrounding points, but may also affect the entire liver. Therefore, each input point in the network may act on the output point, that is, each output point must have a receptive domain that spans the entire input. The structure of the convolutional neural network in the embodiment of this application is similar to the U-net structure. The input data size of the network is 64×64×64×5, and the output of the network is the displacement vector of the liver, which is 64×64×64×3 . The convolutional neural network includes two parts: encoder and decoder. The encoder is used to reduce the resolution and learn data characteristics, and the jump connection is used to copy the front-end information to the decoder part. The decoder uses average pooling to reduce the data resolution Increase the number of channels. Unlike U-Net, this network uses three-dimensional input data, and all convolutions are calculated in three-dimensional space. The decoder has three upsamplings, and uses nearest interpolation to double the resolution to ensure that the output resolution of the network is the same as the input resolution. The size of the convolution kernel is 3, padding=1, and the SoftSign function is used as the activation function after convolution.

Network optimization module: used to construct the loss function to obtain the optimized convolutional neural network; among them, the loss function adds additional down-sampling displacement estimation and label-corresponding down-sampling phase error calculation, so that the network can be in the encoder and the bottom layer More focused on reducing errors, the decoder is more focused on improving resolution. The error function is expressed as:

Data experiments prove that the convolutional neural network proposed in this application has high reliability and can be used in practical applications.

Fig. 4 is a schematic diagram of the hardware device structure of the liver deformation prediction method provided by an embodiment of the present application. As shown in Figure 4, the device includes one or more processors and memory. Taking a processor as an example, the device may also include: an input system and an output system.

The processor, the memory, the input system, and the output system may be connected by a bus or in other ways. In FIG. 4, the connection by a bus is taken as an example.

As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor executes various functional applications and data processing of the electronic device by running non-transitory software programs, instructions, and modules stored in the memory, that is, realizing the processing methods of the foregoing method embodiments.

The memory may include a program storage area and a data storage area, where the program storage area can store an operating system and an application program required by at least one function; the data storage area can store data and the like. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely provided with respect to the processor, and these remote memories may be connected to the processing system through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input system can receive input digital or character information, and generate signal input. The output system may include display devices such as a display screen.

The one or more modules are stored in the memory, and when executed by the one or more processors, the following operations of any of the foregoing method embodiments are performed:

Step a: Obtain the deformation data of the liver tetrahedron;

The above-mentioned products can execute the methods provided in the embodiments of the present application, and have functional modules and beneficial effects corresponding to the execution methods. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of this application.

The embodiments of the present application provide a non-transitory (non-volatile) computer storage medium. The computer storage medium stores computer-executable instructions, and the computer-executable instructions can perform the following operations:

Step a: Obtain the deformation data of the liver tetrahedron;

The embodiment of the present application provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer To make the computer do the following:

Step a: Obtain the deformation data of the liver tetrahedron;

The liver deformation prediction method, system and electronic equipment of the embodiments of the present application obtain the displacement of part of the liver surface under the laparoscopic image, and then use the convolutional neural network to obtain the displacement information of the entire liver surface and the internal structure of the liver; Displacement has a very effective prediction effect and can greatly reduce the time used for prediction.

The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined in this application can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to the embodiments shown in this application, but should conform to the widest scope consistent with the principles and novel features disclosed in this application.

Claims

A method for predicting liver deformation, characterized in that it comprises the following steps:

Step a: Obtain the deformation data of the liver tetrahedron;

Step b: performing re-sampling interpolation and preprocessing on the deformation data of the liver tetrahedron to generate a liver data set;

Step c: Construct a convolutional neural network based on the liver data set, and the convolutional neural network predicts the overall liver deformation after obtaining the liver surface displacement.
The method for predicting liver deformation according to claim 1, wherein, in the step a, the obtaining the deformation data of the liver tetrahedron is specifically: obtaining liver data from the liver CT image of the patient before the operation, and The liver data is reconstructed to obtain a triangular mesh on the liver surface, and a corresponding liver tetrahedral mesh is generated; an area of a set size is randomly selected on the liver surface as a zero-displacement boundary condition to simulate the fixed place between the liver and adjacent organs, and apply A certain amount of force is applied to another randomly set size area to give the liver deformation power; the Elmer algorithm is used to calculate the deformed result to obtain the displacement vector information of the corresponding point; and the liver data whose maximum displacement is greater than a certain size is removed area.
The liver deformation prediction method according to claim 2, wherein in the step b, the re-sampling, interpolation and preprocessing of the deformation data of the liver tetrahedron are specifically: first, re-sampling the liver to 64 In a ×64×64 regular grid, calculate the distance from each point in the regular grid to the nearest surface point of the liver, and set the distance from the point outside the liver surface to the nearest surface point of the liver to be positive, and the point inside the liver surface The distance to the nearest surface point of the liver is negative; then, the part of the liver zero displacement point is marked to distinguish it from the non-liver internal points; for a regular grid, each point contains five data of triple information, namely: vector Three directions, the closest distance to the liver surface, and whether it is a zero displacement point.
The liver deformation prediction method according to any one of claims 1 to 3, wherein in the step c, the convolutional neural network includes an encoder and a decoder, and the encoder is used to reduce the resolution and learn data Features: Use skip connection to copy the front-end information to the decoder. The decoder uses average pooling to reduce the data resolution and increase the number of channels; the decoder has three upsampling and uses the nearest interpolation to double the resolution, making the network The output resolution is the same as the input resolution.
The liver deformation prediction method according to claim 4, characterized in that, after step c, it further comprises: constructing a loss function to obtain an optimized convolutional neural network; the loss function adds an additional down-sampling displacement estimation corresponding to the label The down-sampling phase error calculation; the error function is expressed as:

In the above formula, u i is the extra output part of the decoder when the resolution is i, u tar,i is the label data is the downsampling result of the corresponding resolution,
For the number of corresponding resolution points, if the point is outside the liver, then O(p)=0; resolution i∈(64,32,16,8), and the final loss function is used as the weighted sum of different resolutions:

In the above formula, select the weights λ 64 = λ 32 = λ 16 = λ 8 = 1.
A liver deformation prediction system is characterized in that it comprises:

Data acquisition module: used to acquire deformation data of the liver tetrahedron;

Data processing module: used to resample, interpolate and preprocess the deformation data of the liver tetrahedron to generate a liver data set;

The network construction module is used to construct a convolutional neural network based on the liver data set, and the convolutional neural network predicts the overall liver deformation after obtaining the liver surface displacement.
The liver deformation prediction system according to claim 6, wherein the data acquisition module acquiring the deformation data of the liver tetrahedron is specifically: segmenting the liver data from the liver CT images of the preoperative patient, and comparing the liver data Reconstruct the liver surface triangle mesh and generate the corresponding liver tetrahedral mesh; randomly select a set size area on the liver surface as the zero-displacement boundary condition, simulate the place where the liver is fixed to the adjacent organs, and apply a certain amount of force Go to another randomly set size area to give the liver deformation power; use the Elmer algorithm to calculate the deformed result to obtain the displacement vector information of the corresponding point; and remove the area with the largest displacement greater than a certain size in the liver data.
The liver deformation prediction system according to claim 7, wherein the data processing module performs re-sampling, interpolation and pre-processing on the deformation data of the liver tetrahedron: firstly, re-sampling the liver to 64×64×64 In the regular grid, calculate the distance from each point in the regular grid to the nearest surface point of the liver, and set the distance from the point outside the liver surface to the nearest surface point of the liver to be positive, and the point inside the liver surface to the nearest surface point of the liver The distance of the points is negative; then, the part of the liver zero displacement point is marked to distinguish it from the non-liver internal points; for a regular grid, each point contains five data of triple information, namely: the three directions of the vector, The closest distance to the liver surface and whether it is the zero displacement point.
The liver deformation prediction system according to any one of claims 6 to 8, wherein the convolutional neural network includes an encoder and a decoder, the encoder is used to reduce the resolution and learn data features, and the jump connection is used to connect The front-end information is copied to the decoder, and the decoder uses average pooling to reduce the data resolution and increase the number of channels; the decoder has three up-sampling and uses the nearest interpolation to double the resolution, making the output resolution of the network and the input resolution The rate is the same.
The liver deformation prediction system according to claim 9, further comprising a network optimization module, which is used to construct a loss function to obtain an optimized convolutional neural network; the loss function adds additional downsampling The displacement estimation and the down-sampling phase error calculation corresponding to the label; the error function is expressed as:

In the above formula, u i is the extra output part of the decoder when the resolution is i, u tar,i is the label data is the downsampling result of the corresponding resolution,
For the number of corresponding resolution points, if the point is outside the liver, then O(p)=0; resolution i∈(64,32,16,8), and the final loss function is used as the weighted sum of different resolutions:

In the above formula, select the weights λ 64 = λ 32 = λ 16 = λ 8 = 1.
An electronic device including:

At least one processor; and

A memory communicatively connected with the at least one processor; wherein,

The memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform the liver deformation described in any one of 1 to 5 above. The following operations of the forecast method:

Step a: Obtain the deformation data of the liver tetrahedron;

Step b: performing re-sampling interpolation and preprocessing on the deformation data of the liver tetrahedron to generate a liver data set;

Step c: Construct a convolutional neural network based on the liver data set, and the convolutional neural network predicts the overall liver deformation after obtaining the liver surface displacement.