CN115862821A - Construction method of intelligent operating room based on digital twins and related device - Google Patents

Abstract

The invention relates to the field of data processing, and discloses a construction method of an intelligent operating room based on digital twins and a related device, which are used for improving the accuracy of patient information processing. The method comprises the following steps: acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin body; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; carrying out mesh division on the initial digital twin body based on multi-angle image information, and determining a plurality of corresponding twin body meshes; modifying the initial digital twins based on the twin grids to obtain target digital twins; and carrying out operation instruction setting on the target digital twin, and transmitting the target digital twin set by the operation instruction to the three-dimensional operation terminal.

Description

Construction method and related devices of smart operating room based on digital twin

technical field

The present invention relates to the field of data processing, in particular to a method for constructing a smart operating room based on digital twins and related devices.

Background technique

During traditional surgery, doctors often encounter various situations, such as being blocked by surgical instruments, etc. Usually, it is necessary to check the characteristics of patients from multiple angles in real time, and there may be errors caused by unclear surgical field of view, which are difficult to find The operating position increases the risk of surgery.

Therefore, in the process of operating on patients, there is an urgent need for a virtual simulation physical model to help doctors find the operating position more accurately, so as to improve the efficiency of patient information processing, and further improve the doctor's selection of operating positions during the operation. the accuracy rate.

Contents of the invention

The invention provides a digital twin-based smart operating room construction method and related devices, which are used to improve the accuracy of patient data processing.

The first aspect of the present invention provides a method for constructing a smart operating room based on a digital twin, and the method for constructing a smart operating room based on a digital twin includes: collecting patient information of a target patient, and performing type matching on the patient information , to obtain the corresponding information type; construct a finite element model based on the information type to obtain the corresponding initial digital twin; perform multi-angle scanning on the initial digital twin to obtain a multi-angle image corresponding to the digital twin information; performing grid division on the initial digital twin body based on the multi-angle image information, and determining corresponding multiple twin body grids; correcting the initial digital twin body based on the multiple twin body grids, Obtaining the target digital twin; performing operation instruction setting on the target digital twin, and transmitting the target digital twin set by the operation instruction to the three-dimensional operation terminal.

In conjunction with the first aspect, in the first implementation of the first aspect of the present invention, the construction of the finite element model based on the information type to obtain the corresponding initial digital twin includes: pairing Perform unit matching on the information type to obtain the corresponding information unit set; perform parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set; perform model construction through the unit parameter information to obtain the corresponding The initial digital twin of .

With reference to the first aspect, in the second embodiment of the first aspect of the present invention, the multi-angle scanning of the initial digital twin to obtain multi-angle image information corresponding to the digital twin includes: The spatial coordinate system is set, and the spatial position of the initial digital twin is determined to obtain the corresponding spatial position coordinates; based on the spatial position coordinates, the corresponding scanning angle information set is determined; based on the scanning angle information set, the The initial digital twin is scanned from multiple angles to obtain multi-angle image information corresponding to the digital twin.

With reference to the second implementation of the first aspect, in the third implementation of the first aspect of the present invention, the initial digital twin is meshed based on the multi-angle image information, and the corresponding multiple twins are determined A volume grid, including: analyzing the key position points of the initial digital twin body through the spatial coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body; according to each of the key position points Region expansion, to obtain a plurality of corresponding extended three-dimensional regions; performing image matching on the plurality of extended three-dimensional regions and the multi-angle image information, to obtain image information to be processed corresponding to each extended three-dimensional region; The to-be-processed image information corresponding to the three-dimensional area is subjected to sweeping division processing respectively, and corresponding multiple twin body meshes are obtained.

In combination with the first aspect, in the fourth embodiment of the first aspect of the present invention, the correction of the initial digital twin based on the plurality of twin grids to obtain the target digital twin includes: Perform virtual point cloud data mapping on multiple twin grids to obtain corresponding virtual point cloud data sets; perform virtual semantic assignment on the virtual point cloud data sets to obtain corresponding virtual semantic point cloud data; The cloud data is corrected for the depth truth value to obtain the target point cloud data set; the initial digital twin is corrected through the target point cloud data set to obtain the target digital twin.

In conjunction with the fourth implementation of the first aspect, in the fifth implementation of the first aspect of the present invention, the virtual point cloud data mapping is performed on the multiple twin grids to obtain the corresponding virtual point cloud data set, Including: performing texture image conversion on the plurality of twin body grids to obtain corresponding multiple twin body texture images; based on a preset feature database, performing two-dimensional feature matching on the plurality of twin body texture images to obtain The target two-dimensional feature set corresponding to the plurality of twin body texture images; performing mapping relationship matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relationship; The volume grid is mapped to virtual point cloud data to obtain the corresponding virtual point cloud data set.

With reference to the fourth implementation of the first aspect, in the sixth implementation of the first aspect of the present invention, the virtual semantic assignment of the virtual point cloud data set to obtain the corresponding virtual semantic point cloud data includes: based on The preset point cloud semantic database performs semantic matching on the virtual point cloud data set to obtain semantic information corresponding to the virtual point cloud data set; performs virtual semantics on the virtual point cloud data set based on the semantic information Assign a value to get the corresponding virtual semantic point cloud data.

The second aspect of the present invention provides a digital twin-based smart operating room construction device, the digital twin-based smart operating room construction device includes:

An acquisition module, configured to acquire patient information of a target patient, and perform type matching on the patient information to obtain a corresponding information type;

A building block for building a finite element model based on the information type to obtain a corresponding initial digital twin;

A scanning module, configured to perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin;

A division module, configured to perform grid division on the initial digital twin based on the multi-angle image information, and determine corresponding multiple twin grids;

A correction module, configured to correct the initial digital twin based on the plurality of twin grids to obtain a target digital twin;

The setting module is configured to perform operation instruction setting on the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal.

The third aspect of the present invention provides a digital twin-based smart operating room construction equipment, including: a memory and at least one processor, instructions are stored in the memory; the at least one processor calls all the instructions in the memory The above instructions are used to make the construction equipment of the digital twin-based smart operating room execute the above-mentioned construction method of the digital twin-based smart operating room.

The fourth aspect of the present invention provides a computer-readable storage medium, and instructions are stored in the computer-readable storage medium, and when it is run on a computer, the computer executes the construction of the above-mentioned digital twin-based smart operating room method.

In the technical solution provided by the present invention, the patient information of the target patient is collected, and the patient information is type-matched to obtain the corresponding information type; the finite element model is constructed based on the information type to obtain the corresponding initial digital twin; the initial digital twin Multi-angle scanning of the body to obtain multi-angle image information corresponding to the digital twin body; based on the multi-angle image information, the initial digital twin body is meshed to determine the corresponding multiple twin body grids; based on multiple twin body grids Correct the initial digital twin to obtain the target digital twin; set the operation instruction for the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal. The present invention uses the finite element model of virtual simulation Carry out the construction of the target digital twin, and then set the operation instruction for the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal, so as to help the doctor find the operation position more accurately, so as to improve the patient's The efficiency of information processing further improves the accuracy of the doctor's selection of the operation position during the operation.

Description of drawings

FIG. 1 is a schematic diagram of an embodiment of a method for constructing a smart operating room based on digital twins in an embodiment of the present invention;

Fig. 2 is the flowchart of carrying out grid division to initial digital twin body in the embodiment of the present invention;

Fig. 3 is a flow chart of correcting the initial digital twin body in the embodiment of the present invention;

FIG. 4 is a schematic diagram of an embodiment of a digital twin-based smart operating room construction device in an embodiment of the present invention;

Fig. 5 is a schematic diagram of an embodiment of a digital twin-based smart operating room construction device in an embodiment of the present invention.

Detailed ways

Embodiments of the present invention provide a digital twin-based smart operating room construction method, device, equipment, and storage medium for improving the accuracy of patient data processing. The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the term "comprising" or "having" and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to those explicitly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

For ease of understanding, the following describes the specific process of the embodiment of the present invention, please refer to Figure 1, an embodiment of the construction method of the digital twin-based smart operating room in the embodiment of the present invention includes:

S101. Collect the patient information of the target patient, and perform type matching on the patient information to obtain the corresponding information type;

It can be understood that the execution subject of the present invention may be a construction device of a smart operating room based on a digital twin, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as an execution subject as an example.

Specifically, the server extracts the entity words in the patient information of the target patient, and performs boundary proofreading on the extracted entity words. In this embodiment, a preset text information classifier is used to perform syntactic analysis on the patient information, and the preset The entity tagging model in the text information classifier marks out the entity words and extracts the labeled entity words. In order to improve the semantic integrity of the entity words in this embodiment, the word segmentation results are used to perform boundary proofreading on each extracted entity word. The proofreaded entity words are analyzed, and the target entity words are screened out. After receiving the patient information of the target patient, semantic analysis may also be performed on the patient information of the target patient, so as to obtain entity words corresponding to the patient information of the target patient. Specifically, the type matching of patient information is first the normalization of entity words and the feature extraction of synonyms. Unification processing, for example, standardize numerals, quantifiers, and units to facilitate later judgment and report generation. In the synonym feature extraction, each extracted entity word can be extracted according to the synonym lexicon to ensure the consistency of information in the process of patient information collection, and to avoid the phenomenon of duplication of the same patient information. After completing the standardization process of the above entity words, perform cluster analysis on multiple entity words to obtain the entity class corresponding to the target patient, and then perform information type matching on the entity class to obtain the information type corresponding to the patient information.

S102. Constructing a finite element model based on the information type to obtain a corresponding initial digital twin;

Specifically, the unit parameters and grid type variables are set based on the information type, and then combined with the intelligent optimization algorithm for combined optimization analysis to obtain the corresponding information unit set, and then the parameter analysis is performed on the information unit set to construct the initial digital twin. In this embodiment, different element types are selected for different information types, and the applicable simulation ranges of these element types are different; for the selection of grid type and grid size, the solid element selects tetrahedron or hexahedron with different numbers of nodes, and the network Whether the grid size is reasonable or not is judged by the unit parameter information after adjusting the grid fineness twice. If the change value of the unit parameter information meets the preset threshold, the model is constructed according to the unit parameter information to obtain the initial digital twin. Among them, the modification of the finite element model and the data transfer of the analysis results are through the optimization process of the unit parameter information to generate a more accurate and reliable initial digital twin and perform load simulation analysis, and then compare and analyze the error of the unit parameter information.

S103. Perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin;

It should be noted that the coordinate values of multiple position points on the initial digital twin body in the digital twin body are determined, where the initial digital twin body is the digital twin model of the medical equipment entity in the smart operating room, and the medical equipment entity is the smart surgery Any equipment entity in the room; based on the coordinate transformation matrix between the real space and the digital twin, the coordinates of multiple position points on the initial digital twin are transformed in the digital twin to obtain the initial digital twin The standard coordinate values of multiple position points on the real space; the position points corresponding to the standard coordinate values of the multiple position points on the initial digital twin body in the real space are determined as the multiple position points on the initial digital twin body. Multiple positioning standard points in the real space, and multiple high-precision positioning devices are respectively installed on multiple standard positioning points, and image information corresponding to each positioning standard point is collected according to multiple positioning standard points to obtain multi-angle image information.

S104. Perform grid division on the initial digital twin based on the multi-angle image information, and determine corresponding multiple twin grids;

Specifically, the initial digital twin can be divided into multiple grid areas based on multi-angle image information. The multiple grid areas are multiple twin grids, and there are digital twin equipment in the initial digital twin , the twin grid where the digital twin equipment is located in the initial digital twin is the corresponding twin grid in the corresponding initial digital twin, and each twin grid corresponds to a central control center, mainly used It is responsible for taking corresponding measures when the smart operating room equipment corresponding to the twin grid is in an abnormal working state. Specifically, when it is recognized that the smart operating room equipment is in an abnormal working state, it can be searched and determined according to the equipment identification number corresponding to the smart operating room equipment and its corresponding twin grid in the initial digital twin, or through image processing technology determines its corresponding twin mesh in the initial digital twin.

S105. Correcting the initial digital twin based on multiple twin grids to obtain a target digital twin;

Specifically, after mapping the virtual point cloud data to the twin body texture image according to the preset feature database, it is equivalent to simultaneously mapping the virtual point cloud data of each twin body grid to the original digital twin body in the twin body texture image. occupied area. Therefore, the target point cloud data refers to the point cloud data in the area covered by the initial digital twin corresponding to each twin mesh on the twin texture image after the initial point cloud is projected onto the twin texture image. According to the preset mapping relationship and the initial digital twin, determine each twin grid in the virtual point cloud data; and obtain the target point cloud covered by the initial digital twin, and map the virtual point cloud data to the twin texture according to the mapping relationship In the image, each twin body grid is mapped to the area occupied by the initial digital twin body in the twin body texture image to obtain the target digital twin body.

S106. Set an operation instruction for the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal.

Specifically, according to the operation instruction mapped on the target digital twin, determine the corresponding position of the operation instruction on the target digital twin, and obtain the operation position; according to the corresponding position of the operation position on the target digital twin, determine that the operation position is on the target The real operation corresponding to the 3D operation terminal on the digital twin, the equipment and components on the target digital twin and the equipment or components in the real operating space have a preset ratio, so that the physical model of the doctor through virtual simulation is more accurate to find the operating position.

In the embodiment of the present invention, the patient information of the target patient is collected, and the patient information is type-matched to obtain the corresponding information type; the finite element model is constructed based on the information type to obtain the corresponding initial digital twin; Multi-angle scanning to obtain multi-angle image information corresponding to the digital twin; based on the multi-angle image information, the initial digital twin is meshed to determine the corresponding multiple twin meshes; based on multiple twin meshes, the initial The digital twin is corrected to obtain the target digital twin; the operation instruction is set for the target digital twin, and the target digital twin set by the operation instruction is transmitted to the three-dimensional operation terminal. The digital twin is constructed, and then the operation instruction is set for the target digital twin, and the target digital twin set by the operation instruction is transmitted to the three-dimensional operation terminal, which helps doctors find the operation position more accurately and improves the patient information. The processing efficiency further improves the accuracy of the operation position selection during the operation of the doctor.

In a specific embodiment, the process of performing step S102 may specifically include the following steps:

(1) Perform unit matching on the information type through the preset finite element model database to obtain the corresponding information unit set;

(2) Perform parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set;

(3) Model construction is carried out through the unit parameter information to obtain the corresponding initial digital twin.

Specifically, the unit matching of the information type is carried out through the preset finite element model database, and the corresponding information unit set is obtained, and the parameter analysis is performed on the information unit set, and the unit parameter information corresponding to the information unit set is obtained, and the model is carried out through the unit parameter information. Construct and obtain the corresponding initial digital twin. Specifically, set unit parameters and grid type variables based on the information type, and then combine the intelligent optimization algorithm for combined optimization analysis to obtain the corresponding information unit set, and then perform Parametric analysis proceeds to build an initial digital twin.

In a specific embodiment, the process of performing step S103 may specifically include the following steps:

(1) Determine the spatial position of the initial digital twin based on the preset spatial coordinate system, and obtain the corresponding spatial position coordinates;

(2) Determine the corresponding scanning angle information set based on the spatial position coordinates;

(3) Based on the scanning angle information set, the initial digital twin is scanned from multiple angles to obtain the multi-angle image information corresponding to the digital twin.

Specifically, the image of the medical equipment entity is taken at the preset position and preset direction of the space coordinate system, wherein the medical equipment entity is any equipment entity in the smart operating room; the space of the medical equipment entity is determined based on the image of the medical equipment entity position, and construct the spatial position coordinates based on the spatial position of the medical equipment entity; in the spatial position coordinates, the preset position and direction are taken as the origin of the field of view, and the position and angle of the initial digital twin are adjusted multiple times. Intercept an image of the initial digital twin to obtain multiple target digital twin matching templates, among which the initial digital twin is the target digital twin of the medical equipment entity; based on the preset image template matching algorithm, determine the twin texture images respectively The matching degree of the matching template and the matching template of multiple target digital twins is obtained to obtain multiple matching degrees; the position and angle of the initial digital twin body corresponding to the maximum matching degree among the multiple matching degrees are determined as the initial digital twin body in the digital The position and angle of the twins are obtained to obtain the set of scanning angle information. Transform the position of an intersection point between the medical equipment entity and the ground of the smart operating room in the multi-angle image information obtained from the shooting to the digital twin as the origin of the grid, centering on the origin of the grid, along the x-axis and y-axis of the digital twin The axis draws grid lines on the ground of the digital twin, sets the unit length of the distance between adjacent grid lines and the number of grid lines, and adjusts the position and angle of the initial digital twin multiple times to obtain a digital twin Corresponding multi-angle image information.

In a specific embodiment, as shown in FIG. 2, the process of performing step S104 may specifically include the following steps:

S201. Analyze the key position points of the initial digital twin body through the spatial coordinate system, and determine multiple key position points corresponding to the initial digital twin body;

S202. Perform area expansion according to each key position point to obtain multiple corresponding expanded three-dimensional areas;

S203. Perform image matching on multiple expanded stereoscopic regions and multi-angle image information, to obtain image information to be processed corresponding to each extended stereoscopic region;

S204. Perform sweeping division processing on the image information to be processed corresponding to each expanded three-dimensional area to obtain multiple corresponding twin volume meshes.

Specifically, analyze the key position points of the initial digital twin body through the spatial coordinate system, determine multiple key position points corresponding to the initial digital twin body, and specifically use the mean shift segmentation algorithm to quickly identify the initial digital twin body and effectively locate the multiple key locations. Then expand the area according to each key position point to obtain multiple corresponding expanded three-dimensional areas. First, divide the smart operating room corresponding to the initial digital twin into multiple areas according to the set size and number each area. The basic attributes of all areas in the smart operating room can be counted, and then based on the multiple areas and the upper limit of the number of grids determined by the number of areas, multiple extended three-dimensional areas can be generated, that is, set according to the total number of divided areas The upper limit of the number of grids (the upper limit of the number of grids), based on the multiple regions and according to the upper limit of the number of grids, a certain number of initial twin body grid division schemes can be generated, which can further be used, for example, across grid lines The number should be as small as possible. According to the preset division target, based on the basic properties of the above-mentioned statistical regions, the evaluation results corresponding to each initial twin body grid division scheme can be obtained, and then the target twin body network can be determined according to the quality of the evaluation results. grid division scheme, and iteratively update the twin body mesh division scheme, and finally find the best grid division scheme as the target twin body mesh division scheme. After obtaining the target twin body mesh division scheme, the initial Multiple twin grids are divided in the area corresponding to the digital twin. Further, performing image matching on a plurality of extended stereoscopic regions and multi-angle image information to obtain image information to be processed corresponding to each extended stereoscopic region; performing sweeping division on the image information to be processed corresponding to each extended stereoscopic region The corresponding multiple twin grids are obtained. Through the scheme of this embodiment, the twin grids that are convenient for each user to manage the relevant smart operating room equipment and deal with the abnormal state in time can be obtained, and finally the corresponding multiple twin grids are obtained. Twin mesh.

In a specific embodiment, as shown in FIG. 3, the process of performing step S105 may specifically include the following steps:

S301. Perform virtual point cloud data mapping on multiple twin grids to obtain corresponding virtual point cloud data sets;

S302. Perform virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data;

S303. Perform depth truth correction on the virtual semantic point cloud data to obtain a target point cloud data set;

S304. Correct the initial digital twin through the target point cloud data set to obtain the target digital twin.

Specifically, the target point cloud data is a collection of point cloud data in an area on the target digital twin that matches the position of the initial digital twin. That is, the target point cloud data refers to the collection of points covering the initial digital twin. The points in the target point cloud data all come from the aforementioned virtual point cloud data. Specifically, perform virtual point cloud data mapping on multiple twin grids to obtain corresponding virtual point cloud data sets; perform virtual semantic assignment on virtual point cloud data sets to obtain corresponding virtual semantic point cloud data and obtain virtual point clouds Data collection of virtual point cloud data on the target digital twin. Among them, the virtual point cloud data set includes the coordinates of each point in the virtual point cloud data on the target digital twin; since the initial point cloud can represent the three-dimensional coordinates of any point in the virtual point cloud data, and the target digital twin also has a corresponding Therefore, after mapping the virtual point cloud data to the target digital twin, the coordinates of each point in the virtual point cloud data mapped to the target digital twin can be obtained, and the coordinates of these points The set is regarded as the above-mentioned virtual point cloud data set. Perform virtual semantic assignment on the virtual point cloud data set to obtain the corresponding virtual semantic point cloud data, perform depth truth correction on the virtual semantic point cloud data, obtain the target point cloud data set, and map the virtual point cloud data to the twin texture image In essence, the spatial position of each point in the virtual point cloud data is mapped to the virtual semantic point cloud data in the target digital twin, and the virtual semantic point cloud data is used to mark the initial digital twin. Correspondingly, since the initial digital twins have been identified from the target digital twins through object recognition related methods, and the initial digital twins are assigned values with virtual semantic point cloud data, therefore, it is determined that the target digital twins are different from the initial digital twins The target point cloud data for body matching, including: obtaining the point cloud data of each twin body grid from the virtual point cloud data; according to the virtual point cloud data set, the point cloud data of each twin The spatial position of the volume grid corresponds to the virtual semantic point cloud data to obtain the target point cloud data; the virtual semantic point cloud data is used to mark the initial digital twin in the target digital twin. The initial digital twin is corrected through the target point cloud data set to obtain the target digital twin.

In a specific embodiment, the process of performing step S301 may specifically include the following steps:

(1) Perform texture image conversion on multiple twin body grids to obtain corresponding multiple twin body texture images;

(2) Based on the preset feature database, perform two-dimensional feature matching on multiple twin body texture images, and obtain target two-dimensional feature sets corresponding to multiple twin body texture images;

(3) Match the mapping relationship based on the target two-dimensional feature set to obtain the corresponding point cloud mapping relationship;

(4) Through the point cloud mapping relationship, virtual point cloud data mapping is performed on multiple twin grids to obtain the corresponding virtual point cloud data set.

Specifically, firstly, texture image conversion is performed on multiple twin body grids to obtain corresponding multiple twin body texture images; based on the preset feature database, two-dimensional feature matching is performed on multiple twin body texture images to obtain multiple The target two-dimensional feature set corresponding to the twin body texture image; then the mapping relationship is matched based on the target two-dimensional feature set to obtain the corresponding point cloud mapping relationship; finally, the virtual point cloud data of multiple twin body grids is obtained through the point cloud mapping relationship Mapping to get the corresponding virtual point cloud data set. Perform mapping relationship matching on the target two-dimensional feature set, and obtain candidate point cloud data corresponding to the target two-dimensional feature set, wherein the candidate point cloud data includes a point cloud image and dense depth point cloud data corresponding to the point cloud image Set; Based on the point cloud mapping relationship of the candidate point cloud data, determine the reference point cloud mapping relationship of the image to be positioned, and determine the same feature points between the point cloud image and the virtual point cloud data; In the dense depth point cloud data set corresponding to the point cloud image, according to the three-dimensional coordinate points corresponding to each of the feature points, determine the target point cloud mapping relationship of the image to be positioned, and use the point cloud mapping relationship to multiple twins The grid performs virtual point cloud data mapping to obtain the corresponding virtual point cloud data set.

In a specific embodiment, the process of performing step S302 may specifically include the following steps:

(1) Based on the preset point cloud semantic database, perform semantic matching on the virtual point cloud data set, and obtain the semantic information corresponding to the virtual point cloud data set;

(2) Based on the semantic information, the virtual semantic value is assigned to the virtual point cloud data set, and the corresponding virtual semantic point cloud data is obtained.

Specifically, firstly, according to the point cloud semantic database of the 3D point cloud, the discrete points containing semantic information are marked in the virtual point cloud data set, and then the discrete points containing semantic information are connected into polygons and classified to obtain the virtual point cloud data. The semantic information corresponding to the set. After obtaining the semantic information corresponding to the virtual point cloud data set, the virtual semantic point cloud data is generated according to the semantic information. The virtual semantic point cloud data is assigned virtual semantic value to the virtual point cloud data set by the semantic information corresponding to the virtual point cloud data set. The semantic information corresponding to the virtual point cloud data set refers to medical equipment and smart operating room in this embodiment. The connection relationship between, establish a KD tree for all virtual point cloud data sets, search the KD tree for the first and last points of the i-th medical device entity, and find the a-th and b-th medical device entities to which the nearest neighbor point belongs, As the forward and backward association of the i-th medical equipment entity, all medical equipment entities are traversed to obtain a complete connection relationship. This embodiment implements virtual semantic assignment based on semantic information, thereby obtaining corresponding virtual semantic point cloud data, and then It improves the accuracy of the operation position selected by the doctor during the operation.

The above describes the construction method of the digital twin-based smart operating room in the embodiment of the present invention. The following describes the construction device of the digital twin-based smart operating room in the embodiment of the present invention. Please refer to FIG. 4. In the embodiment of the present invention, based on An embodiment of the digital twin smart operating room construction device includes:

An acquisition module 401, configured to acquire patient information of a target patient, and perform type matching on the patient information to obtain a corresponding information type;

A construction module 402, configured to construct a finite element model based on the information type to obtain a corresponding initial digital twin;

The scanning module 403 is configured to perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin;

A division module 404, configured to perform grid division on the initial digital twin based on the multi-angle image information, and determine corresponding multiple twin grids;

A correction module 405, configured to correct the initial digital twin based on the plurality of twin grids to obtain a target digital twin;

The setting module 406 is configured to set an operation instruction for the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal.

Through the cooperation of the above components, the patient information of the target patient is collected, and the patient information is type-matched to obtain the corresponding information type; the finite element model is constructed based on the information type to obtain the corresponding initial digital twin; the initial digital twin The twin is scanned from multiple angles to obtain the multi-angle image information corresponding to the digital twin; based on the multi-angle image information, the initial digital twin is meshed to determine the corresponding multiple twin grids; based on multiple twin networks The grid corrects the initial digital twin to obtain the target digital twin; sets the operation instruction for the target digital twin, and transmits the target digital twin set by the operation instruction to the three-dimensional operation terminal. The model constructs the target digital twin, and then sets the operation instruction for the target digital twin, and transmits the target digital twin set by the operation instruction to the 3D operation terminal, which helps doctors find the operation position more accurately, and improves the accuracy of the operation. The processing efficiency of patient information further improves the accuracy of the doctor's operation position selection during operation.

FIG. 5 is a schematic structural diagram of a digital twin-based smart operating room construction device provided by an embodiment of the present invention. The digital twin-based smart operating room construction device 500 may have relatively large differences due to different configurations or performances. One or more processors (central processing units, CPU) 510 (for example, one or more processors) and memory 520 may be included, and one or more storage media 530 for storing application programs 533 or data 532 (for example, one or more above mass storage devices). Wherein, the memory 520 and the storage medium 530 may be temporary storage or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations for the construction device 500 of the digital twin-based smart operating room. Further, the processor 510 may be configured to communicate with the storage medium 530, and execute a series of instruction operations in the storage medium 530 on the construction device 500 of the digital twin-based smart operating room.

The digital twin-based smart operating room construction device 500 may also include one or more power sources 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and/or, one or more operating System 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the construction equipment structure of the digital twin-based smart operating room shown in Figure 5 does not constitute a limitation on the construction equipment of the digital twin-based smart operating room, and may include more or less than the illustration components, or combinations of certain components, or different arrangements of components.

The present invention also provides a digital twin-based smart operating room construction device, the digital twin-based smart operating room construction device includes a memory and a processor, and computer-readable instructions are stored in the memory, and the computer-readable instructions are processed When the processor is executed, the processor is made to execute the steps of the method for constructing a digital twin-based smart operating room in the above-mentioned embodiments.

The present invention also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium may also be a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions are run on the computer, the computer is made to execute the steps of the method for constructing the smart operating room based on digital twins.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random accessS memory, RAM), magnetic disk or optical disk, and various media that can store program codes. .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions recorded in each embodiment are modified, or some of the technical features are replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A construction method based on the wisdom operating room of digital twin, it is characterized in that, the construction method of the wisdom operating room based on digital twin comprises: Collecting patient information of the target patient, and performing type matching on the patient information to obtain the corresponding information type; Constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin; performing grid division on the initial digital twin based on the multi-angle image information, and determining corresponding multiple twin grids; modifying the initial digital twin based on the plurality of twin grids to obtain a target digital twin; An operation instruction is set for the target digital twin, and the target digital twin set by the operation instruction is transmitted to the three-dimensional operation terminal. 2. the construction method based on the intelligent operating room of digital twin according to claim 1, is characterized in that, described based on described information type carries out finite element model construction, obtains corresponding initial digital twin body, comprises: Perform unit matching on the information type through the preset finite element model database to obtain the corresponding information unit set; Perform parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set; Model construction is carried out through the unit parameter information to obtain the corresponding initial digital twin. 3. The construction method of the smart operating room based on digital twin according to claim 1, wherein the multi-angle scanning is carried out to the initial digital twin to obtain a multi-angle image corresponding to the digital twin information, including: Based on the preset spatial coordinate system, the spatial position of the initial digital twin is determined to obtain the corresponding spatial position coordinates; Determine a corresponding scan angle information set based on the spatial position coordinates; Multi-angle scanning is performed on the initial digital twin based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin. 4. The construction method of the intelligent operating room based on digital twins according to claim 3, characterized in that, the initial digital twins are meshed based on the multi-angle image information, and a plurality of corresponding ones are determined. Twin volume meshes, including: Analyzing the key position points of the initial digital twin body through the spatial coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body; performing area expansion according to each of the key position points to obtain a plurality of corresponding expanded three-dimensional areas; performing image matching on the plurality of extended stereo regions and the multi-angle image information to obtain image information to be processed corresponding to each extended stereo region; The image information to be processed corresponding to each expanded three-dimensional area is subjected to sweeping and division processing respectively to obtain a plurality of corresponding twin volume grids. 5. The method for constructing a smart operating room based on digital twins according to claim 1, wherein the initial digital twins are corrected based on the multiple twin grids to obtain the target digital twins ,include: Carrying out virtual point cloud data mapping to the plurality of twin body grids to obtain corresponding virtual point cloud data sets; Carry out virtual semantic assignment to described virtual point cloud data set, obtain corresponding virtual semantic point cloud data; Carrying out depth truth correction to the virtual semantic point cloud data to obtain the target point cloud data set; Correcting the initial digital twin through the target point cloud data set to obtain a target digital twin. 6. the construction method based on the intelligent operating room of digital twin according to claim 5, is characterized in that, described described a plurality of twin body grids are carried out virtual point cloud data mapping, obtains corresponding virtual point cloud data set ,include: performing texture image conversion on the multiple twin body grids to obtain corresponding multiple twin body texture images; performing two-dimensional feature matching on the plurality of twin body texture images based on a preset feature database, to obtain a target two-dimensional feature set corresponding to the plurality of twin body texture images; performing mapping relationship matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relationship; Perform virtual point cloud data mapping on the plurality of twin body grids through the point cloud mapping relationship to obtain a corresponding virtual point cloud data set. 7. the construction method based on the intelligent operating room of digital twin according to claim 5, is characterized in that, described described virtual point cloud data set is carried out virtual semantic assignment, obtains corresponding virtual semantic point cloud data, comprising: Perform semantic matching on the virtual point cloud data set based on the preset point cloud semantic database to obtain semantic information corresponding to the virtual point cloud data set; Perform virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data. 8. A construction device based on a digital twin-based smart operating room, characterized in that, the construction device based on a digital twin-based smart operating room comprises: An acquisition module, configured to acquire patient information of a target patient, and perform type matching on the patient information to obtain a corresponding information type; A building block for building a finite element model based on the information type to obtain a corresponding initial digital twin; A scanning module, configured to perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin; A division module, configured to perform grid division on the initial digital twin based on the multi-angle image information, and determine corresponding multiple twin grids; A correction module, configured to correct the initial digital twin based on the plurality of twin grids to obtain a target digital twin; The setting module is configured to perform operation instruction setting on the target digital twin, and transmit the target digital twin set by the operation instruction to the three-dimensional operation terminal. 9. A construction device for a smart operating room based on a digital twin, characterized in that, the construction device for a smart operating room based on a digital twin includes: a memory and at least one processor, and instructions are stored in the memory; The at least one processor calls the instructions in the memory, so that the construction equipment of the digital twin-based smart operating room performs the digital twin-based smart surgery according to any one of claims 1-7 Chamber construction method. 10. A computer-readable storage medium, wherein instructions are stored on the computer-readable storage medium, wherein when the instructions are executed by a processor, the digital-based The construction method of the twin smart operating room.

Images