CN115862821B - Digital twinning-based intelligent operating room construction method and related device - Google Patents
Digital twinning-based intelligent operating room construction method and related device Download PDFInfo
- Publication number
- CN115862821B CN115862821B CN202310122298.2A CN202310122298A CN115862821B CN 115862821 B CN115862821 B CN 115862821B CN 202310122298 A CN202310122298 A CN 202310122298A CN 115862821 B CN115862821 B CN 115862821B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- digital twin
- cloud data
- information
- twin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention relates to the field of data processing, and discloses a digital twinning-based intelligent operating room construction method and a related device, which are used for improving the accuracy of patient information processing. The method comprises the following steps: collecting patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; grid division is carried out on the initial digital twin body based on the multi-angle image information, and a plurality of corresponding twin body grids are determined; correcting the initial digital twin based on the twin grids to obtain a target digital twin; and setting an operation instruction on the target digital twin body, and transmitting the target digital twin body set by the operation instruction to the three-dimensional operation terminal.
Description
Technical Field
The invention relates to the field of data processing, in particular to a digital twinning-based intelligent operating room construction method and a related device.
Background
In the traditional operation process, a doctor often encounters various conditions, such as the condition that the vision is blocked by a surgical instrument, the characteristics of a plurality of angles of a patient are usually required to be checked in real time, and the error condition caused by unclear surgical vision is likely to be caused, so that the operating position point is difficult to find, and the operation risk is increased.
Therefore, in the operation process of the patient, a virtual simulation physical model is needed to help the doctor to find the operation position more accurately, so as to improve the processing efficiency of the patient information and further improve the accuracy of selecting the operation position when the doctor operates in the operation process.
Disclosure of Invention
The invention provides a digital twinning-based intelligent operating room construction method and a related device, which are used for improving the accuracy of patient data processing.
The first aspect of the invention provides a method for constructing a digital twinning-based intelligent operating room, which comprises the following steps: collecting patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; grid dividing the initial digital twin body based on the multi-angle image information to determine a plurality of corresponding twin body grids; correcting the initial digital twin based on the twin grids to obtain a target digital twin; and setting the operation instruction on the target digital twin body, and transmitting the target digital twin body set by the operation instruction to a three-dimensional operation terminal.
With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the performing finite element model construction based on the information type to obtain a corresponding initial digital twin includes: performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set; performing parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set; and constructing a model through the unit parameter information to obtain a corresponding initial digital twin body.
With reference to the first aspect, in a second implementation manner of the first aspect of the present invention, the performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body includes: based on a preset space coordinate system, determining the space position of the initial digital twin body to obtain a corresponding space position coordinate; determining a corresponding scanning angle information set based on the spatial position coordinates; and performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect of the present invention, the meshing of the initial digital twin based on the multi-angle image information, determining a corresponding plurality of twin grids includes: performing key position point analysis on the initial digital twin body through the space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body; performing region expansion according to each key position point to obtain a plurality of corresponding expansion three-dimensional regions; performing image matching on the plurality of extended three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each extended three-dimensional area; and respectively carrying out sweep division processing on the image information to be processed corresponding to each extended three-dimensional region to obtain a plurality of corresponding twin grids.
With reference to the first aspect, in a fourth implementation manner of the first aspect of the present invention, the correcting the initial digital twin based on the multiple twin grids to obtain a target digital twin includes: performing virtual point cloud data mapping on the twin grids to obtain corresponding virtual point cloud data sets; performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data; performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set; and correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body.
With reference to the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect of the present invention, the performing virtual point cloud data mapping on the multiple twin grids to obtain a corresponding virtual point cloud data set includes: performing texture image conversion on the twin grids to obtain a plurality of corresponding twin texture images; performing two-dimensional feature matching on the twin texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin texture images; performing mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation; and performing virtual point cloud data mapping on the twin grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
With reference to the fourth implementation manner of the first aspect, in a sixth implementation manner of the first aspect of the present invention, performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data includes: based on a preset point cloud semantic database, carrying out semantic matching on the virtual point cloud data set to obtain semantic information corresponding to the virtual point cloud data set; and performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
The second aspect of the present invention provides a digital twinning-based intelligent operating room construction apparatus, comprising:
the acquisition module is used for acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
the construction module is used for constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
the scanning module is used for performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
the dividing module is used for dividing grids of the initial digital twin body based on the multi-angle image information, and determining a plurality of corresponding twin body grids;
the correction module is used for correcting the initial digital twin based on the twin grids to obtain a target digital twin;
the setting module is used for setting the operation instruction on the target digital twin body and transmitting the target digital twin body set by the operation instruction to the three-dimensional operation terminal.
A third aspect of the present invention provides a digital twinning-based intelligent operating room construction apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the digital twinning-based intelligent operating room construction device to perform the digital twinning-based intelligent operating room construction method described above.
A fourth aspect of the present invention provides a computer readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the above-described method of constructing a digital twinned-based intelligent operating room.
In the technical scheme provided by the invention, the patient information of a target patient is acquired, and the type of the patient information is matched to obtain the corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; grid division is carried out on the initial digital twin body based on the multi-angle image information, and a plurality of corresponding twin body grids are determined; correcting the initial digital twin based on the twin grids to obtain a target digital twin; according to the invention, the target digital twin body is constructed through a virtual simulated finite element model, so that the target digital twin body is subjected to operation instruction setting, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, thereby realizing the purpose of helping doctors to find the operation position more accurately, improving the processing efficiency of patient information, and further improving the accuracy of selecting the operation position in the operation process of doctors.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a method for constructing a digital twinning-based intelligent operating room in accordance with an embodiment of the present invention;
FIG. 2 is a flow chart of meshing an initial digital twin volume in an embodiment of the present invention;
FIG. 3 is a flow chart of the modification of an initial digital twin in an embodiment of the present invention;
FIG. 4 is a schematic diagram of an embodiment of a device for constructing a digital twinning-based intelligent operating room in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of an embodiment of a digital twinning-based intelligent operating room construction apparatus in accordance with an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device, equipment and a storage medium for constructing an intelligent operating room based on digital twinning, which are used for improving the accuracy of patient data processing. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, a specific flow of an embodiment of the present invention is described below with reference to fig. 1, and an embodiment of a method for constructing a digital twin-based intelligent operating room in the embodiment of the present invention includes:
s101, collecting patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
it will be appreciated that the implementation subject of the present invention may be a digital twinning-based intelligent operating room building device, or may be a terminal or a server, which is not limited herein. The embodiment of the invention is described by taking a server as an execution main body as an example.
Specifically, the server extracts entity words in patient information of a target patient, and performs boundary correction on the extracted entity words, wherein in the embodiment, a preset text information classifier is adopted to carry out syntactic analysis on the patient information, the entity words are marked out through an entity marking model in the preset text information classifier, and the marked entity words are extracted. After receiving the patient information of the target patient, semantic analysis can be performed on the patient information of the target patient, so that entity words corresponding to the patient information of the target patient are obtained. Specifically, the type matching of the patient information is firstly entity word normalization processing and paraphrasing feature extraction, wherein in the entity word normalization processing, each extracted entity word can be subjected to entity word normalization processing according to a normalized word stock, for example, standard normalization is carried out on the number words, the graduated words and the units, and the later judgment and report generation work are facilitated. In the extraction of the characteristics of the hyponyms, the extracted characteristics of each entity word can be extracted according to the hyponym word bank, so that the consistency of information in the process of collecting patient information is ensured, and the phenomenon of repeating the same patient information can be avoided. After the standardization processing of the entity words is completed, clustering analysis is carried out on the plurality of entity words to obtain entity classes corresponding to the target patient, and then information type matching is carried out on the entity classes to obtain information types corresponding to the patient information.
S102, constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
specifically, setting unit parameters and grid type variables based on information types, then combining an intelligent optimization algorithm to perform combined optimization analysis to obtain a corresponding information unit set, and performing parameter analysis on the information unit set to construct an initial digital twin body. In this embodiment, different unit types are selected according to different information types, and the simulation ranges applicable to the unit types are different; for the selection of the grid type and the grid size, the entity unit selects tetrahedrons or hexahedrons with different node numbers, whether the grid size is reasonable or not is judged by adjusting the unit parameter information after the grid fineness twice, and if the change value of the unit parameter information accords with a preset threshold value, model construction is carried out according to the unit parameter information, so that an initial digital twin is obtained. The finite element model modification and analysis result data transfer are to generate a more accurate and reliable initial digital twin body and perform load simulation analysis through the optimization process of the unit parameter information, so that error comparison and analysis are performed on the unit parameter information.
S103, performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
it should be noted that, determining coordinate values of a plurality of position points on an initial digital twin body on the digital twin body, wherein the initial digital twin body is a digital twin model of a medical equipment entity in the intelligent operating room, and the medical equipment entity is any equipment entity in the intelligent operating room; based on a coordinate transformation matrix between the real space and the digital twin body, respectively carrying out coordinate transformation on coordinate values of a plurality of position points on the initial digital twin body in the digital twin body to obtain standard coordinate values of the plurality of position points on the initial digital twin body in the real space; and determining the position points corresponding to the standard coordinate values of the plurality of position points on the initial digital twin body in the real space as a plurality of positioning standard points of the plurality of position points on the initial digital twin body in the real space, respectively installing a plurality of high-precision positioning devices at a plurality of standard positioning points, and acquiring image information corresponding to each positioning standard point according to the plurality of positioning standard points to obtain multi-angle image information.
S104, carrying out grid division on the initial digital twin body based on the multi-angle image information, and determining a plurality of corresponding twin body grids;
Specifically, the initial digital twin body can be divided into a plurality of grid areas based on multi-angle image information, the grid areas are a plurality of twin body grids, digital twin devices are arranged in the initial digital twin body, the twin body grids of the digital twin devices in the initial digital twin body are corresponding twin body grids in the corresponding initial digital twin body, and each twin body grid corresponds to a central control center and is mainly used for taking corresponding processing measures when intelligent operating room devices corresponding to the twin body grids are in abnormal working states. Specifically, when the intelligent operating room equipment is identified to be in an abnormal working state, the twin grid corresponding to the intelligent operating room equipment in the initial digital twin body can be searched and determined according to the equipment identification number corresponding to the intelligent operating room equipment, and the twin grid corresponding to the intelligent operating room equipment in the initial digital twin body can also be determined through an image processing technology.
S105, correcting the initial digital twin based on a plurality of twin grids to obtain a target digital twin;
specifically, after mapping the virtual point cloud data onto the twin texture image according to the preset feature database, mapping the virtual point cloud data of each twin grid onto the area occupied by the initial digital twin in the twin texture image is equivalent. Thus, the target point cloud data refers to point cloud data of an initial digital twin corresponding to each twin grid in an area covered on the twin texture image after the initial point cloud is projected on the twin texture image. Determining each twin grid in the virtual point cloud data according to a preset mapping relation and an initial digital twin; and acquiring a target point cloud covered by the initial digital twin body, and mapping virtual point cloud data into a twin body texture image according to the mapping relation so as to map each twin body grid to the occupied area of the initial digital twin body in the twin body texture image, thereby obtaining the target digital twin body.
S106, setting an operation instruction on the target digital twin body, and transmitting the target digital twin body set by the operation instruction to the three-dimensional operation terminal.
Specifically, according to the operation instruction mapped on the target digital twin body, determining the corresponding position of the operation instruction on the target digital twin body to obtain an operation position; according to the corresponding position of the operation position on the target digital twin body, determining the real operation corresponding to the three-dimensional operation terminal of the operation position on the target digital twin body, wherein the preset proportion exists between the equipment and the parts on the target digital twin body and the equipment or the parts in the real operation space, so that a doctor can find the operation position more accurately through a virtual simulated physical model.
In the embodiment of the invention, the patient information of a target patient is acquired, and the type of the patient information is matched to obtain the corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; grid division is carried out on the initial digital twin body based on the multi-angle image information, and a plurality of corresponding twin body grids are determined; correcting the initial digital twin based on the twin grids to obtain a target digital twin; according to the invention, the target digital twin body is constructed through a virtual simulated finite element model, so that the target digital twin body is subjected to operation instruction setting, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, thereby realizing the purpose of helping doctors to find the operation position more accurately, improving the processing efficiency of patient information, and further improving the accuracy of selecting the operation position in the operation process of doctors.
In a specific embodiment, the process of executing step S102 may specifically include the following steps:
(1) Performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set;
(2) Carrying out parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set;
(3) And constructing a model through the unit parameter information to obtain a corresponding initial digital twin body.
Specifically, unit matching is performed on information types through a preset finite element model database to obtain a corresponding information unit set, parameter analysis is performed on the information unit set to obtain unit parameter information corresponding to the information unit set, model construction is performed through the unit parameter information to obtain a corresponding initial digital twin body, specifically, unit parameters and grid type variables are set based on the information types, then combination optimization analysis is performed by combining an intelligent optimization algorithm to obtain the corresponding information unit set, and parameter analysis is performed on the information unit set to further construct the initial digital twin body.
In a specific embodiment, the process of executing step S103 may specifically include the following steps:
(1) Based on a preset space coordinate system, determining the space position of the initial digital twin body to obtain a corresponding space position coordinate;
(2) Determining a corresponding scanning angle information set based on the spatial position coordinates;
(3) And performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
Specifically, shooting images of medical equipment entities at preset positions and preset directions of a space coordinate system, wherein the medical equipment entities are any equipment entity in an intelligent operating room; determining a spatial position of the medical device entity based on the image of the medical device entity, and constructing spatial position coordinates based on the spatial position of the medical device entity; in the space position coordinates, the position and the angle of an initial digital twin body are adjusted for a plurality of times by taking a preset position and a preset direction as view origins, and an image of the initial digital twin body is intercepted once for each adjustment to obtain a plurality of target digital twin body matching templates, wherein the initial digital twin body is a target digital twin body of a medical equipment entity; based on a preset image template matching algorithm, respectively determining matching degrees of the twin texture image matching templates and a plurality of target digital twin matching templates to obtain a plurality of matching degrees; and determining the position and the angle of the initial digital twin body corresponding to the maximum matching degree in the plurality of matching degrees as the position and the angle of the initial digital twin body in the digital twin body, and obtaining a scanning angle information set. And transforming the position of an intersection point of the medical equipment entity in the photographed multi-angle image information and the ground of the intelligent operating room to a digital twin body to serve as a grid origin, drawing grid lines on the ground of the digital twin body along the x axis and the y axis of the digital twin body by taking the grid origin as the center, setting the unit length of the distance between adjacent grid lines and the number of the grid lines, and carrying out multiple adjustment on the position and the angle of the initial digital twin body to obtain the multi-angle image information corresponding to the digital twin body.
In a specific embodiment, as shown in fig. 2, the process of executing step S104 may specifically include the following steps:
s201, carrying out key position point analysis on an initial digital twin body through a space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body;
s202, performing region expansion according to each key position point to obtain a plurality of corresponding expansion three-dimensional regions;
s203, performing image matching on the plurality of extended three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each extended three-dimensional area;
s204, respectively carrying out sweep division processing on the image information to be processed corresponding to each expansion three-dimensional region to obtain a plurality of corresponding twin grids.
Specifically, the initial digital twin is subjected to key position point analysis through a space coordinate system, a plurality of key position points corresponding to the initial digital twin are determined, specifically, the initial digital twin is rapidly identified through a mean shift segmentation algorithm, and the plurality of key position points are effectively located. And then carrying out region expansion according to each key position point to obtain a plurality of corresponding expansion solid regions, wherein the intelligent operating room corresponding to the initial digital twin is firstly divided into a plurality of regions according to the set size and each region is numbered, the basic attribute of all regions in the intelligent operating room can be counted, then the plurality of expansion solid regions are generated based on the plurality of regions and according to the grid number upper limit value determined by the number of regions, namely, the upper limit (grid number upper limit value) of the grid number is set according to the total number of the divided regions, a certain number of initial twin grid division schemes are generated based on the plurality of regions and according to the grid number upper limit value, thereby further estimating and obtaining the estimation result corresponding to each initial twin grid division scheme according to the preset division target based on the basic attribute of each region counted, further determining the target twin grid division scheme according to the estimation result, and further iteratively updating the twin grid division scheme, finally, finding the optimal division scheme as the target twin grid division scheme, and obtaining the target twin grid division scheme according to the calculated according to the estimation result. Further, performing image matching on the plurality of extended three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each extended three-dimensional area; the image information to be processed corresponding to each extended three-dimensional area is respectively subjected to sweep division processing to obtain a plurality of corresponding twin grids, and the twin grids which are convenient for each user to manage the related intelligent operating room equipment and timely process abnormal states can be reasonably divided through the scheme of the embodiment, so that the corresponding twin grids are finally obtained.
In a specific embodiment, as shown in fig. 3, the process of executing step S105 may specifically include the following steps:
s301, performing virtual point cloud data mapping on a plurality of twin grids to obtain a corresponding virtual point cloud data set;
s302, performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data;
s303, performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set;
s304, correcting the initial digital twin body through the target point cloud data set to obtain the target digital twin body.
Specifically, the target point cloud data is a set of point cloud data in an area on the target digital twin that matches the initial digital twin location. That is, the target point cloud data refers to a set of points that cover the initial digital twin. The points in the target point cloud data are all from the virtual point cloud data. Specifically, virtual point cloud data mapping is carried out on the twin grids to obtain corresponding virtual point cloud data sets; performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data, and obtaining a virtual point cloud data set of the virtual point cloud data on the target digital twin body. The virtual point cloud data set comprises coordinates of each point in the virtual point cloud data on a target digital twin body; since the initial point cloud can represent three-dimensional coordinates of any point in the virtual point cloud data, and the target digital twin body also has a corresponding image coordinate system, after the virtual point cloud data is mapped onto the target digital twin body, coordinates of each point in the virtual point cloud data mapped onto the target digital twin body can be obtained, and a set of coordinates of the points can be regarded as the virtual point cloud data set. Performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data, performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set, wherein mapping the virtual point cloud data to the twin texture image is to map the spatial positions of points in the virtual point cloud data to the virtual semantic point cloud data in the target digital twin, and the virtual semantic point cloud data is used for labeling the initial digital twin. Accordingly, since the initial digital twin is already identified from the target digital twin by the object identification correlation mode, and the initial digital twin is respectively assigned by using the virtual semantic point cloud data, determining the target point cloud data matched with the initial digital twin in the target digital twin includes: acquiring point cloud data of each twin grid from the virtual point cloud data; according to the virtual point cloud data set, the point cloud data of each twin grid, namely the spatial position of each twin grid, is corresponding to the virtual semantic point cloud data, and target point cloud data are obtained; the virtual semantic point cloud data is used for annotating an initial digital twin in the target digital twin. And correcting the initial digital twin body through the target point cloud data set to obtain the target digital twin body.
In a specific embodiment, the process of executing step S301 may specifically include the following steps:
(1) Converting texture images of the twin grids to obtain corresponding twin texture images;
(2) Performing two-dimensional feature matching on the twin texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin texture images;
(3) Performing mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation;
(4) And performing virtual point cloud data mapping on the twin grids through the point cloud mapping relationship to obtain a corresponding virtual point cloud data set.
Specifically, firstly, converting texture images of a plurality of twin grids to obtain a plurality of corresponding twin texture images; performing two-dimensional feature matching on the twin texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin texture images; performing mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation; and finally, performing virtual point cloud data mapping on the twin grids through a point cloud mapping relation to obtain a corresponding virtual point cloud data set. Performing mapping relation matching on a target two-dimensional feature set to obtain candidate point cloud data of the target two-dimensional feature set, wherein the candidate point cloud data comprises a point cloud image and a dense depth point cloud data set corresponding to the point cloud image; determining a reference point cloud mapping relation of the image to be positioned based on the point cloud mapping relation of the candidate point cloud data, and determining the same characteristic points between the point cloud image and the virtual point cloud data; and in the dense depth point cloud data set corresponding to the point cloud image, determining a target point cloud mapping relation of the image to be positioned according to the three-dimensional coordinate points corresponding to the feature points, and performing virtual point cloud data mapping on a plurality of twin grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
In a specific embodiment, the process of executing step S302 may specifically include the following steps:
(1) Based on a preset point cloud semantic database, carrying out semantic matching on the virtual point cloud data set to obtain semantic information corresponding to the virtual point cloud data set;
(2) And performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
Specifically, firstly, marking discrete points containing semantic information in a virtual point cloud data set according to a point cloud semantic database of a three-dimensional point cloud, and then connecting the discrete points containing semantic information into polygons and classifying to obtain semantic information corresponding to the virtual point cloud data set. And after semantic information corresponding to the virtual point cloud data set is obtained, generating virtual semantic point cloud data according to the semantic information. Virtual semantic point cloud data performs virtual semantic assignment on a virtual point cloud data set by semantic information corresponding to the virtual point cloud data set, wherein the semantic information corresponding to the virtual point cloud data set refers to a connection relation between medical equipment and an intelligent operating room in the embodiment, KD trees are built for all the virtual point cloud data sets, KD trees are searched for the head and tail points of an ith medical equipment entity respectively, an a-th medical equipment entity and a b-th medical equipment entity of the nearest point are found and used as forward and backward association of the ith medical equipment entity, and all the medical equipment entities are traversed to obtain a complete connection relation.
The method for constructing a smart operating room based on digital twin in the embodiment of the present invention is described above, and the apparatus for constructing a smart operating room based on digital twin in the embodiment of the present invention is described below, referring to fig. 4, one embodiment of the apparatus for constructing a smart operating room based on digital twin in the embodiment of the present invention includes:
the acquisition module 401 is configured to acquire patient information of a target patient, and perform type matching on the patient information to obtain a corresponding information type;
a construction module 402, configured to perform finite element model construction based on the information type, so as to obtain a corresponding initial digital twin;
the scanning module 403 is configured to perform multi-angle scanning on the initial digital twin to obtain multi-angle image information corresponding to the digital twin;
a dividing module 404, configured to divide the initial digital twin body into grids based on the multi-angle image information, and determine a plurality of corresponding twin body grids;
a correction module 405, configured to correct the initial digital twin based on the multiple twin grids to obtain a target digital twin;
the setting module 406 is configured to set the operation instruction for the target digital twin body, and transmit the target digital twin body set by the operation instruction to the three-dimensional operation terminal.
Through the cooperation of the components, the patient information of the target patient is acquired, and the type of the patient information is matched to obtain the corresponding information type; constructing a finite element model based on the information type to obtain a corresponding initial digital twin; performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body; grid division is carried out on the initial digital twin body based on the multi-angle image information, and a plurality of corresponding twin body grids are determined; correcting the initial digital twin based on the twin grids to obtain a target digital twin; according to the invention, the target digital twin body is constructed through a virtual simulated finite element model, so that the target digital twin body is subjected to operation instruction setting, and the target digital twin body set through the operation instruction is transmitted to the three-dimensional operation terminal, thereby realizing the purpose of helping doctors to find the operation position more accurately, improving the processing efficiency of patient information, and further improving the accuracy of selecting the operation position in the operation process of doctors.
Fig. 5 is a schematic structural diagram of a digital twinning-based intelligent operating room construction device 500 according to an embodiment of the present invention, where the digital twinning-based intelligent operating room construction device 500 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (e.g., one or more processors) and a memory 520, one or more storage media 530 (e.g., one or more mass storage devices) storing application programs 533 or data 532. Wherein memory 520 and storage medium 530 may be transitory or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the digital twinned intelligent operating room-based building apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the digital twinned intelligent operating room based build device 500.
The digital twinning-based intelligent operating room building apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input/output interfaces 560, and/or one or more operating systems 531, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the digital twinning-based intelligent operating room construction apparatus structure shown in fig. 5 is not limiting and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The invention also provides a construction device of the intelligent operating room based on the digital twin, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the construction method of the intelligent operating room based on the digital twin in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and may also be a volatile computer readable storage medium, where instructions are stored in the computer readable storage medium, when the instructions are executed on a computer, cause the computer to perform the steps of the method for constructing a digital twin-based intelligent operating room.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random acceS memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The method for constructing the intelligent operating room based on the digital twin is characterized by comprising the following steps of:
collecting patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
constructing a finite element model based on the information type to obtain a corresponding initial digital twin;
performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
grid dividing the initial digital twin body based on the multi-angle image information to determine a plurality of corresponding twin body grids;
Correcting the initial digital twin based on the twin grids to obtain a target digital twin; performing virtual point cloud data mapping on the twin grids to obtain corresponding virtual point cloud data sets; performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data; performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set; correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body;
and setting the operation instruction on the target digital twin body, and transmitting the target digital twin body set by the operation instruction to a three-dimensional operation terminal.
2. The method for constructing a digital twinning-based intelligent operating room according to claim 1, wherein the finite element model construction based on the information type is performed to obtain a corresponding initial digital twinning body, and the method comprises the following steps:
performing unit matching on the information types through a preset finite element model database to obtain a corresponding information unit set;
performing parameter analysis on the information unit set to obtain unit parameter information corresponding to the information unit set;
And constructing a model through the unit parameter information to obtain a corresponding initial digital twin body.
3. The method for constructing a digital twinning-based intelligent operating room according to claim 1, wherein the performing multi-angle scanning on the initial digital twinning body to obtain multi-angle image information corresponding to the digital twinning body includes:
based on a preset space coordinate system, determining the space position of the initial digital twin body to obtain a corresponding space position coordinate;
determining a corresponding scanning angle information set based on the spatial position coordinates;
and performing multi-angle scanning on the initial digital twin body based on the scanning angle information set to obtain multi-angle image information corresponding to the digital twin body.
4. The method for constructing a digital twinning-based intelligent operating room according to claim 3, wherein the meshing of the initial digital twinning based on the multi-angle image information, determining a corresponding plurality of twin grids, comprises:
performing key position point analysis on the initial digital twin body through the space coordinate system, and determining a plurality of key position points corresponding to the initial digital twin body;
Performing region expansion according to each key position point to obtain a plurality of corresponding expansion three-dimensional regions;
performing image matching on the plurality of extended three-dimensional areas and the multi-angle image information to obtain image information to be processed corresponding to each extended three-dimensional area;
and respectively carrying out sweep division processing on the image information to be processed corresponding to each extended three-dimensional region to obtain a plurality of corresponding twin grids.
5. The method for constructing a digital twinning-based intelligent operating room according to claim 1, wherein the performing virtual point cloud data mapping on the plurality of twinning grids to obtain a corresponding virtual point cloud data set includes:
performing texture image conversion on the twin grids to obtain a plurality of corresponding twin texture images;
performing two-dimensional feature matching on the twin texture images based on a preset feature database to obtain a target two-dimensional feature set corresponding to the twin texture images;
performing mapping relation matching based on the target two-dimensional feature set to obtain a corresponding point cloud mapping relation;
and performing virtual point cloud data mapping on the twin grids through the point cloud mapping relation to obtain a corresponding virtual point cloud data set.
6. The method for constructing a digital twinning-based intelligent operating room according to claim 1, wherein the performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data includes:
based on a preset point cloud semantic database, carrying out semantic matching on the virtual point cloud data set to obtain semantic information corresponding to the virtual point cloud data set;
and performing virtual semantic assignment on the virtual point cloud data set based on the semantic information to obtain corresponding virtual semantic point cloud data.
7. The utility model provides a construction device of wisdom operating room based on digital twin which characterized in that, the construction device of wisdom operating room based on digital twin includes:
the acquisition module is used for acquiring patient information of a target patient, and performing type matching on the patient information to obtain a corresponding information type;
the construction module is used for constructing a finite element model based on the information type to obtain a corresponding initial digital twin body;
the scanning module is used for performing multi-angle scanning on the initial digital twin body to obtain multi-angle image information corresponding to the digital twin body;
The dividing module is used for dividing grids of the initial digital twin body based on the multi-angle image information, and determining a plurality of corresponding twin body grids;
the correction module is used for correcting the initial digital twin based on the twin grids to obtain a target digital twin; performing virtual point cloud data mapping on the twin grids to obtain corresponding virtual point cloud data sets; performing virtual semantic assignment on the virtual point cloud data set to obtain corresponding virtual semantic point cloud data; performing depth truth value correction on the virtual semantic point cloud data to obtain a target point cloud data set; correcting the initial digital twin body through the target point cloud data set to obtain a target digital twin body;
the setting module is used for setting the operation instruction on the target digital twin body and transmitting the target digital twin body set by the operation instruction to the three-dimensional operation terminal.
8. A digital twinning-based intelligent operating room construction apparatus, characterized in that the digital twinning-based intelligent operating room construction apparatus comprises: a memory and at least one processor, the memory having instructions stored therein;
The at least one processor invoking the instructions in the memory to cause the digital twinned intelligent operating room construction apparatus to perform the digital twinned intelligent operating room construction method of any one of claims 1-6.
9. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the method of constructing a digital twinning-based intelligent operating room of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310122298.2A CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310122298.2A CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115862821A CN115862821A (en) | 2023-03-28 |
| CN115862821B true CN115862821B (en) | 2023-05-05 |
Family
ID=85658160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310122298.2A Active CN115862821B (en) | 2023-02-16 | 2023-02-16 | Digital twinning-based intelligent operating room construction method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115862821B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116228991B (en) * | 2023-05-08 | 2023-07-14 | 河北光之翼信息技术股份有限公司 | Coordinate conversion method and device, electronic equipment and storage medium |
| CN116912430B (en) * | 2023-09-14 | 2023-12-15 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853746A (en) * | 2019-10-14 | 2020-02-28 | 边缘智能研究院南京有限公司 | Intelligent hospital system based on digital twin and construction method thereof |
| CN112002401A (en) * | 2020-08-25 | 2020-11-27 | 上海至数企业发展有限公司 | Space environment monitoring method and system based on digital twin and storage medium |
| CN114386150A (en) * | 2022-01-13 | 2022-04-22 | 上海理工大学 | Digital and intelligent building applying 6G digital twins |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210005324A1 (en) * | 2018-08-08 | 2021-01-07 | Hc1.Com Inc. | Methods and systems for a health monitoring command center and workforce advisor |
| US20190005200A1 (en) * | 2017-06-28 | 2019-01-03 | General Electric Company | Methods and systems for generating a patient digital twin |
| US20190087544A1 (en) * | 2017-09-21 | 2019-03-21 | General Electric Company | Surgery Digital Twin |
| CN112380704B (en) * | 2020-11-16 | 2022-05-20 | 北京航空航天大学 | Digital twin model correction method and system based on machine vision |
| DE102020214654A1 (en) * | 2020-11-20 | 2022-05-25 | Siemens Healthcare Gmbh | Long-distance communication with a medical device using a digital twin |
| CN113379860B (en) * | 2021-04-25 | 2023-03-14 | 上海索骥信息科技有限公司 | Medical image data generation method and system with marker based on digital object |
| CN114224528B (en) * | 2021-10-26 | 2023-07-18 | 上海交通大学医学院附属第九人民医院 | Oral cavity digital twin model system based on virtual reality interaction and establishment method |
| CN115054367A (en) * | 2022-06-20 | 2022-09-16 | 上海市胸科医院 | Focus positioning method and device based on mixed reality and electronic equipment |
-
2023
- 2023-02-16 CN CN202310122298.2A patent/CN115862821B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853746A (en) * | 2019-10-14 | 2020-02-28 | 边缘智能研究院南京有限公司 | Intelligent hospital system based on digital twin and construction method thereof |
| CN112002401A (en) * | 2020-08-25 | 2020-11-27 | 上海至数企业发展有限公司 | Space environment monitoring method and system based on digital twin and storage medium |
| CN114386150A (en) * | 2022-01-13 | 2022-04-22 | 上海理工大学 | Digital and intelligent building applying 6G digital twins |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115862821A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115862821B (en) | Digital twinning-based intelligent operating room construction method and related device | |
| CN102341813B (en) | System and method to match images | |
| CN111414809B (en) | Three-dimensional pattern recognition method, device, equipment and storage medium | |
| CN104268849A (en) | Image Matching Device And Device For Positioning Patient | |
| US20190147650A1 (en) | Method for establishing a deformable 3d model of an element, and associated system | |
| CN110969649A (en) | Matching evaluation method, medium, terminal and device of laser point cloud and map | |
| WO2007072391A2 (en) | Automatic 3-d object detection | |
| EP2216726A2 (en) | System and method for prototyping by learning from examples | |
| CN110480075A (en) | Curve surface of workpiece outline compensation system and method and medium based on point cloud data | |
| CN116596755B (en) | Method, device, equipment and storage medium for splicing point cloud data | |
| CN112541950A (en) | Method and device for calibrating external parameter of depth camera | |
| CN112241952A (en) | Method and device for recognizing brain central line, computer equipment and storage medium | |
| CN111862176B (en) | Three-dimensional oral cavity point cloud orthodontic front and back accurate registration method based on palatine fold | |
| US7388584B2 (en) | Method and program for determining insides and outsides of boundaries | |
| CN114372970B (en) | Surgical reference information generation method and device | |
| CN114374931A (en) | Fingerprint positioning method based on measurement learning of neighbor component analysis | |
| CN112774181A (en) | Radar data processing method, processing system and computer storage medium | |
| CN113324473B (en) | House measuring method and measuring equipment | |
| CN111694911B (en) | Method, equipment and system for generating random distribution points in specified range | |
| CN114295138B (en) | Robot, map extension method, apparatus, and readable storage medium | |
| CN115222799B (en) | Method and device for acquiring image gravity direction, electronic equipment and storage medium | |
| JP7164060B1 (en) | Information processing device, information processing method and information processing program | |
| CN114485664B (en) | Boundary simplifying method, path planning method, device, equipment and system | |
| CN117952322B (en) | Engineering management system based on BIM technology | |
| JP2011108000A (en) | Highly accurate data classification apparatus based on tree diagram creation utilizing spherical self-organization map and program for creating the highly accurate data classification apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |