CN113384361B - Visual positioning method, system, device and storage medium - Google Patents
Visual positioning method, system, device and storage medium Download PDFInfo
- Publication number
- CN113384361B CN113384361B CN202110556239.7A CN202110556239A CN113384361B CN 113384361 B CN113384361 B CN 113384361B CN 202110556239 A CN202110556239 A CN 202110556239A CN 113384361 B CN113384361 B CN 113384361B
- Authority
- CN
- China
- Prior art keywords
- lead
- dimensional code
- dimensional
- coordinate
- registration parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/90—Identification means for patients or instruments, e.g. tags
- A61B90/94—Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text
- A61B90/96—Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
- A61B2090/3966—Radiopaque markers visible in an X-ray image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
- A61B2090/3983—Reference marker arrangements for use with image guided surgery
Abstract
The invention discloses a visual positioning method, a system, a device and a storage medium, wherein the method comprises the steps of spraying or pasting a plurality of lead two-dimensional codes on the surface of a patient; scanning a patient, establishing a three-dimensional model before an operation, and acquiring a first coordinate set; acquiring a second coordinate set; identifying and coding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code; calculating to obtain three groups of registration parameters according to the first coordinate group and the second coordinate group; calculating a reprojection error of the center of each lead two-dimensional code according to the three sets of registration parameters; optimizing the three groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters; and determining the real lesion position of the patient in the operation according to the optimized registration parameters. The device required by the invention is simple, high in precision and short in time; the focus region in the intraoperative camera vision system can be registered with the focus region in the preoperative three-dimensional imaging system; the invention can be widely applied to the technical field of surgical navigation.
Description
Technical Field
The invention relates to the technical field of surgical navigation, in particular to a visual positioning method, a system, a device and a storage medium.
Background
The prior surgical operation often needs preoperative three-dimensional image data to assist in positioning a focus area. During the surgical procedure, three-dimensional image data (from CT computed tomography, MRI, etc.) of the preoperative patient is integrated with the focal zone of the patient during the actual surgical procedure using computer technology.
At present, the technology for unifying the three-dimensional image data and the real lesion area coordinates of the patient in the operation includes: a traditional three-point positioning mode; registration is realized through anatomical landmark points, markers pre-embedded in the body and the like; the use of larger registration devices and the installation of fixation devices at the operating bedside to assist registration; registration is performed using infrared light, laser, structured light, and the like.
The drawbacks of the above techniques include: (1) The traditional three-point positioning mode has larger registration error, and the main reason is that when a patient scans CT and starts to register, the mark point can change along with the change of the body state of the patient, and the error can be even in the centimeter level and cannot be overcome; (2) The method for registering the anatomical landmark points and the embedded markers in the body is invasive to the patient, extra risks are increased, the preparation time for preoperative registration is long, the operation time is prolonged, and more recovery time is needed after the operation; (3) The method for assisting registration by a large registration device and a fixing device arranged beside an operation bed has the disadvantages of very complicated operation, need of manually placing markers and fixing devices, long preoperative preparation time and low efficiency; and, over time, registration accuracy can quickly degrade due to physical wear; (4) The image registration technology is carried out by adopting structured light, so that the number of equipment is too much, equipment required in other operations also needs to be placed in a small space above an operating table, more space is difficult to move for the registration technology, and the operation is inconvenient; in addition, a camera coordinate system on the structured light navigator, a static reference coordinate system near a focus region, a coordinate system of the end effector, and a navigation image coordinate system of the surgical navigation system need to be converted and overlapped for many times, so that the accumulated error is relatively increased.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a visual positioning method, a system, a device and a storage medium.
The technical scheme adopted by the invention is as follows:
in one aspect, embodiments of the present invention include a visual positioning method, including:
spraying and printing or pasting a plurality of lead two-dimensional codes on the surface of a patient;
scanning a patient, establishing a three-dimensional model before operation, and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in the three-dimensional model;
acquiring a second coordinate set, wherein the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
identifying and coding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code;
calculating to obtain a plurality of groups of registration parameters according to the first coordinate group and the second coordinate group;
calculating a reprojection error of the center of each lead two-dimensional code according to the multiple groups of registration parameters;
optimizing the multiple groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters;
and determining the real lesion position of the patient in the operation according to the optimized registration parameters.
Further, the lead two-dimensional code is formed by at least one of the following modes:
spraying and painting the two-dimensional code by using lead powder material;
or attaching the two-dimensional code on the lead layer.
Further, the lead two-dimensional code is arranged in the following pattern:
is composed of 6 × 6 black and white quadrangles, each of which has a size of 0.5cm × 0.5cm and ranges from 49 to 100 pixels.
Further, the step of identifying and encoding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code includes:
detecting each edge in the two-dimensional code image according to the gradient to obtain an edge image;
acquiring and screening out a required quadrilateral pattern from the edge image;
and determining the unique mark ID of each lead two-dimensional code according to the gray value and the threshold value of each quadrilateral pattern.
Further, the step of detecting each edge in the two-dimensional code image according to the gradient to obtain an edge image includes:
calculating the gradient direction and amplitude of each pixel point in the two-dimensional code image;
classifying pixels with the same gradient direction and the same amplitude into one class;
and acquiring an edge line segment by using a Felzenszwalb cluster algorithm to obtain an edge image.
Further, the step of obtaining and screening out a desired quadrilateral pattern in the edge image includes:
acquiring the line segment by adopting an iterative algorithm with the depth of 4 and taking the end point of the non-closed loop line segment as a starting point;
after 4 iterations, if a closed loop is formed in the threshold range, determining the pattern to be a quadrilateral pattern;
and screening the quadrilateral patterns to obtain the required quadrilateral patterns.
Further, the step of determining the unique mark ID of each lead two-dimensional code according to the gray value and the threshold value of each quadrilateral pattern includes:
acquiring the gray level mean value of a black quadrangle and the gray level mean value of a white quadrangle in the quadrangle pattern;
calculating to obtain a threshold value according to the gray level average value of the black quadrangle and the gray level average value of the white quadrangle;
traversing pixels in each quadrilateral;
and if the gray value of the pixel is larger than the threshold value, carrying out AND operation on the coding variable TagCode and 1 and shifting the coding variable TagCode and the coding variable TagCode by one bit to the left so as to obtain the unique mark ID of each lead two-dimensional code.
On the other hand, the embodiment of the invention also comprises a visual positioning system, which comprises a preoperative three-dimensional imaging system, an intraoperative camera visual system, a two-dimensional code detection and coding system and a computer processing system; the intraoperative camera vision system and the two-dimensional code detection and coding system are both connected with the computer processing system;
the preoperative three-dimensional imaging system is used for scanning a patient, establishing a preoperative three-dimensional model and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of a plurality of lead two-dimensional codes in the three-dimensional model;
the intraoperative camera vision system is used for acquiring a second coordinate set, and the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
the two-dimension code detection and coding system is used for identifying and coding each lead two-dimension code to obtain a unique mark ID of each lead two-dimension code;
the computer processing system is used for calculating a plurality of groups of registration parameters according to the first coordinate group and the second coordinate group; calculating a reprojection error of the center of each lead two-dimensional code according to the multiple groups of registration parameters; optimizing the multiple groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters; and determining the real lesion position of the patient in the operation according to the optimized registration parameters.
In another aspect, an embodiment of the present invention further includes a visual positioning apparatus, including:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement the visual positioning method.
In another aspect, the embodiments of the present invention further include a computer readable storage medium, on which a program executable by a processor is stored, the program executable by the processor being used for implementing the visual positioning method when being executed by the processor.
The invention has the beneficial effects that:
(1) According to the invention, a plurality of lead two-dimensional codes are sprayed and printed or adhered on the body surface of a patient, the lead two-dimensional codes can take the center of the lead two-dimensional codes as an original point, three vertexes of a tetragon of the lead two-dimensional codes are taken as references to form a coordinate system, and the relative position of each vertex of the two-dimensional codes is fixed and cannot be changed due to the deformation of the body surface skin; meanwhile, final registration can be realized only by spraying or pasting a plurality of lead two-dimensional codes on the body surface of the patient, and the method is simple and convenient in operation steps and low in cost;
(2) According to the method, the unique mark ID of each lead two-dimensional code is obtained by identifying and encoding the lead two-dimensional code, namely each lead two-dimensional code can be uniquely determined in the registration process, and can be in one-to-one correspondence during preoperative registration, so that the lead two-dimensional codes are not easy to be confused; even if the lead two-dimensional code falls off due to loose adhesion for some reasons before registration, the lead two-dimensional code which falls off is easily identified, the method is more convenient than the traditional mark point positioning registration, error calibration caused by rotation deviation is avoided, the registration time before operation is saved, and the method has high use value;
(3) According to the invention, multiple groups of registration parameters are obtained through calculation according to the first coordinate group and the second coordinate group, and the multiple groups of registration parameters are optimized, so that the registration parameters are more accurate, and the accuracy of final registration positioning is further improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flowchart illustrating steps of a visual positioning method according to an embodiment of the present invention;
fig. 2 is a diagram illustrating a design example of rotation robustness of a lead two-dimensional code according to an embodiment of the present invention;
fig. 3 is an exemplary diagram of a lead two-dimensional code according to an embodiment of the present invention;
fig. 4 is another exemplary diagram of a rotation robustness design of a lead two-dimensional code according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an encoding algorithm according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a visual positioning apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
The embodiments of the present application will be further explained with reference to the drawings.
Referring to fig. 1, an embodiment of the present invention includes a visual positioning method including, but not limited to, the steps of:
s1, spraying or pasting a plurality of lead two-dimensional codes on the surface of a patient;
s2, scanning the patient, establishing a three-dimensional model before operation, and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in the three-dimensional model;
s3, obtaining a second coordinate set, wherein the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
s4, identifying and coding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code;
s5, calculating to obtain a plurality of groups of registration parameters according to the first coordinate group and the second coordinate group;
s6, calculating a reprojection error of the center of each lead two-dimensional code according to the multiple groups of registration parameters;
s7, optimizing the multiple groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters;
and S8, determining the real focus position of the patient in the operation according to the optimized registration parameters.
The embodiment of the invention provides a visual positioning method, which can be used for registering a focus region in an intraoperative camera visual system and a focus region in an preoperative three-dimensional imaging system based on a lead two-dimensional code marker; the main parts involved are: the system comprises a lead two-dimensional code marker, a two-dimensional code detection and coding system, a preoperative three-dimensional imaging system, an intraoperative camera vision system and a computer processing system, wherein the computer processing system is connected with the intraoperative camera vision system and the two-dimensional code detection and coding system, and all data are gathered into the computer processing system for registration.
The lead two-dimensional code marker is formed by two modes of lead powder material painting two-dimensional codes or attaching the two-dimensional codes to a lead layer, lead belongs to a developing material, the lead two-dimensional codes have identification very easily, and a unique marker ID can be obtained through a two-dimensional code detection and coding system and is very easily identified by a computer; referring to fig. 2, the lead two-dimensional code determines the direction of the two-dimensional code by setting a reference for three of four corners of a square, so as to determine the rotation angle difference between the three-dimensional image model and the position of the intraoperative real patient, and help the computer processing system to perform quick registration.
Specifically, the lead material in the lead two-dimensional code marker is used for developing and imaging on the CT image, so that registration is carried out; the two-dimensional code in the lead two-dimensional code marker is used for being identified by an intraoperative camera vision system and providing unique information of the marker; before the preoperative three-dimensional imaging system scans the patient, at least three lead two-dimensional code markers are attached to the patient, as shown in fig. 3, and are used for registration.
The lead two-dimensional code is required to be designed into a pattern with certain specified characteristics, specifically:
(1) The whole two-dimensional code is limited to be composed of 6 multiplied by 6 black and white quadrangles (quad); the size of each quadrangle is 0.5cm × 0.5cm. Each quadrilateral ranges from 49 to 100 pixels. Due to the design, the two-dimensional code can be clearly identified by a camera and can be quickly detected and encoded;
(2) Quadrangles on the periphery of the two-dimensional code are all set to be black, quadrangles at the upper left, upper right and lower left positions on the periphery of the second part are white, and quadrangles at the lower right position are black, as shown in fig. 4. The design is to enable the lead two-dimensional code marker to have rotation robustness, and through the mode, the camera can easily identify the direction of the two-dimensional code and can be quickly registered with a preoperative three-dimensional modeling system.
In the embodiment, after three lead two-dimensional codes are sprayed or pasted on the body surface of a patient, the patient is scanned through a preoperative three-dimensional imaging system, a preoperative three-dimensional model is established, and the coordinate positions of the centers of three lead two-dimensional code markers under the three-dimensional model system are obtained, namely a first coordinate set is obtained and is used for registering a focus area in an operation; and then, acquiring an image of the body surface of the patient in the operation through an intraoperative camera vision system, acquiring coordinates of the centers of the three two-dimensional code markers in an intraoperative camera coordinate system, namely acquiring a second coordinate set, and handing the second coordinate set to a two-dimensional code detection and coding system for processing. The two-dimensional code detection and coding system detects each edge in the two-dimensional code image according to the gradient; finding out a needed quadrilateral pattern in the edge image and screening; and determining a series of strings of 0 and 1 according to the gray value of each quadrilateral pattern and the size of the threshold value to obtain the information of the two-dimensional code.
Specifically, a two-dimensional code on a marker is identified through a two-dimensional code detection and coding system and is coded to obtain unique information; the two-dimension code identification specifically refers to searching possible two-dimension code images in an image scene, namely trying to search a quadrangle with black inside and white outside, including line segment detection and quadrangle detection; the line segment detection specifically comprises the steps of calculating the gradient direction and the amplitude of each pixel point in an image, classifying pixels with the same gradient direction and amplitude into one class, and finally obtaining a line segment through a Felzenszwalb clustering algorithm. The mode is insensitive to illumination change and is suitable for illumination with various brightness in an operating room. Specifically, the quadrangle detection is to adopt an iterative algorithm with the depth of 4, take the end point of the non-closed loop segment as the starting point, acquire the segment, and after 4 iterations, if a closed loop can be formed within the threshold range, the quadrangle detection is successful.
Referring to fig. 5, the encoding process specifically includes: respectively obtaining the gray average Value Black Value of the Black quadrangle and the gray average Value White Value of the White quadrangle, calculating a Threshold Value Threshold through the Black Value and the White Value, traversing the pixels in each quadrangle, and if the gray Value of the pixel is greater than the Threshold Value, performing AND operation on the coding variables TagCode and 1 and shifting the coding variables by one bit to the left.
The two-dimensional code detection and coding system identifies the two-dimensional code on the marker, codes the two-dimensional code, and returns the two-dimensional code to the computer processing system after obtaining the unique information, and the computer processing system calculates a plurality of sets of registration parameters (namely, a rotation matrix and a translation vector of coordinates of the preoperative three-dimensional imaging system and the intraoperative camera vision system) between the preoperative three-dimensional imaging system and the intraoperative camera vision system by combining a first coordinate set provided by the preoperative three-dimensional imaging system and a second coordinate set provided by the intraoperative camera vision system. And calculating a reprojection error of the center of each lead two-dimensional code according to the obtained multiple groups of registration parameters, and optimizing the parameters by using a nonlinear least square method to minimize the reprojection error and obtain a final registration result.
In the embodiment, in the registration process, the unique mark ID can effectively correspond to the mark, registration deviation caused by rotation disturbance is prevented, each lead two-dimensional code takes the center of the lead two-dimensional code as an origin, a coordinate system is established, the preoperative three-dimensional imaging system and the intraoperative camera vision system can be registered according to the coordinate system transformation principle, and at least three lead two-dimensional codes are attached to each patient, so that at least three groups of registration parameters can be obtained.
The visual positioning method provided by the embodiment of the invention has the following technical effects:
(1) The registration precision is not influenced by the deformation of the skin on the body surface of the patient; according to the visual positioning method based on the plumbous two-dimensional code markers, each two-dimensional code marker can take the center of the marker as an original point and three vertexes of a tetragon of the marker as reference to form a coordinate system, and the relative position of each vertex of the two-dimensional code is fixed and cannot be changed due to the deformation of the skin on the body surface;
(2) The registration parameters are optimized so that the registration parameters are more accurate; according to the embodiment of the invention, at least three two-dimensional code markers are attached to a patient, a coordinate system can be established for each two-dimensional code to obtain a registration parameter, then the three parameters are respectively re-projected, the re-projection error is calculated, and the parameters are optimized through nonlinear least squares, so that the parameters are more accurate;
(3) The operation steps are simple, convenient and fast, and the cost is low; the embodiment of the invention provides a lightweight registration method based on the lead two-dimensional code markers, only three lead two-dimensional code markers are needed to be pasted on the body surface of a patient for each registration, and the method is simple, convenient and low in cost and can be thrown away after use;
(4) The two-dimensional codes contain specific semantic information, so that each two-dimensional code marker can be uniquely determined in the registration process, can correspond to each other one by one during preoperative registration and is not easy to be confused; even if the markers fall off due to untight adhesion of some reasons before registration, the markers can be easily identified, compared with the traditional marker positioning and registration, the method is convenient, the error calibration caused by rotation deviation can be avoided, the preoperative registration time is saved, and the method has high use value;
(5) The embodiment of the invention utilizes a two-dimensional code detection and coding system, combines a rapid and powerful line detection system, and has a stronger digital coding system and higher robustness on shielding, warping and lens distortion.
The embodiment of the invention also provides a visual positioning system, which comprises a preoperative three-dimensional imaging system, an intraoperative camera visual system, a two-dimensional code detection and coding system and a computer processing system; the intraoperative camera vision system and the two-dimensional code detection and coding system are both connected with the computer processing system;
the preoperative three-dimensional imaging system is used for scanning a patient, establishing a preoperative three-dimensional model and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of a plurality of lead two-dimensional codes in the three-dimensional model;
the intraoperative camera vision system is used for acquiring a second coordinate set, and the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
the two-dimension code detection and coding system is used for identifying and coding each lead two-dimension code to obtain a unique mark ID of each lead two-dimension code;
the computer processing system is used for calculating three sets of registration parameters according to the first coordinate set and the second coordinate set; calculating a reprojection error of the center of each lead two-dimensional code according to the three groups of registration parameters; optimizing the three groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters; and determining the real lesion position of the patient in the operation according to the optimized registration parameters.
Referring to fig. 6, an embodiment of the present invention further provides a visual positioning apparatus 200, which specifically includes:
at least one processor 210;
at least one memory 220 for storing at least one program;
when executed by the at least one processor 210, causes the at least one processor 210 to implement the method as shown in fig. 1.
The memory 220, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs and non-transitory computer-executable programs. The memory 220 may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 220 may optionally include remote memory located remotely from processor 210, and such remote memory may be connected to processor 210 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
It will be understood that the device structure shown in fig. 6 does not constitute a limitation of device 200, and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
In the apparatus 200 shown in fig. 6, the processor 210 may retrieve the program stored in the memory 220 and execute, but is not limited to, the steps of the embodiment shown in fig. 1.
The above-described embodiments of the apparatus 200 are merely illustrative, and the units illustrated as separate components may or may not be physically separate, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purposes of the embodiments.
Embodiments of the present invention also provide a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used for implementing the method shown in fig. 1 when being executed by the processor.
The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and executed by the processor to cause the computer device to perform the method illustrated in fig. 1.
It will be understood that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as is well known to those skilled in the art.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
Claims (9)
1. A visual positioning method, comprising:
spraying and printing or pasting a plurality of lead two-dimensional codes on the surface of a patient;
scanning a patient, establishing a three-dimensional model before operation, and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in the three-dimensional model;
acquiring a second coordinate set, wherein the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
identifying and coding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code; the step of identifying and encoding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code includes: detecting each edge in the two-dimensional code image according to the gradient to obtain an edge image; acquiring and screening out a required quadrilateral pattern from the edge image; determining a unique mark ID of each lead two-dimensional code according to the gray value and the threshold value of each quadrilateral pattern;
calculating to obtain a plurality of groups of registration parameters according to the first coordinate group and the second coordinate group;
calculating a reprojection error of the center of each lead two-dimensional code according to the multiple groups of registration parameters;
optimizing the multiple groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters;
and determining the real lesion position of the patient in the operation according to the optimized registration parameters.
2. The visual positioning method according to claim 1, wherein the lead two-dimensional code is formed by at least one of the following methods:
the two-dimensional code is sprayed and painted by lead powder;
or attaching the two-dimensional code on the lead layer.
3. The visual positioning method according to claim 1, wherein the lead two-dimensional code is set in the following pattern:
is composed of 6 × 6 black-and-white quadrangles each having a size of 0.5cm × 0.5cm, each quadrangle ranging from 49 to 100 pixels.
4. The visual positioning method of claim 1, wherein the step of detecting each edge in the two-dimensional code image according to the gradient to obtain an edge image comprises:
calculating the gradient direction and amplitude of each pixel point in the two-dimensional code image;
classifying pixels with the same gradient direction and the same amplitude into one class;
and acquiring an edge line segment by using a Felzenszwalb cluster algorithm to obtain an edge image.
5. The visual positioning method of claim 1, wherein the step of obtaining and screening out the desired quadrilateral pattern from the edge image comprises:
acquiring the line segment by adopting an iterative algorithm with the depth of 4 and taking the end point of the non-closed loop line segment as a starting point;
after 4 iterations, if a closed loop is formed in the threshold range, determining the closed loop as a quadrilateral pattern;
and screening the quadrilateral patterns to obtain the required quadrilateral patterns.
6. The visual positioning method of claim 1, wherein the quadrilateral patterns are composed of a plurality of black quadrilaterals and a plurality of white quadrilaterals, and the step of determining the unique mark ID of each lead two-dimensional code according to the gray value size and the threshold size of each quadrilateral pattern comprises the following steps:
acquiring the gray level mean value of a black quadrangle and the gray level mean value of a white quadrangle in the quadrangle pattern;
calculating to obtain a threshold value according to the gray level average value of the black quadrangle and the gray level average value of the white quadrangle;
traversing pixels in each quadrilateral;
and if the gray value of the pixel is larger than the threshold value, performing AND operation on the coding variable TagCode and 1 and shifting the coding variable TagCode and the coding variable TagCode by one bit to the left so as to obtain the unique mark ID of each lead two-dimensional code.
7. A visual positioning system is characterized by comprising a preoperative three-dimensional imaging system, an intraoperative camera visual system, a two-dimensional code detection and coding system and a computer processing system; the intraoperative camera vision system and the two-dimensional code detection and coding system are both connected with the computer processing system;
the preoperative three-dimensional imaging system is used for scanning a patient, establishing a preoperative three-dimensional model and acquiring a first coordinate set, wherein the first coordinate set is a coordinate of the centers of a plurality of lead two-dimensional codes in the three-dimensional model;
the intraoperative camera vision system is used for acquiring a second coordinate set, and the second coordinate set is a coordinate of the centers of the plurality of lead two-dimensional codes in an intraoperative camera coordinate system;
the two-dimensional code detection and coding system is used for identifying and coding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code; the step of identifying and encoding each lead two-dimensional code to obtain a unique mark ID of each lead two-dimensional code includes: detecting each edge in the two-dimensional code image according to the gradient to obtain an edge image; acquiring and screening out a required quadrilateral pattern from the edge image; determining a unique mark ID of each lead two-dimensional code according to the gray value and the threshold value of each quadrilateral pattern;
the computer processing system is used for calculating a plurality of groups of registration parameters according to the first coordinate group and the second coordinate group; calculating a reprojection error of the center of each lead two-dimensional code according to the multiple groups of registration parameters; optimizing the multiple groups of registration parameters by using a nonlinear least square method to obtain optimized registration parameters; and determining the real lesion position of the patient in the operation according to the optimized registration parameters.
8. A visual positioning device, comprising:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of any one of claims 1-6.
9. Computer-readable storage medium, characterized in that a program executable by a processor is stored thereon, which program, when being executed by the processor, is adapted to carry out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110556239.7A CN113384361B (en) | 2021-05-21 | 2021-05-21 | Visual positioning method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110556239.7A CN113384361B (en) | 2021-05-21 | 2021-05-21 | Visual positioning method, system, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113384361A CN113384361A (en) | 2021-09-14 |
| CN113384361B true CN113384361B (en) | 2022-10-28 |
Family
ID=77618825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110556239.7A Active CN113384361B (en) | 2021-05-21 | 2021-05-21 | Visual positioning method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113384361B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114288023A (en) * | 2022-01-26 | 2022-04-08 | 北京柏惠维康科技有限公司 | Calibration method, device, equipment and medium for positioning guide plate in surgical navigation equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2445961B (en) * | 2006-10-31 | 2009-02-04 | Prosurgics Ltd | Fiducial marker placement |
| CN111291584B (en) * | 2016-07-22 | 2023-05-02 | 创新先进技术有限公司 | Method and system for identifying two-dimensional code position |
| CN109833092A (en) * | 2017-11-29 | 2019-06-04 | 上海复拓知达医疗科技有限公司 | Internal navigation system and method |
| JP7440534B2 (en) * | 2019-04-04 | 2024-02-28 | センターライン バイオメディカル，インコーポレイテッド | Spatial registration of tracking system and images using 2D image projection |
| CN111899823A (en) * | 2020-06-11 | 2020-11-06 | 上海梅斯医药科技有限公司 | Method and system for processing gauge information, terminal equipment and storage medium |
-
2021
- 2021-05-21 CN CN202110556239.7A patent/CN113384361B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113384361A (en) | 2021-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| US11123144B2 (en) | Registration of frames of reference | |
| US10060739B2 (en) | Method for determining a position and orientation offset of a geodetic surveying device and such a surveying device | |
| JP4739004B2 (en) | Information processing apparatus and information processing method | |
| US9560291B2 (en) | Automatic improvement of tracking data for intraoperative C-arm images in image guided surgery | |
| CN111882612A (en) | Vehicle multi-scale positioning method based on three-dimensional laser detection lane line | |
| Gschwandtner et al. | Infrared camera calibration for dense depth map construction | |
| JP2016167229A (en) | Coordinate transformation parameter determination device, coordinate transformation parameter determination method, and computer program for coordinate transformation parameter determination | |
| CN106999256A (en) | Optical tracking method and system based on passive marker | |
| WO2022078513A1 (en) | Positioning method and apparatus, self-moving device, and storage medium | |
| JP2007164631A (en) | Indicator identification method and indicator identification device | |
| WO2022078488A1 (en) | Positioning method and apparatus, self-moving device, and storage medium | |
| CN108022265B (en) | Method, equipment and system for determining pose of infrared camera | |
| CN106952219B (en) | Image generation method for correcting fisheye camera based on external parameters | |
| CN112435300A (en) | Positioning method and device | |
| CN111964680B (en) | Real-time positioning method of inspection robot | |
| CN110490943B (en) | Rapid and accurate calibration method and system of 4D holographic capture system and storage medium | |
| CN113384361B (en) | Visual positioning method, system, device and storage medium | |
| CN106886976B (en) | Image generation method for correcting fisheye camera based on internal parameters | |
| JP2015194397A (en) | Vehicle location detection device, vehicle location detection method, vehicle location detection computer program and vehicle location detection system | |
| CN111210478A (en) | Method, medium and system for calibrating external parameters of common-view-free multi-camera system | |
| CN105513074B (en) | A kind of scaling method of shuttlecock robot camera and vehicle body to world coordinate system | |
| CN113884081B (en) | Method and equipment for measuring three-dimensional coordinates of positioning point | |
| Leventon et al. | Multiple view 2D-3D mutual information registration | |
| CN110021035A (en) | The marker of Kinect depth camera and virtual tag object tracking based on the marker |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |