WO2022206434A1 - 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 - Google Patents
一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 Download PDFInfo
- Publication number
- WO2022206434A1 WO2022206434A1 PCT/CN2022/081727 CN2022081727W WO2022206434A1 WO 2022206434 A1 WO2022206434 A1 WO 2022206434A1 CN 2022081727 W CN2022081727 W CN 2022081727W WO 2022206434 A1 WO2022206434 A1 WO 2022206434A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- registration
- registration device
- module
- identifiable
- preset
- Prior art date
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims description 51
- 210000003484 anatomy Anatomy 0.000 claims abstract description 35
- 230000033001 locomotion Effects 0.000 claims description 146
- 238000004364 calculation method Methods 0.000 claims description 36
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000009471 action Effects 0.000 claims description 7
- 238000011065 in-situ storage Methods 0.000 abstract 5
- 230000008569 process Effects 0.000 description 31
- 238000004422 calculation algorithm Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 239000003550 marker Substances 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000001356 surgical procedure Methods 0.000 description 5
- 206010011224 Cough Diseases 0.000 description 4
- 208000012661 Dyskinesia Diseases 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 206010028347 Muscle twitching Diseases 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000036461 convulsion Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004659 sterilization and disinfection Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000001020 rhythmical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
Definitions
- the present application relates to the medical field, and in particular, to a cross-registration system, method, electronic device and readable storage medium for surgical navigation.
- the surgical navigation system accurately corresponds the preoperative or intraoperative image data of the patient with the patient's anatomical structure on the operating bed, tracks the surgical instruments during the operation, and updates and displays the position of the surgical instruments on the patient image in real time in the form of a virtual probe, allowing doctors to See at a glance the position of surgical instruments relative to the patient's anatomy, making surgery faster, more precise, and safer.
- augmented reality for surgical navigation, first of all, it is necessary to store the surgical plan information in the computer in advance; secondly, it is also necessary to convert the medical image data obtained by preoperative image scanning with the real-time images of the subject during the operation through coordinate transformation. way to achieve data registration.
- the point cloud is usually extracted from the scene images captured multiple times, and the registration between the point cloud of the scene scene and the computer model is performed. The higher the accuracy of data registration, the more accurate the navigation information obtained by the operator.
- the operator or the operating object will inevitably have various situations such as movement, shaking, breathing, heartbeat, etc. These situations will increase the difficulty of the registration process and reduce the accuracy.
- the present application provides a cross-registration system, method, electronic device and readable storage medium for surgical navigation.
- a first aspect of the present application provides an interactive registration system for surgical navigation, characterized in that the device includes:
- a medical image acquisition module an on-site image acquisition module, an identification module, a judgment module and an execution module;
- the medical image acquisition module is used to acquire the preoperative medical image of the operation object, and obtain the anatomical structure model of the operation object according to the preoperative medical image;
- the on-site image acquisition module is used to acquire on-site images of the operation site
- the identification module is used to identify the position of the registration device in the operation scene according to the scene image
- the judging module is used to judge whether the preset registration condition is currently met according to the position state of the registration device; when the preset registration condition is met, the anatomical structure model of the operation object and For the scene image, the anatomical structure model is registered with the operation scene.
- the registration device includes at least an identifiable part and a marked part; the preset registration conditions are:
- the preset motion parameters are motion parameters corresponding to body motions caused by human respiration or heartbeat.
- the motion parameter includes at least one of motion frequency, motion amplitude, motion speed and motion trajectory.
- the judging module further includes a computing module, and the computing module is configured to perform the following calculations:
- the distance between any point of the identifiable part of the registration device relative to its geometric mean position is less than or equal to a first preset threshold, it is determined that the identifiable part of the registration device is in a stable state.
- the distance between any point of the marking part of the registration device and its geometric mean position; is less than or equal to the second preset threshold, it is determined that the marking part of the registration device is in a stable state.
- the judging module further includes a recording module, and the recording module is specifically used for:
- the average value of the coordinates, and the set time is the time when the motion parameter is the smallest in the motion parameter cycle.
- a prompt module is also included, which is used to guide the operator to perform a specific operation.
- the prompt module is configured to display prompt information to prompt the operator to make a specified behavior to meet the preset registration conditions; the prompt information specifically includes:
- the operator is guided to maintain a specified positional relationship with respect to the operation target.
- a dynamic pattern for representing the progress of time is displayed.
- the prompting module prompts the operator to maintain the specified state again and restart the calculation of the specified time.
- a second aspect of the present application includes:
- the registration device includes at least an identifiable part and a marked part; the registration conditions are:
- the moving speed of the marking part of the registration device at each moment in the time period is calculated.
- the distance between any point of the identifiable part of the registration device and its geometric mean position is less than or equal to a first preset threshold, it is determined that the identifiable part of the registration device is in a stable state.
- the distance between any point of the marking part of the registration device and its geometric mean position; is less than or equal to the second preset threshold, it is determined that the marking part of the registration device is in a stable state.
- the present invention also includes the following recording steps:
- the identifiable part of the registration device When the identifiable part of the registration device is in a stable state and the motion parameters of the marked part of the registration device match the preset motion parameters, record the average value of the coordinates of the identifiable part at the set moment , and the set time is the time when the motion parameter is the smallest in the motion parameter cycle.
- the present invention further includes a prompting step for guiding the operator to perform a specific operation.
- the prompting step of the present invention is used to display prompt information to prompt the operator to make a specified behavior to meet the preset registration conditions;
- the prompt information specifically includes:
- the operator is guided to maintain a specified positional relationship with respect to the operation target.
- a dynamic pattern for representing the progress of time is displayed.
- the prompting module prompts the operator to maintain the specified state again and restart the calculation of the specified time.
- an electronic device in a third aspect of the present application, includes a memory and a processor; wherein, the memory is used to store executable program codes; the computer instructions are executed by the processor to implement any of the second aspect of the present application. one of the methods described.
- a fourth aspect of the present application a computer-readable storage medium storing a computer program, characterized in that, when the computer instruction is executed by a processor, any one of the second aspect of the present application is implemented.
- the technical solution in the embodiment of the present invention can automatically identify the registration device included in the image of the operation scene, and according to the anatomical structure model of the operation object and the scene image, the anatomical structure model and the corresponding anatomical structure of the operation scene Perform location registration.
- the invention enables the operator to automatically complete the registration of one or more spatial points in the whole process without other operations or instructions, such as operating a keyboard, a mouse, a voice password, a gesture, and the like.
- the registration of specific spatial points can also be completed under the condition that the spatial points move synchronously with the physiological movements (such as respiration, heartbeat, shaking, etc.) of the surgical object.
- physiological movements such as respiration, heartbeat, shaking, etc.
- the present invention can also help the operator to complete the registration process more quickly and accurately by providing the whole-process augmented reality prompt information.
- implementing the technical solution of the present invention can improve the accuracy and efficiency of spatial position registration.
- FIG. 1 is a structural block diagram of an interactive registration system for surgical navigation disclosed in an embodiment of the present application
- FIG. 2 is a structural block diagram of another interactive registration system for surgical navigation disclosed in an embodiment of the present application.
- FIG. 3 is a schematic diagram of a surgical scene disclosed in an embodiment of the present application.
- FIG. 4 is a motion parameter diagram of a marked part of a registration device disclosed in an embodiment of the present application.
- FIG. 5 is a motion parameter diagram of another registration device marking part disclosed in an embodiment of the present application.
- Fig. 6 is a flowchart of an interactive registration method for surgical navigation disclosed in an embodiment of the present application.
- FIG. 7 is a structural block diagram of another electronic device disclosed in an embodiment of the present application.
- FIG. 8 is a schematic structural diagram of a computer system disclosed in an embodiment of the present application.
- an interactive registration system for surgical navigation includes: a medical image acquisition module, an on-site image acquisition module, an identification module, a calculation module and an execution module; wherein,
- the medical image acquisition module is used to acquire the preoperative medical image of the operation object, and obtain the anatomical structure model of the operation object according to the preoperative medical image;
- the on-site image acquisition module is used to acquire on-site images of the operation site
- the identification module is used for identifying the position of the registration device in the operation site according to the scene image, and the registration device at least includes an identifiable part and a marked part;
- the judging module is used to judge whether the preset registration condition is currently met according to the position state of the registration device; when the preset registration condition is met, the anatomical structure model of the operation object and For the scene image, the anatomical structure model is registered with the operation scene.
- the technical solution in the embodiment of the present invention can automatically identify the registration equipment included in the surgical scene image, and perform position registration between the anatomical structure model and the corresponding anatomical structure on the surgical site according to the anatomical structure model of the operating object and the on-site image.
- the present invention enables the operator to automatically complete the registration of one or more spatial points in the whole process without other operations or instructions, such as operating a keyboard, a mouse, a voice password, a gesture, and the like.
- the identification module of the present invention is used for identifying the position of the registration device in the operation scene according to the scene image.
- the registration device includes at least an identifiable part and a marked part.
- the identifiable part has at least one of specific optical features, pattern features and geometric features, so that the image obtained by photographing the marker has specific image features, for example, the pattern information is a two-dimensional code.
- the marked part can be used to touch/click on a designated position, and the designated position can be a predetermined point on the body of the surgery object, and the spatial coordinates of these points have been configured in the interactive registration system in advance.
- the number of the designated positions is 2 or more.
- the recognition module in this embodiment of the present invention may use an existing image recognition algorithm to perform image recognition, such as a spot detection algorithm, a corner detection algorithm, and the like.
- a corresponding appropriate algorithm can be selected according to the form of the marker. For example, when the marker is a two-dimensional code set on an operating table or a surgical instrument, a corresponding two-dimensional code identification algorithm can be directly used.
- the relative coordinate relationship between the identifiable part and the marked part of the registration device can be obtained by calculation from the associated structural information in the database.
- the spatial coordinates of the marked site can be obtained by registering the optical features on the device.
- the space coordinates of the identifiable parts and the space coordinates of the marked parts can be converted to the world coordinate system through coordinate transformation.
- the registration device in the embodiment of the present application is preferably a device consistent with a surgical instrument, and more preferably a medical device integrating a registration function and a surgical implementation function, which can reduce data errors caused by instrument replacement.
- the registration device can be a puncture needle, and the identifiable part is an optical feature set on the puncture needle, such as a two-dimensional code; the marked part can be the tip of the puncture needle. In this way, the registration device satisfies one of two conditions: re-sterilization or single-use aseptic use.
- the judgment module in the embodiment of the present invention is used for judging whether the position of the registration device complies with a preset registration condition.
- judging whether the position of the registration device complies with the preset registration conditions according to the anatomical structure model of the operation object and the scene image, position the anatomical structure model and the operation scene registration.
- the preset registration condition is: when the identifiable part and the marked part of the registration device are in a stable state; and/or, when the identifiable part of the registration device is in a stable state; stable state, and the motion parameters of the marked part of the registration device match the preset motion parameters.
- the preset motion parameter may be a parameter value, or may be a certain value range.
- the position registration device meets the registration conditions, and the position registration operation can be performed.
- the registration operation is not performed.
- the position of the registration device meets the situation (1), it can be considered that the registration device has been in a stable state, and the registration can be started; when the position of the registration device meets the situation (2), it can be Considering that the identified site of the registration device is in a stable state with respect to the body surface of the subject, the registration can be started. In this way, the recording of a specific spatial point can be completed under the condition that the designated collection position moves synchronously with the physiological movement of the surgical object. That is to say, the spatial position registration system of the present invention can eliminate the influence caused by the specific movement of the body surface of the operation object.
- the preset motion parameters are motion parameters corresponding to body motions caused by human respiration or heartbeat.
- the motion parameter includes at least one of motion frequency, motion amplitude, motion speed and motion trajectory.
- the motion parameters can be provided in advance or collected on site. The on-site collection can be carried out in the existing manner in the prior art.
- the judgment module further includes a calculation module, and the calculation module is configured to perform the following calculations:
- the moving speed of the marking part of the registration device at each moment in the time period is calculated.
- the calculation module specifically performs the following calculations:
- V 2i B i B i-1 / ⁇ t ⁇ Formula 3>
- D 1i is the distance from any point of the identifiable part of the registration device to its geometric mean position within the ⁇ t time period;
- a i is the position coordinate of the identifiable part of the registration device in the ⁇ t time period, and the position coordinate of A i is represented by (X 1i , Y 1i , Z 1i );
- O 1 is the geometric mean coordinate of the position coordinates of the identifiable part of the registration device in the time period ⁇ t, and the position coordinates of O 1 are To represent;
- D 2i is the distance between any point of the marking part of the registration device relative to its geometric mean position in the ⁇ t time period
- B i is the position coordinate of the marking part of the registration device in the ⁇ t time period, and the position coordinate of B i is represented by (X 2i , Y 2i , Z 2i );
- O 2 is the geometric mean coordinate of the position coordinates of the marking part of the registration device in the ⁇ t time period, and the position coordinates of O 2 are To represent;
- V 2i is the movement speed of the marking part of the registration device in the time period ⁇ t;
- the coordinate positions can be characterized in world coordinates or space coordinates.
- the calculation process is as follows:
- V 2i B i B i-1 / ⁇ t ⁇ Formula 3>;
- the coordinate position is represented by world coordinates or space coordinates.
- the judging module when all D 1i are less than or equal to the first preset threshold, it is judged that the identifiable part of the registration device is in a stable state.
- the first preset value is 1-3 mm; more preferably, the first preset value is 2 mm.
- the second preset value is 1-3mm; more preferably, the second preset value is 2mm.
- the third preset value is 1-3 mm/s; more preferably, the third preset value is 2 mm/s.
- the system starts to perform the registration operation when all D 1i are smaller than 2 mm, and when all D 2i are smaller than 2 mm.
- the motion parameter is motion amplitude
- the horizontal axis represents time
- the vertical axis represents the movement range of the spatial position of the marking part of the registration device. It can be seen from the figure that at time T1, the spatial position of the marked part reaches the highest position; at time T2, the spatial position of the marked part reaches the lowest position; at time T3, the spatial position of the marked part reaches the highest position again.
- the collected motion amplitude curve is compared with the motion amplitude curve of the preset position point collected in real time or pre-stored with respiration, and if the two match, it is considered that in the time period from T1 to T3, the The marked site of the quasi-device is in a stable state with respect to the body surface, and the registration operation can be performed.
- the present invention can also include a recording module, the recording module is used to obtain the judgment result of the specified position point, and when the judgment result is that the position of the registration device meets the registration conditions, the coordinate information of the current specified position point is recorded. . If there is movement during recording that exceeds either threshold, the recording is stopped.
- the number of the designated positions is 2 or more.
- the recording module in the present invention can be used to continuously record the coordinate information of multiple designated position points within a period of time.
- the average coordinates of the marked part of the registration device at each moment are recorded.
- D 1i of the identifiable part of the registration device is less than a first preset threshold, such as 2 mm
- D 2i of the marked part is less than a second preset threshold, such as 2 mm
- the position of the identifiable part of the registration device is in a stable state and the motion parameters of the marked part of the registration device match the preset motion parameters
- recording the The average value of the coordinates of the identifiable part at a set time where the set time is the time when the motion parameter is the smallest in the motion parameter cycle.
- a first preset threshold such as 2 mm
- the motion parameters of the marked part of the registration device match the motion parameters of the human heartbeat
- the average value of the coordinates at a fixed time, and the set time is the time when the heartbeat amplitude is the smallest in the human heartbeat cycle.
- the recording process is specifically as follows: in acquiring n motion parameter periods, the registration device marks the part B (X 2 , Y 2 , Z 2 ) at the moment of the minimum motion parameter value in each motion period The corresponding position coordinates B' 1 (X 21 , Y 21 , Z 21 )...B' n (X 2n , Y 2n , Z 2n ), calculate the geometric mean position coordinates, denoted as record stated If there is any movement beyond any threshold during the recording, the recording will be stopped and the judgment phase will be restarted.
- the interactive registration system for surgical navigation described in the embodiment of the present invention further includes a prompting module for guiding the operator to perform a specific operation.
- the prompting module of the present invention is used to display prompting information to prompt the operator to perform a specified behavior to meet the preset registration conditions; the prompting information specifically includes: instructing the operator to register The device moves to a specified position, or guides the operator to move itself relative to the object of operation according to a specified positional relationship, or instructs the operator to control the registration device to maintain a specified state for a specified time, or guide the operation It keeps itself in the specified positional relationship with respect to the cast object.
- the prompting module further includes a determination module and a display module.
- the determination module is used to determine the specific stage of registration; the display module is used to display the navigation stage and specific guidance information, and different registration stages display different guidance information.
- the display module at least displays and guides the operator to move the registration device to a specified position, or guides the operator to make himself relative to the operation object according to a specified positional relationship Movement information;
- the specific stage of registration is the position recording stage, the display module at least displays and guides the operator to keep the registration device in a specified state, or guides the operator to maintain a specified positional relationship with respect to the surgical object Information.
- the interactive registration system for surgical navigation of the present invention can realize the function of differentiating reminders according to the specific stages of registration, thereby improving the efficiency and accuracy of the registration process.
- the instructing the operator to display a dynamic pattern for representing the progress of time when the registration device is instructed to maintain a specified state for a specified period of time. For example, hourglass, progress bar, circle, etc.
- the prompt for guiding the operator to perform a specific operation is implemented in one or more forms of text, arrow, voice, color, pattern, animation, and rhythm sound.
- the prompt form includes one or more of characters, arrows, colors, patterns, and animations. More preferably, when the specific stage of registration is the position registration stage, the prompt form is arrows or text; when the specific stage of registration is the position recording stage, the prompt form is animation or text.
- the prompting manner of the present invention can be set by the operator. The advantage of this is that it can deepen the operator's memory and enhance the degree of personalization of the system.
- the display module when the specific stage of registration is the position registration stage, the display module displays the text "Currently is the registration stage, please move the registration device to the specified position", and is accompanied by a direction arrow; when the specific stage of registration is the position recording stage, the display module will display the text "Have entered the recording stage, please keep the current action", accompanied by animation of circles, hourglasses, or progress bars.
- selecting a specific color indicates that the current stage is the registration stage, and the text "Please move the registration device to the specified position" is displayed in the display module. , and accompanied by a direction arrow;
- select a specific color to indicate that the current is the registration stage, and the text "Please keep the current action” is displayed in the display module, accompanied by circles and hourglasses. , or the animation of the progress bar.
- the prompting module if the registration device fails to maintain the specified state within the specified time, the prompting module prompts the operator to maintain the specified state again and restart the calculation of the specified time. This is to prevent errors in the registration data caused by abnormal jitter during the registration process. For example, the operator may sneeze, cough, or perform other abnormal movements within the specified time when the system prompts "please maintain the current action", and the subject may also experience abnormal movements such as body twitches and sudden changes in position. If the quasi-equipment fails to maintain the specified state for the specified time, it is necessary to interrupt the ongoing registration process and start a new round of position calculation and registration.
- the purpose of maintaining the designated state is to complete the registration process.
- the system prompt module displays prompt information instructing the operator to direct the registration device to maintain a specified state for a specified time
- the system is performing a position registration process.
- the system prompt module displays prompt information instructing the operator to direct the registration device to maintain a specified state for a specified time
- the system is performing a recording process.
- a judgment method similar to that of the recording process can be adopted. For example, during the registration process, if the registration device moves beyond any threshold, the current position registration and time calculation will be stopped, and then a new registration and time calculation will be restarted. Specifically, when there is an abnormal action caused by the operator's coughing during the registration process, it is determined whether the identifiable part and the marked part of the registration device have moved beyond any threshold, and if the determination result is yes, then stop the current Position registration and time calculation, if there is a recording process, stop together, and re-execute the judgment of calculating the position coordinates of the registration device and the registration conditions.
- Figure 6 shows a second aspect of the present invention, a cross-registration method for surgical navigation, comprising:
- S601 Obtain a preoperative medical image of an operation object, and obtain an anatomical structure model of the operation object according to the preoperative medical image;
- S604 Determine whether the preset registration condition is currently met according to the position state of the registration device; when the preset registration condition is met, according to the anatomical structure model of the operation object and the on-site image, The anatomical structure model is positionally registered with the surgical site.
- the technical solution in the embodiment of the present invention can automatically identify the registration equipment included in the surgical scene image, and perform position registration between the anatomical structure model and the corresponding anatomical structure on the surgical site according to the anatomical structure model of the operating object and the on-site image.
- the present invention enables the operator to automatically complete the registration of one or more spatial points in the whole process without other operations or instructions, such as operating a keyboard, a mouse, a voice password, a gesture, and the like.
- Fig. 3 is the operation performed by the operator 1 to the frame area 3 according to the camera wearing the head-mounted device 2 scene image.
- the identification step identifies the position of the registration device in the operation scene according to the scene image.
- the registration device includes at least an identifiable part and a marked part.
- the identifiable part has at least one of specific optical features, pattern features and geometric features, so that the image obtained by photographing the marker has specific image features, for example, the pattern information is a two-dimensional code.
- the marked part can be used to touch/click on a designated position, and the designated position can be a predetermined point on the body of the surgery object, and the spatial coordinates of these points have been configured in the interactive registration system in advance.
- the number of the designated positions is 2 or more.
- the recognition module in this embodiment of the present invention may use an existing image recognition algorithm to perform image recognition, such as a spot detection algorithm, a corner detection algorithm, and the like.
- a corresponding appropriate algorithm can be selected according to the form of the marker. For example, when the marker is a two-dimensional code set on an operating table or a surgical instrument, a corresponding two-dimensional code identification algorithm can be directly used.
- the relative coordinate relationship between the identifiable part and the marked part of the registration device can be obtained by calculation from the associated structural information in the database.
- the spatial coordinates of the marked site can be obtained by registering the optical features on the device.
- the space coordinates of the identifiable parts and the space coordinates of the marked parts can be converted to the world coordinate system through coordinate transformation.
- the registration device is preferably a device consistent with a surgical instrument, more preferably a medical device that integrates a registration function and a surgical implementation function, so that data errors caused by instrument replacement can be reduced.
- the registration device can be a puncture needle, and the identifiable part is an optical feature set on the puncture needle, such as a two-dimensional code; the marked part can be the needle tip of the puncture needle. In this way, the registration device satisfies one of two conditions: re-sterilization or single-use aseptic use.
- Step S604 Determine whether the position of the registration device meets the preset registration condition, and when the position of the registration device meets the registration condition, according to the anatomical structure model of the operation object and the live image, The anatomical structure model is positionally registered with the surgical site.
- step S604 the preset registration conditions are:
- the preset motion parameter may be a parameter value, or may be a certain value range.
- the position registration device meets the registration conditions, and the position registration operation can be performed.
- the registration operation is not performed.
- the position of the registration device conforms to the (1) case it can be considered that the registration device has been in a stable state, and the registration can be started;
- the position of the registration device conforms to the (2) case it can be Considering that the identified site of the registration device is in a stable state with respect to the body surface of the subject, the registration can be started. In this way, the recording of a specific spatial point can be completed under the condition that the designated collection position moves synchronously with the physiological movement of the surgical object. That is to say, the spatial position registration system of the present invention can eliminate the influence caused by the specific movement of the body surface of the operation object.
- the preset motion parameters are motion parameters corresponding to body motion caused by human respiration or heartbeat.
- the motion parameter includes at least one of motion frequency, motion amplitude, motion speed, and motion trajectory.
- the motion parameters can be provided in advance or collected on site. The on-site collection can be carried out in the existing manner in the prior art.
- step S704 the following calculation steps are also included:
- the moving speed of the marking part of the registration device at each moment in the time period is calculated.
- the coordinate position may be characterized in the form of world coordinates or space coordinates.
- the specific process of the calculation step is as follows:
- V 2i B i B i-1 / ⁇ t ⁇ Formula 3>
- D 1i is the distance from any point of the identifiable part of the registration device to its geometric mean position within the ⁇ t time period;
- a i is the position coordinate of the identifiable part of the registration device in the ⁇ t time period, and the position coordinate of A i is represented by (X 1i , Y 1i , Z 1i );
- O 1 is the geometric mean coordinate of the position coordinates of the identifiable part of the registration device in the time period ⁇ t, and the position coordinates of O 1 are To represent;
- D 2i is the distance between any point of the marking part of the registration device relative to its geometric mean position in the ⁇ t time period
- B i is the position coordinate of the marking part of the registration device in the ⁇ t time period, and the position coordinate of B i is represented by (X 2i , Y 2i , Z 2i );
- O 2 is the geometric mean coordinate of the position coordinates of the marking part of the registration device in the ⁇ t time period, and the position coordinates of O 2 are To represent;
- V 2i is the movement speed of the marking part of the registration device in the time period ⁇ t;
- the coordinate positions are characterized in world or space coordinates.
- the specific calculation process is as follows:
- V 2i B i B i-1 / ⁇ t ⁇ Formula 3>;
- the coordinate positions are characterized in world or space coordinates.
- step S604 when all D 1i are less than or equal to the first preset threshold, it is determined that the identifiable part of the registration device is in a stable state.
- the first preset value is 1-3 mm; more preferably, the first preset value is 2 mm.
- the second preset value is 1-3mm; more preferably, the second preset value is 2mm.
- the third preset value is 1-3 mm/s; more preferably, the third preset value is 2 mm/s.
- the system starts to perform the registration operation when all D 1i are smaller than 2 mm, and when all D 2i are smaller than 2 mm.
- the motion parameter is motion amplitude
- the horizontal axis represents time
- the vertical axis represents the movement range of the spatial position of the marking part of the registration device. It can be seen from the figure that at time T1, the spatial position of the marked part reaches the highest position; at time T2, the spatial position of the marked part reaches the lowest position, and at time T3, the spatial position of the marked part reaches the highest position again.
- the collected motion amplitude curve is compared with the motion amplitude curve of the preset position point collected in real time or pre-stored with respiration, and if the two match, it is considered that in the time period from T1 to T3, the The marked site of the quasi-device is in a stable state with respect to the body surface, and the registration operation can be performed.
- the present invention may further include the following recording step: obtaining the judgment result of the specified position point, and when the judgment result is that the position of the registration device meets the registration conditions, recording the coordinates of the current specified position point information. If there is movement during recording that exceeds either threshold, the recording is stopped.
- the number of the designated positions is 2 or more.
- the recording module in the present invention can be used to continuously record the coordinate information of multiple designated position points within a period of time.
- the average coordinates of the marked part of the registration device at each moment are recorded.
- D 1i of the identifiable part of the registration device is less than a first preset threshold, such as 2 mm
- D 2i of the marked part is less than a second preset threshold, such as 2 mm
- the position of the identifiable part of the registration device is in a stable state and the motion parameters of the marked part of the registration device match the preset motion parameters
- recording the The average value of the coordinates of the identifiable part at a set time where the set time is the time when the motion parameter is the smallest in the motion parameter cycle.
- a first preset threshold such as 2 mm
- the motion parameters of the marked part of the registration device match the motion parameters of the human heartbeat
- the average value of the coordinates at a fixed time, and the set time is the time when the heartbeat amplitude is the smallest in the human heartbeat cycle.
- the recording process is specifically as follows: in acquiring n motion parameter periods, the registration device marks the part B (X 2 , Y 2 , Z 2 ) at the moment of the minimum motion parameter value in each motion period The corresponding position coordinates B' 1 (X 21 , Y 21 , Z 21 )...B' n (X 2n , Y 2n , Z 2n ), calculate the geometric mean position coordinates, denoted as record stated If there is any movement beyond any threshold during the recording, the recording will be stopped and the judgment phase will be restarted.
- the present invention further includes a prompt step S605, which is used to guide the operator to perform a specific operation.
- the prompting module of the present invention is used to display prompting information to prompt the operator to perform a specified behavior to meet the preset registration conditions; the prompting information specifically includes: instructing the operator to register The device moves to a specified position, or guides the operator to move itself relative to the object of operation according to a specified positional relationship, or instructs the operator to control the registration device to maintain a specified state for a specified time, or guide the operation It keeps itself in the specified positional relationship with respect to the cast object.
- the prompting step further includes a determining step and a displaying step.
- the determining step is used to determine the specific stage of registration; the displaying step is used to display the navigation stage and specific guidance information, and different registration stages display different guidance information.
- the display step at least instructs the operator to move the registration device to a specified position, or instructs the operator to make himself relative to the operation object in a specified positional relationship information of movement; when the specific stage of registration is the position recording stage, the display step at least instructs the operator to keep the registration device in a specified state, or guides the operator to maintain a specified positional relationship with respect to the surgical object Information.
- the interactive registration system for surgical navigation of the present invention can realize the function of differentiating reminders according to the specific stages of registration, thereby improving the efficiency and accuracy of the registration process.
- the instructing the operator to display a dynamic pattern representing the progress of time while directing the registration device to remain in a specified state for a specified period of time For example, hourglass, progress bar, circle, etc.
- step S605 the guiding the operator to perform a specific operation includes:
- the prompt form includes one or more of characters, arrows, colors, patterns, and animations. More preferably, when the specific stage of registration is the position registration stage, the prompt form is arrows or text; when the specific stage of registration is the position recording stage, the prompt form is animation or text.
- the prompting manner of the present invention can be set by the operator. The advantage of this is that it can deepen the operator's memory and enhance the degree of personalization of the system.
- the display module when the specific stage of registration is the position registration stage, the display module displays the text "Currently is the registration stage, please move the registration device to the specified position", and is accompanied by a direction arrow; when the specific stage of registration is the position recording stage, the display module will display the text "Have entered the recording stage, please keep the current action", accompanied by animation of circles, hourglasses, or progress bars.
- selecting a specific color indicates that the current stage is the registration stage, and the text "Please move the registration device to the specified position" is displayed in the display module. , and accompanied by a direction arrow;
- select a specific color to indicate that the current is the registration stage, and the text "Please keep the current action” is displayed in the display module, accompanied by circles and hourglasses. , or the animation of the progress bar.
- step S605 if the registration device fails to maintain the specified state within the specified time, the operator is prompted to maintain the specified state again and restarts the calculation of the specified time. This is to prevent errors in the registration data caused by abnormal jitter during the registration process. For example, the operator may sneeze, cough, or perform other abnormal movements within the specified time when the system prompts "please maintain the current action", and the subject may also experience abnormal movements such as body twitches and sudden changes in position. If the quasi-equipment fails to maintain the specified state for the specified time, it is necessary to interrupt the ongoing registration process and start a new round of position calculation and registration.
- the purpose of maintaining the designated state is to complete the registration process.
- the system is performing a position registration process when the system displays prompt information instructing the operator to direct the registration device to maintain a specified state for a specified time.
- the system prompt module displays prompt information instructing the operator to direct the registration device to maintain a specified state for a specified time
- the system is performing a recording process.
- a judgment method similar to that of the recording process can be adopted. For example, during the registration process, if the registration device moves beyond any threshold, the current position registration and time calculation will be stopped, and then a new registration and time calculation will be restarted. Specifically, when there is an abnormal action caused by the operator's coughing during the registration process, it is determined whether the identifiable part and the marked part of the registration device have moved beyond any threshold, and if the determination result is yes, then stop the current Position registration and time calculation, if there is a recording process, stop together, and re-execute the judgment of calculating the position coordinates of the registration device and the registration conditions.
- an electronic device 700 includes a memory 701 and a processor 702.
- the memory 701 is used for storing computer instructions, and the computer instructions are executed by the processor 702 to implement any information interaction method in the embodiments of the present application.
- the present application also provides a readable storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, implement any of the information interaction methods in the embodiments of the present application.
- FIG. 8 is a schematic structural diagram of a computer system suitable for implementing the method of an embodiment of the present application.
- the computer system includes a processing unit 801, which can execute the programs shown in the above figures according to a program stored in a read only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803 Various processing in the embodiment of .
- RAM 803 various programs and data necessary for system operation are also stored.
- the processing unit 801 , the ROM 802 and the RAM 803 are connected to each other through a bus 804 .
- An input/output (I/O) interface 805 is also connected to bus 804 .
- the following components are connected to the I/O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc. ; and a communication section 809 including a network interface card such as a LAN card, a modem, and the like.
- the communication section 809 performs communication processing via a network such as the Internet.
- a drive 8010 is also connected to the I/O interface 805 as needed.
- a removable medium 8011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 8010 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.
- the processing unit 801 may be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, and an NPU.
- the method described above may be implemented as a computer software program.
- embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a readable medium thereof, the computer program comprising program code for performing the methods of the Figures.
- the computer program may be downloaded and installed from the network through the communication section 809, and/or installed from the removable medium 8011.
- first and second are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with “first”, “second” may expressly or implicitly include at least one of that feature.
- plurality means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Robotics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Analysis (AREA)
Abstract
一种用于手术导航的交互配准系统，包括：医学影像获取模块、现场图像获取模块、识别模块、判断模块；其中，医学影像获取模块，用于获取施术对象的术前医学影像；现场图像获取模块，用于获取手术现场的现场图像；识别模块，用于根据现场图像识别手术现场中配准设备的位置配准设备至少包括可识别部位和标记部位；判断模块，用于判断配准设备的位置是否符合预设的配准条件；当配准设备的位置符合预设的配准条件时，根据施术对象的解剖结构模型以及现场图像，将解剖结构模型与手术现场进行位置配准。可以提高空间位置配准的精确性和效率。
Description
本申请涉及医疗领域，尤其涉及一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质。
手术导航系统，是将病人术前或术中影像数据和手术床上病人解剖结构准确对应，手术中跟踪手术器械并将手术器械的位置在病人影像上以虚拟探针的形式实时更新显示，使医生对手术器械相对病人解剖结构的位置一目了然，使外科手术更快速、更精确、更安全。
利用增强现实进行手术导航，首先需要提前将手术计划信息预先存储在计算机中；其次，还需要将术前影像扫描获得的医学影像数据，与术中施术对象实时拍摄的现场图像通过坐标变换的方式实现数据配准。现有技术通常通过多次捕捉到的现场画面来提取点云，进行现场场景点云与计算机模型的配准。数据配准的精确度越高，手术操作者获得的导航信息越精准。但是，在现场拍摄现场画面过程中，操作者或操作对象难免会产生移动、抖动、呼吸、心跳等各种情况，这些情况给配准过程带来难度增大，精度下降的问题。
发明内容
为了解决上述技术问题中的至少一个，本申请提供了用于手术导航的交互配准系统、方法、电子设备和可读存储介质。
本申请的第一方面，提供一种用于手术导航的交互配准系统，其特征在于，该装置包括：
医学影像获取模块、现场图像获取模块、识别模块、判断模块和执行模块；其中，
所述医学影像获取模块，用于获取施术对象的术前医学影像，并 根据所述术前医学影像获得所述施术对象的解剖结构模型；
所述现场图像获取模块，用于获取手术现场的现场图像；
所述识别模块，用于根据所述现场图像识别手术现场中配准设备的位置；
所述判断模块，用于根据所述配准设备的位置状态判断当前是否符合预设的配准条件；当符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
所述配准设备至少包括可识别部位和标记部位；所述预设配准条件为：
当所述配准设备的可识别部位和标记部位处于稳定状态；
和/或，
当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配。
可选的，所述预设运动参数为与人体呼吸或心跳导致的身体运动相应的运动参数。可选的，所述运动参数包括运动频率、运动幅度、运动速度和运动轨迹中的至少一种。
可选的，所述判断模块还包括计算模块，所述计算模块用于进行以下计算：
获取一段时间内所述配准的所述可识别部位和所述标记部位各时刻的坐标，并
计算该时间段内，所述配准设备的可识别部位的平均坐标；
计算该时间段内，所述配准设备的标记部位的平均坐标；
计算该时间段内，所述配准设备的所述可识别部位各时刻的坐标相对所述可识别部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的坐标相对其所述标记部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的移动速度；
可选的，当所述配准设备的可识别部位任意一点相对其几何平均 位置的距离均小于或等于第一预设阈值时，判定所述配准设备的可识别部位处于稳定状态。
当所述配准设备的标记部位任意一点相对其几何平均位置的距离；均小于或等于第二预设阈值时，判定所述配准设备的标记部位处于稳定状态。
当所述配准设备的标记部位在该时间段内，出现最大运动速度时刻前后，均出现了小于或等于第三预设值的速度时，判定所述配准设备的标记部位的运动参数和预设运动参数相匹配。
可选的，所述判断模块还包括记录模块，所述记录模块具体用于：
当所述配准设备的可识别部位和标记部位处于稳定状态时，记录各时刻所述配准设备的标记部位的平均坐标；
当所述配准设备的可识别部位的位置处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配时，记录所述配准设备标记部位在设定时刻的坐标的平均值，所述设定时刻为运动参数周期中运动参数最小的时刻。
可选的，还包括提示模块，用于引导操作者进行特定的操作。
可选的，所述提示模块用于显示提示信息提示所述操作者做出指定行为以符合所述预设的配准条件；所述提示信息具体包括：
指导所述操作者将配准设备移动至指定位置，或
引导所述操作者使其自身相对施术对象按照指定位置关系移动，或
指导所述所述操作者支配所述配准设备在指定时间内保持指定状态，或
引导所述操作者使其自身相对施术对象保持指定位置关系。
可选的，所述指导所述操作者支配所述配准设备在指定时间内保持指定状态时，显示用于表征时间进度的动态图案。
可选的，若所述配准设备在指定时间内保持指定状态失败，所述提示模块提示所述操作者重新保持指定状态并重新开始计算指定时间。
本申请的第二方面，一种用于手术导航的交互配准方法，包括：
获取施术对象的术前医学影像，并根据所述术前医学影像获得所述施术对象的解剖结构模型；
获取手术现场的现场图像；
根据所述现场图像识别手术现场中配准设备的位置；
根据所述配准设备的位置状态判断当前是否符合预设的配准条件，当所述配准设备的位置符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
可选的，所述配准设备至少包括可识别部位和标记部位；所述配准条件为：
当所述配准设备的可识别部位和标记部位处于稳定状态；
和/或，
当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配。
可选的，还包括以下计算步骤：
获取一段时间内所述配准设备的所述可识别部位和所述标记部位在各时刻的坐标，并
计算该时间段内，所述配准设备的可识别部位的平均坐标；
计算该时间段内，所述配准设备的标记部位的平均坐标；
计算该时间段内，所述配准设备的所述可识别部位的各时刻的坐标相对所述可识别部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的坐标相对所述标记部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的移动速度。
可选的，当所述配准设备的可识别部位任意一点相对其几何平均位置的距离均小于或等于第一预设阈值时，判定所述配准设备的可识别部位处于稳定状态。
当所述配准设备的标记部位任意一点相对其几何平均位置的距离；均小于或等于第二预设阈值时，判定所述配准设备的标记部位处于稳定状态。
当所述配准设备的标记部位在该时间段内，出现最大运动速度时刻前后，均出现了小于或等于第三预设值的速度时，判定所述配准设备的标记部位的运动参数和预设运动参数相匹配。
可选的，本发明还包括以下记录步骤：
当所述配准设备的可识别部位和标记部位处于稳定状态时，记录所述配准设备的标记部位的所有位置的几何平均坐标；
当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配时，记录所述可识别部位在设定时刻的坐标的平均值，所述设定时刻为运动参数周期中运动参数最小的时刻。
可选的，本发明还包括提示步骤，用于引导操作者进行特定的操作。
可选的，本发明的提示步骤用于显示提示信息提示所述操作者做出指定行为以符合所述预设的配准条件；所述提示信息具体包括：
指导所述操作者将配准设备移动至指定位置，或
引导所述操作者使其自身相对施术对象按照指定位置关系移动，或
指导所述操作者支配所述配准设备在指定时间内保持指定状态，或
引导所述操作者使其自身相对施术对象保持指定位置关系。
可选的，所述指导所述操作者支配所述配准设备在指定时间内保持指定状态时，显示用于表征时间进度的动态图案。
可选的，若所述配准设备在指定时间内保持指定状态失败，所述提示模块提示所述操作者重新保持指定状态并重新开始计算指定时间。
本申请的第三方面，一种电子设备，其包括存储器和处理器；其中， 所述存储器用于存储可执行程序代码；所述计算机指令被所述处理器执行以实现本申请第二方面任一项所述的方法。
本申请的第四方面，一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，该计算机指令被处理器执行时实现本申请第二方面任一项所述的方法。
通过实施本申请的技术方案可以取得以下有益技术效果：
本发明实施例中的技术方案，可以自动识别手术场景图像中所包含的配准设备，根据所述施术对象的解剖结构模型以及所述现场图像，将解剖结构模型与手术现场的相应解剖结构进行位置配准。
本发明使得操作者在全程无需其他操作或指令，如操作键盘、鼠标、语音口令、手势等情况下，全程自动完成一个或多个空间点的配准。
本发明还可在空间点与施术对象生理运动(如呼吸、心跳、抖动等)同步运动的情况下，完成特定空间点的配准。
此外，本发明还可以通过提供全程增强现实提示信息，帮助操作者更快速、更精准地完成配准过程。相对于现有技术，实施本发明的技术方案可以提高空间位置配准的精确性和效率。
附图示出了本公开的示例性实施方式，并与其说明一起用于解释本公开的原理，其中包括了这些附图以提供对本公开的进一步理解，并且附图包括在本说明书中并构成本说明书的一部分。
图1是本申请实施例公开的用于手术导航的交互配准系统的结构框图；
图2是本申请实施例公开的另一种用于手术导航的交互配准系统的结构框图；
图3是本申请实施例公开的一种手术场景示意图；
图4是本申请实施例公开配准设备标记部位运动参数图；
图5是本申请实施例公开的另一配准设备标记部位运动参数图；
图6是本申请实施例公开的用于手术导航的交互配准方法的流程 图；
图7是本申请实施例公开的另一种电子设备的结构框图；
图8是本申请实施例公开的一种计算机系统的结构示意图。
下面结合附图和实施方式对本公开作进一步的详细说明。可以理解的是，此处所描述的具体实施方式仅用于解释相关内容，而非对本公开的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本公开相关的部分。
需要说明的是，在不冲突的情况下，本公开中的实施方式及实施方式中的特征可以相互组合。虽然每个实施方式代表了发明的单一组合，但是本发明不同实施例可以替换，或者合并组合，因此本发明也可认为包含所记载的相同和/或不同实施例的所有可能组合。因而，如果一个实施例包含A、B、C，另一个实施例包含B和D的组合，那么本发明也应视为包括含有A、B、C、D的一个或多个所有其他可能的组合的实施例，尽管该实施例可能并未在以下内容中有明确的文字记载。
下面将参考附图并结合实施方式来详细说明本公开。
参见图1，一种用于手术导航的交互配准系统,包括：医学影像获取模块、现场图像获取模块、识别模块、计算模块和执行模块；其中，
所述医学影像获取模块，用于获取施术对象的术前医学影像，并根据所述术前医学影像获得所述施术对象的解剖结构模型；
所述现场图像获取模块，用于获取手术现场的现场图像；
所述识别模块，用于根据所述现场图像识别手术现场中配准设备的位置，所述配准设备至少包括可识别部位和标记部位；
所述判断模块，用于根据所述配准设备的位置状态判断当前是否符合预设的配准条件；当符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
本发明实施例中的技术方案，可以自动识别手术场景图像中所包含的配准设备，根据施术对象的解剖结构模型以及现场图像，将解剖结构模型与手术现场的相应解剖结构进行位置配准。本发明可以使得操作者在全程无需其他操作或指令，如操作键盘、鼠标、语音口令、手势等情况下，全程自动完成一个或多个空间点的配准。
其中，可以知道的，操作者可以通过佩戴头戴式设备的摄像头来拍摄手术场景以采集手术场景图像，参见图3，图3中是操作者1根据通过佩戴头戴式设备2的摄像头拍摄到方框区域3的手术场景图像。
本发明的识别模块，用于根据所述现场图像识别手术现场中配准设备的位置。
作为识别模块的可选实施方式，所述配准设备至少包括可识别部位和标记部位。可识别部位具有特定的光学特征、图案特征和几何特征中的至少一种，以使得拍摄标识物所得到的图像具有特定的图像特征，例如，所述图案信息为二维码。所述标记部位可用于触碰/点击指定位置，所述指定位置可以为施术对象身体上预先确定好的点，这些点的空间坐标已提前配置在交互配准系统中。优选地，所述指定位置的数量为2个或多个。
作为识别模块的可选实施方式，本发明实施例中的识别模块可以利用现有的图像识别算法进行图像识别，例如斑点检测算法、角点检测算法等。具体的，可根据标识物的形式选择相应合适的算法，例如标识物是设置在手术台或手术器械上的二维码时，可直接采用相应的二维码识别算法。
所述配准设备的可识别部位和标记部位的相对坐标关系可以通过数据库中相关联的结构信息计算获得。如此，所述标记部位的空间坐标可以通过配准设备上的光学特征来获得。可识别部位的空间坐标及标记部位的空间坐标可以通过坐标变换转换至世界坐标系。
本申请实施例中的配准设备优先选用与手术器械一致的设备，更优选为集配准功能与手术实施功能一体的医疗器械，这样可以减少因器械更换带来的数据误差。例如，配准设备可以为穿刺针，可识别部位为设置在穿刺针上的光学特征，例如：二维码；标记部位可以为穿 刺针的针尖。如此，配准设备即可满足重复灭菌或一次性无菌使用这两个条件之一。
本发明实施例中的判断模块，用于判断所述配准设备的位置是否符合预设的配准条件。当判断所述配准设备的位置是否符合所述的预设配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
作为判断模块的可选实施方式，所述预设配准条件为：当所述配准设备的可识别部位和标记部位处于稳定状态；和/或，当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配。所述预设运动参数可以是参数数值，也可以是一定的数值范围。
也就是说，判断所述配准设备的位置是否符合配准条件时，可以通过以下方式来确定：
(1)当所述配准设备的可识别部位和标记部位处于稳定状态；
(2)当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配；
(3)当(1)和(2)同时满足。
当出现上述三种情况之一时，即认为配准设备的位置符合配准条件，可以执行位置配准操作。当配准设备的位置不符合上述配准条件时，则不执行配准操作。具体地，当配准设备地位置符合第(1)种情况时，可以认为配准设备已经处于稳定状态，可以开始执行配准；当配准设备的位置符合第(2)种情况时，可以认为配准设备的识别部位相对于对象体表处于稳定状态，可以开始执行配准。这样可以使得在指定采集位置与施术对象生理运动同步运动的情况下，完成特定空间点的记录。也就是说，本发明的空间位置配准系统可以消除操作对象体表特定运动所带来的影响。
作为判断模块的可选实施方式，所述预设运动参数为与人体呼吸或心跳导致的身体运动相应的运动参数。作为判断模块的可选实施方式，所述运动参数包括运动频率、运动幅度、运动速度和运动轨迹中的至少一种。例如，施术对象身体随呼吸、移动、抖动、心跳等引起 的指定采集位置的运动幅度、运动速度，和/或运动轨迹等。所述运动参数可提前提供，也可现场采集。现场采集可采用现有技术中已有的方式进行。
作为判断模块的可选实施方式，所述判断模块还包括计算模块，所述计算模块用于进行以下计算：
获取一段时间内所述配准设备的所述可识别部位和所述标记部位在各时刻的坐标，并
计算该时间段内，所述配准设备的可识别部位的平均坐标；
计算该时间段内，所述配准设备的标记部位的平均坐标；
计算该时间段内，所述配准设备的所述可识别部位的各时刻的坐标相对所述可识别部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的坐标相对所述标记部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的移动速度。
根据本发明的一个实施方式，所述计算模块具体执行以下计算：
D
1i＝A
iO
1 <公式1>
D
2i＝B
iO
2 <公式2>
V
2i＝B
iB
i-1/Δt <公式3>
公式1中，
D
1i为Δt时间段内，所述配准设备的可识别部位任意一点相对其几何平均位置的距离；
A
i为Δt时间段内，所述配准设备的可识别部位的位置坐标，A
i的位置坐标以(X
1i,Y
1i,Z
1i)来表示；
O
1为Δt时间段内，所述配准设备的可识别部位的位置坐标的几何平均坐标，O
1的位置坐标以
来表示；
类似的，公式2中，
D
2i为Δt时间段内，所述配准设备的标记部位任意一点相对其几何平均位置的距离；
B
i为Δt时间段内，所述配准设备的标记部位的位置坐标，B
i的位 置坐标以(X
2i,Y
2i,Z
2i)来表示；
O
2为Δt时间段内，所述配准设备的标记部位的位置坐标的几何平均坐标，O
2的位置坐标以
来表示；
公式3中，
V
2i为Δt时间段内，所述配准设备的标记部位的移动速度；
所述坐标位置可以以世界坐标或空间坐标予以表征。
根据本发明的又一个实施方式，所述计算过程如下：
获取Δt时间段内所述配准设备可识别部位A(X
1,Y
1,Z
1)所有位置坐标，记为A
1(X
11,Y
11,Z
11)…A
i(X
1i,Y
1i,Z
1i)，计算几何平均坐标，记为
获取同一段时间内所述配准设备标记部位B(X
2,Y
2,Z
2)的全部位置坐标，记为B
1(X
21,Y
21,Z
21)…B
i(X
2i,Y
2i,Z
2i)，计算几何平均坐标，记为
计算Δt时间段内，所述配准设备的可识别部位任意一点相对其几何平均位置的距离，计算公式如下：
D
1i＝A
iO
1 <公式1>，
计算同样时间段内，所述配准设备的标记部位任意一点相对其几何平均位置的距离，计算公式如下：
D
2i＝B
iO
2 <公式2>；
计算同样时间段内所述配准设备的标记部位的移动速度，计算公式如下：
V
2i＝B
iB
i-1/Δt <公式3>；
其中，所述坐标位置以世界坐标或空间坐标予以表征。
作为判断模块的可选实施方式，当所有D
1i均小于或等于第一预设阈值时，判定所述配准设备的可识别部位处于稳定状态。优选的，所述第一预设值为1-3mm；更优选地，所述第一预设值为2mm。
当所有D
2i均小于或等于第二预设阈值时，判定所述配准设备的标记部位处于稳定状态。优选的，所述第二预设值为1-3mm；更优选地，所述第二预设值为2mm。
当所述配准设备的标记部位B在Δt时间段内，出现最大运动速 度时刻前的后，均出现了小于或等于第三预设值的速度时，判定所述配准设备的标记部位的运动参数和预设运动参数相匹配。优选的，所述第三预设值为1-3mm/s；更优选地，所述第三预设值为2mm/s。
根据本发明的一个实施例，当所有D
1i均小于2mm，且当所有D
2i均小于2mm时，系统开始执行配准操作。
根据本发明的另一个实施例，当所有D
2i均小于2mm，当所述配准设备的标记部位B在Δt时间段内，出现最大运动速度时刻的前后，均出现了小于2mm/s的速度时，系统开始执行配准操作。
若Δt时间段内，出现任一D
2i大于第二预设阈值，或出现最大运动速度时刻的前后，没有同时出现小于第三预设值的速度，则系统不执行配准操作。
当运动参数为运动幅度时，参见图4。图4中，横轴表示时间，纵轴表示配准设备标记部位空间位置的运动幅度。从图中可以看出，在T1时刻，标记部位的空间位置到达了最高位置；在T2时刻，标记部位的空间位置到达了最低位置；在T3时刻，标记部位的空间位置再次到达了最高位置。此时，将采集的该运动幅度曲线与实时采集的或预先存储的预设位置点随呼吸而出现的运动幅度曲线相比较，如果两者相匹配，则认为在T1至T3时间段内，配准设备的标记部位相对于体表处于稳定状态，可以执行配准操作。
运动参数为运动速度时，参见图5。从图中可以看到，在T1、T2时刻，标记部位运动速度趋近于零，在T4时刻附近，运动速度达到了最大值。也就是说，达到运动最大值的T4时刻之前和之后，分别出现了运动速度趋近于零的时刻，此时即可判定配准设备的标记部位相对于体表处于稳定状态，可以执行配准操作。
进一步地，本发明还可以包括记录模块，所述记录模块用于获取指定位置点的判断结果，当判断结果为所述配准设备的位置符合配准条件时，记录当前指定位置点的坐标信息。若记录过程中出现了超出任一阈值的移动，则停止记录。优选地，所述指定位置的数量为2个或多个。具体地，本发明中的记录模块可以用于连续记录一段时间内的多个指定位置点的坐标信息。
具体地，当所述配准设备的可识别部位和标记部位的位置处于稳定状态时，记录各时刻所述配准设备的标记部位的平均坐标。根据本发明的一个实施例，当配准设备的可识别部位的D
1i小于第一预设阈值，如2mm，并且标记部位的D
2i小于第二预设阈值，如2mm时，此时可以认为配准设备已处于稳定状态，记录此时配准设备标记部位的几何平均坐标
根据本发明的另一种实施方式，当所述配准设备的可识别部位的位置处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配时，记录所述可识别部位在设定时刻的坐标的平均值，所述设定时刻为运动参数周期中运动参数最小的时刻。例如，当配准设备的可识别部位的D
1i小于第一预设阈值，如2mm，且所述配准设备的标记部位的运动参数和人体心跳运动参数相匹配时，记录可识别部位在设定时刻的坐标的平均值，设定时刻为人体心跳周期中心跳幅度最小的时刻。进一步地，心跳周期的数量可以为2个或2个以上。根据本发明的一个实施例，记录过程具体如下：获取n个运动参数周期中，所述配准设备标记部位B(X
2,Y
2,Z
2)在每个运动周期中运动参数最小值时刻所对应的位置坐标B’
1(X
21,Y
21,Z
21)…B’
n(X
2n,Y
2n,Z
2n)，计算其几何平均位置坐标，记为
记录所述
若记录过程中出现了超出任一阈值的移动，则停止记录，重新开始判断阶段。
可选地，本发明实施例中所述的一种用于手术导航的交互配准系统,还包括提示模块，用于引导操作者进行特定的操作。
可选地，本发明的提示模块，用于显示提示信息提示所述操作者做出指定行为以符合所述预设的配准条件；所述提示信息具体包括：指导所述操作者将配准设备移动至指定位置，或引导所述操作者使其自身相对施术对象按照指定位置关系移动，或指导所述操作者支配所述配准设备在指定时间内保持指定状态，或引导所述操作者使其自身相对施术对象保持指定位置关系。
作为提示模块的可选实施方式，提示模块还包括确定模块和显示模块。确定模块用于确定配准的具体阶段；显示模块用于显示导航阶段和具体的引导信息，不同的配准阶段显示不同的引导信息。例如， 当配准的具体阶段为位置配准阶段时，显示模块至少显示指导所述操作者将配准设备移动至指定位置，或引导所述操作者使其自身相对施术对象按照指定位置关系移动的信息；当配准的具体阶段为位置记录阶段时，显示模块至少显示指导所述操作者将配准设备保持指定状态，或引导所述操作者使其自身相对施术对象保持指定位置关系的信息。如此，本发明所述用于手术导航的交互配准系统，可以实现根据配准的具体阶段的不同而进行差异化提醒的功能，进而提高配准过程的效率和准确性。
作为提示模块的可选实施方式，所述指导所述操作者支配所述配准设备在指定时间内保持指定状态时，显示用于表征时间进度的动态图案。例如，沙漏、进度条、画圈等。
作为提示模块的可选实施方式，所述引导所述操作者进行特定操作的提示通过文字、箭头、语音、颜色、图案、动画、节奏音中的一种或多种形式予以实现。优选的，提示形式包括文字、箭头、颜色、图案、动画中的一种或多种。更优选地，当配准的具体阶段为位置配准阶段时，提示形式为箭头或文字；当配准的具体阶段为位置记录阶段时，提示形式为动画或文字。优选地，本发明的提示方式可以由操作者自行设定。这样做的好处是可以加深操作者的记忆，增强系统个性化的程度。
根据本发明的一个实施例，当配准的具体阶段为位置配准阶段时，显示模块中显示文字“当前为配准阶段，请移动配准设备至指定位置”，并配以方向箭头；当配准的具体阶段为位置记录阶段时，显示模块中显示文字“已进入记录阶段，请保持目前动作”，并配以画圈、沙漏，或进度条的动画。
根据本发明的另一个实施例，当配准的具体阶段为位置配准阶段时，选取特定的颜色表示当前为配准阶段，并在显示模块中显示文字“请移动配准设备至指定位置”，并配以方向箭头；当配准的具体阶段为位置记录阶段时，选取特定的颜色表示当前为配准阶段，在显示模块中显示文字“请保持目前动作”，并配以画圈、沙漏，或进度条的动画。
作为提示模块的可选实施方式，若所述配准设备在指定时间内保持指定状态失败，所述提示模块提示所述操作者重新保持指定状态并重新开始计算指定时间。这样是为了防止配准过程中出现异常抖动而引起配准数据出现误差。例如，操作者有可能在系统提示“请保持目前动作”的指定时间内打喷嚏、咳嗽或其他异常动作，施术对象也可能出现身体抽动、位置突然变化等异常移动，这些情况都可能导致配准设备在指定时间内保持指定状态失败，此时需要中断正在进行的配准过程，开始新一轮的位置计算和配准。
本发明中，所述保持指定状态是为了完成配准过程。根据本发明的一个实施例，当系统提示模块显示指导所述操作者支配所述配准设备在指定时间内保持指定状态的提示信息时，系统正在执行位置配准过程。根据本发明的又一个实施例，当系统提示模块显示指导所述操作者支配所述配准设备在指定时间内保持指定状态的提示信息时，系统正在执行记录过程。
对于配准设备在指定时间内保持指定状态失败的判断可采取与记录过程类似的判断方式。例如，在配准过程中，配准设备若出现超出任一阈值的移动，则停止当前位置配准和时间计算，然后重新开始新的配准和时间计算。具体地，当配准过程中出现因操作者咳嗽而引起的异常动作时，判断配准设备的可识别部位和标记部位是否出现了超出任一阈值的移动，若判断结果为是，则停止当前位置配准和时间计算，如有记录过程则一并停止，重新执行计算配准设备的位置坐标和配准条件的判断。
图6表示本发明的第二方面，一种用于手术导航的交互配准方法，包括：
S601:获取施术对象的术前医学影像，并根据所述术前医学影像获得所述施术对象的解剖结构模型；
S602:获取手术现场的现场图像；
S603:根据所述现场图像识别手术现场中配准设备的位置；
S604:根据所述配准设备的位置状态判断当前是否符合预设的配 准条件；当符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
本发明实施例中的技术方案，可以自动识别手术场景图像中所包含的配准设备，根据施术对象的解剖结构模型以及现场图像，将解剖结构模型与手术现场的相应解剖结构进行位置配准。本发明可以使得操作者在全程无需其他操作或指令，如操作键盘、鼠标、语音口令、手势等情况下，全程自动完成一个或多个空间点的配准。
操作者可以通过佩戴头戴式设备的摄像头来拍摄手术场景以采集手术场景图像，参见图3，图3中是操作者1根据通过佩戴头戴式设备2的摄像头拍摄到方框区域3的手术场景图像。
本发明步骤S603识别步骤根据所述现场图像识别手术现场中配准设备的位置。作为步骤S603的可选实施方式，所述配准设备至少包括可识别部位和标记部位。可识别部位具有特定的光学特征、图案特征和几何特征中的至少一种，以使得拍摄标识物所得到的图像具有特定的图像特征，例如，所述图案信息为二维码。所述标记部位可用于触碰/点击指定位置，所述指定位置可以为施术对象身体上预先确定好的点，这些点的空间坐标已提前配置在交互配准系统中。优选地，所述指定位置的数量为2个或多个。
作为步骤S603的可选实施方式，本发明实施例中的识别模块可以利用现有的图像识别算法进行图像识别，例如斑点检测算法、角点检测算法等。具体的，可根据标识物的形式选择相应合适的算法，例如标识物是设置在手术台或手术器械上的二维码时，可直接采用相应的二维码识别算法。
所述配准设备的可识别部位和标记部位的相对坐标关系可以通过数据库中相关联的结构信息计算获得。如此，可以通过配准设备上的光学特征来获得所述标记部位的空间坐标。可识别部位的空间坐标及标记部位的空间坐标可以通过坐标变换转换至世界坐标系。
作为步骤S603的可选实施方式，配准设备优先选用与手术器械一致的设备，更优选为集配准功能与手术实施功能一体的医疗器械，这 样可以减少因器械更换带来的数据误差。例如，配准设备可以为穿刺针，可识别部位为设置在穿刺针上的光学特征，例如：二维码；标记部位可以为穿刺针的针尖。如此，配准设备即可满足重复灭菌或一次性无菌使用这两个条件之一。
步骤S604：判断所述配准设备的位置是否符合预设的配准条件，当所述配准设备的位置符合配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
作为步骤S604的可选实施方式，所述预设配准条件为：
当所述配准设备的可识别部位和标记部位处于稳定状态；和/或，当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配。所述预设运动参数可以是参数数值，也可以是一定的数值范围。
也就是说，判断所述配准设备的位置是否符合配准条件时，可以通过以下方式来确定：
(1)当所述配准设备的可识别部位和标记部位处于稳定状态；
(2)当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配；
(3)当(1)和(2)同时满足。
当出现上述三种情况之一时，即认为配准设备的位置符合配准条件，可以执行位置配准操作。当配准设备的位置不符合上述配准条件时，则不执行配准操作。具体地，当配准设备地位置符合第(1)种情况时，可以认为配准设备已经处于稳定状态，可以开始执行配准；当配准设备的位置符合第(2)种情况时，可以认为配准设备的识别部位相对于对象体表处于稳定状态，可以开始执行配准。这样可以使得在指定采集位置与施术对象生理运动同步运动的情况下，完成特定空间点的记录。也就是说，本发明的空间位置配准系统可以消除操作对象体表特定运动所带来的影响。
作为步骤S604的可选实施方式，所述预设运动参数为与人体呼吸 或心跳导致的身体运动相应的运动参数。作为步骤S604的可选实施方式，所述运动参数包括运动频率、运动幅度、运动速度、和运动轨迹中的至少一种。例如，施术对象身体随呼吸、移动、抖动、心跳等引起的指定采集位置的运动幅度、运动速度，和/或运动轨迹等。所述运动参数可提前提供，也可现场采集。现场采集可采用现有技术中已有的方式进行。
作为步骤S704的可选实施方式，还包括以下计算步骤：
获取一段时间内所述配准设备的所述可识别部位和所述标记部位在各时刻的坐标，并
计算该时间段内，所述配准设备的可识别部位的平均坐标；
计算该时间段内，所述配准设备的标记部位的平均坐标；
计算该时间段内，所述配准设备的所述可识别部位的各时刻的坐标相对所述可识别部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的坐标相对所述标记部位的平均坐标的距离；
计算该时间段内，所述配准设备的所述标记部位的各时刻的移动速度。
所述坐标位置可以以世界坐标或空间坐标的形式予以表征。
根据本发明的一种实施方式，计算步骤具体过程如下：
D
1i＝A
iO
1 <公式1>
D
2i＝B
iO
2 <公式2>
V
2i＝B
iB
i-1/Δt <公式3>
公式1中，
D
1i为Δt时间段内，所述配准设备的可识别部位任意一点相对其几何平均位置的距离；
A
i为Δt时间段内，所述配准设备的可识别部位的位置坐标，A
i的位置坐标以(X
1i,Y
1i,Z
1i)来表示；
O
1为Δt时间段内，所述配准设备的可识别部位的位置坐标的几何平均坐标，O
1的位置坐标以
来表示；
类似的，公式2中，
D
2i为Δt时间段内，所述配准设备的标记部位任意一点相对其几何平均位置的距离；
B
i为Δt时间段内，所述配准设备的标记部位的位置坐标，B
i的位置坐标以(X
2i,Y
2i,Z
2i)来表示；
O
2为Δt时间段内，所述配准设备的标记部位的位置坐标的几何平均坐标，O
2的位置坐标以
来表示；
公式3中，
V
2i为Δt时间段内，所述配准设备的标记部位的移动速度；
所述坐标位置以世界坐标或空间坐标予以表征。
根据本发明的一个具体实施例，具体计算过程如下：
获取Δt时间段内所述配准设备可识别部位A(X
1,Y
1,Z
1)所有位置坐标，记为A
1(X
11,Y
11,Z
11)…A
i(X
1i,Y
1i,Z
1i)，计算Δt时间段内所有位置坐标几何平均坐标，记为
获取同一段时间内所述配准设备标记部位B(X
2,Y
2,Z
2)的全部位置坐标，记为B
1(X
21,Y
21,Z
21)…B
i(X
2i,Y
2i,Z
2i)，计算Δt时间段内所有位置坐标几何平均坐标，记为
计算Δt时间段内，所述配准设备的可识别部位任意一点相对其几何平均位置的距离，计算公式如下：
D
1i＝A
iO
1 <公式1>，
计算同样时间段内，所述配准设备的标记部位任意一点相对其几何平均位置的距离，计算公式如下：
D
2i＝B
iO
2 <公式2>；
计算同样时间段内所述配准设备的标记部位的移动速度，计算公式如下：
V
2i＝B
iB
i-1/Δt <公式3>；
所述坐标位置以世界坐标或空间坐标予以表征。
作为步骤S604的可选实施方式，当所有D
1i均小于或等于第一预设阈值时，判定所述配准设备的可识别部位处于稳定状态。优选的，所述第一预设值为1-3mm；更优选地，所述第一预设值为2mm。
当所有D
2i均小于或等于第二预设阈值时，判定所述配准设备的标记部位处于稳定状态。优选的，所述第二预设值为1-3mm；更优选地，所述第二预设值为2mm。
当所述配准设备的标记部位B在Δt时间段内，出现最大运动速度时刻前的后，均出现了小于或等于第三预设值的速度时，判定所述配准设备的标记部位的运动参数和预设运动参数相匹配。优选的，所述第三预设值为1-3mm/s；更优选地，所述第三预设值为2mm/s。
根据本发明的一个实施例，当所有D
1i均小于2mm，且当所有D
2i均小于2mm时，系统开始执行配准操作。
根据本发明的另一个实施例，当所有D
2i均小于2mm，当所述配准设备的标记部位B在Δt时间段内，出现最大运动速度时刻的前后，均出现了小于2mm/s的速度时，系统开始执行配准操作。
若Δt时间段内，出现任一D
2i大于第二预设阈值，或出现最大运动速度时刻的前后，没有同时出现小于第三预设值的速度，则系统不执行配准操作。
当运动参数为运动幅度时，参见图4。图4中，横轴表示时间，纵轴表示配准设备标记部位空间位置的运动幅度。从图中可以看出在T1时刻，标记部位的空间位置到达了最高位置；在T2时刻，标记部位的空间位置到达了最低位置，在T3时刻，标记部位的空间位置再次到达了最高位置。此时，将采集的该运动幅度曲线与实时采集的或预先存储的预设位置点随呼吸而出现的运动幅度曲线相比较，如果两者相匹配，则认为在T1至T3时间段内，配准设备的标记部位相对于体表处于稳定状态，可以执行配准操作。
运动参数为运动速度时，参见图5。从图中可以看到，在T1、T2时刻，标记部位运动速度趋近于零，在T4时刻附近，运动速度达到了最大值。也就是说，达到运动最大值的T4时刻之前和之后，分别出现了运动速度趋近于零的时刻，此时即可判定配准设备的标记部位相对于体表处于稳定状态，可以执行配准操作。
作为步骤S604的可选实施方式，本发明还可以包括如下记录步骤：获取指定位置点的判断结果，当判断结果为所述配准设备的位置 符合配准条件时，记录当前指定位置点的坐标信息。若记录过程中出现了超出任一阈值的移动，则停止记录。优选地，所述指定位置的数量为2个或多个。具体地，本发明中的记录模块可以用于连续记录一段时间内的多个指定位置点的坐标信息。
具体地，当所述配准设备的可识别部位和标记部位的位置处于稳定状态时，记录各时刻所述配准设备的标记部位的平均坐标。根据本发明的一个实施例，当配准设备的可识别部位的D
1i小于第一预设阈值，如2mm，并且标记部位的D
2i小于第二预设阈值，如2mm时，此时可以认为配准设备已处于稳定状态，记录此时配准设备标记部位的几何平均坐标
根据本发明的另一种实施方式，当所述配准设备的可识别部位的位置处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配时，记录所述可识别部位在设定时刻的坐标的平均值，所述设定时刻为运动参数周期中运动参数最小的时刻。例如，当配准设备的可识别部位的D
1i小于第一预设阈值，如2mm，且所述配准设备的标记部位的运动参数和人体心跳运动参数相匹配时，记录可识别部位在设定时刻的坐标的平均值，设定时刻为人体心跳周期中心跳幅度最小的时刻。进一步地，心跳周期的数量可以为2个或2个以上。根据本发明的一个实施例，记录过程具体如下：获取n个运动参数周期中，所述配准设备标记部位B(X
2,Y
2,Z
2)在每个运动周期中运动参数最小值时刻所对应的位置坐标B’
1(X
21,Y
21,Z
21)…B’
n(X
2n,Y
2n,Z
2n)，计算其几何平均位置坐标，记为
记录所述
若记录过程中出现了超出任一阈值的移动，则停止记录，重新开始判断阶段。
可选地，本发明还包括S605提示步骤，用于引导操作者进行特定的操作。可选地，本发明的提示模块，用于显示提示信息提示所述操作者做出指定行为以符合所述预设的配准条件；所述提示信息具体包括：指导所述操作者将配准设备移动至指定位置，或引导所述操作者使其自身相对施术对象按照指定位置关系移动，或指导所述操作者支配所述配准设备在指定时间内保持指定状态，或引导所述操作者使其自身相对施术对象保持指定位置关系。
作为步骤S605的可选实施方式，提示步骤还包括确定步骤和显示步骤。确定步骤用于确定配准的具体阶段；显示步骤用于显示导航阶段和具体的引导信息，不同的配准阶段显示不同的引导信息。例如，当配准的具体阶段为位置配准阶段时，显示步骤至少显示指导所述操作者将配准设备移动至指定位置，或引导所述操作者使其自身相对施术对象按照指定位置关系移动的信息；当配准的具体阶段为位置记录阶段时，显示步骤至少显示指导所述操作者将配准设备保持指定状态，或引导所述操作者使其自身相对施术对象保持指定位置关系的信息。如此，本发明所述用于手术导航的交互配准系统，可以实现根据配准的具体阶段的不同而进行差异化提醒的功能，进而提高配准过程的效率和准确性。
所述指导所述操作者支配所述配准设备在指定时间内保持指定状态时，显示用于表征时间进度的动态图案。例如，沙漏、进度条、画圈等。
作为步骤S605的可选实施方式，所述引导所述操作者进行特定的操作包括：
在增强现实设备内使用文字、箭头、语音、颜色、图案、动画、节奏音的一种或多种形式进行提示。优选的，提示形式包括文字、箭头、颜色、图案、动画中的一种或多种。更优选地，当配准的具体阶段为位置配准阶段时，提示形式为箭头或文字；当配准的具体阶段为位置记录阶段时，提示形式为动画或文字。优选地，本发明的提示方式可以由操作者自行设定。这样做的好处是可以加深操作者的记忆，增强系统个性化的程度。
根据本发明的一个实施例，当配准的具体阶段为位置配准阶段时，显示模块中显示文字“当前为配准阶段，请移动配准设备至指定位置”，并配以方向箭头；当配准的具体阶段为位置记录阶段时，显示模块中显示文字“已进入记录阶段，请保持目前动作”，并配以画圈、沙漏，或进度条的动画。
根据本发明的另一个实施例，当配准的具体阶段为位置配准阶段时，选取特定的颜色表示当前为配准阶段，并在显示模块中显示文字 “请移动配准设备至指定位置”，并配以方向箭头；当配准的具体阶段为位置记录阶段时，选取特定的颜色表示当前为配准阶段，在显示模块中显示文字“请保持目前动作”，并配以画圈、沙漏，或进度条的动画。
作为步骤S605的可选实施方式，若所述配准设备在指定时间内保持指定状态失败，则提示所述操作者重新保持指定状态并重新开始计算指定时间。这样是为了防止配准过程中出现异常抖动而引起配准数据出现误差。例如，操作者有可能在系统提示“请保持目前动作”的指定时间内打喷嚏、咳嗽或其他异常动作，施术对象也可能出现身体抽动、位置突然变化等异常移动，这些情况都可能导致配准设备在指定时间内保持指定状态失败，此时需要中断正在进行的配准过程，开始新一轮的位置计算和配准。
本发明中，所述保持指定状态是为了完成配准过程。根据本发明的一个实施例，当系统显示指导所述操作者支配所述配准设备在指定时间内保持指定状态的提示信息时，系统正在执行位置配准过程。根据本发明的又一个实施例，当系统提示模块显示指导所述操作者支配所述配准设备在指定时间内保持指定状态的提示信息时，系统正在执行记录过程。
对于配准设备在指定时间内保持指定状态失败的判断可采取与记录过程类似的判断方式。例如，在配准过程中，配准设备若出现超出任一阈值的移动，则停止当前位置配准和时间计算，然后重新开始新的配准和时间计算。具体地，当配准过程中出现因操作者咳嗽而引起的异常动作时，判断配准设备的可识别部位和标记部位是否出现了超出任一阈值的移动，若判断结果为是，则停止当前位置配准和时间计算，如有记录过程则一并停止，重新执行计算配准设备的位置坐标和配准条件的判断。
参见图7，一种电子设备700，包括存储器701和处理器702，存储器701用于存储计算机指令，计算机指令被处理器702执行以实现本申请实施例中任一的信息交互方法。
本申请还提供了一种可读存储介质，其上存储有计算机指令，该计算机指令被处理器执行时实现本申请实施例中任一的信息交互方法。
图8为适于用来实现本申请一实施方式的方法的计算机系统的结构示意图。
参见图8，计算机系统包括处理单元801，其可以根据存储在只读存储器(ROM)802中的程序或者从存储部分808加载到随机访问存储器(RAM)803中的程序而执行上述附图所示的实施方式中的各种处理。在RAM803中，还存储有系统操作所需的各种程序和数据。处理单元801、ROM802以及RAM803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
以下部件连接至I/O接口805：包括键盘、鼠标等的输入部分806；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分807；包括硬盘等的存储部分808；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分809。通信部分809经由诸如因特网的网络执行通信处理。驱动器8010也根据需要连接至I/O接口805。可拆卸介质8011，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器8010上，以便于从其上读出的计算机程序根据需要被安装入存储部分808。其中，处理单元801可实现为CPU、GPU、TPU、FPGA、NPU等处理单元。
特别地，根据本申请的实施方式，上文描述的方法可以被实现为计算机软件程序。例如，本申请的实施方式包括一种计算机程序产品，其包括有形地包含在及其可读介质上的计算机程序，计算机程序包含用于执行附图中的方法的程序代码。在这样的实施方式中，该计算机程序可以通过通信部分809从网络上被下载和安装，和/或从可拆卸介质8011被安装。
本领域的技术人员应当理解，上述实施方式仅仅是为了清楚地说明本公开，而并非是对本公开的范围进行限定。对于所属领域的技术人员而言，在上述公开的基础上还可以做出其它变化或变型，并且这些变化或变型仍处于本公开的范围内。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有 “第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
Claims (14)
- 一种用于手术导航的交互配准系统，其特征在于，该系统包括：医学影像获取模块、现场图像获取模块、识别模块、判断模块；其中，所述医学影像获取模块，用于获取施术对象的术前医学影像，并根据所述术前医学影像获得所述施术对象的解剖结构模型；所述现场图像获取模块，用于获取手术现场的现场图像；所述识别模块，用于根据所述现场图像识别手术现场中配准设备的位置；所述判断模块，用于根据所述配准设备的位置状态判断当前是否符合预设的配准条件；当符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
- 根据权利要求1所述的一种用于手术导航的交互配准系统,其特征在于，所述配准设备至少包括可识别部位和标记部位；所述预设配准条件为：当所述配准设备的可识别部位和标记部位处于稳定状态；和/或，当所述配准设备的可识别部位处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配。
- 根据权利要求2所述的一种用于手术导航的交互配准系统,其特征在于，所述预设运动参数为与人体呼吸或心跳导致的身体运动相应的运动参数。
- 根据权利要求2所述的一种用于手术导航的交互配准系统,其特征在于，所述运动参数包括运动频率、运动幅度、运动速度和运动轨迹中的至少一种。
- 根据权利要求2所述的一种用于手术导航的交互配准系统,其特征在于，所述判断模块还包括计算模块，所述计算模块用于进行以下计算：获取一段时间内所述配准设备的所述可识别部位和所述标记部位在各时刻的坐标，并计算该时间段内，所述配准设备的可识别部位的平均坐标；计算该时间段内，所述配准设备的标记部位的平均坐标；计算该时间段内，所述配准设备的所述可识别部位的各时刻的坐标相对所述可识别部位的平均坐标的距离；计算该时间段内，所述配准设备的所述标记部位的各时刻的坐标相对所述标记部位的平均坐标的距离；计算该时间段内，所述配准设备的所述标记部位的各时刻的移动速度。
- 根据权利要求5所述的一种用于手术导航的交互配准系统,其特征在于，当所述配准设备的可识别部位任意一点相对其几何平均位置的距离均小于或等于第一预设阈值时，判定所述配准设备的可识别部位处于稳定状态；当所述配准设备的标记部位任意一点相对其几何平均位置的距离；均小于或等于第二预设阈值时，判定所述配准设备的标记部位处于稳定状态；当所述配准设备的标记部位在该时间段内，出现最大运动速度时刻前后，均出现了小于或等于第三预设值的速度时，判定所述配准设备的标记部位的运动参数和预设运动参数相匹配。
- 根据权利要求2所述的一种用于手术导航的交互配准系统,其特征在于，所述判断模块还包括记录模块，所述记录模块具体用于：当所述配准设备的可识别部位和标记部位的位置处于稳定状态时，记录各时刻所述配准设备的标记部位的平均坐标；当所述配准设备的可识别部位的位置处于稳定状态，且所述配准设备的标记部位的运动参数和预设运动参数相匹配时，记录所述可识别部位在设定时刻的坐标的平均值，所述设定时刻为运动参数周期中运动参数最小的时刻。
- 根据权利要求1所述的一种用于手术导航的交互配准系统,其特征在于，还包括提示模块，用于引导操作者进行特定的操作。
- 根据权利要求8所述的一种用于手术导航的交互配准系统,其 特征在于，所述提示模块用于显示提示信息提示所述操作者做出指定行为以符合所述预设的配准条件；所述提示信息具体包括：指导所述操作者将配准设备移动至指定位置；或指导所述操作者支配所述配准设备在指定时间内保持指定状态。
- 根据权利要求9所述的一种用于手术导航的交互配准系统,其特征在于，所述指导所述操作者支配所述配准设备在指定时间内保持指定状态时，显示用于表征时间进度的动态图案。
- 根据权利要求10所述的一种用于手术导航的交互配准系统,其特征在于，若所述配准设备在指定时间内保持指定状态失败，所述提示模块提示所述操作者重新保持指定状态并重新开始计算指定时间。
- 一种用于手术导航的交互配准方法，其特征在于，包括：获取施术对象的术前医学影像，并根据所述术前医学影像获得所述施术对象的解剖结构模型；获取手术现场的现场图像；根据所述现场图像识别手术现场中配准设备的位置；根据所述配准设备的位置状态判断当前是否符合预设的配准条件，当所述配准设备的位置符合所述预设的配准条件时，根据所述施术对象的解剖结构模型以及所述现场图像，将所述解剖结构模型与所述手术现场进行位置配准。
- 一种电子设备，其包括存储器和处理器；其中，所述存储器用于存储可执行程序代码；所述处理器用于读取所述存储器中存储的可执行程序代码以执行根据权利要求12所述的方法。
- 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求12所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110358149.7A CN113133828B (zh) | 2021-04-01 | 2021-04-01 | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 |
| CN202110358149.7 | 2021-04-01 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022206434A1 true WO2022206434A1 (zh) | 2022-10-06 |
Family
ID=76810384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2022/081727 WO2022206434A1 (zh) | 2021-04-01 | 2022-03-18 | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113133828B (zh) |
| WO (1) | WO2022206434A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113133828B (zh) * | 2021-04-01 | 2023-12-01 | 上海复拓知达医疗科技有限公司 | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020082498A1 (en) * | 2000-10-05 | 2002-06-27 | Siemens Corporate Research, Inc. | Intra-operative image-guided neurosurgery with augmented reality visualization |
| US20090137907A1 (en) * | 2007-11-22 | 2009-05-28 | Kabushiki Kaisha Toshiba | Imaging diagnosis apparatus having needling navigation control system and a needling navigation controlling method |
| US20130293578A1 (en) * | 2012-05-02 | 2013-11-07 | Empire Technology Development LLC. a corporation | Four Dimensional Image Registration Using Dynamical Model For Augmented Reality In Medical Applications |
| US20140081659A1 (en) * | 2012-09-17 | 2014-03-20 | Depuy Orthopaedics, Inc. | Systems and methods for surgical and interventional planning, support, post-operative follow-up, and functional recovery tracking |
| US20170215969A1 (en) * | 2014-06-11 | 2017-08-03 | Tsinghua University | Human organ movement monitoring method, surgical navigation system and computer readable medium |
| US20180303558A1 (en) * | 2016-08-17 | 2018-10-25 | Monroe Milas Thomas | Methods and systems for registration of virtual space with real space in an augmented reality system |
| CN111388092A (zh) * | 2020-03-17 | 2020-07-10 | 京东方科技集团股份有限公司 | 一种定位跟踪件、配准方法、存储介质及电子设备 |
| CN111973273A (zh) * | 2020-08-31 | 2020-11-24 | 上海交通大学医学院附属第九人民医院 | 基于ar技术的手术导航系统、方法、设备和介质 |
| CN112155727A (zh) * | 2020-08-31 | 2021-01-01 | 上海市第一人民医院 | 基于三维模型的手术导航系统、方法、设备和介质 |
| CN113133828A (zh) * | 2021-04-01 | 2021-07-20 | 上海复拓知达医疗科技有限公司 | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102949240B (zh) * | 2011-08-26 | 2014-11-26 | 高欣 | 一种影像导航肺部介入手术系统 |
| CN103040525B (zh) * | 2012-12-27 | 2016-08-03 | 深圳先进技术研究院 | 一种多模医学影像手术导航方法及系统 |
| US10506947B2 (en) * | 2014-12-11 | 2019-12-17 | Koninklijke Philips N.V. | Automated selection of optimal calibration in tracked interventional procedures |
| WO2017030913A2 (en) * | 2015-08-14 | 2017-02-23 | Intuitive Surgical Operations, Inc. | Systems and methods of registration for image-guided surgery |
| EP3414737A4 (en) * | 2015-12-07 | 2019-11-20 | M.S.T. Medical Surgery Technologies Ltd. | AUTONOMOUS SYSTEM FOR DETERMINING CRITICAL POINTS IN LAPAROSCOPIC SURGERY |
| US11135016B2 (en) * | 2017-03-10 | 2021-10-05 | Brainlab Ag | Augmented reality pre-registration |
| CN107049489B (zh) * | 2017-03-29 | 2019-09-27 | 中国科学院苏州生物医学工程技术研究所 | 一种手术导航方法及系统 |
| US10786311B2 (en) * | 2017-12-22 | 2020-09-29 | Acclarent, Inc. | Apparatus and method for registering facial landmarks for surgical navigation system |
| US10504250B2 (en) * | 2018-01-27 | 2019-12-10 | Uih America, Inc. | Systems and methods for correcting mismatch induced by respiratory motion in positron emission tomography image reconstruction |
| CN110772320A (zh) * | 2018-07-30 | 2020-02-11 | 上海术凯机器人有限公司 | 一种配准方法、配准装置及计算机可读存储介质 |
| CN111772792B (zh) * | 2020-08-05 | 2023-11-28 | 山东省肿瘤防治研究院（山东省肿瘤医院） | 基于增强现实和深度学习的内窥镜手术导航方法、系统和可读存储介质 |
| CN112006776A (zh) * | 2020-09-27 | 2020-12-01 | 安徽埃克索医疗机器人有限公司 | 一种手术导航系统及手术导航系统的配准方法 |
| CN112155734B (zh) * | 2020-09-29 | 2022-01-28 | 苏州微创畅行机器人有限公司 | 可读存储介质、骨建模配准系统及骨科手术系统 |
-
2021
- 2021-04-01 CN CN202110358149.7A patent/CN113133828B/zh active Active
-
2022
- 2022-03-18 WO PCT/CN2022/081727 patent/WO2022206434A1/zh active Application Filing
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020082498A1 (en) * | 2000-10-05 | 2002-06-27 | Siemens Corporate Research, Inc. | Intra-operative image-guided neurosurgery with augmented reality visualization |
| US20090137907A1 (en) * | 2007-11-22 | 2009-05-28 | Kabushiki Kaisha Toshiba | Imaging diagnosis apparatus having needling navigation control system and a needling navigation controlling method |
| US20130293578A1 (en) * | 2012-05-02 | 2013-11-07 | Empire Technology Development LLC. a corporation | Four Dimensional Image Registration Using Dynamical Model For Augmented Reality In Medical Applications |
| WO2013165408A1 (en) * | 2012-05-02 | 2013-11-07 | Empire Technology Development Llc | Four dimensional image registration using dynamical model for augmented reality in medical applications |
| US20140081659A1 (en) * | 2012-09-17 | 2014-03-20 | Depuy Orthopaedics, Inc. | Systems and methods for surgical and interventional planning, support, post-operative follow-up, and functional recovery tracking |
| US20150282796A1 (en) * | 2012-09-17 | 2015-10-08 | DePuy Synthes Products, Inc. | Systems And Methods For Surgical And Interventional Planning, Support, Post-Operative Follow-Up, And, Functional Recovery Tracking |
| US20170215969A1 (en) * | 2014-06-11 | 2017-08-03 | Tsinghua University | Human organ movement monitoring method, surgical navigation system and computer readable medium |
| US20180303558A1 (en) * | 2016-08-17 | 2018-10-25 | Monroe Milas Thomas | Methods and systems for registration of virtual space with real space in an augmented reality system |
| CN111388092A (zh) * | 2020-03-17 | 2020-07-10 | 京东方科技集团股份有限公司 | 一种定位跟踪件、配准方法、存储介质及电子设备 |
| CN111973273A (zh) * | 2020-08-31 | 2020-11-24 | 上海交通大学医学院附属第九人民医院 | 基于ar技术的手术导航系统、方法、设备和介质 |
| CN112155727A (zh) * | 2020-08-31 | 2021-01-01 | 上海市第一人民医院 | 基于三维模型的手术导航系统、方法、设备和介质 |
| CN113133828A (zh) * | 2021-04-01 | 2021-07-20 | 上海复拓知达医疗科技有限公司 | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113133828A (zh) | 2021-07-20 |
| CN113133828B (zh) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102013866B1 (ko) | 실제수술영상을 이용한 카메라 위치 산출 방법 및 장치 | |
| JP6511050B2 (ja) | イメージング装置を追跡装置と位置合わせする位置合わせシステム、イメージングシステム、介入システム、位置合わせ方法、イメージングシステムの作動方法、位置合わせコンピュータプログラム、及びイメージングコンピュータプログラム | |
| CN111670018A (zh) | 用于定位患者和手术机器人的引导 | |
| US11660142B2 (en) | Method for generating surgical simulation information and program | |
| US20210346103A1 (en) | Support for a medical intervention | |
| CN111839727A (zh) | 基于增强现实的前列腺粒子植入路径可视化方法及系统 | |
| WO2022206434A1 (zh) | 一种用于手术导航的交互配准系统、方法、电子设备和可读存储介质 | |
| WO2022206435A1 (zh) | 一种手术导航系统、方法、电子设备和可读存储介质 | |
| EP3075342B1 (en) | Microscope image processing device and medical microscope system | |
| KR20190080706A (ko) | 수술보조 영상 표시방법, 프로그램 및 수술보조 영상 표시장치 | |
| CN110613519A (zh) | 动态配准定位装置及方法 | |
| CN116420107A (zh) | 使用增强现实的深脑部路径规划 | |
| CN109934798A (zh) | 体内目标物信息标注方法及装置、电子设备、存储介质 | |
| CN109106448A (zh) | 一种手术导航方法和装置 | |
| US20220370147A1 (en) | Technique of Providing User Guidance For Obtaining A Registration Between Patient Image Data And A Surgical Tracking System | |
| CN108852513A (zh) | 一种骨科手术导引系统的器械导引方法 | |
| KR20190088419A (ko) | 수술 시뮬레이션 정보 생성방법 및 프로그램 | |
| EP4140411A1 (en) | Oral image marker detection method, and oral image matching device and method using same | |
| JP2011131020A (ja) | トロカーポート位置決定シミュレーション方法及びその装置 | |
| US10049480B2 (en) | Image alignment device, method, and program | |
| KR102378114B1 (ko) | 증강 현실을 이용한 수술 가이드 방법 및 장치 | |
| KR20190133423A (ko) | 수술 시뮬레이션 정보 생성방법 및 프로그램 | |
| CN115444553B (zh) | 进针点投影方法、装置、设备和存储介质 | |
| KR101940706B1 (ko) | 수술 시뮬레이션 정보 생성방법 및 프로그램 | |
| CN116795213A (zh) | 基于增强现实技术的智能人机交互系统及方法 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22778617 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 22778617 Country of ref document: EP Kind code of ref document: A1 |