WO2022206415A1

WO2022206415A1 - Prompt information display device and method for augmented reality surgical navigation system

Info

Publication number: WO2022206415A1
Application number: PCT/CN2022/081504
Authority: WO
Inventors: 孙非; 朱奕; 郭晓杰; 崔芙粒; 单莹
Original assignee: 上海复拓知达医疗科技有限公司
Priority date: 2021-04-01
Filing date: 2022-03-17
Publication date: 2022-10-06
Also published as: CN113113149A

Abstract

The present application provides a prompt information display device and method for an augmented reality surgical navigation system. The prompt information display device for the augmented reality surgical navigation system comprises an obtaining module used for obtaining prompt information to be displayed; a first determining module used for determining the priority of the prompt information according to the information type to which the prompt information belongs; a second determining module used for determining an information display position of the prompt information according to the information content category to which the prompt information belongs; and a display module used for displaying the prompt information according to the priority of the prompt information and the information display position of the prompt information. By adopting the technical solution of the present application, the negative influence of the prompt information of the augmented reality surgical navigation system on an operator can be reduced.

Description

A prompt information display device and method of an augmented reality surgical navigation system

technical field

The present application relates to the field of computers, and in particular, to a prompt information display device and method of an augmented reality surgical navigation system.

Background technique

The augmented reality surgical navigation system allows the wearer to display surgical-related information at the same time, and cooperate with the position information of the surgical instruments in use obtained by the head-mounted camera or other positioning and acquisition devices to achieve rapid surgical navigation. However, displaying too much information to the operator irregularly will distract the operator's attention, reduce the operator's work efficiency, and even create risks.

SUMMARY OF THE INVENTION

In order to solve at least one of the above technical problems, the present application provides a prompt information display device and method of an augmented reality surgical navigation system.

A first aspect of the present application is a prompt information display device of an augmented reality surgical navigation system, comprising:

The acquisition module is used to acquire the prompt information to be displayed;

a first determining module, configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs;

a second determining module, configured to determine the information display position of the prompt information according to the information content category to which the prompt information belongs;

The display module is configured to display the prompt information according to the priority of the prompt information and the information display position of the prompt information.

Optionally, when the first determining module is configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs, it is specifically used to:

If the information type category to which the prompt information belongs is normal information, the priority of the prompt information is determined according to the information display position category to which the prompt information belongs;

If the information type category to which the prompt information belongs is abnormal information, determining the priority of the prompt information according to the information content category to which the prompt information belongs;

Wherein, the information display position category includes a first category, a second category and a third category, the first category indicates that the display position of the prompt information belonging to this category is related to the position of the display area, and the second category indicates that the display position belongs to this category The display position of the prompt information of the category is related to the patient position, and the third category indicates that the display position of the prompt information belonging to this category is related to the surgical instrument position.

The priority of the prompt information is determined according to the information type category to which the prompt information belongs and the information display time category to which the prompt information belongs, where the information display time category includes permanent display and conditional display.

Optionally, when the second determining module is configured to determine the information display position of the prompt information according to the information content category to which the prompt information belongs, it is specifically used for:

According to the information content category to which the prompt information belongs, determine the information display position category to which the prompt information belongs;

The information display position of the prompt information is determined according to the information display position category to which the prompt information belongs.

Optionally, when the second determining module is configured to determine the information display location category to which the prompt information belongs according to the information content category to which the prompt information belongs, it is specifically configured to:

If the information content category to which the prompt information belongs is the device status category, determine that the information display position category to which the prompt information belongs is the first category, and the first category indicates the display position and display area of the prompt information belonging to the category location related;

If the information content category to which the prompt information belongs is the patient information category, determine that the information display position category to which the prompt information belongs is the second category, and the second category indicates the display position and the patient position of the prompt information belonging to this category related;

If the information content category to which the prompt information belongs is a surgical instrument category, it is determined that the information display position to which the prompt information belongs is classified as a third category, and the third category indicates the display position of the prompt information belonging to this category and the surgical instrument Location related.

Optionally, the display module is used to display the prompt information according to the priority of the prompt information and the information display position of the prompt information, specifically for:

According to the priority of the prompt information and the current information display status, determine whether to display the prompt information;

If it is determined that the prompt information is displayed, the prompt information is displayed on the information display position.

Optionally, when the display module is used to judge whether to display the prompt information according to the priority of the prompt information and the current information display status, it is specifically used for:

It is judged whether the displayed quantity of the current information exceeds a set value, and when it exceeds the set value, it is judged whether to display the prompt information according to the priority of the prompt information.

Optionally, the device includes:

A display mode determination module, configured to obtain the target direction of the surgical instrument corresponding to the prompt information when it is judged that the information display position category to which the prompt information belongs is the third category, and determine the display of the prompt information according to the target direction direction, wherein the third category indicates that the display position of the prompt information belonging to this category is related to the position of the surgical instrument.

In a second aspect of the present application, a method for displaying prompt information of an augmented reality surgical navigation system, comprising:

Get the prompt information to be displayed;

Determine the priority of the prompt information according to the information type category to which the prompt information belongs;

Determine the information display position of the prompt information according to the information content category to which the prompt information belongs;

The prompt information is displayed according to the priority of the prompt information and the information display position of the prompt information.

Optionally, the information type category includes normal type information and abnormal type information;

The determining the priority of the prompt information according to the information type category to which the prompt information belongs, includes:

Optionally, determining the priority of the prompt information according to the information type category to which the prompt information belongs, including:

Optionally, determining the information display position of the prompt information according to the information content category to which the prompt information belongs, including:

Optionally, the information content categories include equipment status, patient information, surgical plans, surgical steps, and surgical instruments;

The determining the information display position category to which the prompt information belongs according to the information content category to which the prompt information belongs includes:

Optionally, displaying the prompt information according to the priority of the prompt information and the information display position of the prompt information, including:

Optionally, determining whether to display the prompt information according to the priority of the prompt information and the current information display status includes:

Optionally, the method further includes:

When it is determined that the information display position category to which the prompt information belongs is the third category, the target direction of the surgical instrument corresponding to the prompt information is acquired, and the display direction of the prompt information is determined according to the target direction, wherein the first The three categories indicate that the display position of the prompt information belonging to this category is related to the position of the surgical instrument.

In a third aspect of the present application, an electronic device includes a memory and a processor, wherein the memory is used to store computer instructions, wherein the computer instructions are executed by the processor to implement the second aspect of the present application the method described.

By implementing the technical solution of the present application, the following beneficial technical effects can be obtained: the technical solution of the present application determines the priority of the prompt information and the information display position respectively according to the information type category and the information content category, and can quickly determine the priority of the prompt information and information display. position; and, displaying the prompt information according to the priority of the prompt information and the information display position can reduce the negative impact of the prompt information of the augmented reality surgical navigation system on the operator.

Description of drawings

The accompanying drawings illustrate exemplary embodiments of the application and, together with the description, serve to explain the principles of the application, are included to provide a further understanding of the application, and are included in and constitute the specification part of the manual.

1 is a structural block diagram of a prompt information display device of an augmented reality surgical navigation system in an embodiment of the present application;

Fig. 2 is a kind of prompt information display schematic diagram in the embodiment of the present application;

3 is a flowchart of a prompt information display method of an augmented reality surgical navigation system in an embodiment of the present application;

FIG. 4 is a structural block diagram of an electronic device in an embodiment of the present application.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related content, rather than limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings.

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other if there is no conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

Referring to Fig. 1, a prompt information display device of an augmented reality surgical navigation system includes:

an obtaining module 101, configured to obtain prompt information to be displayed;

The first determining module 102 is configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs;

The second determining module 103 is configured to determine the information display position of the prompt information according to the information content category to which the prompt information belongs;

The display module 104 is configured to display the prompt information according to the priority of the prompt information and the information display position of the prompt information.

In this embodiment, according to the specific situation, the system pre-sets the classification of the prompt information that needs to be displayed to the user, so that when the system obtains a prompt information to be displayed, it determines to display the prompt according to the category to which the prompt information belongs. Location and priority of information. For example, the information type categories of the prompt information are divided into normal information and abnormal information; the information content categories of the prompt information are divided into equipment status, patient information, surgical planning, surgical steps, and surgical instruments. In this embodiment, the priority indicates the priority of displaying the prompt information; the priority can be used to determine whether the prompt information is hidden or blocked by other prompt information, for example, to determine which prompt information is displayed when two prompt information are displayed at the same position The prompt information is displayed in the front, which prompt information is blocked; it is used to judge which prompt information is displayed and which prompt information is hidden when two prompt information is not allowed to be displayed in the same set position; it is used to display the prompt information exceeding the setting When the number is displayed, it is determined which prompt information is hidden according to the display priority.

The technical solution of this embodiment can respectively determine the priority of the prompt information and the information display position according to the information type category and the information content category of the prompt information, so as to quickly obtain the priority and the information display position, and then according to the priority of the prompt information and the information display position The prompt information is displayed at the location. Displaying the prompt information according to the priority of the prompt information and the information display position can reduce the probability of confusion in the display of the prompt information, reduce the distraction caused by the prompt information to the operator, and reduce the prompt information of the augmented reality surgical navigation system. Negative effects caused by the operator.

In order to facilitate intuitive understanding, refer to Figure 2. The information display position shown in Figure 2 displays "AR glasses power", "signal", "patient information", "surgical step prompt", "patient lesion related information", "surgery" Device-related information” and other prompt information. Among them, "patient information" is displayed at or near the patient 201, "surgical instrument related information" is displayed at or near the surgical instrument 202, and "AR glasses power" and "signal" are displayed on the display of the AR glasses The fixed position of the area, such as displayed in the upper left or upper right corner of the display area of the AR glasses.

In one embodiment, when the first determining module is configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs, it is specifically used to:

If the information type category to which the prompt information belongs is abnormal information, the priority of the prompt information is determined according to the information content category to which the prompt information belongs;

Among them, abnormal information refers to information related to various abnormal states that need to be clearly prompted to the user, such as abnormal system connection, extremely low system power, medical device recognition failure, patient registration failure and other information. Since abnormal information may affect the normal operation of the operation, it needs to be displayed to the user with a high priority to avoid affecting the operation process because the user does not know the abnormal information in time. Normal information refers to information other than abnormal information prompted to the user, such as successful system connection, power display when the system power is normal, successful identification of medical devices, successful patient configuration, etc., as well as the operation steps displayed to the user during the operation Navigation tips, etc.

Among them, the information display position category includes the first category, the second category and the third category. The first category indicates that the display position of the prompt information belonging to this category is related to the position of the display area. For example, the device battery belongs to the first category and is displayed in the AR The upper left or upper right corner of the display area of the glasses; the second category indicates that the display position of the prompt information belonging to this category is related to the patient's position, for example, the patient's lesion location prompt information belongs to the second category, and the information is superimposed in an augmented reality manner Displayed at the corresponding location on the patient's body. The third category indicates that the display position of the prompt information belonging to this category is related to the position of the surgical instrument. For example, in the navigation of the puncture operation, the extension line of the needle body of the puncture needle is displayed on the front end of the needle tip of the puncture needle in an augmented reality manner to help the user. Determine whether the needle tip of the puncture needle is aligned with the location of the lesion.

In this embodiment, the priority of the abnormal information is higher than that of the normal information, so that the display of the abnormal information is not easily affected by the normal information. At the same time, when the two prompt messages are abnormal information, the information content type is used to determine the priority. , and when both prompt messages are normal messages, the priority is determined by the message placement category.

Specifically, the normal information is based on the actual distance from the wearer's field of vision, according to the first category of the information display position category "related to the position of the display area", the third category "related to the position of the surgical instrument" and the second category "related to the position of the patient" The priority of ” judges occlusion, which can be specifically: the priority related to the position of the display area > the priority related to the position of the surgical instrument > the priority related to the position of the patient. The priority of abnormal information is always higher than that of normal information. The priority of abnormal information is determined according to the information content category. Specifically, it can be: equipment status category > device information category > surgical plan category > patient information category > surgical procedure category.

Specifically, the priority of the conditional display > the priority of the permanent display, where the permanent display is defined as a display that will always exist in the space or screen after the device is started (for example, the information of the displayable area of the screen will always appear on the screen, and the patient information will always be fixed. In the operating room or patient-specific location); conditional display is defined as if and only when the trigger condition is met, the relevant information is displayed, and when the trigger condition does not exist, the relevant information is hidden. For example, the device power information is displayed permanently, and the operation step prompt information is conditional display. In the case that these two prompt information belong to normal information (when the power is not extremely low), if the two prompt information are displayed in the field of vision If there is a conflict, the prompt information of the surgical steps will be displayed first to ensure the smooth operation of the operation.

In one embodiment, the second determination module is used to determine the information display position of the prompt information according to the information content category to which the prompt information belongs, and is specifically used for:

According to the information content category to which the prompt information belongs, determine the information display location category to which the prompt information belongs;

In this embodiment, the information display position category to which the prompt information belongs is first determined according to the information content category, and then the information display position of the prompt information is determined according to the information display position category to which the prompt information belongs. Information placement categories are primarily used to determine prompt information.

The information display position category includes three types of information related to the display position and the display area position, the display position and the patient position, and the display position and the surgical instrument position. Among them, since the prompt information is generally displayed through the display screen, the display area position correlation is generally the display screen position correlation, for example, the border-related prompt information is displayed on the edge of the display screen.

The above-mentioned position correlation includes relative fixed position and relative movement relative to position. Among them, the position is relatively moved. For example, the prompt information related to the position of the display area can be moved from the upper left corner of the display area to the lower left corner of the display area, and the prompt information related to the patient's position can be moved from the patient's feet to the head, related to the position of the surgical instrument. The prompt information can be moved from the root of the surgical instrument to the head of the surgical instrument.

In one embodiment, when the second determining module is configured to determine the information display location category to which the prompt information belongs according to the information content category to which the prompt information belongs, it is specifically used for:

If the information content category to which the prompt information belongs is the device status category, then it is determined that the information display position category to which the prompt information belongs is the first category, and the first category indicates that the display position of the prompt information belonging to this category is related to the position of the display area;

If the information content category to which the prompt information belongs is the patient information category, determine that the information display position category to which the prompt information belongs is the second category, and the second category indicates that the display position of the prompt information belonging to this category is related to the patient's position;

If the information content category to which the prompt information belongs is a surgical instrument category, it is determined that the information display position to which the prompt information belongs is classified as a third category, and the third category indicates that the display position of the prompt information belonging to this category is related to the position of the surgical instrument.

Optionally, if the information content category to which the prompt information belongs is a surgical plan category or a surgical procedure category, the information display location category to which the prompt information belongs is determined according to the specific content of the prompt information.

As can be seen from the above, the information content categories may include equipment status, patient information, surgical plans, surgical steps, and surgical instruments.

Specifically, the display position of equipment status prompt information is always related to the position of the display area, as the first priority prompt; the display position of patient information prompt information is always related to the patient position; the display position of surgical instrument prompt information is related to the surgical instrument Position-related; while in the surgical planning information, the surgical target and surrounding organs are related to the patient's position. At different stages of surgery, the prediction of instrument orientation can be related to the instrument or the patient's position (related to the surgical target); surgical steps are related to the position of the patient. The text prompt part of the prompt information is related to the position of the display area, the part of the used instrument is related to the position of the device, and the part of the occurrence position is related to the position of the patient.

Among them, the device status category includes device storage space, power, network communication status, screen displayable area and camera identifiable area, etc. The device status class can be further divided into the augmented reality device itself and the monitoring and surgical devices that connect the signal source to it. Among them, the prompt information related to the augmented reality device itself is generally of high importance, and is related to the screen position with the highest priority and cannot be ignored. The monitoring equipment connected to the signal source has other operation and usage specifications, and the importance is relatively low. It can be related to the environmental location, permanently displayed, and can be blocked.

Patient information includes name, gender, case number, surgical site, physiological signals (such as blood pressure, blood oxygen, respiration), etc.

The surgical plan includes the target lesion, the target surgical area, surrounding organs, and the planned puncture location, depth, and angle.

Surgical steps include textual or pictorial reminders of instrument manipulation steps to be taken in sequence to achieve the current surgical plan.

The categories of information display modes of the prompt information in the embodiments of the present application include static text, 2D graphics, and 3D graphics, and dynamic text, 2D graphics, and 3D graphics.

In one embodiment, the display module is configured to display the prompt information according to the priority of the prompt information and the information display position of the prompt information, and is specifically used for:

The current information display status may be the current total number of information displayed, whether information has been displayed at a specific display position, and the priority comparison between the displayed prompt information and the prompt information to be displayed.

In one embodiment, the display module is used to judge whether to display the prompt information according to the priority of the prompt information and the current information display status, and is specifically used for:

It is judged whether the display quantity of the current information exceeds the set value, and when it exceeds the set value, it is judged whether to display the prompt information according to the priority of the prompt information.

In one embodiment, the apparatus includes:

The display mode determination module is used to obtain the target direction of the surgical instrument corresponding to the prompt information when it is determined that the information display position category to which the prompt information belongs is the third category, and determine the display direction of the prompt information according to the target direction, wherein the third category represents The placement of prompts that fall into this category is relative to the surgical instrument location. For example, in the puncture operation navigation process, the prompt information to be displayed is specifically the puncture path direction of the puncture needle, which is determined according to the orientation of the puncture needle when it is aimed at the puncture target.

In this embodiment, the display direction of the prompt information is determined according to the target direction, so that after the prompt information is straightened, the corresponding surgical instrument is also straightened, that is, the direction of the surgical instrument faces the target direction. The technical solution of this embodiment can enable the user to adjust the corresponding surgical instrument according to the prompt information in a natural state. For example, the text of the prompt information is upside down, and the corresponding surgical instrument needs to be rotated by 180° to meet the requirements of the surgical instrument. Requirements.

Referring to Fig. 3, a prompt information display method of an augmented reality surgical navigation system includes:

S301, obtaining prompt information to be displayed;

S302, determine the priority of the prompt information according to the information type category to which the prompt information belongs;

S303, according to the information content category to which the prompt information belongs, determine the information display position of the prompt information;

S304, displaying the prompt information according to the priority of the prompt information and the information display position of the prompt information.

In this embodiment, according to the specific situation, the system pre-sets the classification of the prompt information that needs to be displayed to the user, so that when the system obtains a prompt information to be displayed, it determines to display the prompt according to the category to which the prompt information belongs. Location and priority of information. For example, the information type categories of the prompt information are divided into normal information and abnormal information; the information content categories of the prompt information are divided into equipment status, patient information, surgical planning, surgical steps, and surgical instruments.

In this embodiment, the priority indicates the priority of displaying the prompt information; the priority can be used to determine whether the prompt information is hidden or blocked by other prompt information, for example, to determine which prompt information is displayed when two prompt information are displayed at the same position The prompt information is displayed in the front, which prompt information is blocked; it is used to judge which prompt information is displayed and which prompt information is hidden when two prompt information is not allowed to be displayed in the same set position; it is used to display the prompt information exceeding the setting When the number is displayed, it is determined which prompt information is hidden according to the display priority.

In one embodiment, the information type category includes normal type information and abnormal type information;

Determine the priority of the prompt information according to the information type category to which the prompt information belongs, including:

The information display position category includes a first category, a second category and a third category. The first category indicates that the display position of the prompt information belonging to this category is related to the position of the display area, and the second category indicates the display of the prompt information belonging to this category. The location is related to the patient location, and the third category indicates that the display location of the prompt information belonging to this category is related to the surgical instrument location.

Specifically, the normal information is based on the actual distance from the wearer's field of vision, according to the first category of the information display position category "related to the position of the display area", the third category "related to the position of the surgical instrument" and the second category "related to the position of the patient" The priority of ” judges occlusion, which can be specifically: the priority related to the position of the display area > the priority related to the position of the surgical instrument > the priority related to the position of the patient. The priority of abnormal information is always higher than that of normal information. The priority among the abnormal information is determined according to the information content category, which may specifically be: equipment status category > device information category > surgical plan category > patient information category > surgical procedure category.

In an optional implementation manner, the priority of the prompt information is determined according to the information type category to which the prompt information belongs, including:

Specifically, the priority of the conditional display > the priority of the permanent display, where the permanent display is defined as a display that will always exist in the space or screen after the device is started (for example, the information of the displayable area of the screen will always appear on the screen, and the patient information will always be fixed. In the operating room or patient-specific location); conditional display is defined as if and only when the trigger condition is met, the relevant information is displayed, and when the trigger condition does not exist, the relevant information is hidden.

In an optional implementation manner, the information display position of the prompt information is determined according to the information content category to which the prompt information belongs, including:

In this embodiment, the information display position category to which the prompt information belongs is first determined according to the information content category, and then the information display position of the prompt information is determined according to the information display position category to which the prompt information belongs. The information placement category is mainly used to determine the prompt information,

The information display position category includes three types of information related to the display position and the position of the display area, the display position and the patient position, and the display position and the position of the surgical instrument. Position correlation includes relative fixed position and relative movement of position. Relative position movement. For example, the prompt information related to the position of the display area can be moved from the upper left corner of the display area to the lower left corner of the display area. The prompt information related to the patient's position can be moved from the patient's feet to the head, and the prompt information related to the position of the surgical instrument It can be moved from the root of the surgical instrument to the head of the surgical instrument, etc.

In an optional embodiment, the information content categories include equipment status, patient information, surgical plans, surgical steps, and surgical instruments;

According to the information content category to which the prompt information belongs, determine the information placement category to which the prompt information belongs, including:

Among them, the display position of the equipment status prompt information is always related to the position of the display area, as the first priority prompt; the display position of the patient information prompt information is always related to the patient position; the display position of the surgical instrument prompt information is related to the position of the surgical instrument In the surgical planning information, the surgical target and surrounding organs are related to the position of the patient. At different stages of the operation, the prediction of the orientation of the instrument can be related to the instrument or the position of the patient (related to the surgical target); the surgical steps prompt The text prompt part of the information is related to the position of the display area, the part of the used instrument is related to the position of the device, and the part of the occurrence position is related to the position of the patient.

Specifically, the device status category includes device storage space, power, network communication status, screen displayable area, camera-recognizable area, and the like. The device status class can be further divided into the augmented reality device itself and the monitoring and surgical devices that connect the signal source to it. Among them, the prompt information related to the augmented reality device itself is generally of high importance, and is related to the screen position with the highest priority and cannot be ignored. The monitoring equipment connected to the signal source has other operation and usage specifications, and the importance is relatively low. It can be related to the environmental location, permanently displayed, and can be blocked.

Patient information includes name, gender, case number, surgical site, physiological signals (such as blood pressure, blood oxygen respiration), etc.

In an optional implementation manner, the prompt information is displayed according to the priority of the prompt information and the information display position of the prompt information, including:

Specifically, according to the priority of the prompt information and the current information display status, it is judged whether to display the prompt information, including:

In an optional embodiment, the method further includes:

When it is determined that the information display location category to which the prompt information belongs is the third category, the target direction of the surgical instrument corresponding to the prompt information is obtained, and the display direction of the prompt information is determined according to the target direction, wherein the third category represents the type of prompt information belonging to the category. The placement is relative to the surgical instrument location.

In an optional embodiment, the dynamic strategy of the prompt information is as follows: based on the classification of the information display mode, at the same time, no more than two dynamic information may appear in the user's field of vision.

The following example illustrates the priority of dynamic prompt information for each information content category:

For example, the information of the area that can be recognized by the camera of an augmented reality device belonging to the device status category is the most important and the highest priority information in the process of navigating surgery. The information display strategy is: the display position is related to the position of the display area, displayed permanently, and never blocked. At the same time, because it belongs to continuous existence, it does not need to attract attention, so it can be a static 2D pattern, reducing the interference to vision in use.

For example, the remaining storage space information of the device status class, because it is not necessary to pay attention to it in normal use, but once the storage space is insufficient, it is likely to cause the failure of the navigation process, so the display strategy of the information can be: related to the position of the display area, Displayed only when the remaining storage space is less than a certain threshold, not blocked (the same as the identifiable area is related to the position of the display area, and the interface design can be used to avoid blocking between the two), static or dynamic 2D pattern.

For example, the name information of the patient information category is generally only used at the beginning of the operation, and its display strategy can be: related to the patient's position, only displayed 2 minutes before the start of the operation, and can be blocked by all other information, static text.

For example, the needle entry point information of body surface puncture in the surgical plan, which should be associated with the patient's position, is displayed when the surgical step is performed to the navigational puncture, and is hidden and displayed when the surgical step is performed to the puncture into the skin, which can be hidden from the same surgical plan. The puncture target inside the body can be occluded by the indication information of the advancing direction of the instrument, and it is a static 2D pattern.

For example, the selection information of body surface feature points of the surgical step class is triggered when the surgical step is executed to this step. After the trigger, static text can be used in the area related to the screen position to prompt the operator that this step should be executed at this time. The area where the first feature point of the patient's body is located uses dynamic 3D to prompt the operator to operate at this position. After the operation in this area is completed, the static text on the screen is changed, and the dynamic 3D of another feature point area on the patient's body is used to prompt the operator to go down. one step position. Screen information cannot be blocked, and spatial information can be blocked by other information.

For example, the information on the distance between the needle tip of the puncture needle of the surgical instrument and the lesion can be triggered and displayed when the surgical step is performed to the puncture navigation. When triggered for the first time, it is fixed outside the patient's body, and the position of the needle tip of the puncture needle in the operator's hand will highlight this information through dynamic text and static 3D patterns to attract attention. The focus of the eye can clearly see this information and the puncture needle tip as the operation target at the same time. The text part of this information is never occluded by non-screen related information, whereas the static 3D part that serves as a foil can be occluded by other information. When the surgical step enters the next step, that is, after the puncture needle pierces the skin, this information should no longer stay at the position of the puncture needle tip, but is converted into surgical planning information associated with the patient's position: the area where the puncture target is located. . At this stage of the surgical procedure, the operator's attention should be focused on the puncture target that is farthest from the field of vision, and at the same time, by controlling the direction of the hand-held puncture needle, the direction of the puncture needle should be as stable as possible within the puncture target. Therefore, the position correlation method of the equally important information "distance between the puncture needle tip and the lesion" should be jumped to the target position, reminding the operator that the focus of attention should be shifted to the puncture target farther away from the needle tip.

Referring to FIG. 4 , an electronic device 400, a processor 401 and a memory 402, the memory 402 is used for storing computer instructions, and the computer instructions are executed by the processor 401 to implement any of the methods described in the embodiments of this application.

The present application also provides a computer system for executing the method of the embodiments of the present application, the computer system includes a processing unit, which can be loaded into a random device according to a program stored in a read only memory (ROM) or from a storage part Various processes in the embodiments shown in the above drawings are executed by accessing a program in a memory (RAM). In the RAM, various programs and data required for system operation are also stored. The processing unit, ROM, and RAM are connected to each other through a bus. Input/output (I/O) interfaces are also connected to the bus.

The following components are connected to the I/O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; The communication part of the network interface card of the LAN card, modem, etc. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the I/O interface as required. Removable media, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are mounted on the drive as needed, so that the computer program read therefrom is installed into the storage section as needed. The processing unit may be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, and an NPU.

In the description of this specification, references to the terms "one embodiment/mode", "some embodiments/modes", "example", "specific example", or "some examples", etc. are intended to be combined with the description of the embodiment/mode A particular feature, structure, material, or characteristic described by way of example or example is included in at least one embodiment/mode or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment/mode or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments/means or examples. Furthermore, those skilled in the art may combine and combine the different embodiments/modes or examples described in this specification and the features of the different embodiments/modes or examples without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

It should be understood by those skilled in the art that the above embodiments are only for clearly illustrating the present application, rather than limiting the scope of the present application. For those skilled in the art, other changes or modifications can also be made on the basis of the above disclosure, and these changes or modifications are still within the scope of the present application.

Claims

A prompt information display device for an augmented reality surgical navigation system, characterized in that it includes:

The acquisition module is used to acquire the prompt information to be displayed;

a first determining module, configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs;

a second determining module, configured to determine the information display position of the prompt information according to the information content category to which the prompt information belongs;

The display module is configured to display the prompt information according to the priority of the prompt information and the information display position of the prompt information.
The prompt information display device of an augmented reality surgical navigation system according to claim 1, wherein the first determination module is configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs level, specifically for:

If the information type category to which the prompt information belongs is normal information, the priority of the prompt information is determined according to the information display position category to which the prompt information belongs;

If the information type category to which the prompt information belongs is abnormal information, determining the priority of the prompt information according to the information content category to which the prompt information belongs;

Wherein, the information display position category includes a first category, a second category and a third category, the first category indicates that the display position of the prompt information belonging to this category is related to the position of the display area, and the second category indicates that the display position belongs to this category The display position of the prompt information of the category is related to the patient position, and the third category indicates that the display position of the prompt information belonging to this category is related to the surgical instrument position.
The prompt information display device of an augmented reality surgical navigation system according to claim 1, wherein the first determination module is configured to determine the priority of the prompt information according to the information type category to which the prompt information belongs level, specifically for:

The priority of the prompt information is determined according to the information type category to which the prompt information belongs and the information display time category to which the prompt information belongs, where the information display time category includes permanent display and conditional display.
The prompt information display device of an augmented reality surgical navigation system according to claim 1 or 2, wherein the second determination module is configured to determine the prompt information according to the information content category to which the prompt information belongs When displaying the location of the information, it is specifically used to:

According to the information content category to which the prompt information belongs, determine the information display position category to which the prompt information belongs;

The information display position of the prompt information is determined according to the information display position category to which the prompt information belongs.
The prompt information display device of an augmented reality surgical navigation system according to claim 4, wherein the second determination module is configured to determine the type of the prompt information to which the prompt information belongs according to the information content category to which the prompt information belongs. Information placement category, specifically for:

If the information content category to which the prompt information belongs is the device status category, determine that the information display position category to which the prompt information belongs is the first category, and the first category indicates the display position and display area of the prompt information belonging to the category location related;

If the information content category to which the prompt information belongs is the patient information category, determine that the information display position category to which the prompt information belongs is the second category, and the second category indicates the display position and the patient position of the prompt information belonging to this category related;

If the information content category to which the prompt information belongs is a surgical instrument category, it is determined that the information display position to which the prompt information belongs is classified as a third category, and the third category indicates the display position of the prompt information belonging to this category and the surgical instrument Location related.
The prompt information display device of an augmented reality surgical navigation system according to claim 1, wherein the display module is configured to display all the prompt information according to the priority of the prompt information and the information display position of the prompt information. When the prompt information is mentioned, it is specifically used for:

According to the priority of the prompt information and the current information display status, determine whether to display the prompt information;

If it is determined that the prompt information is displayed, the prompt information is displayed on the information display position.
The prompt information display device of an augmented reality surgical navigation system according to claim 6, wherein the display module is configured to determine whether to display the prompt according to the priority of the prompt information and the current information display status information, specifically for:

It is judged whether the displayed quantity of the current information exceeds the set value, and when it exceeds the set value, it is judged whether to display the prompt information according to the priority of the prompt information.
The prompt information display device of an augmented reality surgical navigation system according to claim 1, wherein the device comprises:

A display mode determination module, configured to obtain the target direction of the surgical instrument corresponding to the prompt information when it is judged that the information display position category to which the prompt information belongs is the third category, and determine the display of the prompt information according to the target direction direction, wherein the third category indicates that the display position of the prompt information belonging to this category is related to the position of the surgical instrument.
A prompt information display method of an augmented reality surgical navigation system, characterized in that it includes:

Get the prompt information to be displayed;

Determine the priority of the prompt information according to the information type category to which the prompt information belongs;

Determine the information display position of the prompt information according to the information content category to which the prompt information belongs;

The prompt information is displayed according to the priority of the prompt information and the information display position of the prompt information.
An electronic device comprising a memory and a processor, wherein the memory is used to store computer instructions, wherein the computer instructions are executed by the processor to implement the method of claim 9 .