CN112509119B - Spatial data processing and positioning method and device for temporal bone and electronic equipment - Google Patents

Abstract

The application discloses a method and a device for processing and positioning spatial data of a temporal bone and electronic equipment. The processing method comprises the following steps: determining a reference surface between the bilateral external semicircular vessel bone tube structures based on a first image containing the bilateral external semicircular vessel bone tube structures; according to the reference surface, determining the symmetrical characteristic points corresponding to the bone tube structures of the bilateral external semicircular canals respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information of the bone tube structure of the corresponding bilateral external semicircular canal in the first image; determining a coronal plane from the transverse axis plane and the median sagittal plane; and establishing a temporal bone space coordinate system based on the coordinate origin, the median sagittal plane, the transverse axis plane and the coronal plane. After a user triggers a calibration request for one point in an image, the embodiment provided by the application can automatically determine and display the coordinate value of the point by using the temporal bone space coordinate system, thereby providing a good basis for subsequent medical research.

Description

Spatial data processing and positioning method and device for temporal bone and electronic equipment

Cross-referencing

Chinese patent application No. 201911337695.1 entitled "method of constructing a temporal bone space coordinate system, a spatial localization method, and an electronic device" filed on 12/23/2019, which is incorporated herein by reference in its entirety.

Technical Field

The application belongs to the technical field of computers, and particularly relates to a temporal bone spatial data processing method, a temporal bone spatial positioning method, a temporal bone data processing device, a temporal bone spatial positioning device and electronic equipment.

Background

The temporal bone is the most delicate and complex bone structure in human skeleton, and contains the important fine structures of auditory system, blood vessels, arteries, etc. At present, doctors or researchers mainly improve the knowledge and research on the temporal bone through a three-dimensional reconstruction model of the temporal bone. In the process of studying the three-dimensional model of the temporal bone, the absolute positioning problem of the temporal bone structure or the spatial point is often required to be analyzed according to a spatial coordinate system. At present, no related technology can establish a space coordinate system capable of accurately calibrating points. Nowadays, the artificial intelligence technology is rapidly developed, and if the artificial intelligence technology can be combined, great help is provided for improving the accuracy of model and point positioning.

Disclosure of Invention

In view of this, the present application provides a method, an apparatus, and an electronic device for processing and positioning temporal bone spatial data, so as to achieve standardized positioning of temporal bone structures or spatial points.

In one embodiment of the present application, a spatial data processing method for a temporal bone is provided. The method comprises the following steps:

receiving a first image containing a bilateral external semicircular vessel bone tube structure introduced by external equipment;

determining a reference surface between the bilateral external semicircular tubular bone pipe structures based on the first image containing the bilateral external semicircular tubular bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system for displaying to a user;

responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system;

and displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values.

In one embodiment of the present application, there is provided a spatial localization method, including: acquiring a point set corresponding to a preset structure in a first image; determining coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system; and highlighting the preset structure and a coordinate value of a point set corresponding to the preset structure in the first image, wherein the temporal bone space coordinate system is a coordinate system established by the data processing method aiming at the temporal bone space.

In one embodiment of the present application, a data processing apparatus is provided. The data processing apparatus includes:

the receiving module is used for receiving a first image which is introduced by external equipment and contains a bilateral external semicircular vessel bone tube structure;

the reference surface determining module is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on the first image containing the bilateral external semicircular vessel bone pipe structures;

the symmetrical characteristic point determining module is used for determining symmetrical characteristic points corresponding to the bilateral external semicircular tubular bone pipe structures according to the reference surface;

the first determining module is used for determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points corresponding to the bilateral external semi-diameter tube bone tube structures respectively;

the second determining module is used for determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

a third determination module for determining a coronal plane from the transverse axis plane and the midsagittal plane;

a coordinate system creating module for creating a temporal bone space coordinate system based on the origin of coordinates, the midsagittal plane, the transverse axis plane and the coronal plane;

the interaction module is used for responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system; and displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values.

In another embodiment of the present application, a spatial locator device is provided. The spatial positioning device includes:

the data processing module is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane;

the acquisition module is used for acquiring a point set corresponding to a preset structure in the first image;

the determining module is used for determining the coordinate value of the point set corresponding to the preset structure based on the temporal bone space coordinate system;

and the display module is used for highlighting and displaying the preset structure and the coordinate value of the point set corresponding to the preset structure in the first image.

In one embodiment of the present application, there is provided an electronic device including: a memory, a processor and a display; wherein the memory is used for storing programs; the processor, coupled with the memory, to execute the program stored in the memory to: receiving a first image containing a bilateral external semicircular vessel bone tube structure introduced by external equipment; determining a reference surface between the bilateral external semicircular tubular bone pipe structures based on the first image containing the bilateral external semicircular tubular bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system for displaying to a user; responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system; and controlling the display to display the coordinate values and the points in the first image in an associated manner, so that a user can perform operations related to the points based on the coordinate values.

The scheme provided by the embodiment of the application can be based on the coordinate origin, the median sagittal plane, the transverse axis plane and the coronal plane which correspond to the bilateral external semicircular canal tubular structures in the first image containing the bilateral external semicircular canal tubular structures to establish a temporal bone space coordinate system. The temporal bone space coordinate system can effectively solve the problem of absolute positioning of temporal bone structures or space points, so as to establish a foundation for researching the temporal bone space position. Namely, after a user triggers a calibration request for a point in an image, the coordinate value of the point can be automatically determined by using the temporal bone space coordinate system, and the coordinate value and the point in the image are displayed in a correlated manner, so that the user can visually see the accurate position of the point, and a good basis is provided for subsequent medical research.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts. In the drawings:

fig. 1 is a schematic flowchart of a spatial data processing method for a temporal bone according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a semi-tubular bone tube structure according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a scanning system according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of three corresponding faces and three axes in human anatomy according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an image to be mapped and a corresponding horizontally flipped image according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a symmetry axis offset corresponding to each layer of an image according to an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of a semicircular tubular bone structure according to yet another embodiment of the present application;

fig. 8 is a schematic flowchart of a spatial positioning method according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a spatial locator device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Before the technical solutions provided by the embodiments of the present application are described, a brief description of specific terms in this document will be provided.

Temporal bone: belongs to one of paired cranial bones in the skull, is positioned at two sides of the skull, is bounded between a sphenoid bone, a parietal bone and an occipital bone, has small volume and irregular shape, has complex internal structure and adjacent relation, is internally provided with a position sensor, an auditory organ, a facial nerve, a vestibular cochlear nerve and an internal carotid artery, and is the finest and most complex structure in the skull.

Semicircular canal bone: is a component of the bone labyrinth and is composed of three mutually perpendicular semicircular canaliculi. Referring to fig. 2, the tubular structure of the semicircular canals is schematically shown, and the tubular structure in the highest position is referred to as an anterior semicircular canals 11, the tubular structure in the latter position is referred to as a posterior semicircular canals 12, and the tubular structure in the approximately horizontal position is referred to as an outer semicircular canals 13. Each semicircular canal bone canal has a single bone foot and a ampulla bone foot, the expansion of the ampulla bone foot is called bone ampulla at the position close to the atrium, and the single bone feet of the front semicircular canal bone canal and the back semicircular canal bone canal are combined into a total bone foot.

The sagittal plane, transverse axis plane and coronal plane are anatomical terms, and their relevant definitions can be specifically referred to the structural diagrams of the corresponding sagittal plane, transverse axis plane and coronal plane in the human anatomy shown in fig. 4. As shown in fig. 4, the sagittal plane 31 is a cross section obtained by longitudinally cutting the human body into left and right halves in the front-rear direction, wherein the cross section obtained by dividing the human body into left and right halves is referred to as a median sagittal plane; the transverse axis plane 32 (also called horizontal plane) is a plane parallel to the ground plane and dividing the human body into an upper part and a lower part; the coronal plane 33 (also called frontal plane) is a cross section obtained by longitudinally cutting the human body into front and rear parts in the left and right directions. The sagittal plane 31, the transverse axis plane 32, and the coronal plane 33 intersect each other two by two and are perpendicular to each other.

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "a" and "an" typically include at least two, but do not exclude the presence of at least one.

It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

It should be understood that although the terms first, second, third, etc. may be used to describe XXX in the embodiments of the present application, these XXX should not be limited to these terms. These terms are only used to distinguish XXX from each other. For example, a first XXX may also be referred to as a second XXX, and similarly, a second XXX may also be referred to as a first XXX, without departing from the scope of embodiments of the present application. The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a monitoring", depending on the context. Similarly, the phrase "if it is determined" or "if it is monitored (a stated condition or event)" may be interpreted as "when determining" or "in response to determining" or "when monitoring (a stated condition or event)" or "in response to monitoring (a stated condition or event)", depending on the context.

It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.

Embodiments of the present application will be described in detail with reference to the drawings and examples, so that how to implement technical means to solve technical problems and achieve technical effects of the present application can be fully understood and implemented.

Currently, temporal bone anatomical structures or landmark points must be located in studies such as spatial position studies of relevant anatomical structures and intelligent detection of anatomical structures. Currently, three methods are mainly used for locating temporal bone structures: the method is characterized in that a relative position method is adopted, namely, one marker structure is used as a reference point to describe the relative position of another structure relative to the marker structure, and the method cannot form a uniform coordinate of any structure of the temporal bone; the second method is a random coordinate method, wherein random coordinate methods are mainly adopted for temporal bone CT, MRI images and the like, the coordinate system of the method is determined according to an acquisition system during image acquisition, the origin of the coordinate is random, the structures of different individuals are different, and the space coordinate is not comparable. And thirdly, a standard coordinate system method is established, a standard space coordinate system taking the total foot bifurcation point of the semicircular canals and the lower edge of the eyeball as references is established, the problem of standardized positioning of the temporal bone structure is solved, but the method is different from the standard layer which is widely applied at present and takes the bilateral external semicircular canals as references, and the practicability is poor. Therefore, there is a need to address the problem of standardized location of temporal bone structures or spatial points. In the following embodiments of the present application, an artificial intelligence technique is also utilized to further improve the precision of the scheme.

Fig. 1 shows a schematic flow chart of a spatial data processing method for a temporal bone according to an embodiment of the present application. The execution main body of the method provided by the embodiment of the present application may be an apparatus, and the apparatus may be, but is not limited to, an apparatus integrated on any terminal device such as a smart phone, a tablet computer, a PDA (Personal Digital Assistant), a smart television, a laptop computer, a desktop computer, and a smart wearable device. As shown in fig. 1, the spatial data processing method for a temporal bone includes:

101. determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures;

102. according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively;

103. determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures;

104. determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

105. determining a coronal plane from the transverse axis plane and the midsagittal plane;

106. based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, a temporal bone space coordinate system is established so as to mark the coordinates of a point in the first image with reference to the temporal bone space coordinate system for showing to a user.

Wherein, the execution sequence of 104 and 101 is not in sequence.

Before step 101, the method may further comprise the steps of: 100. a first image containing a bilateral external semicircular vessel bone tube structure introduced by an external device is received. The external device may: a server storing image data, a storage medium used by a user (e.g., a usb disk, a mobile hard disk), an intelligent terminal used by the user (e.g., a mobile phone, a notebook computer), or an image capturing device, which is not particularly limited in this embodiment.

The following steps may be further included after step 106:

107. responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system;

108. and displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values.

In the embodiment of the present application, the operation performed by the user based on the coordinate value is not specifically limited, and may be a statistical operation, a marking operation, and the like performed for scientific research.

In the foregoing 101, the first image may be an image obtained by analyzing a three-dimensional reconstruction model, where the three-dimensional reconstruction model may be obtained by performing correlation processing based on a series of two-dimensional second images acquired from different angles for the bilateral lateral semicircular canal of a patient by using an existing three-dimensional reconstruction technology (e.g., a 3D printer, and three-dimensional reconstruction software).

Specifically, the three-dimensional reconstructed model can be processed by the following steps:

acquiring a plurality of second images acquired for bilateral external semicircular canals;

performing image recognition on the plurality of second images to obtain three-dimensional modeling parameters of the bilateral lateral semicircular canal;

and constructing a three-dimensional reconstruction model corresponding to the bilateral external semi-diameter tubular bones based on the three-dimensional modeling parameters.

The second images may be tomographic images of bilateral lateral semicircular canals. The plurality of tomographic images may be base data for medical research stored from an archival database. The image recognition of the second images in the above steps can be implemented by using an artificial intelligence technique, for example, the image recognition of the second images is performed by using a neural network model. The plurality of tomographic images may be the subject of the method of the present embodiment introduced by an external device. The external device may be: a server storing archived data, or a storage medium (such as a usb flash disk, a removable hard disk, etc.), or a scanning device (such as a CT device), etc. I.e. the above steps can also be characterized as the following steps:

step 1, receiving a plurality of section scanning image images which are introduced by external equipment and are required by reconstructing a bilateral external semi-diameter tube bone tube model;

and 2, performing three-dimensional modeling on the plurality of section scanning images by using the neural network model to obtain a three-dimensional reconstruction model.

The step 2 may specifically include: identifying the plurality of tomographic images by using the neural network model to obtain three-dimensional modeling parameters; and constructing the three-dimensional reconstruction model based on the three-dimensional modeling parameters. Specifically, the neural network model may include: deep neural network models, recurrent neural network models, convolutional neural network models, and so forth. The plurality of tomographic images are respectively analyzed and identified using a neural network model to extract key features (i.e., three-dimensional modeling parameters) from the respective tomographic images. The neural network model is trained in advance by using training samples. And taking the plurality of tomographic images as input of a neural network model, and executing the neural network model to obtain a pixel point set belonging to bilateral external semicircular canal bone tubes in each tomographic image. These pixel point sets can be used as three-dimensional modeling parameters; the three-dimensional modeling software can reconstruct a three-dimensional model of the bilateral external semicircular canal according to three-dimensional modeling parameters, namely pixel point sets of all fault layers belonging to the bilateral external semicircular canal.

Further, the method further comprises the following execution steps: acquiring a plurality of second images acquired for the bilateral external semicircular vessel bone tube structure; and performing image recognition on the plurality of second images to obtain voxel point information corresponding to the bilateral external semicircular vessel bone tube structure. The voxel point information corresponding to the bilateral external semicircular canal bone tube structure can be as follows: the voxel point information of the tube surface of the bilateral external semicircular canal bone tube or the voxel point information of the tube surface of the bilateral external semicircular canal bone tube structure and the bilateral external semicircular canal bone tube or the voxel point information of the outer line of the right external semicircular canal bone tube structure and the left external semicircular canal bone tube structure or the voxel point information of the inner line of the right external semicircular canal bone tube structure and the left external semicircular canal bone tube structure or the voxel point information of the center line of the right external semicircular canal bone tube structure and the left external semicircular canal bone tube structure. Similarly, the above-mentioned "performing image recognition on the plurality of second images to obtain the voxel point information corresponding to the bilateral external semicircular canal bone structure" may also be implemented by using an artificial intelligence technique, that is, performing image recognition on the plurality of second images by using a neural network model (such as a deep neural network model, a cyclic neural network model, a convolutional neural network model, or the like) to obtain the voxel point information.

Further, the bilateral external semi-gauge tubular bone tube structure comprises: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure; correspondingly, the symmetric feature points include: the left symmetrical characteristic points corresponding to the left-side external semicircular tube bone tube structure and the right symmetrical characteristic points corresponding to the right-side external semicircular tube bone tube structure.

In practical application, in order to ensure that the information in the first image is consistent with the human anatomy structure, the first image needs to obtain a plurality of section scanning images aiming at the bilateral external semicircular canal bone tube structure on the basis of a medical image corresponding to a volunteer individual; wherein the medical image may be obtained by, but not limited to, retrieval from an image database, direct scan acquisition by a scanning device, and the like. When the plurality of section scanning images are used for three-dimensional modeling, the three-dimensional modeling parameters are firstly obtained, and the plurality of section scanning images are modeled according to the three-dimensional modeling parameters to obtain a plurality of reconstructed section images; acquiring construction parameters of a three-dimensional geometric model of the bilateral external semicircular canal bone tube structure according to the plurality of reconstructed section images; and constructing a three-dimensional reconstruction model of the bilateral external semi-diameter tube bone tube structure according to the plurality of reconstruction section images and the construction parameters.

As a practical application scenario, a computer tomography system (i.e., CT system) is shown in fig. 3. Referring to fig. 3, a user (e.g., a scanning device operator) sets scanning parameters (e.g., layer thickness, scanning mode, matrix, tube current amount, etc.) of a CT scanning device 21 through a computer operating console 22 (which may also be referred to as a terminal device), thereby completing scanning of a volunteer and acquiring data information of a plurality of sections acquired by aiming at bilateral external semicircular canals of the volunteer; after the data information is sent to the terminal device 22, the terminal device 22 processes the data information to obtain a corresponding image; the image can be displayed on the display of the terminal device 22, or can be photographed and printed out by a printing device 23 or transmitted to other display terminals through a network, so as to be further analyzed and processed by other people; in addition, the user may also recognize the reconstructed image through an application (e.g., three-dimensional reconstruction software) on the terminal device 22 to obtain three-dimensional modeling parameters of the bilateral lateral semicircular canal bone canal, and then construct a three-dimensional reconstruction model of the bilateral lateral semicircular canal bone canal by using the application based on the three-dimensional modeling parameters.

Further, in the foregoing 101, "determining a reference plane between the bilateral lateral semicircular tubular bone structures based on the first image containing the bilateral lateral semicircular tubular bone structures" may specifically be:

1011. acquiring a horizontal turning image corresponding to each second image in the plurality of second images corresponding to the first image and a preset angle range;

1012. calculating to obtain a correlation score corresponding to each second image by using the second image, the horizontal turnover image and the angle in the preset angle range;

1013. determining the maximum rotation angle based on the corresponding correlation score of each second image;

1014. obtaining values of a layer where each second image in the plurality of second images is located;

1015. acquiring the migration quantity from the symmetry axis of each second image in the plurality of second images to a calibration axis respectively;

1016. and determining the reference surface based on the maximum rotation angle, the migration amount corresponding to each second image and the value of the layer where each second image is located.

Alternatively, the horizontally flipped image may be a mapped image obtained by mapping the second image according to the axis of symmetry.

In 1012, the "calculating the correlation score corresponding to each second image by using the second image, the horizontally flipped image, and the angle within the preset angle range" may be implemented by the following formula:

Score_i(θ_j)＝{corr(P_i，rot(2θ_j，flipH(P_i)))}

wherein, P_iFor the ith second image, theta_jThe jth angle value may be specifically an angle of the symmetry axis corresponding to each of the second images deviating from the vertical center line thereof, where θ is determined according to the preset angle range and the preset angle step length_jIs a preset value. Theta_jThe range of (a), namely the preset angle range, can be-16 degrees to 16 degrees; the corr (·) function is used to calculate the symmetry dependence of the outer semicircular canal; FlipH (P)_i) For the horizontally flipped image corresponding to the ith second image, the rot (-) function is used to calculate the second image P_iHorizontally flipped image flipH (P)_i) Rotation 2 theta_jThe latter image. Score_i(θ_j) The angle of the ith second image in the preset angle range is theta_jThe corresponding relevance score.

It should be noted that the value range of i may be 1-M, and the value range of j is N, where M is the number of all the second images, and N is the number of angles obtained by dividing the angle in the preset angle range into the angles according to the preset angle step length.

With respect to 2 theta_jCan be based onIt is to be understood that bilateral symmetry is defined as: assuming that an image to be mapped is A, and a mapped image generated by mapping the image to be mapped through a symmetry axis is B, the mapped image B is similar to the image to be mapped A. Specifically, referring to the image to be mapped 201 and the corresponding horizontally flipped image 202 (i.e., the image after mapping) shown in fig. 5, if the angle of the symmetry axis 51 of the image to be mapped 201 deviating from the vertical center line 52 is θ, the angle of the symmetry axis 51 of the image after mapping 202 deviating from the vertical center line 52 is — θ, so the image after mapping 202 needs to be rotated by 2 θ to achieve the maximum similarity score with the image to be mapped 201 (i.e., the image after mapping 202 needs to be rotated by 2 θ to achieve the maximum approximation with the image to be mapped 201). Here the angle of rotation 2 theta is required.

In 1013, the "determining the maximum rotation angle based on the correlation score corresponding to each second image" may specifically include:

and traversing all the correlation scores corresponding to the second images, and taking the angle corresponding to the highest correlation score in all the correlation scores corresponding to all the second images as the maximum rotation angle.

In summary, the correlation Score is utilized_i(θ_j) A maximum rotation angle shared by the symmetry axes corresponding to the second images in all the layers where the outer semicircular canal appears can be obtained.

In the above 1016, "determining the reference plane based on the maximum rotation angle, the migration amount corresponding to each second image, and the value of the layer where each image is located" may specifically include:

s21, calculating to obtain a first parameter and a second parameter according to the maximum rotation angle;

s22, determining a third parameter and a fourth parameter according to the migration amount corresponding to each second image and the value of the layer where each second image is located;

s23, substituting the first parameter, the second parameter, the third parameter and the fourth parameter into the following spatial plane mathematical expression to obtain the reference surface:

aX+bY+cZ+d＝0

wherein a is a first parameter, b is a second parameter, c is a third parameter, and d is a fourth parameter.

Specifically, the maximum rotation angle is denoted as θ_maxCan be calculated according to the formula a ═ cos θ_maxAnd b ═ sin θ_maxThe values of the first parameter a and the second parameter b in the spatial plane mathematical expression described in the above step S23 are obtained. In addition, referring to the structural schematic diagram of the symmetry axis offset corresponding to each layer of the second image shown in fig. 6, the migration amount corresponding to each layer of the second image and the value of the layer where each layer of the second image is located can be obtained. As shown in fig. 6, the ith layer of the second image P_iThe migration from the axis of symmetry 51 to a calibration axis Z is designated as tranP_iAnd the value of the layer is recorded as Z_iThen, the formula tranP is used_i＝cZ_i+ d, the third parameter c and the fourth parameter d in the mathematical expression of the spatial plane in step S23 are obtained, where the calibration axis may be the vertical center line corresponding to the second image of each layer. Substituting the first parameter a, the second parameter b, the third parameter c and the fourth parameter d, which are obtained by calculating the migration amount corresponding to each second image and the value of the layer where each second image is located, into the spatial plane mathematical expression in the step S23: and aX + bY + cZ + d is 0, the reference plane can be determined.

Further, the "determining the symmetric feature points corresponding to the bilateral external semicircular canal bone structures according to the reference plane" in the step 102 may specifically include:

1021. searching a voxel point farthest from the reference surface on the left outer semicircular tube bone tube structure as a left characteristic point;

1022. searching a voxel point which is farthest away from the reference surface on the right side outer semicircular tube bone tube structure as a right characteristic point;

1023. re-determining the reference surface based on the left feature point and the right feature point, so as to continuously search a left feature point and a right feature point which are farthest from the re-determined reference surface on the left-side outer semicircular tubular bone pipe structure and the right-side outer semicircular tubular bone pipe structure respectively until an optimal left feature point and an optimal right feature point are searched, and determining an optimal reference surface;

1024. and respectively taking the optimal left characteristic point and the optimal right characteristic point as the respective corresponding symmetrical characteristic points of the bilateral external semicircular tubular bone tube structure.

Specifically, see fig. 7 for a schematic view of the outer semicircular canal. Assuming that the reference plane determined in 1016 above is a plane α, as shown in FIG. 7, the voxel point located farthest from the reference plane α is found on the left lateral external semicircular canal bone structure 41 as a₁Then the voxel is pointed at a₁As a left feature point; finding out the voxel point b farthest from the reference surface alpha on the right external semicircular tubular bone tube structure 42₁Then the voxel is placed at b₁As a right feature point; connecting the voxel points a₁And b₁A straight line a is obtained₁b₁Taking the straight line a₁b₁Corresponding to the midpoint O1, the passing point O1 is perpendicular to the straight line a₁b₁Perpendicular plane alpha of₁Perpendicular to the plane alpha₁As a new said reference surface; and then continuously searching the reference surface alpha with the redetermined distance on the left side outer semicircular tube bone tube structure and the right side outer semicircular tube bone tube structure respectively₁The farthest left characteristic point and the farthest right characteristic point until the optimal left characteristic point (such as the point L) and the optimal right characteristic point (such as the point R) are found, and the optimal reference surface (such as the plane 31) is determined; and respectively taking the optimal left characteristic point and the optimal right characteristic point as symmetrical characteristic points corresponding to the bone tube structures of the bilateral external semi-conventional tubes, namely, the optimal left characteristic point is a left symmetrical characteristic point corresponding to the left external semi-conventional tube structure, and the optimal right characteristic point is a right symmetrical characteristic point corresponding to the right external semi-conventional tube structure.

In some optional embodiments of the present application, determining the symmetric feature points corresponding to each of the bilateral external semicircular canal bone structures according to the reference plane may also be implemented by: searching a voxel point closest to the reference surface on the left outer semicircular tube bone tube structure as a left characteristic point; searching a voxel point closest to the reference surface on the right side outer semicircular tube bone tube structure as a right characteristic point; re-determining the reference surface based on the left feature point and the right feature point, so as to continuously search a left feature point and a right feature point which are closest to the re-determined reference surface on the left-side outer semicircular tubular bone pipe structure and the right-side outer semicircular tubular bone pipe structure respectively until an optimal left feature point and an optimal right feature point are searched, and determining an optimal reference surface; and respectively taking the optimal left characteristic point and the optimal right characteristic point as the respective corresponding symmetrical characteristic points of the bilateral external semicircular tubular bone tube structure.

It should be noted that, since the bilateral external semicircular tubular bone structure is a semicircular tubule, in the process that 1023 repeatedly searches the farthest left and right feature points corresponding to the bilateral external semicircular tubular bone structure according to the newly determined reference surface, the distance from the voxel point corresponding to the middle semicircular tubule (for example, the middle semicircular tubule L1L2 corresponding to the left external semicircular tubular bone structure 41 and the middle semicircular tubule R1R2 corresponding to the right external semicircular tubular bone structure 42 in fig. 7) on the bilateral external semicircular tubular bone structure to the reference surface may also be searched.

Further, the "determining the origin of coordinates and the midsagittal plane according to the symmetric feature points corresponding to the bilateral external semicircular tubular bone tube structures" in the above step 103 may specifically include:

1031. acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure;

1032. determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint;

1033. and taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left outer semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right outer semicircular tubular bone pipe structure as the median sagittal plane.

And the "determining the transverse axial plane by using the voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image" in the above step 104 may specifically include:

fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

Specifically, as shown in fig. 7, assuming that the symmetric characteristic point corresponding to the left-side external semi-circular tubular bone structure 41 is a point L and the symmetric characteristic point corresponding to the right-side external semi-circular tubular bone structure 42 is a point R, a straight line LR is obtained by connecting the point L and the point R, and the midpoint O corresponding to the straight line LR is taken, so that the point O is the coordinate origin. A vertical plane 31 perpendicular to the straight line LR is drawn through the origin of coordinates O, and the vertical plane 31 is the median sagittal plane.

In addition, a plane 32 perpendicular to the midsagittal plane can be fit using known techniques (e.g., least squares) through points L and R, with the plane 32 being the transverse axis plane. It should be noted that: the sum of the distances of the voxel points corresponding to the bilateral external semicircular canal bone tube structure from the transverse axial plane 32 is the smallest.

Further, in the above 105, "determining a coronal plane according to the transverse axis plane and the median sagittal plane" may specifically be: acquiring normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively; determining a normal vector corresponding to the coronal plane by using a right-handed screw rule according to normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively; and determining the coronal plane according to the normal vector corresponding to the coronal plane.

Furthermore, as can be seen from fig. 4, the sagittal plane 31, the transverse axis plane 32 and the coronal plane 33 intersect with each other two by two and are perpendicular to each other, and the intersecting lines of the three lines form a spatial coordinate system. Thus, the "establishing a temporal bone space coordinate system based on the origin of coordinates, the midsagittal plane, the transverse axis plane and the coronal plane" in 106 may specifically include:

1051. acquiring a first straight line which is formed by intersecting the coronal plane and the transverse axis plane and passes through the coordinate origin, and taking the first straight line as an X axis;

1052. acquiring a second straight line which is formed by the intersection of the median sagittal plane and the transverse axis plane and passes through the origin of coordinates, and taking the second straight line as a Y axis;

1053. and acquiring a third straight line which is formed by the intersection of the median sagittal plane and the coronal plane and passes through the coordinate origin, wherein the third straight line is taken as a Z axis.

The above step 106 is described with reference to a practical application scenario. Assuming that the median sagittal plane, the transverse axial plane, and the coronal plane determined by the above steps 103, 104, and 105 correspond to the sagittal plane 31, the transverse axial plane 32, and the coronal plane 33, respectively, shown in fig. 4, a temporal bone space coordinate system may be established based on the origin of coordinates, the transverse axial plane, the coronal plane, and the median sagittal plane. Specifically, the method comprises the following steps: a first straight line which is formed by the intersection of the acquired coronal plane and the transverse axis plane and passes through the coordinate origin can be taken as an X axis, and the positive direction of the coronal axis (namely, the left direction) in the anatomy is taken as the positive direction of the X axis; a second straight line which is formed by the intersection of the median sagittal plane and the transverse axis plane and passes through the coordinate origin is taken as a Y axis, and the positive direction (namely the rear direction) of the sagittal axis in the anatomy is taken as the positive direction of the Y axis; and taking a third straight line which is formed by the intersection of the middle sagittal plane and the coronal plane and passes through the coordinate origin as a Z axis, and taking the positive direction of the vertical axis in the anatomy (namely the upper direction or the head direction) as the positive direction of the Z axis.

Further, the method provided by the embodiment of the present application may further include the following steps:

109. acquiring a point set corresponding to a point to be calibrated and/or a preset structure in a first image;

110. determining coordinate values of the points to be calibrated and/or coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system;

111. and highlighting the point to be marked and the coordinate value of the point to be marked in the first image, and/or the preset structure and the coordinate value of a point set corresponding to the preset structure.

The preset structure may be a preset human body part structure, for example: inner ear structures, cochlear structures, etc. In addition, the coordinate information of all voxel points in the first image in the temporal bone space coordinate system can be determined and displayed based on the temporal bone space coordinate system.

Specifically, a point set corresponding to a fixed point and/or a preset structure is arbitrarily calibrated in the image, and then a specific coordinate value of the point set corresponding to the fixed point to be calibrated and/or the preset structure is determined according to the constructed temporal bone space coordinate system, and the coordinate value can be output. In addition, the coordinate values of the point to be calibrated and the point set corresponding to the point to be calibrated, and/or the coordinate values of the point set corresponding to the preset structure and the preset structure may be highlighted in the image, where the highlighting may be performed by color highlighting or thickening the coordinate values of the point to be calibrated and the preset structure. And the coordinate values of the point set corresponding to the calibration point and/or the preset structure can be displayed in an (x, y, z) mode.

The embodiment provides a technical scheme, a temporal bone space coordinate system can be constructed based on the coordinate origin, the median sagittal plane, the transverse axis plane and the coronal plane which are obtained in the first image and correspond to the bilateral external semi-radial tube bone tube structures, and a straight line which is formed by pairwise intersection of the coordinate origin is passed through, so that the problem of absolute positioning of temporal bone structures or space points is solved, and a foundation is established for researching the temporal bone space position. Meanwhile, the integrity of the bilateral images can be effectively ensured by utilizing the temporal bone space coordinate system, so that the medical image reading system is more consistent with the current medical habit in clinical image reading, and a basis is provided for the automatic image processing calculation of a computer.

Fig. 8 shows a schematic flow chart of a spatial location method according to an embodiment of the present application. The method comprises the following steps:

601. acquiring a point set corresponding to a point to be calibrated and/or a preset structure in a first image;

602. determining coordinate values of the points to be calibrated and/or coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system;

603. and highlighting the point to be marked and the coordinate value of the point to be marked in the first image, and/or the preset structure and the coordinate value of a point set corresponding to the preset structure, wherein the temporal bone space coordinate system is a coordinate system established by the temporal bone space data processing method.

Fig. 9 illustrates a data processing apparatus according to an embodiment of the present application. As shown in fig. 9, the data processing apparatus includes:

the reference surface determining module 61 is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures;

a symmetrical feature point determining module 62, configured to determine, according to the reference plane, symmetrical feature points corresponding to the bilateral external semi-diameter tube bone tube structures respectively;

the first determining module 63 is configured to determine an origin of coordinates and a median sagittal plane according to respective corresponding symmetric feature points of the bilateral external semi-diameter tubular bone structures;

a second determining module 64, configured to determine a transverse axial plane by using voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

a third determination module 65 for determining a coronal plane from said transverse axis plane and said median sagittal plane;

a coordinate system creating module 66, configured to create a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system, so as to be displayed to a user.

Further, the apparatus provided in this embodiment may further include a receiving module 67 for receiving a first image of a bilateral external semicircular vessel bone tube structure introduced by an external device.

Still further, the apparatus provided in this embodiment may further include an interaction module 68. The interaction module 68 is configured to determine, in response to a user's calibration request for a point in the first image, a coordinate value of the point using the temporal bone space coordinate system; and displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values.

Further, the data processing apparatus provided in this embodiment further includes an obtaining module, configured to obtain a plurality of second images acquired for the bilateral external semicircular tubular bone structure; and performing image recognition on the plurality of second images to obtain voxel point information corresponding to the bilateral external semicircular vessel bone tube structure.

Further, the bilateral external semicircular canal bone structure may include: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure; correspondingly, the symmetric feature points include: the left symmetrical characteristic points corresponding to the left-side external semicircular tube bone tube structure and the right symmetrical characteristic points corresponding to the right-side external semicircular tube bone tube structure.

Further, the reference surface determining module 61 is specifically configured to determine a reference surface between the bilateral lateral semicircular tubular bone structures based on the first image containing the bilateral lateral semicircular tubular bone structures, and to: acquiring a horizontal turning image corresponding to each second image in the plurality of second images corresponding to the first image and a preset angle range; calculating to obtain a correlation score corresponding to each second image by using the second image, the horizontal turnover image and the angle in the preset angle range; determining the maximum rotation angle based on the corresponding correlation score of each second image; obtaining values of a layer where each second image in the plurality of second images is located; acquiring the migration quantity from the symmetry axis of each second image in the plurality of second images to a calibration axis respectively; and determining the reference surface based on the maximum rotation angle, the migration amount corresponding to each second image and the value of the layer where each second image is located.

Further, when the reference surface determining module 61 is configured to determine the maximum rotation angle based on the correlation score corresponding to each second image, specifically: and traversing all the correlation scores corresponding to the second images, and taking the angle corresponding to the highest correlation score in all the correlation scores corresponding to all the second images as the maximum rotation angle.

Further, the reference plane determining module 61 is configured to determine the reference plane based on the maximum rotation angle, the migration amount corresponding to each second image, and a value of a layer where each second image is located, and specifically configured to: calculating to obtain a first parameter and a second parameter according to the maximum rotation angle; determining a third parameter and a fourth parameter according to the migration quantity corresponding to each second image and the value of the layer where each second image is located; substituting the first parameter, the second parameter, the third parameter and the fourth parameter into the following spatial plane mathematical expression to obtain the reference surface:

aX+bY+cZ+d＝0

wherein a is a first parameter, b is a second parameter, c is a third parameter, and d is a fourth parameter.

Further, when the symmetric feature point determining module 62 determines the symmetric feature points corresponding to the bilateral external semicircular canal bone structures according to the reference plane, it is specifically configured to: searching a voxel point farthest from the reference surface on the left outer semicircular tube bone tube structure as a left characteristic point; searching a voxel point which is farthest away from the reference surface on the right side outer semicircular tube bone tube structure as a right characteristic point; re-determining the reference surface based on the left feature point and the right feature point, so as to continuously search a left feature point and a right feature point which are farthest from the re-determined reference surface on the left-side outer semicircular tubular bone pipe structure and the right-side outer semicircular tubular bone pipe structure respectively until an optimal left feature point and an optimal right feature point are searched, and determining an optimal reference surface; and respectively taking the optimal left characteristic point and the optimal right characteristic point as the respective corresponding symmetrical characteristic points of the bilateral external semicircular tubular bone tube structure.

Further, the first determining module 63 is configured to determine the origin of coordinates and the midsagittal plane according to the symmetric feature points corresponding to the bilateral external semicircular tubular bone structures, and specifically configured to: acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure; determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint; taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left lateral external semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right lateral external semicircular tubular bone pipe structure as the median sagittal plane; and further, the second determining module 64, when configured to determine a transverse plane by using voxel point information corresponding to the bilateral external semicircular canal bone structure in the first image, is specifically configured to: fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

Further, the third determination module 65, when configured to determine a coronal plane from the transverse axis plane and the median sagittal plane, is specifically configured to: acquiring normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively; determining a normal vector corresponding to the coronal plane by using a right-handed screw rule according to normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively; and determining the coronal plane according to the normal vector corresponding to the coronal plane.

Further, the coordinate system creating module 66, when configured to establish a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane, is specifically configured to: acquiring a first straight line which is formed by intersecting the coronal plane and the transverse axis plane and passes through the coordinate origin, and taking the first straight line as an X axis; acquiring a second straight line which is formed by the intersection of the median sagittal plane and the transverse axis plane and passes through the origin of coordinates, and taking the second straight line as a Y axis; and acquiring a third straight line which is formed by the intersection of the median sagittal plane and the coronal plane and passes through the coordinate origin, wherein the third straight line is taken as a Z axis.

Further, the data processing apparatus further comprises a positioning module configured to: acquiring a point set corresponding to a point to be calibrated and/or a preset structure in a first image; determining coordinate values of the points to be calibrated and/or coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system; and highlighting the point to be marked and the coordinate value of the point to be marked in the first image, and/or the preset structure and the coordinate value of a point set corresponding to the preset structure.

The working principle and process of each module of the data processing apparatus provided in the embodiment of the present application may refer to the corresponding content in the above method embodiment, and are not described herein again.

Fig. 10 shows a schematic structural diagram of a spatial positioning apparatus provided in an embodiment of the present application. As shown, the spatial locating device comprises:

the data processing module 81 is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane;

an obtaining module 82, configured to obtain a point set corresponding to a preset structure in a first image;

a determining module 83, configured to determine, based on the temporal bone space coordinate system, a coordinate value of a point set corresponding to the preset structure;

a display module 84, configured to highlight and display the preset structure and the coordinate values of the point set corresponding to the preset structure in the first image.

The working principle and process of the data processing module in the embodiment of the present application may refer to the content in the above method embodiment, which is not described herein again.

Fig. 11 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 11, the electronic apparatus includes: a memory 71, a processor 72, and a display 73; wherein the content of the first and second substances,

the memory 71 is used for storing programs;

the processor 72, coupled to the memory, is configured to execute the program stored in the memory to:

receiving a first image containing a bilateral external semicircular vessel bone tube structure introduced by external equipment;

determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system for displaying to a user;

responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system;

and controlling the display to display the coordinate values and the points in the first image in an associated manner, so that a user can perform operations related to the points based on the coordinate values.

The memory 71 described above may be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. The memory 71 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The processor 72 may also perform other functions besides the above functions when executing the program in the memory 71, which may be specifically referred to the description of the foregoing embodiments.

Further, as shown in fig. 11, the electronic apparatus further includes: power components 74, communication components 75, and the like. Only some of the components are schematically shown in fig. 11, and it is not meant that the electronic device includes only the components shown in fig. 11.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the steps or functions of the spatial data processing method for the temporal bone provided in the foregoing embodiments.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (14)

1. A spatial data processing method for a temporal bone, comprising:

receiving a first image containing a bilateral external semicircular vessel bone tube structure introduced by external equipment;

determining a reference surface between the bilateral external semicircular tubular bone pipe structures based on the first image containing the bilateral external semicircular tubular bone pipe structures;

according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively;

determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures;

determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

determining a coronal plane from the transverse axis plane and the midsagittal plane;

establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system for displaying to a user;

responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system;

displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values;

wherein, the outer semi-normal pipe tubular bone structure of bilateral side includes: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure;

according to the symmetrical characteristic point that two side outer semi-diameter tubular bone tube structures correspond respectively, confirm coordinate origin and median sagittal plane, include:

acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure;

determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint;

taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left lateral external semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right lateral external semicircular tubular bone pipe structure as the median sagittal plane; and

determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image, wherein the method comprises the following steps:

fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

2. The method of claim 1, further comprising:

acquiring a plurality of second images acquired for the bilateral external semicircular vessel bone tube structure;

and carrying out image recognition on the plurality of second images by utilizing a neural network model so as to obtain voxel point information corresponding to the bilateral external semicircular vessel bone tube structure.

3. The method of claim 2, wherein the symmetric feature points comprise: the left symmetrical characteristic points corresponding to the left-side external semicircular tube bone tube structure and the right symmetrical characteristic points corresponding to the right-side external semicircular tube bone tube structure.

4. The method of claim 3, wherein determining a reference plane between the bilateral lateral semicircular tubular bone structures based on the first image containing the bilateral lateral semicircular tubular bone structures comprises:

acquiring a horizontal turning image corresponding to each second image in the plurality of second images corresponding to the first image and a preset angle range;

calculating to obtain a correlation score corresponding to each second image by using the second image, the horizontal turnover image and the angle in the preset angle range;

determining the maximum rotation angle based on the corresponding correlation score of each second image;

obtaining values of a layer where each second image in the plurality of second images is located;

acquiring the migration quantity from the symmetry axis of each second image in the plurality of second images to a calibration axis respectively;

and determining the reference surface based on the maximum rotation angle, the migration amount corresponding to each second image and the value of the layer where each second image is located.

5. The method of claim 4,

determining the maximum rotation angle based on the correlation scores corresponding to the second images comprises:

and traversing all the correlation scores corresponding to the second images, and taking the angle corresponding to the highest correlation score in all the correlation scores corresponding to all the second images as the maximum rotation angle.

6. The method according to claim 4, wherein determining the reference plane based on the maximum rotation angle, the migration amount corresponding to each second image, and the value of the layer in which each second image is located comprises:

calculating to obtain a first parameter and a second parameter according to the maximum rotation angle;

determining a third parameter and a fourth parameter according to the migration quantity corresponding to each second image and the value of the layer where each second image is located;

substituting the first parameter, the second parameter, the third parameter and the fourth parameter into the following spatial plane mathematical expression to obtain the reference surface:

aX+bY+cZ+d＝0

wherein a is a first parameter, b is a second parameter, c is a third parameter, and d is a fourth parameter.

7. The method of claim 6, wherein determining the respective symmetric feature points of the bilateral, external semi-gauge tubular bone structures from the reference surface comprises:

searching a voxel point farthest from the reference surface on the left outer semicircular tube bone tube structure as a left characteristic point;

searching a voxel point which is farthest away from the reference surface on the right side outer semicircular tube bone tube structure as a right characteristic point;

re-determining the reference surface based on the left feature point and the right feature point, so as to continuously search a left feature point and a right feature point which are farthest from the re-determined reference surface on the left-side outer semicircular tubular bone pipe structure and the right-side outer semicircular tubular bone pipe structure respectively until an optimal left feature point and an optimal right feature point are searched, and determining an optimal reference surface;

and respectively taking the optimal left characteristic point and the optimal right characteristic point as the respective corresponding symmetrical characteristic points of the bilateral external semicircular tubular bone tube structure.

8. The method of claim 1, wherein determining a coronal plane from the transverse axis plane and the midsagittal plane comprises:

acquiring normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively;

determining a normal vector corresponding to the coronal plane by using a right-handed screw rule according to normal vectors corresponding to the transverse axis plane and the median sagittal plane respectively;

and determining the coronal plane according to the normal vector corresponding to the coronal plane.

9. The method of claim 8, wherein establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane comprises:

acquiring a first straight line which is formed by intersecting the coronal plane and the transverse axis plane and passes through the coordinate origin, and taking the first straight line as an X axis;

acquiring a second straight line which is formed by the intersection of the median sagittal plane and the transverse axis plane and passes through the origin of coordinates, and taking the second straight line as a Y axis;

and acquiring a third straight line which is formed by the intersection of the median sagittal plane and the coronal plane and passes through the coordinate origin, wherein the third straight line is taken as a Z axis.

10. The method of any one of claims 1 to 9, further comprising:

acquiring a point set corresponding to a preset structure in the first image;

determining coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system;

and highlighting the preset structure and the coordinate value of the point set corresponding to the preset structure in the first image.

11. A spatial localization method, comprising:

acquiring a point set corresponding to a preset structure in a first image;

determining coordinate values of a point set corresponding to the preset structure based on the temporal bone space coordinate system;

and highlighting the preset structure and coordinate values of a point set corresponding to the preset structure in the first image, wherein the temporal bone space coordinate system is a coordinate system established by the temporal bone space data processing method according to any one of claims 1 to 10.

12. A data processing apparatus, comprising:

the receiving module is used for receiving a first image which is introduced by external equipment and contains a bilateral external semicircular vessel bone tube structure;

the reference surface determining module is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on the first image containing the bilateral external semicircular vessel bone pipe structures;

the symmetrical characteristic point determining module is used for determining symmetrical characteristic points corresponding to the bilateral external semicircular tubular bone pipe structures according to the reference surface;

the first determining module is used for determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points corresponding to the bilateral external semi-diameter tube bone tube structures respectively;

the second determining module is used for determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

a third determination module for determining a coronal plane from the transverse axis plane and the midsagittal plane;

a coordinate system creating module for creating a temporal bone space coordinate system based on the origin of coordinates, the midsagittal plane, the transverse axis plane and the coronal plane;

the interaction module is used for responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system; displaying the coordinate values and the points in the first image in an associated mode, so that a user can execute operations related to the points based on the coordinate values;

wherein, the outer semi-normal pipe tubular bone structure of bilateral side includes: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure;

the first determining module is specifically configured to:

acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure;

determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint;

taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left lateral external semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right lateral external semicircular tubular bone pipe structure as the median sagittal plane; and

the second determining module is specifically configured to:

fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

13. A spatial locator device, comprising:

the data processing module is used for determining a reference surface between the bilateral external semicircular vessel bone pipe structures based on a first image containing the bilateral external semicircular vessel bone pipe structures; according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively; determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures; determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image; determining a coronal plane from the transverse axis plane and the midsagittal plane; establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane, and the coronal plane;

the acquisition module is used for acquiring a point set corresponding to a preset structure in the first image;

the determining module is used for determining the coordinate value of the point set corresponding to the preset structure based on the temporal bone space coordinate system;

the display module is used for highlighting and displaying the preset structure and the coordinate value of the point set corresponding to the preset structure in the first image;

wherein, the outer semi-normal pipe tubular bone structure of bilateral side includes: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure;

the data processing module is specifically configured to, when determining the origin of coordinates and the midsagittal plane:

acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure;

determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint;

taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left lateral external semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right lateral external semicircular tubular bone pipe structure as the median sagittal plane;

the data processing module is specifically configured to, when determining the horizontal plane:

fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

14. An electronic device, comprising: a memory, a processor and a display; wherein the content of the first and second substances,

the memory is used for storing programs;

the processor, coupled with the memory, to execute the program stored in the memory to:

receiving a first image containing a bilateral external semicircular vessel bone tube structure introduced by external equipment;

determining a reference surface between the bilateral external semicircular tubular bone pipe structures based on the first image containing the bilateral external semicircular tubular bone pipe structures;

according to the reference surface, determining symmetrical characteristic points corresponding to the bilateral external semicircular tube bone tube structures respectively;

determining a coordinate origin and a median sagittal plane according to the symmetrical characteristic points of the bilateral external semi-diameter tube bone tube structures;

determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image;

determining a coronal plane from the transverse axis plane and the midsagittal plane;

establishing a temporal bone space coordinate system based on the origin of coordinates, the median sagittal plane, the transverse axis plane and the coronal plane, so as to calibrate coordinates of a point in the first image with reference to the temporal bone space coordinate system for displaying to a user;

responding to a calibration request of a user for one point in the first image, and determining a coordinate value of the point by using the temporal bone space coordinate system;

controlling the display to display the coordinate values in association with the points in the first image for a user to perform operations related to the points based on the coordinate values;

wherein, the outer semi-normal pipe tubular bone structure of bilateral side includes: the left side external semi-diameter tube bone tube structure and the right side external semi-diameter tube bone tube structure;

according to the symmetrical characteristic point that two side outer semi-diameter tubular bone tube structures correspond respectively, confirm coordinate origin and median sagittal plane, include:

acquiring symmetrical characteristic points corresponding to the left-side external semicircular vessel bone tube structure and the middle point of a connecting line of the symmetrical characteristic points corresponding to the right-side external semicircular vessel bone tube structure;

determining the origin of coordinates of the bilateral external semicircular canal bone tube structure according to the midpoint;

taking a plane which passes through the coordinate origin and is perpendicular to a connecting line of the symmetrical characteristic points corresponding to the left lateral external semicircular tubular bone pipe structure and the symmetrical characteristic points corresponding to the right lateral external semicircular tubular bone pipe structure as the median sagittal plane; and

determining a transverse axial plane by utilizing voxel point information corresponding to the bilateral external semicircular canal bone tube structure in the first image, wherein the method comprises the following steps:

fitting a plane perpendicular to the sagittal plane according to the voxel point information corresponding to the left lateral external semicircular canal bone tube structure and the voxel point information corresponding to the right lateral external semicircular canal bone tube structure, and taking the plane as the transverse axis plane; and the sum of the distances from all voxel points on the bilateral external semicircular canal bone tube structure to the transverse axial plane is minimum.

Images