CN108354578A - A kind of capsule endoscope positioning system - Google Patents

A kind of capsule endoscope positioning system Download PDF

Info

Publication number
CN108354578A
CN108354578A CN201810210665.3A CN201810210665A CN108354578A CN 108354578 A CN108354578 A CN 108354578A CN 201810210665 A CN201810210665 A CN 201810210665A CN 108354578 A CN108354578 A CN 108354578A
Authority
CN
China
Prior art keywords
alimentary canal
capsule endoscope
picture
network model
depth network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810210665.3A
Other languages
Chinese (zh)
Other versions
CN108354578B (en
Inventor
袁建
白家莲
梁东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Jinshan Medical Technology Research Institute Co Ltd
Original Assignee
Chongqing Jinshan Medical Appliance Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Jinshan Medical Appliance Co Ltd filed Critical Chongqing Jinshan Medical Appliance Co Ltd
Priority to CN201810210665.3A priority Critical patent/CN108354578B/en
Publication of CN108354578A publication Critical patent/CN108354578A/en
Application granted granted Critical
Publication of CN108354578B publication Critical patent/CN108354578B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/041—Capsule endoscopes for imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147—Holding or positioning arrangements
    • A61B1/00158—Holding or positioning arrangements using magnetic field
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0661—Endoscope light sources
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/273—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Optics & Photonics (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Signal Processing (AREA)
  • Gastroenterology & Hepatology (AREA)
  • Endoscopes (AREA)

Abstract

The invention discloses a kind of capsule endoscope positioning systems, including：Acquisition module is used for：Obtain the alimentary canal picture of capsule endoscope acquisition, the picture luminance of the alimentary canal picture and the lens parameters for acquiring the camera lens for realizing picture collection when the alimentary canal picture in the capsule endoscope；Locating module is used for：The alimentary canal picture is input in depth network model trained in advance, obtains the alimentary canal position corresponding with the alimentary canal picture of the depth network model output；And the distance between described distance of camera lens alimentary canal mucous membrane when determining acquisition corresponding with the picture luminance and the lens parameters alimentary canal picture based on predetermined correspondence；Output module is used for：The capsule endoscope is acquired to the posture information output when alimentary canal picture, which includes that the alimentary canal picture corresponds to the distance between alimentary canal position and the distance of camera lens alimentary canal mucous membrane.To realize to capsule endoscope being accurately positioned in vivo.

Description

A kind of capsule endoscope positioning system
Technical field
The present invention relates to the field of medical instrument technology, more specifically to a kind of capsule endoscope positioning system.
Background technology
The image or video of human body alimentary canal inner wall are shot while capsule endoscope is used to move in human body alimentary canal, and Wirelessly reach external signal receiving device, compared with conventional medical endoscope, capsule endoscope have it is easy to operate, Hurtless measure, no pain, without cross-infection, do not influence patient normal work the advantages that, especially to disease of intestine inspection have it is very high Medical diagnosis value.
The movement of capsule endoscope in vivo is divided into：Passive type and two kinds active, passive type are capsule endoscope with digestion Road wriggles and advances, and movement has uncontrollable and randomness, it is understood that there may be the excessive imaging of certain anatomical positions or missing inspection； And active is capsule endoscope as the control of external externally-applied magnetic field advances, retreat, pitching or rolling, movement tool There is controllability, can carry out more efficient to alimentary canal or lesions position and comprehensively check.
For active capsule endoscope (later abbreviation capsule endoscope), capsule endoscope residing dissection position in vivo The automatic positioning set, is necessary checking process, the automatic positioning and parsing of capsule endoscope residing anatomical position in vivo, Can to operator provide capsule endoscope residing for anatomical position judgement, lesions position can preferably be carried out different direction and The observation of scale contributes to operator to formulate and more fully checks that route avoids missing inspection.The identifying schemes of alimentary canal position at present Mostly it is to rely on the image data that capsule endoscope is passed back by medical staff substantially to identify the location of capsule endoscope, It is easy to be influenced by the subjective factor of medical staff, cannot achieve capsule endoscope being accurately positioned in vivo.
In conclusion how to provide it is a kind of can realize the pinpoint technical solution of capsule endoscope in vivo, be mesh Preceding those skilled in the art's urgent problem to be solved.
Invention content
The object of the present invention is to provide a kind of capsule endoscope positioning system, to realize capsule endoscope in vivo accurate fixed Position.
To achieve the goals above, the present invention provides the following technical solutions：
A kind of capsule endoscope positioning system, including：
Acquisition module is used for：It obtains the alimentary canal picture of capsule endoscope acquisition, the picture luminance of the alimentary canal picture and adopts Collect the lens parameters for the camera lens for realizing picture collection when the alimentary canal picture in the capsule endoscope；
Locating module is used for：The alimentary canal picture is input in depth network model trained in advance, is obtained described The alimentary canal position corresponding with the alimentary canal picture of depth network model output；And it is true based on predetermined correspondence Make distance of camera lens alimentary canal when acquisition corresponding with the picture luminance and the lens parameters alimentary canal picture The distance between mucous membrane；
Output module is used for：The capsule endoscope is acquired to the posture information output when alimentary canal picture, the pose Information includes that the alimentary canal picture corresponds to the distance between alimentary canal position and the distance of camera lens alimentary canal mucous membrane.
Preferably, further include：
Attitude Calculation module, is used for：Detect the acceleration when capsule endoscope acquires the alimentary canal picture；Judge institute The outside for stating alimentary canal position residing for capsule endoscope presets whether magnetic field meets preset condition, if it is, detecting the capsule The magnetic induction intensity of alimentary canal position residing for scope, and the magnetic induction intensity and the acceleration are substituted into preset formula simultaneously Calculate the attitude angle of the capsule endoscope；If it is not, then detect the capsule endoscope around itself presetting the dynamic angular speed of three shaft rotations, And each angular speed is carried out to the attitude angle of capsule endoscope described in integral and calculating respectively；
The attitude angle of the capsule endoscope is added into the posture information.
Preferably, the Attitude Calculation module includes：
First computing unit, is used for：Pass through formula
Calculate the pitch angle pitch of the capsule endoscope；
Pass through formula
Calculate the roll angle roll of the capsule endoscope；
Pass through formula
Yaw=ξ+θ
Calculate the yaw angle yaw of the capsule endoscope；And
Wherein, ξ indicates the deflection angle when external default magnetic field meets preset condition in the horizontal direction；θ isWith Angle in the horizontal direction；For the x-axis base vector of the capsule endoscope, if For the capsule endoscope Y-axis base vector, if For the z-axis base vector of the capsule endoscope, if It is described Capsule endoscope acquires the acceleration when alimentary canal picture, if To disappear residing for the capsule endoscope The magnetic induction intensity of the positions Hua Dao, if ForThrowing in the x-y plane of the capsule endoscope Shadow, if AndWhenAnd A ∈ R, A>When 0, roll=| Roll |, whenAnd when A ∈ R, A≤0, roll=- | roll |；WhenAnd A ∈ R, A>When 0, θ =| θ |, whenAnd when A ∈ R, A≤0, θ=- | θ |.
Preferably, the Attitude Calculation module includes：
Second computing unit, is used for：Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting x-axis rotation；
Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting y-axis rotation；
Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting z-axis rotation；
Then the attitude angle of the capsule endoscope can be expressed as by spin matrix：
Wherein, α0、β0And γ0Respectively capsule endoscope described in integral constant item is around the initial rotation angle for itself presetting x-axis Degree, the capsule endoscope turn around the initial rotation angle and the capsule endoscope for itself presetting y-axis around the initial of z-axis itself is preset Dynamic angle；ωx、ωyAnd ωzThe respectively described capsule endoscope is around itself presetting the dynamic angular speed of three shaft rotations, the integral constant item For the default magnetic field in outside is being changed into the glue for being unsatisfactory for the instantaneous moment of preset condition and being calculated from meeting preset condition The attitude angle of intracapsular mirror.
Preferably, further include：
Model training module is used for：Training set and test set are obtained, includes digestion in the training set and the test set Road picture and each alimentary canal picture of expression correspond to the label of alimentary canal position；
The depth network model based on deep learning frame is chosen as current depth network model, utilizes the training set Depth network model is trained, the depth network model tested after training using the test set obtains depth network model Accuracy of identification data, judge whether the accuracy of identification data meet default required precision, if so, determine training after depth Network model is the depth network model for completing training, if not, it is determined that after being adjusted to the depth network model after training Obtained depth network model is depth network model, and return execution is described to carry out depth network model using the training set Trained step.
Preferably, further include：
Preprocessing module is used for：After obtaining the training set and the test set, the training set and the survey are determined It is unknown class picture that examination concentration corresponding alimentary canal position, which is unknown alimentary canal picture, and using described in the exclusion of perceptual hash algorithm Similarity is more than the picture of predetermined threshold value in unknown class picture；
Predetermined angle rotation processing and image are carried out to the alimentary canal picture for including in the training set and the test set Enhancing is handled.
Preferably, the model training module includes：
First training unit, is used for：The alimentary canal picture for including in the training set is combined into multiple sub- training sets, In every sub- training set alimentary canal picture for including it is not exactly the same；Depth network is respectively trained using the multiple sub- training set Model obtains corresponding multiple depth network models, and is surveyed respectively to multiple depth network model using the test set Examination obtains the accuracy of identification data of corresponding depth network model, chooses corresponding accuracy of identification statistics indicate that the highest depth of accuracy of identification Degree network model is the depth network model after being trained to depth network model using the training set.
Preferably, the model training module includes：
Second training unit, is used for：It is pressed using the test set test depth network model, and based on test acquired results The recognition correct rate and positive prediction rate that the accuracy of identification data that the depth network model is calculated according to following equation include：
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in recognition correct rate=test set/ The total quantity * 100% of the alimentary canal picture of alimentary canal position is corresponded in test set；
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in positive prediction rate=test set/ The total quantity * 100% of the alimentary canal picture of the correspondence alimentary canal position automatically identified in test set.
Preferably, further include：
Discrimination module is used for：Before posture information output when the capsule endoscope to be acquired to the alimentary canal picture, sentence Whether the disconnected alimentary canal position is unknown class, if it is, the posture information exported apart from the last time at moment is carried out defeated Go out, if it is not, then indicating that the output module executes the pose when capsule endoscope to acquire to the alimentary canal picture The step of information exports.
Preferably, the output module includes：
Display unit is used for：The pose of pre-rendered simulation capsule endoscope is arranged to, is acquired with the capsule endoscope The corresponding pose of posture information when the alimentary canal picture, and the simulation capsule endoscope is shown.
The present invention provides a kind of capsule endoscope positioning system, which includes：Acquisition module is used for：It obtains in capsule It the alimentary canal picture of mirror acquisition, the picture luminance of the alimentary canal picture and acquires when the alimentary canal picture real in the capsule endoscope The lens parameters of the camera lens of existing picture collection；Locating module is used for：The alimentary canal picture is input to depth trained in advance In network model, the alimentary canal position corresponding with the alimentary canal picture of the depth network model output is obtained；And it is based on Predetermined correspondence determines the acquisition alimentary canal picture corresponding with the picture luminance and the lens parameters The distance between Shi Suoshu distance of camera lens alimentary canal mucous membranes；Output module is used for：The capsule endoscope is acquired into the alimentary canal Posture information output when picture, which includes that the alimentary canal picture corresponds to alimentary canal position and the distance of camera lens The distance between alimentary canal mucous membrane.Technical solution provided by the invention is realized by depth network model to alimentary canal picture pair It answers the identification of alimentary canal position, and is determined pair by the lens parameters of picture luminance and alimentary canal picture collection moment camera lens The distance between camera lens and alimentary canal mucous membrane for answering, to realize to capsule endoscope being accurately positioned in vivo, and will it is final must To posture information exported, realized for operator and further check that programme path has good directive significance.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of capsule endoscope localization method provided in an embodiment of the present invention；
Fig. 2 is the three-axis reference of capsule endoscope and inspection in a kind of capsule endoscope localization method provided in an embodiment of the present invention The schematic diagram of the acceleration, magnetic field vector that measure；
Fig. 3 is a kind of structural schematic diagram of capsule endoscope positioning system provided in an embodiment of the present invention；
Fig. 4 is a kind of capsule endoscope localization method provided in an embodiment of the present invention and system corresponds to the structure of hardware system and shows It is intended to.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.
It, can be with referring to Fig. 1, it illustrates a kind of flow chart of capsule endoscope localization method provided in an embodiment of the present invention Including：
S11：It obtains the alimentary canal picture of capsule endoscope acquisition, the picture luminance of the alimentary canal picture and acquires the alimentary canal The lens parameters of the camera lens of picture collection are realized when picture in capsule endoscope.
Wherein, the lens parameters of camera lens may include the gain of camera lens, time for exposure etc., and the capsule endoscope in the application Magnetic control capsule endoscope can specifically be referred to, corresponding alimentary canal figure can be obtained by picture collection of the capsule endoscope in human body Piece.A kind of execution subject of capsule endoscope localization method provided in an embodiment of the present invention can be pair The capsule endoscope localization method answered.
S12：Alimentary canal picture is input in depth network model trained in advance, obtains the output of depth network model Alimentary canal corresponding with alimentary canal picture position；And it determines to join with picture luminance and camera lens based on predetermined correspondence The distance between distance of camera lens alimentary canal mucous membrane when number corresponding acquisition alimentary canal picture.
The alimentary canal picture that capsule endoscope acquires is input to the depth network model that training is completed in advance, obtains depth net The corresponding alimentary canal position of the alimentary canal picture of network model output.The alimentary canal picture of wherein capsule endoscope acquisition is to need The alimentary canal picture is input to the depth network mould of training completion by the alimentary canal picture for identifying its corresponding alimentary canal position Type, depth network model can then export the corresponding alimentary canal position of the alimentary canal picture, to realize to the alimentary canal picture The identification of alimentary canal position.
The distance of the distance of camera lens alimentary canal mucous membrane of capsule endoscope, Ke Yishi are calculated according to the pictorial information of alimentary canal picture It is obtained by the correspondence between synchronization picture luminance, lens parameters and distance of camera lens alimentary canal mucous membrane, the relationship It can be experiment gained, be illustrated for the gain of camera lens and for the time for exposure by lens parameters in the application, above-mentioned correspondence The acquisition process of relationship can be：Experimental simulation vivo environment, in the effective shooting distance of camera lens of capsule endoscope, by determining step It is long to change the distance between object and capsule endoscope, in the case of object and certain capsule endoscope distance, adjust capsule The gain of the camera lens of scope and time for exposure obtain related different imagings by calculating the collected picture luminance of capsule endoscope The gain of the camera lens of distance (image-forming range is the distance between distance of camera lens alimentary canal mucous membrane) and capsule endoscope, time for exposure Mapping table between picture luminance.In specific implementation process, image-forming range and capsule endoscope camera lens gain, exposure Relationship between time and picture luminance corresponds to table and is obtained by testing, and is stored in and realizes technology provided in an embodiment of the present invention In the positioning system of scheme, pass through gain, the exposure of the picture luminance and combination capsule endoscope camera lens of the alimentary canal picture of acquisition Time, by testing the mapping table obtained, the camera lens and object for obtaining capsule endoscope (can refer to alimentary canal in the application The distance between mucous membrane).
S13：Capsule endoscope is acquired to posture information output when alimentary canal picture, which includes alimentary canal picture Corresponding the distance between alimentary canal position and distance of camera lens alimentary canal mucous membrane.
Can be wherein it will be shown on corresponding display unit, so that work for the output of above-mentioned posture information Personnel can intuitively be rapidly obtained the posture information of capsule endoscope based on the information shown on display unit, naturally it is also possible to Other settings are carried out according to actual needs within protection scope of the present invention.It is further to note that the present invention is implemented Posture information in example corresponds to same alimentary canal picture, namely the pose of corresponding synchronization capsule endoscope.
Technical solution provided by the invention corresponds to alimentary canal position by the realization of depth network model to alimentary canal picture Identification, and determine corresponding camera lens and digestion by the lens parameters of picture luminance and alimentary canal picture collection moment camera lens The distance between mucous membrane, to realize to capsule endoscope being accurately positioned in vivo, and by finally obtained posture information into Row output is realized for operator and further checks that programme path has good directive significance.
A kind of capsule endoscope localization method provided in an embodiment of the present invention can also include：
Detect acceleration when capsule endoscope acquisition alimentary canal picture；
Judge that the outside of alimentary canal position residing for capsule endoscope presets whether magnetic field meets preset condition, if it is, inspection The magnetic induction intensity of alimentary canal position residing for capsule endoscope is surveyed, and the magnetic induction intensity and acceleration are substituted into preset formula simultaneously The middle attitude angle for calculating capsule endoscope；If it is not, then detection capsule endoscope is around itself presetting the dynamic angular speed of three shaft rotations, and will be each Angular speed carries out the attitude angle of integral and calculating capsule endoscope respectively；
The attitude angle of capsule endoscope is added into posture information.
When above-mentioned steps are specifically described, it is believed that detection acceleration is first step, and it is second to calculate attitude angle Posture information is added as third step in attitude angle by a step, i.e., each small paragraph corresponds to one in above-mentioned also included step A step.
Wherein, in a first step, acceleration when capsule endoscope acquisition alimentary canal picture, the acceleration are detected first For vector, there is size and Orientation, it, can be by the 3-axis acceleration that is arranged in capsule endoscope specifically, in this step Sensor detects the acceleration thirdly in axis direction.Signified three axis directions herein, i.e., three formulated on the basis of capsule endoscope Dimension coordinate system, X, Y, Z axis therein can be arbitrarily designated on capsule endoscope, not influence the Accurate Determining of its posture.But for convenience It discusses, the present embodiment the following contents be right-handed coordinate system is that standard is said using the axial direction of capsule endoscope as Z axis It is bright.In this way, after 3-axis acceleration sensor detects the acceleration in every axis direction of capsule endoscope, it later can be by three axis On each acceleration be added, calculate acceleration of the resultant acceleration as capsule endoscope.
It is external default due to being needed by space magnetic field vector auxiliary measuring capsule endoscope attitude angle in second step Magnetic field meets certain condition --- and it need to be that horizontal orientation magnetic field or the magnetic induction line depth of parallelism are higher than that generally magnetic field is preset in the outside 95% horizontal magnetic field, and external default magnetic field is phasic Chang in operation, therefore firstly the need of judging in capsule The outside of alimentary canal position residing for mirror presets whether magnetic field meets preset condition.If it is, explanation can pass through space at this time The auxiliary measuring means of magnetic field vector carry out attitude angle measurement.The magnetic strength of alimentary canal position residing for so i.e. detectable capsule endoscope Intensity is answered, the acceleration of the magnetic induction intensity of the detection and the corresponding alimentary canal picture of capsule endoscope acquisition, which is then carried out data, melts It closes, the two substitutes into preset formula calculated simultaneously, you can accurately determines the attitude angle of capsule endoscope.If it is not, then Illustrate that attitude angle measurement can not be carried out by the supplementary means of space magnetic field vector at this time, it is contemplated that the attitude angle of capsule endoscope becomes When change, change around itself presetting the dynamic angle of three shaft rotations and also synchronizing, therefore it is pre- around itself to can detect capsule endoscope at this time If then three angular speed are carried out integral and calculating by the dynamic angular speed of three shaft rotations respectively, you can calculate capsule endoscope around certainly Body presets the dynamic angle of three shaft rotations, and then accurately measures the attitude angle of capsule endoscope.
In this way, above-mentioned technical proposal can pass through the acceleration of capsule endoscope when the default magnetic field in outside meets preset condition Data fusion is carried out with the magnetic induction intensity of residing alimentary canal position, is precisely calculated the attitude angle of capsule endoscope；And in outside When default magnetic field is unsatisfactory for preset condition, can by the acceleration of capsule endoscope with it around itself presetting the dynamic angular speed of three shaft rotations Data fusion is carried out, the attitude angle of capsule endoscope is precisely calculated.To sum up, in the environment of the default changes of magnetic field in outside, this reality The full attitude determination of capsule endoscope can accurately be realized by applying the above-mentioned technical proposal that example is provided.
A kind of capsule endoscope localization method provided in an embodiment of the present invention substitutes into magnetic induction intensity and acceleration pre- simultaneously If calculating the attitude angle of capsule endoscope in formula, may include：
Pass through formula
Calculate the pitch angle pitch of capsule endoscope；
Pass through formula
Calculate the roll angle roll of capsule endoscope；
Pass through formula
Yaw=ξ+θ
Calculate the yaw angle yaw of capsule endoscope；And
Wherein, ξ indicates the deflection angle when external default magnetic field meets preset condition in the horizontal direction；θ isWith Angle in the horizontal direction；For the x-axis base vector of capsule endoscope, if For the y-axis base of capsule endoscope Vector, if For the z-axis base vector of capsule endoscope, if Disappear for capsule endoscope acquisition Change acceleration when road picture, if For the magnetic induction intensity of alimentary canal position residing for capsule endoscope, If ForProjection in the x-y plane of capsule endoscope, ifAndWhenAnd A ∈ R, A>When 0, roll=| roll |, whenAnd A ∈ When R, A≤0, roll=- | roll |；WhenAnd A ∈ R, A>When 0, θ=| θ |, whenAnd when A ∈ R, A≤0, θ=- | θ |.
It, specifically can be by being arranged in capsule endoscope when detecting the magnetic induction intensity of alimentary canal position residing for capsule endoscope Magnetic field sensor detect its magnetic induction intensity.As the acceleration of capsule endoscope, which is similarly vector, tool There is size and Orientation, is stepwise varying, therefore its magnetic induction intensity since the default magnetic field in outside is in normal operation It is in variation at any time.And magnetic field is preset when meeting preset condition, forming horizontal orientation magnetic field in outside, magnetic direction is opposite It is inclined in the horizontal direction of preset standard axle (horizontal axis that may generally be the earth axes established centered on human body) Gyration is known, if its angle is ξ.While glue is calculated in preset formula substituting into magnetic induction intensity and acceleration simultaneously When the attitude angle of intracapsular mirror, specifically, first three-axis reference structure can be carried out to capsule endoscope.With capsule in the present embodiment The axial direction of scope be Z-direction, cross-sectional direction be X-Y plane direction for illustrate.In this way, the x of capsule endoscope Axis base vectorAs [1 0 0], and the y-axis base vector of capsule endoscopeAs [0 1 0], z-axis base vectorAs [0 0 1].The acceleration detected can be set simultaneouslyFor [gx gy gz], the magnetic induction intensity of alimentary canal position residing for capsule endoscopeFor [mx my mz], it is as shown in Figure 2 to correspond to schematic diagram.Meanwhile the posture of capsule endoscope is determined by attitude angle, and attitude angle master To include pitch angle, roll angle and yaw angle, therefore, it is pitch, roll angle roll, yaw angle yaw that can enable pitch angle.
Wherein, pitch angle pitch can be by between the acceleration and capsule endoscope local Coordinate System Z axis of capsule endoscope Angle acquires, so pitch angle can pass through formula：
It is calculated.
The computational methods of roll angle roll similarly, can set the acceleration of capsule endoscope in capsule endoscope local Coordinate System first Projection vector on x/y plane isThenFor [gx gy 0], such roll angle roll can pass through formula：
It is calculated.
Wherein, whenAnd A ∈ R, A>When 0, roll=| roll |,
WhenAnd when A ∈ R, A≤0, roll=- | roll |.
Yaw angle yaw needs acceleration and magnetic induction intensity carrying out Data Fusion, while also needing to using aforementioned The default deflection angle ξ of magnetic field in the horizontal direction in outside.
θ is set first as the horizontal direction angle of magnetic induction intensity and capsule endoscope local Coordinate System Z-direction, secondly, is enabledWherein,ForWithThe normal vector of plane is formed,ForWithForm plane Normal vector so then has：
Wherein, whenAnd A ∈ R, A>When 0, θ=| θ |,
WhenAnd when A ∈ R, A≤0, θ=- | θ |；
Above-mentioned A is coefficient.
After θ is calculated, itself and ξ are added as yaw angle yaw, i.e. yaw=ξ+θ.
So far, the pitch angle pitch of capsule endoscope, roll angle roll and yaw angle yaw, which have calculated that, comes, can be smooth Know the attitude angle of capsule endoscope, realizes and the full posture of capsule endoscope is parsed.
A kind of capsule endoscope localization method provided in an embodiment of the present invention, integral and calculating capsule is carried out by each angular speed respectively The attitude angle of scope may include：
Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting x-axis rotation；
Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting y-axis rotation；
Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting z-axis rotation；
Then the attitude angle of capsule endoscope can be expressed as by spin matrix：
Wherein, α0、β0And γ0Respectively in integral constant item capsule endoscope around itself preset x-axis initial rotation angle, Capsule endoscope is around the initial rotation angle and capsule endoscope for itself presetting y-axis around the initial rotation angle for itself presetting z-axis；ωx、 ωyAnd ωzRespectively capsule endoscope is that the default magnetic field in outside exists around the dynamic angular speed of three shaft rotations, integral constant item itself is preset It is unsatisfactory for the attitude angle of the capsule endoscope that the instantaneous moment of preset condition is calculated from meeting preset condition and being changed into.
Specifically, when angular speed to be carried out to the attitude angle of integral and calculating capsule endoscope respectively, can also include：It will be external Default magnetic field is being unsatisfactory for the posture of the capsule endoscope that the instantaneous moment of preset condition is calculated from meeting preset condition and be changed into As integral constant item, the formula to each angular speed to be carried out to integral and calculating respectively is modified at angle.
When the default magnetic field in outside is unsatisfactory for preset condition, need to detect capsule endoscope around itself presetting the dynamic angle of three shaft rotations Speed, and each angular speed is subjected to integral and calculating respectively, to measure attitude angle.Specifically, pre- around itself in detection capsule endoscope If when the angular speed that three shaft rotations are moved, specifically its angular speed can be detected by the angular-rate sensor being arranged in capsule endoscope.Together When, it is contemplated that in the fields MEMS, angular-rate sensor can be brought because what is measured is angular speed rather than angle itself for system Error.Angle can be acquired by angular speed definite integral, due to the shadow of the factors such as measurement error, sampling error during integral It rings, what is obtained is an amount for introducing error, within the short time, influences and little, can neglect on the precision of attitude determination Slightly, but when long lasting for attitude angle is measured by way of being integrated angular speed, system cumulative errors can be with the time It elapses increasing.In view of this, being additionally arranged amendment link in the present embodiment.
Since when the default magnetic field in outside meets preset condition, the attitude angle measured is accurate poor, therefore can The attitude angle measured in this time is recorded, preset magnetic field when outside changes suddenly, and is unsatisfactory for preset condition When, it is being changed into the attitude angle for being unsatisfactory for the instantaneous moment of preset condition and being recorded from meeting preset condition in the default magnetic field in outside As last group is without accumulated error data, and as integral constant item, as subsequently through angular speed integral operation Corrected parameter is eliminated with this and passes through the system accumulated error that is formed during angular speed Integration Solving attitude angle in long-time.
Specifically, when each angular speed to be carried out to the attitude angle of integral and calculating capsule endoscope respectively, formula can be passed through first：
Calculate t moment capsule endoscope around itself preset x-axis rotation angle,
It can pass through formula simultaneously：
Calculate t moment capsule endoscope around itself preset y-axis rotation angle,
It can pass through formula simultaneously：
T moment capsule endoscope is calculated around the angle for itself presetting z-axis rotation.
Wherein, ωx、ωyAnd ωzRespectively capsule endoscope is around itself presetting the dynamic angular speed of three shaft rotations.
Since t be to be counted from the default magnetic field in outside meeting preset condition and being changed into the instantaneous moment for being unsatisfactory for preset condition When time, and α0、β0And γ0Capsule endoscope is around the initial rotation angle for itself presetting x, y and z axes respectively in integral constant item Degree, each initial rotation angle is related without accumulated error data to last group of record, can pass through the pitching in this group of data Angle pitch, roll angle roll and yaw angle yaw are converted and are obtained.
Later, you can capsule endoscope is calculated around the calculation formula for itself presetting the dynamic angle of three shaft rotations according to above three, The attitude angle of capsule endoscope is solved, for convenience of discussing, the present embodiment uses rolling-pitching-beat representation in dynamics, Pass through spin matrix RrpyThe attitude angle of capsule endoscope is expressed as by (φ, θ, ψ)：
And by the matrix data carry out operation can obtain respectively pitch angle pitch, roll angle roll and partially The general expression of boat angle yaw, details are not described herein again.
It is further to note that the detection of angular speed can be realized by the gyroscope being set in capsule endoscope.
A kind of capsule endoscope localization method provided in an embodiment of the present invention, training depth network model may include：
Training set and test set are obtained, includes alimentary canal picture and each alimentary canal picture of expression in training set and test set The label of corresponding alimentary canal position；
The depth network model based on deep learning frame is chosen as current depth network model, using training set to depth Degree network model is trained, and the depth network model tested after training using test set obtains the identification essence of depth network model Degrees of data, judges whether the accuracy of identification data meet default required precision, if so, determining the depth network model after training To complete the depth network model of training, if not, it is determined that the depth obtained after being adjusted to the depth network model after training Degree network model is depth network model, returns and executes the step of being trained to depth network model using training set.
Data set prepares to include the acquisition to training set and test set, and wherein training set is for realizing depth network model Training, test set is for realizing the test of depth network model, and the alimentary canal picture that test set and training set include is (in the application May be simply referred to as picture) it can be by coming from Gastroenterology dept. Endoscope room diagosis medical worker to the collected full picture of capsule endoscope Manual markings are carried out to obtain, the alimentary canal position of label can include but is not limited to oesophagus, cardia, stomach bottom, body of stomach, stomach angle, Antrum, pylorus, duodenum etc. are labeled different anatomic position using different labels, and label principle is can foundation Single picture information carries out human eye and recognizes anatomical position, and training set is different from the alimentary canal picture for including in test set, thus, it is possible to Enough improve the accuracy of identification of the depth network model trained.For example bright, the alimentary canal picture that N number of patient may be used is made For data set, using the alimentary canal picture of wherein i-th patient as test set, the remaining N-1 other than i-th of patient is a The alimentary canal picture of patient is as training set (1≤i≤N).
Realize the identification of alimentary canal position in the application using depth network model, and the selection of depth network model and its The setting of model parameter can be set previously according to actual needs, and depth network model may be used based on before CNN in the application Alexnet, Resnet, Googlenet, VGG etc. for presenting convolutional neural networks model, by above-mentioned network in embodiment hereof After model is trained and tests it was found that, for capsule endoscope picture (alimentary canal dissect picture) i.e. in the application Speech, Alexne network models have relatively high recognition correct rate and Predict masculine gender rate, therefore preferred Alexne in the application Network model realizes corresponding function, namely chooses the depth network model based on deep learning frame as current depth network mould Type may include：The Alexne network models based on deep learning frame are chosen as current depth network model.
Specifically, the depth network model selected in the application is preferably based on the realization of Alexne network models, should The basic number of plies of Alexne network models is 8 layers, 5 layers of convolutional layer, 3 layers of full articulamentum, and the output layer of the last one full articulamentum is Precision layer with loss function output, the output number of output layer are the number of alimentary canal position；In first layer convolutional layer (conv1) it is later normalization layer (norm) with second layer convolutional layer (conv2), in each convolutional layer and full articulamentum (FC) RELU operations are followed by used, i.e. activation primitive is used for solving nonlinear problem, after norm1, norm2, conv5 For pond layer (pooling).And default required precision is preset according to being actually needed, if accuracy of identification data fit Default required precision, then illustrate that the accuracy of identification of the corresponding depth network model of the accuracy of identification data reaches to accuracy of identification It is required that then determining that training is completed at this time, otherwise, then the depth network model is instructed again after adjusting corresponding depth network model Practice, to ensure that the accuracy of identification for being finally completed trained depth network model is higher.
In above-mentioned technical proposal disclosed by the embodiments of the present invention, obtain comprising alimentary canal picture and corresponding alimentary canal position mark The training set and test set of note train depth network model using training set, test the depth network model using test set and obtain To the accuracy of identification data for the accuracy of identification for indicating the depth network model, and in the corresponding accuracy of identification of accuracy of identification data The step of executing using training set training depth network model is returned to when undesirable after percentage regulation network model, until deep Until degree network model corresponds to the requirement of accuracy of identification data fit, the accuracy of identification to ensure that depth network model is higher.
A kind of capsule endoscope localization method provided in an embodiment of the present invention can be with after obtaining training set and test set Including：
It is unknown class picture to determine that alimentary canal position is corresponded in training set and test set is unknown alimentary canal picture, and profit The picture that similarity in unknown class picture is more than predetermined threshold value is excluded with perceptual hash algorithm；
Predetermined angle rotation processing and image enhancement processing are carried out to the alimentary canal picture for including in training set and test set.
The picture labeled as unknown class is further comprised in the alimentary canal picture that training set and test set include, this kind of picture is The remaining picture behind alimentary canal position can be distinguished by being removed in the collected complete alimentary canal picture of capsule endoscope, such as by bubble Some human eyes that either other alimentary canal contents block by the alimentary canal of individual discrimination can not dissect picture or close to disappearing Change that mucous membrane is more indiscernible can be marked as class, in the present invention, in order to make the picture labeled as alimentary canal position class Certain harmony is quantitatively kept with the picture labeled as unknown class, unknown class picture is excluded using perceptual hash algorithm The picture of similarity high (i.e. similarity is more than previously according to the determining given threshold of actual needs), the low picture of similarity is stayed The lower unknown class as the training of depth network model and test.Specifically, similar pictures are excluded using perceptual hash algorithm Process can be as follows：
Adjacent two pictures are narrowed down to 8*8 sizes by S1；
S2 if it is color image, then carries out it processing for reducing gray level, specifically may be used for the picture after diminution To drop to 64 grades of gray scales；
S3 calculates the average gray of 8*8 picture all pixels；
The gray scale of each pixel is compared by S4 with average gray, and the pixel more than or equal to average value is denoted as 1, Pixel less than average value is denoted as 0, obtains the cryptographic Hash of corresponding picture；The cryptographic Hash of two pictures is compared, sees correspondence Whether the cryptographic Hash of pixel is equal, records unequal number of pixels, if the number reaches one set according to actual needs Fixed threshold value illustrates two pictures dissmilarity；If the number is not up to above-mentioned threshold value, two pictures are similar, exclude first, Retain second to repeat the above process with next pictures, until the whole pictures of traversal.
In addition the picture for including in training set and test set can be directly used in the application, but in order to make last depth Degree network model can adapt to isotropic recognition result, all pictures can be concentrated to carry out one data in the present embodiment Determine angle rotation processing and image enhancement processing, wherein the angle rotated can be set according to actual needs, and at rotation Reason and image enhancement processing are consistent with the realization principle for corresponding to technical solution in the prior art, and details are not described herein.Certainly also may be used Only to carry out predetermined angle rotation to the picture in training set and test set or only carry out image enhancement processing etc., in this hair Within bright protection domain.
A kind of capsule endoscope localization method provided in an embodiment of the present invention instructs depth network model using training set Practice, may include：
The alimentary canal picture for including in training set is combined into multiple sub- training sets, wherein every sub- training set disappearing of including It is not exactly the same to change road picture；Depth network model is respectively trained using multiple sub- training sets and obtains corresponding multiple depth networks Model, and using test set tested to obtain the identification essence of corresponding depth network model to multiple depth network model respectively Degrees of data chooses corresponding accuracy of identification statistics indicate that the highest depth network model of accuracy of identification is using training set to depth net Network model be trained after depth network model.
The alimentary canal picture for including in training set can be combined into multiple sub- training sets, as when a comprising n in training set , can be using the picture of other patients except j-th of patient as sub- training set when the picture of patient, j distinguishes value by 1 to n, from And n sub- training sets can be obtained.Corresponding depth network model is obtained by multiple sub- training sets and is selected in optimal Model realization subsequent step, further improve the precision of depth network model that training obtains.
In addition it can form multiple sub- training sets and test set by data set, as when including N number of patient in data set When picture, can using the picture of other patients except i-th of patient as sub- training set, and the picture of i-th of patient as pair Sub- test set, i is answered to distinguish value by 1 to N, so as to obtain N number of sub- training set and N number of sub- test set, thus using any Realize that the test of the depth network model, final choose are known after sub- training set training depth network model using corresponding sub- test set The other highest depth network model of precision is as the depth network model after being trained to depth network model using training set It realizes subsequent step, to realize the training to depth network model using data set to greatest extent, and improves depth The precision of network model.
A kind of capsule endoscope localization method provided in an embodiment of the present invention, is obtained using test set test depth network model The accuracy of identification data of depth network model may include：
The depth net is calculated according to the following formula using test set test depth network model, and based on test acquired results The recognition correct rate and positive prediction rate that the accuracy of identification data of network model include：
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in recognition correct rate=test set/ The total quantity * 100% of the alimentary canal picture of alimentary canal position is corresponded in test set；
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in positive prediction rate=test set/ The total quantity * 100% of the alimentary canal picture of the correspondence alimentary canal position automatically identified in test set.
It can specifically include using test set test depth network model：Using the picture of test set as depth network model Input, obtain depth network model output picture correspond to alimentary canal position, the label that picture has had corresponds to alimentary canal Then determination correctly identifies the picture to the alimentary canal position consistency that position is exported with depth network model, otherwise then thinks incorrect The picture is identified, so as to count the alimentary canal picture of certain the correct alimentary canal position automatically identified in test set Quantity, alimentary canal position is corresponded in test set the total quantity of alimentary canal picture, automatically identify in test set it is correct The alimentary canal picture of the correspondence alimentary canal position automatically identified in the quantity of the alimentary canal picture of certain alimentary canal position, test set Total quantity, and then calculate based on the obtained above-mentioned quantity of statistics the recognition correct rate and positive prediction rate of depth network model, If recognition correct rate and positive prediction rate reach corresponding numerical value in default required precision, illustrate depth network model Accuracy of identification meets the requirements, and training is completed, and otherwise, then illustrates that the accuracy of identification of depth network model is undesirable, and training is not It completes and continues follow-up training step.
In addition the alimentary canal picture that capsule endoscope acquires is input to the depth network model of training completion, obtain depth net The corresponding alimentary canal position of the alimentary canal picture of network model output may include：
The alimentary canal picture that capsule endoscope acquires is input to the depth network model of training completion, obtains the depth network The alimentary canal picture of the acquisition for the softmax layers output with probability output that model includes corresponds to the general of different alimentary canal positions Rate, and determine that the alimentary canal position of the maximum probability is the corresponding alimentary canal position of alimentary canal picture of acquisition.
The softmax layers with probability, corresponding depth network may be used in the output layer of depth network model in the application Model output includes that the alimentary canal picture of capsule endoscope acquisition corresponds to the probability of each alimentary canal position, therefrom chooses probability most Large digestive tract position is the alimentary canal position of corresponding picture, to ensure that accuracy of identification.
It is further to note that for the figure of capsule endoscope acquisition in above-mentioned technical proposal provided in an embodiment of the present invention Piece, the mode that reading picture internal storage data or picture storage path may be used inputs depth network model, and can will instruct Practice the picture that collection includes to be stored on hard disk by the way of training list, for obtaining when training.
Depth network model is trained using training set in above-mentioned technical proposal, may include：Instruction is utilized on GPU Practice set pair depth network model to be trained.
The training of depth network model can be realized on GPU, CPU etc., in the present embodiment, in order to enable instruction The speed for practicing depth network model faster, preferentially uses the instruction that depth network model is realized in supercomputing performance card GPU platform Practice process, can also be carried out on other hardware platforms according to the actual application certainly, protection scope of the present invention it It is interior.
In addition depth network model based on deep learning frame can be Caffe, Caffe2, Tensorflow, Theano, Torch, CNTK etc. can specifically be set, within protection scope of the present invention according to actual needs.This Preferably use the networks the Alexnet model under Caffe frames as depth network learning model in text, it is corresponding to Caffe frames Corresponding order line statement is called to realize the training to depth network learning model under frame.
The depth network model obtained after being adjusted to the depth network model after training is determined in above-mentioned technical proposal For depth network model, may include：Determine the depth network obtained after adjusting to the current depth network model after training Model is current depth network model, and adjustment is comprising to the network hyper parameter of current depth network model and the adjustment of the number of plies.
It should be noted that can adjust the network hyper parameter and layer of network model to the adjustment of depth network model Number can specifically be realized by the content in the corresponding configuration file solver.prototxt of change, can include mainly training kind Basic learning rate, learning rate adjustable strategies and maximum iteration etc., the picture number that training set includes can also be increased； To improve the precision of the depth network model trained.Above-mentioned adjustment in the present embodiment can be that fine-tuning is finely tuned Processing.
Capsule endoscope is acquired position when alimentary canal picture by a kind of capsule endoscope localization method provided in an embodiment of the present invention Before the output of appearance information, can also include：
Judge whether alimentary canal position is unknown class, if it is, the posture information that will be exported apart from the last time at moment It is exported, if it is not, then instruction output module executes posture information output when capsule endoscope to be acquired to alimentary canal picture Step.
Judge that alimentary canal position is unknown class, i.e. alimentary canal position is the classification that can not manually determine its actual position, Then think that the last alimentary canal position determined (is not exported apart from the last time at current time for the posture information of unknown class Posture information) be posture information and output that capsule endoscope acquires the alimentary canal picture moment so that extraneous staff can To recognize the pose of capsule endoscope.
Capsule endoscope is acquired position when alimentary canal picture by a kind of capsule endoscope localization method provided in an embodiment of the present invention Appearance information exports：
The pose of pre-rendered simulation capsule endoscope is arranged to, with capsule endoscope acquisition alimentary canal picture when pose The corresponding pose of information, and the simulation capsule endoscope is shown.
Above-mentioned display can be realized by display unit namely display unit for capsule endoscope posture information it is aobvious Show it can is by pre-rendered simulation capsule endoscope, be arranged to and the posture information pair when capsule endoscope acquisition alimentary canal picture The pose answered and display, so that staff intuitively can quickly recognize true glue by the simulation capsule endoscope The posture information of intracapsular mirror.It should be noted that simulation capsule endoscope can be according to true capsule endoscope 1:1 ratio It draws, and the size for the simulation capsule endoscope that can zoom in or out under the control of staff, to facilitate staff Check.In addition, display unit may include touch screen, simulation capsule endoscope is shown by touch screen as a result, same When, staff can realize the control of size, angle to simulating capsule endoscope etc. by touch screen.
The embodiment of the present invention additionally provides a kind of capsule endoscope positioning system, as shown in Fig. 2, may include：
Acquisition module 11, is used for：Obtain capsule endoscope acquisition alimentary canal picture, the alimentary canal picture picture luminance and The lens parameters of the camera lens of picture collection are realized when acquiring the alimentary canal picture in capsule endoscope；
Locating module 12, is used for：Alimentary canal picture is input in depth network model trained in advance, obtains depth net Alimentary canal corresponding with the alimentary canal picture position of network model output；And it is determined based on predetermined correspondence and picture The distance between distance of camera lens alimentary canal mucous membrane when brightness and lens parameters corresponding acquisition alimentary canal picture；
Output module 13, is used for：Capsule endoscope is acquired to posture information output when alimentary canal picture, the posture information packet It includes alimentary canal picture and corresponds to the distance between alimentary canal position and distance of camera lens alimentary canal mucous membrane.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention can also include：
Attitude Calculation module, is used for：Detect acceleration when capsule endoscope acquisition alimentary canal picture；Judge capsule endoscope institute The outside of place alimentary canal position presets whether magnetic field meets preset condition, if it is, alimentary canal position residing for detection capsule endoscope The magnetic induction intensity set, and the magnetic induction intensity and acceleration are substituted into preset formula to the posture for calculating capsule endoscope simultaneously Angle；If it is not, then detection capsule endoscope is around itself presetting the dynamic angular speed of three shaft rotations, and each angular speed is subjected to integrating meter respectively Calculate the attitude angle of capsule endoscope；The attitude angle of capsule endoscope is added into posture information.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention, Attitude Calculation module may include：
First computing unit, is used for：Pass through formula
Calculate the pitch angle pitch of capsule endoscope；
Pass through formula
Calculate the roll angle roll of capsule endoscope；
Pass through formula
Yaw=ξ+θ
Calculate the yaw angle yaw of capsule endoscope；And
Wherein, ξ indicates the deflection angle when external default magnetic field meets preset condition in the horizontal direction；θ isWith Angle in the horizontal direction；For the x-axis base vector of capsule endoscope, if For the y-axis base of capsule endoscope Vector, if For the z-axis base vector of capsule endoscope, if Disappear for capsule endoscope acquisition Change acceleration when road picture, if For the magnetic induction intensity of alimentary canal position residing for capsule endoscope, If ForProjection in the x-y plane of capsule endoscope, ifAndWhenAnd A ∈ R, A>When 0, roll=| roll |, whenAnd A ∈ When R, A≤0, roll=- | roll |；WhenAnd A ∈ R, A>When 0, θ=| θ |, whenAnd when A ∈ R, A≤0, θ=- | θ |.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention, Attitude Calculation module may include：
Second computing unit, is used for：Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting x-axis rotation；
Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting y-axis rotation；
Pass through formula
T moment capsule endoscope is calculated around the angle for itself presetting z-axis rotation；
Then the attitude angle of capsule endoscope can be expressed as by spin matrix：
Wherein, α0、β0And γ0Respectively in integral constant item capsule endoscope around itself preset x-axis initial rotation angle, Capsule endoscope is around the initial rotation angle and capsule endoscope for itself presetting y-axis around the initial rotation angle for itself presetting z-axis；ωx、 ωyAnd ωzRespectively capsule endoscope is that the default magnetic field in outside exists around the dynamic angular speed of three shaft rotations, integral constant item itself is preset It is unsatisfactory for the attitude angle of the capsule endoscope that the instantaneous moment of preset condition is calculated from meeting preset condition and being changed into.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention can also include：
Model training module is used for：Obtain training set and test set, in training set and test set comprising alimentary canal picture and Indicate that each alimentary canal picture corresponds to the label of alimentary canal position；Choose the depth network model conduct based on deep learning frame Current depth network model is trained depth network model using training set, and the depth after training is tested using test set Network model obtains the accuracy of identification data of depth network model, judges whether the accuracy of identification data meet default precision and want It asks, if so, determining that the depth network model after training is the depth network model for completing training, if not, it is determined that training The depth network model that depth network model afterwards obtains after being adjusted is depth network model, returns to execution and utilizes training set The step of depth network model is trained.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention can also include：
Preprocessing module is used for：After obtaining training set and test set, determines in training set and test set and correspond to alimentary canal Position is that unknown alimentary canal picture is unknown class picture, and it is big to utilize perceptual hash algorithm to exclude similarity in unknown class picture In the picture of predetermined threshold value；Predetermined angle rotation processing and image are carried out to the alimentary canal picture for including in training set and test set Enhancing is handled.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention, model training module may include：
First training unit, is used for：The alimentary canal picture for including in training set is combined into multiple sub- training sets, wherein often The alimentary canal picture that a sub- training set includes is not exactly the same；Depth network model is respectively trained using multiple sub- training sets to obtain Corresponding multiple depth network models, and respectively multiple depth network model is tested to obtain using test set and correspond to deeply The accuracy of identification data of network model are spent, choose corresponding accuracy of identification statistics indicate that the highest depth network model of accuracy of identification is Depth network model after being trained to depth network model using training set.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention, model training module may include：
Second training unit, is used for：Using test set test depth network model, and based on test acquired results under The recognition correct rate and positive prediction rate that the accuracy of identification data that row formula calculates the depth network model include：
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in recognition correct rate=test set/ The total quantity * 100% of the alimentary canal picture of alimentary canal position is corresponded in test set；
The quantity of the alimentary canal picture of certain the correct alimentary canal position automatically identified in positive prediction rate=test set/ The total quantity * 100% of the alimentary canal picture of the correspondence alimentary canal position automatically identified in test set.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention can also include：
Discrimination module is used for：Before posture information output when capsule endoscope to be acquired to alimentary canal picture, alimentary canal is judged Whether position is unknown class, if it is, the posture information exported apart from the last time at moment is exported, if it is not, then Indicate the step of output module executes posture information output when capsule endoscope to be acquired to alimentary canal picture.
A kind of capsule endoscope positioning system provided in an embodiment of the present invention, output module may include：
Display unit is used for：The pose of pre-rendered simulation capsule endoscope is arranged to, is digested with capsule endoscope acquisition The corresponding pose of posture information when road picture, and the simulation capsule endoscope is shown.
The explanation of relevant portion refers to of the invention real in a kind of capsule endoscope positioning system provided in an embodiment of the present invention The detailed description of corresponding part in a kind of capsule endoscope localization method of example offer is applied, details are not described herein.In addition the present invention is real Apply in the above-mentioned technical proposal of example offer the part consistent with technical solution realization principle is corresponded in the prior art not specifically It is bright, in order to avoid excessively repeat.
It is further to note that as shown in figure 4, realizing the hardware system of above-mentioned technical proposal provided in an embodiment of the present invention System may include：Capsule endoscope, including capsule endoscope shell, light illuminating unit, image acquisition units (including camera lens), microprocessor Device, Transmit-Receive Unit, acceleration transducer, magnetic field sensor, internal magnets, battery；External magnetic field device, including external magnetic field hair Generating apparatus, external magnetic field detection device；Processor, including capsule attitude calculation unit, capsule anatomical position recognition unit, Metrics calculation unit and display unit.The component consistent with technical solution realization principle is corresponded in the prior art exists in said program This is not illustrated.
Wherein, acceleration transducer is used for the detection of capsule endoscope self-acceleration vector；Magnetic field sensor is used for capsule The magnetic field vector generated by external magnetic field generator at endoscope position；Outer field generator is for generating external drive magnetic , for being reached to capsule endoscope generation pulling force and torsional forces to drive capsule endoscope to be rolled, rotated and banking motion The purpose moved in vivo to active control capsule endoscope；External magnetic field detection device is for being detected external magnetic field.Glue The acceleration and magnetic field that capsule spatial attitude computing unit is used to be obtained according to capsule endoscope acceleration transducer and magnetic field sensor Vector calculates the attitude angle of capsule endoscope according to preset formula, and capsule anatomical position recognition unit is used to utilize depth network model The determination of alimentary canal position is carried out according to the collected alimentary canal picture of capsule endoscope.Metrics calculation unit according to picture for believing Breath calculates the distance of capsule endoscope distance of camera lens alimentary canal mucous membrane.Display unit is used for over the display to the pose of capsule endoscope Information is shown, to achieve the purpose that capsule endoscope positions in vivo, further inspection programme path is realized for operator With good directive significance.
The foregoing description of the disclosed embodiments enables those skilled in the art to realize or use the present invention.To this A variety of modifications of a little embodiments will be apparent for a person skilled in the art, and the general principles defined herein can Without departing from the spirit or scope of the present invention, to realize in other embodiments.Therefore, the present invention will not be limited It is formed on the embodiments shown herein, and is to fit to consistent with the principles and novel features disclosed in this article widest Range.

Claims (10)

1. a kind of capsule endoscope positioning system, which is characterized in that including：
Acquisition module is used for：Obtaining the alimentary canal picture of capsule endoscope acquisition, the picture luminance of the alimentary canal picture and acquisition should The lens parameters of the camera lens of picture collection are realized when alimentary canal picture in the capsule endoscope；
Locating module is used for：The alimentary canal picture is input in depth network model trained in advance, obtains the depth The alimentary canal position corresponding with the alimentary canal picture of network model output；And it is determined based on predetermined correspondence Distance of camera lens alimentary canal mucous membrane when acquisition corresponding with the picture luminance and the lens parameters alimentary canal picture The distance between；
Output module is used for：The capsule endoscope is acquired to the posture information output when alimentary canal picture, the posture information The distance between alimentary canal position and the distance of camera lens alimentary canal mucous membrane are corresponded to including the alimentary canal picture.
2. system according to claim 1, which is characterized in that further include：
Attitude Calculation module, is used for：Detect the acceleration when capsule endoscope acquires the alimentary canal picture；Judge the glue The outside of alimentary canal position residing for intracapsular mirror presets whether magnetic field meets preset condition, if it is, detecting the capsule endoscope The magnetic induction intensity of residing alimentary canal position, and the magnetic induction intensity and the acceleration are substituted into preset formula simultaneously and calculated The attitude angle of the capsule endoscope；If it is not, then the capsule endoscope is detected around itself presetting the dynamic angular speed of three shaft rotations, and will Each angular speed carries out the attitude angle of capsule endoscope described in integral and calculating respectively；By the attitude angle of the capsule endoscope be added to In the posture information.
3. system according to claim 2, which is characterized in that the Attitude Calculation module includes：
First computing unit, is used for：Pass through formula
Calculate the pitch angle pitch of the capsule endoscope；
Pass through formula
Calculate the roll angle roll of the capsule endoscope；
Pass through formula
Yaw=ξ+θ
Calculate the yaw angle yaw of the capsule endoscope；And
Wherein, ξ indicates the deflection angle when external default magnetic field meets preset condition in the horizontal direction；θ isWithIn water Square upward angle；For the x-axis base vector of the capsule endoscope, if For the y of the capsule endoscope Axis base vector, if For the z-axis base vector of the capsule endoscope, if For the glue Intracapsular mirror acquires the acceleration when alimentary canal picture, if To be digested residing for the capsule endoscope The magnetic induction intensity of road position, if ForProjection in the x-y plane of the capsule endoscope, If AndWhenAnd A ∈ R, A>When 0, roll=| roll |, whenAnd when A ∈ R, A≤0, roll=- | roll |；WhenAnd A ∈ R, A>When 0, θ=| θ |, whenAnd when A ∈ R, A≤0, θ=- | θ |.
4. system according to claim 3, which is characterized in that the Attitude Calculation module includes：
Second computing unit, is used for：Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting x-axis rotation；
Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting y-axis rotation；
Pass through formula
Capsule endoscope described in t moment is calculated around the angle for itself presetting z-axis rotation；
Then the attitude angle of the capsule endoscope can be expressed as by spin matrix：
Wherein, α0、β0And γ0Respectively capsule endoscope described in integral constant item around itself preset x-axis initial rotation angle, The capsule endoscope is around the initial rotation angle and the capsule endoscope for itself presetting y-axis around the initial rotation for itself presetting z-axis Angle；ωx、ωyAnd ωzThe respectively described capsule endoscope is around the dynamic angular speed of three shaft rotations, the integral constant item itself is preset The default magnetic field in outside is being changed into the capsule for being unsatisfactory for the instantaneous moment of preset condition and being calculated from meeting preset condition The attitude angle of scope.
5. system according to claim 1, which is characterized in that further include：
Model training module is used for：Training set and test set are obtained, includes alimentary canal figure in the training set and the test set Piece and each alimentary canal picture of expression correspond to the label of alimentary canal position；Choose the depth network based on deep learning frame Model is trained depth network model as current depth network model, using the training set, utilizes the test set Depth network model after test training obtains the accuracy of identification data of depth network model, whether judges the accuracy of identification data Meet default required precision, if so, determine that the depth network model after training is the depth network model for completing training, if It is no, it is determined that the depth network model obtained after being adjusted to the depth network model after training is depth network model, is returned The step of depth network model is trained using the training set described in receipt row.
6. system according to claim 5, which is characterized in that further include：
Preprocessing module is used for：After obtaining the training set and the test set, the training set and the test set are determined Middle corresponding alimentary canal position is that unknown alimentary canal picture is unknown class picture, and it is described unknown to utilize perceptual hash algorithm to exclude Similarity is more than the picture of predetermined threshold value in class picture；To the alimentary canal picture that includes in the training set and the test set into Row predetermined angle rotation processing and image enhancement processing.
7. system according to claim 5, which is characterized in that the model training module includes：
First training unit, is used for：The alimentary canal picture for including in the training set is combined into multiple sub- training sets, wherein often The alimentary canal picture that a sub- training set includes is not exactly the same；Depth network model is respectively trained using the multiple sub- training set Corresponding multiple depth network models are obtained, and respectively multiple depth network model test using the test set To the accuracy of identification data of corresponding depth network model, corresponding accuracy of identification is chosen statistics indicate that the highest depth net of accuracy of identification Network model is the depth network model after being trained to depth network model using the training set.
8. system according to claim 5, which is characterized in that the model training module includes：
Second training unit, is used for：Using the test set test depth network model, and based on test acquired results under The recognition correct rate and positive prediction rate that the accuracy of identification data that row formula calculates the depth network model include：
Quantity/test of the alimentary canal picture of certain the correct alimentary canal position automatically identified in recognition correct rate=test set Concentrate the total quantity * 100% of the alimentary canal picture of corresponding alimentary canal position；
Quantity/test of the alimentary canal picture of certain the correct alimentary canal position automatically identified in positive prediction rate=test set Concentrate the total quantity * 100% of the alimentary canal picture of the correspondence alimentary canal position automatically identified.
9. system according to claim 1, which is characterized in that further include：
Discrimination module is used for：Before posture information output when the capsule endoscope to be acquired to the alimentary canal picture, institute is judged State whether alimentary canal position is unknown class, if it is, the posture information exported apart from the last time at moment is exported, such as Fruit is no, it indicates that it is defeated that the output module executes the posture information when capsule endoscope to acquire to the alimentary canal picture The step of going out.
10. according to claim 1 to 9 any one of them system, which is characterized in that the output module includes：
Display unit is used for：The pose of pre-rendered simulation capsule endoscope is arranged to, with the capsule endoscope acquisition described in The corresponding pose of posture information when alimentary canal picture, and the simulation capsule endoscope is shown.
CN201810210665.3A 2018-03-14 2018-03-14 Capsule endoscope positioning system Active CN108354578B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810210665.3A CN108354578B (en) 2018-03-14 2018-03-14 Capsule endoscope positioning system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810210665.3A CN108354578B (en) 2018-03-14 2018-03-14 Capsule endoscope positioning system

Publications (2)

Publication Number Publication Date
CN108354578A true CN108354578A (en) 2018-08-03
CN108354578B CN108354578B (en) 2020-10-30

Family

ID=63000276

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810210665.3A Active CN108354578B (en) 2018-03-14 2018-03-14 Capsule endoscope positioning system

Country Status (1)

Country Link
CN (1) CN108354578B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109464115A (en) * 2018-12-17 2019-03-15 深圳开立生物医疗科技股份有限公司 A kind of endoscopic system and a kind of AI diagnostic device
CN110613454A (en) * 2019-10-09 2019-12-27 北京华亘安邦科技有限公司 Method and system for searching position of capsule endoscope
CN110897596A (en) * 2019-12-05 2020-03-24 重庆金山医疗技术研究院有限公司 Method for automatically adjusting capsule shooting rate, endoscope, recorder and system
CN111493805A (en) * 2020-04-23 2020-08-07 重庆金山医疗技术研究院有限公司 State detection device, method, system and readable storage medium
WO2020244582A1 (en) 2019-06-04 2020-12-10 北京理工大学 Active magnetically controlled capsule robot detection system and detection method
CN112200250A (en) * 2020-10-14 2021-01-08 重庆金山医疗器械有限公司 Digestive tract segmentation identification method, device and equipment of capsule endoscope image
CN113743287A (en) * 2021-08-31 2021-12-03 之江实验室 Robot self-adaptive grabbing control method and system based on impulse neural network

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102458215A (en) * 2009-06-10 2012-05-16 奥林巴斯医疗株式会社 Capsule type endoscope device
CN104203068A (en) * 2012-05-14 2014-12-10 奥林巴斯医疗株式会社 Capsule therapy device and therapy system
CN106097335A (en) * 2016-06-08 2016-11-09 安翰光电技术（武汉）有限公司 Digestive tract focus image identification system and recognition methods
CN106934799A (en) * 2017-02-24 2017-07-07 安翰光电技术（武汉）有限公司 Capsule endoscope image aids in diagosis system and method
CN107374566A (en) * 2017-07-13 2017-11-24 重庆金山医疗器械有限公司 A kind of full attitude determination method and system of capsule endoscope based on variation magnetic field
CN107437100A (en) * 2017-08-08 2017-12-05 重庆邮电大学 A kind of picture position Forecasting Methodology based on the association study of cross-module state

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102458215A (en) * 2009-06-10 2012-05-16 奥林巴斯医疗株式会社 Capsule type endoscope device
CN104203068A (en) * 2012-05-14 2014-12-10 奥林巴斯医疗株式会社 Capsule therapy device and therapy system
CN106097335A (en) * 2016-06-08 2016-11-09 安翰光电技术（武汉）有限公司 Digestive tract focus image identification system and recognition methods
CN106934799A (en) * 2017-02-24 2017-07-07 安翰光电技术（武汉）有限公司 Capsule endoscope image aids in diagosis system and method
CN107374566A (en) * 2017-07-13 2017-11-24 重庆金山医疗器械有限公司 A kind of full attitude determination method and system of capsule endoscope based on variation magnetic field
CN107437100A (en) * 2017-08-08 2017-12-05 重庆邮电大学 A kind of picture position Forecasting Methodology based on the association study of cross-module state

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109464115A (en) * 2018-12-17 2019-03-15 深圳开立生物医疗科技股份有限公司 A kind of endoscopic system and a kind of AI diagnostic device
WO2020244582A1 (en) 2019-06-04 2020-12-10 北京理工大学 Active magnetically controlled capsule robot detection system and detection method
CN110613454A (en) * 2019-10-09 2019-12-27 北京华亘安邦科技有限公司 Method and system for searching position of capsule endoscope
CN110613454B (en) * 2019-10-09 2022-07-26 北京华亘安邦科技有限公司 Method and system for searching position of capsule endoscope
CN110897596A (en) * 2019-12-05 2020-03-24 重庆金山医疗技术研究院有限公司 Method for automatically adjusting capsule shooting rate, endoscope, recorder and system
CN111493805A (en) * 2020-04-23 2020-08-07 重庆金山医疗技术研究院有限公司 State detection device, method, system and readable storage medium
CN112200250A (en) * 2020-10-14 2021-01-08 重庆金山医疗器械有限公司 Digestive tract segmentation identification method, device and equipment of capsule endoscope image
CN113743287A (en) * 2021-08-31 2021-12-03 之江实验室 Robot self-adaptive grabbing control method and system based on impulse neural network
CN113743287B (en) * 2021-08-31 2024-03-26 之江实验室 Robot self-adaptive grabbing control method and system based on impulse neural network

Also Published As

Publication number Publication date
CN108354578B (en) 2020-10-30

Similar Documents

Publication Publication Date Title
CN108354578A (en) A kind of capsule endoscope positioning system
CN109091098A (en) Magnetic control capsule endoscopic diagnostic and examination system
US20050182341A1 (en) Posture diagnosis equipment and program therefor
CN102149325B (en) Line-of-sight direction determination device and line-of-sight direction determination method
CN110473192A (en) Digestive endoscope image recognition model training and recognition methods, apparatus and system
US20100245555A1 (en) Method and system to measure body volume/surface area, estimate density and body composition based upon digital image assessment
CN105996975A (en) Method, device and terminal for testing vision
CN113139962B (en) System and method for scoliosis probability assessment
CN108182377A (en) Human eye sight detection method and device based on photogrammetric technology
WO2009043927A1 (en) Apparatus for acquiring and processing information relating to human eye movements
US20180070816A1 (en) System and method for capturing spatially and temporally coherent eye gaze and hand data during performance of a manual task
CN108563023B (en) Augmented reality processing method and system, control equipment and augmented reality equipment
CN108269309B (en) Parachute jumping operation action quantitative evaluation method and system
CN110505383B (en) Image acquisition method, image acquisition device and endoscope system
CN108427988A (en) A kind of alimentary canal anatomical position identification device
KR20210033290A (en) Swing posture training method using golf simulation device
CN107374566A (en) A kind of full attitude determination method and system of capsule endoscope based on variation magnetic field
KR20180080848A (en) Rehabilitation system and the method for the same which use virtual reality provided by game engine
JP7189355B2 (en) Computer program, endoscope processor, and information processing method
CN116740618A (en) Motion video action evaluation method, system, computer equipment and medium
Neumann et al. A camera-based scoring system for evaluating performance accuracy during a golf putting task
KR101398193B1 (en) Device and Method for Calibration
CN110008995A (en) Image-recognizing method and system
CN115105062A (en) Hip and knee joint coordination evaluation method, device and system and storage medium
CN114832349A (en) Yuanzhou swimming teaching auxiliary system and use method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20210927

Address after: 401120 1-1, 2-1, 3-1, building 5, No. 18, Cuiping Lane 2, Huixing street, Yubei District, Chongqing

Patentee after: Chongqing Jinshan Medical Technology Research Institute Co.,Ltd.

Address before: 401120 1 office buildings, Jinshan International Industrial City, 18 of Nei sang Road, Hui Xing street, Yubei District, Chongqing.

Patentee before: CHONGQING JINSHAN MEDICAL APPLIANCE Co.,Ltd.

TR01 Transfer of patent right