EP4622544A1 - Wireless thermal detection capsule - Google Patents
Wireless thermal detection capsuleInfo
- Publication number
- EP4622544A1 EP4622544A1 EP23895378.0A EP23895378A EP4622544A1 EP 4622544 A1 EP4622544 A1 EP 4622544A1 EP 23895378 A EP23895378 A EP 23895378A EP 4622544 A1 EP4622544 A1 EP 4622544A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- capsule
- tissue
- subject
- thermistors
- controller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
- A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00011—Operational features of endoscopes characterised by signal transmission
- A61B1/00016—Operational features of endoscopes characterised by signal transmission using wireless means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00174—Optical arrangements characterised by the viewing angles
- A61B1/00177—Optical arrangements characterised by the viewing angles for 90 degrees side-viewing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/041—Capsule endoscopes for imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/046—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for infrared imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
- A61B5/0004—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by the type of physiological signal transmitted
- A61B5/0008—Temperature signals
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
- A61B5/015—By temperature mapping of body part
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/07—Endoradiosondes
- A61B5/073—Intestinal transmitters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6846—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be brought in contact with an internal body part, i.e. invasive
- A61B5/6847—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be brought in contact with an internal body part, i.e. invasive mounted on an invasive device
- A61B5/6861—Capsules, e.g. for swallowing or implanting
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00064—Constructional details of the endoscope body
- A61B1/00071—Insertion part of the endoscope body
- A61B1/0008—Insertion part of the endoscope body characterised by distal tip features
- A61B1/00097—Sensors
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/02—Details of sensors specially adapted for in-vivo measurements
- A61B2562/0271—Thermal or temperature sensors
Definitions
- VCE Video capsule endoscopy
- various embodiments provide a swallowable, wireless, thermal detection (e.g., using temperature sensors and/or infrared (IR) imaging) capsule that allows for non- invasive diagnosis and monitoring of inflammatory gastrointestinal disease and other gastrointestinal pathologies.
- VCE has become a routine method used in medical practice for identifying gastrointestinal disease and has had success as a non-invasive tool for Crohn’s disease diagnosis and monitoring of disease progression.
- IR infrared
- a swallowable, wireless capsule technology capable of identifying inflammatory activity can change this outlook by providing one or more of: early diagnosis, routine monitoring, treatment optimization, and/or a way to locally deliver therapies in conditions such as Crohn's disease.
- Active inflammation in other organ systems generates a temperature rise in tissue detectable through infrared (IR) blackbody radiation.
- IR infrared
- IR sensor arrays may be limited by their low pixel resolution; therefore, we have instead developed a detection (e.g., via temperature sensors and/or thermal IR imaging) capsule that can utilize super- resolution neural networks to increase resolution and identify intestinal pathology (e.g., by detecting inflammation or other tissue conditions) that arises deep in the submucosal/muscular layers.
- a detection e.g., via temperature sensors and/or thermal IR imaging
- Embodiments of the disclosed thermal imaging capsule provide a mechanism for more sensitively detecting inflammation or other indications of pathology at any depth in the bowel wall, with little or no bowel prep, which is expected to make the device particularly suitable for Crohn’s disease monitoring.
- the thermal imaging capsule may also allow inflamed tissue to be treated directly, mitigating systemic side effects, thereby providing better outcomes for Crohn’s patients.
- the disclosure provides one or more of: 1) a wireless thermal detection capsule system, 2) a super-resolution neural network for capsule endoscopy, and 3) mechanisms for detecting inflammatory gut foci in conditions such as Crohn’s disease.
- the various embodiments provide a platform for both diagnosis and routine monitoring to enable therapeutic optimization and targeted treatment of Crohn’s disease and/or other pathologies.
- Another embodiment provides an apparatus for detecting gastrointestinal pathology, including: a capsule including a thermal imaging sensor, the capsule being configured to be swallowed by a subject, and the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject.
- FIG. 14 Another embodiment provides a system for detecting gastrointestinal pathology, including: a capsule including a thermal imaging sensor in communication with a controller and a wireless transmitter; and a plurality of antennas in communication with the capsule, the capsule being configured to be swallowed by a subject, the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject, the controller being configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject, the controller being configured to transmit the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas, and the plurality of antennas being configured to receive the at least one image from the wireless transmitter.
- Still another embodiment provides a method for detecting gastrointestinal pathology, including: providing a capsule including a thermal imaging sensor, a wireless transmitter, and a plurality of antennas in communication with a controller, the capsule being configured to be swallowed by a subject, detecting, by the thermal imaging sensor, infrared radiation emitted by a tissue of the subject; generating, by the controller, at least one image of the detected infrared radiation emitted by the tissue of the subject; transmitting, by the controller, the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas; and receiving, by the plurality of antennas, the at least one image from the wireless transmitter.
- FIG.4 shows several embodiments of reflectors for reflecting light from a sample onto the IR sensor; in the left panel, the photo below shows the IR sensor with a conical reflector and the diagram above the photo shows the field of view of the IR sensor with the conical reflector, with an indication of where there is a "blind spot" on the IR sensor which does not receive reflected light; in the center panel, the photo below shows the IR sensor with a wedge-shaped reflector and the diagram above the photo shows the field of view of the IR sensor with the wedge-shaped reflector, with an indication of where there are "blind spots” on the IR sensor which do not receive reflected light; in the right panel, the photo below shows the catadioptric reflector of FIGS.2 and 3 and the diagram above the photo shows the field of view of the IR sensor with the catadioptric reflector, with an indication of where there are "blind spots” on the IR sensor which do not receive reflected light; as shown in the right panel, the catadiopt
- FIG.5 top panel, shows a photo of a PCB for use with constructions of a wireless thermal IR imaging capsule, where the PCB has an IR sensor attached thereto as well as a snap-off programmer element which provides connections for programming the device and which can be removed before placing the PCB into the capsule;
- the bottom panel of FIG.5 shows a construction of a wireless thermal IR imaging capsule as in FIG.2 with a 4 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 rounded reflector.
- FIG.6A shows a diagram of a receiver module attached to a subject (an animal subject used during testing) with several antennas connected to the receiver module via coaxial connectors.
- FIG.9 shows a layered dielectric model of human tissue for RF transmission from the lumen to the skin surface. Dimensions are based on anatomical CT and ultrasound studies on the US population accounting for gender, weight, and variations of diseased states of tissue to provide a maximum upper bound on total attenuation (dielectric parameters are taken from the IT’IS foundation’s tissue database).
- FIG.10 shows, using the model of FIG.9, the reflection and transmission loss which were calculated at 433MHz using the max thickness of recorded tissues in both healthy and diseased patients. The results were validated in a 43kg swine tissue model.
- FIG.11 shows a prototype IR capsule PCB (panel a), capsule length (panel b), and capsule front (panel c).
- FIG.12 shows a block diagram of the electronic communications and powering of the IR capsule device.
- FIG.13 shows a photo of an experimental setup for performing an ex vivo test of a construction of a wireless thermal IR imaging capsule.
- FIG.14 shows a diagram of an experimental setup for performing ex vivo tests of a wireless thermal IR imaging capsule.
- FIG.15 shows an ideal image (left panel) and 8-frame averaged images for an insertion 1cm inside (center panel) the heated section and at the very end of the heated section of the tissue (3cm, right panel). Images were taken by the capsule inside the intestine.
- FIG.16A shows a general schematic of one construction of a thermal IR imaging capsule with a single IR sensor.
- the outer shell is made of PMMA.
- FIG.16B shows a general schematic of one construction of a thermal IR imaging capsule with two IR sensors, one at each end of the capsule.
- FIG.16C shows a diagram of another construction of a thermal IR imaging capsule with one IR sensor.
- FIG.17 shows an overview of an entire IR capsule system from a data stream perspective.
- PCB-Printed Circuit Board EEPROM-Electrically Erasable Programmable Read-Only Memory
- IR-Infrared IR-Infrared.
- FIG.18 shows a schematic of an experimental temperature measurement apparatus.
- FIG.19 shows an example of a system for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter.
- FIG.20 shows an example of hardware that can be used to implement computing device and server in accordance with some embodiments of the disclosed subject matter.
- FIG.21 shows an example of a process for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter.
- FIG.22 shows another example of a process for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter.
- FIG.23 shows a construction of a wireless thermal detection capsule according to the disclosure, indicating a front cap, a thermistor block, a processing unit and wireless transmission, a main power source, and a back cap.
- FIG.24 shows an exploded cross-sectional view of a wireless thermal detection capsule indicating a front cap, a capsule wall, a thermistor ring, a drug reservoir, a battery, a printed circuit board 1 (PCB1), a printed circuit board 2 (PCB2), a printed circuit board 3 (PCB3), a printed circuit board 4 (PCB4), a battery, and a back cap. 6
- PCB1 printed circuit board 1
- PCB2 printed circuit board 2
- PCB3 printed circuit board 3
- PCB4 printed circuit board 4
- FIG.25 shows a perspective view of a wireless thermal detection capsule.
- FIG.26A shows a side view of the thermistor block of FIG.24, indicating a capsule wall and thermistor rings.
- FIG.26B shows a front view of printed circuit board 1 (PCB1) of FIG.24, including thermistor rings.
- FIG.27A shows a front view of the printed circuit board 2 (PCB2) of FIG.24.
- FIG.27B shows a processing unit and wireless transmission of FIG.23, including a printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), and an antenna.
- FIG.27C shows a rear view of printed circuit board 3 (PCB3) of FIG.24.
- FIG.28A shows a front view of printed circuit board 2 (PCB2) of FIG.24.
- FIG.28B shows a side view of the processing unit and wireless transmission of FIG.23, including locations of printed circuit board 2 (PCB2) and printed circuit board 3 (PCB3).
- FIG.28C shows a rear perspective view of printed circuit board 3 (PCB3) of FIG.24.
- FIG.29A shows a processing unit and wireless transmission of FIG.23, including a drug reservoir, printed circuit board 2 (PCB2), and printed circuit board 3 (PCB3).
- FIG.29B shows a processing unit and wireless transmission of FIG.23, including a drug reservoir, an antenna, printed circuit board 2 (PCB2), and printed circuit board 3 (PCB3).
- FIG.30A shows a side view of the main power source of FIG.23.
- information including one or more of the current position of the capsule, the at least one of the type of tissue or the tissue thickness, and/or the data obtained from the at least one temperature sensor may be processed by a second neural network to determine at least 13 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 one of a pathology depth, a pathology type, or a pathology severity in the tissue of the subject.
- the information from the plurality of receiver belt antennas may be processed (e.g., using a neural network or other procedure) to identify at least one of a type of tissue or a tissue thickness between the capsule and a surface of the subject adjacent the plurality of receiver belt antennas.
- the apparatus and/or systems above may be used to carry out a method for detecting gastrointestinal pathology.
- the controller of the capsule may be configured to obtain data from the thermistors relating to the detected heat emitted from the tissue of the subject at a first time, where the data may be used (e.g., by the controller, the external device, and/or a further system) to determine a first position of the capsule at the first time based on information obtained from the plurality of receiver belt antennas.
- a second image of the tissue of the subject at the second position may be generated (e.g., by the controller, the external device, and/or a further system) based on the data relating to the detected heat emitted from the tissue of the subject at the second time.
- a map of heat emitted from the tissue of the subject may be generated (e.g., by the controller, the external device, and/or a further system) based on the first image at the first position and the second image at the second position.
- information from the first and second images may be combined to create an improved single third image, which may also be used to generate the map of emitted heat.
- the disclosed procedures employ this insight to provide embodiments of the capsule that include IR imaging capability.
- certain embodiments may provide a swallowable, wireless, IR imaging-capable capsule that images mucosal temperature and thus inflammatory activity, along with systems and methods for collecting and processing the data from the capsule.
- the capsule may also identify bowel inflammation which arises deep in the submucosal/muscular layers; as a result, the disclosed thermal IR imaging capsule- based procedures may require little or no colonic bowel preparation.
- CD is characterized by intermittent patches of inflammation while ulcerative colitis occupies a continuous swath of the colon. While ulcerative colitis can be treated and even cured with the removal of the colon, there currently is no cure for CD. Instead, treatment focuses on immunomodulators or TNF agonists.
- Early treatment for CD is associated with reduced risk of intestinal and perianal surgery as well as complications such as strictures.
- Current methods fall short of early detection of the disease. Forty percent (40%) of patients who are diagnosed with CD develop bowel damage within one year as a result of delayed diagnosis and are associated with worse outcomes such as high rates of surgery and hospitalization.
- the present capsule device can perform thermal IR imaging through fecal matter and use information obtained about the depth or thickness of fecal matter (e.g. obtained using an ultrasound transducer in the capsule) to adjust the signal strength in the images that are obtained.
- the apparatus may include a capsule (FIG.1) including a thermal imaging sensor and may be configured to be swallowed by a subject (e.g. by making the capsule suitably small and/or having a shape that is conducive to swallowing such as that of a pill with rounded ends, as disclosed herein).
- the thermal imaging sensor may be configured to detect infrared radiation emitted by a tissue of the subject (FIG.1).
- the IR capsule may include a printed circuit board (PCB) housing a microcontroller, IR sensor, power regulator and radio frequency (RF) transmitter.
- the electronics may be hermetically sealed in a polycarbonate capsule with a custom thermal IR lens.
- the capsule may further include a controller (e.g. including a processor and memory) coupled to the thermal imaging sensor.
- the controller may be configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject.
- the capsule may also include a wireless transmitter and the controller may be configured to transmit collected image data (e.g.
- the window may include a biocompatible material, which in some embodiments may include low density polyethylene (LDPE) (FIG.2).
- the capsule may further include a reflector disposed adjacent to the thermal imaging sensor, where the reflector may be configured to direct infrared radiation from the tissue towards the thermal imaging sensor.
- the reflector may be configured to direct infrared radiation from a circumferential region around the capsule. For example, since the typical GI tract sample is a luminal (i.e.
- a Nooelec NESDR SMArt XR software defined radio receives the FSK via 433MHz monopole with a 2MHz sampling rate. The data is processed in MATLAB and SIMULINK to display the video feed and store the image data for further processing.
- FIG.12 shows a block diagram of the device.
- Power [142] Each component of the IR capsule was isolated during typical operation and 25 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 their current was measured while the operating voltage was held at 3.3V. The resulting current and power consumptions are documented in Table 3.
- FIG.17 depicts an overview of the data stream for the thermal capsule and recording system. Data is transmitted from the capsule to the receiver belt and stored on two SD cards.
- the SD cards on the front of the main module of the receiver belt store image data, while those on the back store position tracking data.
- the SD cards can be removed and replaced with new or the same SD cards at any time to monitor the contents and extract data such as the position of the capsule.
- Antennas should be placed on the subject as shown in FIG.35 using the xyphoid process as a benchmark.
- the receiver belt then is worn using a strap (e.g., like a purse), and the subject swallows the wireless thermal capsule.
- N thermal images will be acquired by each IR sensor and digitized.
- the N frames will be transferred to the microcontroller via an I2C serial communications protocol bus.
- the microcontroller contains a processor that will average the N frames to decrease noise and increase thermal detection sensitivity.
- the averaged image will be transmitted from the capsule using a carrier RF frequency of 433 MHz.
- the RF signal will be transmitted wirelessly through the body and detected by a multitude of spatially offset receivers that reside within a belt worn by the patient.
- the image data will then be demodulated and stored by the patient-worn recorder. Images will be transferred from the recorder to a computer that will apply custom super-resolution algorithms that operate on M averaged frames to reconstruct the final image which has a greater number of pixels than those of the individual sensors.
- the senor will be a Heimann IR 32x32 array, which has the lowest commercial power consumption of 5-7 mA at 3V and is capable of running on non- combustible button cell batteries.
- the two wide view, sensor-mounted lenses (for embodiments which use lenses to direct light to the sensor) at each end of the capsule will enable ⁇ 360° visualization of the luminal surface.
- the IR sensor may include two thermistor rings that each include a thermistor array comprising thermistor temperature sensors lined on the wall of the capsule (FIGS.23, 24, 26A, 26B); by collecting temperature readings from each thermistor in the array at particular intervals as the capsule moves through the tissue, the collected data can be used to generate one or more images (e.g., 2D images) of the heat that is being emitted by the tissue.
- the precision contact thermistors are configured to measure temperature by an onboard microcontroller and converted to a digital value.
- the capsule will then send the data wirelessly to an external battery-powered receiver belt using Gaussian Frequency Shift Keying at a frequency range of 430-440 MHz, which is a frequency range that has optimal electrokinetic properties for transmitting through human tissue.
- the receiver belt may determine the position of the capsule using relative signal strength across eight different antennas.
- the receiver belt may also store both the temperature and position data on two separate SD cards.
- Sensitivity One area of focus has been the development of methods to achieve high IR detection sensitivity so that small temperature changes can be measured. Because we cannot change the detector array itself, we have instead used procedures based on averaging successive frames, which reduces thermal noise and thus increases sensitivity, by the square root of the number of frames averaged.
- IR sensors have a pre-measured noise equivalent temperature difference (NETD) that is the standard deviation in pixel value in milli-Kelvin.
- NETD noise equivalent temperature difference
- frame averaging lowers the NETD by the inverse square root of the number of frames average, so achieving 100mK requires around 12 frames to be averaged.
- frame averaging will be implemented on the capsule’s microcontroller to achieve a target temperature resolution/variance of 0.1 oC, a value that is well within the range of temperature increases associated with inflammatory activity in other organ systems.
- aqueous button cells aqueous cells are immune to thermal runaway and combustion
- Silver-Zinc (Ag-Zn) chemistry a size 312 button Ag-Zn cell can provide 5-10 mA at 1.8 V with 35 mWh capacity.
- 10 mA can be supplied at 3.3V with 20 mA bursts for transmission.
- the average power draw of our current design is 21.96 mW giving about 3.1 hours of continuous imaging time. Operations of the device including transmission will be delayed post ingestion by a given amount (e.g. about 2-3 hours in certain embodiments) to allow the device to reach the ileum.
- the Lithium Manganese Dioxide battery is used in various embodiments due to its non-volatile chemistry, long life, and ability to power medium current devices.
- the capsule is further configured to include both a hardware and a software check on the battery voltage to determine when the battery is nearing depletion and initiate a graceful shutdown.
- the software checks the battery voltage using an ADC input on a GPIO pin.
- the battery is specified to operate between 2.0-3.0V, so when the battery reaches 2.2V the software triggers a shutdown process and the capsule then enters the off state after turning off all microprocessor sub-systems.
- the main regulator hardware checks if the battery is near depletion and if so, performs a hardware disconnect of the power supply.
- the capsule poses no risk to the subject when power is turned off either gracefully at the end of the battery life, or abruptly by the hardware cutoff.
- Wireless transmission The MAX41464 transmitter will transmit encoded I2C image sensor data at 433 MHz, providing a data rate of 200 kbps. When transmitting through the human body, RF signals are both attenuated and reflected when transitioning from different tissue layers.
- PCB Printed circuit board
- the four components can be combined onto a single PCB prototype, as shown in FIGS.13a, 14, and 17.
- the board will occupy 8.8 mm x 17 mm x 1.6 mm to allow space for two 3V batteries inside the capsule’s housing.
- FIGS.14 and 17 show internal block diagrams of the different components on the board that communicate with each other.
- PCB1 may be connected to and placed within the array of thermistor rings for improved thermal conductivity (FIGS.26 & 32).
- one or more thermistor rings may be used.
- the thermistor sensors are not required to be perfectly aligned and in some embodiments can be offset from one another by rotating the individual rings of the thermistors by any amount, e.g., any number of degrees.
- Two PCBs, PCB2 and PCB3 may be configured to be located within the processing unit and wireless transmission that further include being surrounded by antennas in the distal direction and a drug reservoir (FIGS.27-29).
- PCB 4 may include a holder for contact to the rear side of the power source (FIG.31).
- PCB1-PCB4 may be connected to one another through a circuit board housing and are configured to be folded and assembled within the capsule (FIGS.32 & 33).
- Receiver belt includes a main module, a battery, antenna leads, and a holster/belt.
- an 8-lead antenna configuration can be used for receiving capsule image data, although in other embodiments various numbers of antennas may be used, e.g. from 1-20 antennas.
- the eight antennas will reside within a wearable belt that transmits data to a recorder and may be positioned from 5 to 10 centimeters apart.
- the holster is attached to both a shoulder strap and belt to ensure it stays in place (FIG. 35).
- Capsule position tracking will be accomplished by triangulating its position from the amplitudes of the received signal to the array of antenna leads incorporated in the belt.
- the main module continuously sweeps all eight antennas, recording signal strength to track the position of the capsule.
- the antenna received the strongest signal strength is constantly updated and used to receive and record the capsule data.
- the thermal data from the wireless thermal capsule is demodulated and decoded from baseband and then stored into an SD card on the front side of the main module.
- a second SD card stores the signal strength data for position tracking from the continuous sweeps.
- Two serial ports, one for each SD card, allow for JTAG to USB serial access to live stream the thermal imaging data and capsule position.
- the data is then sent in an encrypted state that can decrypt and display in real time the assessment of the capsule position and device 33 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 functionality.
- Current VCE capsules achieve 3.8 cm position tracking resolution. We believe location resolution is limited by the high attenuation and variance in signal strength as the capsule moves.
- ADC analog to digital converter
- This configuration should enable mm-level capsule position tracking resolution. Based on recent research conducted in our lab, we have found that a 20° cone encompasses the total transmitted signal.
- the main module is powered by a 3.7V rechargeable battery, and the PCB and battery are held in separate compartments in a plastic case.
- the plastic case is placed in a cloth pouch that is attached to a shoulder strap, which provides vertical support, and a belt, which prevents swaying motion (FIGS.35A & 35B).
- the design is configured to comfortably holds the main module near the subject for the duration of use while minimizing the possibility that any antennas will be pulled off the subject.
- the receiver belt main module is reusable, and the antennas are single use.
- the receiver belt may use a Lithium Polymer battery.
- the software checks the battery voltage using an ADC input on a GPIO pin.
- the battery is specified to operate between 3.7-2.2V, so when the battery reaches a voltage of 2.2V or less the software triggers a shutdown process and the receiver then enters the off state after turning off all microprocessor sub-systems.
- the main voltage regulator also checks if the battery voltage is below 2.2V and if so, cuts power to the device.
- the lithium polymer battery has a built-in control circuit that cuts the battery off if a short-circuit condition is detected.
- Table 7 exhibits the specifications for the wireless thermal receiver belt.
- the receiver belt is configured to have a small footprint (e.g., occupying a volume of 7cm x 7cm x 2cm or less) and fit inside a cloth pouch to be worn by the subject.
- the total weight of the wireless thermal receiver belt may be 1.1 lbs. (about 0.5 kg) or less and the antennas are designed using medical adhesive so as to minimize any skin irritation.
- low noise chipsets with -120dBm noise floors were chosen to minimize the power necessary from the capsule.
- a frequency of 430-440MHz was chosen to minimize attenuation and interactions with the human body.
- a blackbody source can be used to validate the capsule’s thermal resolution/sensitivity and to optimize the angular field of view.
- the capsule’s capacity to measure temperature in blackbody radiation phantoms and thermally modulated swine intestines ex vivo can be tested using a flexible thermoelectric generator (FlexTEG) apparatus.
- FlexTEG flexible thermoelectric generator
- the swine intestines will be sutured water tight at one and submerged in a thermally controlled 37°C water bath, with thermally modulated areas (areas with 0.5°C to 1.5°C rises) created from resistive and Peltier heaters simulating inflammation spots. Temperatures along the tissue and at inflammation sites will be measured with non-metallic fiber optic temperature sensors (TS5, Micronor).
- TS5, Micronor non-metallic fiber optic temperature sensors
- Capsule prototypes will be introduced (AdvanceCE® Delivery Device) in the duodena of three cohorts (with prep, minimal prep, no prep) of adult 35 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 (50-75 kg) swine 7 days following DSS or TNBS-EtOH administration.
- a modified version of the receiver/recorder belt will be attached to the swine and parameters will be slightly adjusted to account for the differences between swine and human anatomy. Swine will be allowed to recover/ambulate, and thermal images will be recorded of the small and large intestines in vivo. Ileocolonic lesions with elevated temperatures will be identified along with their corresponding locations in the intestine.
- Capsule localization will be confirmed via anteroposterior and lateral X-rays taken every 30 minutes. After the capsule has passed through the GI tract, the swine will be sacrificed, small and large intestines prosected, and normal and high temperature lesions (determined by capsule localization) will be submitted for histology. Histology will be scored by pathologists blinded to thermal imaging data. Temperature in regions determined to be inflamed by pathology will be compared to temperatures in non-inflamed regions using t-tests. [182] Validation of the IR imaging capsule in a swine model.
- IHC immunohistochemical
- the onboard microprocessor can calculate the standard deviation between a prior frame and the current one and calculate how many more frames are needed for adequate image uniformity. If this should prove ineffective a transceiver can be incorporated to allow the position tracking algorithm to determine the number of frames needed based on device speed and the frame rate can be communicated to the capsule.
- a transceiver can be incorporated to allow the position tracking algorithm to determine the number of frames needed based on device speed and the frame rate can be communicated to the capsule.
- the receiver belt to reconstruct the capsule’s position and velocity we can judge how many frames are reasonable to average. As described above, if we incorporate a receiver inside the capsule, we will be able to adaptively change frame rates during a procedure so that the images have lower thermal noise when the device is moving more slowly. Alternatively, in other embodiments we can actively cool the sensor using a miniature Peltier element mounted to the imaging sensors inside the capsule to reduce background thermal noise.
- the primary benchmarks of success include a capsule design outputs that meet the design input specifications (Tables 4, 5), as determined by verification and preclinical validation. Another benchmark of success is an animal study that demonstrates the capacity to detect 1°C elevations in GI tract temperature that corresponds to histological evidence of inflammation.
- Example 1 Characterization of relationship between inflammation and temperature in the intestine
- the temperature of inflamed tissue typically increases by 0.1-5°C owing to increases in blood flow and increased metabolic rate of inflammatory cells. While this phenomenon has been studied for a variety of human diseases, little is known about the temperature changes seen in the inflamed bowel wall. A better understanding of the relationship between GI temperature elevation and the underlying inflammatory signature is therefore needed characterize temperature-based inflammatory changes in Crohn’s and optimally design a diagnostic thermal imaging capsule.
- histopathology/IHC will be evaluated as a function of distance from the luminal surface to determine the relationship between intraluminal temperature and inflammatory activity depth.
- Animal models [204] As heat generated by inflammation is a fundamental biophysical phenomenon that is conserved across vertebrates and inflammatory diseases, we will study GI intraluminal temperatures in adult (50-75 kg) transmural swine injury models of ileocolonic inflammation. Swine also have similar anatomy and bowel wall thicknesses to those of humans, making results of this Example more clinically translatable.
- bowel diameters of the swine will allow us to conduct intraluminal thermography in vivo with commercially available infrared borescopes that are similar to endoscopes but typically used for industrial purposes.
- Our swine injury models will include intragastric administration of dextran sulfate sodium (DSS) or intra-ileal injection of 2,4,6-trinitrobenzene sulphonic acid (TNBS)- EtOH, both of which are established to produce transmural inflammation in swine. Following the chemical challenge, swine will recover for 7 days at which time a follow up procedure will be conducted to measure intraluminal temperatures along the GI tract.
- DSS dextran sulfate sodium
- TNBS 2,4,6-trinitrobenzene sulphonic acid
- the primary temperature metrics will be the normalized mucosal temperature mean and variance for different grades/metrics of inflammatory activity states and levels. Additional parameters derived from the thermal 40 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 images, including spatial moments and features, will also be investigated. [210] Histopathologic analysis [211] Standard H&E and IHC processing will take place. IHC will include CD45, CD3, CD4, CD20, CD68 and enzyme staining (e.g., metalloproteinases, cathepsins, neutrophil elastase) known to be elevated in inflammatory processes. All slides will be digitized using a whole slide imaging system (Nanozoomer).
- the histopathologic metric e.g., CD45 %area staining
- temperature have a correlation coefficient of 0.6, and an intra-animal correlation of 0.5
- straightforward injury-induced models of inflammation here owing to their simplicity and capacity to be generated in swine that have anatomy similar to humans, more sophisticated models that may more closely resemble Crohn’s disease could be considered if we find that the temperature changes seen in injury models are not conserved in Crohn’s.
- a key benchmark of success will be the demonstration that an increase in temperature reflects histological metrics of inflammation in the small and large bowel. A secondary benchmark will be to show that temperature increases can be distinguished when inflammation is located below the surface of the bowel wall.
- Preliminary results of initial animal studies [220] Two swine studies have been conducted and analyzed. The first study (Swine Study 1) showed a 0.7-1.2 Celsius temperature rise in inflammation with TNBS treatment (to artificially induce inflammation) and histology confirms inflammatory response. Another outcome of this study was that we determined that forward viewing from the capsule is blocked by mucus.
- Example 2 Determine the capacity of mucosal temperature to be measured through fecal contents [222] Since feces are predominantly made up of water that readily transfers heat, fecal temperature should be reflective of that of the bowel wall. Thus, a potential advantage of thermal imaging is the capability of diagnosing bowel wall inflammation through fecal contents. This prospect could enable colonic inflammation detection without requiring patients to undergo full bowel prep.
- Temperature will be monitored at the luminal surface using a thermocouple, a common and reliable electronic device for measuring temperature.
- the voltage to the Peltier cooler will be controlled using feedback from the thermocouple to maintain a constant bowel wall temperature.
- the entire temperature modulation apparatus will be placed in a controlled humidity environment at 37 °C to simulate intraluminal conditions.
- Temperature measurements [230] Thermal imaging will take place for each cleaned bowel segment to measure temperature without feces. Then, a layer of feces from the same anatomical segment will be placed between the mucosal surface and the thermal camera, covered by a glass plate. Fecal thicknesses will be measured using high-frequency (>30 MHz) ultrasound.
- Thermal images of the of the bowel with varying overlying thicknesses of fecal contents and over the range of temperatures representative of presumed none to severe ileocolic inflammation (e.g., 37-42 °C) will be acquired, at increments of ⁇ 0.1°C.
- Thermal equilibration determined to occur when thermal images no longer vary, will take place prior to each temperature measurement.
- the mean and standard deviation of the temperature obtained from the thermal images will be computed.
- the equivalence of temperature measured with and without varying thickness fecal layers will be determined using the student’s t-tests or trimmed t-test assuming unequal variances whenever appropriate.
- the cutoff fecal layer thickness for which equivalence is no longer demonstrated will be ascertained, as a function of thermal detection sensitivity.
- the standard deviation of the temperature measurement is 0.1 °C
- 22 43 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 pairs of measurements per segment (with/without interposed feces) will provide 80% power to provide a two-sided 90% confidence interval that excludes a difference in means of more than 0.1 °C.
- Example 3 Conduct pilot clinical studies using the wireless thermal imaging capsule in Crohn’s patients [237] In this Example, we will conduct pilot clinical studies to demonstrate feasibility and obtain an estimate of efficacy of the thermal imaging capsule for detecting inflammation in Crohn’s patients with and without bowel preparation. [238] Experimental strategy. Patients with Crohn’s disease who have undergone bowel preparation will swallow the thermal imaging capsule and thermal imaging data will be continuously recorded along with capsule position.
- Inclusion criteria are: 1) Confirmed diagnosis of ileocolonic 44 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 Crohn’s disease, 2) Able to consent, 3) Age 18-75 years, and 4) Moderate to severe Crohn’s disease as defined by 220 ⁇ Crohn’s disease activity index (CDAI) ⁇ 450.
- CDAI Crohn’s disease activity index
- Exclusion criteria are: 1) Evidence of active infection, 2) History of ileocolonic resection, 3) Evidence of a small bowel or colonic stricture or obstructive symptoms, 4) Evidence of a small bowel or colonic fistula, 5) Presence of intra-abdominal abscess, 6) Diagnosis of ulcerative or indeterminate colitis, or 7) Pregnancy.
- Protocol To ensure there are no capsule-retaining strictures, subjects will first swallow a patency capsule (e.g., PillcamTM patency capsule) and then will undergo a 24-hr follow up abdominal X-ray to confirm that the capsule has passed.
- a patency capsule e.g., PillcamTM patency capsule
- Subjects (10 who have undergone split-dose 4L PEG-ELS bowel prep, and 10 without bowel prep) will then swallow the thermal imaging capsule disclosed herein. Subjects will don the receiver/recorder belt for 8-10 hours as the capsule traverses the entire GI tract, recording corresponding ileocolonic thermal images and capsule position. After the capsule procedure, regions of low/normal and elevated temperatures will be identified and their locations within the bowel registered. Patients will then undergo sedated ileocolonoscopy 2-3 days after the thermal capsule imaging procedure. During endoscopy, biopsies will be taken from the sites identified by thermal capsule imaging. [241] Data analysis. Biopsy slides will be evaluated for inflammation as described in Example 1. Thermal capsule signatures for normal vs.
- inflamed regions as determined by histology, will be compared using t-tests.
- the sensitivity and specificity of thermal capsule imaging for discriminating inflamed from uninflamed bowel wall will be determined using histology as the gold standard.
- Sensitivity/specificity for cohorts with and without bowel prep will be compared using a Fisher’s exact test.
- Receiver operating characteristic (ROC) curve will be used to assess the classification power of the proposed method.
- Example 4 Development of a Super Resolution Neural Network for IR Image Enhancement
- GAN Generative Adversarial Network
- ToF methods rely on rapidly taking subsequent low resolution (LR) images with known shifts in position and then interpolating between images to recover sub-pixel information.
- Prior endoscopy reconstruction techniques would estimate these displacements using the optical flow of RGB data from the endoscope camera.
- GAN networks While ToF requires multiple images to produce a single super resolution image, GAN networks have a 1:1 input to output for images. GAN networks function by training two competing networks: A generating network creates a high resolution version of a low resolution image, and simultaneously a discriminator network attempts to determine whether the generated image is a generated high resolution image or ground truth initially taken high resolution image. These competing networks allow for the generation of images with accurate resolution increases of 10-12x increase in one dimension. [252] Prior works have adapted various loss functions for medical images.
- Pixel loss evaluates pixel by pixel differences of a generated super resolution (SR) image to its corresponding ground truth HR image via an L1 loss function.
- Content loss evaluates the feature maps of the SR image to its HR image using pre-extracted feature maps typically from pre-trained, off the shelf feature extractors like VGG or ResNet using the Euclidean distance between feature representations.
- Texture loss is defined as the Gram matrix formed by the inner products of the vectorized feature maps for a given convolutional layer. This helps to introduce directional correlations to the learning process.
- adversarial loss has been shown to increase stability during training and result in better convergence.
- FIG.19 an example 1900 of a system (e.g. a data collection and processing system) for detecting gastrointestinal inflammation is shown in accordance with some embodiments of the disclosed subject matter.
- a computing device 47 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 1910 can receive thermal IR data from a thermal IR capsule 1900.
- computing device 1910 can execute at least a portion of a system for detecting gastrointestinal inflammation 1904 to detect gastrointestinal inflammation based on the thermal IR data received from thermal IR capsule 1900.
- computing device 1910 can communicate information about the thermal IR data received from thermal IR capsule 1900 to a server 1920 over a communication network 1906, which can execute at least a portion of system for detecting gastrointestinal inflammation 1904 to detect gastrointestinal inflammation based on the thermal IR data.
- server 1920 can return information to computing device 1910 (and/or any other suitable computing device) indicative of an output of system for detecting gastrointestinal inflammation 1904, such as the thermal IR information. This information may be transmitted and/or presented to a user (e.g. a researcher, an operator, a clinician, etc.) and/or may be stored (e.g. as part of a research database or a medical record associated with a subject).
- computing device 1910 and/or server 1920 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, etc.
- IR sensor 1902 can be located locally and/or remotely from computing device 1910, and can communicate information to computing device 1910 (and/or server 1920) via a communication network (e.g., communication network 1906).
- communication network 1906 can be any suitable communication network or combination of communication networks.
- communication network 1906 can include a Wi-Fi network (which can include one or more 48 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 4G network, a 5G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, etc.
- Wi-Fi network which can include one or more 48 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 wireless routers, one or more switches, etc.
- a peer-to-peer network e.g., a Bluetooth network
- a cellular network e.g., a 4G network, a 5G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE
- communication network 1906 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks.
- Communications links shown in FIG.19 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, etc.
- FIG.20 shows an example 2000 of hardware that can be used to implement computing device 1910 and server 1920 in accordance with some embodiments of the disclosed subject matter.
- computing device 1910 can include a processor 2002, a display 2004, one or more inputs 2006, one or more communication systems 2008, and/or memory 2010.
- processor 2002 can be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc.
- display 2004 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc.
- inputs 2006 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, etc.
- communications systems 2008 can include any suitable hardware, firmware, and/or software for communicating information over communication network 1906 and/or any other suitable communication networks.
- communications systems 2008 can include one or more transceivers, one or more communication chips and/or chip sets, etc.
- communications systems 2008 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, etc.
- memory 2010 can include any suitable storage device or devices that can be used to store instructions, values, etc., that can be used, for example, by processor 2002 to present content using display 2004, to communicate with server 1920 via communications system(s) 2008, etc.
- server 1920 can include a processor 2012, a display 2014, one or more inputs 2016, one or more communications systems 2018, and/or memory 2020.
- processor 2012 can be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc.
- display 2014 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc.
- inputs 2016 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, etc.
- communications systems 2018 can include any suitable hardware, firmware, and/or software for communicating information over communication network 1906 and/or any other suitable communication networks.
- communications systems 2018 can include one or more transceivers, one or more communication chips and/or chip sets, etc.
- communications systems 2018 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, etc.
- memory 2020 can include any suitable storage device or devices that can be used to store instructions, values, etc., that can be used, for example, by processor 2012 to present content using display 2014, to communicate with one or more computing devices 1910, etc.
- Memory 2020 can include any suitable volatile memory, non- volatile memory, storage, or any suitable combination thereof.
- memory 2020 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, etc.
- memory 2020 can have encoded thereon a server program for controlling operation of server 1920.
- processor 2012 can execute at least a portion of the server program to transmit information and/or content (e.g., results of a tissue identification and/or classification, a user interface, etc.) to one or more computing devices 1910, receive information and/or content 50 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 from one or more computing devices 1910, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), etc.
- any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein.
- computer readable media can be transitory or non-transitory.
- non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as RAM, Flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
- media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as RAM, Flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
- EPROM electrically programmable read only
- transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
- the optical signals are detected by photodiodes. It should be recognized that any opto-electronic conversion device including but not limited to photo detectors, photodiodes, line-scan and two-dimensional cameras, and photodiode arrays can be used to perform this detection function.
- the term mechanism can encompass hardware, software, firmware, or any suitable combination thereof.
- FIG.21 shows an example 1200 of a process for detecting gastrointestinal inflammation in accordance with some embodiments of the disclosed subject matter.
- process 2100 can provide a capsule comprising a thermal imaging sensor, where the capsule may be configured to be swallowed by a subject.
- process 2100 can detect, using the thermal imaging sensor, infrared radiation emitted by a tissue of the subject.
- FIG.22 shows an example 2200 of a process for detecting gastrointestinal inflammation in accordance with some embodiments of the disclosed subject matter.
- process 2200 can provide a capsule comprising a thermal imaging sensor, a wireless transmitter, and a plurality of antennas in communication with a controller, where the capsule may be configured to be swallowed by a subject.
- process 2200 can detect, by the thermal imaging sensor, infrared radiation emitted by a tissue of the subject.
- process 2200 can generate, by the controller, at least one image of the 51 Q B ⁇ 85925099.1 MGH 2021-602-03 Quarles 125141.04435 detected infrared radiation emitted by the tissue of the subject.
- process 2200 can transmit, by the controller, the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas. Finally, at 2210, process 2200 can receive, by the plurality of antennas, the at least one image from the wireless transmitter.
- FIG.34 shows a system block diagram of operation of embodiments of the device, going from capsule measurement transmission then reception to processing and storage between the capsule and the receiver belt.
- the thermistor sensor array of the capsule measures luminal temperature of the user.
- the on board controller of the capsule then processes and encrypts the data.
- the capsule wirelessly transmits the data through the tissue to the external receiver of the antenna array to the receiver belt.
- the external receiver sweeps and stores all the antenna amplitudes from the signal and then updates the receiver antenna to record the signal to be the highest fidelity input, at step 50. Simultaneous with step 40, the external receiver also receives and stores the data at step 42.
- the receiver processes the data in the neural networks to determine a position of the capsule at step 44. Finally, at step 46 the receiver transmits the signal back to the capsule to perform one or more function including: releasing of the drug payload, providing information on the position, adjusting the sensitivity and filter settings for thermal measurements.
- the data is processed by the receiver belt to send a signal back to the wireless capsule to adjust its sensor settings and framerate, as well as to guide further actions, such as releasing drug payloads or informing the capsule of its current position.
- a first network (Network 1) is configured to resolve tissue thickness and determine the position of the capsule.
- the neural network for the capsule is trained using a generated dataset that simulates the electromagnetic fields generated 58 by the capsule for various tissue depths, capsule angles, and capsule positions relative to an antenna. Simultaneously, the input antenna configuration is used to generate a multi-antenna measurement.
- FIG.37 shows a second network (Network 2) that is configured to determine tissue inflammation and pathology severity 68.
- the second network takes the capsule position, computed local tissue thicknesses, and thermal readings to determine the pathology depth and/or severity in tissue 66.
- the network can compute using the past, present, and future readings for all inputs at once.
- the second neural network is trained by generating thermal fields that are produced by varying size, intensity, depth, heat type, and location of pathology are simulated within the tissue and propagating to the capsule body 70. These are varied for tissue thickness, capsule position, and rotation, and allowed to be guided by any combination of current, past, and present measurements. Reasoning for each input is given below.
- Input from the first network takes capsule position and tissue thickness as inputs, as well as the past, present, and if evaluated, post-procedure future inputs. Tissue thickness helps determine thermal diffusion to resolve depth and calculate thermal physics to judge the depth and severity of a site.
- Position helps determine if the capsule is near any major thermal bodies, such as an artery or thermally significant organ due to metabolism or perfusion, such as the heart, thyroid gland, kidneys, liver, or bladder.
- the past and present thermal readings as an input help judge the ambient temperatures and determine if the capsule is in the presence of a large swatch of pathology. If evaluated post-procedure, the future inputs are also processed.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Optics & Photonics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Evolutionary Computation (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Endoscopes (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
An apparatus for detecting gastrointestinal pathology, including: a capsule including at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject.
Description
MGH 2021-602-03 Quarles 125141.04435 WIRELESS THERMAL DETECTION CAPSULE CROSS-REFERENCE TO RELATED APPLICATIONS [1] The present application is based on and claims priority from U.S. Patent Application Ser. No.63/384,564, filed on November 21, 2022, the entire disclosure of which is incorporated herein by reference. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH [2] N/A BACKGROUND [3] Video capsule endoscopy (VCE) has shown promise as a non-invasive tool for Crohn’s disease diagnosis and monitoring but has several drawbacks including the fact that it cannot obtain data from below the tissue surface, it is unable to detect disease in the early stages, and it typically requires good bowel prep for adequate visualization of the colon. SUMMARY OF THE INVENTION [4] Accordingly, new systems, methods, and apparatus for detecting gastrointestinal pathology are desirable. [5] Thus, various embodiments provide a swallowable, wireless, thermal detection (e.g., using temperature sensors and/or infrared (IR) imaging) capsule that allows for non- invasive diagnosis and monitoring of inflammatory gastrointestinal disease and other gastrointestinal pathologies. VCE has become a routine method used in medical practice for identifying gastrointestinal disease and has had success as a non-invasive tool for Crohn’s disease diagnosis and monitoring of disease progression. [6] One barrier to the optimal management of inflammatory conditions such as Crohn's disease has been the lack of a sensitive, non-invasive, and easily administered way to diagnose inflammation in the gut. A swallowable, wireless capsule technology capable of identifying inflammatory activity can change this outlook by providing one or more of: early diagnosis, routine monitoring, treatment optimization, and/or a way to locally deliver therapies in conditions such as Crohn's disease. [7] Active inflammation in other organ systems generates a temperature rise in tissue detectable through infrared (IR) blackbody radiation. Thus, while VCE is limited to 1 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 viewing the luminal surface and is blind to early disease, heat diffuses readily through water rich tissue which could allow thermal imaging to overcome the limitations of VCE. The diagnostic capability of one potential thermal imaging sensing technology, IR sensor arrays, may be limited by their low pixel resolution; therefore, we have instead developed a detection (e.g., via temperature sensors and/or thermal IR imaging) capsule that can utilize super- resolution neural networks to increase resolution and identify intestinal pathology (e.g., by detecting inflammation or other tissue conditions) that arises deep in the submucosal/muscular layers. [8] Embodiments of the disclosed thermal imaging capsule provide a mechanism for more sensitively detecting inflammation or other indications of pathology at any depth in the bowel wall, with little or no bowel prep, which is expected to make the device particularly suitable for Crohn’s disease monitoring. In certain embodiments which are equipped with a local therapy delivery mechanism, the thermal imaging capsule may also allow inflamed tissue to be treated directly, mitigating systemic side effects, thereby providing better outcomes for Crohn’s patients. [9] Accordingly, in various embodiments the disclosure provides one or more of: 1) a wireless thermal detection capsule system, 2) a super-resolution neural network for capsule endoscopy, and 3) mechanisms for detecting inflammatory gut foci in conditions such as Crohn’s disease. The various embodiments provide a platform for both diagnosis and routine monitoring to enable therapeutic optimization and targeted treatment of Crohn’s disease and/or other pathologies. [10] Accordingly, one embodiment provides an apparatus for detecting gastrointestinal pathology, including: a capsule including at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject. [11] Another embodiment provides a system for detecting gastrointestinal pathology, including: a capsule comprising at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject; and a receiver belt configured to be coupled to the subject. [12] Yet another embodiment provides a method for detecting gastrointestinal pathology, including: providing a capsule comprising at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature 2 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 sensor being configured to circumferentially detect heat emitted by a tissue of the subject; and circumferentially detecting, using the at least one temperature sensor, heat emitted by a tissue of the subject. [13] Another embodiment provides an apparatus for detecting gastrointestinal pathology, including: a capsule including a thermal imaging sensor, the capsule being configured to be swallowed by a subject, and the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject. [14] Another embodiment provides a system for detecting gastrointestinal pathology, including: a capsule including a thermal imaging sensor in communication with a controller and a wireless transmitter; and a plurality of antennas in communication with the capsule, the capsule being configured to be swallowed by a subject, the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject, the controller being configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject, the controller being configured to transmit the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas, and the plurality of antennas being configured to receive the at least one image from the wireless transmitter. [15] Yet another embodiment provides a method for detecting gastrointestinal pathology, including: providing a capsule including a thermal imaging sensor, the capsule being configured to be swallowed by a subject; and detecting, using the thermal imaging sensor, infrared radiation emitted by a tissue of the subject. [16] Still another embodiment provides a method for detecting gastrointestinal pathology, including: providing a capsule including a thermal imaging sensor, a wireless transmitter, and a plurality of antennas in communication with a controller, the capsule being configured to be swallowed by a subject, detecting, by the thermal imaging sensor, infrared radiation emitted by a tissue of the subject; generating, by the controller, at least one image of the detected infrared radiation emitted by the tissue of the subject; transmitting, by the controller, the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas; and receiving, by the plurality of antennas, the at least one image from the wireless transmitter. BRIEF DESCRIPTION OF THE DRAWINGS [17] Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed 3 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements. [18] FIG.1 shows a diagram of a system for detecting gastrointestinal inflammation or other pathology, depicting a thermal IR imaging capsule moving through the gastrointestinal tract and obtaining thermal IR image information which can then be presented as images, e.g. using a pseudocolor scale that represents different temperatures. [19] FIG.2 shows a construction of a wireless thermal IR imaging capsule according to the disclosure, indicating a catadioptric reflector, an IR sensor, a power supply/battery, a viewing window (LDPE), a printed circuit board (PCB), and an antenna. [20] FIG.3 shows a construction of a catadioptric reflector such as that shown with the capsule of FIG.2; the left panel shows a perspective view of the pyramid-shaped reflector with an indication of which portions of the reflector reflect which portions of the IR sensor (the top of the pyramid reflecting the inner 16x16 pixels of the IR detector and the lower portion of the pyramid reflecting the remaining outer pixels of the IR detector); the two right panels show ZEMAX diagrams of the light paths off the reflector. [21] FIG.4 shows several embodiments of reflectors for reflecting light from a sample onto the IR sensor; in the left panel, the photo below shows the IR sensor with a conical reflector and the diagram above the photo shows the field of view of the IR sensor with the conical reflector, with an indication of where there is a "blind spot" on the IR sensor which does not receive reflected light; in the center panel, the photo below shows the IR sensor with a wedge-shaped reflector and the diagram above the photo shows the field of view of the IR sensor with the wedge-shaped reflector, with an indication of where there are "blind spots" on the IR sensor which do not receive reflected light; in the right panel, the photo below shows the catadioptric reflector of FIGS.2 and 3 and the diagram above the photo shows the field of view of the IR sensor with the catadioptric reflector, with an indication of where there are "blind spots" on the IR sensor which do not receive reflected light; as shown in the right panel, the catadioptric reflector minimizes blind spots and maximizes utilization of the IR sensor. [22] FIG.5, top panel, shows a photo of a PCB for use with constructions of a wireless thermal IR imaging capsule, where the PCB has an IR sensor attached thereto as well as a snap-off programmer element which provides connections for programming the device and which can be removed before placing the PCB into the capsule; the bottom panel of FIG.5 shows a construction of a wireless thermal IR imaging capsule as in FIG.2 with a 4 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 rounded reflector. [23] FIG.6A shows a diagram of a receiver module attached to a subject (an animal subject used during testing) with several antennas connected to the receiver module via coaxial connectors. [24] FIG.6B shows a diagram of a subject with a capsule device in their GI tract with an array of antennas attached to the subject's body using adhesives. [25] FIG.7 shows a photo of a circuit board for the receiver module which includes an RF multiplexer connected to eight input channels, each of which includes a coaxial connector for establishing a connection to an antenna. [26] FIG.8 shows (upper panels) an experimental setup for generating an elevated temperature spot for performing ex vivo tests of a wireless thermal IR imaging capsule and (lower panels) thermal IR imaging results obtained from viewing the high temperature spot with the capsule device through an excised portion of ileum tissue. [27] FIG.9 shows a layered dielectric model of human tissue for RF transmission from the lumen to the skin surface. Dimensions are based on anatomical CT and ultrasound studies on the US population accounting for gender, weight, and variations of diseased states of tissue to provide a maximum upper bound on total attenuation (dielectric parameters are taken from the IT’IS foundation’s tissue database). [28] FIG.10 shows, using the model of FIG.9, the reflection and transmission loss which were calculated at 433MHz using the max thickness of recorded tissues in both healthy and diseased patients. The results were validated in a 43kg swine tissue model. [29] FIG.11 shows a prototype IR capsule PCB (panel a), capsule length (panel b), and capsule front (panel c). [30] FIG.12 shows a block diagram of the electronic communications and powering of the IR capsule device. [31] FIG.13 shows a photo of an experimental setup for performing an ex vivo test of a construction of a wireless thermal IR imaging capsule. [32] FIG.14 shows a diagram of an experimental setup for performing ex vivo tests of a wireless thermal IR imaging capsule. [33] FIG.15 shows an ideal image (left panel) and 8-frame averaged images for an insertion 1cm inside (center panel) the heated section and at the very end of the heated section of the tissue (3cm, right panel). Images were taken by the capsule inside the intestine. An area of overlap of the resistive heating pad resulted in a warm corner of the images and is 5 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 marked "T.C." in the images. [34] FIG.16A shows a general schematic of one construction of a thermal IR imaging capsule with a single IR sensor. The outer shell is made of PMMA. IR-Infrared; PCB-printed circuit board; PMMA-Polymethyl methacrylate. FIG.16B shows a general schematic of one construction of a thermal IR imaging capsule with two IR sensors, one at each end of the capsule. PCB – printed circuit board; PMMA - Polymethyl methacrylate. FIG.16C shows a diagram of another construction of a thermal IR imaging capsule with one IR sensor. [35] FIG.17 shows an overview of an entire IR capsule system from a data stream perspective. PCB-Printed Circuit Board; EEPROM-Electrically Erasable Programmable Read-Only Memory; IR-Infrared. [36] FIG.18 shows a schematic of an experimental temperature measurement apparatus. [37] FIG.19 shows an example of a system for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter. [38] FIG.20 shows an example of hardware that can be used to implement computing device and server in accordance with some embodiments of the disclosed subject matter. [39] FIG.21 shows an example of a process for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter. [40] FIG.22 shows another example of a process for detecting gastrointestinal inflammation or other pathology in accordance with some embodiments of the disclosed subject matter. [41] FIG.23 shows a construction of a wireless thermal detection capsule according to the disclosure, indicating a front cap, a thermistor block, a processing unit and wireless transmission, a main power source, and a back cap. [42] FIG.24 shows an exploded cross-sectional view of a wireless thermal detection capsule indicating a front cap, a capsule wall, a thermistor ring, a drug reservoir, a battery, a printed circuit board 1 (PCB1), a printed circuit board 2 (PCB2), a printed circuit board 3 (PCB3), a printed circuit board 4 (PCB4), a battery, and a back cap. 6 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [43] FIG.25 shows a perspective view of a wireless thermal detection capsule. [44] FIG.26A shows a side view of the thermistor block of FIG.24, indicating a capsule wall and thermistor rings. [45] FIG.26B shows a front view of printed circuit board 1 (PCB1) of FIG.24, including thermistor rings. [46] FIG.27A shows a front view of the printed circuit board 2 (PCB2) of FIG.24. [47] FIG.27B shows a processing unit and wireless transmission of FIG.23, including a printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), and an antenna. [48] FIG.27C shows a rear view of printed circuit board 3 (PCB3) of FIG.24. [49] FIG.28A shows a front view of printed circuit board 2 (PCB2) of FIG.24. [50] FIG.28B shows a side view of the processing unit and wireless transmission of FIG.23, including locations of printed circuit board 2 (PCB2) and printed circuit board 3 (PCB3). [51] FIG.28C shows a rear perspective view of printed circuit board 3 (PCB3) of FIG.24. [52] FIG.29A shows a processing unit and wireless transmission of FIG.23, including a drug reservoir, printed circuit board 2 (PCB2), and printed circuit board 3 (PCB3). [53] FIG.29B shows a processing unit and wireless transmission of FIG.23, including a drug reservoir, an antenna, printed circuit board 2 (PCB2), and printed circuit board 3 (PCB3). [54] FIG.30A shows a side view of the main power source of FIG.23. [55] FIG.30B shows a side view of the battery of the main power source of FIG. 23. [56] FIG.30C shows a front view of the main power source of FIG.23. [57] FIG.31A shows a front view of printed circuit board 4 (PCB4) of FIG.24. [58] FIG.31B shows a side view of printed circuit board 4 (PCB4) of FIG.24. [59] FIG.31C shows a rear side view of printed circuit board 4 (PCB4) mount for contact with the main power source of FIG.23. [60] FIG.32A shows an example of internal circuit board housing of printed circuit board 1 (PCB1), printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), and printed circuit board 4 (PCB4) of FIG.24. [61] FIG.32B shows an example of internal circuit board housing of printed circuit 7 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 board 1 (PCB1), printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), and printed circuit board 4 (PCB4) of FIG.24. [62] FIG.33A shows the example folded and assembled internal circuit board housing of printed circuit board 1 (PCB1), printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), printed circuit board 4 (PCB4), and the thermistor rings of FIG.24. [63] FIG.33B shows the example folded and assembled internal circuit board housing of printed circuit board 1 (PCB1), printed circuit board 2 (PCB2), printed circuit board 3 (PCB3), printed circuit board 4 (PCB4), and the thermistor rings of FIG.24. [64] FIG.34 shows a system block diagram of the capsule of FIG.23 showing steps from measurement to transmission, then showing steps from reception to processing and storage. [65] FIG.35A shows a receiver belt, indicating antenna positions, a shoulder strap, a belt, and a holster. [66] FIG.35B shows the antenna positions of FIG.12A, including a shoulder strap, a holster, and a belt. [67] FIG.36 shows a functional block diagram of Network 1. [68] FIG.37 shows a functional block diagram of Network 2. DETAILED DESCRIPTION [69] In accordance with some embodiments of the disclosed subject matter, mechanisms (which can include systems, methods, and apparatus) for detecting gastrointestinal pathology are provided. [70] Current capsule endoscopy operates in the visible or near infrared spectrum. As a result, only the surface of the lumen can be inspected, bowel prep is required, and the method is blind to early stages of inflammatory disease. Given that inflammation in other body systems has been shown to induce notable temperature rises, the disclosed embodiments are directed at enabling more sensitive and earlier detection of inflammatory diseases of the GI tract, including but not limited to Crohn’s disease, with a swallowable capsule configured to detect heat emitted by a tissue of a subject, e.g., using one or more temperature sensors and/or thermal infrared (IR) imaging. [71] Thus, certain embodiments provide an apparatus, system, and method for detecting gastrointestinal pathology which may include a capsule including at least one temperature sensor, where the capsule may be configured to be swallowed by a subject. The 8 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 at least one temperature sensor may be configured to circumferentially detect heat emitted by a tissue of the subject. [72] In some embodiments, the at least one temperature sensor may include a thermistor including a thermistor ring; the thermistor ring may include a plurality of thermistors arranged around a cylindrical wall of the capsule (FIGS.23-37). In particular embodiments, the plurality of thermistors in the thermistor ring may be configured to detect a spatial pattern of heat emitted by the tissue of the subject. In various embodiments, the plurality of thermistors may be arranged partway around the cylindrical wall of the capsule (e.g., around one half, one third, one fourth etc. of the capsule circumference) or may be arranged completely around the cylindrical wall of the capsule. [73] In certain embodiments the capsule may include two or more groups of thermistors that are arranged around the cylindrical wall of the capsule. In some embodiments, the various arrays, groups, or rings of thermistors may be aligned with one another and in other embodiments the groups or rings of thermistors may be offset from one another. When the arrays, groups, or rings of thermistors are arranged in an offset manner around the cylindrical wall of the capsule, they are able to obtain information from the tissue at a higher level of resolution than a single array or several arrays that are aligned. [74] In various embodiments, data obtained from multiple arrays of thermistors arranged around the cylindrical wall of the capsule can be used to improve a spatial resolution of an image formed using the data, determine a velocity of movement of the capsule within the tissue of the subject, and/or reduce an amount of noise related to transmission of the data. [75] In some embodiments, the cylindrical wall of the capsule may include or be composed of a thermally conductive material, where the arrays of thermistors may be in thermal contact with the thermally conductive material of the cylindrical wall of the capsule. In various embodiments, the cylindrical wall may be made of a material including any linear combination of the following material as the core conductive element in the form of a solid, thermal paste, or thermally conductive adhesive. Materials could then be protected by a thin layer of biocompatible materials such as Polycarbonate, PMMA, Polystyrene, etc. The core conductive element may be or include one or more of: metals or other allows such as platinum, gold, silver, copper, bronze, and/or aluminum; or carbon composites and crystals such as diamond, carbon nanotube sheets, and/or carbon matrix composites (CAMCs). In other embodiments, the cylindrical wall may be made without a thermally conductive 9 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 material or with a material with low thermal conductance such that in the absence of thermal conductors, contact between the tissue and the temperature sensors may be facilitated by application of pressure between the tissue and the capsule wall as well as minimizing the capsule wall thickness at the points of contact of the sensors to reduce thermal insulation from the sensor through the wall to the tissue. [76] In some embodiments, the capsule may include a flexible printed circuit board, where the arrays or thermistors may be coupled to the flexible printed circuit board (FIGS.33-34). In certain embodiments the capsule may include a power source such as a battery which is in electrical contact with the printed circuit board (FIGS.23-26, 31). [77] In various embodiments, the capsule may further include a controller (e.g., including a processor and/or memory) in communication with the thermistor array(s). The controller may be configured to obtain data from the thermistor arrays pertaining to the detected heat that is emitted from the tissue of the subject. In some embodiments, the controller may be further configured to encrypt the data obtained from the at least one temperature sensor, e.g., to maintain data integrity and/or patient confidentiality. [78] In particular embodiments, the capsule may further include a wireless transmitter and an antenna in communication with the controller. The controller may be configured to transmit the data that is obtained from the thermistors via the wireless transmitter and the antenna to an external device for at least one of storage or analysis of the transmitted data. The external device may be a control unit that is attached to the subject using a receiver belt (see below). In some embodiments, the controller may be further configured to generate at least one image based on the data obtained from the at least one temperature sensor; in other embodiments, the external device which receives the transmitted data may be configured to generate an image. In still other embodiments, the external device may transfer the data to another component (e.g., a remote computer or server) for further processing, including generation of images another information based on the data. [79] In various embodiments, the controller, external device, or another system may be further configured to generate a plurality of images based on the data obtained from the thermistors. The plurality of images may be based on data obtained at a particular rate of data capture from the at least one temperature sensor. In some embodiments, the controller may be further configured to receive a signal from the external device which instructs the controller to adjust at least one of the rate of data capture (e.g., from 0.1-1 MHz) from the thermistors or an operational setting of the thermistors. In various embodiments, a number of 10 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 frames may be averaged to create a final image frame for further analysis and/or viewing. In some embodiments the frames may be sampled at a rate of 100 kHz, where these sampled frames may then be averaged to further improve accuracy by reducing thermal noise (e.g., by 1 / (sqrt(# frames)); in general, the number of frames that are averaged to produce a single final frame may be varied based on a tradeoff between speed and noise levels. Thus, in certain embodiments, the operational settings of the thermistors may include "Voltage/Current thermistors are driven with", "Number of frames averaged from thermistors prior to transmission", and/or "Waveform thermistors are driven with (duty cycle, sin vs square vs triangle vs saw tooth vs trapezoid vs sinc vs custom shape)". [80] In some embodiments, the capsule may further include a drug reservoir, where the controller may be configured to receive a signal from the external device and release a drug payload from the drug reservoir based on receiving the signal from the external device. In some embodiments, the controller may be further configured to detect a target site in the tissue based on the detected heat emitted from the tissue of the subject and release a drug payload from the drug reservoir based on detecting the target site. In various embodiments, the drug payload may include one or more of corticosteroids, immunosuppressants (e.g., chemotherapy agents), biologics (e.g., Anti TNF-Alpha, Jak inhibitors, etc.), gene therapies (e.g., adenovirus delivered siRNA, gene transfer, CART, TIL, mRNA, RNAi, oncolytic virus therapy, and gene editing CRISPR-Cas9, TALEN, prodrug activation, immunotherapy, and/or small molecule therapy) and/or other agents (see Table 1 for list of possible drugs). Corticoids Immunosuppressants Biologics ^ Budesonide (mild to ^ Thiopurines ^ Anti-TNF
11 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 ^ Clinical remission in refractory and luminal Crohn’s )
g p y p p [82] In various embodiments, the controller may be further configured to receive a signal from the external device which includes information regarding a current position of the capsule within the subject. In particular embodiments, the wireless transmitter may be configured to transmit to and from the external device at a frequency in a range of between 300-500 MHz, and in certain embodiments in a range of between 430MHz and 440MHz, which is suitable for transmitting through body tissue. [83] In some embodiments, the capsule may further include a thermal imaging sensor which is configured to detect infrared radiation emitted by the tissue of the subject. In particular embodiments, the capsule may include a cylindrical housing and the thermal imaging sensor may be disposed at an end of the cylindrical housing. In certain embodiments, the thermal imaging sensor may be configured to detect infrared radiation comprising a wavelength in a range of between 7μm to 14μm. In particular embodiments, the thermal imaging sensor may include a thermopile sensor. [84] In some embodiments, the capsule may further include an ultrasound transducer coupled to the controller. The ultrasound transducer may be configured to obtain data that is indicative of a depth of fecal matter between the capsule and the tissue of the subject. In various embodiments, the controller may be further configured to obtain data from the ultrasound transducer that is indicative of a depth of fecal matter between the capsule and the tissue. The controller may be further configured to transmit the data from the ultrasound transducer that is indicative of the depth of fecal matter between the capsule and the tissue to the external device using the wireless transmitter and the antenna. [85] In some embodiments, the capsule may include a window which transmits 12 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 (e.g., is transparent to or semi-transparent to) infrared radiation, wherein the window may be adjacent to the thermal imaging sensor. In particular embodiments, the capsule may also include a reflector disposed adjacent to the thermal imaging sensor, where the reflector may be configured to direct infrared radiation from the tissue towards the thermal imaging sensor. In some embodiments, the reflector may be configured to direct infrared radiation from a circumferential region around the capsule, e.g., from a ring of tissue surrounding the capsule. In various embodiments, the reflector may be conical or pyramid shaped, and in particular embodiments the pyramid shaped reflector may include an apex including a triangular wedge. [86] In various embodiments, the capsule may be part of a system which includes a receiver belt including an external device and a plurality of receiver belt antennas in communication with the external device, where the receiver belt is worn by the subject and the external device is in communication with the capsule during a procedure. In some embodiments, the receiver belt include a waist belt and a shoulder strap, where the waist belt and the shoulder strap may be configured to hold the plurality of receiver belt antennas in a fixed position relative to the body of the subject. [87] In certain embodiments, the controller may be further configured to receive a signal from the external device, where the signal received from the external device may include information regarding a current position of the capsule within the subject. In particular embodiments, the information regarding the current position of the capsule within the subject may be determined based on information from the plurality of receiver belt antennas. The information from the receiver belt antennas may be used to determine the current position of the capsule within the subject based on a comparison of the information to a predetermined set of data regarding the relationship between antenna signal attenuation vs. tissue depth. [88] In some embodiments, the information regarding the position of the capsule may be obtained using neural networks. In particular embodiments, the information from the plurality of receiver belt antennas may be processed by a first neural network to identify at least one of a type of tissue or a tissue thickness between the capsule and a surface of the subject adjacent the plurality of receiver belt antennas, where the current position of the capsule may be determined based on the type of tissue and/or the tissue thickness. In various embodiments, information including one or more of the current position of the capsule, the at least one of the type of tissue or the tissue thickness, and/or the data obtained from the at least one temperature sensor may be processed by a second neural network to determine at least 13 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 one of a pathology depth, a pathology type, or a pathology severity in the tissue of the subject. In certain embodiments, the information from the plurality of receiver belt antennas may be processed (e.g., using a neural network or other procedure) to identify at least one of a type of tissue or a tissue thickness between the capsule and a surface of the subject adjacent the plurality of receiver belt antennas. [89] In various embodiments, the apparatus and/or systems above may be used to carry out a method for detecting gastrointestinal pathology. In one embodiment, the controller of the capsule may be configured to obtain data from the thermistors relating to the detected heat emitted from the tissue of the subject at a first time, where the data may be used (e.g., by the controller, the external device, and/or a further system) to determine a first position of the capsule at the first time based on information obtained from the plurality of receiver belt antennas. [90] In some embodiments, a first image of the tissue of the subject at the first position of the capsule may be generated (e.g., by the controller, the external device, and/or a further system) based on the data from the first plurality of thermistors and the second plurality of thermistors relating to the detected heat emitted from the tissue of the subject at the first time. In another embodiment, a second set of data may be obtained from the thermistors relating to the detected heat emitted from the tissue of the subject at a second time, and this second set of data may be used to determine a second position of the capsule at the second time based on information from the plurality of receiver belt antennas. In certain embodiments, a second image of the tissue of the subject at the second position may be generated (e.g., by the controller, the external device, and/or a further system) based on the data relating to the detected heat emitted from the tissue of the subject at the second time. In some embodiments, a map of heat emitted from the tissue of the subject may be generated (e.g., by the controller, the external device, and/or a further system) based on the first image at the first position and the second image at the second position. In other embodiments, information from the first and second images may be combined to create an improved single third image, which may also be used to generate the map of emitted heat. Accordingly, a detailed map of the tissue of the subject along with information relating to pathologies of the tissue may be obtained from the thermistors as the capsule moves through the subject’s body. [91] Although in the discussion above the temperature sensor has been referred to primarily as a thermistor, in various embodiments the temperature sensor may include one or more of a thermistor, a thermocouple, a semiconductor-based sensor, and/or a resistance- 14 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 based temperature sensor. Furthermore, the various calculations or actions may be conducted by one or more of the controller in the capsule, the external device, and/or another unit that is in communication with the capsule controller and/or the external device. [92] Given that active inflammation may generate an increase in tissue temperature that is manifested as radiated infrared (IR) light (wavelength: 7-14 μm), in various embodiments the disclosed procedures employ this insight to provide embodiments of the capsule that include IR imaging capability. Thus, certain embodiments may provide a swallowable, wireless, IR imaging-capable capsule that images mucosal temperature and thus inflammatory activity, along with systems and methods for collecting and processing the data from the capsule. Notably, because heat diffuses through water-rich tissue and fecal contents, in various embodiments the capsule may also identify bowel inflammation which arises deep in the submucosal/muscular layers; as a result, the disclosed thermal IR imaging capsule- based procedures may require little or no colonic bowel preparation. One or more embodiments provides procedures for at least one of: 1) characterizing the relationship between bowel wall inflammation and temperature, 2) measuring the bowel wall temperature through feces, 3) operating a wireless thermal imaging capsule, and/or 4) detecting inflammatory gut foci in subjects with inflammatory bowel conditions such as Crohn’s disease. Embodiments of the disclosed procedures provide a new platform technology that can be used for diagnosis, routine monitoring of disease activity, therapeutic optimization, and targeted treatment of conditions such as Crohn’s disease. [93] Crohn’s Disease (CD) is a chronic disease characterized by inflammation and irritation of any part of intestinal tract from the mouth to the anus. Patients with Crohn’s disease experience severe abdominal pain, fever, bowel obstruction, diarrhea, and blood or mucus, or both, in fecal matter. Extra-intestinal symptoms commonly include anemia, arthritis, and low bone density, with less common complications characterized by inflammation of the skin, eyes, and liver. Patients also run an increased risk of developing colon cancer. These painful symptoms drastically affect a patient’s quality of life, productivity, and healthy lifespan. [94] The prevalence of Crohn’s disease in the US population was recently re- evaluated based on a 61-63 million annually reported patient conglomerate of U.S. healthcare databases. The study found a total pediatric prevalence of Crohn’s disease was 45.9 per 100,000 in 2016 rising 148% from 18.5 per 100,000 in 2007. Interestingly, the vast majority of pediatric cases were initially diagnosed in the 10-17 year age group. Adult prevalence was 15 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 197.7 per 100,000 in 2016 up by 125% from 88.0 out of 100,000 in 2007. [95] The risk of developing Crohn’s disease is influenced by both environmental and genetic factors. Genetic factors have seen the most progress in being delineated, but only lead to a minor increase in risk with odds ratios of 1.1-1.2. A small subset of genes is responsible for the risk loci for Crohn’s disease: The pattern recognition receptor gene NOD2, the autophagy gene ATG16L1 and the IL-23 receptor gene IL23R. In Asian cohorts, it has been noted that TNFSF15 has the most prominent associated risk, while NOD2 is more heavily identified in white populations. Only 13.1% of disease heritability is explained by genetic factors, leaving non-genetic environmental factors and epigenetic factors as major contributors to CD risk. [96] Smoking doubles the risk of developing CD and has been identified as the single largest environmental factor for the disease. Oral contraceptives also show strong positive association with disease onset while the following only show mild positive correlation: appendectomy, low dietary vitamin D, NSAID use (also has strong positive association with disease progression), antibiotic use, depression, and psychological stress. Low dietary fiber has a negative association with disease onset and high dietary fat and protein show no association with disease onset. [97] Diagnosis of Crohn’s disease is a long process, requiring both several months and the input of several specialists, as many of its symptoms are shared with a variety of diseases. A diagnosis requires assessment of clinical history, physical examination, serological and fecal biomarkers, cross-sectional and endoscopic imaging, and finally histological examination of biopsy specimens. CD can be classified three ways based on its location in the large or small intestine. About a third of patients present with small bowel CD, one third with large bowel CD, and one third with ileocolic CD. Currently ileocolonoscopy is the gold standard for diagnosis of CD, though small bowel CD can go undetected in this procedure. Patients who have a negative ileocolonoscopy but are strongly suspected of having CD receive small bowel video capsule endoscopy (VCE). The diagnostic yield, meaning the likelihood a test will provide adequate information for a diagnosis, of CD is higher for VCE than that for ileocolonoscopy (47% versus 21%, P=0.009) and VCE has a significant advantage of being better able to image parts of the small bowel proximal to the stomach. [98] Because of the similarities with ulcerative colitis, endoscopy has proven extremely important in differentiating between the two diseases. The key differentiator 16 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 between the two diseases is that CD is characterized by intermittent patches of inflammation while ulcerative colitis occupies a continuous swath of the colon. While ulcerative colitis can be treated and even cured with the removal of the colon, there currently is no cure for CD. Instead, treatment focuses on immunomodulators or TNF agonists. Early treatment for CD is associated with reduced risk of intestinal and perianal surgery as well as complications such as strictures. Unfortunately, current methods fall short of early detection of the disease. Forty percent (40%) of patients who are diagnosed with CD develop bowel damage within one year as a result of delayed diagnosis and are associated with worse outcomes such as high rates of surgery and hospitalization. Accordingly, a more sensitive and convenient method for diagnosis is needed for early detection of Crohn’s disease to improve treatment efficacy. [99] While previous investigators have performed infrared imaging with capsule devices, this has been at shorter near-infrared wavelengths (e.g.700nm-2500nm) and has generally included an IR source, such that what is being detected is reflected infrared light. In contrast, the present disclosure involves long IR wavelengths (e.g.7μm or 8μm up to 14μm or 15μm) and does not employ a light source; instead, the IR sensor in the present disclosure detects long IR light that is emitted from tissue, generally tissue with inflammation, which is associated with elevated tissue temperatures, which is referred to as thermal IR imaging. [100] The techniques of the present disclosure would not have been possible without relatively recent advances in thermal IR detectors which use less power and are more compact than previous generations of these devices. Older detectors were too large to fit into a capsule that could be swallowed, and/or the older thermal IR detectors generally needed to be cooled with liquid nitrogen. [101] Given that the capsule is untethered, there needs to be a way to track the position of the capsule over time so that the imaging information can be related to specific anatomical locations within the subject. Embodiments of the present disclosure address this issue by collecting information transmitted from the device wirelessly using a number of antennas placed around the subject (e.g. attached to a belt coupled to the subject); in addition to collecting the transmitted image information itself, the relative signal strength at each antenna can be used to help determine the capsule location by triangulation/trilateration, as disclosed herein. Thus, in various embodiments, position tracking algorithms may use preset measurements of attenuation of a signal vs depth so that the signal strength received at different antennas can be used for trilateration based on the preset measurements. The use of such a catalog of preset measurements as well as the catalog itself, along with any correction 17 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 algorithms, may be used in various other capsule or catheter based tracking systems and are not limited to the thermal IR embodiments disclosed herein. [102] To facilitate more frequent and/or less difficult checkups, embodiments of the present capsule device can perform thermal IR imaging through fecal matter and use information obtained about the depth or thickness of fecal matter (e.g. obtained using an ultrasound transducer in the capsule) to adjust the signal strength in the images that are obtained. [103] Thus, in various embodiments the disclosure provides an apparatus, system, and/or method for detecting gastrointestinal inflammation. The apparatus may include a capsule (FIG.1) including a thermal imaging sensor and may be configured to be swallowed by a subject (e.g. by making the capsule suitably small and/or having a shape that is conducive to swallowing such as that of a pill with rounded ends, as disclosed herein). The thermal imaging sensor may be configured to detect infrared radiation emitted by a tissue of the subject (FIG.1). In certain embodiments, the IR capsule may include a printed circuit board (PCB) housing a microcontroller, IR sensor, power regulator and radio frequency (RF) transmitter. The electronics may be hermetically sealed in a polycarbonate capsule with a custom thermal IR lens. [104] In some embodiments, the capsule may further include a controller (e.g. including a processor and memory) coupled to the thermal imaging sensor. The controller may be configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject. The capsule may also include a wireless transmitter and the controller may be configured to transmit collected image data (e.g. the at least one image) to the wireless transmitter, which in turn transmits the at least one image to an external antenna (FIG.7). An external device (e.g. a remote computing system) connected to the external antenna may receive the image signals and process them for analysis and/or display (FIG.1) to facilitate diagnosis and/or treatment of the subject. In various embodiments the wireless transmitter may be configured to transmit at a frequency in a range of between 400MHz and 500MHz, and in one particular embodiment may be configured to transmit at a frequency of 433MHz. [105] In certain embodiments, the controller may be further configured to generate a plurality of images of the detected infrared radiation emitted by the tissue of the subject and in some embodiments may be configured to generate at least one image per second of the detected infrared radiation emitted by the tissue of the subject. 18 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [106] In various embodiments, the capsule may further include a power supply including at least one battery. [107] In particular embodiments, the thermal imaging sensor may be configured to detect infrared radiation including a wavelength in a range of between 7μm to 14μm. In some embodiments, the thermal imaging sensor may include a thermopile sensor. [108] In some embodiments, the thermal imaging sensor further may include an ultrasound transducer coupled to the controller. In various embodiments, the controller may be further configured to obtain data from the ultrasound transducer indicative of a depth of fecal matter between the capsule and the tissue. In particular embodiments, the controller may be further configured to transmit the data from the ultrasound transducer indicative of the depth of fecal matter between the capsule and the tissue from the wireless transmitter to an external device. The information pertaining to the depth of fecal matter may in turn be used to adjust the thermal imaging data, which facilitates obtaining thermal information from the tissue without the subject having to perform a bowel prep. [109] In some embodiments, the thermal imaging sensor may be configured to detect infrared radiation following a delay period; that is, after the subject swallows the capsule there is a period of time before the controller in the capsule begins collecting thermal imaging information and transmitting the information via the transmitter to the external antennas. This delay period, which in various embodiments may be at least two hours, allows time for the capsule to reach the particular portion of the gastrointestinal tract (e.g. the ileum) from which data is to be obtained. The delay period extends the battery life of the device by not collecting data any sooner than needed. [110] In certain embodiments, the capsule may include a window which transmits infrared radiation, wherein the window is adjacent to the thermal imaging sensor. In particular embodiments the window may include a biocompatible material, which in some embodiments may include low density polyethylene (LDPE) (FIG.2). [111] In various embodiments, the capsule may further include a reflector disposed adjacent to the thermal imaging sensor, where the reflector may be configured to direct infrared radiation from the tissue towards the thermal imaging sensor. In certain embodiments, the reflector may be configured to direct infrared radiation from a circumferential region around the capsule. For example, since the typical GI tract sample is a luminal (i.e. tube-shaped) sample, the reflector may direct thermal IR light from a circumferential "ring" of the sample adjacent to the end of the capsule onto the IR sensor 19 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 face. The reflector, which may be a catadioptric reflector, in some embodiments may include a pyramid shape and particular embodiments may include an apex including a triangular wedge (FIGS.2-4). In other embodiments, the reflector may have a rounded shape (FIG.5). [112] In some embodiments, the at least one image may include a plurality of images and the controller may be configured to average together the plurality of images and transmit the averaged image to the wireless transmitter for transmission to the external antenna. As disclosed herein, averaging two or more images together can improve the signal to noise ratio and therefore permit smaller differences in thermal IR levels to be detected. [113] In various embodiments, the system for detecting gastrointestinal inflammation may include a capsule including a thermal imaging sensor in communication with a controller and a wireless transmitter. The system may also include a plurality of antennas in communication with the capsule, where the capsule may be configured to be swallowed by a subject. In some embodiments, the thermal imaging sensor may be configured to detect infrared radiation emitted by a tissue of the subject and the controller may be configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject. In certain embodiments, the controller may be configured to transmit the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas, and the plurality of antennas may be configured to receive the at least one image from the wireless transmitter (FIG.7). [114] In some embodiments, the system may further include a computing system in communication with the plurality of antennas, where the computing system may be configured to determine a location of the capsule based on the plurality of antennas receiving the at least one image from the wireless transmitter. In various embodiments, the computing system may be further configured to determine the location of the capsule based on an amplitude of a signal received by each of the plurality of antennas. In certain embodiments, the computing system may be further configured to determine the location of the capsule with an uncertainty of 1cm or less; in other embodiments, the uncertainty may be 10cm or less, 5cm or less, 2cm or less, 0.5cm or less, or 0.1cm or less. [115] In various embodiments, the plurality of antennas may include eight antennas. In other embodiments, the number of antennas may range from 1-20 and may include 1, 2, 3, 4, 5, 10, 15, or other number of antennas. In some embodiments, the antennas may be coupled to a belt, where the belt may be configured to be coupled to the subject (FIG.6A). In other embodiments the antennas may be attached to the subject using adhesives (FIG.6B). 20 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [116] In some embodiments, the at least one image may include a plurality of images each having a first image resolution and the computing system may be further configured to process the plurality of images to obtain at least one super-resolution image having a second resolution greater than the first resolution. In certain embodiments, at least two of the plurality of images may correspond to at least two different positions of the capsule and the computing system may be further configured to obtain the at least one super- resolution image based on interpolating the at least two of the plurality of images corresponding to the at least two different positions of the capsule. [117] Thus, in various embodiments the disclosed apparatus, methods, and systems may be used for detecting deep sites of inflammation not visible to visible light endoscopy. Thermal imaging presents a potentially more sensitive diagnostic tool allowing physicians to locate the site of inflammation with higher confidence than existing technologies. [118] Applications of this technology include, but are not limited to, detecting any form of inflammation inside the GI tract, such as colitis not associated with IBD, IBD (Crohn's Disease, Ulcerative Colitis associated with IBD, etc.), infections, ulcers, cancer, diverticulitis, etc. In addition, the disclosed technology also may be used for detecting and evaluating cancer and inflammatory conditions of other luminal organs. Embodiments of the disclosed device have much broader applicability than IBD for many very common conditions like diverticulitis, cancer, and ulcers. It also could also be relevant for many other infectious diseases like H. pylori and could be useful for eosinophilic esophagitis and gastritis [119] Considerations for embodiments of the various components of the system are disclosed below. [120] IR Sensors [121] Thermal imaging detects light in the long wave infrared regime (LWIR) from 8 to 14 microns. Early forms of thermal imaging required the sensor itself to be cooled via Peltier elements or liquid nitrogen to prevent the thermal radiation of the sensor from interfering with measurement. Fueled by military contracts in the 1990’s, modern uncooled focal plane arrays were developed that utilized micro-bolometer, pyroelectric technologies, and thermopiles. This development enabled uncooled thermal measurements on processes compatible with modern silicon fabrication. Micro-bolometers formed the majority of commercial uncooled sensors as of 2010 taking up 95% of the market. Micro-bolometers rely on a material whose resistance changes based on exposure to LWIR radiation such as vanadium oxide, while thermopiles include micron-sized thermocouples, which generate a 21 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 voltage in response to heat and IR radiation. Currently the top of the line miniature thermal imaging micro bolometers achieving a sensing efficiency of 34μW per pixel with 12μm pixel size while top-of-the-line thermopiles achieve 20.2 μW per pixel with 40μm pixel size. As a result, selection of IR sensor array type is dependent both on power and spatial budgets for a pillcam like device. [122] The ability for thermography to detect inflammation across the body has been documented by those in the field. In a rodent model, injections of 1% carrageenin to the pleural space, lower lip, and paws induced an up regulation of histamine, serotonin, prostaglandin, kinin and interleukins while showing at least a 0.7°C rise in all injected regions. In rabbits, atherosclerotic plaques demonstrated archipelagos of macrophage-induced temperature rises of 0.5°C with normal temperature regions <1mm away. Similarly, surface thermography of patient faces with sinusitis versus without sinusitis displayed a 0.5°C temperature rise in the difference between nasal region surface temperature and axillary body temperature. Finally, a human deep tissue incision model has shown the resultant inflammation from a deep tissue incision (1.5cm x 0.25cm) causes a 0.25°-0.9°C local temperature rise. [123] In various embodiments, the IR sensor may include 16x16 pixel arrays, 32x32 pixel arrays, 64x64 pixel arrays, or other configurations of pixels as needed for a particular application. Reflector [124] FIG.4 shows several embodiments of reflectors for reflecting light from a sample onto the IR sensor, showing the extent to which the IR sensor is covered by light with each type of reflector. In the left panel of FIG.4, the photo below shows the IR sensor with a conical reflector and the diagram above the photo shows the field of view of the IR sensor with the conical reflector, with an indication of where there is a "blind spot" on the IR sensor which does not receive reflected light. In the center panel of FIG.4, the photo below shows the IR sensor with a wedge-shaped reflector and the diagram above the photo shows the field of view of the IR sensor with the wedge-shaped reflector, with an indication of where there are "blind spots" on the IR sensor which do not receive reflected light. In the right panel of FIG.4, the photo below shows the catadioptric reflector of FIGS.2 and 3 and the diagram above the photo shows the field of view of the IR sensor with the catadioptric reflector, with an indication of where there are "blind spots" on the IR sensor which do not receive reflected light. As shown in the right panel, the catadioptric reflector minimizes blind spots and maximizes utilization of the IR sensor. While different types of reflectors can be used, in 22 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 each case the thermal IR energy that is reflected onto the IR sensor can be mapped back into the portion of tissue from which the energy arose in order to identify the location(s) having elevated temperatures. This mapping is based at least in part on the assumption that the original tissue is a luminal (tubular) structure that surrounds the capsule. In conjunction with the time and location of the capsule when each image is taken, the image information can be mapped onto the correct portion of the GI tract. [125] Battery technology [126] One of the greatest limitations of IR sensors has been their power draw, e.g. a top of the line single pixel miniature thermopile consumes 20.2μW. For comparison, an endoscopic visible light camera sensor consumes 300nW per pixel. As a result, batteries will need to support a power draw on the scale of 20mW for a full sensor, independent of any additional technologies that may be included on such a device. For a typical 8-hour capsule endoscopy procedure, this would require 55mAh from a 3V battery to provide continuous imaging. Aqueous cells are preferable for an ingestible device as they are resistant to thermal runaway and combustion. Silver zinc (Ag-Zn) chemistry offers the highest commercial power density in an aqueous cell. For reference, a size 312 silver zinc battery can provide 5-10mA at 1.8V with 35mWh capacity. For many implants, such as pacemakers, lithium iodide and lithium photocells are also used. These allow for higher current draw and capacity. Occupying the same space as 8 size 312 batteries, a size 2LZ6 button cell provides 60-80mA at 3V with 480mWh capacity, however temperature monitoring circuits are required to prevent thermal runaway and combustion. [127] Wireless transmission through human tissue [128] Radio frequency (RF) attenuation in human tissue can be accurately modeled by approximating organs and tissues as blocks of homogenous dielectrics. Using this model there are two types of losses that occur, reflection at dielectric boundaries and tissue absorption, where the RF energy is converted to thermal energy. The absorption (A) coefficient of radiation by human tissue is given by Eq. (1) while the reflection coefficient (R) is given by Eq. (2), assuming normal incidence and uniform permeability between tissues. (1) (2)
MGH 2021-602-03 Quarles 125141.04435 [131] Where ^^′ and ^^" are the positive real and imaginary components of the wave vector respectively, ^^ is the angular frequency of the wave, ^^ is the depth the wave has travelled through the tissue, ^^ and ^^ are the permeability and permittivity of the tissue respectively, ^^ is the conductivity of the tissue, and ^^ ^^1 ^^ ^^2 are the permittivities of the two tissues meeting at the reflection boundary. A comprehensive database modeling lossy tissues has been well established over the past decade and is used in finite difference time domain (FDTD) simulations throughout the RF industry. FIG.9 shows a dielectric model for a diseased obese patient where the receiver is 4.7 inches from the intestinal lumen. [132] Preliminary data [133] Wireless Transmission. When working with RF signals in lossy environments such as human tissue, it is desirable to select a transmission power that ensures the received power is large enough for detection. Using the model shown in FIG.9, we calculated the losses due to reflection and transmission as shown in FIG.10 for varying depths of both visceral and subcutaneous fat at 433 MHz. To verify the results, we performed a tissue model using a sacrificed 43kg swine where a 433MHz MAX41464 transmitter with an omnidirectional half dipole was implanted inside the ileocolonic region. We observed 22 dB of loss at a similar depth to the model in FIGS.9-10. Given our receiver sensitivity of - 45dBm, a transmission power of 15dBm will more than satisfy the link budget. We also observed a 20° cone of transmission before total internal reflection is observed. This result matches prior works findings. [134] [135]
R 2.1f 32p x 32p thermopile camera with window substrates of varying thickness immediately in front of the sensor. An IR blackbody source was used to generate temperatures in the expected biological range of interest (FIG.8). FIG.8 shows (top panels) an experimental setup for generating a high temperature spot for performing ex vivo tests of a wireless thermal IR imaging capsule and (lower panels) thermal IR imaging results obtained from viewing the high temperature 24 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 spot with the capsule device through an excised portion of ileum tissue. [136] Lens [137] Typical commercial lens and window designs for long wavelength infrared (LWIR) wavelengths use silicon, potassium bromide, zinc selenide, and germanium windows. Of these, only silicon is known to be biocompatible and even then, may demonstrate mild toxicity. In addition, silicon windows may also be brittle, risking fracturing due to pressure in the GI tract. As a result, we sought to identify a biocompatible flexible polymer that could pass LWIR light. We tested three materials commonly used in our labs prior Optical Coherence Tomography (OCT) capsules: Polycarbonate (PC), Polymethyl methacrylate (PMMA) and low-density polyethylene (LDPE). Based on prior literature, all 3 materials had different length passbands in the 8-14μm range. We used a calibrated blackbody source (SR-33, CI Systems) to display 37-38°C temperatures and then measured the voltage recorded by a 32p x 32p thermopile array based LWIR camera. Based on the results shown above in Table 2, the LDPE lens has the lowest amount of attenuation, showing that its sharp stopband at 14um has minimal impact on the temperature range of interest. [138] Prototype [139] One embodiment of a prototype device (FIG.11) was constructed using a Heimann IR 32p x 32p thermopile array, MAX41464 transmitter and MAX32630 microcontroller. Components were selected based on being the most power efficient and compact units on the market. [140] The IR sensor communicates over a 2 wire 1MHz I2C bus to the 1.2mm x 1.2mm MAX32630 microcontroller where initial IR image construction is performed to interpret the thermopile’s transient voltages, offset voltages, gradient, and ambient temperatures using the calibration data stored on the IR sensor’s electrically erasable programmable read-only memory (EEPROM). From there the microcontroller sends each frame over 400kHz I2C to the MAX414642mm x 5mm transmitter chip where the data is transmitted over a solenoid half wave dipole at 433MHz using Gaussian shaped FSK with a bandwidth of 200kHz and a baud-rate of 100kHz. A Nooelec NESDR SMArt XR software defined radio (SDR) receives the FSK via 433MHz monopole with a 2MHz sampling rate. The data is processed in MATLAB and SIMULINK to display the video feed and store the image data for further processing. FIG.12 shows a block diagram of the device. [141] Power [142] Each component of the IR capsule was isolated during typical operation and 25 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 their current was measured while the operating voltage was held at 3.3V. The resulting current and power consumptions are documented in Table 3. Note that while the MAX41464 transmitter had the highest power consumption during transmission, we are transmitting 16dBm (40mW) so only 8.7mW is truly being consumed by the device during operation. [143] [144] apsule. I2C
communication requires pull-up resistors, which are lumped in with the microcontroller power. [145] We tested three types of batteries for powering the device: Silver Zinc (AgZn) 312 cells from ZPower, Silver Oxide size 394 cells from Energizer, and Li/MnO22L76 size cells (Table 4). The same volume of cells was used to compare options with Li/MnO2 providing the best performance. For initial swine testing we plan to use Li/MnO2 chemistry in a 13mm capsule as we contract a third party to make a smaller diameter longer Li/MnO2 cell for an 11mm capsule for patient trials. [146]
sed at the same volume. [148] Ex-Vivo [149] For our ex-vivo study a 30cm long section of a 52kg swine’s large intestine was flushed of fecal matter and sutured at one end to be watertight. The middle 3cm (from 26 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 intestine lengths 13.5cm to 16.5cm) of the intestine was wrapped with a resistive heating element. A thermocouple was placed in between the heating element and the tissue. The resistive heater and thermocouple were connected to a switching power supply thermostat (DROK electronic thermostat controller). The thermostat was set to 38°C with a +/-0.2°C tolerance. The tissue and heating apparatus was placed into a thermal bath heated to 37°C (FIG.13). The device shown in FIG.12 was advanced towards the simulated inflammation site while the images were transmitted and received wirelessly at a rate of one frame per second. A diagram of the experimental setup is shown in FIG.14 and the resulting images are shown in FIG.15. [150] Development and validation of wireless infrared imaging capsules for detecting and localizing inflammation in the GI tract [151] The following sections describe the development and validation of embodiments of wireless thermal IR imaging capsules that will provide significant advances in diagnosis of various conditions including Crohn’s disease. The non-invasive capsule device will be more sensitive for inflammatory activity than other imaging approaches (e.g. MRI, CT, endoscopy/VCE) and will provide information on inflammation deep within the bowel wall, requiring minimal to no bowel preparation. [152] The IR capsule is being developed in our lab using an FDA-recommended, industry standard, design control process that ensures that the device satisfies the end user needs and safely works as intended. This process includes establishing design inputs, regular design reviews, risk analysis, design verification, and validation, all of which are documented in a design history file. Once the device is fabricated, it will be verified in phantoms and validated preclinically in a swine injury model of intestinal inflammation. [153] Design input specifications. Tables 4 and 5 depict two sets of initial target design input specifications of the wireless thermal imaging capsule that are based on specifications of existing VCE devices and our current understanding of temperature changes associated with tissue inflammation. Refinement of these specifications will take place as we obtain results from various GI inflammation experiments. 27 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435
[154] [155]
[156] [157]
[158] High-level schematics of embodiments of the wireless infrared capsule are shown in FIGS.16A and 16C (single IR sensor) and 16B (dual IR sensors). While the embodiments shown in the embodiments of FIGS.16A-16C include lenses or rounded windows to direct light to the IR sensors, in other embodiments disclosed herein a catadioptric reflector may be placed in front of the IR sensor to direct light towards the sensor (e.g. FIG.2). The IR sensor may be controlled by a microcontroller on the PCB which will communicate to an RF transmitter chip that will send data through a quarter dipole coil antenna housed around the PCB. As with commercial colonic VCE devices, to simultaneously obtain images in the forward and backward directions, the capsule in certain 28 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 embodiments may contain an imaging detector array (sensor) at each end of the capsule (FIG. 16B). Each IR sensor will be associated with a microcontroller, a transmitter, and a battery mounted to a printed circuit board (PCB). All components will be contained in a sealed, 11x32 mm Polymethyl methacrylate (PMMA) shell that is biocompatible and transmits IR light. In various embodiments, the capsule will be the same size as the Given PillCam™ Colon 2 VCE device. [159] FIG.17 depicts an overview of the data stream for the thermal capsule and recording system. Data is transmitted from the capsule to the receiver belt and stored on two SD cards. The SD cards on the front of the main module of the receiver belt store image data, while those on the back store position tracking data. The SD cards can be removed and replaced with new or the same SD cards at any time to monitor the contents and extract data such as the position of the capsule. Antennas should be placed on the subject as shown in FIG.35 using the xyphoid process as a benchmark. The receiver belt then is worn using a strap (e.g., like a purse), and the subject swallows the wireless thermal capsule. Briefly, N thermal images will be acquired by each IR sensor and digitized. The N frames will be transferred to the microcontroller via an I2C serial communications protocol bus. The microcontroller contains a processor that will average the N frames to decrease noise and increase thermal detection sensitivity. The averaged image will be transmitted from the capsule using a carrier RF frequency of 433 MHz. The RF signal will be transmitted wirelessly through the body and detected by a multitude of spatially offset receivers that reside within a belt worn by the patient. The image data will then be demodulated and stored by the patient-worn recorder. Images will be transferred from the recorder to a computer that will apply custom super-resolution algorithms that operate on M averaged frames to reconstruct the final image which has a greater number of pixels than those of the individual sensors. Data from different receivers in the belt will also be input into an algorithm that recovers the 3D position of the capsule for each image. The capsule is designed to be off when in its magnetic housing and will start transmitting within 5-6 seconds of being removed from the magnetic housing. These components and algorithms are described in more detail below. [160] Sensors. Until recently, the power requirements and sizes of IR detector arrays has prohibited the development of a wireless thermal imaging capsule. However, 32x32 pixel IR arrays have now been produced that can be incorporated into a thermal imaging capsule, which has facilitated the development of the disclosed capsule devices. For some 29 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 embodiments of the device, the sensor will be a Heimann IR 32x32 array, which has the lowest commercial power consumption of 5-7 mA at 3V and is capable of running on non- combustible button cell batteries. The two wide view, sensor-mounted lenses (for embodiments which use lenses to direct light to the sensor) at each end of the capsule will enable ~360° visualization of the luminal surface. [161] In some embodiments, the IR sensor may include two thermistor rings that each include a thermistor array comprising thermistor temperature sensors lined on the wall of the capsule (FIGS.23, 24, 26A, 26B); by collecting temperature readings from each thermistor in the array at particular intervals as the capsule moves through the tissue, the collected data can be used to generate one or more images (e.g., 2D images) of the heat that is being emitted by the tissue. The precision contact thermistors are configured to measure temperature by an onboard microcontroller and converted to a digital value. The capsule will then send the data wirelessly to an external battery-powered receiver belt using Gaussian Frequency Shift Keying at a frequency range of 430-440 MHz, which is a frequency range that has optimal electrokinetic properties for transmitting through human tissue. The receiver belt may determine the position of the capsule using relative signal strength across eight different antennas. The receiver belt may also store both the temperature and position data on two separate SD cards. [162] Sensitivity. One area of focus has been the development of methods to achieve high IR detection sensitivity so that small temperature changes can be measured. Because we cannot change the detector array itself, we have instead used procedures based on averaging successive frames, which reduces thermal noise and thus increases sensitivity, by the square root of the number of frames averaged. IR sensors have a pre-measured noise equivalent temperature difference (NETD) that is the standard deviation in pixel value in milli-Kelvin. The Heimann sensor we have selected due to its low power requirement, small packaging and highest resolution has an NETD of 340mK. As noted above, frame averaging lowers the NETD by the inverse square root of the number of frames average, so achieving 100mK requires around 12 frames to be averaged. Thus, in various embodiments frame averaging will be implemented on the capsule’s microcontroller to achieve a target temperature resolution/variance of 0.1 ºC, a value that is well within the range of temperature increases associated with inflammatory activity in other organ systems. [163] Super-resolution. Since 32x32 pixel arrays produce relatively low-resolution images, we are developing new algorithms to improve resolution by multiplexing spatially 30 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 offset images into a single higher resolution image. Since each successive averaged frame acquired by the arrays will be slightly spatially offset (e.g. due to movement of the capsule between frames), we are implementing super-resolution algorithms that take a set of M shifted, averaged images and interpolate them to obtain sub-pixel information. In various embodiments, acquisition of 16 sequential 32x32 images can allow VGA-level (512x512) resolution to be achieved, which is similar to that of commercial VCE devices. [164] Microcontroller. High efficiency mm-sized microcontroller and transmitters are available commercially that communicate via I2C protocols, requiring minimum firmware development. In one embodiment, a prototype device using the MAX32630 microcontroller 1.2 mm x 1.2 mm chip has been constructed which has been used to perform 5-frame averaging while consuming only 200μW peak power (FIG.15). [165] Frame Rate. The sensor and microcontroller are capable of 1MHz I2C communication with each frame requiring 4224 to 5248 bits of information yielding a frame rate on average of 24 frames per second. The transmitter can communicate over 400kHz I2C and transmit with a 100KHz baud rate. As result, each frame takes 0.516 seconds giving a framerate of 1.94 frames per second. [166] Battery power system. The highest energy density in a commercial aqueous button cells (aqueous cells are immune to thermal runaway and combustion) is achieved by Silver-Zinc (Ag-Zn) chemistry. For reference, a size 312 button Ag-Zn cell can provide 5-10 mA at 1.8 V with 35 mWh capacity. For two 312 Ag-Zn cells, 10 mA can be supplied at 3.3V with 20 mA bursts for transmission. When imaging continuously, the average power draw of our current design is 21.96 mW giving about 3.1 hours of continuous imaging time. Operations of the device including transmission will be delayed post ingestion by a given amount (e.g. about 2-3 hours in certain embodiments) to allow the device to reach the ileum. To extend battery life to enable complete ileocolonic recording over 8 hours, in some embodiments imaging can be cycled on-off for alternating 5-second periods. Based on the preliminary data shown in Tables 2 and 3 we require 30mAh per hour from a typical Li- MnO2 3V battery. For an 8 hour procedure this means 240mAh are required from the battery at 3V. [167] In some embodiments, the wireless capsule may be powered by a Lithium Manganese Dioxide battery that is positioned at near the rear cap of the capsule between a third and a fourth printed circuit board (FIGS.24 & 30) that may draw an average current of approximately 6 mA at 3V, with a peak draw of 9 mA. This is 400% below the pulse current 31 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 specification of the Lithium Manganese Dioxide battery generally used in the capsule. The Lithium Manganese Dioxide battery is used in various embodiments due to its non-volatile chemistry, long life, and ability to power medium current devices. [168] The capsule is further configured to include both a hardware and a software check on the battery voltage to determine when the battery is nearing depletion and initiate a graceful shutdown. The software checks the battery voltage using an ADC input on a GPIO pin. The battery is specified to operate between 2.0-3.0V, so when the battery reaches 2.2V the software triggers a shutdown process and the capsule then enters the off state after turning off all microprocessor sub-systems. [169] The main regulator hardware checks if the battery is near depletion and if so, performs a hardware disconnect of the power supply. The capsule poses no risk to the subject when power is turned off either gracefully at the end of the battery life, or abruptly by the hardware cutoff. [170] Wireless transmission. The MAX41464 transmitter will transmit encoded I2C image sensor data at 433 MHz, providing a data rate of 200 kbps. When transmitting through the human body, RF signals are both attenuated and reflected when transitioning from different tissue layers. One approach for determining whether the wireless signal can be reliably received is to model human tissue as layers of lossy dielectrics (FIG.9). Using this model, we have estimated RF attenuation losses, calculated our link budget, and have ensured that a strong enough signal will be transmitted through the body. For our 0 dBm (1mW), 433 MHz signal, our models predict a 27 dB transmission loss for a human, assuming the receiver is 4.7 inches from the intestinal lumen (FIG.9). Additionally, in a preliminary tissue model using a sacrificed 43kg pig, we observed a 22 dB of loss at a similar depth. Given our conservative receiver sensitivity of -45 dBm, this data indicates that our 0 dBm transmission power will more than satisfy our link budget. [171] Printed circuit board (PCB). In various embodiments, the four components (sensors, microcontroller, batteries, transmitter) can be combined onto a single PCB prototype, as shown in FIGS.13a, 14, and 17. In some embodiments the board will occupy 8.8 mm x 17 mm x 1.6 mm to allow space for two 3V batteries inside the capsule’s housing. FIGS.14 and 17 show internal block diagrams of the different components on the board that communicate with each other. The IR sensor will communicate over a two wire I2C bus to the microcontroller where initial IR image processing can be performed to interpret the thermopile information and apply the calibration data stored in the IR sensor’s electrically 32 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 erasable programmable read-only memory (EEPROM). From there, the microcontroller will perform frame averaging and the averaged image will be transferred across a separate I2C line to the transmitter. [172] Further shown in FIGS.23 and 24, in some embodiments, the capsule may include four printed circuit boards (PCBs) positioned throughout the capsule and around the thermistor rings, drug reservoir, and battery. For example, PCB1 may be connected to and placed within the array of thermistor rings for improved thermal conductivity (FIGS.26 & 32). In further embodiments, one or more thermistor rings may be used. The thermistor sensors are not required to be perfectly aligned and in some embodiments can be offset from one another by rotating the individual rings of the thermistors by any amount, e.g., any number of degrees. Two PCBs, PCB2 and PCB3, may be configured to be located within the processing unit and wireless transmission that further include being surrounded by antennas in the distal direction and a drug reservoir (FIGS.27-29). PCB 4 may include a holder for contact to the rear side of the power source (FIG.31). PCB1-PCB4 may be connected to one another through a circuit board housing and are configured to be folded and assembled within the capsule (FIGS.32 & 33). [173] Receiver belt. The receiver belt includes a main module, a battery, antenna leads, and a holster/belt. In particular embodiments, an 8-lead antenna configuration can be used for receiving capsule image data, although in other embodiments various numbers of antennas may be used, e.g. from 1-20 antennas. The eight antennas will reside within a wearable belt that transmits data to a recorder and may be positioned from 5 to 10 centimeters apart. The holster is attached to both a shoulder strap and belt to ensure it stays in place (FIG. 35). Capsule position tracking will be accomplished by triangulating its position from the amplitudes of the received signal to the array of antenna leads incorporated in the belt. The main module continuously sweeps all eight antennas, recording signal strength to track the position of the capsule. The antenna received the strongest signal strength is constantly updated and used to receive and record the capsule data. The thermal data from the wireless thermal capsule is demodulated and decoded from baseband and then stored into an SD card on the front side of the main module. On the reverse side of the main module, a second SD card stores the signal strength data for position tracking from the continuous sweeps. Two serial ports, one for each SD card, allow for JTAG to USB serial access to live stream the thermal imaging data and capsule position. The data is then sent in an encrypted state that can decrypt and display in real time the assessment of the capsule position and device 33 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 functionality. Current VCE capsules achieve 3.8 cm position tracking resolution. We believe location resolution is limited by the high attenuation and variance in signal strength as the capsule moves. To obtain better resolution, we will utilize a 16-bit analog to digital converter (ADC), allowing 1 mdBm resolution for an RF detector with 70 dB dynamic range. This configuration should enable mm-level capsule position tracking resolution. Based on recent research conducted in our lab, we have found that a 20° cone encompasses the total transmitted signal. We will thus configure our 8-lead receiver belt to collect RF data over this angular range for all capsule positions to avoid dead zones in the signal. The main module is powered by a 3.7V rechargeable battery, and the PCB and battery are held in separate compartments in a plastic case. The plastic case is placed in a cloth pouch that is attached to a shoulder strap, which provides vertical support, and a belt, which prevents swaying motion (FIGS.35A & 35B). The design is configured to comfortably holds the main module near the subject for the duration of use while minimizing the possibility that any antennas will be pulled off the subject. The receiver belt main module is reusable, and the antennas are single use. [174] In various embodiments the receiver belt may use a Lithium Polymer battery. Both a hardware and a software check on the battery voltage to determine when the battery is nearing depletion and initiate a graceful shutdown. [175] The software checks the battery voltage using an ADC input on a GPIO pin. The battery is specified to operate between 3.7-2.2V, so when the battery reaches a voltage of 2.2V or less the software triggers a shutdown process and the receiver then enters the off state after turning off all microprocessor sub-systems. The main voltage regulator also checks if the battery voltage is below 2.2V and if so, cuts power to the device. In addition to this, the lithium polymer battery has a built-in control circuit that cuts the battery off if a short-circuit condition is detected. [176] Table 7 exhibits the specifications for the wireless thermal receiver belt. The receiver belt is configured to have a small footprint (e.g., occupying a volume of 7cm x 7cm x 2cm or less) and fit inside a cloth pouch to be worn by the subject. The total weight of the wireless thermal receiver belt may be 1.1 lbs. (about 0.5 kg) or less and the antennas are designed using medical adhesive so as to minimize any skin irritation. For signal fidelity, low noise chipsets with -120dBm noise floors were chosen to minimize the power necessary from the capsule. A frequency of 430-440MHz was chosen to minimize attenuation and interactions with the human body. 34 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 For Wireless Thermal Receiver Belt Typical Value Unit Battery life under normal use >24 Hours
f signal strength from the receiver belt. This means that for a final 1cm uncertainty, each measurement requires 1/3cm uncertainty. Given our preliminary data of 22dB of loss in a swine model at depth 14.8cm we need to resolve 0.5dB difference at signal strengths around - 15dBm requiring a noise floor higher than -26dBm. We will construct our receiver using a Nesdr Nooelec Smart radio which has an intrinsic noise floor of -90dBm and sensitivity of - 45dBm, easily achieving this requirement. [179] Capsule performance verification. A blackbody source can be used to validate the capsule’s thermal resolution/sensitivity and to optimize the angular field of view. The capsule’s capacity to measure temperature in blackbody radiation phantoms and thermally modulated swine intestines ex vivo can be tested using a flexible thermoelectric generator (FlexTEG) apparatus. [180] We will use an extended area blackbody source (SR-33, CI-Systems) to project a uniform temperature surface inside an insulated temperature controlled box to generate a bi-linear mapping between the measured temperature and both pixel values and ambient temperature. This is necessary as thermopile voltage varies based on ambient temperature. Once calibrated we will then perform an ex-vivo study in which the capsule will image the lumen of swine intestines. The swine intestines will be sutured water tight at one and submerged in a thermally controlled 37°C water bath, with thermally modulated areas (areas with 0.5°C to 1.5°C rises) created from resistive and Peltier heaters simulating inflammation spots. Temperatures along the tissue and at inflammation sites will be measured with non-metallic fiber optic temperature sensors (TS5, Micronor). [181] Animal study validation. Capsule prototypes will be introduced (AdvanceCE® Delivery Device) in the duodena of three cohorts (with prep, minimal prep, no prep) of adult 35 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 (50-75 kg) swine 7 days following DSS or TNBS-EtOH administration. A modified version of the receiver/recorder belt will be attached to the swine and parameters will be slightly adjusted to account for the differences between swine and human anatomy. Swine will be allowed to recover/ambulate, and thermal images will be recorded of the small and large intestines in vivo. Ileocolonic lesions with elevated temperatures will be identified along with their corresponding locations in the intestine. Capsule localization will be confirmed via anteroposterior and lateral X-rays taken every 30 minutes. After the capsule has passed through the GI tract, the swine will be sacrificed, small and large intestines prosected, and normal and high temperature lesions (determined by capsule localization) will be submitted for histology. Histology will be scored by pathologists blinded to thermal imaging data. Temperature in regions determined to be inflamed by pathology will be compared to temperatures in non-inflamed regions using t-tests. [182] Validation of the IR imaging capsule in a swine model. By inducing an inflammatory response along swine intestines and knowing the site of inflammation we can verify that the IR capsule is capable of consistent detection of an inflammatory response. In some embodiments a swine injury model of intragastric administration of dextran sulfate sodium (DSS) or intra-ileal injection of 2,4,6-trinitrobenzene sulphonic acid (TNBS) will be used. Both methods have been established for inducing transmural inflammation in swine. Swine will undergo a 7-day recovery period after which the inflammation sites will be examined both with a high resolution IR borescope (TB-1710, Vividia) and our capsule devices mounted on a capsule deployer (AdvanceCE® Delivery Device). The swine will then be sacrificed and the small and large intestines prosected. The study will be repeated for animals with minimal and no bowel prep. [183] Histopathologic analysis. Standard H&E and immunohistochemical (IHC) processing will be performed including CD45 (Leukocytes), CD3 (T lymphocyte, T helper cell), CD4 (T regulatory cell, monocytes, macrophages and dendritic cells), CD68 (Monocytes, macrophages, dendritic cells) and metalloproteinase and cathepsin (lysosome) enzyme staining. These IHC stains are all known to be elevated in inflammatory processes. Slides will be digitized using a whole slide imaging system (Nanozoomer) and scored by two pathologists blinded to thermal imaging, at different 100 micron depth intervals. Inflammatory cell counts, inflammation/enzymatic activity areas and staining percent will be automatically computed from digitized slides and tabulated by depth. [184] Statistical rationale for number of animals. Assuming an increase in 36 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 temperature of 1°C for inflamed vs. non-inflamed regions and a standard deviation of 1°C, a moderate degree of within-animal correlation coefficient of 0.5, a total of 13 inflamed lesions and 13 non-inflamed regions will provide 90% power to detect a difference between the two groups using two-sided paired t-test with α=0.05. Given that we expect to image 2-3 well- delineated, inflamed lesions per animal, ~5 animals will be required for each (bowel prep, minimal bowel prep, and no bowel prep) study. Thus, a total of 15-20 animals will be used for pre-clinical device validation. [185] Sensitivity is insufficient. If the thermal noise in an image is too high, we can incorporate an adaptive frame-averaging algorithm to increase/decrease the number of frames. The onboard microprocessor can calculate the standard deviation between a prior frame and the current one and calculate how many more frames are needed for adequate image uniformity. If this should prove ineffective a transceiver can be incorporated to allow the position tracking algorithm to determine the number of frames needed based on device speed and the frame rate can be communicated to the capsule. [186] Fecal contents in swine are different from that of humans. Swine used in this study will be maintained on a human diet. However, thermal properties of the swine fecal matter will be collected and compared to human fecal matter from colonoscopies with inadequate bowel prep to ensure efficacy of our swine model. [187] Determination of optimal architecture for IR capsule endoscopy. Because of the size and power limitations of an IR sensor array, the images are lower resolution than visible light sensors. Video capsule endoscopy already faced limitations in resolution due to similar size and power constraints and neural networks have been shown to be a clinically successful way of improving diagnostic power through increasing resolution. While visible images produce colored texture images with specific shapes using RGB combinations, IR presents more uniform images using a single scalar value. As a result, much complexity can and likely should be eliminated to create the optimal super resolution algorithm for IR capsule endoscopy. [188] We will begin with the prior state-of-the-art capsule endoscopy super resolution EndoL2h. We will use data collected using a high resolution IR borescope, with such images acting as our training set for the adversarial network while the capsule images will be fed into the generator. [189] We will adapt the EndoL2h loss function given in Eq.3
MGH 2021-602-03 Quarles 125141.04435 Eq. (3) [191] Where ^^ ^^2 ^^, Ladv, Ltexture, Lcontent, ^^ ^^ℎ ^^ are the EndoL2h loss, adversarial loss, texture loss, content loss, and Charbonnier loss for pixel loss, respectively. α, β and γ are the hyper parameters controlling the weight of each loss type. We will adapt the EndoL2h loss function to the IR case by randomly sub-sampling 30 image pairs from our dataset and test 10 randomly generated hyper-parameter sets for each combination of α, β and γ ( [α, β, γ],[ α, β, 0], [α, 0, γ], [0, β, γ], [α, 0, 0], [0, β, 0], [0, 0, γ], [0,0,0] ). Images will be assessed by their peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) to determine the optimal loss combinations for IR images. [192] Because the capsule is moving and its velocity is determined by triangulation of the position over time, an interpolation between two or more images with rates dependent on the speed of the device can enable higher resolution images on top of those generated by GAN’s. [193] The center of the image will be determined by eccentricity of the content and a radial velocity vector will be assumed towards said center. The velocity determined from position tracking and angular field of view of the sensor gives the magnitude of the velocity of vector. Bicubic up-sampling is then performed along the velocity vector estimating the subpixel values from the subpixel velocity. Once again, images will be judged by PSNR and SSIM for both the case with and without a GAN network. [194] Number of frames are insufficient. Due to our low frame rate to achieve a low noise image we may not be achieving enough frames to have sub-pixel velocity values. We will begin testing the algorithms for this aim while other work is under way and if the framerate is insufficient, we will consider pursuing image compression on our microcontroller as a means of increasing framerate. [195] A key step in super-resolution algorithms is determining the subpixel shifts in successively acquired images. If the device experiences a period of stasis, then we will have little sub-pixel information to use to reconstruct higher resolution images. Alternatively, if the device moves over a large distance quickly, successive images will be completely different, also prohibiting the recovery of sub-pixel information. If we find that stasis or rapid capsule movements are a frequent problem, incorporating a receiver inside the capsule will allow us to control the sampling period and frame rate. We can then use the capsule position tracking velocity to control the capsule so that thermal images are acquired only when capsule motion rates are optimal for super-resolution reconstruction. 38 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [196] Should temperature variations due to bowel inflammation be smaller than anticipated, we can increase temperature sensitivity by performing more frame averaging. For example, in some embodiments averaging frames over one second should yield a temperature sensitivity of ~0.1 °C while in other embodiments averaging over two seconds will increase sensitivity to 0.07 °C. Using the receiver belt to reconstruct the capsule’s position and velocity, we can judge how many frames are reasonable to average. As described above, if we incorporate a receiver inside the capsule, we will be able to adaptively change frame rates during a procedure so that the images have lower thermal noise when the device is moving more slowly. Alternatively, in other embodiments we can actively cool the sensor using a miniature Peltier element mounted to the imaging sensors inside the capsule to reduce background thermal noise. [197] The primary benchmarks of success include a capsule design outputs that meet the design input specifications (Tables 4, 5), as determined by verification and preclinical validation. Another benchmark of success is an animal study that demonstrates the capacity to detect 1°C elevations in GI tract temperature that corresponds to histological evidence of inflammation. [198] EXAMPLES [199] The following are non-limiting examples according to embodiments of the disclosure. [200] Example 1: Characterization of relationship between inflammation and temperature in the intestine [201] The temperature of inflamed tissue typically increases by 0.1-5°C owing to increases in blood flow and increased metabolic rate of inflammatory cells. While this phenomenon has been studied for a variety of human diseases, little is known about the temperature changes seen in the inflamed bowel wall. A better understanding of the relationship between GI temperature elevation and the underlying inflammatory signature is therefore needed characterize temperature-based inflammatory changes in Crohn’s and optimally design a diagnostic thermal imaging capsule. Since Crohn’s is a transmural disease, it is furthermore important to understand how heat generated deep within the bowel wall diffuses to the surface where it emits IR radiation. [202] The temperature of inflamed swine ileocolic bowel wall segments will be measured in vivo and compared to corresponding quantitative immunohistochemistry (IHC) of inflammatory cells/enzymatic activity and histopathologic inflammation scoring systems. 39 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 One potential advantage of IR thermal imaging is that heat caused by inflammation deep within the bowel wall may diffuse to the surface and be detected. Thus, in addition to determining overall inflammation, histopathology/IHC will be evaluated as a function of distance from the luminal surface to determine the relationship between intraluminal temperature and inflammatory activity depth. [203] Animal models [204] As heat generated by inflammation is a fundamental biophysical phenomenon that is conserved across vertebrates and inflammatory diseases, we will study GI intraluminal temperatures in adult (50-75 kg) transmural swine injury models of ileocolonic inflammation. Swine also have similar anatomy and bowel wall thicknesses to those of humans, making results of this Example more clinically translatable. In addition, the bowel diameters of the swine will allow us to conduct intraluminal thermography in vivo with commercially available infrared borescopes that are similar to endoscopes but typically used for industrial purposes. [205] Our swine injury models will include intragastric administration of dextran sulfate sodium (DSS) or intra-ileal injection of 2,4,6-trinitrobenzene sulphonic acid (TNBS)- EtOH, both of which are established to produce transmural inflammation in swine. Following the chemical challenge, swine will recover for 7 days at which time a follow up procedure will be conducted to measure intraluminal temperatures along the GI tract. [206] Animal surgical procedure [207] Seven days after administration of DSS/TNBS, an open abdominal surgical procedure will be performed on bowel-prepped swine. Temperature will be measured intraluminally using an IR borescope inserted through an incision in the terminal ileum and colon. Conventional white light endoscopy will also be conducted through the same bowel segments. Cautery will be used to mark sites containing images with both elevated and low/normal temperatures and regions that are visibly normal and abnormal. Animals will then be sacrificed, the ileum and colon prosected, and histologic sections taken from the regions with cautery marks. [208] Temperature metrics [209] We will develop algorithms for normalizing elevated intestinal temperatures to normal regions to compare data across animals. The primary temperature metrics will be the normalized mucosal temperature mean and variance for different grades/metrics of inflammatory activity states and levels. Additional parameters derived from the thermal 40 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 images, including spatial moments and features, will also be investigated. [210] Histopathologic analysis [211] Standard H&E and IHC processing will take place. IHC will include CD45, CD3, CD4, CD20, CD68 and enzyme staining (e.g., metalloproteinases, cathepsins, neutrophil elastase) known to be elevated in inflammatory processes. All slides will be digitized using a whole slide imaging system (Nanozoomer). Two pathologists blinded to thermal imaging will apply histologic scoring, at different depth intervals (e.g.0-100, 100- 200, etc. micrometers from the luminal surface) and scoring consensus will be obtained. Inflammatory cell counts and inflammation/enzymatic activity expression areas will be automatically computed from the digitized histology/IHC slides and cell counts/area and staining %area will be tabulated in a depth dependent manner. [212] Data analysis [213] Temperature will be compared to %area staining of inflammatory cells and enzyme expression from corresponding digitized slides using linear regression with repeated measurements. Non-linear association between temperature and histologic scoring will be estimated using Spearman correlation. Analyses will be conducted for the entire bowel wall and at different depths from the luminal surface. In addition, normalized temperature thresholds for diagnosing inflammation (none-mild, moderate-severe) will be determined and sensitivity and specificity retrospectively evaluated. The smallest inflammatory foci that can be detected will also be tabulated as a function of temperature measurement variance. [214] Statistical rationale for number of animals. If three inflamed regions are identified in each animal, the histopathologic metric (e.g., CD45 %area staining) and temperature have a correlation coefficient of 0.6, and an intra-animal correlation of 0.5, data simulations show that 10 animals will provide >80% power to detect the correlation between temperature and histopathologic metric using a two-tailed test with α=0.05 in a random effects model. [215] While we have chosen to use straightforward injury-induced models of inflammation here owing to their simplicity and capacity to be generated in swine that have anatomy similar to humans, more sophisticated models that may more closely resemble Crohn’s disease could be considered if we find that the temperature changes seen in injury models are not conserved in Crohn’s. [216] If inflammation is too heterogeneous, we may have difficulty matching thermal images to histology. In this case, we can locally challenge the gut with DSS or TNBS 41 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 by distilling the toxic agent between two inflated balloons so that only a limited region of the bowel is exposed. [217] Should the DSS or TNBS models not produce sufficient variations in depth dependent inflammation, we will investigate an alteration of the protocol where we will endoscopically inject the DSS, TNBS, or lipopolysaccharide (LPS) at different depths within the bowel wall. [218] A key benchmark of success will be the demonstration that an increase in temperature reflects histological metrics of inflammation in the small and large bowel. A secondary benchmark will be to show that temperature increases can be distinguished when inflammation is located below the surface of the bowel wall. [219] Preliminary results of initial animal studies [220] Two swine studies have been conducted and analyzed. The first study (Swine Study 1) showed a 0.7-1.2 Celsius temperature rise in inflammation with TNBS treatment (to artificially induce inflammation) and histology confirms inflammatory response. Another outcome of this study was that we determined that forward viewing from the capsule is blocked by mucus. In the second study (Swine Study 2), we tested a catadioptric lens and showed that we could image tissue, and that a distance of 15cm is necessary between imaging site and laparoscopic incision for evaporative cooling. Furthermore, we determined that additional improvements are needed on the mirror/reflector design. [221] Example 2: Determine the capacity of mucosal temperature to be measured through fecal contents [222] Since feces are predominantly made up of water that readily transfers heat, fecal temperature should be reflective of that of the bowel wall. Thus, a potential advantage of thermal imaging is the capability of diagnosing bowel wall inflammation through fecal contents. This prospect could enable colonic inflammation detection without requiring patients to undergo full bowel prep. Here, we investigate this possibility by measuring the temperature of the bowel wall with varying thickness of interposed fecal contents. [223] Experimental strategy [224] Clean swine distal ileum and colonic segments will be heated to a known temperature ex vivo. Bowel surface temperature will be measured via IR thermal imaging. Fecal contents of varying thicknesses will then be applied on the luminal surface of the bowel and temperature measurement via IR imaging repeated. The equivalence between temperature measured with and without overlying feces will be determined for different 42 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 intestinal temperatures and fecal burdens. [225] Animal models [226] As swine intestine is readily available and anatomically similar to that of humans, it serves as an ideal model for this study. Terminal ileum and colorectal segments (cecum, ascending, transverse, descending, sigmoid, rectum) from sacrificed adult swine (50- 75kg) on human diets will be prosected. Since fecal content composition depends on bowel segment, feces will be collected and separated by anatomical origin. [227] Experimental apparatus [228] Since measurement of fecal thickness will be far easier for flat specimens, each intestinal segment will be longitudinally sectioned, cleaned, pinned flat, and placed with the luminal surface up on a Peltier thermoelectric heating plate (FIG.18). Temperature will be monitored at the luminal surface using a thermocouple, a common and reliable electronic device for measuring temperature. The voltage to the Peltier cooler will be controlled using feedback from the thermocouple to maintain a constant bowel wall temperature. The entire temperature modulation apparatus will be placed in a controlled humidity environment at 37 °C to simulate intraluminal conditions. [229] Temperature measurements [230] Thermal imaging will take place for each cleaned bowel segment to measure temperature without feces. Then, a layer of feces from the same anatomical segment will be placed between the mucosal surface and the thermal camera, covered by a glass plate. Fecal thicknesses will be measured using high-frequency (>30 MHz) ultrasound. Thermal images of the of the bowel with varying overlying thicknesses of fecal contents and over the range of temperatures representative of presumed none to severe ileocolic inflammation (e.g., 37-42 °C) will be acquired, at increments of ~0.1°C. Thermal equilibration, determined to occur when thermal images no longer vary, will take place prior to each temperature measurement. [231] The mean and standard deviation of the temperature obtained from the thermal images will be computed. The equivalence of temperature measured with and without varying thickness fecal layers will be determined using the student’s t-tests or trimmed t-test assuming unequal variances whenever appropriate. If present, the cutoff fecal layer thickness for which equivalence is no longer demonstrated will be ascertained, as a function of thermal detection sensitivity. [232] Assuming the standard deviation of the temperature measurement is 0.1 °C, if there is truly no difference between temperatures measured with and without feces, then 22 43 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 pairs of measurements per segment (with/without interposed feces) will provide 80% power to provide a two-sided 90% confidence interval that excludes a difference in means of more than 0.1 °C. We estimate that we will require approximately 10 swine to obtain 22 pairs of measurements for each bowel segment. [233] Even though the swine used in this study will be on a human diet, it is possible that the model may not completely recapitulate human fecal contents. To investigate this factor, we can obtain stool from volunteers or collect human feces from endoscopic procedures with inadequate bowel prep and repeat a subset of the studies described here to determine if equivalency thresholds are similar. [234] In addition, it is possible that thermal diffusion of feces depends on dietary and other physiological conditions. To investigate this potential confounding factor, we will measure the thermal diffusion of feces from swine on a variety of diets and from different anatomical locations to determine whether there is significant variation in fecal thermal diffusion with these variables. [235] The primary benchmark of success for this Example will be demonstration that temperatures at the surface of clinically relevant fecal depositions are equivalent to those at the surface of the bowel wall. This outcome will pave the way towards utilizing the thermal capsule to diagnose inflammation in patients without requiring bowel prep. [236] Example 3: Conduct pilot clinical studies using the wireless thermal imaging capsule in Crohn’s patients [237] In this Example, we will conduct pilot clinical studies to demonstrate feasibility and obtain an estimate of efficacy of the thermal imaging capsule for detecting inflammation in Crohn’s patients with and without bowel preparation. [238] Experimental strategy. Patients with Crohn’s disease who have undergone bowel preparation will swallow the thermal imaging capsule and thermal imaging data will be continuously recorded along with capsule position. Bowel wall regions with normal and elevated temperature readings will subsequently be endoscopically biopsied. Capsule-based thermal images will then be correlated to quantitative histology from corresponding biopsies. The same study will be conducted in patients who have not undergone bowel prep. The capacity of capsule imaging to obtain thermal images that correspond to intestinal inflammation will be compared between the two groups. [239] Patients. Patients with moderate to severe ileocolonic Crohn’s disease (n=20) will be enrolled in this study. Inclusion criteria are: 1) Confirmed diagnosis of ileocolonic 44 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 Crohn’s disease, 2) Able to consent, 3) Age 18-75 years, and 4) Moderate to severe Crohn’s disease as defined by 220 ≤ Crohn’s disease activity index (CDAI) < 450. Exclusion criteria are: 1) Evidence of active infection, 2) History of ileocolonic resection, 3) Evidence of a small bowel or colonic stricture or obstructive symptoms, 4) Evidence of a small bowel or colonic fistula, 5) Presence of intra-abdominal abscess, 6) Diagnosis of ulcerative or indeterminate colitis, or 7) Pregnancy. [240] Protocol. To ensure there are no capsule-retaining strictures, subjects will first swallow a patency capsule (e.g., Pillcam™ patency capsule) and then will undergo a 24-hr follow up abdominal X-ray to confirm that the capsule has passed. Subjects (10 who have undergone split-dose 4L PEG-ELS bowel prep, and 10 without bowel prep) will then swallow the thermal imaging capsule disclosed herein. Subjects will don the receiver/recorder belt for 8-10 hours as the capsule traverses the entire GI tract, recording corresponding ileocolonic thermal images and capsule position. After the capsule procedure, regions of low/normal and elevated temperatures will be identified and their locations within the bowel registered. Patients will then undergo sedated ileocolonoscopy 2-3 days after the thermal capsule imaging procedure. During endoscopy, biopsies will be taken from the sites identified by thermal capsule imaging. [241] Data analysis. Biopsy slides will be evaluated for inflammation as described in Example 1. Thermal capsule signatures for normal vs. inflamed regions, as determined by histology, will be compared using t-tests. The sensitivity and specificity of thermal capsule imaging for discriminating inflamed from uninflamed bowel wall will be determined using histology as the gold standard. Sensitivity/specificity for cohorts with and without bowel prep will be compared using a Fisher’s exact test. Receiver operating characteristic (ROC) curve will be used to assess the classification power of the proposed method. [242] Statistical rationale for number of patients [243] Assuming a mean temperature difference between inflamed and non-inflamed bowel wall of 1 °C, and a standard deviation of 1 °C, a total of 13 normal and 13 inflamed bowel segments will provide >80% power to detect the difference using a two-sided paired t- test with α=0.05. Assuming an average of 3 inflamed bowel wall regions per patient and estimating a sensitivity/specificity of capsule thermal imaging of 90% for diagnosing inflammation, enrollment of 10 patients per cohort will provide 95% confidence intervals of ± 11% for a per lesion analysis. [244] We acknowledge that inflammation in areas other than the terminal ileum may 45 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 be difficult to locate by the capsule and confirmed through colonoscopy. If it is difficult to separate inflammation in the terminal ileum to proximal colon, we will enroll patients with either only ileal or only colonic disease and will stratify our analyses by location. [245] Further, the capsule location will have a 1cm accuracy, however, endoscopy location can be more difficult to determine. Locations will be marked on the patency x-rays for the surgeon’s reference, and imaging modalities such as ultrasound or x-ray may be needed to confirm the exact location detected by position tracking is sampled by ileocolonoscopy. [246] If patients with CDAI between 220-450 do not exhibit sufficient GI inflammation, we can narrow the enrollment criteria to include only patients with elevated inflammatory markers such as ESR, CRP, and stool calprotectin or recent endoscopic confirmation of active ileocolonic disease. In the unlikely case that there are insufficient normal bowel segments, we can enroll an age- and sex-matched cohort without Crohn’s disease for comparison. [247] Expected outcomes/results. The primary outcome of these studies will be verification that the capsule detects inflammatory activity in Crohn’s patients with and without bowel preparation. We expect to see high correlations between temperature measurements and histologic activity which will significantly increase the resolution of this technology as compared to other currently available imaging modalities (i.e., CT, MRI, or VCE). [248] Example 4: Development of a Super Resolution Neural Network for IR Image Enhancement [249] Determination of optimal architecture for IR capsule endoscopy: Current gold standard Generative Adversarial Network (GAN) machine learning architectures common to endoscopy super-resolution for color images will be adapted for IR images, evaluated on datasets obtained using embodiments of the disclosed capsule, and compared to the ground truth high resolution IR borescope data. The resulting increased resolution is expected to increase diagnostic power of an IR capsule by allowing detection of small, early stage gut foci at the mm scale. [250] Implementation of Time of Flight pixel tracking: The position tracking of the capsule endoscopy system will be investigated to help further predict velocity vectors for resolving sub-pixel information as an add-on to the GAN based super-resolution network to further increase resolution. 46 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [251] Super resolution in Machine Learning refers to the concept of generating high- resolution images from low-resolution images. The two methods that are most successful in endoscopy are Time of Flight (ToF) and generative adversarial networks (GAN). ToF methods rely on rapidly taking subsequent low resolution (LR) images with known shifts in position and then interpolating between images to recover sub-pixel information. Prior endoscopy reconstruction techniques would estimate these displacements using the optical flow of RGB data from the endoscope camera. While ToF requires multiple images to produce a single super resolution image, GAN networks have a 1:1 input to output for images. GAN networks function by training two competing networks: A generating network creates a high resolution version of a low resolution image, and simultaneously a discriminator network attempts to determine whether the generated image is a generated high resolution image or ground truth initially taken high resolution image. These competing networks allow for the generation of images with accurate resolution increases of 10-12x increase in one dimension. [252] Prior works have adapted various loss functions for medical images. Pixel loss evaluates pixel by pixel differences of a generated super resolution (SR) image to its corresponding ground truth HR image via an L1 loss function. Content loss evaluates the feature maps of the SR image to its HR image using pre-extracted feature maps typically from pre-trained, off the shelf feature extractors like VGG or ResNet using the Euclidean distance between feature representations. Texture loss is defined as the Gram matrix formed by the inner products of the vectorized feature maps for a given convolutional layer. This helps to introduce directional correlations to the learning process. Finally, adversarial loss has been shown to increase stability during training and result in better convergence. Prior works outside of endoscopy have used these losses individually to characterize their super resolution algorithms; however, capsule endoscopy has shown the greatest success when all losses are used each with their own optimized weighted hyper-parameter. While video endoscopy images are rich with information and low noise, small bowel thermal imaging presents a unique challenge where, as our preliminary data shows, the image is mostly uniform. As a result, it is unclear if the same complexity is necessary to generate complete images. [253] Computer systems [254] Turning to FIG.19, an example 1900 of a system (e.g. a data collection and processing system) for detecting gastrointestinal inflammation is shown in accordance with some embodiments of the disclosed subject matter. As shown in FIG.19, a computing device 47 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 1910 can receive thermal IR data from a thermal IR capsule 1900. In some embodiments, computing device 1910 can execute at least a portion of a system for detecting gastrointestinal inflammation 1904 to detect gastrointestinal inflammation based on the thermal IR data received from thermal IR capsule 1900. Additionally or alternatively, in some embodiments, computing device 1910 can communicate information about the thermal IR data received from thermal IR capsule 1900 to a server 1920 over a communication network 1906, which can execute at least a portion of system for detecting gastrointestinal inflammation 1904 to detect gastrointestinal inflammation based on the thermal IR data. In some such embodiments, server 1920 can return information to computing device 1910 (and/or any other suitable computing device) indicative of an output of system for detecting gastrointestinal inflammation 1904, such as the thermal IR information. This information may be transmitted and/or presented to a user (e.g. a researcher, an operator, a clinician, etc.) and/or may be stored (e.g. as part of a research database or a medical record associated with a subject). [255] In some embodiments, computing device 1910 and/or server 1920 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, etc. As described herein, system for detecting gastrointestinal inflammation 1904 can present information about the thermal IR data, and/or the thermal IR information to a user (e.g., researcher and/or physician). [256] In some embodiments, thermal IR capsule 1900 may include an IR sensor 1902, which can be any sensor suitable for detecting IR such as a thermopile sensor. In other embodiments, IR sensor 1902 can be associated with computing device 1910. For example, IR sensor 1902 may be combined with computing device 1910 (e.g., computing device 1910 can be configured as part of a capsule for obtaining thermal IR information). As another example, IR sensor 1902 may be connected to computing device 1910 by a cable, a direct wireless link, etc. Additionally or alternatively, in some embodiments, IR sensor 1902 can be located locally and/or remotely from computing device 1910, and can communicate information to computing device 1910 (and/or server 1920) via a communication network (e.g., communication network 1906). [257] In some embodiments, communication network 1906 can be any suitable communication network or combination of communication networks. For example, communication network 1906 can include a Wi-Fi network (which can include one or more 48 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 4G network, a 5G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, etc. In some embodiments, communication network 1906 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG.19 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, etc. [258] FIG.20 shows an example 2000 of hardware that can be used to implement computing device 1910 and server 1920 in accordance with some embodiments of the disclosed subject matter. As shown in FIG.20, in some embodiments, computing device 1910 can include a processor 2002, a display 2004, one or more inputs 2006, one or more communication systems 2008, and/or memory 2010. In some embodiments, processor 2002 can be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc. In some embodiments, display 2004 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc. In some embodiments, inputs 2006 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, etc. [259] In some embodiments, communications systems 2008 can include any suitable hardware, firmware, and/or software for communicating information over communication network 1906 and/or any other suitable communication networks. For example, communications systems 2008 can include one or more transceivers, one or more communication chips and/or chip sets, etc. In a more particular example, communications systems 2008 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, etc. [260] In some embodiments, memory 2010 can include any suitable storage device or devices that can be used to store instructions, values, etc., that can be used, for example, by processor 2002 to present content using display 2004, to communicate with server 1920 via communications system(s) 2008, etc. Memory 2010 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 2010 can include RAM, ROM, EEPROM, one or more flash drives, one or more 49 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 hard disks, one or more solid state drives, one or more optical drives, etc. In some embodiments, memory 2010 can have encoded thereon a computer program for controlling operation of computing device 1910. In such embodiments, processor 2002 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables, etc.), receive content from server 1920, transmit information to server 1920, etc. [261] In some embodiments, server 1920 can include a processor 2012, a display 2014, one or more inputs 2016, one or more communications systems 2018, and/or memory 2020. In some embodiments, processor 2012 can be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc. In some embodiments, display 2014 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc. In some embodiments, inputs 2016 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, etc. [262] In some embodiments, communications systems 2018 can include any suitable hardware, firmware, and/or software for communicating information over communication network 1906 and/or any other suitable communication networks. For example, communications systems 2018 can include one or more transceivers, one or more communication chips and/or chip sets, etc. In a more particular example, communications systems 2018 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, etc. [263] In some embodiments, memory 2020 can include any suitable storage device or devices that can be used to store instructions, values, etc., that can be used, for example, by processor 2012 to present content using display 2014, to communicate with one or more computing devices 1910, etc. Memory 2020 can include any suitable volatile memory, non- volatile memory, storage, or any suitable combination thereof. For example, memory 2020 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, etc. In some embodiments, memory 2020 can have encoded thereon a server program for controlling operation of server 1920. In such embodiments, processor 2012 can execute at least a portion of the server program to transmit information and/or content (e.g., results of a tissue identification and/or classification, a user interface, etc.) to one or more computing devices 1910, receive information and/or content 50 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 from one or more computing devices 1910, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), etc. [264] In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as RAM, Flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media. [265] In some embodiments, the optical signals are detected by photodiodes. It should be recognized that any opto-electronic conversion device including but not limited to photo detectors, photodiodes, line-scan and two-dimensional cameras, and photodiode arrays can be used to perform this detection function. [266] It should be noted that, as used herein, the term mechanism can encompass hardware, software, firmware, or any suitable combination thereof. [267] FIG.21 shows an example 1200 of a process for detecting gastrointestinal inflammation in accordance with some embodiments of the disclosed subject matter. As shown in FIG.21, at 2102, process 2100 can provide a capsule comprising a thermal imaging sensor, where the capsule may be configured to be swallowed by a subject. Finally, at 2104, process 2100 can detect, using the thermal imaging sensor, infrared radiation emitted by a tissue of the subject. [268] FIG.22 shows an example 2200 of a process for detecting gastrointestinal inflammation in accordance with some embodiments of the disclosed subject matter. As shown in FIG.22, at 2202, process 2200 can provide a capsule comprising a thermal imaging sensor, a wireless transmitter, and a plurality of antennas in communication with a controller, where the capsule may be configured to be swallowed by a subject. At 2204, process 2200 can detect, by the thermal imaging sensor, infrared radiation emitted by a tissue of the subject. At 2206, process 2200 can generate, by the controller, at least one image of the 51 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 detected infrared radiation emitted by the tissue of the subject. At 2208, process 2200 can transmit, by the controller, the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas. Finally, at 2210, process 2200 can receive, by the plurality of antennas, the at least one image from the wireless transmitter. [269] It should be understood that the above described steps of the processes of FIGS.21 and 22 can be executed or performed in any order or sequence not limited to the order and sequence shown and described in the figures. Also, some of the above steps of the processes of FIGS.21 and 22 can be executed or performed substantially simultaneously where appropriate or in parallel to reduce latency and processing times. [270] FIG.34 shows a system block diagram of operation of embodiments of the device, going from capsule measurement transmission then reception to processing and storage between the capsule and the receiver belt. As shown in FIG.34, at step 37, the thermistor sensor array of the capsule measures luminal temperature of the user. At step 38, the on board controller of the capsule then processes and encrypts the data. Following step 38, at step 40 the capsule wirelessly transmits the data through the tissue to the external receiver of the antenna array to the receiver belt. At step 48, the external receiver sweeps and stores all the antenna amplitudes from the signal and then updates the receiver antenna to record the signal to be the highest fidelity input, at step 50. Simultaneous with step 40, the external receiver also receives and stores the data at step 42. In some embodiments the receiver processes the data in the neural networks to determine a position of the capsule at step 44. Finally, at step 46 the receiver transmits the signal back to the capsule to perform one or more function including: releasing of the drug payload, providing information on the position, adjusting the sensitivity and filter settings for thermal measurements. In some embodiments, the data is processed by the receiver belt to send a signal back to the wireless capsule to adjust its sensor settings and framerate, as well as to guide further actions, such as releasing drug payloads or informing the capsule of its current position. [271] With machine learning, two networks are used within the thermal wireless capsule system. As shown in FIG.36, a first network (Network 1) is configured to resolve tissue thickness and determine the position of the capsule. This is achieved by taking the antenna signals’ amplitude input 62 from the receiver belt and, through a neural network, uses them to output the tissue thickness 64 for the region along the signal path between the capsule and the surface of the patient. The network also outputs the capsule position and rotation in (x,y,z) coordinates 64. 52 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 [272] The neural network for the capsule is trained using a generated dataset that simulates the electromagnetic fields generated 58 by the capsule for various tissue depths, capsule angles, and capsule positions relative to an antenna. Simultaneously, the input antenna configuration is used to generate a multi-antenna measurement. This is achieved by iterating expected capsule positions that generates the antenna field for all the antennas 60. Based on the input number and position of antennas this is then used to extrapolate the fields seen by the entire array of antennas. These are then used to train the network to learn the capsule position, rotation, and the various tissue depths for given field strength patterns. [273] FIG.37 shows a second network (Network 2) that is configured to determine tissue inflammation and pathology severity 68. The second network takes the capsule position, computed local tissue thicknesses, and thermal readings to determine the pathology depth and/or severity in tissue 66. When performed post-procedure, the network can compute using the past, present, and future readings for all inputs at once. When performed during the procedure, only the past and present inputs are available for a given new datapoint. [274] The second neural network is trained by generating thermal fields that are produced by varying size, intensity, depth, heat type, and location of pathology are simulated within the tissue and propagating to the capsule body 70. These are varied for tissue thickness, capsule position, and rotation, and allowed to be guided by any combination of current, past, and present measurements. Reasoning for each input is given below. [275] Input from the first network takes capsule position and tissue thickness as inputs, as well as the past, present, and if evaluated, post-procedure future inputs. Tissue thickness helps determine thermal diffusion to resolve depth and calculate thermal physics to judge the depth and severity of a site. Position helps determine if the capsule is near any major thermal bodies, such as an artery or thermally significant organ due to metabolism or perfusion, such as the heart, thyroid gland, kidneys, liver, or bladder. The past and present thermal readings as an input help judge the ambient temperatures and determine if the capsule is in the presence of a large swatch of pathology. If evaluated post-procedure, the future inputs are also processed. [276] Thus, while the invention has been described above in connection with particular embodiments and examples, the invention is not necessarily so limited, and that numerous other embodiments, examples, uses, modifications and departures from the embodiments, examples and uses are intended to be encompassed by the claims attached hereto. 53 Q B\85925099.1
Claims
MGH 2021-602-03 Quarles 125141.04435 CLAIMS What is claimed is: 1. An apparatus for detecting gastrointestinal pathology, comprising: a capsule comprising at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject. 2. The apparatus of claim 1, wherein the at least one temperature sensor comprises a thermistor comprising a thermistor ring including a plurality of thermistors arranged around a cylindrical wall of the capsule, wherein the plurality of thermistors in the thermistor ring are configured to detect a spatial pattern of heat emitted by the tissue of the subject. 3. The apparatus of claim 2, wherein the plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 4. The apparatus of claim 2, wherein the thermistor ring comprises a first thermistor ring and wherein the plurality of thermistors comprises a first plurality of thermistors, and wherein the at least one temperature sensor further comprises a thermistor comprising a second thermistor ring including a second plurality of thermistors arranged around the cylindrical wall of the capsule. 5. The apparatus of claim 4, wherein the second plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 6. The apparatus of claim 4, wherein the second plurality of thermistors of the second thermistor ring are arranged in a spatially offset manner from the first plurality of thermistors of the first thermistor ring. 54 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 7. The apparatus of claim 6, wherein the first plurality of thermistors and the second plurality of thermistors are configured to detect the spatial pattern of heat emitted by the tissue of the subject, and wherein data pertaining to the detected spatial pattern of heat is processed to at least one of: improve a spatial resolution of an image formed using the data, determine a velocity of movement of the capsule within the tissue of the subject, or reduce an amount of noise related to transmission of the data. 8. The apparatus of claim 6, wherein the cylindrical wall of the capsule comprises a thermally conductive material, and wherein at least one of the first plurality of thermistors or the second plurality of thermistors are in thermal contact with the thermally conductive material of the cylindrical wall of the capsule. 9. The apparatus of claim 8, further comprising a flexible printed circuit board, wherein at least one of the first thermistor ring or the second thermistor ring are coupled to the flexible printed circuit board. 10. The apparatus of claim 9, further comprising a battery in electrical contact with the flexible printed circuit board. 11. The apparatus of any one of claims 1-10, further comprising a controller in communication with the at least one temperature sensor, wherein the controller is configured to obtain data from the at least one temperature sensor relating to the detected heat emitted from the tissue of the subject. 12. The apparatus of claim 11, wherein the controller is further configured to encrypt the data obtained from the at least one temperature sensor. 13. The apparatus of claim 11, further comprising a wireless transmitter and an antenna in communication with the controller, 55 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wherein the controller is further configured to transmit the data via the wireless transmitter and the antenna to an external device for at least one of storage or analysis of the transmitted data. 14. The apparatus of claim 13, wherein the controller is further configured to generate at least one image based on the data obtained from the at least one temperature sensor. 15. The apparatus of claim 13, wherein the controller is further configured to generate a plurality of images based on the data obtained from the at least one temperature sensor, wherein the plurality of images are based on data obtained at a rate of data capture from the at least one temperature sensor. 16. The apparatus of claim 15, wherein the controller is further configured to: receive a signal from the external device, adjust, based on the signal from the external device, at least one of the rate of data capture from the at least one temperature sensor or an operational setting of the at least one temperature sensor. 17. The apparatus of claim 13, further comprising a drug reservoir, and wherein the controller is further configured to: receive a signal from the external device, and release a drug payload from the drug reservoir based on receiving the signal from the external device. 18. The apparatus of claim 13, further comprising a drug reservoir, and wherein the controller is further configured to: detect a target site in the tissue based on the detected heat emitted from the tissue of the subject, and release a drug payload from the drug reservoir based on detecting the target site. 19. The apparatus of claim 13, wherein the controller is further configured to receive a signal from the external device, 56 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wherein the signal received from the external device comprises information regarding a current position of the capsule within the subject. 20. The apparatus of claim 13, wherein the wireless transmitter is configured to transmit at a frequency in a range of between 430MHz and 440MHz. 21. The apparatus of claim 13, further comprising a thermal imaging sensor, wherein the thermal imaging sensor is configured to detect infrared radiation emitted by the tissue of the subject. 22. The apparatus of claim 21, wherein the capsule comprises a cylindrical housing, and wherein the thermal imaging sensor is disposed at an end of the cylindrical housing. 23. The apparatus of claim 21, wherein the thermal imaging sensor is configured to detect infrared radiation comprising a wavelength in a range of between 7μm to 14μm. 24. The apparatus of claim 21, wherein the thermal imaging sensor comprises a thermopile sensor. 25. The apparatus of claim 21, wherein the thermal imaging sensor further comprises an ultrasound transducer coupled to the controller. 26. The apparatus of claim 25, wherein the ultrasound transducer is configured to obtain data indicative of a depth of fecal matter between the capsule and the tissue of the subject. 27. The apparatus of claim 25, wherein the controller is further configured to obtain data from the ultrasound transducer indicative of a depth of fecal matter between the capsule and the tissue. 28. The apparatus of claim 27, wherein the controller is further configured to transmit the data from the ultrasound transducer indicative of the depth of fecal matter between the capsule and the tissue to the external device using the wireless transmitter and the antenna. 57 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 29. The apparatus of claim 21, wherein the capsule comprises a window which transmits infrared radiation, wherein the window is adjacent to the thermal imaging sensor. 30. The apparatus of claim 21, further comprising a reflector disposed adjacent to the thermal imaging sensor, wherein the reflector is configured to direct infrared radiation from the tissue towards the thermal imaging sensor. 31. The apparatus of claim 30, wherein the reflector is configured to direct infrared radiation from a circumferential region around the capsule. 32. The apparatus of claim 30, wherein the reflector comprises a pyramid shape. 33. The apparatus of claim 32, wherein the pyramid shape comprises an apex comprising a triangular wedge. 34. The apparatus of any one of claims 1-33, wherein the at least one temperature sensor comprises at least one of a thermistor, a thermocouple, a semiconductor-based sensor, or a resistance-based temperature sensor. 35. A system for detecting gastrointestinal pathology, comprising: a capsule comprising at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject; and a receiver belt configured to be coupled to a body of the subject. 36. The system of claim 35, wherein the at least one temperature sensor comprises a thermistor comprising a thermistor ring including a plurality of thermistors arranged around a cylindrical wall of the capsule, wherein the plurality of thermistors in the thermistor ring are configured to detect a spatial pattern of heat emitted by the tissue of the subject. 58 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 37. The system of claim 36, wherein the plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 38. The system of claim 36, wherein the thermistor ring comprises a first thermistor ring and wherein the plurality of thermistors comprises a first plurality of thermistors, and wherein the at least one temperature sensor further comprises a thermistor comprising a second thermistor ring including a second plurality of thermistors arranged around the cylindrical wall of the capsule. 39. The system of claim 38, wherein the second plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 40. The system of claim 38, wherein the second plurality of thermistors of the second thermistor ring are arranged in a spatially offset manner from the first plurality of thermistors of the first thermistor ring. 41. The system of claim 40, wherein the first plurality of thermistors and the second plurality of thermistors are configured to detect the spatial pattern of heat emitted by the tissue of the subject, and wherein data pertaining to the detected spatial pattern of heat is processed to at least one of: improve a spatial resolution of an image formed using the data, determine a velocity of movement of the capsule within the tissue of the subject, or reduce an amount of noise related to transmission of the data. 42. The system of claim 40, wherein the cylindrical wall of the capsule comprises a thermally conductive material, and wherein at least one of the first plurality of thermistors or the second plurality of thermistors are in thermal contact with the thermally conductive material of the cylindrical wall of the capsule. 59 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 43. The system of claim 42, wherein the capsule further comprises a flexible printed circuit board, wherein at least one of the first thermistor ring or the second thermistor ring are coupled to the flexible printed circuit board. 44. The system of claim 43, wherein the capsule further comprises a battery in electrical contact with the flexible printed circuit board. 45. The system of any one of claims 35-44, wherein the capsule further comprises a controller in communication with the at least one temperature sensor, wherein the controller is configured to obtain data from the at least one temperature sensor relating to the detected heat emitted from the tissue of the subject. 46. The system of claim 45, wherein the controller is further configured to encrypt the data obtained from the at least one temperature sensor. 47. The system of claim 45, wherein the capsule further comprises an antenna in communication with the controller, wherein the controller is further configured to transmit the data via the antenna to an external device for at least one of storage or analysis of the transmitted data. 48. The system of claim 47, wherein the external device is coupled to the receiver belt. 49. The system of claim 48, wherein the receiver belt comprises a plurality of receiver belt antennas in communication with the external device. 50. The system of claim 49, wherein the receiver belt comprises a waist belt and a shoulder strap, wherein the waist belt and the shoulder strap are configured to hold the plurality of receiver belt antennas in a fixed position relative to the body of the subject. 51. The system of claim 47, wherein the controller is further configured to: 60 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 receive a signal from the external device, adjust, based on the signal from the external device, at least one of a rate of data capture from the at least one temperature sensor or an operational setting of the at least one temperature sensor. 52. The system of claim 47, wherein the capsule further comprises a drug reservoir, and wherein the controller is further configured to: receive a signal from the external device, and release a drug payload from the drug reservoir based on receiving the signal from the external device. 53. The system of claim 47, wherein the capsule further comprises a drug reservoir, and wherein the controller is further configured to: detect a target site in the tissue based on the detected heat emitted from the tissue of the subject, and release a drug payload from the drug reservoir based on detecting the target site. 54. The system of claim 49, wherein the controller is further configured to receive a signal from the external device, wherein the signal received from the external device comprises information regarding a current position of the capsule within the subject. 55. The system of claim 54, wherein the information regarding the current position of the capsule within the subject is determined based on information from the plurality of receiver belt antennas. 56. The system of claim 55, wherein the information regarding the current position of the capsule within the subject is determined based on a comparison of the information from the plurality of receiver belt antennas and data regarding a relationship between antenna signal attenuation vs. tissue depth. 61 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 57. The system of claim 55, wherein the information from the plurality of receiver belt antennas is processed by a first neural network to identify at least one of a type of tissue or a tissue thickness between the capsule and a surface of the subject adjacent the plurality of receiver belt antennas, and wherein the current position of the capsule is determined based on the at least one of the type of tissue or the tissue thickness. 58. The system of claim 57, wherein the current position of the capsule, the at least one of the type of tissue or the tissue thickness, and the data obtained from the at least one temperature sensor are processed by a second neural network to determine at least one of a pathology depth, a pathology type, or a pathology severity in the tissue of the subject. 59. The system of claim 49, wherein information from the plurality of receiver belt antennas is processed to identify at least one of a type of tissue or a tissue thickness between the capsule and a surface of the subject adjacent the plurality of receiver belt antennas. 60. The system of any one of claims 35-59, wherein the at least one temperature sensor comprises at least one of a thermistor, a thermocouple, a semiconductor-based sensor, or a resistance-based temperature sensor. 61. A method for detecting gastrointestinal pathology, comprising: providing a capsule comprising at least one temperature sensor, the capsule being configured to be swallowed by a subject, and the at least one temperature sensor being configured to circumferentially detect heat emitted by a tissue of the subject; and circumferentially detecting, using the at least one temperature sensor, heat emitted by a tissue of the subject. 62. The method of claim 61, wherein the at least one temperature sensor comprises a thermistor comprising a thermistor ring including a plurality of thermistors arranged around a cylindrical wall of the capsule, and wherein providing a capsule further comprises: 62 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 providing the capsule comprising the plurality of thermistors in the thermistor ring arranged around the cylindrical wall of the capsule, and wherein circumferentially detecting heat emitted by a tissue of the subject further comprises: circumferentially detecting a spatial pattern of heat emitted by the tissue of the subject using the plurality of thermistors in the thermistor ring. 63. The method of claim 62, wherein the plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 64. The method of claim 62, wherein the thermistor ring comprises a first thermistor ring and wherein the plurality of thermistors comprises a first plurality of thermistors, and wherein the at least one temperature sensor further comprises a thermistor comprising a second thermistor ring including a second plurality of thermistors arranged around the cylindrical wall of the capsule. wherein providing a capsule further comprises: providing the capsule comprising the second plurality of thermistors in the second thermistor ring arranged around the cylindrical wall of the capsule, and wherein circumferentially detecting heat emitted by a tissue of the subject further comprises: circumferentially detecting a spatial pattern of heat emitted by the tissue of the subject using the second plurality of thermistors in the second thermistor ring. 65. The method of claim 64, wherein the second plurality of thermistors are arranged partway around or completely around the cylindrical wall of the capsule. 66. The method of claim 64, wherein the second plurality of thermistors of the second thermistor ring are arranged in a spatially offset manner from the first plurality of thermistors of the first thermistor ring. 67. The method of claim 66, wherein the first plurality of thermistors and the second plurality of thermistors are configured to detect the spatial pattern of heat emitted by the tissue of the subject, and 63 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wherein the method further comprises: processing data pertaining to the detected spatial pattern of heat to at least one of: improve a spatial resolution of an image formed using the data, determine a velocity of movement of the capsule within the tissue of the subject, or reduce an amount of noise related to transmission of the data. 68. The method of claim 66, wherein the cylindrical wall of the capsule comprises a thermally conductive material, and wherein at least one of the first plurality of thermistors or the second plurality of thermistors are in thermal contact with the thermally conductive material of the cylindrical wall of the capsule. 69. The method of claim 68, wherein the capsule further comprises a controller in communication with the first plurality of thermistors and the second plurality of thermistors, wherein the method further comprises: obtaining, using the controller, data from the first plurality of thermistors and the second plurality of thermistors relating to the detected heat emitted from the tissue of the subject at a first time. 70. The method of claim 69, wherein the capsule further comprises an antenna in communication with the controller, wherein the method further comprises: transmitting, using the controller, the data via the antenna to an external device for at least one of storage or analysis of the transmitted data, wherein the external device is coupled to a receiver belt comprising a plurality of receiver belt antennas in communication with the external device. 71. The method of claim 70, further comprising: determining a first position of the capsule at the first time based on information from the plurality of receiver belt antennas. 64 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 72. The method of claim 71, further comprising: generating a first image of the tissue of the subject at the first position of the capsule of the subject based on the data from the first plurality of thermistors and the second plurality of thermistors relating to the detected heat emitted from the tissue of the subject at the first time. 73. The method of claim 72, further comprising: obtaining, using the controller, data from the first plurality of thermistors and the second plurality of thermistors relating to the detected heat emitted from the tissue of the subject at a second time, determining a second position of the capsule at the second time based on information from the plurality of receiver belt antennas, and generating a second image of the tissue of the subject at the second position of the capsule of the subject based on the data from the first plurality of thermistors and the second plurality of thermistors relating to the detected heat emitted from the tissue of the subject at the second time. 74. The method of claim 73, further comprising: generating a map of heat emitted from the tissue of the subject based on the first image at the first position and the second image at the second position. 75. The method of claim 74, further comprising: determining at least one of the first position of the capsule or the second position of the capsule based on a comparison of the information from the plurality of receiver belt antennas and data regarding a relationship between antenna signal attenuation vs. tissue depth. 76. The method of any one of claims 61-75, wherein the at least one temperature sensor comprises at least one of a thermistor, a thermocouple, a semiconductor-based sensor, or a resistance-based temperature sensor. 77. An apparatus for detecting gastrointestinal pathology, comprising: a capsule comprising a thermal imaging sensor, 65 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 the capsule being configured to be swallowed by a subject, and the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject. 78. The apparatus of claim 77, wherein the capsule further comprises a controller coupled to the thermal imaging sensor and configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject. 79. The apparatus of claim 78, wherein the capsule further comprises a wireless transmitter, and wherein the controller is configured to transmit the at least one image to the wireless transmitter which transmits the at least one image to an external antenna. 80. The apparatus of claim 79, wherein the wireless transmitter is configured to transmit at a frequency in a range of between 400MHz and 500MHz. 81. The apparatus of claim 80, wherein the wireless transmitter is configured to transmit at a frequency of 433MHz. 82. The apparatus of claim 78, wherein the controller is further configured to generate a plurality of images of the detected infrared radiation emitted by the tissue of the subject. 83. The apparatus of claim 82, wherein the controller is further configured to generate at least one image per second of the detected infrared radiation emitted by the tissue of the subject. 84. The apparatus of claim 77, wherein the capsule further comprises a power supply comprising at least one battery. 85. The apparatus of claim 77, wherein the thermal imaging sensor is configured to detect infrared radiation comprising a wavelength in a range of between 7μm to 14μm. 66 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 86. The apparatus of claim 85, wherein the thermal imaging sensor comprises a thermopile sensor. 87. The apparatus of claim 79, wherein the thermal imaging sensor further comprises an ultrasound transducer coupled to the controller. 88. The apparatus of claim 87, wherein the controller is further configured to obtain data from the ultrasound transducer indicative of a depth of fecal matter between the capsule and the tissue. 89. The apparatus of claim 88, wherein the controller is further configured to transmit the data from the ultrasound transducer indicative of the depth of fecal matter between the capsule and the tissue from the wireless transmitter to an external device. 90. The apparatus of claim 77, wherein the thermal imaging sensor is configured to detect infrared radiation following a delay period. 91. The apparatus of claim 90, wherein the delay period is at least two hours. 92. The apparatus of claim 77, wherein the capsule comprises a window which transmits infrared radiation, wherein the window is adjacent to the thermal imaging sensor. 93. The apparatus of claim 92, wherein the window comprises a biocompatible material. 94. The apparatus of claim 93, wherein the window comprises low density polyethylene (LDPE). 95. The apparatus of claim 77, further comprising a reflector disposed adjacent to the thermal imaging sensor, wherein the reflector is configured to direct infrared radiation from the tissue towards the thermal imaging sensor. 67 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 96. The apparatus of claim 95, wherein the reflector is configured to direct infrared radiation from a circumferential region around the capsule. 97. The apparatus of claim 95, wherein the reflector comprises a pyramid shape. 98. The apparatus of claim 97, wherein the pyramid shape comprises an apex comprising a triangular wedge. 99. The apparatus of claim 79, wherein the at least one image comprises a plurality of images, and wherein the controller is configured to average together the plurality of images to produce an averaged image and transmit the averaged image to the wireless transmitter for transmission to the external antenna. 100. A system for detecting gastrointestinal pathology, comprising: a capsule comprising a thermal imaging sensor in communication with a controller and a wireless transmitter; and a plurality of antennas in communication with the capsule, the capsule being configured to be swallowed by a subject, the thermal imaging sensor being configured to detect infrared radiation emitted by a tissue of the subject, the controller being configured to generate at least one image of the detected infrared radiation emitted by the tissue of the subject, the controller being configured to transmit the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas, and the plurality of antennas being configured to receive the at least one image from the wireless transmitter. 101. The system of claim 100, further comprising a computing system in communication with the plurality of antennas, 68 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wherein the computing system is configured to determine a location of the capsule based on the plurality of antennas receiving the at least one image from the wireless transmitter. 102. The system of claim 101, wherein the computing system is further configured to determine the location of the capsule based on an amplitude of a signal received by each of the plurality of antennas. 103. The system of claim 102, wherein the computing system is further configured to determine the location of the capsule with an uncertainty of 1cm or less. 104. The system of claim 100, wherein the plurality of antennas comprises eight antennas. 105. The system of claim 100, wherein each of the plurality of antennas is coupled to a belt, and wherein the belt is configured to be coupled to the subject. 106. The system of claim 101, wherein the at least one image comprises a plurality of images each having a first image resolution, and wherein the computing system is further configured to process the plurality of images to obtain at least one super-resolution image having a second resolution greater than the first resolution. 107. The system of claim 106, wherein at least two of the plurality of images correspond to at least two different positions of the capsule, and wherein the computing system is further configured to obtain the at least one super- resolution image based on interpolating the at least two of the plurality of images corresponding to the at least two different positions of the capsule. 108. A method for detecting gastrointestinal pathology, comprising: providing a capsule comprising a thermal imaging sensor, the capsule being configured to be swallowed by a subject; and 69 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 detecting, using the thermal imaging sensor, infrared radiation emitted by a tissue of the subject. 109. The method of claim 108, further comprising: generating, using a controller coupled to the thermal imaging sensor, at least one image of the detected infrared radiation emitted by the tissue of the subject. 110. The method of claim 109, wherein the capsule further comprises a wireless transmitter, and wherein the method further comprises: transmitting, using the controller, the at least one image to the wireless transmitter which transmits the at least one image to an external antenna. 111. The method of claim 110, wherein the wireless transmitter transmits the at least one image to the external antenna at a frequency in a range of between 400MHz and 500MHz. 112. The method of claim 111, wherein the wireless transmitter transmits the at least one image to the external antenna at a frequency of 433MHz. 113. The method of claim 109, further comprising: generating, by the controller, a plurality of images of the detected infrared radiation emitted by the tissue of the subject. 114. The method of claim 113, further comprising: generating, by the controller, at least one image per second of the detected infrared radiation emitted by the tissue of the subject. 115. The method of claim 108, wherein the capsule further comprises a power supply comprising at least one battery. 116. The method of claim 108, wherein detecting infrared radiation further comprises: detecting, using the thermal imaging sensor, infrared radiation comprising a wavelength in a range of between 7μm to 14μm. 70 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 117. The method of claim 116, wherein the thermal imaging sensor comprises a thermopile sensor. 118. The method of claim 110, wherein the thermal imaging sensor further comprises an ultrasound transducer coupled to the controller, and wherein the method further comprises: obtaining data from the ultrasound transducer indicative of a depth of fecal matter between the capsule and the tissue. 119. The method of claim 118, further comprising: transmitting, by the controller, the data from the ultrasound transducer indicative of the depth of fecal matter between the capsule and the tissue from the wireless transmitter to an external device. 120. The method of claim 108, wherein detecting infrared radiation further comprises: detecting infrared radiation following a delay period. 121. The method of claim 120, wherein the delay period is at least two hours. 122. The method of claim 108, wherein the capsule comprises a window adjacent to the thermal imaging sensor, and wherein the method further comprises: transmitting infrared radiation through the window to the thermal imaging sensor. 123. The method of claim 122, wherein the window comprises a biocompatible material. 124. The method of claim 123, wherein the window comprises low density polyethylene (LDPE). 125. The method of claim 108, further comprising a reflector disposed adjacent to the thermal imaging sensor, and 71 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 wherein the method further comprises: directing, by the reflector, infrared radiation from the tissue towards the thermal imaging sensor. 126. The method of claim 125, wherein directing infrared radiation from the tissue towards the thermal imaging sensor further comprises: directing infrared radiation from a circumferential region around the capsule towards the thermal imaging sensor. 127. The method of claim 125, wherein the reflector comprises a pyramid shape. 128. The method of claim 127, wherein the pyramid shape comprises an apex comprising a triangular wedge. 129. The method of claim 110, wherein the at least one image comprises a plurality of images, and wherein the method further comprises: averaging together the plurality of images to produce an averaged image and transmitting the averaged image to the wireless transmitter for transmission to the external antenna. 130. A method for detecting gastrointestinal pathology, comprising: providing a capsule comprising a thermal imaging sensor, a wireless transmitter, and a plurality of antennas in communication with a controller, the capsule being configured to be swallowed by a subject, detecting, by the thermal imaging sensor, infrared radiation emitted by a tissue of the subject; generating, by the controller, at least one image of the detected infrared radiation emitted by the tissue of the subject; transmitting, by the controller, the at least one image to the wireless transmitter which transmits the at least one image to the plurality of antennas; and receiving, by the plurality of antennas, the at least one image from the wireless transmitter. 72 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 131. The method of claim 130, further comprising: providing a computing system in communication with the plurality of antennas, and determining, using the computing system, a location of the capsule based on the plurality of antennas receiving the at least one image from the wireless transmitter. 132. The method of claim 131, further comprising: determining, using the computing system, the location of the capsule based on an amplitude of a signal received by each of the plurality of antennas. 133. The method of claim 132, further comprising: determining, using the computing system, the location of the capsule with an uncertainty of 1cm or less. 134. The method of claim 130, wherein the plurality of antennas comprises eight antennas. 135. The method of claim 130, wherein each of the plurality of antennas is coupled to a belt, and wherein the method further comprises: coupling the antennas to the subject using the belt. 136. The method of claim 131, wherein the at least one image comprises a plurality of images each having a first image resolution, and wherein the method further comprises: processing, using the computing system, the plurality of images to obtain at least one super-resolution image having a second resolution greater than the first resolution. 137. The method of claim 136, wherein at least two of the plurality of images correspond to at least two different positions of the capsule, and wherein the obtain at least one super-resolution image further comprises: 73 Q B\85925099.1
MGH 2021-602-03 Quarles 125141.04435 obtaining, using the computing system, the at least one super-resolution image based on interpolating the at least two of the plurality of images corresponding to the at least two different positions of the capsule. 138. The method of claim 131, wherein the capsule further comprises an ultrasound transducer coupled to the controller, wherein the method further comprises: obtaining, using the controller, data from the ultrasound transducer indicative of a depth of fecal matter between the capsule and the tissue, transmitting, using the controller, the data from the ultrasound transducer indicative of the depth of fecal matter between the capsule and the tissue from the wireless transmitter to the computing system via the plurality of antennas, and adjusting, using the computing system, the at least one image to compensate for the depth of fecal matter. 74 Q B\85925099.1
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202263384564P | 2022-11-21 | 2022-11-21 | |
| PCT/US2023/080662 WO2024112736A1 (en) | 2022-11-21 | 2023-11-21 | Wireless thermal detection capsule |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4622544A1 true EP4622544A1 (en) | 2025-10-01 |
Family
ID=91196625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP23895378.0A Pending EP4622544A1 (en) | 2022-11-21 | 2023-11-21 | Wireless thermal detection capsule |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4622544A1 (en) |
| JP (1) | JP2025539826A (en) |
| WO (1) | WO2024112736A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119650173B (en) * | 2024-12-12 | 2025-09-26 | 电银配售电有限公司 | A smart monitoring long-life cable and its preparation method |
| CN121324974B (en) * | 2025-12-15 | 2026-04-07 | 电子科技大学 | Lithium sulfur battery thermal runaway monitoring method and system based on neural network |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS5588732A (en) * | 1978-12-26 | 1980-07-04 | Olympus Optical Co | Endoscope |
| WO2008030480A2 (en) * | 2006-09-06 | 2008-03-13 | Innurvation, Inc. | Ingestible low power sensor device and system for communicating with same |
| KR102397233B1 (en) * | 2014-09-25 | 2022-05-12 | 프로제너티, 인크. | Electromechanical pill device with localization capabilities |
| US11564629B2 (en) * | 2014-10-22 | 2023-01-31 | GI Bionics, LLC | Devices for testing distal colonic and anorectal function |
| CN109843149B (en) * | 2016-07-22 | 2020-07-07 | 普罗秋斯数字健康公司 | Electromagnetic sensing and detection of ingestible event markers |
| WO2019176157A1 (en) * | 2018-03-15 | 2019-09-19 | 三菱電機株式会社 | Biological material measurement device |
| IL307631B2 (en) * | 2021-04-12 | 2026-04-01 | Niche Biomedical Inc | Bleeding sensor for a patient's gastrointestinal tract |
-
2023
- 2023-11-21 EP EP23895378.0A patent/EP4622544A1/en active Pending
- 2023-11-21 WO PCT/US2023/080662 patent/WO2024112736A1/en not_active Ceased
- 2023-11-21 JP JP2025529736A patent/JP2025539826A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024112736A9 (en) | 2024-07-11 |
| JP2025539826A (en) | 2025-12-09 |
| WO2024112736A1 (en) | 2024-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Miley et al. | Video capsule endoscopy and ingestible electronics: emerging trends in sensors, circuits, materials, telemetry, optics, and rapid reading software | |
| US12374464B2 (en) | Apparatus, systems, and methods for rapid cancer detection | |
| Sliker et al. | Flexible and capsule endoscopy for screening, diagnosis and treatment | |
| Gulati et al. | The future of endoscopy: Advances in endoscopic image innovations | |
| Sahafi et al. | Edge artificial intelligence wireless video capsule endoscopy | |
| ES2704827T3 (en) | Tissue characterization probe with effective sensor to tissue contact | |
| EP4622544A1 (en) | Wireless thermal detection capsule | |
| WO2007138566A2 (en) | Method of infrared thermography for earlier diagnosis of gastric colorectal and cervical cancer | |
| CN103815858A (en) | Capsular endoscope with multiple built-in sensors | |
| Fontana et al. | State of the art on advancements in wireless capsule endoscopy telemetry: a systematic approach | |
| Alam et al. | IoT-Based intelligent capsule endoscopy system: A technical review | |
| CN106419942B (en) | A kind of capsule endoscope and its application for gastrointestinal protuberant lesions screening | |
| Pahlavan et al. | A novel cyber physical system for 3-D imaging of the small intestine in vivo | |
| WO2025160584A1 (en) | Wireless ingestible luminal diagnostic capsule that autonomously treats a condition | |
| CN106377221A (en) | Radial scanning type capsule endoscopy applied to digestive tract pathologic change screening and application thereof | |
| Bao | On Simultaneous Localization and Mapping Inside the Human Body (Body-SLAM). | |
| CN206462965U (en) | A kind of ray scanning formula capsule endoscope system for Alimentary tract disease examination | |
| Filip | Self–stabilizing capsule endoscope for early detection and biopsy of colonic polyps | |
| Romero et al. | Diagnosis and treatment of cancer using thermal therapies: Minimal and Non-invasive techniques | |
| US11045106B2 (en) | System and method for detecting and diagnosing diseases and use of same | |
| CN202821277U (en) | Thermal Imaging Capsule Endoscopy Device | |
| CN206852604U (en) | A kind of capsule endoscope system for gastrointestinal protuberant lesions examination | |
| CN201929946U (en) | Human surface blood vessel display with infrared thermal scanning function | |
| WO2023076631A1 (en) | Infrared thermal endoscopy | |
| Gregersen | Bionic devices and measurements in the digestive tract |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20250620 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) |