WO2024044613A2 - Apparatus and methods for inflatable tactile sensing - Google Patents

Abstract

Exemplary embodiments of the present disclosure include methods and apparatus for flexible tactile sensing. Certain embodiments include a housing and an inflatable member coupled to the housing, as well as a lumen in fluid communication with the inflatable member. Particular embodiments may also include an imaging device as well as a sensor.

Description

DESCRIPTION

APPARATUS AND METHODS FOR INFLATABLE TACTILE SENSING

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of United States Provisional Patent Application No. 63/401,318 filed August 26, 2022, the entirety of which is incorporated herein by reference.

BACKGROUND INFORMATION

Existing vision-based inspection devices depend heavily on the visual feedback provided by a camera utilized in inspection devices and the ability of the operator to notice potential signs of damage or other issues in the inspected area. Vision-based inspection devices, including endoscopic tools, are omnipresent in modem operating rooms and will continue to play an important role with the growth of robotic minimally invasive procedures. In addition, industrial applications utilize vision-based inspection devices (e.g., for inspection of the pipes and other surfaces).

Existing vision-based endoscopes, colonoscopes, borescopes, and diagnosis tools typically use a frontal view camera that might get occluded and depend heavily on the operator’s ability to notice potential signs of damages, lesions, or polyps. The challenge is compounded when the tool’s lens view is blocked by fluids and semi-solids. This clearly presents a need to supplement endoscopic tools’ vision with other high-resolution modes of sensing.

Such devices rely on the subjective judgment of the operator, which can be compromised when inspection device lens view is blocked or occluded by fluids and semisolids.

Accordingly, a need exists to supplement such inspection devices vision with other high-resolution modes of texture and stiffness sensing.

SUMMARY

Exemplary embodiments of the present disclosure relate to apparatus and methods for tactile sensing and can have medical applications (including for example endoscopic and colonoscopy examinations) and industrial applications (including for example, the diagnosis of pipes to find small cracks or blockages).

Embodiments of the present disclosure are sometimes referred to herein as an Active Inflatable Tactile Sensor (AITS). In certain embodiments, AITS can detect textural features based on the observed deformation of its flexible balloon layer. AITS may be actuated using the inspected device or passively map the internal surface of the environments, and the stiffness of the flexible balloon can be adjusted by fluid pressure depending on the environment.

AITS enable operators to be able to detect features in operations where direct access is not possible. For example, they can help physicians to better diagnose cancerous polyps inside the colon in a colonoscopy procedure that otherwise would have gone unnoticed. Furthermore, with known polyps, AITS can enable physicians to be able to make early-stage diagnosis of the polyp type before in-depth biopsies are performed. AITS can be mounted on the surface of existing inspection devices to safely inspect environments with hard and soft stiffness using their high-resolution tactile sensing capability. These environments can be oil or water pipes or a part of human anatomy such as an artery, upper airway, or colon that is diagnosed for various medical purposes. AITS can include sensors that can provide high resolution topographic mapping and stiffness of the inspected area

In exemplary embodiments, AITS is an active inflatable high resolution vision-based tactile sensor that can be mounted on the external surface or merged within the external surface of an inspection device (e.g., a camera, borescopes, endoscope, colonoscope, and catheter) for inspection and accurate topographic surface mapping and stiffness detection of internal surface of cavities with hard and soft stiffness.

AITS can include an inflatable member (e.g., a stretchable balloon made of silicone or other flexible materials) that can be secured around a rigid/flexible component e.g., an annular or ring-shaped housing) or an inspection device (e.g., the surface of a borescope, catheter, colonoscope). In certain embodiments, the external surface of the AITS contains one or multiple cameras and illumination devices (e.g., LEDs and or optical fibers) to provide adequate lighting conditions for the camera(s). For example, for a ring shape rigid component, based on the camera’s field of view, multiple cameras can be mounted on the cylindrical surface of the ring together with appropriate LEDs. One or multiple air channels and appropriate pressure regulators can be used to inflate and deflate the balloon, as needed during topographic mapping. AITS can be mounted to the external surface of a diagnostic device using the rigid component or can be embedded in the outer surface of the inspection device

Depending on the length of the diagnosis device (e.g., an endoscope or catheter), one or multiple AITS can be used along the length of the device for complete and accurate topographic and surface mapping of the internal surface of the environment. These AITS can be actively actuated or be passively mounted to the external surface of the inspection device. Exemplary embodiments of AITS may comprise small microphones inside the rigid component for hearing the sounds during diagnosis. Sound can additionally help for stiffness classification and better diagnosis and inspection.

AITS can easily be mounted to the current diagnosis devices with a small footprint. For example, exemplary embodiments can easily be mounted to current colonoscopes or endoscopes for simultaneous surface mapping, stiffness detection and frontal typical visual feedback. Exemplary embodiments of AITS include an active inflatable high resolution visionbased tactile sensor that can be used for various applications and particularly medical diagnosis. For example, the concept of catheter balloons can be modified such that an operator cannot only open an occluded vessel using the balloon, but with minor modifications and addition of a camera and illumination system, AITS can completely map the blockage inside the vessel with a high resolution and also simultaneously obtain its stiffness. The same feature can be used for various catheter-balloon interventional radiology procedures. While current endoscopic technologies rely on only vision, AITS utilizes vision and touch to allow operators to detect polyps and lesions in early stages thanks to the highly sensitive environment of the AITS, and even perform early diagnosis. AITS is able to also provide high resolution texture and stiffness mapping of the polyps which is not possible with current technologies.

In exemplary embodiments, AITS stiffness is adjustable and active. In other words, by regulating the air pressure inside the inflatable balloon, an operator can control the level of compliance of the sensor. As opposed to the other available tactile sensors, this feature allows an operator to safely map texture of both hard and soft objects.

Exemplary embodiments of AITS can be actively actuated using a robotic manipulator or current manual colonoscopes. AITS can also be passively mounted on the surface of current devices or can be mounted to a surgical robotic system and be controlled remotely using a teleoperation system. Certain embodiments of AITS may comprise physical markers e.g., miniature dots and specific tags such as ArucoTags) which can help a user to quantify the deformation of the sensor within the environment when there is an interaction with different objects. AITS may also include UV LEDs for the illumination in addition to the standard LEDs, in order to detect hidden properties of the environment which cannot be seen with the visible spectrum wavelengths.

In specific embodiments, AITS inflatable member can be designed in an asymmetric way (e.g., within the camera(s) field or view(s) different regions of the balloon with various thicknesses and hardness values) in order to actively control each region for the specific purpose depending on the tasks. Using computer vision algorithms, it is possible to stitch the images captured by multiple cameras and create a panoramic view of the environment. This can be significant for diagnosing the environment over time and comparing the changes that have occurred to the environment.

Aside from texture mapping, embodiments of AITS can provide stiffness mapping that, together with texture mapping, is useful information for users to detect the type of damage. Moreover, exemplary embodiments address the existing challenge of miniaturization and sensitivity of the current vision-based tactile sensor. The AITS can also be actively actuated and mounted to the current devices and has the potential of miniaturization for various medical and surgical applications. Thanks to the inflation feature of the balloon, tiny sensors can be embedded within the current colonoscope and catheters and provide clinicians with the lost sense of touch of internal anatomies.

AITS is able to detect textural details that are difficult or impossible to distinguish with only visual cues. However, with the tactile sensing of AITS, an operator may note that individual polyps are clearly displayed. Additionally, the fine details of the sample polyps e.g., ridges, imperfections) that the camera missed are also visible in sensor representation. This presents a significant advantage over the existing endoscopic tools that depend solely on vision. Vision supplemented by high fidelity tactile feedback of AITS may allow the operators to not only identify possible damage or issues on the internal surface of the environment, but also allow them to characterize the damage more precisely to determine which type it may be. This is of significant importance for various medical applications, such as cancer polyp classifications. Similar to endoscopic tools, such classifications can be supplemented with computer vision algorithms. Because texture sensing does not depend on lighting conditions and occlusion, the algorithms display a remarkable ability to detect visually nonobvious features and classify them based on their type. Through utilizing typical image stitching algorithms, the whole internal surface of the diagnosed/inspected environment can be mapped with high resolution.

Exemplary embodiments include a flexible tactile sensing apparatus comprising: a housing; an inflatable member coupled to the housing; an imaging device; and a lumen in fluid communication with the inflatable member; and a sensor. Particular embodiments further comprise a gel layer coupled to the inflatable member, and in some embodiments the gel layer is translucent or transparent. In specific embodiments the gel layer is between 3.0 and 5.0 millimeters thick, and in certain embodiments the gel layer comprises a silicone material.

Tn particular embodiments the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member. In some embodiments the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member. In specific embodiments the plurality of sensors is configured to obtain data simultaneously in one-step. In certain embodiments the plurality of sensors is configured to obtain data sequentially. In particular embodiments the housing is coupled to an elongated member. In some embodiments the elongated member comprises a distal end with an axial portion, and the housing is coupled to the axial portion of the distal end of the elongated member. In specific embodiments the inflatable member is configured to expand in a direction along a primary axis of the elongated member. In particular embodiments the elongated member comprises a distal end;, and the housing is coupled to the distal end of the elongated member such that the housing extends circumferentially around the distal end of the elongated member. In some embodiments the inflatable member is configured to primarily expand radially outward from the elongated member.

Certain embodiments further comprise an illumination device. In particular embodiments the housing is an annular member comprising a central opening configured to receive an elongated member. In some embodiments the elongated member is a medical scope, and in specific embodiments the medical scope is a colonoscope or an endoscope. In certain embodiments the elongated member is a borescope, and in particular embodiments the housing is approximately planar. In some embodiments the housing is flexible. In specific embodiments the housing is configured to couple to a robotic hand. In certain embodiments the sensor is configured to measure a pressure within the inflatable member. In particular embodiments the inflatable member is asymmetrical. In some embodiments the inflatable member comprises an outer surface with extended sections and recessed sections. In some embodiments the extended sections and recessed sections form a flower or star pattern. In specific embodiments the inflatable member is symmetrical. In certain embodiments the inflatable member is configured in a ring shape or a hemispherical shape.

In particular embodiments the inflatable member is formed from a single material, including for example, a silicone material. In some embodiments the inflatable member is formed from a plurality of materials, which may comprise at least one silicone material, a gel, and/or a first silicone material and a second silicone material. In specific embodiments the at least one silicone material is coated. In certain embodiments the at least one silicone material is coated with a reflective material, including for example, a reflective aluminum paint. In particular embodiments the at least one silicone material is coated with a pigment which may comprise an aluminum powder in some embodiments.

In specific embodiments the inflatable member has a thickness of 0.1 millimeters to 5.0 millimeters. In certain embodiments the illumination device comprises a light emitting diode (LED) or optical fibers. In particular embodiments the LED comprises a red LED, a blue LED, a green LED, a white LED and/or an ultraviolet LED. In some embodiments the imaging device is configured to provide an endoscopic image and/or a radial image.

Specific embodiments further comprise a computer readable medium containing program instructions for topographic mapping classification of a surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of: (a) obtaining imaging data of the surface; and (b) analyzing the imaging data of the surface. In certain embodiments the computer readable medium contains program instructions for stiffness classification of the surface, wherein execution of the program instructions by one or more processors of the computer system causes the one or more processors to carry out the steps of: (c) obtaining pressure data from the sensor; and (d) analyzing the pressure data from the sensor.

Particular embodiments include analyzing the pressure data from the sensor comprises comparing the pressure within the inflatable member to a parameter of the inflatable member. In certain embodiments the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface. In some embodiments the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member. In specific embodiments the parameter is a volume within the inflatable member. In particular embodiments analyzing the imaging data of the surface comprises analyzing a contrast of the surface. In certain embodiments analyzing the imaging data of the surface comprises analyzing a color of the surface. In particular embodiments analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface.

In some embodiments obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. In specific embodiments the biological tissue is gastrointestinal (GI) tract tissue. In particular embodiments the biological tissue is arterial tissue. In certain embodiments the surface is an interior surface of a pipe. In particular embodiments the pipe is configured to transmit a hydrocarbon. In some embodiments the surface is accessed by a mobile robot. In specific embodiments the surface is contacted by a robotic hand. In certain embodiments the imaging data comprises shape, size, and texture data. Particular embodiments further comprise a sound detector. Certain embodiments further comprise: a power supply; an air supply system; and a controller, wherein the controller is configured to control the air supply system.

Exemplary embodiments include method of analyzing a surface, where the method comprises: (a) positioning an apparatus proximal to a surface, wherein the apparatus comprises: a housing; an inflatable member coupled to the housing; an imaging device; a lumen in fluid communication with the inflatable member; and a sensor; (b) inflating the inflatable member coupled to the housing; (c) obtaining imaging data of the surface with the imaging device; and (d) analyzing the imaging data of the surface.

In particular embodiments step (a) comprises positioning a plurality of apparatus proximal to the surface, wherein each apparatus in the plurality of apparatus comprises: a housing; an inflatable member coupled to the housing; an imaging device; a lumen in fluid communication with the inflatable member; and a sensor; step (b) comprises inflating each inflatable member; step (c) comprises obtaining imaging data from the surface from each of the imaging devices; and step (d) comprises analyzing the imaging data of the surface.

In some embodiments the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member. In specific embodiments the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member. In certain embodiments step (c) comprises obtaining imaging data of the surface simultaneously in one-step with the plurality of imaging devices. In particular embodiments step (c) comprises obtaining imaging data of the surface sequentially with the plurality of imaging devices.

In certain embodiments the method further comprises: (e) obtaining pressure data of the inflatable member from the sensor; and (f) analyzing the pressure data of the inflatable member. In particular embodiments the method further comprises: (g) deflating the inflatable member. Some embodiments further comprise: (h) moving the plurality of apparatus relative to the surface; and (i) repeating steps (a)-(c).

In particular embodiments the apparatus proximal to the surface comprises a computer readable medium containing program instructions for topographic mapping and stiffness classification of the surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of: (c) obtaining imaging data of the surface with the imaging device; and (d) analyzing the imaging data of the surface. In some embodiments the computer readable medium contains program instructions for topographic mapping and stiffness classification of the surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out steps of: (e) obtaining pressure data of the inflatable member from the sensor; and (f) analyzing the pressure data of the inflatable member.

In certain embodiments analyzing the imaging data of the surface comprises calculating the texture, size, shape, or stiffness of the surface. In particular embodiments analyzing the pressure data of the inflatable member comprises calculating the stiffness of the surface. In some embodiments analyzing the pressure data of the surface comprises comparing the pressure within the inflatable member to a parameter of the inflatable member. In specific embodiments the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface. In certain embodiments the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member. In particular embodiments the plurality of points on the inflatable member are marked with a plurality of fiducial markers. In some embodiments the plurality of fiducial markers comprises fiducial markers of different sizes. In specific embodiments the parameter is a volume within the inflatable member. In certain embodiments analyzing the imaging data of the surface comprises analyzing a contrast of the surface. In particular embodiments analyzing the imaging data of the surface comprises analyzing a color of the surface. In some embodiments analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface. In specific embodiments obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. In certain embodiments the biological tissue is gastrointestinal (Gl) tract tissue. In particular embodiments the biological tissue is arterial tissue. In some embodiments the surface is an interior surface of a pipe, and in specific embodiments the pipe is configured to transmit a hydrocarbon. In certain embodiments the imaging data comprises shape, size, and texture data. Particular embodiments further comprise detecting sound.

Exemplary embodiments include an apparatus comprising: an annular member comprising a central opening configured to receive a medical scope; an illumination device; an imaging device; an inflatable member coupled to the annular member; a lumen in fluid communication with the annular member; and a sensor configured to measure a pressure within the inflatable member. Certain embodiments further comprise a gel layer coupled to the inflatable member. In particular embodiments the gel layer is translucent or transparent. In some embodiments the gel layer is between 3.0 and 5.0 millimeters thick. In specific embodiments the gel layer comprises a silicone material.

In specific embodiments the illumination device is a first illumination device in a plurality of illumination devices; the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of illumination devices is contained within the inflatable member; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member. In certain embodiments the plurality of illumination devices, the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member. In particular embodiments the plurality of illumination devices is configured to obtain data simultaneously in one-step. In some embodiments the plurality of illumination devices is configured to obtain data sequentially.

In certain embodiments the medical scope is a colonoscope or an endoscope. In particular embodiments the inflatable member is asymmetrical. In some embodiments the inflatable member is symmetrical. In specific embodiments the inflatable member is formed from a single material. In certain embodiments the single material is a silicone material. In particular embodiments the inflatable member is formed from a plurality of materials. In some embodiments the plurality of materials comprises at least one silicone material. In specific embodiments the plurality of materials comprises a first silicone material and a second silicone material. In certain embodiments the inflatable member has a thickness of 0.1 millimeters to 5.0 millimeters. In particular embodiments the illumination device comprises a light emitting diode (LED). In some embodiments the LED comprises a red LED, a blue LED, a green LED, a white LED or an ultraviolet LED. In specific embodiments the imaging device is configured to provide an endoscopic image and/or a radial image.

Certain embodiments further comprise a computer readable medium containing program instructions for topographic mapping classification of a surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of: (a) obtaining imaging data of the surface; and(b) analyzing the imaging data of the surface. In particular embodiments the computer readable medium contains program instructions for stiffness classification of the surface, wherein execution of the program instructions by one or more processors of the computer system causes the one or more processors to carry out the steps of: (c) obtaining pressure data from the sensor; and (d) analyzing the pressure data from the sensor.

In some embodiments analyzing the pressure data from the sensor comprises comparing the pressure within the inflatable member to a parameter of the inflatable member. In specific embodiments the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface. In certain embodiments the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member. In particular embodiments the parameter is a volume within the inflatable member. In some embodiments analyzing the imaging data of the surface comprises analyzing a contrast of the surface. In specific embodiments analyzing the imaging data of the surface comprises analyzing a color of the surface. In certain embodiments analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface. In particular embodiments obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. In some embodiments the biological tissue is gastrointestinal (Gl) tract tissue. In specific embodiments the biological tissue is arterial tissue. In certain embodiments the imaging data comprises shape, size, and texture data. Particular embodiments further comprise a sound detector. Specific embodiments include a method of analyzing a surface, the method comprising: (a) positioning an apparatus proximal to a surface, where the apparatus comprises: a housing; an inflatable member coupled to the housing; a plurality of sensors contained within the inflatable member; and an imaging device; (b) inflating the inflatable member coupled to the housing; and (c) obtaining data of the surface with the plurality of sensors.

In certain embodiments the plurality of sensors is spaced axially along the inflatable member. In particular embodiments obtaining data of the surface with the plurality of sensors comprises obtaining data simultaneously in one-step with the plurality of sensors. In specific embodiments obtaining data of the surface with the plurality of sensors comprises obtaining data sequentially with the plurality of sensors.

In the present disclosure, the term “coupled” is defined as connected, although not necessarily directly, and not necessarily mechanically.

The use of the word “a” or “an” when used in conjunction with the term “comprising” in the claims and/or the specification may mean “one,” but it is also consistent with the meaning of “one or more” or “at least one.” The terms “approximately, “about” or “substantially” mean, in general, the stated value plus or minus 10%. The use of the term “or” in the claims is used to mean “and/or” unless explicitly indicated to refer to alternatives only or the alternative are mutually exclusive, although the disclosure supports a definition that refers to only alternatives and “and/or.”

The terms “comprise” (and any form of comprise, such as “comprises” and “comprising”), “have” (and any form of have, such as “has” and “having”), “include” (and any form of include, such as “includes” and “including”) and “contain” (and any form of contain, such as “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises,” “has,” “includes” or “contains” one or more steps or elements, possesses those one or more steps or elements, but is not limited to possessing only those one or more elements. Likewise, a step of a method or an element of a device that “comprises,” “has,” “includes” or “contains” one or more features, possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.

Other objects, features and advantages of the present invention will become apparent from the following detailed description. It should be understood, however, that the detailed description and the specific examples, while indicating specific embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will be apparent to those skilled in the art from this detailed description.

BRIEF DESCRIPTION OF THE FIGURES

The following drawings form part of the present specification and are included to further demonstrate certain aspects of the present invention. The invention may be better understood by reference to one or more of these drawings in combination with the detailed description of specific embodiments presented herein.

FIG. 1 illustrates a perspective view of an exemplary embodiment of an apparatus according to the present disclosure with an inflatable member in an inflated state.

FIG. 2 illustrates a partial assembly of the embodiment of FIG. 1 without an inflatable member.

FIG. 3 illustrates a side view of the embodiment of FIG. 1 with the inflatable member in a deflated state.

FIG. 4 illustrates a perspective view of an apparatus comprising embodiments of FIG. 1 with inflatable members in an inflated state.

FIG. 5 illustrates different embodiments of inflatable members according to the present disclosure.

FIG. 6 illustrates exemplary embodiments of the present disclosure coupled to a robotic hand.

FIG. 7 illustrates exemplary embodiments of the present disclosure in different medical applications.

FIG. 8 illustrates a colonoscopy apparatus comprising embodiments of FIG. 1.

FIG. 9 illustrates embodiments of the present disclosure in pipe inspection applications.

FIG. 10 illustrates embodiments of the present disclosure coupled to a mobile robot.

FIG. 11 illustrates perspective views of the embodiment of FIG. 1 with an inflatable member comprising a plurality of materials.

FIG. 12 illustrates image data obtained from a surface according to exemplary embodiments of the present disclosure. FIG. 13 illustrates a plurality of fiducial markers coupled to a gel layer configured for coupling to an inflatable member according to exemplary embodiments of the present disclosure.

FIG. 14 illustrates an embodiment of the present disclosure configured for sequential mapping during inspection of a pipe.

FIG. 15 illustrates exemplary embodiments of the present disclosure comprising a plurality of sensors contained within a single inflatable member.

FIG. 16 illustrates the embodiments of FIG. 15 during use.

FIG. 17 illustrates an embodiment of the present disclosure comprising a plurality of sensors contained within a single inflatable member during use.

FIG. 18 illustrates an embodiment of the present disclosure comprising an inflatable configured to primarily expand radially outward during use.

FIG. 19 illustrates an embodiment of the present disclosure comprising an inflatable configured to expand in an axial direction from the distal end of a scope during use.

FIG. 20 illustrates an embodiment of the present disclosure coupled to a distal end of a colonoscope.

FIG. 21 illustrates an exploded view embodiment of the present disclosure comprising an inflatable member with a sensor and a gel layer.

FIG. 22 illustrates an assembled view of the embodiment of FIG. 21 in a deflated state.

FIG. 23 illustrates an assembled view of the embodiment of FIG. 21 in an inflated state.

FIG. 24 illustrates a schematic view of an embodiment of the present disclosure comprising a controller, power supply and air supply components.

FIG. 25 illustrates a force model of one embodiment according to the present disclosure.

FIG. 26 illustrates a schematic view of an embodiment according to the present disclosure used to obtain data disclosed herein. FIG. 27 illustrates graphical force-displacement experimental results of embodiments according to the present disclosure.

FIG. 28 illustrates graphical stiffness-air pressure experimental results of embodiments according to the present disclosure.

FIG. 29 illustrates a schematic view of components according to the present disclosure to obtain experimental data disclosed herein.

FIG. 30 illustrates a first set of imaging data obtained from the embodiment according to FIG. 29.

FIG. 31 illustrates a second set of imaging data obtained from the embodiment according to FIG. 29.

FIG. 32 illustrates an apparatus according to the present disclosure during an inflation.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

For purposes of clarity, certain elements in some of the figures may not be labeled with a reference number. Referring initially to FIGS. 1-4, one exemplary embodiment of the present disclosure includes an apparatus 100 for flexible tactile sensing, which comprises an inflatable member 110 coupled to a housing 120, and a lumen 130 in fluid communication with inflatable member 110. Lumen 130 can be coupled to a fluid source providing fluid (e.g., gas or liquid) to inflate and deflate inflatable member 110. In addition, apparatus 100 comprises an imaging device 140 and a sensor 150. In certain embodiments, sensor 150 is configured to measure a pressure and/or a sound within the inflatable member 110.

In the embodiment shown, housing 120 is an annular member comprising a central opening 125 configured to receive an elongated member 180, but as explained further below, other embodiments may comprise different configurations. In this illustrated embodiment, apparatus 100 also comprises an illumination device 160, which may comprise a light emitting diode (LED) or optical fibers. Illumination device 160 may include a red LED 161, a blue LED 162, a green LED 163, a white LED 164 and/or an ultraviolet LED 165 as shown in FIG. 4. In particular embodiments, inflatable member 110 may have a thickness of 0.1 millimeters to 5.0 millimeters. Imaging device 140 can be configured to provide an endoscopic and/or radial image depending on the orientation of imaging device 140. While an annular ring configuration for inflatable member 110 may be used in certain embodiments, other embodiments may comprise different configurations for inflatable member 110, including both asymmetrical and symmetrical configurations. As shown in FIG. 5, inflatable member 110 may comprise an outer surface with extended sections 111 and recessed sections 112 that form a flower or star pattern 113. Inflatable member 110 may also comprise a hemispherical shape 115 configured for use with housing 120 that is approximately planar. In still other embodiments, inflatable member 110 may comprise a hemispherical shape 116 with a central aperture 117.

In certain embodiments, housing 120 is configured such that apparatus 100 can be coupled to a robotic hand 170 as shown in FIG. 6. In certain embodiments apparatus 100 can be attached on all fingers and any place on the fingers or the palm of hand 170. In particular embodiments, the hemispherical configuration shown in the center of FIG. 5 can be coupled to robotic hand 170. As shown in FIG. 5, the hemispherical configuration of inflatable member 110 can be coupled to a housing 120 that is substantially planar. In certain embodiments, a substantially planar housing 120 can be flexible and can be coupled to robotic hand 170 an adhesive or other suitable mechanisms.

As shown in FIG. 7-8, in certain embodiments elongated member 180 can be used in medical applications. For example, in particular embodiments elongated member 180 is a colonoscope 181 or a catheter 182 (further comprising an endoscope 183 and/or other medical instruments). In some embodiments elongated member 180 is a laparoscopic tool 184. In particular embodiments, apparatus 100 can be configured such that it can be retrofitted to an elongated member 180 that is an existing device, while in other embodiments apparatus 100 may be a component of a new device created specifically for an inspection or analysis task.

In addition to medical applications, certain embodiments of the present disclosure can be configured to use in industrial applications. For example, as shown in FIG. 9 in some embodiments apparatus 100 can be coupled to an elongated member 180 that is a borescope 185 configured for inspection of a pipe 190. In some embodiments apparatus 100 may be coupled to a pipe inspection robot 191 configured to traverse inside pipe 190. As shown in FIG. 10, in still other embodiments, apparatus 100 can be coupled to a mobile robot 171 which can be used to access surfaces in environments unsafe for human inspectors. Mobile robot 171 can be controlled remotely using a teleoperation system. In particular embodiments inflatable member 110 is formed from a single material, including for example a silicone material. As shown in FIG. 11 in other embodiments, inflatable member 110 is formed from a plurality of materials, including for example, one or more silicone materials. In particular embodiments inflatable member 110 may comprise one or more gel layers 118, and/or a reflective material 119. In particular embodiments a portion of inflatable member 110 (including for example, gel layer 118) may be coated with a reflective material 119. In specific embodiments, reflective material 119 may be an aluminum paint. In addition, all, or portions of inflatable member 110 may be coated with a pigment, which may comprise an aluminum powder in certain embodiments. Gel layers 118 may comprise different material properties (e.g., transparency and stiffness) and may also be coated for adding to the sensitivity of a sensor of apparatus 100. Gel layers 118 may be attached to any symmetric or asymmetric designs of inflatable member 110 and may be located in front of each imaging device 140 such that a gel layer 118 covers the field of view of imaging devices 140.

FIG. 12 shows image data 145 obtained from a gel layer interacting with a surface 195. FIG. 12 also shows image data 146 obtained from a gel layer interacting with a surface 196. As shown in the image data, the contours of surfaces 195 and 196 are clearly visible in image data 145 and 146, showing the raised and recessed portions of surfaces 195 and 196.

Depending on the configuration of inflatable member 110 and housing 120, apparatus 100 may be placed within a confined space (e.g., a gastrointestinal [GI] tract or a pipe) or placed on a surface to obtain and analyze imaging data of the surface. In particular embodiments, apparatus 100 may comprise a computer readable medium containing program instructions for topographic mapping classification of the surface, where execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of obtaining and analyzing the imaging data of the surface from imaging device 140 and/or obtaining and analyzing the pressure data of the surface from sensor 150. Referring back now to FIG. 8, in one specific embodiment processors 200 may be part of a system 250 specifically configured for performing a colonoscopy on a patient 220. It is understood that other systems utilizing apparatus 100 and processors 200 may be used in other applications, including other medical and industrial applications.

In certain embodiments, analyzing the pressure data from sensor 150 comprises comparing the pressure within inflatable member 110 to a parameter of inflatable member 110. In specific embodiments, the parameter may be a volume within inflatable member 110. In other embodiments, the parameter is a distance and/or an orientation from a reference point to an interface between inflatable member HO and the surface that is being analyzed (e.g., a biological tissue or a surface of a pipe or other structure). As shown in FIG. 13, the reference points may be indicated by fiducial markers 109 coupled to a gel layer 118. In specific embodiments fiducial markers 109 may be configured as miniature dots or specific tags such as ArucoTags, which can help a user to quantify the deformation of inflatable member 110 within the environment when there is an interaction with different objects. In certain embodiments, a plurality of fiducial markers 109 can be coupled to gel layer 118 with different sizes e.g., 0.2 mm- 10 mm) and in different distribution and distances with respect to each other. In the example shown in the FIG. 13, twenty-five fiducial markers are coupled to gel layer 118. In certain embodiments, a computer processor can analyze the position and orientation of each marker with respect to an imaging device.

In certain embodiments, analyzing the imaging data of the surface comprises analyzing a contrast, color, shape, size, stiffness, and/or texture of the surface. As previously noted, the surface may be a biological tissue, including gastrointestinal (GI) tract tissue or arterial tissue, or the surface could be non-biological tissue, including for example, the interior surface of a pipe, which may be configured to transmit hydrocarbons.

In summary, certain embodiments of apparatus 100 can be used to analyze a surface by: (a) positioning apparatus 100 proximal to the surface to be analyzed; (b) inflating inflatable member 110 (e.g., via lumen 130); (c) obtaining imaging data of the surface with imaging device 140; and (d) analyzing the imaging data of the surface. In particular embodiments apparatus 100 can also obtain pressure data of inflatable member 1 10 from sensor 150, and then analyze the pressure data. As previously mentioned, certain embodiments of apparatus 100 may also comprise a computer readable medium containing program instructions for topographic mapping and stiffness classification of the surface.

Particular embodiments can comprise a plurality of sensors spaced along the axial length of an elongated member where the sensors are configured to provide sequential mapping (e.g. imaging and/or detection of particular parameters) of a conduit into which the sensors have been inserted. FIG. 14 illustrates the interaction of an object with the sensing module of an apparatus 100, sometime referred to herein as a Soft and Inflatable Vision-based Tactile Sensing (SI-VITS) apparatus. This interaction results in a visible deformation that can be captured by an embedded camera in the frame. In an exemplary tactile sensing procedure, as depicted in FIG. 14, the following steps are considered. Initially, one or more deflated SI-VTSs are integrated with a flexible manual or robotic inspection device (e.g., an endoscope for pipe inspection or colonoscope for cancer diagnosis). Subsequently, following a typical inspection procedure, the user can insert the inspection robot/device into a generic constrained environment with symmetric (e.g., circular) or asymmetric (non-uniform rectangular) internal geometry (as shown in FIG. 14) and advance it while sequentially inflating the SI-VTS, whenever inspection is needed. This inflation process allows for capturing high-resolution textural images of the environment surface, while ensuring a safe interaction force between the sensor and environment.

In the embodiment shown in FIG. 14, a plurality of apparatus 100 are coupled to elongated member 180 (e.g. a borescope) configured for inspection of pipe 190. When an apparatus 100 is proximal to a region of interest 197 or 198, inflatable member 110 can be inflated and sensor 150 activated to detect parameters of regions of interest 197 and 198, as described herein. In the embodiment shown in FIG. 14, image data 297 and 298 is obtained of regions of interest 197 and 198 respectively. Accordingly the ability to provide sequential mapping with multiple axial sensors 150 can reduce time by obtaining parameters of multiple regions of interest at the same time. In addition, the incorporation of multiple axial sensors 150 can provide for more detailed analysis of a conduit surface by incrementally advancing the sensors 150 through the conduit and obtaining data from all of the sensors at specified intervals.

In still other embodiments, multiple axial sensors may be contained within a single inflatable member. In certain embodiments, the axial sensors may be configured to obtain data simultaneously in one-step, while in other embodiments the axial sensors may be configured to obtain data sequentially.

Referring now to FIG. 15 embodiments of an apparatus 100 are shown comprising a plurality of sensors 150 spaced axially along elongated member 180, where the plurality of sensors is contained within a single inflatable member 110. As described in other embodiments disclosed herein, sensors 150 may comprise imaging devices, pressure sensors, sound sensors or other sensing devices as appropriate. The individual components of sensors 150 are not labeled in FIG. 16 for purposes of clarity. FIG. 16 illustrates the embodiments of FIG. 15 in use performing an inspection of a gastrointestinal [GI] tract 192 comprising regions of interest 198. In particular embodiments, GI tract 192 may be a colon, regions of interest 198 may by polyps and apparatus 100 can be coupled to colonoscope 181. In other embodiments, apparatus 100 can be coupled to an endoscope or cystoscope and be used for inspection of a bladder, stomach or other biological tissue. While a GI tract is shown in FIG. 16, it is understood that other embodiments may comprise multiple sensors within a single inflatable member to perform inspections of other confined spaces, including for example, industrial applications to inspect pipes or conduits as needed.

The incorporation of multiple sensors spaced within a single inflatable member can allow a user to simultaneously in one-step inspect a larger surface area of the object being inspected (e.g. GI tract or pipe) without having to sequentially obtain inspection data and sequentially advance the apparatus to obtain inspection data at multiple locations. This can reduce the time needed for inspection and therefore minimize the potential for complications, particularly during medical procedures. The reduced inspection time can also reduce the associated costs for a particular procedure. For example, an embodiment of apparatus 100 shown in FIG. 17 comprises a sufficient number of sensors 150 contained within inflatable member 110 to detect multiple regions of interest 198 (e.g. polyps) over a large, curved portion of GI tract 192. The detailed views of regions of interest 198 shown in FIG. 17 include the actual surface on the right and the image data obtained by apparatus 100 on the left.

Referring now to FIGS. 18 and 19, embodiments of an apparatus 100 are shown coupled to an endoscope 87 and a cystoscope 88 during inspection of a stomach 187 and a bladder 188, respectively. An overall schematic view of the procedure inspecting stomach 187 is shown on the left portion of FIGS. 18 and 19, while an overall schematic view of the procedure inspecting bladder 188 is shown on the right portion of FIGS. 18 and 19. Enlarged detailed views of the inspection of stomach 187 and bladder 188 are shown in the central left and central right portions of FIGS. 18 and 19 respectively. In the enlarged views in the central portion of the figures, inflatable member 110 is shown in a deflated state during the inspection of stomach 187 and an inflated state during the inspection of bladder 188. In FIG. 18, apparatus 100 is configured such that it is coupled circumferentially around the distal end of endoscope 87 or cystoscope 88. Accordingly, inflatable member 110 is configured to primarily expand radially outward from endoscope 87 or cystoscope 88 and utilize a side view and contact approach during the inspection procedure.

In the embodiment shown in FIG. 19, apparatus 100 is configured to couple to axial portion 288 of the distal end of cystoscope 88 or axial portion 287 of endoscope 87 (in contrast to the embodiment of FIG. 18, in which apparatus 100 extends circumferentially around endoscope 87 or cystoscope 88). Accordingly, in the embodiment shown in FIG. 19, inflatable member 110 is configured to expand in an axial direction from the distal end of endoscope 87 or cystoscope 88 (e.g. in a direction along the primary axis of endoscope 87 or cystoscope 88). The configuration shown in FIG. 19 can reduce the overall diameter of apparatus 100 and allow for inspection of very small cavities.

FIG. 20 illustrates an embodiment of an apparatus 100 coupled to a distal end of a colonoscope 89. While specific dimensions are shown of various components in FIG. 20, it is understood that other embodiments may have dimensions different than the illustrated embodiment. Panel (a) illustrates a ring-shaped housing 120 comprising a peripheral notch 121 near each end of housing 121. In the embodiment shown, housing 120 is approximately 26.5 mm in diameter and 30 mm in length. An imaging device 140 (e.g. camera) and illumination devices 160 (e.g. LEDs) are coupled to housing 120.. Panel (b) of FIG. 20 shows apparatus 100 in the deflated mode of inflatable member 110 coupled to housing 120 and integrated with the distal end of colonoscope 89 (having a diameter of 13.2 mm). Panel (c) illustrates the inflated mode of inflatable member 110 coupled to housing 120 and integrated with the distal end of colonoscope 89. In the embodiment shown, inflatable member 110 can expand to an inflated diameter of approximately 60 mm. Panel (d) of FIG. 20 illustrates the deflated mode of inflatable member 110 integrated with the distal end of a colonoscope, with lumen 130 (configured to provide air or other suitable fluid to inflate inflatable member 110) and electronic cable 131 configured to provide electronic communication (e.g. electric power, data transfer and control signals) to imaging device 140 and illumination devices 160. In the embodiment shown, inflatable member 110 has a length of approximately 25 mm and an uninflated diameter of approximately 40 mm. Panel (e) illustrates apparatus 100 coupled to a distal end of colonoscope 89 with inflatable member 110 expanded. In this embodiment, a gel layer 151 is coupled to inflatable member 110, as discussed in further detail below.

An exemplary embodiment of an apparatus 100 is shown in FIGS. 21-23. FIG. 21 shows an exploded view of the components, while FIG. 22 shows an assembled view in a deflated state and FIG. 23 shows an assembled view in an inflated state. As depicted in one embodiment of apparatus 100 shown in FIG. 21, the SLVTS comprises a ring-shape frame or housing 120 that houses an imaging device 140 (e.g. camera), a torus-shape inflatable member 110 coupled to housing 120, an a stretchable sensing module 150 adhered to inflatable member 110. Housing 120 of apparatus 100 further comprises lumen 130 in fluid communication with inflatable member 110, allowing for inflation or deflation of inflatable member 110, as shown in FIGS. 23 and 22 respectively. As shown in FIG. 23, imaging device 140 provides a wide field of view (FOV) greater than 90 degrees, which extends across a majority of a gel layer 151, as discussed further below. Additionally, small illumination devices 160 (e.g. LEDs) are present to provide illumination within the inflated region of inflatable member 110, ensuring sufficientlighting conditions for imaging device 140. In the embodiment shown, housing 120 also comprises one or more lumens 161 to allow for communication (e.g. wires or other suitable transmission means) with illumination devices 160 and/or imaging device 140.

In certain embodiments, the ring-shapegeometry of housing 120 and the torus-shape geometry of inflatable member 110 have been designed for its integration with a flexible industrial or medical endoscopes. This geometry can be readily changed based on the considered application and the integrating device. Accordingly, the configuration of apparatus 100 replaces the rigid components of typical vision-based tactile sensors (VTSs) with inflatable member 110, and the stiffness of inflatable member 110 can actively be adjusted by controlling the input airpressure. As a result of using the inflatable structure shown in FIGS. 21-23, not only can the overall size of the sensor be miniaturized by avoiding the need for a rigid housing structure and supporting plate, but the system also gains conformable and stiffness-tuning characteristics. Of note, this feature also addresses the need for a safe interaction with the sensitive environments.

In addition, the ability to inflate inflatable member 110 eliminates the need for an additional actuation mechanism to apply the desired amount of force needed to deform sensor module 150 through contact with the region of interest. This provides for a more compact design and reduces the overall size of apparatus 100, allowing for the inspection of confined spaces. In addition, the interaction force between inflatable member 110 and the region of interest can also be tuned and adjusted depending on the stiffness of the object and sensitivity of the application.

In certain embodiments sensing module 150 comprises a gel layer 151. In specific embodiments, gel layer 151 may be transparent or translucent, stretchable, and readily deformable e.g. less stiff than the region of interest being analyzed). In particular embodiments, gel layer 151 may comprise a stretchable and soft silicon that can adhere well to inflatable member 110 and uniformly inflate and deflate with inflatable member 110. The transparent or translucent properties of gel layer 151 allow sending module 150 to clearly reflect the textural features of the region of interest to be recorded with imaging device 140. In specific embodiments, gel layer 151 may comprise Ecoflex® 00-31 Near Clear or Ecoflex ® 00-10 (Smooth-On, Inc.™). Ecoflex® 00-31 Near Clear is a more transparent and stiffer (i.e., 00-31 Shore hardness) silicon compared with Ecoflex® 00-10, which is a water white translucent and softer (00-10 Shore hardness) silicone. Both silicones are soft, very strong, and highly stretchable without distortion.

Aside from material properties of the sensing module 150, the shape and, particularly, the thickness of this gel layer 151 can determine the detection area and sensitivity of apparatus 100, respectively. Considering the deflated and inflated geometry of the inflatable member (shown in FIGS. 21 and 23, respectively), the dimensions and geometry of gel layer can be determined based on the target application and size of the inspection space. In the embodiment shown in FIG. 22, gel layer 151 comprises a length L_sensor, a width Wsensor and a thickness t_sensor. In non-limiting examples, L_senSor, may be between 1 and 80 millimeters, width W_sensor may be between 1 and 60 mm, and a thickness tsensor may be between 1 and 5 mm, or more particularly 3 mm.

Referring now to FIG. 24, a schematic is shown of one embodiment of an apparatus 100 coupled to a controller 101, a power supply 102 and an air supply system 103 (e.g. comprising one or more air compressors, vacuum pumps and valves). The inflation and deflation of inflatable member 110 is controlled using the pneumatic actuation system (Programmable Air, Crowd Supply). This system comprises two air pumps, three solenoid valves, and an air pressure sensor. Using this circuit, air pressure inside the inflatable member 1 10 is actively measured using the pressure sensor and compensated in case of an abnormal pressure decrease after inflation. Additionally, SI-VTS’s three LEDs and camera are controlled by a DC power supply and a single-board computer (Raspberry Pi 4 Model B, Raspberry Pi Foundation), respectively). It is understood that this embodiment is only one exemplary embodiment, and other embodiments within the scope of this disclosure may comprises different components than those illustrated in FIG. 24.

FIG. 25 illustrates a force model 175 of one embodiment according to the present disclosure. For a typical VTS, to increase sensitivity of the sensor and obtain high-quality textural outputs during interaction of an object with the sensing layer, a large relative deformation A between the object and the sensing layer is preferred. This relative deformation is defined by the stiffness of the object and the sensing layer Ksensor (FIG. 25, upper left). For an ideal VTS, therefore, to increase sensitivity of the sensor and create high- quality textural outputs, deformation of the sensing module needs to be larger than the object. To address this point in a typical VTS such as the Gelsight sensor [27], the flexible sensing module is typically fabricated with a low stiffness silicone that is supported by a rigid plexiglass layer. Nevertheless, in exemplary embodiments of SI-VITS of the present disclosure, the inflated balloon functions as the rigid plexiglass in a VTS and its internal pressure defines the stiffness of the balloon KBaiioon (FIG. 25, upper right) in apparatus 100. In other words, when SI- VTS interacts with an object, aside from deformation of the sensing module and object, the balloon may also undergo deformation, which is not desired. For an ideal SI-VTS, the inflated balloon should perform equivalent to a rigid plexiglass. Characterizing the stiffness of SI-VTS (Ks _tem) in interaction with the object is therefore important and needs to be appropriately designed for obtaining high-quality textural images.

As shown in FIG. 25 model 175 (lower center) when the SI-VTS interacts with an object with force F , since both sensing module and the inflated balloon are adhered together, both layers experience an equivalent deformation (i.e., Asensor = Abaiioon). Therefore, we can model the overall stiffness of the SI-VTs as two parallel springs with variable stiffness of aKsensor for the sensing layer and KBaiioon for the balloon at different internal balloon pressures Pair

where, a is a constant denoting the rate of change in the stiffness of sensing module at different balloon’s internal pressures. Of note, a is found experimentally. See the section entitled “Stiffness Characterization Experiments” for additional information regarding finding this contant.

EVALUATION EXPERIMENTS

To thoroughly evaluate the performance of the proposed SI-VTS and investigate its performance on creating high-quality textural images, the investigators first fabricated two different types of sensors with identical geometry and size. For the first SI-VTS (i.e., Type- A), the investigators fabricated both stretchable balloon and sensing module using Ecoflex 00-31 Near Clear silicone. The second type of SI- VTS (i.e., Type-B) utilized Ecoflex 00-31 Near Clear silicone and Ecoflex 00-10 for fabrication of the balloon and sensing module, respectively. Therefore, Type-A utilizes a transparent and stiffer silicone compared with Type- B in which a softer and translucent silicone is used. These two types helped the investigators to analyze the effect of these silicone materials on the performance of the SI-VTS through various experiments. Experiments included characterizing the stiffness of the SI-VTSs, analyzing the textural outputs of both types of sensors in interaction with different objects, identifying the resolution of these sensors, and evaluating their conformability and selfactuation in a constrained and confined environment.

Stiffness Characterization Experiments

To characterize the performance of the fabricated Type-A and Type-B sensors using the proposed model in (1), the experimental setup shown in FIG. 26 was used. This setup consisted of:: [a] a single-row linear stage with 1 pm precision (M-UMR12.40, Newport); [b] a digital force gauge with 0.02 N resolution (Mark- 10 Series 5, Mark-10 Corporation) attached to the linear stage to precisely push a force gauge to the SI-VTS and measure the applied interaction force; [c] the SI-VTS; and [d] the pneumatic actuation system (Programmable Air, Crowd Supply) for controlling the inflation and deflation of the SI- VTS as well as measuring the pressure inside the balloon. Using this setup, the investigators performed three independent experiments to experimentally characterize stiffness of SI-VTS components (i. e., the stretchable balloon V^Baiioon, the sensing module, Ksensor), and the SI-VTS V system.

First, to solely measure the stiffness of the sensing module Ksensor, as depicted in FIG. 25, upper left, the sensing module was adhered to a flat surface. Next, the force gauge probe with a circular cross section (diameter = 4.8 mm) was pushed on the surface of the sensing module while its displacement and interaction forces were being recorded. For this experiment, the force measurements were recorded for every 1 mm move- ment of the linear stage for a total displacement of 10 mm. Of note, this experiment was repeated three times for both Type- A and Type-B sensing module. Graphs (a) and (b) in FIG. 27 display the average experimental results obtained for the Type-A and Type-B sensors and trials, respectively.

As the second type of experiments, to solely measure stiffness of an inflated balloon in different pressures, the investigators first integrated the SI-VTS with the ring-shape frame (as shown in FIG. 25, upper right and FIG. 26). Since the stiffness values of Keaiioon directly depends on the internal pressure of the inflated balloon, for this experiment, the investigators followed the same experimental procedure performed for measuring stiffness of the sensing module, but here the force gauge probe was pushing on the balloon layer during its inflation. Specifically, for every pressure ranging from 6 kPa to 10 kPa with 2 kPa intervals, the investigators recorded force measurements for every 1 mm movement of the linear stage on the inflated balloon for a total displacement of 10 mm. Note that these pressures were found after performing preliminary tests and realizing pressures under 6 kPa would not create enough internal pressure for the balloon and pressures after 11 kPa can cause the balloon failure. The investigators repeated this test three times for both Sl-VTS Type-A and Type-B. Since these sensors were made of different silicone materials, the stiffness results were not identical (see graphs (a) and (b) in FIG. 27).

Finally, as shown in FIG. 25 (lower center) and FIG. 26, to measure the stiffness of the SI-VTS (i.e., the sensing module integrated with the inflated balloon)- denoted by Ky,,,,,-,,. the investigators followed the same procedure performed for measuring the stiffness of the inflated balloon. Of note, here the force gauge probe was pushed on the sensing module during the balloon’s inflation. Similarly, the investigators recorded the measured forces at each pressure and force gauge displacements. The investigators repeated this experiment for both Type-A and Type-B SI-VTSs. Graphs (a) and (b) in FIG. 27 display the obtained experimental results for these sensors, respectively. Further, Table I summarizes the calculated slope of the plots shown in FIG. 27 representing the measured stiffness of the balloon, sensing module, and SI- VTS for Type-A and Type-B sensors at different internal balloon pressures. Table 1 also summarizes the calculated constant a representing the rate of change of the stiffness of sensing layer using (1) and measured KBalloon, Ksensor, and Ksystem. Moreover, FIG. 28 also visually represents the change of stiffness for the Ksaiioon, Ksensor, and

with internal pressure of the balloon.

6 0.20 0.34 0.54

Type-A 8 0.26 0.35 0.44 0.35

10 0.46 0.56 0.38

6 0.26 0.28 0.15

Type-B 8 0, 3 0,42 0.45 0.2.3

10 0.51 0.56 0.38

Textural Image Output Evaluation

To characterize and evaluate the performance of the fabricated SI-VTS in creating image outputs of different objects with distinct textural patterns, we used the experimental setup shown in FIG. 29. FIG. 29 illustrates: [a] a single-row linear stage (M-UMR12.40, Newport); [b] digital force gauge (Mark-10 Series 5, Mark-10 Corporation); [c] the SI- VTS in its deflated mode; [d] the SI-VTS in its inflated mode ; [e] Raspberry® Pi 4 Model B; [f] pneumatic actuation system (Programmable Air, Crowd Supply), [g] Gelsight sensor; and [h] sandpapers used for finding the resolution of SI-VTS; and [i] seven objects used for evaluation of the SI-VTS image quality This setup uses the stiffness experimental setup together with a Raspberry Pi 4 Model B for stream- ing and recording the video for the image processing algorithm, and the pneumatic actuation system (Programmable Air, Crowd Supply) for controlling the inflation and deflation of the SI-VTS as well as measuring the pressure inside the balloon. The investigators also utilized MESUR Lite basic data acquisition software (Mark- 10 Corporation) to record the interaction forces between the SI-VTS and the objects

For the analysis of the relationship between the external force exerted on the proposed system and the resulting textural image outputs (as shown in FIG. 29), a total of seven objects were selected. These objects included a phantom representing a colorectal cancer polyp based on [9], a leaf, an M5 screw, and a Lego® block (front and back). FIG. 29 represents dimensions of these objects. The selection of these objects was based on their fine textures as well as previous studies such as [?], [30], [34], [36]. As shown in FIG. 29, the investigators also designed and 3D printed two custom-designed objects using the Form 3 printer (Formlabs Inc.) and Clear resin. These objects were used to thoroughly evaluate performance of SI-VTS on sensing different geometrical objects. The first object was constructed as a 5 5 array of cuboids with 0.2 mm cross sectional dimensions and height of 0.3 mm distanced 1.8 mm from each other. The second object comprised of three different rows with different shapes, including four square, four circles, and four equilateral triangles. Dimensions of these objects are shown in FIG. 29. FIG. 30 illustrates the textural outputs of Type-A and Type-B sensors in interaction with these objects. FIG. 30 also summarizes the corresponding interaction forces and balloon pressures to obtain these textural images.

Furthermore, to compare the performance of the fabricated SI-VTS s with a typical VTS (i.e. without an inflatable member), the investigators followed the procedure proposed in [30] and fabricated a GelSight sensor. To ensure a fair comparison between this sensor and the proposed SI-VTS, the investigators fabricated a Gelsight sensor with an identical camera (5MP Omnivision OV5647, Arducam), an array of high-power Red, Green, and Blue LEDs (XBDRED-00-0000- 000000701, XBDGRN-00-0000-000000D01, XBDBLU-00-0000- 000000202, Cree LED), and a sensing module with iden- tical rectangular shape and thickness (i.e., 40 mm 25mm 3 mm). As recommended by Yuan et al. [30], instead of using the stretchable silicone materials utilized for the fabrication of SI-VTSs, the investigators used a soft transparent platinum cure two-part silicone (P- 565, Silicones Inc.) and covered the surface of the Gelsight sensor with the matte-colored aluminum powder (AL-101, Atlantic Equipment Engineers). Moreover, the investigators poured the same silicone with the mixture of black and white pigments (Silc Pig Black, Silc Pig White, Smooth-On Inc.) to protect the sensing layer. Of note, similar to SI-VTS Type-B sensing layer, the investigators selected the lower stiffness (z.<?., 00-10 Shore Hardness) for the fabrication of the sensing layer of the Gelsight sensor to increase its sensitivity. To ensure having an identical sensing layer stiffness between the Gelsight sensor and SI-VTS Type- B, the investigators measured the stiffness of the fabricated sensing layer by a Shore 00 scale durometer (Model 1600 Dial Shore 00, Rex Gauge Company). FIG. 29 shows the fabricated Gelsight sensor used for the experiments. Also, FIG. 30 illustrates and compares textural outputs of this sensor in interaction with the used seven objects and the corresponding interaction forces during the experiments. Textural Image Resolution Evaluation

To measure the sensing resolution of the SI-VTS Type- A and Type-B, the investigators used the setup shown in FIG. 29 and utilized sandpapers (YXYL-0006, YXYL) with known Grit grades. Of note, the grit number refers to the size of abrasive particles present on the sandpaper. A higher grit number signifies a smoother finish, while a lower grit number indicates a coarser finish. For our experiments, the investigators utilized 10 mm 10mm pieces of sandpapers with the following grit grades and their corresponding abrasive particle dimensions inside the parentheses: a 150 grit (89 microns), 180 (76 microns), 220 grit (63 microns), and 240 grit (53.5 microns). It is worth mentioning that the investigators found this grit range, and particularly the larger grit number, after performing a few preliminary tests and checking the quality of obtained images. To perform the experiments, the investigators followed the procedure performed in the Textural Image Output Evaluation section and collected the SI-VTSs image outputs and their corresponding balloon internal pressure and interaction forces. FIG. 31 summarizes the results of these experiments.

Conformability and Self-actuation Evaluation

To validate the proposed working principle and tactile sensing procedure as well as the conformablity and self-actuation of the proposed SI-VTS in inspection of a confined and constrained environment (i.e., a pipe), the investigators first integrated the fabricated sensor with a colonoscope (PENTAX® EC 3840 L). The investigators then conducted a set of experiments inside a transparent round acrylic tube (Meccanixity® Inc.) with 200 mm length and inner diameter of 86 mm, connected with a semi-transparent acrylic 90 degree elbow (Jumpanny ™ Inc.) with 89 mm inner diameter and identical height and width of 145 mm (see FIG. 32). The inflation sequences of the SI-VTS inside this constrained environment are presented in FIG. 32, clearly showing the successful operation of the sensor inside the pipe and elbow. In this figure, each row displays the inflation sequences of the SI-VTS while inspecting and conforming with a segment of the pipe and elbow.

Discussion

FIG. 27 demonstrates the results of the stiffness experiments performed on the balloon, sensing module Type-A and Type-B, and the corresponding SI-VTSs. As can clearly be observed from these plots, independent of the type of the used silicone 29aterials (i.e., Ecoflex® 00-31 Near Clear or Ecoflex® 00-10), the relationship between the exerted force and displacements is linear. More importantly, this relationship stays linear at different internal balloon pressures. Also, based on equation (1), the slope of these plots demonstrates the stiffness of the balloon, sensing module, and the SI-VTS. As can be observed from FIG. 28, stiffness of the balloon and the fabricated Type-A and Type-B SI-VTSs almost linearly increases due to the increase of the internal pressure. Since the investigators only measured the stiffness of the sensing module Type-A and Type-B in a deflated condition (see FIG. 26), the stiffness of these layers are constant. As expected and summarized in Table I, due to the use of a stiffer silicone (i.e., Ecoflex® 00-31 Near Clear), stiffness of Type-A is twice of Type-B (i.e., 0.26 N/mm versus 0.13 N/mm). It is worth emphasizing that during inflation, the original thickness of the sensing layer (i.e., 3 mm) is reduced and, therefore, the measured stiffness of sensing module is changing with the pressure. As described in the section discussing modeling, the investigators modeled this stiffness variation with parameter a as a constant denoting the rate of change in the stiffness of sensing module. Table I summarizes the calculated values for this constant for Type-A and Type-B sensors in different pressures using the proposed model in equation (1). Investigation of FIG. 26, FIG. 28, and Table I shows that at 10 kPa internal pressure, regardless of the type of constituting material for the sensing layer, the stiffness of both SI-VTSs are identical (i.e., 0.56 N/mm). This can be attributed to the fact that at high internal pressures, the stiffness of the balloon mechanism becomes dominant determining the overall stiffness of the SI-VTS (see Table I). This analysis clarifies that at high pressures, SI- VTSs with sensing layers made of different silicone materials can perform similarly regardless of their distinct stiffness at their deflated states. Nevertheless, aside from similar stiffness properties and as shown in FIG. 30, these sensors may create different textural outputs due to their different levels of transparency.

Investigation of FIG. 30 and FIG. 31 indicates the following observations: (1) Regardless of the interacting object, one can rank the quality of the output images of the used sensors at similar interaction forces as: Type-A> Type-B> Gelsight. Of note, this can clearly be correlated to the transparency level of the used silicone materials for Type-A and Type-B sensors in which Type-A is made of a near-clear silicone whereas Type-B uses a translucent silicone. Therefore, the images obtained from a transparent silicon is expected to have a better quality and be more clear. Also, as reported in various literature (e.g., [30], [39]) typically Gelsight sensor generates a more clear image at higher interaction forces than the one studied in our experiments; (2) Compared with Type-B, Type-A sensor requires more internal pressure to create similar interaction force between the SI-VTS and the object and, subsequently, equal interaction surface area and image quality. This higher internal pressure clearly corresponds with the results shown in FIG. 28 and Table I, demonstrating a higher stiffness for Type- A sensor. Notably, the choice of selecting a stiffer but more transparent silicone- demanding more internal balloon pressure, but resulting better quality textural images- depends on the target application of SI-VTS. For example, in medical applications such as colonoscopy, the interaction force and pressure of pneumatic medical devices are typically limited to <13.5 N [40] and <7.6 kPa [41], respectively. Nevertheless, for industrial application (such as pipe inspection) the interaction force and pressure can be much higher; (3) Aside from quality and sharpness of images, the performed experiments clearly demonstrate the high- sensitivity of both Type-A and Type-B sensors compared with Gelsight sensor. As can be observed, both SI- VTSs, regardless of the interacting object, create very visible and detailed textural images at low interaction forces > 2.6 N. Of note, after this critical interaction force, the textural details of the objects do not dramatically change with the increase of force/pressure; (4) The performed experiments with sandpaper quantitatively demonstrated the performance of both SI-VTS s in creating high-resolution textural images as small as 53 pm. This high-resolution is also clearly visible in the performed experiments on the Lego® Block (i.e., object 6) in which the word ''LEGO"® on the surface of the block is clearly visible in the obtained images after > 2.6 N interaction force; and (5) All of the above-mentioned points indicatesthat the utilized soft and inflatable balloon mechanism in SI- VTS can successfully replace the used rigid plexiglass and structure in typical VTSs in creating high-quality textural images of different objects. Moreover, it also indicates the functionality of the proposed soft structure and fabrication procedure for the SLVTS.

The performed experiments displayed in FIG. 32 demonstrates the self-actuation and conformability of the SI-VTS in constrained environment and its application for inspection of confined and asymmetric cavities. Particularly, second, third, and fourth rows of FIG. 32 illustrate sequences in which SLVTS is interacting and conforming with an asymmetric geometry of the entrance and the middle of 90 degree elbow. As mentioned before, this important characteristic is attributed to the inflatable and soft balloon mechanism of SI-VTS. This conformability is very visible in sequences (o)-(s) in the third row and (v)-(y) of the fourth row in FIG. 32.

Accordingly, embodiments of the present disclosure include an inflatable and soft VTSs that benefits from a balloon mechanism and flexible sensing module enabling unique features such as self-actuation, miniaturization ability, and conformability for this sensor. Additionally, the proposed SI-VTS can readily be integrated with the existing inspection devices/robots (e.g., industrial or clinical endoscopes) to enable inspection of confined and constrained environments with complex geometries. Various experiments on two distinct typesof SI-VTSs were performed to characterize the impact of (i) material and stiffness properties of the constituting materials for the balloon mechanism and the sensing module, and (ii) balloon internal pressure and interaction force on the performance of these sensors. The textural image outputs, resolution, and sensitivity of these sensors were also evaluated in interaction with 8 different objects and under different interaction forces and internal pressures. Results demonstrated that regardless of the type of the utilized silicone for the sensing module, very visible and detailed textural images (with 53 pm resolution) can be created at low interaction forces > 2.6 N. Nevertheless, the sharpness and clarity of obtained images directly depended on the transparency level of the utilized silicone. Also, conformability and self- actuation of the proposed sensorwere successfully verified by integrating the sensor with a colonoscope and performing various experiments inside a pipeconnected with a 90 degree elbow

All of the apparatus, systems and/or methods disclosed and claimed herein can be made and executed without undue experimentation in light of the present disclosure. While the devices, systems and methods of this invention have been described in terms of particular embodiments, it will be apparent to those of skill in the art that variations may be applied to the devices, systems and/or methods in the steps or in the sequence of steps of the method described herein without departing from the concept, spirit and scope of the invention. All such similar substitutes and modifications apparent to those skilled in the art are deemed to be within the spirit, scope and concept of the invention as defined by the appended claims.

References: The contents of the following references are incorporated by reference herein:

[1] Dahiya, R., Valle, M. & Others Tactile sensing for robotic applications.

[2] Sensors, Focus On Tactile, Force And Stress Sensors. Pp. 298-304 (2008) [2]Won, C., Lee, J. & Saleheen, F. Tactile sensing systems for tumor characterization: A review. IEEE Sensors Journal. 21, 12578-12588(2021)

[3] Nguyen, T. & Lee, J. Recent development of flexible tactile sensors andtheir applications. Sensors. 22, 50 (2022)

[4] Zhang, S., Chen, Z., Gao Y., Wan, W Shan J Xue, H., Sun, F., Yang, Y. & Fang, B. Hardware Technology of Vision-Based Tactile Sensor: AReview. IEEE Sensors Journal. (2022)

[5] Zou, L., Ge, C., Wang, Z., Cretu, E. & Li, X. Novel tactile sensor technology and smart tactile sensing systems: A review. Sensors. 17, 2653(2017)

[6] Yamaguchi, A. & Atkeson, C. Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?. Advanced Robotics. 33, 661-673 (2019)

[7] Gabler, V., Huber, G. & Wollherr, D. A Force-Sensitive Grasping Con- troller Using Tactile Gripper Fingers and an Industrial Position-ControllcdRobot. 2022 International Conference On Robotics And Automation(ICRA) . Pp. 770-776 (2022)

[8] Li, G., Liu, S., Wang, L. & Zhu, R. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Science Robotics.5, eabc8134 (2020)

[9] Kara, O., Ikoma, N. & Alambeigi, F. HySenSe: A Hyper-Sensitive and High-Fidelity Vision-Based Tactile Sensor. 2022 IEEE Sensors. Pp. 1-4(2022)

[10] Venkatayogi, N., Kara, O., Bonyun, J., Ikoma, N. & Alambeigi, F. Classification of Colorectal Cancer Polyps via Transfer Learning and Vision-Based Tactile Sensing. 2022 IEEE Sensors. Pp. 1-4 (2022)

[11] Chu, Z., Sarro, P. & Middelhoek, S. Silicon three-axial tactile sensor. Sensors And Actuators A: Physical. 54, 505-510 (1996)

[12] ]Leineweber, M., Pelz G., Schmidt, M., Kappert, H. & Zimmer, G. New tactile sensor chip with silicone rubber cover. Sensors And Actuators A: Physical. 84, 236-245 (2000)

[13] Morimura, H., Shigematsu, S. & Machida, K. A novel sensor cell architecture and sensing circuit scheme for capacitive fingerprint sensors. IEEE Journal Of Solid-State Circuits. 35, 724-731 (2000)

[14] Naidu, A., Patel, R. & Naish, M. Low-cost disposable tactile sensors for palpation in minimally invasive surgery. IEEE/ASME Transactions On Mechatronics. 22, 127-137 (2016)

[15] Kolesar, E. & Dyson, C. Object imaging with a piezoelectric robotic tactile sensor. Journal Of Microelectromechanical Systems. 4, 87-96(1995)

[16] Samaun, S., Wise, K., Nielsen, E. & Angell, J. An IC piezoresistive pressure sensor for biomedical instrumentation. 1971 IEEE International Solid-State Circuits Conference. Digest Of Technical Papers. 14 pp. 104-105 (1971)

[17] Kong, D., Yang, G., Pang, G., Ye, Z., Lv, H., Yu, Z., Wang, F., Wang, X., Xu, K.

& Yang, H. Bioinspired Co-Design of Tactile Sensor and Deep Learning Algorithm for Human-Robot Interaction. Advanced Intelligent Systems. 4, 2200050 (2022)

[18] De Rossi, D., Domenici, C. & Pastacaldi, P. Piezoelectric properties of dry human skin. IEEE Transactions On Electrical Insulation., 511-517(1986)

[19] Zhang, J., Yao, H., Mo, J., Chen, S Xie, Y., Ma S., Chen, R., Luo, T., Ling, W., Qin, L. & Others Finger-inspired rigid-soft hybrid tactile sensorwith superior sensitivity at high frequency. Nature Communications. 13, 5076 (2022)

[20] Mohammadi, A., Xu, Y., Tan, Y., Choong, P. & Octomo, D. Magnetic- based soft tactile sensors with deformable continuous force transfer medium for resolving contact locations in robotic grasping and manipu-lation. Sensors. 19, 4925 (2019)

[21] Katz, M. Introduction to geometrical optics. (World scientific, 2002) [22]Guo, J., Shang, C., Gao, S., Zhang, Y., Fu, B. & Xu, L. Flexible

[22] Plasmonic Optical Tactile Sensor for Health Monitoring and Artificial Haptic Perception. Advanced Materials Technologies. Pp. 2201506 (2023)

33

4886-9714-7514, v. 1 [23] Dargahi, J., Paramcswaran, M. & Payandch, S. A micromachincd piezoelectric tactile sensor for an endoscopic grasper-theory, fabrication and experiments. Journal Of Microelectromechanical Systems. 9, 329-335(2000)

[24] Krishna, G. & Rajanna, K. Tactile sensor based on piezoelectric reso-nance. IEEE Sensors Journal. 4, 691-697 (2004)

[25] Schaffner, F., Wellscheid, R. & Jungnickel, B. The hydrostatic piezoelec- trie coefficient of pvdf/pmma blends. IEEE Transactions On Electrical Insulation. 26, 78-84 (1991)

[26] Cheung, E. & Lumelsky, V. A sensitive skin system for motion control of robot arm manipulators. Robotics And Autonomous Systems. 10, 9-32(1992)

[27] Johnson, Micah K., and Edward H. Adelson. Retrographic sensing for the measurement of surface texture and shape. 2009 IEEE Conference onComputer Vision and Pattern Recognition. Pp. 1070-1077 (2009)

[28] Li, R., & Adelson, E. H. Sensing and recognizing surface textures using a gelsight sensor. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Pp. 1241-1247 (2013)

[29] Dong, Siyuan, Wenzhen Yuan, and Edward H. Adelson. Improved gelsight tactile sensor for measuring geometry and slip. 2017 IEEE/RS J International Conference on Intelligent Robots and Systems. (2017)

[30] Yuan, W., Dong, S. & Adelson, E. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors. 17, 2762 (2017)

[31] Li, R., Platt, R., Yuan, W., ten Pas, A., Roscup, N., Srinivasan, M.A. and Adelson, E. Localization and manipulation of small parts using gelsighttactile sensing. 2014 1EEE/RSJ International Conference on Intelligent Robots and Systems. Pp. 3988-3993 (2014)

[32] Yuan, W., Zhu, C., Owens, A., Srinivasan, M.A. and Adelson, E.H. Shapeindependent hardness estimation using deep learning and a gelsighttactile sensor. 2017 IEEE International Conference on Robotics and Automation. Pp. 951-958 (2017)

[33] She, Y., Liu, S., Yu, P. & Adelson, E. Exoskeleton-covered soft finger with vision-based proprioception and tactile sensing. 2020 IEEE Interna- tional Conference On Robotics And Automation (ICRA). Pp. 10075-10081 (2020)

[34] Romero, B., Vciga, F. & Adelson, E. Soft, round, high resolution tactile fingertip sensors for dexterous robotic manipulation. 2020 lEEEInternational Conference On Robotics And Automation (ICRA). Pp. 4796-4802 (2020)

[35] Donlon, E., Dong, S., Liu, M., Li, J., Adelson, E. & Rodrigucz, A.Gclslim: A high- resolution, compact, robust, and calibrated tactile-sensing finger. 2018 IEEE/RS J International Conference On Intelligent RobotsAnd Systems (IROS). Pp. 1927-1934 (2018)

[36] Gomes, D., Lin, Z. & Luo, S. GelTip: A finger-shaped optical tactile sensor for robotic manipulation. 2020 IEEE/RS J International Conference On Intelligent Robots And Systems (IROS). Pp. 9903-9909 (2020)

[37] Ward-Cherrier, B., Pestell, N., Cramphorn, L., Winstone, B., Giannac- cini, M., Rossiter, J. & Lepora, N. The tactip family: Soft optical tactile sensors with 3d-printed biomimetic morphologies. Soft Robotics. 5, 216-227 (2018)

[38] Kara, O. C., Kim, H., Ikoma, N., & Alambeigi, F. Design and De- velopment of a Novel Soft and Inflatable Tactile Sensin Balloon forEarly Diagnosis of Colorectal Cancer Polyps. To be appeared in 2023IEEE/RSJ International Conference On Intelligent Robots And Sy stems (IROS), (2023).

[39] Jia, X., Li, R., Srinivasan, M.A. and Adelson, E.H. Lump detection with a gelsight sensor.2013 World Haptics Conference (WHC). Pp. 175-179(2013)

[40] Heijnsdijk, E.A.M., Van Der Voort, M., De Visser, H., Dankelman, J. andGouma, D.J. Inter-and intraindividual variabilities of perforation forces ofhuman and pig bowel tissue. Surgical Endoscopy and other interventionalTechniques. 17, 1923-1926 (2003)

[41] Kozarek, R., Earnest, D., Silverstein, M. & Smith, R. Air-pressure- induced colon injury during diagnostic colonoscopy. Gastroenterology.1 , 7-14 (1980) U.S. Pat. Pub 2004/102733

U.S. Pat. Pub. 20070151390

U.S. Pat. Pub. 2008/0009746

U.S. Pat. Pub. 2016/0044218

U.S. Pat. Pub. 2021/0318116

U.S. Pat. 7,390,408

U.S. Pat. 9,127,938

U.S. Pat. 9,538,056

U.S. Pat. 10,186,546

U.S. Pat. RE44856

PCT Pat. Pub. WO2019084162

Han, M., Chen, L., Aras, K. et al. Catheter-integrated soft multilayer electronic arrays for multiplexed sensing and actuation during cardiac surgery. Nat Biomed Eng (2020). https ://doi.org/l 0.1038/s41551 -020-00604-w.

Claims

CLAIMS:

1. A flexible tactile sensing apparatus comprising: a housing; an inflatable member coupled to the housing; an imaging device; a lumen in fluid communication with the inflatable member; and a sensor.

2. The flexible tactile sensing apparatus of claim 1 further comprising a gel layer coupled to the inflatable member.

3. The flexible tactile sensing apparatus of claim 2 wherein the gel layer is translucent or transparent.

4. The flexible tactile sensing apparatus of claim 2 or claim 3 wherein the gel layer is between 3.0 and 5.0 millimeters thick.

5. The flexible tactile sensing apparatus of any one of claims 2-4 wherein the gel layer comprises a silicone material.

6. The flexible tactile sensing apparatus of any one of claims 2-5 wherein: the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member.

7. The flexible tactile sensing apparatus of claim 6 wherein the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member.

8. The flexible tactile sensing apparatus of claim 6 or claim 7 wherein the plurality of sensors is configured to obtain data simultaneously in one-step. he flexible tactile sensing apparatus of claim 6 or claim 7 wherein the plurality of sensors is configured to obtain data sequentially. The tactile sensing apparatus of any one of claims 1-9 wherein the housing is coupled to an elongated member. The tactile sensing apparatus of claim 10 wherein: the elongated member comprises a distal end with an axial portion; and the housing is coupled to the axial portion of the distal end of the elongated member. The tactile sensing apparatus of claim 11 wherein the inflatable member is configured to expand in a direction along a primary axis of the elongated member. The tactile sensing apparatus of claim 10 wherein: the elongated member comprises a distal end; and the housing is coupled to the distal end of the elongated member such that the housing extends circumferentially around the distal end of the elongated member. The tactile sensing apparatus of claim 13 wherein the inflatable member is configured to primarily expand radially outward from the elongated member. The flexible tactile sensing apparatus of any one of claims 1-14 further comprising an illumination device. The flexible tactile sensing apparatus of claim 1 or claim 15 wherein the housing is an annular member comprising a central opening configured to receive an elongated member. The flexible tactile sensing apparatus of claim 16 wherein the elongated member is a medical scope. The flexible tactile sensing apparatus of claim 17 wherein the medical scope is a colonoscope or an endoscope. The flexible tactile sensing apparatus of claim 16 wherein the elongated member is a borescope. The flexible tactile sensing apparatus of claim 1 or claim 15 wherein the housing is approximately planar. The flexible tactile sensing apparatus of claim 1 or claim 15 wherein the housing is flexible. The flexible tactile sensing apparatus of claim 21 wherein the housing is configured to couple to a robotic hand. The flexible tactile sensing apparatus of any one of claims 1-20 wherein the sensor is configured to measure a pressure within the inflatable member. The flexible tactile sensing apparatus of any one of claims 1-23 wherein the inflatable member is asymmetrical. The flexible tactile sensing apparatus of claim 24 wherein the inflatable member comprises an outer surface with extended sections and recessed sections. The flexible tactile sensing apparatus of claim 25 wherein the extended sections and recessed sections form a flower or star pattern. The flexible tactile sensing apparatus of any one of claims 1-24 wherein the inflatable member is symmetrical. The flexible tactile sensing apparatus of claim 27 wherein the inflatable member is configured in a ring shape. The flexible tactile sensing apparatus of claim 27 wherein the wherein the inflatable member is configured in a hemispherical shape. The flexible tactile sensing apparatus of any one of claims 1-27 wherein the inflatable member is formed from a single material. The flexible tactile sensing apparatus of claim 30 wherein the single material is a silicone material. The flexible tactile sensing apparatus of any one of claims 1-27 wherein the inflatable member is formed from a plurality of materials. The flexible tactile sensing apparatus of claim 32 wherein the plurality of materials comprises at least one silicone material. The flexible tactile sensing apparatus of claim 32 or claim 33 wherein the plurality of materials comprises a gel. The flexible tactile sensing apparatus of claim 32 wherein the plurality of materials comprises a first silicone material and a second silicone material. The flexible tactile sensing apparatus of any one of claims 33-35 wherein the at least one silicone material is coated. The flexible tactile sensing apparatus of 23 wherein the at least one silicone material is coated with a reflective material. The flexible tactile sensing apparatus of 24 wherein the reflective material comprises a reflective aluminum paint. The flexible tactile sensing apparatus of 23 wherein the at least one silicone material is coated with a pigment. The flexible tactile sensing apparatus of 26 wherein the pigment comprises an aluminum powder.

41. The flexible tactile sensing apparatus of any one of claims 1-35 wherein the inflatable member has a thickness of 0.1 millimeters to 5.0 millimeters.

42. The flexible tactile sensing apparatus of any one of claims 15-27 wherein the illumination device comprises a light emitting diode (LED) or optical fibers.

43. The flexible tactile sensing apparatus of claim 42 wherein the LED comprises a red LED, a blue LED, a green LED, a white LED and/or an ultraviolet LED.

44. The flexible tactile sensing apparatus of any one of claims 1-43 wherein the imaging device is configured to provide an endoscopic image.

45. The flexible tactile sensing apparatus of any one of claims 1-44 wherein the imaging device is configured to provide a radial image.

46. The flexible tactile sensing apparatus of any one of claims 1-45 further comprising a computer readable medium containing program instructions for topographic mapping classification of a surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of:

(a) obtaining imaging data of the surface; and

(b) analyzing the imaging data of the surface.

47. The flexible tactile sensing apparatus of claim 46 wherein the computer readable medium contains program instructions for stiffness classification of the surface, wherein execution of the program instructions by one or more processors of the computer system causes the one or more processors to carry out the steps of:

(c) obtaining pressure data from the sensor; and

(d) analyzing the pressure data from the sensor.

48. The flexible tactile sensing apparatus of claim 47 wherein analyzing the pressure data from the sensor comprises comparing the pressure within the inflatable member to a parameter of the inflatable member. The flexible tactile sensing apparatus of claim 48 wherein the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface. The flexible tactile sensing apparatus of claim 48 wherein the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member. The flexible tactile sensing apparatus of claim 48 wherein the parameter is a volume within the inflatable member. The flexible tactile sensing apparatus of any one of claims 46-51 wherein analyzing the imaging data of the surface comprises analyzing a contrast of the surface. The flexible tactile sensing apparatus of any one of claims 46-52 wherein analyzing the imaging data of the surface comprises analyzing a color of the surface. The flexible tactile sensing apparatus of any one of claims 46-53 wherein analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface. The flexible tactile sensing apparatus of any one of claims 46-54 wherein obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. The flexible tactile sensing apparatus of claim 55 wherein the biological tissue is gastrointestinal (GI) tract tissue. The flexible tactile sensing apparatus of claim 55 wherein the biological tissue is arterial tissue. The flexible tactile sensing apparatus of any one of claims 46-54 wherein the surface is an interior surface of a pipe. The flexible tactile sensing apparatus of claim 58 wherein the pipe is configured to transmit a hydrocarbon. The flexible tactile sensing apparatus of any one of claims 46-54 wherein the surface is accessed by a mobile robot. The flexible tactile sensing apparatus of any one of claims 46-54 wherein the surface is contacted by a robotic hand. The flexible tactile sensing apparatus of any one of claims 46-61 wherein the imaging data comprises shape, size, and texture data. The flexible tactile sensing apparatus of any one of claims 1-62 further comprising a sound detector. The flexible tactile sensing apparatus of any one of claims 1-63 further comprising: a power supply; an air supply system; and a controller, wherein the controller is configured to control the air supply system. A method of analyzing a surface, the method comprising:

(a) positioning an apparatus proximal to a surface, wherein the apparatus comprises: a housing; an inflatable member coupled to the housing; an imaging device; a lumen in fluid communication with the inflatable member; and a sensor;

(b) inflating the inflatable member coupled to the housing;

(c) obtaining imaging data of the surface with the imaging device; and

(d) analyzing the imaging data of the surface. The method of claim 65 wherein: step (a) comprises positioning a plurality of apparatus proximal to the surface, wherein each apparatus in the plurality of apparatus comprises: a housing; an inflatable member coupled to the housing; an imaging device; a lumen in fluid communication with the inflatable member; and a sensor; step (b) comprises inflating each inflatable member; step (c) comprises obtaining imaging data from the surface from each of the imaging devices; and step (d) comprises analyzing the imaging data of the surface. The method of claim 65 wherein: the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member. The method of claim 67 wherein the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member. The method of any one of claims 66 - 68 wherein step (c) comprises obtaining imaging data of the surface simultaneously in one-step with the plurality of imaging devices. The method of any one of claims 66 - 68 wherein step (c) comprises obtaining imaging data of the surface sequentially with the plurality of imaging devices. The method of any one of claims 65 - claim 69 wherein the method further comprises:

(e) obtaining pressure data of the inflatable member from the sensor; and

(f) analyzing the pressure data of the inflatable member. The method of any one of claims 65 - 71 further comprising:

(g) deflating the inflatable member. The method of any one of claims 66-72 further comprising:

(h) moving the plurality of apparatus relative to the surface; and

(i) repeating steps (a)-(c). The method of any one of claims 65-73 wherein the apparatus proximal to the surface comprises a computer readable medium containing program instructions for topographic mapping and stiffness classification of the surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of:

(c) obtaining imaging data of the surface with the imaging device; and id) analyzing the imaging data of the surface. The method of claim 74 wherein the computer readable medium contains program instructions for topographic mapping and stiffness classification of the surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out steps of:

(e) obtaining pressure data of the inflatable member from the sensor; and

(f) analyzing the pressure data of the inflatable member. The method of any one of claims 65-74 wherein analyzing the imaging data of the surface comprises calculating the texture, size, shape, or stiffness of the surface. The method of any one of claims 71-76 wherein analyzing the pressure data of the inflatable member comprises calculating the stiffness of the surface. The method of any one of claims 71-77 wherein analyzing the pressure data of the surface comprises comparing the pressure within the inflatable member to a parameter of the inflatable member. The method of claim 78 wherein the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface. The method of claim 78 wherein the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member. The method of claim 80 wherein the plurality of points on the inflatable member are marked with a plurality of fiducial markers. The method of claim 80 wherein the plurality of fiducial markers comprises fiducial markers of different sizes. The method of claim 78 wherein the parameter is a volume within the inflatable member. The method of any one of claims 65-83 wherein analyzing the imaging data of the surface comprises analyzing a contrast of the surface. The method of any one of any one of claims 65-84 wherein analyzing the imaging data of the surface comprises analyzing a color of the surface The method of any one of any one of claims 65-85 wherein analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface. The method of any one of any one of claims 65-86 wherein obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. The method of claim 87 wherein the biological tissue is gastrointestinal (GI) tract tissue. The method of claim 88 wherein the biological tissue is arterial tissue. The method of any one of claims 65-89 wherein the surface is an interior surface of a pipe. The method of claim 90 wherein the pipe is configured to transmit a hydrocarbon. The method of any one of claims 65-91 wherein the imaging data comprises shape, size, and texture data. The method of any one of claims 65-92 further comprising detecting sound. An apparatus comprising: an annular member comprising a central opening configured to receive a medical scope; an illumination device; an imaging device; an inflatable member coupled to the annular member; a lumen in fluid communication with the annular member; and a sensor configured to measure a pressure within the inflatable member. The apparatus of claim 94 further comprising a gel layer coupled to the inflatable member. The apparatus of claim 95 wherein the gel layer is translucent or transparent. The apparatus of claim 95 or claim 96 wherein the gel layer is between 3.0 and 5.0 millimeters thick. The apparatus of any one of claims 95-97 wherein the gel layer comprises a silicone material. The apparatus of any one of claims 94-98 wherein: the illumination device is a first illumination device in a plurality of illumination devices; the imaging device is a first imaging device in a plurality of imaging devices; the sensor is a first sensor in a plurality of sensors; the plurality of illumination devices is contained within the inflatable member; the plurality of imaging devices is contained within the inflatable member; and the plurality of sensors is contained within the inflatable member. . The apparatus of claim 99 wherein the plurality of illumination devices, the plurality of sensors and the plurality of imaging devices are spaced axially along the inflatable member. . The apparatus of claim 99 or claim 100 wherein the plurality of illumination devices is configured to obtain data simultaneously in one-step.

102. The apparatus of claim 99 or claim 100 wherein the plurality of illumination devices is configured to obtain data sequentially.

103. The apparatus of any one of claim 94 - claim 100 wherein the medical scope is a colonoscope or an endoscope.

104. The apparatus of any one of claim 94 - claim 103 wherein the inflatable member is asymmetrical.

105. The apparatus of any one of claims 94-104 wherein the inflatable member is symmetrical.

106. The apparatus of any one of claims 94-105 wherein the inflatable member is formed from a single material.

107. The apparatus of claim 106 wherein the single material is a silicone material.

108. The apparatus of any one of claims 94-105 wherein the inflatable member is formed from a plurality of materials.

109. The apparatus of claim 108 wherein the plurality of materials comprises at least one silicone material.

110. The apparatus of claim 108 wherein the plurality of materials comprises a first silicone material and a second silicone material.

111. The apparatus of any one of claims 94- 110 wherein the inflatable member has a thickness of 0.1 millimeters to 5.0 millimeters.

112. The apparatus of any one of claims 94-111 wherein the illumination device comprises a light emitting diode (LED).

113. The apparatus of claim 112 wherein the LED comprises a red LED, a blue LED, a green LED, a white LED or an ultraviolet LED.

114. The apparatus of any one of claims 94-113 wherein the imaging device is configured to provide an endoscopic image.

115. The apparatus of any one of claims 94-114 wherein the imaging device is configured to provide a radial image.

116. The apparatus of any one of claims 94-115 further comprising a computer readable medium containing program instructions for topographic mapping classification of a surface, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the steps of:

(a) obtaining imaging data of the surface; and

(b) analyzing the imaging data of the surface.

117. The apparatus of any one of claims 94-116 wherein the computer readable medium contains program instructions for stiffness classification of the surface, wherein execution of the program instructions by one or more processors of the computer system causes the one or more processors to carry out the steps of:

(c) obtaining pressure data from the sensor; and

(d) analyzing the pressure data from the sensor.

118. The apparatus of claim 117 wherein analyzing the pressure data from the sensor comprises comparing the pressure within the inflatable member to a parameter of the inflatable member.

119. The apparatus of claim 118 wherein the parameter is a distance and/or an orientation from a reference point to an interface between the inflatable member and the surface.

120. The apparatus of claim 118 wherein the parameter is a distance and/or an orientation from a reference point a plurality of points on the inflatable member.

121. The apparatus of claim 118 wherein the parameter is a volume within the inflatable member.

. The apparatus of any one of claims 116-121 wherein analyzing the imaging data of the surface comprises analyzing a contrast of the surface. . The apparatus of any one of claims 116-122 wherein analyzing the imaging data of the surface comprises analyzing a color of the surface. . The apparatus of any one of claims 116-123 wherein analyzing the imaging data of the surface comprises analyzing a shape, size, stiffness, and/or texture of the surface. . The apparatus of any one of claims 116-124 wherein obtaining imaging data of the surface comprises obtaining imaging data of a biological tissue. . The apparatus of claim 125 wherein the biological tissue is gastrointestinal (GI) tract tissue. . The apparatus of claim 56 wherein the biological tissue is arterial tissue. . The apparatus of any one of claims 116-127 wherein the imaging data comprises shape, size, and texture data. . The apparatus of any one of claims 94-128 further comprising a sound detector. . A method of analyzing a surface, the method comprising:

(a) positioning an apparatus proximal to a surface, wherein the apparatus comprises: a housing; an inflatable member coupled to the housing; a plurality of sensors contained within the inflatable member; and an imaging device;

(b) inflating the inflatable member coupled to the housing; and

(c) obtaining data of the surface with the plurality of sensors. . The method of claim 130 wherein the plurality of sensors is spaced axially along the inflatable member.

. The method of claim 130 or claim 131 wherein obtaining data of the surface with the plurality of sensors comprises obtaining data simultaneously in one-step with the plurality of sensors. . The method of claim 130 or claim 131 wherein obtaining data of the surface with the plurality of sensors comprises obtaining data sequentially with the plurality of sensors.