DE112020005495T5 - Combination of ultrasound and endoscopy - Google Patents

Abstract

A combined ultrasound and endoscopy system (100) includes a cannula (240) having a distal tip (250) configured for insertion into an internal organ or other internal body structure. The distal tip (250) is configured to include both an ultrasound probe head (252) and a camera module (320). The ultrasound and direct view endoscope images (154, 156) can be simultaneously displayed to a user on a display monitor (150, 152). The ultrasound probe head (252) can be rotated and steered to scan any location within the lumen of a human organ. The ultrasonic probe (410) can be intended for multiple or single use. The system (100) can be configured with a handheld portion (110) having a reusable handle portion (130) and a single-use portion (120) configured for disposal after a single use. The system (100) can also be configured with a conventional reusable endoscope with working channels (920, 1020) and an endoscopy processing tower system (112, 116, 118).

Description

REFERENCE TO RELATED APPLICATION

This patent application claims benefit of, and is incorporated by reference, the following provisional application: 62/933,216 filed November 8, 2019.

AREA

The present specification relates generally to a medical device for use in tissue and organ examination. In particular, some embodiments relate to combined ultrasound and endoscopy systems for examining internal organs and other internal body structures.

BACKGROUND

Ultrasound is sound waves with frequencies higher than that which humans can hear (>20,000 Hz). Ultrasound images, also called sonograms, are created by sending pulses of ultrasound into tissue with a probe. The ultrasound pulses are reflected by tissues with different reflection properties, recorded and displayed as an image. Medical ultrasound is commonly used to create images of internal organs as well as other body structures such as tendons, muscles, joints and blood vessels. Although medical ultrasound typically uses transducers designed for external application, e.g., through the lower abdominal wall in the case of gynecologic ultrasound examination, the ultrasound transducer is occasionally configured for insertion into an internal organ or other structure. One such example is the sonohysterogram technique used to visualize the uterus. In this procedure, fluid and an ultrasound probe are inserted into the uterus, and sonographic images of the uterine structures can be obtained. Although the sonohysterogram procedure may be preceded by an endoscopy procedure to obtain images of the uterine walls under direct vision, a sonohysterogram procedure is traditionally performed “blindly” or without any live visual aids during ultrasound probe insertion.

SUMMARY

According to some embodiments, an integrated vision and ultrasound device includes: an ergonomic handle configured for hand gripping and having proximal and distal portions, a cannula extending distally from the distal portion of the handle, and a distal portion having extending along a longitudinal axis; a distally facing camera attached to the distal portion of the cannula and having a camera field of view (FOV) comprising a selected solid angle and a camera line of sight (DOV) angled relative to said axis; an ultrasound probe mounted on the distal portion of the cannula for both rotation about said axis relative to the distal portion of the cannula and for tilting relative to said axis; a probe steering mechanism mounted on the proximal end of the handle and operatively coupled to the ultrasound probe for selectively tilting the ultrasound probe relative to said axis through a selected angular range; and a probe rotation mechanism mounted on the proximal end of the handle and operatively coupled to the ultrasound probe for selectively rotating the ultrasound probe about said axis relative to the cannula.

According to some embodiments, said integrated vision and ultrasound device may also include one or more of the following features: the cannula may include at least one lumen, and may further include a shaft connecting the probe steering mechanism and the ultrasound probe, said shaft being removably engageable in said lumen is included; a belt located in said shaft can be coupled to and driven by the probe rotation mechanism, and a gear box can be attached to said ultrasound probe and driven by said belt to selectively rotate the ultrasound probe about said axis; the probe rotation mechanism may be configured to rotate the ultrasound probe at least 180 degrees; the ultrasound probe is attachable to a rotary plate which rotates about a pivot axis transverse to said longitudinal axis, and a rod inside said shaft can couple said probe steering mechanism to said rotary plate and respond to rotation of the steering mechanism about the rotary plate and thereby pivoting the ultrasound probe relative to said longitudinal axis; said steering mechanism is configurable to tilt said ultrasound probe in two opposite directions relative to said longitudinal axis through an angle of up to 180 degrees in at least one of said directions; said steering mechanism is configurable for tilting said ultrasound probe through different angular ranges in said two opposite directions relative to said longitudinal axis; said handle may comprise: (i) a reusable portion and imaging electronics therein coupled to said camera and said ultrasound probe, and (ii) a disposable portion removably attached to said reusable portion and housing said rotating mechanism and steering mechanism; said cannula may be flexible so that it flexes when inserted into a patient's bladder or ureter; an ultrasound image processor may be operatively coupled to said ultrasound probe, and an ultrasound image display may be configured to display ultrasound images provided by said ultrasound probe and processed by said ultrasound processor, and a camera image processor and camera image display may be configured so that they display images provided by said camera and processed by said camera image processor; said ultrasound image display and camera image display being configurable to display said ultrasound and camera images simultaneously on a single screen; the ultrasound and visual aspects can be integrated by inserting the ultrasound probe through a working channel formed in the cannula; said cannula can be configured so that at least a portion thereof has a stiffness characteristic selected from a group consisting of: rigid, semi-rigid, and flexible; the DOV can range from 0 to 30 degrees; a cannula rotation mechanism positioned at the proximal portion of the handle and operatively coupled to the cannula for selectively rotating the cannula, and thereby the camera, about said axis relative to the handle; said probe turning mechanism may be a probe turning wheel; a rotation sensor may be operatively coupled to the probe rotation mechanism and configured to provide an electronic signal indicative of rotation of the ultrasound probe about said axis; a processing system may be configured to process ultrasound images from said ultrasound probe and automatically generate three-dimensional ultrasound images therefrom based in part on said electronic signals from said rotary sensor; and said probe steering mechanism may be a probe steering wheel.

According to some embodiments, a medical device includes: an elongate shaft having a distal portion extending along a longitudinal axis and a proximal portion, the shaft being for insertion into a sheath or working channel of an endoscope cannula configured for insertion into a patient , shaped and dimensioned; an ultrasound probe located on the distal portion of the shaft and configured to protrude from a distal end of the sheath or cannula and provide ultrasound images; a housing attached to the proximal portion of the shaft; a probe rotation mechanism mounted on or in the housing and operatively coupled to the shaft for rotating the shaft, and thereby the ultrasound probe, about said axis through a selected range of angles of rotation; and a probe steering mechanism mounted on or in the housing and operatively coupled to the ultrasound probe for tilting the ultrasound probe relative to said axis through a selected range of tilt angles.

The medical device may further include one or more of the following features: a rotation sensor operatively coupled to the probe rotation mechanism and configured to provide an electronic signal indicative of rotation of the ultrasound probe about said axis; and said shaft may be shaped and dimensioned for insertion into a working channel of an endoscope having a camera at its distal end, the ultrasound probe being configured to project distally from said camera when the shaft is inserted into said working channel is.

According to some embodiments, a method includes: providing an integrated camera and ultrasound imaging device having an elongated cannula with a distal portion extending along a longitudinal axis; inserting the cannula into an object; operating a camera mounted on a distal portion of the cannula to provide camera images of an interior portion of the object taken with a camera viewing direction angled relative to said axis; operating an ultrasound probe, also mounted on the distal portion of the cannula and projecting distally from said camera, to provide ultrasound images of an interior portion of the object; selectively rotating the cannula and camera about said axis through a selected rotation angle to view internal portions of the object from different directions under manual control exercised at a proximal portion of the imaging device; selectively rotating the ultrasound probe about said axis and selectively tilting the ultrasound probe relative to said axis to obtain ultrasound images of selected portions of the object taken from different directions at a solid angle greater than 180 degrees at a proximal portion of the imaging device exercised manual control; wherein operating the camera includes: including at least a partial image of the ultrasound probe in at least some of the camera-supplied images; processing the camera and ultrasound images; and displaying the resulting processed camera images and ultrasound images.

According to some embodiments, the method may further include one or more of: detecting the rotation of the ultrasound probe and controlling the display of ultrasound images as a function of the detected rotation of the ultrasound probe; providing a handle configured to be gripped by the hand, said cannula being attached to said handle; providing a handle having a disposable grip portion to which said device is attached and a reusable grip portion releasably attached to the disposable portion and containing electronics for processing the ultrasound images from the ultrasound probe; selectively withdrawing the ultrasound probe from the cannula while the cannula remains inserted into the subject and inserting a surgical instrument into a cannula lumen vacated by the withdrawn ultrasound probe; selectively bending the cannula; wherein said rotating of the cannula occurs under manual control by rotating said integrated camera and ultrasound imaging device; wherein said selectively rotating the cannula occurs under manual control by rotating a cannula rotating mechanism operatively coupled to the cannula to selectively rotate the cannula and thereby the camera; and performing a surgical procedure on the subject.

character list

• 1A-1C schematische Diagramme, die Beispiele für kombinierte Ultraschall- und Endoskopiesysteme (CUES) gemäß einigen Ausführungsformen zeigen;
• 2A, 2B, 2C und 2D rechte, linke, obere und vordere Ansichten des handgehaltenen Abschnitts eines kombinierten Ultraschall- und Endoskopiesystems (CUES) gemäß einigen Ausführungsformen;
• 3A und 3B perspektivische Ansichten und Draufsichten des distalen Endes eines handgehaltenen Abschnitts eines kombinierten Ultraschall- und Endoskopiesystems (CUES) gemäß einigen Ausführungsformen;
• 4A und 4B perspektivische Teilansichten einer Kanüle, die Teil eines kombinierten Ultraschall- und Endoskopiesystems (CUES) ist, gemäß einigen Ausführungsformen;
• 5A, 5B und 5C perspektivische Ansichten, die weitere Details von Mechanismen zum Lenken und Drehen eines Ultraschallsondenkopfes zeigen, der Teil eines kombinierten Ultraschall- und Endoskopiesystems (CUES) ist, gemäß einigen Ausführungsformen;
• 6 eine perspektivische Ansicht, die weitere Details der distalen Spitze eines kombinierten Ultraschall- und Endoskopiesystems (CUES) gemäß einigen Ausführungsformen zeigt;
• 7 eine perspektivische Ansicht eines flexiblen handgehaltenen Abschnitts, der Teil eines kombinierten Ultraschall- und Endoskopiesystems (CUES) ist, gemäß einigen Ausführungsformen;
• 8A und 8B Diagramme, die weitere Details des trennbaren handgehaltenen Abschnitts eines kombinierten Ultraschall- und Endoskopiesystems (CUES) gemäß einigen Ausführungsformen zeigen;
• 9A-9C Diagramme, die weitere Details der distalen Spitze und Kanüle einer kombinierten Ultraschall- und Endoskopiesonde gemäß einigen Ausführungsformen zeigen;
• 10A-10C Diagramme, die weitere Details der distalen Spitze und der Kanüle einer kombinierten Ultraschall- und Endoskopiesonde gemäß einigen Ausführungsformen veranschaulichen; und
• 11A und 11B schematische Darstellungen, die weitere Beispiele für kombinierte Ultraschall- und Endoskopiesysteme (CUES) gemäß einigen Ausführungsformen zeigen.

The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description, setting forth illustrative embodiments in which principles of the invention are utilized, and the accompanying drawings. Show in it:

• 1A-1C schematic diagrams showing examples of combined ultrasound and endoscopy systems (CUES) according to some embodiments;
• 2A , 2 B , 2C and 2D right, left, top, and front views of the handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;
• 3A and 3B perspective and plan views of the distal end of a hand-held portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;
• 4A and 4B partial perspective views of a cannula that is part of a combined ultrasound and endoscopy system (CUES) according to some embodiments;
• 5A , 5B and 5C perspective views showing further details of mechanisms for steering and rotating an ultrasound probe head that is part of a combined ultrasound and endoscopy system (CUES), according to some embodiments;
• 6 12 is a perspective view showing further details of the distal tip of a combined ultrasound and endoscopy system (CUES) according to some embodiments;
• 7 14 is a perspective view of a flexible handheld portion that is part of a combined ultrasound and endoscopy system (CUES), according to some embodiments;
• 8A and 8B Diagrams showing further details of the separable hand-held portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments;
• 9A-9C Diagrams showing further details of the distal tip and cannula of a combined ultrasound and endoscopic probe according to some embodiments;
• 10A-10C Diagrams illustrating further details of the distal tip and cannula of a combined ultrasound and endoscopic probe according to some embodiments; and
• 11A and 11B schematic diagrams showing further examples of combined ultrasound and endoscopy systems (CUES) according to some embodiments.

DETAILED DESCRIPTION

A detailed description of examples of preferred embodiments follows. Although several embodiments are described, it should be understood that the novel subject matter described herein is not limited to any particular embodiment or combination of embodiments, but encompasses numerous alternatives, modifications, and equivalents. Furthermore, while numerous specific details are set forth in the following description in order to provide a thorough understanding, some embodiments may be practiced without some or all of these details. Also, for the sake of clarity, certain prior art material has not been described in detail in order to avoid the novelty described herein not to obscure the subject unnecessarily. It should be understood that individual features of one or more of the specific embodiments described herein can be used in combination with features of other described embodiments or with other features. Furthermore, like reference numbers and designations in the different drawings indicate like elements.

As used herein, the term processor includes one or more processors, such as a single processor or multiple processors of a distributed processing system. A controller or processor, as described herein, generally includes a tangible medium for storing instructions to implement steps of a process, and the processor may include, for example, one or more of a central processing unit, programmable array logic, gate array logic or a field programmable gate array.

As used herein, the terms distal and proximal refer to locations referenced from the device, which may be opposite to the anatomical references. For example, a distal location of a probe may correspond to a proximal location of an elongate member of the patient, and a proximal location of the probe may correspond to a distal location of the elongate member of the patient.

While some exemplary embodiments are directed to cystoscopes and/or hysteroscopes, those skilled in the art will understand that this is not intended to be limiting and that the devices described herein may be used for other therapeutic or diagnostic procedures and in other anatomical regions of a patient's body.

According to various embodiments, a device includes a probe portion for direct insertion into a body cavity. The probing section is placed in proximity to the tissue and/or area to be examined. A probe, as used herein, includes an object that is inserted into a subject, such as a patient.

According to some embodiments, an Endoscopy Ultrasound Hysterography and Cysterograph System (EUHCS) is described. According to some embodiments, the system may be referred to more generically as a combined ultrasound and endoscopy system (CUES). EUHCS and CUES are medical devices that allow doctors to obtain video images and organ information from inside the uterus, bladder or other organs. According to some embodiments, the real-time endoscopy and ultrasound images are displayed simultaneously on one monitor or two separate monitors so that the physician can see both the surface and the interior of the tissue of organs, and they can be transmitted electronically to other devices such as workstations that may be located in remote locations, and/or PACS (Picture Archiving and Communication Systems) transmitted.

In the case of clinical applications in gynecology, according to some embodiments, applications for the EUHCS include: (1) early diagnosis of endometrial cancer; (2) providing information on the stage of endometrial cancer based on the depth, area and extent of cancer infiltration of the myometrium; (3) monitoring of hysteroscopic procedures to improve surgical safety and accuracy and success rate; and (4) detection, diagnosis and staging of ovarian and/or fallopian tube cancer.

In providing combined ultrasound and endoscopic images of the uterus, the devices and methods of the present specification can take the shape of the uterus into account. The cervical canal and part of the uterine cavity are cylindrical in shape. The inferior surface of the superior uterine fundus is horizontally oriented in relation to the cervical canal and inferior uterine cavity. According to some embodiments, the hysteroscope and the ultrasound probe are configured to generate images in both vertical and horizontal directions.

Similarly, the shape of the bladder can be taken into account when providing combined ultrasound and endoscopy images of the bladder. The bladder is spherical in shape. According to some embodiments, a flexible cystoscope and an ultrasound catheter probe can be used to image all or at least a desired portion of the bladder.

1A-1C 12 are schematic diagrams showing examples of combined ultrasound and endoscopy systems (CUES) according to some embodiments. In 1A For example, system 100 includes a handheld portion 110 and a tower system 112 interconnected by cables 134 and 136 and processing units 180, 182 and 184. According to some embodiments, the handheld portion 110 is configured as a single-use unit and can be discarded after a single use. According to some other embodiments, the handheld portion 110 is for separating into one disposable upper portion 120 and a multiple use lower handle portion 130 are configured. In such cases, the single-use portion 120 is detachable from the handle portion 130, for example along dashed line 132, such that the handle portion 130 is configured for multiple use. According to some embodiments, different types of versions of the disposable sections may be provided for mating with the same reusable section. in the in 1 In the example shown, three versions of disposable portions in sterile packages or pouches 121, 122 and 123 are shown. In pouch 121 a complete hand held portion 110 is supplied. A detachable, single-use upper portion is provided in pouch 123 for mating with the same reusable portion provided in pouch 121, or with a similar reusable portion, and pouch 124 provides a flexible cannula version. As will be described in more detail below, all devices may have a camera module, LED illumination and ultrasound transducer modules at their distal tips, and one or more internal lumens for fluid transport. The tower system 112 includes a column 140 mounted on a wheeled base 142 . The tower system 112 includes two displays 150 and 152, a keyboard 160, a mouse 162 and a processing system 170. The processing system 170 may functionally include an ultrasound image processing unit 182, an optical endoscopy image processing unit 180 (referred to in the figures as a hysteroscope unit) and include a graphics unit 184 for image display and management processing. According to some embodiments, the display 150 is configured to display ultrasound images 154 from the ultrasound transducer module on the distal tip of the unit 110 and the display 152 is configured to display live live visual images 156 from the camera module on the distal tip of the unit 110 . According to some embodiments, display monitors 150 and 152 may be touch-sensitive to receive user input and have high resolution. According to some embodiments, displays 150 and 152 are each configured to display high-resolution graphics with pixel resolutions of 1280x720, 1920x1080, 2048x1080, 2560x1440, 3840x2160, or higher. According to some embodiments, the camera image display 152 shows a live video image 156 in an orientation of the anatomy inside the human body cavity and the relative position of the anatomy to the tip of the camera. The ultrasound image 154 shown on the display 152 is generated by an ultrasound transducer in a known position relative to the camera. According to some embodiments, the ultrasound image and the camera image may be related to position and orientation within the human body cavity (e.g., uterus 102, in 1 shown) can be correlated exactly. The side-by-side display of the anatomy ultrasound 154 and the camera image 156 allows the physician to simultaneously see the ultrasound image guided by the camera image in real time. In accordance with some embodiments, the camera module is configured and mounted at the distal tip of unit 110 such that at least a portion of ultrasound probe head 252 is visible to the operator in camera image 156 as ultrasound probe head portion 158 . Providing the ultrasound probe head portion 158 on the live camera image 156 can provide the operator with valuable information and feedback about the current orientation and position of the ultrasound probe head 252 .

According to some embodiments, the processing system 170 includes an ultrasound image processing unit 180, an endoscopy image processing/hysteroscope unit 182, and an image display and management system/graphics unit 184. The handheld unit 110 is connected to the ultrasound processing unit 182 and the hysteroscope unit 180 via cables 134 and 136, respectively. The processing system 170 may also include a suitable personal computer or workstation, the / the one or more processing units 174, input / output devices such as CD and / or DVD drives, internal memory 142 such as RAM, PROM, EPROM and magnetic storage media such one or more hard drives 172 for storing the medical images and associated databases and other information; and graphics processors suitable for operating the graphics displayed on displays 150 and 152. According to some embodiments, the tower system 112 is powered by a medical grade power supply (not shown). According to some embodiments, a fluid control system 186 is coupled to handheld portion 110 via fluid line(s) 132 . In some cases there are two fluid lines so that inflowing and outflowing fluids can be controlled.

1B shows a similar system as 1A with a different display configuration according to some embodiments. A single display monitor 150 can be used instead of two displays, as in 1B shown. The endoscopy image 156 and the ultrasound image 154 are combined by the graphics unit 184 . The endoscopy image 156 and the ultrasound image 154 are simultaneously displayed side-by-side on a single high-resolution monitor 150, as shown. In some cases, this display configuration can offer the doctor a better visual effect during diagnostic and surgical procedures.

1C 12 shows another example of a combined ultrasound and endoscopy system (CUES) according to some embodiments. In this example, the handheld portion 110 of the CUES system 100 is connected to two separate tower systems 116 and 118. Each of the systems 116 and 118 can be used with the in 1A and 1B be identical or similar to the tower system 112 shown. in the in 1C In the case shown, however, the ultrasound images 154 are displayed on the monitor 150 of the tower system 116, while the endoscopy images 156 are displayed on the monitor 152 of the tower system 118. The two tower systems 116 and 118 are positioned side by side with the monitors 150 and 152 so that the physician can see both images at the same time. Note that in this case, the hysteroscope unit 180 can be included in the tower system 118 and the ultrasound processing unit 182 can be included in the tower system 116 .

2A , 2 B , 2C and 2D 12 are right, left, top, and front views, respectively, of the handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments. Hand-held portion 110 includes a cannula 240 and a distal tip 250. According to some embodiments, cannula 240 may be rigid, flexible (like cannula 740 in 7 ) or semi-rigid. When semi-rigid, the cannula 240 may be made of a slightly pliable material so that the operator can adjust the cannula slightly, eg, 5-20 degrees, by manually pushing or pulling on the cannula shaft. The distal tip includes an ultrasound probe head 252 that can be rotated and "steered". In particular, the ultrasound probe head 252 is configured to be rotated about a central longitudinal axis 254, as shown by dotted arrow 256, and steered, as indicated by dotted arrow 258, about the ultrasound probe head 252 relative to axis 254 to bend According to some embodiments, both the rotating and the steering of the probe head 252 at the proximal end 260 of the cannula 240 are controlled. According to some embodiments, at proximal end 260 steering of probe head 252 is controlled by turning a wheel 262 and turning of probe head 252 is controlled by turning a wheel 264 . Mechanisms other than a wheel may also be used to control the rotation of the probe head 252 according to some embodiments. Such mechanisms include, without limitation, levers, sliders, sticks, and trackballs.

According to some embodiments, the ultrasound probe head 252 includes an ultrasound transducer made from a conventional piezoelectric material such as PZT or from semiconductor materials. The transducer can have dimensions of 2-4 mm wide and 10-20 mm long. The transducer in head 252 may contain up to 128 elements (or more) (see e.g. ultrasonic transducer array 610 in 6 ). According to some embodiments, the ultrasound transducer array can be configured as a linear or phased array, or as a single element transducer with a frequency of 5 MHz to 10 MHz for general imaging to cover the entire uterus and bladder; and 10 MHz to 30 MHz for superficial images of the uterus, endometrium, and bladder lining. According to some embodiments, when fewer transducer elements, eg, 64 elements or fewer, are used in the array in head 252, each transducer element can have its own cable. The cables can be bundled and connected to an ultrasonic processing unit (e.g. to the in 1 shown unit 182) can be connected. According to some embodiments, ultrasound processing unit 182 may be integrated into processing system 170 (in 1 1) may be integrated and form part thereof, and in some other embodiments, the ultrasound processing unit 182 may be separate from the processing system 170 in whole or in part. According to some embodiments, when the handle 130 is configured as part of the disposable portion, the handle 130 can include a compact ultrasound processing unit 272 as shown in FIG 2A shown included. The compact ultrasound processing unit 272 in the handle 130 can be manufactured at a lower cost and have less functionality than in cases where the handle is not configured for single use, such as that in FIG 8A handle 830, as shown. According to some embodiments, an ASIC may be integrated into the probe head 252 to reduce the number of cables extending from the probe head 252. The ASIC may contain high voltage switches and control circuitry for driving the individual transducer elements and routing the echo signals to the processing unit 252. In such cases, a reduced number of coaxial cables can then be used to carry ultrasound transmit and receive signals, control signals, and electrical power between the probe head 252 and the ultrasound processing unit 182 .

Again referring to 2A-2D , the cannula 240 can be long, thin and rigid or semi-rigid. According to some embodiments, the cross-section of cannula 240 perpendicular to its major longitudinal axis may be substantially circular. It should be noted that the cross-section can have any suitable shape, such as oval. The diameter of the cannula can vary depending on the type of endoscopy, eg from 3 mm to 15 mm. The cannula may contain a working channel (the distal Working channel port 280 is in 2D shown). The working channel may be accessible through a port (not shown) at the rear of proximal end 260 . The cannula 240 may include one or more fluid channels in fluid communication with various fluid ports. The cannula 240 may include a channel shared by an inflow and an outflow. Alternatively, the cannula can comprise two or more channels with separate inflow and outflow. According to some embodiments, cannula 240 also includes one or more fluid lumens that are fluidically isolated from the working channel. The fluid lumens may lead to fluid ports 246 and 244 located on the left and right sides of the distal end of cannula 240, respectively. The fluid lumens may be in fluid communication with fluid ports 230 and 232 on the underside of handle 130 . According to some embodiments, right side fluid port 230 is connected to right side fluid ports 244 and left side fluid port 232 is connected to left side fluid ports 246 . The cannula 240 is also configured to receive a plurality of electrical conductors used to supply power and control signals to and receive video/image data from the camera and illumination modules 270 and the ultrasound probe head 252 at the distal tip 250 . In some cases, the conductors may be isolated and placed in a separate lumen within cannula 240, in other cases some or all of the conductors may be placed in a lumen that is also used for another purpose (e.g., fluid and/or device /tool channel). According to some embodiments, one or more optical fibers may pass through cannula 240 to transmit data and/or deliver illumination light to distal tip 250 . The cannula 240 may also contain one or more separate lumens for one or more rods or rods used to control the rotation and/or steering of the ultrasound probe head 252 . 9A-9C and 10A-10C show further details on possible cannula designs. The handle portion 130 has a main body sized and shaped to be securely and ergonomically gripped by the operator's hand. The handle portion 130 also includes one or more buttons 212 that can be configured to allow common tasks to be performed during use. For example, the button 212 can be programmed to control the LED illumination level (from LEDs on the distal tip 250), capture still images and/or start and stop recording video images, and capture ultrasound data and/or ultrasound images and/or their Recording starts and stops. According to some embodiments, an upper case 242 may also be provided, as shown.

According to some embodiments, the camera and lighting modules 270 at the distal tip of the cannula 240 include a camera module that has a field of view of approximately 120 degrees. According to some embodiments, the camera module can be mounted so that its line of sight (DOV) is at an oblique angle to the main longitudinal axis of the cannula 240 . According to some embodiments, the oblique angle is between zero and 30 degrees. An optical component such as a prism can also be used to create this oblique angle. Further details on techniques for changing the direction of view of the camera module can be found in co-pending US patent application Serial No. 16/268,819, which is incorporated herein by reference.

According to some embodiments, the cannula 240 is connected to a rotary wheel 290 located near the proximal end of the upper housing 242 . Rotating cannula rotation wheel 290 as indicated by dashed arrow 292 can also rotate cannula 240 and camera and lighting modules 270 as indicated by dashed arrow 294 . According to some embodiments, the cannula 240 can be configured to rotate 180 degrees (or more) so that the camera and lighting modules 270 can provide at least a 180 degree angle of view into the uterus, bladder, and other organ cavities.

According to some embodiments, the handle portion 130 includes an electronics set 274 that filters raw image data and transmits it to the hysteroscope image processing unit 180, which may be part of the processing system 170 (in 1 shown). According to some embodiments, the wheel 264 includes a rotation sensor 276 configured to measure or detect the rotational position of the ultrasound probe head 252 . The data from the rotation sensor 276 is processed by the ultrasonic processing unit 182 (in 1 shown) used to create three-dimensional ultrasound images.

3A and 3B 12 are perspective and plan views of the distal end of a handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments. 3A shows the camera and lighting modules 270 with a camera module 320 and two LEDs 330 and 332. In 3B For example, the dotted outlines 312 and 314 illustrate the steering range of the ultrasound probe 252. The probe 252 is indicated by a solid outline in a vertical or "neutral" position tion 310 shown. In the example shown, the probe head 252 rotates about a hub 312 and can be positioned approximately 90 degrees in either direction as shown. According to some embodiments, fluid flushing may be accomplished by introducing fluid into and/or out of the working channel distal port 280 and/or the lateral fluid ports 246 and 244 . In one instance, the working channel distal port 280 is used to introduce or direct fluid into the organ/tissue of interest, while the lateral fluid ports 246 and 244 are used to drain or receive flowing fluid from the organ/tissue of interest. In this case, one of the fluid ports 230 and 232 on the underside of the handle 130 can be used for inflating and the other for deflating.

4A and 4B 12 are partial perspective views of an ultrasound probe that is part of a combined ultrasound and endoscopy system (CUES), according to some embodiments. Shown is the ultrasound probe assembly 410 configured for insertion through a channel of an endoscopic cannula. 4A Figure 12 illustrates further details of the steering control of the ultrasound probe head 252. Rotating the wheel 262 at the proximal end 260, as indicated by the dotted arrow 462, controls the steering of the probe head 252 to the right and left, as indicated by the dotted arrow 452 and the dotted outline 412 and 414 indicated. According to some embodiments, mechanisms other than a wheel may be used to control the steering of the probe head 252 . These mechanisms include, without limitation, levers, sliders, sticks, and trackballs. Also shown is shaft 440 which extends the length of assembly 410 from probe head 252 to proximal end 260. FIG. 4B 12 shows further details of the rotational control of ultrasound probe head 252 about axis 254 as indicated by arrow 454. FIG. The rotation of the probe head 252 is controlled by the rotary wheel 264 as indicated by arrow 464 . According to some embodiments, the entire in 4A and 4B The ultrasound assembly 410 shown can be configured for use in the working channel of a conventional endoscopy system to enable ultrasound of an endoscopy system to form a combined ultrasound and endoscopy system (CUES). In such cases, a cable connector (not shown) is provided at the proximal end 260 of assembly 410 . The cable connector is used for power, control, and transmission and reception of ultrasound signals and/or data to and from the ultrasound probe head 252 .

5A , 5B and 5C 12 are perspective views showing further details of mechanisms for steering and rotating an ultrasound probe head that is part of a combined ultrasound and endoscopy system (CUES) according to some embodiments. 5A shows further details of the steering of the ultrasound probe head 252. The steering movement is indicated by the dotted arrow 552. According to some embodiments, steering is controlled by axial displacement of a push rod 520 as indicated by arrow 518 . Forward or backward movement of rod 520 pushes or pulls rotary plate 550, which rotates about axis 512 on a hub (not shown). According to some embodiments, the pivot plate 550 is configured to provide a range of motion of at least 120 degrees. In some embodiments, the steering system can be configured so that the steering range is equal in both directions, but in other embodiments the steering range is not equal, resulting in greater steering/probe deflection in one direction. Note that by allowing for rotation of the probe head 252 and/or rotation of the entire cannula shaft 1240, the ultrasound transducer assembly 610 (in 6 shown) in the probe head 252 can be oriented with respect to the tissues of interest over a relatively wide range, thereby enhancing the capability for clear and useful ultrasound imaging.

5B 14 shows further detail of the rotation of ultrasound probe head 252. Rotational movement about axis 254 is controlled by movement of a belt 570 as indicated by arrow 574. FIG. Belt 570 rotates a gear 540 about axis 542 . Gear 540 has a bevel gear that meshes with a bevel gear on the proximal end of probe head 252 . 5B 12 also shows steering control pushrod 520. According to some embodiments, probe head 252 may be configured with a range of rotational movement about axis 254 of approximately 360 degrees.

5C FIG. 12 illustrates further details of the operation of the distal end steering and pivot steering wheels 262 and 264. FIG. Steering wheel 262 rotates as indicated by arrow 536 . Gear 262 is rod connected to a bevel gear which meshes with a bevel gear on wheel 566 which rotates as indicated by arrow 536 . The wheel 566 has another gear surface that meshes with a worm gear formed on the steering control push rod 520 . The mediated axial movement of rod 520 is indicated by arrow 526 . 5C Also shows rotary control wheel 264 rotating as indicated by arrow 564 . Wheel 264 turns in Bevel gear meshing with another bevel gear which in turn moves belt 570 as indicated by arrow 572 . The rotation sensor 276 is also shown. The rotation sensor 276 may be connected to the hub of the wheel 264 as shown or may be located inside the rotation wheel 264 . The rotation sensor 276 can be an optical encoder or other measurement device such as a potentiometer. The rotation sensor 276 provides the ultrasound transducer position at which a frame of ultrasound data is acquired. The position information is attached to an ultrasound data frame, which is used to create a 3D ultrasound image. The rotational position information can also be sent to the ultrasound processing unit 182 . According to some embodiments, the position information may be used to trigger an ultrasound timing unit to initiate transmission of ultrasound waves and reception of ultrasound data. With the rotational position detection of the sensor 276, the distance between two frames of ultrasound images can be precisely controlled with a spatial resolution of about 0.1 mm.

6 12 is a perspective view showing further details of the distal tip of a combined ultrasound and endoscopy system (CUES) according to some embodiments. The distal tip 250 includes the camera and lighting modules 270, the distal working channel connector 280 and the ultrasound probe head 252. According to some embodiments, the working channel has an inner diameter of approximately 3.2 mm, so that standard surgical instruments for performing various surgical procedures can be placed therein. Examples of such devices are: biopsy needles, hypodermic needles, forceps, tubes, knives, snares, probes, coagulation devices, brushes, laser devices, microwave devices (e.g. for ablation) and photodynamic tools. The camera and lighting modules 270 include a camera module 320 and two LEDs 330 and 332. According to some embodiments, a camera module 320 may include a CMOS image sensor such as Omni Vision Technologies, Inc. model OVM6946. According to some embodiments, the camera module 320 is approximately 1.05 mm x 1.05 mm, has a resolution of approximately 400 x 400 pixels, a field of view of approximately 120 degrees, and is capable of capturing video at 30 frames per second.

7 12 is a perspective view of a flexible handheld portion that is part of a combined ultrasound and endoscopy system (CUES) according to some embodiments. The flexible handheld portion 710 is for connection to a tower system, for example by cables 134 and 136 to a tower system 112 as in FIG 1 shown, configured. Again referring to 7 , the cannula 740 has flexible cables that allow the cannula 740 to be steered or flexed in multiple directions. According to some embodiments, mechanical control can be provided for both rotation and steering, eg as in FIG 3B and 5A-C2A shown. The hand held portion 710 includes a proximal gripping portion 734 configured for an ergonomic hand grip. According to some embodiments, the flexible handheld portion 710 has a distal tip 750 that includes a camera and illumination module 770 , an ultrasound probe head 752 , and a distal working channel port 780 . These components may be identical or similar to the camera and illumination module(s) 270, ultrasound probe head 252, and distal working channel port 280 described and shown elsewhere herein. In cases where the handheld portion 710 is integrated and has mechanical rotation and steering controls, the ultrasound probe head 752 need not have its own rotation capability. According to some embodiments, the ultrasound probe head 752 is configured to reduce the range of steering and/or rotational movement compared to the probe head 252 shown and described elsewhere herein. According to some embodiments, two proximal fluid ports 730 and 732 are configured to provide fluid inflow and fluid outflow and are analogous to those shown in FIG 2A-2D shown fluid connections 230 and 232.

According to some embodiments, a flexible CUES, as in 7 shown to be more appropriate than a rigid CUES for male urology patients. For example, sites in the bladder may be more accessible and less painful for male patients when a flexible CUES is used. According to some embodiments, the flexible CUES can be used to gain better access to the ureter and perform ultrasound scans of the kidney. The flexible CUES may also be better suited to reach specific sites in the uterus and obtain ultrasound images of those sites in the uterus that are not easily accessible with a rigid hysteroscope.

8A and 8B 12 are diagrams showing further details of the detachable handheld portion of a combined ultrasound and endoscopy system (CUES) according to some embodiments. In this case, the handheld portion 810 has many components that are similar and/or identical to the handheld portion 110 shown and described elsewhere herein. In this case, the hand-held portion 810 is configured to fold into an upper Disposable portion 840 and a lower reusable portion 830 can be separated. The two sections 830 and 840 can be plugged together and separated by manual operation without tools. In particular, handle portion 830 includes a socket 860 sized for mating with male mating portion 861 protruding from disposable portion 840 . The assembly and disassembly process is indicated by the dotted arrow 866 . Electrical connectors 862 and 864 protrude from mating portion 861 . According to some embodiments, the handle portion 830 may include or include components configured to process image data, generate control signals, supply power, or establish communication with other external devices. In some cases, the communication can be wireless or wired. The wireless communication can include, for example, Wi-Fi, radio communication, Bluetooth, IR communication, or other direct communication types. In some embodiments, the handle portion may include a sensor assembly for measuring a relative position between the cannula and the handle portion. In other embodiments, the sensor array can measure the relative position or orientation of the handle to its surroundings. Examples of such sensor arrangements are described further below. 8B 12 shows the disposable portion 840 as supplied in a sterile package or pouch 123. FIG. Because the lower portion of the handle 830 is multiple use (rather than single use like the one in 2A-2D handle 130) shown, greater processing capabilities are incorporated therein, according to some embodiments. For example, electronics unit 872 may be included in lower handle portion 830 to perform some or all of the functions of ultrasound processing unit 182 (in 1 shown) to provide.

Further details of the operation of a combined ultrasound and endoscopy system (CUES) will now be described. In the case of a rigid endoscope and ultrasound probe, such as hand-held sections 110 and 810 shown and described herein, the following procedure may be used: (1) Guided by live video images from camera module 320 (in 3A and 6 shown) at the tip of the endoscope, the endoscope is inserted into the uterus or bladder. (2) To image the cervical canal, the ultrasound probe is oriented in its vertical position (e.g. position 310 in 3B) . The rotary control wheel 264 is actuated to rotate the ultrasound probe 252 about the axis 254 (eg, in 5B shown). In this way, ultrasound images of the cervical wall and/or the vertical section of the uterine wall are obtained. (3) The distal tip 250 is moved into the uterine or bladder cavity. Steering wheel 262 is operated to set probe head 252 at a desired angle, then rotary control wheel 264 is operated to rotate ultrasound probe head 252 about axis 254 (e.g., in 5B shown) so that ultrasound images of the upper uterine or upper bladder wall(s) are obtained.

In the case of a flexible endoscope and ultrasound probe, such as those in 7 With the hand-held portions 710 shown, the following procedure can be used. (1) The distal tip 750 of the flexible cannula 740 is inserted into the uterine or bladder cavity. During insertion, the ultrasound probe head 752 may be in a retracted position in which the distal tip of the probe head 752 is recessed proximally from the distal end of the cannula 740 . (2) Based on live video endoscopy images from the camera module, an area where the ultrasound image is to be recorded is searched. (3) The ultrasound probe is pushed distally so that the probe head 752 protrudes distally from the distal tip 750 (as in 7 shown). (4) The ultrasonic steering control wheel 762 is used to adjust the probe head to a desired angle, then the ultrasonic rotary control wheel 764 is operated to rotate the ultrasonic probe head 752 to a desired angle, thereby obtaining appropriate ultrasonic images.

9A-9C 12 are diagrams showing further details of the distal tip and cannula of a combined ultrasound and endoscopic probe according to some embodiments. 9A 12 is a cross-sectional schematic diagram showing cannula 240 having an upper lumen or channel 910 and a lower lumen or channel 920. FIG. 9B 12 is a perspective view of distal tip 250 showing ultrasound probe assembly 410 deployed in lower channel 920. FIG. Note that in the in 9A-9C In the example shown, ultrasound probe assembly 410 is deployed within cannula 240, which is configured with a single working channel (channel 920). In some cases, the cannula 240 can have multiple working channels, as in 10A-10C shown. 9C 12 shows a cross-section of cannula 240 with a single working channel. In this case, the cannula 240 has an outer wall 900 that houses four separate channels. Upper channel 910 is used for cables to and from camera module 320, LEDs 330 and 332, and possibly other electrical and/or fiber optic cables. The two side channels 930 and 932 are used for drainage of fluid (ie, fluid from the patient's organ or cavity into the device). The side channels 930 and 932 can with the side fluid ports 244 (in 2A and 3B shown) or 246 can be connected. The lower channel 920 is shared by the influent fluid (ie, fluid flowing into the organ or cavity), the ultrasound probe 410, and possibly other surgical tools. According to some embodiments, the lower or working channel 920 has an inner diameter ranging from 1 mm to 4 mm. According to some embodiments, the overall outer diameter of the endoscope cannula 240 in the cases of a single working channel, as in FIG 9A-9C shown less than about 7 mm. During a surgical procedure, the uterine or bladder cavity is filled with fluid through the channel 920. A surgical tool can be inserted into the cavity to perform a surgical procedure. During the surgical procedure, the surgeon can pull out the surgical tool and insert the ultrasonic probe 410 through the working channel 920 to measure the wall thickness of the cavity and the shape of a surgical target and other information to help the surgeon make a decision about the completion of the surgical intervention can meet.

10A-10C 12 are diagrams showing further details of the distal tip and cannula of a combination ultrasound and endoscopic probe according to some embodiments. In the cases shown, the cannula 240 has two working channels. 10A 14 is a perspective view of distal tip 250 when cannula 240 has two working channels 920 and 1020. FIG. In the case shown, the ultrasound probe assembly 410 is inserted through the working channel 920 while the working channel 1020 is empty. 10B 14 is a cross-section of cannula 240. As in the case of the single working channel, there is an upper channel 910 used for cables for camera module 320, LEDs 330 and 332, and possibly other electrical and/or fiber optic cables. Similarly, the two side channels 930 and 932 for fluid drainage (ie, fluid flow from the patient's organ or cavity into the device) are connected through side fluid ports 244 (in 2A and 3B shown) or 246 used. The working channel 920 is used for the ultrasound probe assembly 410, while a separate working channel 1020 can be used for both fluid delivery and the surgical tool(s). According to some embodiments, the diameter of each of the working channels 920 and 1020 is in the range of 1 mm and 4 mm, and the overall outside diameter of the endoscope cannula 240 is less than about 10 mm. During a surgical procedure, the uterine or bladder cavity is filled with fluid through channel 1020 and the ultrasound probe 410 is inserted through channel 920 into the uterine or bladder cavity. After the uterine or bladder cavity is adequately filled with fluid (eg, saline), the surgical tool is inserted through channel 1020 into the uterine or bladder cavity. During the surgical procedure, the use of the surgical tool can be temporarily stopped while the ultrasonic probe is moved to the surgical site to generate an ultrasonic image. The ultrasound image(s) may provide information about the wall thicknesses of the uterine or bladder cavity, the shape of a surgical target, and other information useful in making surgical decisions. When performing a biopsy, the ultrasound image can guide a biopsy instrument to take tissue samples.

11A and 11B 12 are schematic diagrams of other examples of combined ultrasound and endoscopy systems (CUES) according to some embodiments. In these examples, the ultrasound probe assembly 410 is inserted into the working channel of a conventional reusable rigid, semi-rigid, or flexible hysteroscope, cystoscope, or other conventional endoscopic system to form a combined ultrasound and endoscopic system (CUES). The ultrasound probe assembly 410 may be similar or identical to the ultrasound probe assemblies described elsewhere herein, including controllable steering and rotating capabilities of the probe head 252. In 11A CUES system 1100 is formed by inserting ultrasound probe assembly 410 through the working channel of rigid hysteroscope 1110. As in 11A and 11B As shown, the shaft 440 of the ultrasound probe assembly 410 is inserted into the working channel of the cannula 1120 so that the ultrasound probe head 252 protrudes from the distal tip 1150 of the hysteroscope 1110. The hysteroscope 1110 includes a proximal grip portion 1130 configured for ergonomic hand gripping. In 11A the CUES system 1100 includes two tower systems 116 and 118, which correspond to the in 1C are similar or identical to those shown. In the case of 11A 1110 the hysteroscope is attached to the hysteroscope unit 180 of the endoscopy tower system 118 via the cable 1136 . The endoscopy tower system 118 includes a monitor 152 configured to display hysteroscopy images 156 as shown. Fluid lines leading to the fluid control system are not shown. The conventional hysteroscope 1110 and tower system 118 can be a conventional, stand-alone endoscopy system. The ultrasound assembly 410 is connected via the cable 134 to the ultrasound processing unit 182, which in this case is located in the ultrasound tower system 116 located. The ultrasound tower system 116 includes a monitor 150 configured to display hysteroscopy images 154 as shown. In this way, the ultrasound probe assembly 410, the cable 134 and the tower system 116 form a self-contained ultrasound unit. By positioning tower systems 116 and 118 in close proximity so that monitors 150 and 152 can be viewed simultaneously by the operator(s), CUES system 1110 may provide advantages as described elsewhere herein , including: enabling the operator(s) to see both the surface and internal tissues of organs; and providing a real-time ultrasound image guided by a camera image, thereby improving the effectiveness of various diagnostic and surgical procedures.

Examples of methods that can be performed with the combined ultrasound and endoscopy systems (CUES) described herein include, without limitation: detection, screening, and/or diagnosis of endometrial cancer; ultrasound-guided surgical planning; ultrasound-assisted therapy planning; surgical surveillance; and assessment of the surface roughness of the uterus. According to some embodiments, the combined ultrasound and endoscopy systems (CUES) described herein can be used to monitor gynecological and urological surgeries such as: uterine wall resection, endometrial ablation, endometrial resection, submucosal myoma resection, intramural myoma resection, transmural myoma resection, cervical and/or cervical canal resection , prostate resection and uterine myomectomy. The combined ultrasound and endoscopy systems (CUES) described here can also be used to perform measurements such as: thickness of the uterine wall, thickness of the endometrium, size of the polyps, thickness of the prostate, measurements inside the uterus, thickness of the urethra. The combined ultrasound and endoscopy systems (CUES) described herein can also be used to generate three-dimensional images of various organs and body parts such as ovaries, fallopian tubes, uterus, prostate and various tumors and/or polyps.

Although the treatment planning and definition of treatment profiles and volumes described herein are presented in the context of urological or gynecological diagnosis or surgery, the methods and devices described herein can be used to treat any body tissue and any organ and vessel of the body, such as the brain, heart, lungs , intestines, eyes, skin, kidneys, liver, pancreas, stomach, uterus, ovaries, testicles, bladder, ear, nose, mouth, soft tissues such as bone marrow, adipose tissue, muscle, glandular and mucous membrane tissue, spinal and nervous tissue, cartilage, biological hard tissues such as teeth, bones and the like, and body lumens and passages such as sinuses, ureters, colon, esophagus, pulmonary passages, blood vessels and pharynx.

The embodiments disclosed herein can be combined in one or more of many ways to provide improved diagnosis and therapy for a patient. The disclosed embodiments can be combined with previous methods and devices to enable improved treatment, such as combination with known methods of urological or gynecological diagnosis, surgery and surgery of other tissues and organs. It is to be understood that any one or more of the structures and steps described herein may be combined with any one or more additional structures and steps of the methods and apparatus described herein, with the drawings and supporting text providing descriptions according to embodiments.

While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are presented by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. It is to be understood that various alternatives to the embodiments of the invention described herein may be used in practicing the invention. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims (30)

An integrated visual and ultrasonic device, comprising: an ergonomic handle configured for hand gripping and having proximal and distal portions; a cannula extending distally from the distal portion of the handle and having a distal portion extending along a longitudinal axis; a distally facing camera attached to the distal portion of the cannula and having a camera field of view (FOV) encompassing a selected solid angle and a camera line of sight (DOV) angled relative to said axis; an ultrasound probe positioned on the distal portion of the cannula for both rotating about said axis relative to the distal portion of the cannula and tilting relative to said axis; a probe steering mechanism mounted on the proximal end of the handle and operatively coupled to the ultrasound probe for selectively tilting the ultrasound probe through a selected angular range relative to said axis; and a probe rotation mechanism mounted on the proximal end of the handle and operatively coupled to the ultrasound probe for selectively rotating the ultrasound probe about said axis relative to the cannula. device after claim 1 wherein said cannula further comprises at least one lumen and further comprises a shaft connecting the probe steering mechanism and the ultrasound probe, said shaft being removably received in said lumen. device after claim 2 further comprising a belt coupled within said shaft to and driven by said probe rotation mechanism, and further comprising a gear box attached to said ultrasound probe and driven by said belt for selectively rotating said ultrasound probe to rotate said axis. device after claim 3 wherein said probe rotation mechanism is configured to rotate the ultrasound probe at least 180 degrees. device after claim 2 further comprising a rotary plate to which the ultrasound probe is attached and which rotates about a pivot axis transverse to said longitudinal axis, and a rod inside said shaft, said rod coupling said probe steering mechanism to said rotary plate and onto the Rotation of the steering mechanism is responsive to pivot the rotary plate, and hence the ultrasonic probe, relative to said longitudinal axis. device after claim 5 wherein said steering mechanism is configured to tilt said ultrasound probe in two opposite directions relative to said longitudinal axis through an angle of up to 180 degrees in at least one of said directions. device after claim 6 wherein said steering mechanism is configured to tilt said ultrasound probe in said two opposite directions relative to said longitudinal axis through different angular ranges. device after claim 1 wherein the handle includes (i) a reusable portion and imaging electronics therein coupled to said camera and said ultrasound probe, and (ii) a disposable portion removably attached to the reusable portion and housing said rotating mechanism and steering mechanism. device after claim 1 wherein said cannula is flexible and flexes when inserted into a patient's bladder or ureter. device after claim 1 further comprising an ultrasound image processor operatively coupled to said ultrasound probe and an ultrasound image display configured to display ultrasound images provided by said ultrasound probe and processed by said ultrasound processor, and a camera image processor and camera image display, configured to display images provided by the camera and processed by said camera image processor. device after claim 10 wherein said ultrasound image display and said camera image display are configured to display said ultrasound and camera images simultaneously on a single screen. device after claim 1 , in which the ultrasound and visual aspects are integrated by inserting the ultrasound probe through a working channel formed in the cannula. device after claim 1 wherein said cannula is configured such that at least a portion thereof has a stiffness characteristic selected from a group consisting of rigid, semi-rigid and flexible. device after claim 1 , where the DOV ranges from 0 to 30 degrees. device after claim 1 further comprising a cannula rotation mechanism disposed on the proximal portion of the handle and operatively coupled to the cannula for selectively rotating the cannula, and thereby the camera, relative to the handle about said axis. device after claim 1 wherein said probe rotation mechanism is a probe rotation wheel. device after claim 1 further comprising a rotation sensor operatively coupled to the probe rotation mechanism and configured to provide an electronic signal indicative of rotation of the ultrasound probe about said axis. 19. Device after Claim 17 further comprising a processing system configured to process ultrasound images from said ultrasound probe and automatically generate three-dimensional ultrasound images therefrom based in part on said electronic signals from said rotary sensor. device after claim 1 wherein said probe steering mechanism is a probe steering wheel. Medical device that includes: an elongate shaft having a distal portion extending along a longitudinal axis and a proximal portion, the shaft being shaped and dimensioned for insertion into a sheath or working channel of an endoscopic cannula configured for insertion into a patient; an ultrasound probe located on the distal portion of the shaft and configured to protrude from a distal end of the sheath or cannula and provide ultrasound images; a housing attached to the proximal portion of the shaft; a probe rotation mechanism mounted on or in the housing and operatively coupled to the shaft for rotating the shaft, and thereby the ultrasound probe, about said axis through a selected range of angles of rotation; and a probe steering mechanism mounted on or in the housing and operatively coupled to the ultrasound probe for tilting the ultrasound probe relative to said axis through a selected range of tilt angles. Medical device after claim 19 further comprising a rotation sensor operatively coupled to the probe rotation mechanism and configured to provide an electronic signal indicative of rotation of the ultrasound probe about the axis. Medical device after claim 19 wherein the shaft is shaped and dimensioned for insertion into a working channel of an endoscope having a camera at its distal end, the ultrasound probe being configured to project distally from the camera when the shaft is inserted into said working channel . Procedure that includes: providing an integrated camera and ultrasound imaging device comprising an elongate cannula having a distal portion extending along a longitudinal axis; inserting the cannula into an object; operating a camera mounted on a distal portion of the cannula to provide camera images of an interior portion of the object taken with a camera viewing direction angled relative to said axis; operating an ultrasound probe, also mounted on the distal portion of the cannula and projecting distally from said camera, to provide ultrasound images of an interior portion of the object; selectively rotating the cannula and camera about said axis through a selected rotation angle to view internal portions of the object from different directions under manual control exercised at a proximal portion of the imaging device; selectively rotating the ultrasonic probe about said axis and selectively tilting the ultrasonic probe relative to said axis to obtain ultrasonic images of selected portions of the object taken from different directions in a solid angle of more than 180 degrees, under manual control, the at applied to a proximal portion of said imaging device; wherein operating the camera includes including at least a partial image of the ultrasound probe in at least some of the images provided by the camera; processing the camera and ultrasound images; and View the resulting processed camera images and ultrasound images. procedure after Claim 22 that further includes detecting rotation of the ultrasound probe and controlling the display of ultrasound images as a function of the detected rotation of the ultrasound probe. procedure after Claim 22 wherein providing the integrated imaging device further includes providing a handle configured to be grasped by the hand, said cannula being attached to said handle. procedure after Claim 22 wherein providing said handle comprises providing a handle having a disposable handle portion to which said device is attached and a multiple use handle portion releasably attached to the disposable portion and electronics for processing the ultrasound images from the Includes ultrasound probe. procedure after Claim 22 which further includes selectively withdrawing the ultrasound probe from the cannula while the cannula remains inserted into the subject and inserting a surgical instrument into a cannula lumen exposed by the withdrawn ultrasound probe. procedure after Claim 22 wherein inserting the cannula into the object includes selectively deflecting the cannula. procedure after Claim 24 wherein said selectively rotating the cannula is performed under manual control by rotating said integrated camera and ultrasound imaging device. procedure after Claim 24 wherein said selectively rotating the cannula is performed under manual control by rotating a cannula rotating mechanism operatively coupled to the cannula to selectively rotate the cannula and thereby the camera. procedure after Claim 24 further including performing a surgical procedure on the subject guided by the displayed camera images and ultrasound images.

Images