US20190174992A1 - Medical endoscope device and medical observation system - Google Patents

Medical endoscope device and medical observation system Download PDF

Info

Publication number
US20190174992A1
US20190174992A1 US16/180,406 US201816180406A US2019174992A1 US 20190174992 A1 US20190174992 A1 US 20190174992A1 US 201816180406 A US201816180406 A US 201816180406A US 2019174992 A1 US2019174992 A1 US 2019174992A1
Authority
US
United States
Prior art keywords
compression
medical
image data
raw image
encoded
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/180,406
Other languages
English (en)
Inventor
Daisuke Fujii
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Olympus Medical Solutions Inc
Original Assignee
Sony Olympus Medical Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Olympus Medical Solutions Inc filed Critical Sony Olympus Medical Solutions Inc
Assigned to SONY OLYMPUS MEDICAL SOLUTIONS INC. reassignment SONY OLYMPUS MEDICAL SOLUTIONS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FUJII, DAISUKE
Publication of US20190174992A1 publication Critical patent/US20190174992A1/en
Priority to US17/488,323 priority Critical patent/US11839353B2/en
Priority to US18/364,486 priority patent/US20230371785A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00011—Operational features of endoscopes characterised by signal transmission
    • A61B1/00016—Operational features of endoscopes characterised by signal transmission using wireless means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00025—Operational features of endoscopes characterised by power management
    • A61B1/00027—Operational features of endoscopes characterised by power management characterised by power supply
    • A61B1/00029—Operational features of endoscopes characterised by power management characterised by power supply externally powered, e.g. wireless
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/00048—Constructional features of the display
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00188—Optical arrangements with focusing or zooming features
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115—Selection of the code volume for a coding unit prior to coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/65—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience

Definitions

  • the present disclosure relates to a medical endoscope device and a medical observation system.
  • An operator, or the like who uses a medical endoscope device performs medical practice while viewing a display screen on which a medical captured image captured by the medical endoscope device is displayed.
  • image data indicating the medical captured image is compression-encoded and the compression-encoded image data is transmitted in wireless communication is assumed, transmitting the image data with low latency is advantageous.
  • the present disclosure proposes a novel and improved medical endoscope device and medical observation system that enable image data indicating a compression-encoded medical captured image to be transmitted with low latency.
  • a medical endoscope device including: an encoding processing unit configured to compression-encode RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body; and a transmission unit configured to wirelessly transmit the compression-encoded RAW image data.
  • a medical reception device including: a reception unit configured to wirelessly receive compression-encoded RAW image data obtained by compression-encoding RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body; and a signal processing unit configured to process the received compression-encoded RAW image data.
  • a medical observation system including: a medical endoscope device including an encoding processing unit configured to compression-encode RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body, and a transmission unit configured to wirelessly transmit the compression-encoded RAW image data; and a medical reception device including a reception unit configured to wirelessly receive the compression-encoded RAW image data, and a signal processing unit configured to process the received compression-encoded RAW image data.
  • FIG. 1 is an explanatory diagram illustrating an example of a configuration of the medical observation system according to an embodiment of the present disclosure
  • FIG. 2 is an explanatory diagram illustrating an example of transmission of image data indicating a medical captured image in a case in which the image data is not compression-encoded;
  • FIG. 3 is an explanatory diagram illustrating an example of transmission of image data indicating a medical captured image in a case in which the image data is compression-encoded;
  • FIG. 4 is an explanatory diagram for describing an example of an error caused by compression-encoding
  • FIG. 5 is an explanatory diagram for describing an example of an error caused by compression-encoding in a case in which a compression-encoding method according to an embodiment of the present disclosure is applied;
  • FIG. 6 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a first embodiment of the present disclosure
  • FIG. 7 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a second embodiment of the present disclosure
  • FIG. 8 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a third embodiment of the present disclosure
  • FIG. 9 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a fourth embodiment of the present disclosure.
  • FIG. 10 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a fifth embodiment of the present disclosure
  • FIG. 11 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a sixth embodiment of the present disclosure.
  • FIG. 12 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device and a medical reception device according to a seventh embodiment of the present disclosure.
  • FIG. 1 is an explanatory diagram illustrating an example of a configuration of a medical observation system 1000 according to the present embodiment of the present disclosure.
  • the medical observation system 1000 illustrated in FIG. 1 has, for example, a medical endoscope device 100 , a medical reception device 200 , and a display device 300 .
  • the medical observation system according to the present embodiment is not limited to the example illustrated in FIG. 1 .
  • the medical observation system may further have, for example, a control device (not illustrated) that controls various operations of the medical endoscope device 100 .
  • a control device for example, an arbitrary apparatus that can perform a process of a transmission method according to the present embodiment such as a “medical controller,” or a “computer such as a server” is exemplified.
  • the control device may be, for example, an integrated circuit (IC) that can be incorporated into the above-described apparatus.
  • the medical observation system may have a plurality of “medical endoscope devices 100 and medical reception devices 200 ” and display devices 300 .
  • each of the “medical endoscope devices 100 and medical reception devices 200 ” performs transmission using a transmission method which will be described below.
  • the “medical endoscope devices 100 and medical reception devices 200 ” and the display devices 300 may be associated with each other one to one, or a plurality of “medical endoscope devices 100 and medical reception devices 200 ” may be associated with one display device 300 .
  • the display device 300 can switch the medical endoscope device 100 that has captured an image to be displayed on a display screen by performing, for example, a switching operation or the like.
  • the medical observation system may have a plurality of medical endoscope devices 100 associated with one medical reception device 200 .
  • the medical reception device 200 may switch the medical endoscope device 100 to perform wireless transmission therewith by performing, for example, a switching operation or the like.
  • the medical observation system according to the present embodiment may not have the display device 300 . Even in the case in which the medical observation system according to the present embodiment does not have the display device 300 , the medical observation system according to the present embodiment can transmit image data indicating a compression-encoded medical captured image with low latency using a transmission method according to the present embodiment which will be described below.
  • the display device 300 is a display section of the medical observation system 1000 , and corresponds to an external display device with respect to each of the medical endoscope device 100 and the medical reception device 200 .
  • the display device 300 displays various images, for example, medical captured images (moving images or a plurality of still images; the same applies below) captured by the medical endoscope device 100 , images relating to a user interface, and the like.
  • the display device 300 may be capable of performing 3D display. Display by the display device 300 is controlled by, for example, the medical endoscope device 100 , the medical reception device 200 , or the control device (not illustrated).
  • the display device 300 of the medical observation system 1000 is installed in an arbitrary place at which the display device can be visually recognized by a person relating to surgery such as an operator within an operating room, for example, a wall surface, a ceiling, a floor of the operating room.
  • a liquid crystal display for example, an organic electro-luminescence (EL) display, a cathode ray tube (CRT) display, or the like is exemplified.
  • EL organic electro-luminescence
  • CRT cathode ray tube
  • the display device 300 is not limited to the above-described example.
  • the display device 300 may be an arbitrary wearable device worn on the body of an operator or the like for use, for example, a head-mounted display, an eyewear-type device, or the like.
  • the display device 300 is driven by, for example, power supplied from an internal power supply included in the display device 300 such as a battery, power supplied from a connected external power supply, or the like.
  • the medical endoscope device 100 is a medical apparatus to be used in endoscopic surgery.
  • the medical endoscope device 100 has the function of an endoscope and at least a function of transmitting signals wirelessly.
  • the medical reception device 200 has at least a function of receiving signals transmitted from the medical endoscope device 100 wirelessly.
  • an operator an example of a user of the medical endoscope device 100 observes an operative site while referring to a medical captured image captured by the medical endoscope device 100 and being displayed on the display screen of the display device 300 , and performs various treatments such as a procedure appropriate for the surgery type on the operative site.
  • the medical endoscope device 100 includes, for example, an insertion member 102 , and a camera head 104 .
  • the medical endoscope device 100 is driven by, for example, power supplied from an internal power supply included in the medical endoscope device 100 such as a battery, power supplied from a connected external power supply, or the like.
  • the insertion member 102 has an elongated shape and includes an optical system that collects incident light.
  • the medical endoscope device 100 has a function of capturing stereoscopic images (medical captured images for the right eye and medical captured images for the left eye)
  • the insertion member 102 may include an optical system for capturing medical captured images for the right eye and an optical system for capturing medical captured images for the left eye.
  • the tip of the insertion member 102 is inserted into, for example, a body cavity of a patient.
  • the rear end of the insertion member 102 is detachably connected to the tip of the camera head 104 .
  • the insertion member 102 receives supply of light from, for example, a light source provided in the camera head 104 or an external light source.
  • the insertion member 102 is connected to the external light source via a light guide and receives supply of light from the external light source via the light guide.
  • the insertion member 102 may have, for example, a material having no flexibility or of a material having flexibility.
  • the medical endoscope device 100 can be called a rigid endoscope or a flexible endoscope depending on a material forming the insertion member 102 .
  • Light supplied to the insertion member 102 is injected from the tip of the insertion member 102 and radiated to an observation target such as a tissue in a body cavity of a patient.
  • an observation target such as a tissue in a body cavity of a patient.
  • light reflected from the observation target is collected by the optical system inside the insertion member 102 .
  • the camera head 104 includes, for example, an image sensor and has a function of capturing observation targets.
  • the insertion member 102 and the camera head 104 of the medical endoscope device 100 play a role of, for example, an “imaging device that is inserted into the body of a patient and captures an image of the inside of the body.”
  • the camera head 104 includes, for example, an encoding processing circuit and has a function of performing the processes of the transmission methods which will be described below.
  • the camera head 104 includes, for example, a transmitter and has at least a function of transmitting signals wirelessly.
  • the transmitter may be an external transmitter connected to the camera head 104 .
  • a configuration example of the camera head 104 will be described below.
  • the medical endoscope device 100 has the function of an endoscope and at least the function of transmitting signals wirelessly since it includes, for example, the camera head 104 and the transmitter.
  • the medical reception device 200 includes, for example, a receiver and is a device having at least a function of receiving signals transmitted from the medical endoscope device 100 wirelessly.
  • the medical reception device 200 includes, for example, a signal processing circuit and has a function of processing signals received by the receiver.
  • the signal processing circuit performs, for example, a decoding process of decoding compression-encoded image data.
  • the signal processing circuit may perform various kinds of processing that can be performed on medical captured images, for example, a demosaic process for adjusting RAW image data (e.g., arbitrary processing for adjusting color, brightness, and the like based on RAW image data; the same applies below), enlargement or reduction of images relating to an electronic zoom function, inter-pixel correction, and the like.
  • the medical reception device 200 may include, for example, a communication device and have a function of controlling display of the display device 300 .
  • the medical reception device 200 transmits image data processed by the signal processing circuit and a display control signal to the display device 300 through the communication device (not illustrated) and thereby controls display of the display device 300 .
  • an IEEE 802.15.1 port and a transmission/reception circuit wireless communication
  • an IEEE 802.11 port and a transmission/reception circuit wireless communication
  • a communication antenna and a radio frequency (RF) circuit wireless communication
  • an optical communication device wireless communication or wireless communication
  • LAN local area network terminal and a transmission/reception circuit (wired communication)
  • wireless communication wireless communication
  • the communication device may be capable of communicating with one or two or more external devices using a plurality of communication methods.
  • the medical reception device 200 can have various functions, for example, a function of recording a medical captured image based on image data processed by the signal processing circuit in an arbitrary recording medium, and the like.
  • Image data indicating medical captured images have tended to increase resulting from, for example, high resolution of imaging devices, high frame rates, stereoscopy, mounting of additional devices for observing special light incident on the imaging devices, and the like.
  • Image data indicating a medical captured image may be simply referred to as “image data” below.
  • FIG. 2 is an explanatory diagram illustrating an example of transmission of image data indicating a medical captured image in a case in which the image data is not compression-encoded.
  • FIG. 3 is an explanatory diagram illustrating an example of transmission of image data indicating a medical captured image in a case in which the image data is compression-encoded.
  • image sensors included in the imaging devices are denoted by “sensor.”
  • transmitters are denoted by “TX,” and receivers are denoted by “RX.”
  • the image data is transmitted from the transmitter to the receiver on a 4-lane communication path T as illustrated in FIG. 2 .
  • the compression-encoded image data is transmitted from the transmitter to the receiver on one lane of the communication path T, for example, as illustrated in FIG. 3 .
  • the image data indicating a medical captured image functioning as a stereo image can be transmitted on one lane of the communication path T.
  • the amount of the image data can be reduced in accordance with the compression ratio and degradation of the quality of the decoded medical captured image can be prevented.
  • RAW image data is, for example, image data that has yet to undergo a demosaic process.
  • advantages of transmitting RAW image data for example, advantages described in (A) and (B) below are exemplified.
  • a transmission amount can be reduced in a case in which RAW image data is transmitted more than in a case in which demosaic-processed image data is transmitted as will be shown in the following example.
  • a transmission amount in a case in which demosaic-processed data with YCbCr 4:2:2 (of 10 [bits], and a frame rate of 59.94 [p]) is transmitted with no compression: about 10 [Gbps]
  • the RAW image data (in Bayer array) can realize a lower transmission rate with higher image quality or higher quality.
  • RAW image data is image data of which information of light at the time of imaging is retained inside the imaging device as it is without adjustment, that is, pre-processed image data that has not undergone a demosaic process or the like.
  • a device on the reception side can perform adjustment of hue, brightness, and the like without degrading the image quality, and can adjust exposure and white balance later.
  • the device on the reception side can perform an arbitrary demosaic process and processing on the RAW image data.
  • the medical endoscope device 100 compression-encodes RAW image data indicating a medical captured image and wirelessly transmits the compression-encoded RAW image data. That is, in the medical observation system 1000 , the medical endoscope device 100 performs compression-encoding on RAW image data without performing the demosaic process. In the medical observation system 1000 , the demosaic process is performed by the medical reception device 200 as will be described below.
  • RAW image data indicating a medical captured image is sometimes referred to simply as “RAW image data” below.
  • the medical observation system 1000 realizes transmission of compression-encoded image data indicating a medical captured image with lower latency.
  • FIG. 4 is an explanatory diagram for describing an example of an error caused by compression-encoding.
  • FIG. 4 conceptually illustrates an error occurring in a case in which an entire medical captured image with 4K resolution (4096 ⁇ 2160 pixels) of a certain frame (an example of an entire frame image) is compression-encoded.
  • the medical endoscope device 100 compression-encodes the RAW image data indicating a medical captured image by each predetermined unit that is smaller than the medical captured image.
  • the predetermined unit for example, the unit of a plurality of lines of the medical captured image, such as every 16 lines, is exemplified.
  • the predetermined unit may be a preset fixed unit or a variable unit that can be changed on the basis of an operation of a user using the medical observation system 1000 , an operation state of a predetermined medical apparatus, or the like.
  • the predetermined unit according to the present embodiment is not limited to a unit of a plurality of lines of a medical captured image.
  • the predetermined unit according to the present embodiment may be, for example, a block unit that includes a plurality of pixels and is smaller than the entire medical captured image.
  • a case in which the predetermined unit according to the present embodiment is a unit of a plurality of lines of a medical captured image will be exemplified.
  • the predetermined unit defined as a unit of a plurality of lines of a medical captured image may be indicated as a “slice unit.”
  • FIG. 5 is an explanatory diagram for describing an example of an error caused by compression-encoding in a case in which the compression-encoding method according to the present embodiment is applied.
  • FIG. 5 conceptually illustrates an error occurring in a case in which an entire medical captured image of 4K resolution of a certain frame is compression-encoded, as in FIG. 4 .
  • a medical captured image is compression-encoded by a slice unit (an example of a predetermined unit), for example, as illustrated in FIG. 5
  • a slice unit an example of a predetermined unit
  • the medical endoscope device 100 can reduce the amount of the RAW image data while further reducing the influence of the error caused by the compression-encoding by compression-encoding the RAW image data using the compression-encoding method according to the present embodiment.
  • the time taken to perform compression-encoding by the predetermined unit becomes shorter than in the case in which the entire medical captured image is compression-encoded as illustrated in FIG. 4 . Therefore, in the case in which the compression-encoding method according to the present embodiment is used, the compression-encoded RAW image data can be transmitted with lower latency than in the case in which the entire medical captured image is compression-encoded as illustrated in FIG. 4 . In addition, such transmission of image data with low latency is beneficial when medical staff performing medical practice while viewing a decoded medical captured image are taken into account.
  • FIG. 6 is a diagram of hardware blocks illustrating an example of a configuration of the medical endoscope device 100 and the medical reception device 200 according to a first embodiment of the present disclosure. Note that, in FIG. 6 , the hardware configuration of the camera head 104 that performs the processes of the transmission methods according to the present embodiment out of the hardware configuration of the medical endoscope device 100 is illustrated.
  • the camera head 104 included in the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , a battery 116 , and a light source 118 .
  • the camera head 104 may further include, for example, a recording medium (not illustrated) on which data to be used by the encoding processing circuit 112 in processing is recorded.
  • a recording medium for example, a flash memory, a read only memory (ROM), or the like is exemplified.
  • the recording medium (not illustrated) may be included in another constituent element included in the camera head 104 , such as the encoding processing circuit 112 .
  • the recording medium (not illustrated) may be an external recording medium of the medical endoscope device 100 .
  • Each of the sensor 110 , the encoding processing circuit 112 , the transmitter 114 , and the light source 118 is electrically connected to the battery 116 (an example of an internal power supply) and operates with power supplied from the battery 116 .
  • the battery 116 for example, a secondary battery such as a lithium-ion secondary battery or the like is exemplified.
  • the sensor 110 images an observation target by photoelectrically converting light reflected from the observation target collected by the insertion member 102 and transfers RAW image data obtained from the imaging (RAW image data indicating a medical captured image) to the encoding processing circuit 112 .
  • CMOSs complementary metal oxide semiconductors
  • CCDs charge coupled devices
  • the sencoding processing circuit 112 transfers first RAW image data of a medical captured image for the right eye and second RAW image data of a medical captured image for the left eye to the encoding processing circuit 112 .
  • the encoding processing circuit 112 is a circuit functioning as an encoding processing unit of the medical endoscope device 100 and compression-encodes RAW image data.
  • the encoding processing circuit 112 performs, for example, a process of the compression-encoding method according to the present embodiment and compression-encodes RAW image data by each predetermined unit that is smaller than a medical captured image.
  • a process of the encoding processing circuit 112 is not limited to the above example.
  • the encoding processing circuit 112 may perform, for example, a process of a first example introduced in (1) and a process of a fifth example introduced in (5) below.
  • the encoding processing circuit 112 compression-encodes RAW image data at a plurality of different compression ratios. Since the encoding processing circuit 112 compression-encodes RAW image data at a plurality of different compression ratios, the medical endoscope device 100 obtains a plurality of pieces of RAW image data compressed at different compression ratios.
  • the encoding processing circuit 112 performs compression-encoding at a compression ratio corresponding to a state of electronic zoom of the imaging device.
  • Electronic zoom of the medical endoscope device 100 is a zoom method of changing a zoom magnification by performing image processing on a medical captured image, for example, without moving a lens included in the optical system of the insertion member 102 .
  • Image processing on medical captured images for electronic zoom is performed by a reception side device, for example, the medical reception device 200 , or the like.
  • zoom magnifications are changed through image processing, and thus zoom magnifications are discontinuously changed.
  • zoom magnifications are discontinuously changed.
  • the encoding processing circuit 112 cuts out a partial region of a medical captured image and then compression-encodes RAW image data corresponding to the cut-out region at a compression ratio corresponding to the state of electronic zoom.
  • a compression ratio corresponding to the state of electronic zoom is determined on the basis of, for example, one or both a size of the cut-out region and a magnification of the electronic zoom.
  • a size of the cut-out region and a magnification of the electronic zoom are acquired from an external device, for example, the medical reception device 200 , an external operation device such as a remote controller, or the like.
  • Data indicating a size of the cut-out region and data indicating a magnification of the electronic zoom can be received using, for example, a reception function of the transmitter 114 , which will be described below. That is, the transmitter 114 can function also as a receiver for the camera head 104 . Note that it is a matter of course that the camera head 104 may further include a receiver that can receive signals transmitted from the outside.
  • the encoding processing circuit 112 determines a compression ratio by referring to, for example, a “table (or database) in which sizes of cut-out regions, magnifications of electronic zoom, and compression ratios are associated with each other” which is stored in a recording medium (not illustrated).
  • the encoding processing circuit 112 may determine a compression ratio corresponding to one or both of a size of the cut-out region and a magnification of the electronic zoom by performing, for example, an arithmetic operation of an arbitrary algorithm with which a compression ratio can be determined in accordance with one or both of the size of the cut-out region and the magnification of the electronic zoom.
  • the encoding processing circuit 112 compression-encodes RAW image data at a compression ratio of 1/4 since the data amount becomes 1/4 when the magnification of electronic zoom is two times.
  • the encoding processing circuit 112 compression-encodes each of the first RAW image data and the second RAW image data.
  • wireless transmission states are likely to be affected by the following factors:
  • Noise of a transmitting apparatus noise of both a transmission side apparatus and a reception side apparatus
  • Interference of noise emanated from a treatment device such as an electrical scalpel or bipolar forceps;
  • LED light emitting diode
  • various distribution cables for distributing satellite broadcasting and the like a clinical sensor, an access point of a wireless LAN (base station), a telemeter telecontroller, a nurse call input/output (I/O), or the like;
  • Separation distance between antennas e.g., the limit of the separation distance is about 100 m for IEEE 802.11ac (5 GHz) and about 10 m for IEEE 802.11ad (60 GHz));
  • Physical shield between antennas e.g., radio waves at a high frequency level of IEEE 802.11ad (60 GHz) do not go through when shielded by a metallic object, a human, or the like).
  • the encoding processing circuit 112 can change a way of compression-encoding of RAW image data on the basis of a wireless transmission state.
  • the encoding processing circuit 112 specifies a way of compression-encoding in accordance with a wireless transmission state by referring to, for example, a “table (or database) in which wireless transmission states are associated with data indicating a way of compression-encoding” stored in a recording medium (not illustrated).
  • the encoding processing circuit 112 may specify a way of compression-encoding in accordance with a wireless transmission state by performing, for example, an arithmetic operation of an arbitrary algorithm with which a way of compression-encoding can be determined in accordance with a wireless transmission state.
  • the encoding processing circuit 112 compression-encodes RAW image data in the specified way of compression-encoding in accordance with the wireless transmission state.
  • the encoding processing circuit 112 specifies a wireless transmission state, for example, on the basis of a result of bidirectional communication with the medical reception device 200 and thus changes a way of compression-encoding of RAW image data.
  • Bidirectional communication with the medical reception device 200 is realized by, for example, the transmitter 114 , which will be described below, having the reception function.
  • the camera head 104 may further include a receiver that can receive signals transmitted from the outside as described above.
  • the encoding processing circuit 112 acquires a transmission amount (transmission speed) using a wireless transmission protocol (e.g., a transmission of a ping command, a response to a ping command, and the like) and specifies a transmission state corresponding to the transmission amount. Then, the encoding processing circuit 112 compression-encodes RAW image data in a way of compression-encoding in accordance with the specified wireless transmission state.
  • a transmission state corresponding to the transmission amount is determined by referring to, for example, a “table (or database) in which transmission amounts are associated with compression ratios” stored in a recording medium (not illustrated). Note that the encoding processing circuit 112 may specify a transmission state corresponding to the transmission amount by performing, for example, an arithmetic operation of an arbitrary algorithm with which a transmission state can be determined in accordance with the transmission amount.
  • the encoding processing circuit 112 can also specify a transmission state on the basis of the comparison result of a set reference value and an actual transmission value of the transmitter 114 and thus change a way of compression-encoding of RAW image data.
  • the encoding processing circuit 112 obtains a transmission rate by performing an arithmetic operation using, for example, the following formula 1.
  • the encoding processing circuit 112 changes a way of compression-encoding of RAW image data using the obtained transmission rate.
  • the transmission rate 25 [%] means that the transmission speed is only a quarter of the expected value.
  • the encoding processing circuit 112 can change a way of compression-encoding of RAW image data by setting, for example, the compression ratio to quadruple or the like.
  • the encoding processing circuit 112 may perform two or more processes to be combined among the process of the first example introduced in (1) to the process of the fourth example introduced in (4) above.
  • the transmitter 114 is a circuit that functions as a transmission unit in the medical endoscope device 100 and wirelessly transmits compression-encoded RAW image data transferred from the encoding processing circuit 112 .
  • the transmitter 114 a communication device compatible with wireless communication in an arbitrary communication method, for example, an IEEE 802.15.1 port and a transmission/reception circuit, an IEEE 802.11 port and a transmission/reception circuit, a communication antenna and an RF circuit, an optical communication device (for wireless communication), or the like is exemplified.
  • the transmitter 114 can have the function of a receiver as described above.
  • the transmitter 114 includes a processor, and also can perform a process relating to an operation, which will be described below, with the process.
  • the medical endoscope device 100 can also include a communication device compatible with wired communication in an arbitrary communication method such as an optical communication device (for wired communication) or a LAN terminal and a transmission/reception circuit, as will be described below.
  • a communication device compatible with wired communication in an arbitrary communication method such as an optical communication device (for wired communication) or a LAN terminal and a transmission/reception circuit, as will be described below.
  • an operation of the transmitter 114 is not limited to the above-described example.
  • the transmitter 114 may perform, for example, an operation of a first example introduced in (i) to an operation of a fifth example introduced in (v) below.
  • the transmitter 114 transmits compression-encoded RAW image data corresponding to a wireless transmission state of the RAW image data compression-encoded at the plurality of compression ratios.
  • the medical endoscope device 100 realizes, for example, “transmission performed such that transmission of data with low compression is prioritized and it is switched to transmission of data with high compression when a wireless transmission state is bad.
  • the medical observation system 1000 maintains the transmission state of RAW image data with low latency even in a case in which a transmission rate deteriorates.
  • the wireless transmission state is specified by, for example, the encoding processing circuit 112 as described above. Note that the wireless transmission state may be specified by the transmitter 114 .
  • the transmitter 114 wirelessly transmits the compression-encoded RAW image data transferred from the encoding processing circuit 112 .
  • the transmitter 114 selects compression-encoded RAW image data corresponding to the wireless transmission state from the transferred compression-encoded RAW image data and wirelessly transmits the selected compression-encoded RAW image data.
  • the transmitter 114 transmits compression-encoded RAW image data at different frequencies.
  • the transmitter 114 transmits, for example, compression-encoded RAW image data at different frequencies at the same time.
  • compression-encoded RAW image data By transmitting compression-encoded RAW image data at different frequencies at the same time, a possibility of the medical reception device 200 being able to receive the compression-encoded RAW image data can be raised.
  • the transmitter 114 may transmit the compression-encoded RAW image data at, for example, any one frequency of the plurality of frequencies.
  • the transmitter 114 transmits compression-encoded RAW image data at a frequency corresponding to a wireless transmission state.
  • the medical endoscope device 100 can transmit the compression-encoded RAW image data more stably while further reducing power consumption for the transmission of the compression-encoded RAW image data.
  • the transmitter 114 can also transmit compression-encoded RAW image data at a frequency corresponding to an operation state of a predetermined medical apparatus.
  • a predetermined medical apparatus for example, a treatment device such as an electrical scalpel or bipolar forceps is exemplified.
  • the transmitter 114 performs transmission at a frequency corresponding to an operation state of a predetermined medical apparatus.
  • An operation state of a predetermined medical apparatus is acquired through, for example, communication with another predetermined medical apparatus (or a control device controlling the predetermined medical apparatus).
  • the transmitter 114 specifies a frequency for an operation state of the predetermined medical apparatus by referring to, for example, a “table (or database) in which operation states of predetermined medical apparatuses are associated with frequencies” stored in a recording medium (not illustrated).
  • the transmitter 114 may specify a frequency for an operation state of the predetermined medical apparatus by performing, for example, an arithmetic operation of an arbitrary algorithm with which a frequency can be determined in accordance with an operation state of a predetermined medical apparatus.
  • the medical endoscope device 100 can transmit the compression-encoded RAW image data more stably while further reducing power consumption for the transmission of the compression-encoded RAW image data.
  • the transmitter 114 can also transmit compression-encoded RAW image data at a frequency corresponding to a wireless transmission state and an operation state of a predetermined medical apparatus.
  • the transmitter 114 wirelessly transmits each of the compression-encoded first RAW image data and the compression-encoded second RAW image data.
  • the transmitter 114 may stop transmission of the compression-encoded first RAW image data or the compression-encoded second RAW image data on the basis of a wireless transmission state.
  • the transmitter 114 stops transmission of one of the compression-encoded first RAW image data and the compression-encoded second RAW image data on the basis of a wireless transmission state
  • the medical endoscope device 100 can transmit the compression-encoded RAW image data more stably.
  • the medical endoscope device 100 has, for example, the configuration illustrated in FIG. 6 .
  • a configuration of the medical endoscope device 100 according to the first embodiment is not limited to that illustrated in FIG. 6 .
  • the transmitter 114 may be provided in the medical endoscope device 100 according to the first embodiment as a separate body from the camera head 104 .
  • substitution substitution (so-called replacement) of the transmitter 114 can be easier.
  • the medical endoscope device 100 can change, for example, reliability in communication, a communication distance, a transmission capacity (communication speed), an error correction method, a communication method, and the like more flexibly. That is, if the medical endoscope device 100 has a replaceable transmitter 114 , for example, version upgrade for hardware relating to communication becomes easier, and thus the medical endoscope device 100 can respond to the evolution of communication more flexibly.
  • the medical reception device 200 includes, for example, a receiver 210 and a signal processing circuit 212 .
  • the medical reception device 200 may further include, for example, a recording medium (not illustrated) on which data to be used by the signal processing circuit 212 in processes is recorded.
  • a recording medium for example, a flash memory, a ROM, or the like is exemplified.
  • the recording medium (not illustrated) may be an external recording medium of the medical reception device 200 .
  • the receiver 210 is a circuit that functions as a reception unit of the medical reception device 200 and wirelessly receives compression-encoded RAW image data.
  • the receiver 210 for example, a communication device compatible with wireless communication of an arbitrary communication method corresponding to the transmitter 114 included in the medical endoscope device 100 is exemplified.
  • the medical reception device 200 can also include a communication device compatible with wired communication of an arbitrary communication method, such as an optical communication device (for wired communication) or a LAN terminal and a transmission/reception circuit, as will be described below.
  • the signal processing circuit 212 is a circuit that functions as a signal processing unit of the medical reception device 200 and processes received compression-encoded RAW image data.
  • the signal processing circuit 212 performs, for example, a decoding process of decoding compression-encoded RAW image data. In addition, the signal processing circuit 212 performs a demosaic process on decoded RAW image data. Furthermore, the signal processing circuit 212 may perform various processes that can be performed on medical captured images such as enlargement or reduction of images relating to the electronic zoom function and inter-pixel correction.
  • the medical reception device 200 has, for example, the configuration illustrated in FIG. 6 .
  • a configuration of the medical reception device 200 according to the first embodiment is not limited to the example illustrated in FIG. 6 .
  • a communication device for performing communication with an external device such as the display device 300 may be further included as described above.
  • the signal processing circuit 212 controls display of the display device 300 through, for example, transmission to the display device 300 via the communication device (not illustrated).
  • FIG. 7 is a diagram of hardware blocks illustrating an example of a configuration of a medical endoscope device 100 and a medical reception device 200 according to a second embodiment.
  • a hardware configuration of a camera head 104 included in the medical endoscope device 100 is illustrated, as in FIG. 6 .
  • the camera head 104 of the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , transmitters 114 A and 114 B, a battery 116 , and a light source 118 .
  • the camera head 104 illustrated in FIG. 7 When the camera head 104 illustrated in FIG. 7 is compared with the camera head 104 according to the first embodiment illustrated in FIG. 6 , the camera head 104 illustrated in FIG. 7 has a difference of including two transmitters.
  • configurations and functions of the sensor 110 , the encoding processing circuit 112 , the battery 116 , and the light source 118 illustrated in FIG. 7 are similar to those of the encoding processing circuit 112 , the battery 116 , and the light source 118 illustrated in FIG. 6 .
  • the transmitter 114 A is a circuit that functions as a transmission unit in the medical endoscope device 100 and wirelessly transmits compression-encoded RAW image data transferred from the encoding processing circuit 112 , similarly to the transmitter 114 illustrated in FIG. 6 .
  • the transmitter 114 B is another circuit that functions as a transmission unit in the medical endoscope device 100 and transmits additional information.
  • additional information arbitrary data other than RAW image data, for example, one or both of control information including control commands in accordance with operations with respect to an operation device (not illustrated) included in the medical endoscope device 100 and remaining amount information indicating a remaining amount of the battery 116 , or the like is exemplified.
  • the medical endoscope device 100 according to the second embodiment has, for example, the configuration illustrated in FIG. 7 .
  • a configuration of the medical endoscope device 100 according to the second embodiment is not limited to the example illustrated in FIG. 7 .
  • one or both of the transmitters 114 A and 114 B may be provided as a separated body from the camera head 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the transmitter 114 B may transmit additional information in wired communication of an arbitrary communication method.
  • the medical reception device 200 includes, for example, receivers 210 A and 210 B, and a signal processing circuit 212 .
  • the medical reception device 200 illustrated in FIG. 7 When the medical reception device 200 illustrated in FIG. 7 is compared with the medical reception device 200 according to the first embodiment illustrated in FIG. 6 , the medical reception device 200 illustrated in FIG. 7 has a difference of including two receivers.
  • functions and configurations of the signal processing circuit 212 illustrated in FIG. 7 are similar to functions and configurations of the signal processing circuit 212 illustrated in FIG. 6 .
  • the receiver 210 A is a circuit that functions as a reception unit in the medical reception device 200 and wirelessly receives compression-encoded RAW image data, similarly to the receiver 210 illustrated in FIG. 6 .
  • a communication device compatible with the transmitter 114 A included in the medical endoscope device 100 is exemplified.
  • the receiver 210 B is another circuit that functions as a reception unit in the medical reception device 200 and wirelessly receives additional information.
  • a communication device compatible with the transmitter 114 B included in the medical endoscope device 100 is exemplified.
  • the medical reception device 200 according to the second embodiment has, for example, the configuration illustrated in FIG. 7 .
  • a configuration of the medical reception device 200 according to the second embodiment is not limited to the example illustrated in FIG. 7 .
  • the medical reception device 200 according to the second embodiment may further include, for example, a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • the transmitter 114 B may receive, for example, additional information in wired communication of an arbitrary communication method.
  • FIG. 8 is a diagram of hardware blocks illustrating an example of configurations of medical endoscope devices 100 and medical reception devices 200 according to a third embodiment.
  • FIG. 8 hardware configurations of camera heads 104 included in the medical endoscope device 100 are illustrated, as in FIG. 6 .
  • external power supplies Po 1 and Po 2 are also illustrated in FIG. 8 .
  • the camera head 104 included in the medical endoscope device 100 illustrated in A of FIG. 8 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , a battery 116 , a light source 118 , and a charging/feeding circuit 120 A.
  • the camera head 104 included in the medical endoscope device 100 illustrated in B of FIG. 8 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , a battery 116 , a light source 118 , and a charging/feeding circuit 120 B.
  • the camera heads 104 illustrated in A and B of FIG. 8 are compared with the camera head 104 according to the first embodiment illustrated in FIG. 6 , there is a difference that the camera heads 104 in A and B of FIG. 8 have the charging/feeding circuit 120 A and the charging/feeding circuit 120 B.
  • functions and configurations of the sensors 110 , the encoding processing circuits 112 , the transmitters 114 , the batteries 116 , and the light sources 118 illustrated in FIG. 8 are similar to those of the encoding processing circuit 112 , the transmitter 114 , the battery 116 , and the light source 118 illustrated in FIG. 6 .
  • Each of the charging/feeding circuits 120 A and 120 B is a circuit with a function of charging the battery 116 with power supplied from an external power supply.
  • the charging/feeding circuit 120 A charges the battery 116 with power supplied from a wire-connected external power supply Po 1 .
  • the charging/feeding circuit 120 B charges the battery 116 with power supplied from a wirelessly connected external power supply Po 2 .
  • each of the charging/feeding circuits 120 A and 120 B may have a function of supplying power accumulated in the battery 116 to an external device.
  • the medical endoscope device 100 according to the third embodiment has, for example, the configuration illustrated in FIG. 8 .
  • a configuration of the medical endoscope device 100 according to the third embodiment is not limited to the example illustrated in FIG. 8 .
  • the medical endoscope device 100 according to the third embodiment may be provided with the transmitters 114 as a separated body from the camera heads 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the medical reception devices 200 each include, for example, receivers 210 and signal processing circuits 212 .
  • Functions and configurations of the receivers 210 and the signal processing circuits 212 illustrated in FIG. 8 are similar to those of the receiver 210 and the signal processing circuit 212 illustrated in FIG. 6 .
  • the medical reception device 200 according to the third embodiment has, for example, the configuration illustrated in FIG. 8 .
  • a configuration of the medical reception device 200 according to the third embodiment is not limited to the example of FIG. 8 .
  • the medical reception device 200 according to the third embodiment may further include a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • FIG. 9 is a diagram of hardware blocks illustrating an example of configurations of a medical endoscope device 100 and a medical reception device 200 according to a fourth embodiment.
  • a hardware configuration of a camera head 104 included in the medical endoscope device 100 is illustrated, as in FIG. 6 .
  • the camera head 104 included in the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , a battery 116 , and a light source 118 .
  • the battery 116 included in the camera head 104 illustrated in FIG. 9 is detachable.
  • functions and configurations of the sensor 110 , the encoding processing circuit 112 , the transmitter 114 , the battery 116 , and the light source 118 illustrated in FIG. 9 are similar to those of the encoding processing circuit 112 , the transmitter 114 , the battery 116 , and the light source 118 illustrated in FIG. 6 .
  • the medical endoscope device 100 according to the fourth embodiment has, for example, the configuration illustrated in FIG. 9 .
  • a configuration of the medical endoscope device 100 according to the fourth embodiment is not limited to the example illustrated in FIG. 9 .
  • the medical endoscope device 100 according to the fourth embodiment may be provided with the transmitter 114 a separate body from the camera head 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the medical reception device 200 includes, for example, a receiver 210 and a signal processing circuit 212 .
  • Configurations and functions of the receiver 210 and the signal processing circuit 212 illustrated in FIG. 9 are similar to those of the receiver 210 and the signal processing circuit 212 illustrated in FIG. 6 .
  • the medical reception device 200 according to the fourth embodiment has, for example, the configuration illustrated in FIG. 9 .
  • the medical reception device 200 according to the fourth embodiment may further include a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • FIG. 10 is a diagram of hardware blocks illustrating an example of configurations of a medical endoscope device 100 and a medical reception device 200 according to a fifth embodiment.
  • a hardware configuration of a camera head 104 included in the medical endoscope device 100 is illustrated, as in FIG. 6 .
  • a battery 16 C that can be attached to the camera head 104 is illustrated together.
  • the camera head 104 included in the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , batteries 116 A and 116 B, and a light source 118 .
  • the camera head 104 illustrated in FIG. 10 When the camera head 104 illustrated in FIG. 10 is compared with the camera head 104 according to the first embodiment illustrated in FIG. 6 , there is a difference that the camera head 104 illustrated in FIG. 10 includes two batteries.
  • configurations and functions of the sensor 110 , the encoding processing circuit 112 , the transmitter 114 , and the light source 118 illustrated in FIG. 10 are similar to those of the encoding processing circuit 112 , the transmitter 114 , and the light source 118 illustrated in FIG. 6 .
  • the battery 116 A is detachable from the camera head 104 , similarly to the battery 116 included in the camera head 104 according to the fourth embodiment illustrated in FIG. 9 . That is, in the camera head 104 illustrated in FIG. 10 , the battery 116 A and the battery 116 C are interchangeable.
  • the battery 116 B is a spare battery included in the camera head 104 . Since the camera head 104 includes the battery 116 B, the medical endoscope device 100 can replace a battery while operating. In addition, since the camera head 104 includes the battery 116 B, a drive time of the medical endoscope device 100 can be extended.
  • the medical endoscope device 100 according to the fifth embodiment has, for example, the configuration illustrated in FIG. 10 .
  • a configuration of the medical endoscope device 100 according to the fifth embodiment is not limited to the example illustrated in FIG. 10 .
  • the medical endoscope device 100 according to the fifth embodiment may be provided with the transmitter 114 as a separate body from the camera head 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the medical reception device 200 includes, for example, a receiver 210 and a signal processing circuit 212 .
  • Configurations and functions of the receiver 210 and the signal processing circuit 212 illustrated in FIG. 10 are similar to those of the receiver 210 and the signal processing circuit 212 illustrated in FIG. 6 .
  • the medical reception device 200 according to the fifth embodiment has, for example, the configuration illustrated in FIG. 10 .
  • the medical reception device 200 according to the fifth embodiment may further include a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • FIG. 11 is a diagram of hardware blocks illustrating an example of configurations of a medical endoscope device 100 and a medical reception device 200 according to a sixth embodiment.
  • a hardware configuration of a camera head 104 included in the medical endoscope device 100 is illustrated, as in FIG. 6 .
  • the camera head 104 included in the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , transmitters 114 A and 114 B, a battery 116 , and a light source 118 .
  • the camera head 104 illustrated in FIG. 11 When the camera head 104 illustrated in FIG. 11 is compared with the camera head 104 according to the first embodiment illustrated in FIG. 6 , there is a difference that the camera head 104 illustrated in FIG. 11 includes two transmitters.
  • configurations and functions of the sensor 110 , the encoding processing circuit 112 , the battery 116 , and the light source 118 illustrated in FIG. 11 are similar to those of the encoding processing circuit 112 , the battery 116 , and the light source 118 illustrated in FIG. 6 .
  • the transmitter 114 A is a circuit that functions as a transmission unit in the medical endoscope device 100 , and wirelessly transmits compression-encoded RAW image data transferred from the encoding processing circuit 112 , similarly to the transmitter 114 illustrated in FIG. 6 .
  • the transmitter 114 B is another circuit that functions as a transmission unit in the medical endoscope device 100 , and transmits compression-encoded RAW image data transferred from the encoding processing circuit 112 in a wired manner.
  • the medical endoscope device 100 has, for example, the configuration illustrated in FIG. 11 .
  • the medical endoscope device 100 can transmit the compression-encoded RAW image data in arbitrary wired communication.
  • a configuration of the medical endoscope device 100 according to the sixth embodiment is not limited to the example illustrated in FIG. 11 .
  • the medical endoscope device 100 according to the sixth embodiment may be provided with one or both of the transmitters 114 A and 114 B as a separate body from the camera head 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the medical reception device 200 includes, for example, receivers 210 A and 210 B, and a signal processing circuit 214 .
  • the medical reception device 200 illustrated in FIG. 11 When the medical reception device 200 illustrated in FIG. 11 is compared with the medical reception device 200 according to the first embodiment illustrated in FIG. 6 , there are differences that the medical reception device 200 illustrated in FIG. 11 has two receivers and the signal processing circuit 214 has a different function.
  • the receiver 210 A is a circuit that functions as a reception unit in the medical reception device 200 , and wirelessly receives compression-encoded RAW image data, similarly to the receiver 210 illustrated in FIG. 6 .
  • a communication device corresponding to the transmitter 114 A included in the medical endoscope device 100 is exemplified.
  • the receiver 210 B is another circuit that functions as a reception unit in the medical reception device 200 , and receives compression-encoded RAW image data in a wired manner.
  • a communication device corresponding to the transmitter 114 B included in the medical endoscope device 100 is exemplified.
  • the signal processing circuit 214 processes compression-encoded RAW image data received by the receiver 210 A or compression-encoded RAW image data received by the receiver 210 B.
  • the signal processing circuit 214 processes received compression-encoded RAW image data, similarly to the signal processing circuit 212 according to the first embodiment illustrated in FIG. 6 .
  • the signal processing circuit 214 performs processes at each set predetermined period, for example.
  • the predetermined period may be a preset fixed period or a variable period that can be changed in accordance with an operation with respect to an operation device such as a remote controller, or the like.
  • the signal processing circuit 214 processes either RAW image data first transferred at a process start timing of compression-encoded RAW image data transferred from the receiver 210 A and compression-encoded RAW image data transferred from the receiver 210 B.
  • the signal processing circuit 214 may process RAW image data of the compression-encoded RAW image data transferred at the process start timing with a higher signal level.
  • the medical reception device 200 according to the sixth embodiment has, for example, the configuration illustrated in FIG. 11 .
  • the medical reception device 200 according to the sixth embodiment may further include a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • FIG. 12 is a diagram of hardware blocks illustrating an example of configurations of a medical endoscope device 100 and a medical reception device 200 according to a seventh embodiment.
  • a hardware configuration of a camera head 104 included in the medical endoscope device 100 is illustrated, as in FIG. 6 .
  • the camera head 104 included in the medical endoscope device 100 includes, for example, a sensor 110 , an encoding processing circuit 112 , a transmitter 114 , a battery 116 , and a light source 118 .
  • Functions and configurations of the sensor 110 , the encoding processing circuit 112 , the transmitter 114 , the battery 116 , and the light source 118 illustrated in FIG. 12 are similar to those of the encoding processing circuit 112 , the transmitter 114 , the battery 116 , and the light source 118 illustrated in FIG. 6 .
  • the medical endoscope device 100 according to the seventh embodiment has, for example, the configuration illustrated in FIG. 12 .
  • a configuration of the medical endoscope device 100 according to the seventh embodiment is not limited to the example illustrated in FIG. 12 .
  • the medical endoscope device 100 according to the seventh embodiment may be provided with the transmitter 114 as a separate body from the camera head 104 , similarly to the medical endoscope device 100 according to the first embodiment.
  • the medical reception device 200 includes, for example, receivers 210 A, . . . , and 210 N, and a signal processing circuit 214 .
  • the medical reception device 200 illustrated in FIG. 12 is compared with the medical reception device 200 according to the first embodiment illustrated in FIG. 6 , there is a difference that the medical reception device 200 illustrated in FIG. 12 has two or more receivers and the signal processing circuit 214 has a different function.
  • Each of the receivers 210 A, . . . , and 210 N is a circuit that functions as a reception unit in the medical reception device 200 and wirelessly receives compression-encoded RAW image data, similarly to the receiver 210 illustrated in FIG. 6 .
  • a communication device corresponding to the transmitter 114 A included in the medical endoscope device 100 is exemplified.
  • the signal processing circuit 214 processes any RAW image data of compression-encoded RAW image data received by each of the receivers 210 A, . . . , and 210 N.
  • the signal processing circuit 214 processes any RAW image data of compression-encoded RAW image data received by each of the receivers 210 A, . . . , and 210 N, for example, similarly to the signal processing circuit 214 according to the sixth embodiment illustrated in FIG. 11 .
  • the medical reception device 200 has, for example, the configuration illustrated in FIG. 12 . Since the medical reception device 200 includes the plurality of receivers, radio wave environments for transmission and reception of compression-encoded RAW image data can be improved.
  • a configuration of the medical reception device 200 according to the seventh embodiment is not limited to the example illustrated in FIG. 12 .
  • the medical reception device 200 according to the seventh embodiment may further include a communication device for communicating with an external device such as the display device 300 , similarly to the medical reception device 200 according to the first embodiment.
  • the medical reception device 200 according to the seventh embodiment may also can process compression-encoded RAW image data received using an external antenna.
  • the signal processing circuit 214 processes, for example, compression-encoded RAW image data received using the included receivers or compression-encoded RAW image data received using an external antenna.
  • the medical reception device 200 according to the seventh embodiment may not include a plurality of receivers.
  • External antennas can be provided in various places in the medical sites, for example, trocars, beds, shadowless light, and the like.
  • a size of data to be transmitted can be further reduced by compression-encoding RAW image data.
  • a low transmission speed or low-compressed high image quality is realized.
  • medical captured images can be compression-encoded by smaller predetermined units by using the compression-encoding method according to the present embodiments, for example, compression with lower latency in which the delay time is far shorter than 1 [msec] can be realized.
  • compression-encoded RAW image data is wirelessly transmitted, it is possible to perform image processing such as the demosaic process in a reception side device. That is, it is not necessary for a medical endoscope device (a transmission side device) having a camera head to include a signal processing circuit that performs the demosaic process. Therefore, power consumption of the medical endoscope device (a transmission side device) having the camera head can be further reduced.
  • the medical endoscope device (a transmission side device) having the camera head is demanded to achieve lower power consumption, miniaturization, and heating prevention, for example, the demands can be satisfied more easily if the medical observation systems according to the present embodiments is used.
  • a program for causing a computer to function as the medical endoscope device e.g., a program for causing the computer to function as an encoding processing unit and a transmission unit, in other words, a program by which the processes of the transmission methods according to the present embodiments can be executed
  • image data indicating a compression-encoded medical captured image can be transmitted with low latency.
  • the present embodiments can also provide a recording medium in which the program is stored therealong.
  • present technology may also be configured as below.
  • an encoding processing unit configured to compression-encode RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body
  • a transmission unit configured to wirelessly transmit the compression-encoded RAW image data.
  • the encoding processing unit compression-encodes the RAW image data at a plurality of different compression ratios
  • the transmission unit transmits the compression-encoded RAW image data corresponding to a wireless transmission state, of the RAW image data compression-encoded at the plurality of compression ratios.
  • the encoding processing unit compression-encodes each of first RAW image data of a medical captured image for a right eye and second RAW image data of a medical captured image for a left eye
  • the transmission unit wirelessly transmits each of the compression-encoded first RAW image data and the compression-encoded second RAW image data.
  • the encoding processing unit compression-encodes the RAW image data by a predetermined unit that is smaller than the medical captured image
  • the transmission unit transmits the RAW image data compression-encoded by the predetermined unit.
  • a reception unit configured to wirelessly receive compression-encoded RAW image data obtained by compression-encoding RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body;
  • a signal processing unit configured to process the received compression-encoded RAW image data.
  • a medical endoscope device including
  • an encoding processing unit configured to compression-encode RAW image data of a medical captured image of an observation target captured by an imaging device that is inserted into a body of a patient and captures an image of an inside of the body
  • a transmission unit configured to wirelessly transmit the compression-encoded RAW image data
  • a reception unit configured to wirelessly receive the compression-encoded RAW image data
  • a signal processing unit configured to process the received compression-encoded RAW image data.
  • the encoding processing unit of the medical endoscope device compression-encodes the RAW image data without performing a demosaic process
  • the signal processing unit of the medical reception device performs the demosaic process on the compression-encoded RAW image data received by the reception unit or the compression-encoded RAW image data received by an external antenna.

Landscapes

  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Endoscopes (AREA)
  • Studio Devices (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
US16/180,406 2017-12-07 2018-11-05 Medical endoscope device and medical observation system Abandoned US20190174992A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/488,323 US11839353B2 (en) 2017-12-07 2021-09-29 Medical endoscope device and medical observation system
US18/364,486 US20230371785A1 (en) 2017-12-07 2023-08-03 Medical endoscope device and medical observation system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017234812A JP7051408B2 (ja) 2017-12-07 2017-12-07 医療用内視鏡装置、および医療用観察システム
JP2017-234812 2017-12-07

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/488,323 Continuation US11839353B2 (en) 2017-12-07 2021-09-29 Medical endoscope device and medical observation system

Publications (1)

Publication Number Publication Date
US20190174992A1 true US20190174992A1 (en) 2019-06-13

Family

ID=66734358

Family Applications (3)

Application Number Title Priority Date Filing Date
US16/180,406 Abandoned US20190174992A1 (en) 2017-12-07 2018-11-05 Medical endoscope device and medical observation system
US17/488,323 Active 2039-01-24 US11839353B2 (en) 2017-12-07 2021-09-29 Medical endoscope device and medical observation system
US18/364,486 Abandoned US20230371785A1 (en) 2017-12-07 2023-08-03 Medical endoscope device and medical observation system

Family Applications After (2)

Application Number Title Priority Date Filing Date
US17/488,323 Active 2039-01-24 US11839353B2 (en) 2017-12-07 2021-09-29 Medical endoscope device and medical observation system
US18/364,486 Abandoned US20230371785A1 (en) 2017-12-07 2023-08-03 Medical endoscope device and medical observation system

Country Status (2)

Country Link
US (3) US20190174992A1 (https=)
JP (1) JP7051408B2 (https=)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180268523A1 (en) * 2015-12-01 2018-09-20 Sony Corporation Surgery control apparatus, surgery control method, program, and surgery system
US11576563B2 (en) 2016-11-28 2023-02-14 Adaptivendo Llc Endoscope with separable, disposable shaft
US20230232050A1 (en) * 2020-05-31 2023-07-20 Dimension, Inc. Improved superresolution-enabled (sre) video codec
USD1018844S1 (en) 2020-01-09 2024-03-19 Adaptivendo Llc Endoscope handle
USD1031035S1 (en) 2021-04-29 2024-06-11 Adaptivendo Llc Endoscope handle
USD1051380S1 (en) 2020-11-17 2024-11-12 Adaptivendo Llc Endoscope handle
USD1066659S1 (en) 2021-09-24 2025-03-11 Adaptivendo Llc Endoscope handle
USD1070082S1 (en) 2021-04-29 2025-04-08 Adaptivendo Llc Endoscope handle

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102391206B1 (ko) * 2020-05-08 2022-04-27 (주)진성 테크템 횡방향으로 연결된 소포장물의 자동 절단 공급장치
US12489483B2 (en) * 2022-08-10 2025-12-02 Zebra Technologies Corporation Wireless communications assemblies for mitigating radio frequency desensitization

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130244580A1 (en) * 2010-11-10 2013-09-19 Olympus Corporation Wireless communication device and wireless communication system
US8545396B2 (en) * 2006-11-16 2013-10-01 Stryker Corporation Wireless endoscopic camera
US20170034484A1 (en) * 2014-05-22 2017-02-02 Olympus Corporation Wireless endoscope system, endoscope, display device, image transmission method, image display method, and program
US20180047165A1 (en) * 2015-07-03 2018-02-15 Olympus Corporation Image processing apparatus and endoscopic system
US20180220873A1 (en) * 2015-10-08 2018-08-09 Olympus Corporation Endoscope system

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004193683A (ja) * 2002-12-06 2004-07-08 Olympus Corp 電子カメラシステム
JP4503987B2 (ja) * 2003-11-12 2010-07-14 オリンパス株式会社 カプセル型内視鏡
JP2005159490A (ja) * 2003-11-21 2005-06-16 Canon Inc 電子撮影装置
US7602981B2 (en) * 2004-03-09 2009-10-13 Kabushiki Kaisha Toshiba Image storage and display system, maintenance system therefor, and image storage and display method
JP4789570B2 (ja) * 2005-10-07 2011-10-12 オリンパス株式会社 被検体内情報取得装置
US8276035B1 (en) * 2008-07-28 2012-09-25 Netmotion Wireless, Inc. High performance digital communications resiliency in a roamable virtual private network
JP5193931B2 (ja) * 2009-04-20 2013-05-08 富士フイルム株式会社 画像処理装置、画像処理方法およびプログラム
EP3018881A1 (en) * 2010-02-19 2016-05-11 Telefonaktiebolaget LM Ericsson (publ) Method and arrangement for adaptation in http streaming
MX2014001917A (es) * 2011-08-26 2014-04-14 Panasonic Corp Sistema de procesamiento de imagenes, dispositivo de transmision, dispositivo de recepcion, metodo de transmision, metodo de recepcion y programa de computadora.
JP5362156B1 (ja) * 2011-11-11 2013-12-11 オリンパスメディカルシステムズ株式会社 色信号伝送装置、無線映像伝送システム及び送信装置
DE102012205273A1 (de) * 2012-03-30 2013-10-02 Siemens Aktiengesellschaft Medizinische Datenkompression zur Datenverarbeitung in einem Cloud-System
JP6108755B2 (ja) * 2012-10-18 2017-04-05 オリンパス株式会社 撮影機器、撮影画像送信方法及び撮影画像送信プログラム
JP2014128423A (ja) * 2012-12-28 2014-07-10 Olympus Medical Systems Corp 内視鏡システム
US9516197B2 (en) * 2014-10-21 2016-12-06 Pixspan, Inc. Apparatus and method for lossless compression of raw color sensor data from a color array filtered image sensor
JP6300708B2 (ja) * 2014-11-28 2018-03-28 オリンパス株式会社 内視鏡装置及び内視鏡装置の画像伝送方法
JP2017176318A (ja) * 2016-03-29 2017-10-05 ソニー・オリンパスメディカルソリューションズ株式会社 医療用立体観察装置、医療用立体観察方法、プログラム、及び医療用立体観察システム
WO2018225534A1 (ja) * 2017-06-09 2018-12-13 ソニーセミコンダクタソリューションズ株式会社 受信装置、送信装置、制御方法、プログラム、および送受信システム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8545396B2 (en) * 2006-11-16 2013-10-01 Stryker Corporation Wireless endoscopic camera
US20130244580A1 (en) * 2010-11-10 2013-09-19 Olympus Corporation Wireless communication device and wireless communication system
US20170034484A1 (en) * 2014-05-22 2017-02-02 Olympus Corporation Wireless endoscope system, endoscope, display device, image transmission method, image display method, and program
US20180047165A1 (en) * 2015-07-03 2018-02-15 Olympus Corporation Image processing apparatus and endoscopic system
US20180220873A1 (en) * 2015-10-08 2018-08-09 Olympus Corporation Endoscope system

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180268523A1 (en) * 2015-12-01 2018-09-20 Sony Corporation Surgery control apparatus, surgery control method, program, and surgery system
US11127116B2 (en) * 2015-12-01 2021-09-21 Sony Corporation Surgery control apparatus, surgery control method, program, and surgery system
US11576563B2 (en) 2016-11-28 2023-02-14 Adaptivendo Llc Endoscope with separable, disposable shaft
USD1018844S1 (en) 2020-01-09 2024-03-19 Adaptivendo Llc Endoscope handle
US20230232050A1 (en) * 2020-05-31 2023-07-20 Dimension, Inc. Improved superresolution-enabled (sre) video codec
US12432388B2 (en) * 2020-05-31 2025-09-30 Dimension, Inc. Superresolution-enabled (SRE) video CODEC
USD1051380S1 (en) 2020-11-17 2024-11-12 Adaptivendo Llc Endoscope handle
USD1031035S1 (en) 2021-04-29 2024-06-11 Adaptivendo Llc Endoscope handle
USD1070082S1 (en) 2021-04-29 2025-04-08 Adaptivendo Llc Endoscope handle
USD1066659S1 (en) 2021-09-24 2025-03-11 Adaptivendo Llc Endoscope handle

Also Published As

Publication number Publication date
JP7051408B2 (ja) 2022-04-11
US20230371785A1 (en) 2023-11-23
JP2019098048A (ja) 2019-06-24
US20220015608A1 (en) 2022-01-20
US11839353B2 (en) 2023-12-12

Similar Documents

Publication Publication Date Title
US11839353B2 (en) Medical endoscope device and medical observation system
US10799089B2 (en) Observation device and observation system for strengthening stability of communication
CN101541228B (zh) 无线内窥摄像机
JP7311600B2 (ja) フォービエイテッドコーディングのスライスサイズマップ制御
US9119527B2 (en) Image transmitting device and image communication system
US9160985B2 (en) Wireless image communication system and wireless image communication apparatus
US9661081B2 (en) Wireless image transfer system and wireless image transfer method
JP2019098048A5 (https=)
US20190076006A1 (en) Medical observation device and medical observation system
US12569113B2 (en) Medical system, processing protocol control method, and signal processing device
CN222464649U (zh) 远程通信系统以及手术机器人系统
CN103338361A (zh) 一种拍摄系统
US20190099058A1 (en) Medical observation device and medical observation system
US11716544B2 (en) Image capture module and electronic terminal
CN208581314U (zh) 手术过程视频图像数据处理系统
CN104883526B (zh) 一种360度高清图像传输系统及方法
JP5379641B2 (ja) 画像符号化装置及び画像処理システム
US20120086816A1 (en) Speed dome camera

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY OLYMPUS MEDICAL SOLUTIONS INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FUJII, DAISUKE;REEL/FRAME:047410/0693

Effective date: 20181030

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE