CN116507262A

CN116507262A - Endoscope image pickup system and image data transmission device thereof

Info

Publication number: CN116507262A
Application number: CN202080106698.9A
Authority: CN
Inventors: 徐涛; 魏开云
Original assignee: Shenzhen Mindray Bio Medical Electronics Co Ltd
Current assignee: Shenzhen Mindray Bio Medical Electronics Co Ltd
Priority date: 2020-10-27
Filing date: 2020-10-27
Publication date: 2023-07-28
Also published as: WO2022087824A1

Abstract

An endoscope image pickup system and an image data transmission apparatus are realized by a first data processing device (20), an optical fiber transmission component (30) and a second data processing device (40) at the time of image data transmission. The first data processing device (20) is at least used for converting the image data based on the first data communication protocol and output by the image sensor (10) into data based on the second data communication protocol, the data are transmitted to the second data processing device (40) by the optical fiber sensing assembly (30), and the second data processing device (40) converts the acquired image data into image data based on the third data communication protocol and outputs the image data to the image processing part (500) for processing.

Description

Endoscope image pickup system and image data transmission device thereof

Technical Field

The present invention relates to an endoscope imaging system and an image data transmission device thereof.

Background

In recent years, an endoscopic imaging system is increasingly widely used in surgical operations and diagnostic examinations; the endoscope imaging system can provide an image of the inside of a human body to a doctor, and the doctor can perform a surgery or an examination stably and accurately by the image.

The resolution of an endoscopic camera system undergoes a development history of High Definition (HD), full High Definition (FHD) to Ultra High Definition (UHD), or 1K, 2K to 4K. While resolution is continuously improved, there are a number of technical problems to be solved.

For example, as the resolution of an endoscope camera system increases, the amount of data acquired by a sensor that performs image data acquisition increases; the collected data needs to be transmitted to a host of the endoscope camera system for processing through a transmission cable assembly, and in order to ensure real-time performance and display efficiency, the transmission cable assembly is required to be capable of transmitting a large amount of data at high speed and good in anti-interference capability.

The transmission of image data from a sensor to an image processing host in an endoscope camera system is generally realized by a multi-channel twisted pair, which has a plurality of defects, such as excessively thick integral cables, deviation of anti-interference capability and general failure to meet the real-time transmission of image data with large resolution.

Technical problem

Technical solution

An image data transmission device for an endoscopic imaging system, the image data transmission device being configured to be connected to an image processing section in the endoscopic imaging system so as to transmit image data to the image processing section; comprising the following steps:

At least a first image sensor and a second image sensor, wherein the first image sensor and the second image sensor are used for generating image data, and the first image sensor and the second image sensor output the image data based on a first data communication protocol through respective data output channels;

the first data processing device is in communication connection with the data output channels of the first image sensor and the second image sensor; the first data processing device comprises at least a first set of data outputs and a second set of data outputs; the first data processing device is used for converting the image data output by the first image sensor into a first group of image data based on a second data communication protocol and outputting the first group of image data through the first group of data output end, and converting the image data output by the second image sensor into a second group of image data based on a second data communication protocol and outputting the second group of image data through the second group of data output end; the second data communication protocol is different from the first data communication protocol;

an optical fiber transmission assembly comprising an electro-optic transducer, an optical-to-electrical transducer, and at least a first optical fiber transmission channel and a second optical fiber transmission channel; the photoelectric converter is used for converting the first group of image data output by the first data processing device from an electric signal to an optical signal, transmitting the optical signal to the photoelectric converter through the first optical fiber transmission channel, and converting the received first group of image data from the optical signal to the electric signal and outputting the electric signal; the photoelectric converter is further used for converting the second group of image data output by the first data processing device from an electric signal to an optical signal, transmitting the optical signal to the photoelectric converter through the second optical fiber transmission channel, and converting the received second group of image data from the optical signal to the electric signal and outputting the electric signal;

A second data processing device comprising at least a first set of data inputs and a second set of data inputs; the second data processing device is used for receiving a first group of image data output by the photoelectric converter through the first group of data input end and converting the first group of image data into image data based on a third data communication protocol to be output; and the second data processing device is further used for receiving a second group of image data output by the photoelectric converter through the second group of data input end and converting the second group of image data into image data based on a third data communication protocol to be output.

In One embodiment, the first set of data output ends and the second set of data output ends of the first data processing device adopt a V-by-One signal transmission interface standard, the first set of data input ends and the second set of data input ends of the second data processing device adopt a V-by-One signal transmission interface standard, and the second data communication protocol is a protocol based on the V-by-One signal transmission interface standard;

or the first group of data output ends and the second group of data output ends of the first data processing device adopt an FPD-LINK signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt the FPD-LINK signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard.

In an embodiment, the first set of data output terminals and the second set of data output terminals of the first data processing device use an FPD-LINK III signal transmission interface standard, the first set of data input terminals and the second set of data input terminals of the second data processing device use an FPD-LINK III signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK III signal transmission interface standard;

or the first group of data output ends and the second group of data output ends of the first data processing device adopt an FPD-LINK IV signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt an FPD-LINK IV signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK IV signal transmission interface standard.

In an embodiment, the third data communication protocol is the same as the first data communication protocol or the third data communication protocol is different from the first data communication protocol.

In an embodiment, the data output channels of the first image sensor and the second image sensor are MIPI CSI interfaces, and the first data communication protocol is MIPI CSI protocol.

In an embodiment, the second data communication protocol satisfies: the image data based on the second data communication protocol has a signal amplitude required for converting the image data from an electrical signal to an optical signal by the electro-optical converter.

In an embodiment, the first fiber optic transmission channel includes a first optical fiber and the second fiber optic transmission channel includes a second optical fiber;

the electro-optic transducer comprises at least a first input end, a second input end, a first output end and a second output end; the photoelectric converter comprises at least a first input end, a second input end, a first output end and a second output end; the first output end of the photoelectric converter is connected with the first input end of the photoelectric converter through the first optical fiber; the second output end of the photoelectric converter is connected with the second input end of the photoelectric converter through the second optical fiber;

the electro-optic converter receives a first group of image data output by the first data processing device through a first input end of the electro-optic converter and converts the first group of image data from an electric signal to an optical signal; the electro-optic converter outputs a first group of image data converted into optical signals through a first output end of the electro-optic converter and transmits the first group of image data through the first optical fiber; the photoelectric converter receives a first group of image data which is transmitted by the first optical fiber and is converted into an optical signal through a first input end of the photoelectric converter, and converts the first group of image data into an electric signal from the optical signal so as to be output through a first output end of the photoelectric converter;

The electro-optic converter receives a second group of image data output by the first data processing device through a second input end of the electro-optic converter and converts the second group of image data from an electric signal to an optical signal; the electro-optic converter outputs a second group of image data converted into optical signals through a second output end of the electro-optic converter and transmits the second group of image data through the second optical fiber; the photoelectric converter receives a second group of image data which is transmitted by the second optical fiber and is converted into optical signals through a second input end of the photoelectric converter, and converts the second group of image data into electric signals from the optical signals so as to be output through a second output end of the photoelectric converter.

In an embodiment, the electrical-to-optical converter comprises a first electrical-to-optical converter and a second electrical-to-optical converter; the photoelectric converter includes a first photoelectric converter and a first photoelectric converter; the first optical fiber transmission channel comprises a first optical fiber, and the second optical fiber transmission channel comprises a second optical fiber;

the first electro-optic transducer includes a first input and a first output; the second electro-optic transducer includes a second input and a second output; the first photoelectric converter comprises a first input end and a first output end; the second photoelectric converter comprises a second input end and a second output end; the first output end of the first photoelectric converter is connected with the first input end of the first photoelectric converter through the first optical fiber; the second output end of the second photoelectric converter is connected with the second input end of the second photoelectric converter through the second optical fiber;

The first electro-optic converter receives a first group of image data output by the first data processing device through a first input end of the first electro-optic converter and converts the first group of image data from an electric signal to an optical signal; the first electro-optic converter outputs a first group of image data converted into optical signals through a first output end of the first electro-optic converter and transmits the first group of image data through the first optical fiber; the first photoelectric converter receives a first group of image data which is transmitted by the first optical fiber and is converted into an optical signal through a first input end of the first photoelectric converter, and converts the first group of image data into an electric signal from the optical signal so as to be output through a first output end of the first photoelectric converter;

the second electro-optical converter receives a second group of image data output by the first data processing device through a second input end of the second electro-optical converter and converts the second group of image data from an electric signal to an optical signal; the second electro-optic converter outputs a second group of image data converted into optical signals through a second output end of the second electro-optic converter and transmits the second group of image data through the second optical fiber; the second photoelectric converter receives the second group of image data converted into optical signals transmitted by the second optical fiber through a second input end of the second photoelectric converter, and converts the second group of image data into electric signals from the optical signals so as to be output through a second output end of the second photoelectric converter.

an image sensor for generating and outputting image data based on a first data communication protocol;

the first data processing device is at least used for converting the image data output by the image sensor into image data based on a second data communication protocol and outputting the image data; the second data communication protocol is different from the first data communication protocol;

the optical fiber transmission assembly is used for converting the image data output by the first data processing device into an optical signal from an electric signal and transmitting the optical signal, and then converting the optical signal into the electric signal and outputting the electric signal;

a second data processing device for receiving the image data transmitted from the optical fiber transmission assembly and converting the image data into image data based on a third data communication protocol for output;

wherein:

the first data processing device and the second data processing device adopt a V-by-One signal transmission interface standard, and the second data communication protocol is a protocol based on the V-by-One signal transmission interface standard;

Alternatively, an FPD-LINK signal transmission interface standard is adopted between the first data processing device and the second data processing device, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard.

In an embodiment, an FPD-LINK III signal transmission interface standard is adopted between the first data processing device and the second data processing device, and the second data communication protocol is a protocol based on the FPD-LINK III signal transmission interface standard;

or, an FPD-LINK IV signal transmission interface standard is adopted between the first data processing device and the second data processing device, and the second data communication protocol is a protocol based on the FPD-LINK IV signal transmission interface standard.

In an embodiment, the first data communication protocol is the MIPI CSI protocol.

In an embodiment, the second data communication protocol satisfies: the image data based on the second data communication protocol has a signal amplitude required for converting the image data from an electrical signal to an optical signal by the optical fiber transmission assembly.

In one embodiment, the fiber optic transmission assembly includes an electro-optic transducer, a fiber optic transmission channel, and an optical-to-electrical transducer;

the electro-optical converter receives the image data output by the first data processing device and converts the image data from an electric signal to an optical signal to output to the optical fiber transmission channel;

the optical fiber transmission channel is used for transmitting the image data converted into optical signals; the optical fiber transmission channel comprises an optical fiber;

the photoelectric converter receives the image data converted into the optical signal transmitted by the optical fiber transmission channel, converts the image data into an electric signal from the optical signal and outputs the electric signal.

An endoscopic imaging system comprising:

a light source section;

a light source control section for controlling the light source section to provide light required for imaging;

an endoscope including an insertion section that can be inserted into a living body;

an image pickup section including the image data transmission device according to any one of the above;

an image processing unit for receiving and processing the image data outputted from the endoscope data transmission device to generate data for displaying an image;

and the display is used for displaying the data for displaying the image.

In one embodiment, the image processing part includes an FPGA or a CPU.

Drawings

Fig. 1 is a schematic structural diagram of an image data transmission device according to an embodiment;

FIG. 2 is a schematic diagram illustrating an embodiment of an image data transmission device;

FIG. 3 is a schematic diagram illustrating an embodiment of an image data transmission device;

FIG. 4 is a schematic diagram illustrating an embodiment of an image data transmission device;

FIG. 5 is a schematic diagram illustrating an embodiment of an image data transmission device;

FIG. 6 is a schematic diagram illustrating an embodiment of an image data transmission device;

FIG. 7 is a schematic diagram of an endoscopic imaging system according to an embodiment;

FIG. 8 is a schematic diagram of an endoscopic imaging system according to an embodiment;

FIG. 9 is a schematic diagram of an endoscopic imaging system according to an embodiment;

FIG. 10 is a schematic diagram of an endoscopic imaging system according to an embodiment;

FIG. 11 is a schematic diagram of an endoscopic imaging system according to an embodiment;

FIG. 12 is a schematic view of an endoscopic imaging system according to an embodiment;

FIG. 13 is a schematic view of an endoscopic imaging system according to an embodiment;

FIG. 14 is a schematic view of an endoscopic imaging system according to an embodiment;

FIG. 15 is a schematic view of an endoscopic imaging system according to an embodiment;

FIG. 16 is a schematic view of an endoscopic imaging system according to an embodiment;

fig. 17 is a schematic diagram of an endoscopic imaging system according to an embodiment.

Embodiments of the invention

The invention will be described in further detail below with reference to the drawings by means of specific embodiments. Wherein like elements in different embodiments are numbered alike in association. In the following embodiments, numerous specific details are set forth in order to provide a better understanding of the present application. However, one skilled in the art will readily recognize that some of the features may be omitted, or replaced by other elements, materials, or methods in different situations. In some instances, some operations associated with the present application have not been shown or described in the specification to avoid obscuring the core portions of the present application, and may not be necessary for a person skilled in the art to describe in detail the relevant operations based on the description herein and the general knowledge of one skilled in the art.

Furthermore, the described features, operations, or characteristics of the description may be combined in any suitable manner in various embodiments. Also, various steps or acts in the method descriptions may be interchanged or modified in a manner apparent to those of ordinary skill in the art. Thus, the various orders in the description and drawings are for clarity of description of only certain embodiments, and are not meant to be required orders unless otherwise indicated.

The numbering of the components itself, e.g. "first", "second", etc., is used herein merely to distinguish between the described objects and does not have any sequential or technical meaning. The terms "coupled" and "connected," as used herein, are intended to encompass both direct and indirect coupling (coupling), unless otherwise indicated.

As described above, the performance of the current image data acquisition and transmission scheme of the endoscope camera system cannot meet the requirement of high definition or even 4K for real-time camera shooting.

Firstly, the current 4K type image sensor generally adopts an image sensor with sub-LVDS interface, but cannot select an image sensor with MIPI CSI interface with better performance. This is because the image acquisition portion (camera/camera handle) of an endoscopic camera system is constrained by size and power consumption, while the better performing MIPI CSI interface image sensor typically has 4 data channels, which results in a higher rate for a single channel, typically exceeding 1.5Gbps, while the sub-LVDS interface image sensor typically has 8-10 data channels, so that the rate for a single channel of the sub-LVDS interface image sensor can be significantly reduced, e.g., less than 1 Gbps, for the same number of channels. Thus, current endoscopic camera systems cannot directly select an image sensor such as MIPI CSI interface in terms of image acquisition.

Secondly, for the 4K type image sensor, the amount of data collected and to be transmitted is very large, in order to ensure real-time performance, the image data collected by the image sensor needs to be transmitted to the image processing host at high speed, but the current image data transmission realized by a multi-channel twisted pair cannot meet the high-speed real-time transmission, and the anti-interference capability is deviated, and the cable is thick.

In view of the large resolution typical image 4K type endoscope imaging system, there are many technical problems to be solved, and the applicant has studied one or more of these technical problems and proposed some solutions, which will be described in detail below.

In some embodiments, an image data transmission device is provided, and the image data transmission device can be applied to occasions and products such as an endoscope camera system. The image data transmission device of the present invention can be connected to an image processing unit, such as an image processing host, in an endoscopic imaging system, and can transmit image data to the image processing unit, and the image data transmitted to the image processing unit can be processed by the image processing unit to generate data for displaying an image. The image data transmission device may include one or more image sensors, as will be described below.

Referring to fig. 1, an image data transmission apparatus in some embodiments includes an image sensor 10, a first data processing device 20, an optical fiber transmission component 30, and a second data processing device 40, which are described in detail below.

The image sensor 10 is used for generating and outputting image data based on a first data communication protocol. In some embodiments, the image sensor 10 includes at least two data output channels, e.g., 10a and 10b, through which the image sensor outputs image data. The image sensor 10 can generate image data according to a specification which can be processed by an AP (Application Processor ) as a CPU for a mobile device.

The first data processing device 20 is in communication connection with a data output channel of the image sensor 10; the first data processing device 20 is at least used for converting the image data output by the image sensor 10 into image data based on the second data communication protocol and outputting the image data; the second data communication protocol is different from the first data communication protocol.

The optical fiber transmission assembly 30 is configured to convert the image data output by the first data processing device 20 from an electrical signal to an optical signal for transmission, and then convert the image data from the optical signal to an electrical signal for output. The fiber optic transmission assembly 30 serves as a pass-through channel only and does not involve protocol packing and unpacking operations.

Referring to fig. 2, in some embodiments, the fiber optic transmission assembly 30 may include an electrical-to-optical converter 32, a fiber optic transmission channel 39a, and an optical-to-electrical converter 36. The electro-optical converter 32 receives the image data output by the first data processing device 20 and converts the image data from an electrical signal to an optical signal for output to the optical fiber transmission channel 39a; the optical fiber transmission channel 39a is used for transmitting the image data converted into the optical signal, and in some examples, the optical fiber transmission channel 39a includes an optical fiber; the photoelectric converter 36 receives the image data converted into an optical signal transmitted from the optical fiber transmission path 39a, converts the optical signal into an electrical signal, and outputs the electrical signal.

The second data processing device 40 is configured to receive the image data transmitted from the optical fiber transmission assembly 30, and convert the image data into image data based on the third data communication protocol for output.

It can be seen that the first data processing device 20, the optical fiber transmission assembly 30 and the second data processing device 40 cooperate to transmit the image data generated by the image sensor 10, for example, to an image processing unit, which may be an FPGA or other CPU processing platform. The optical fiber transmission assembly 30 mainly performs electro-optical conversion, transmits optical signals through an optical fiber, and performs photoelectric conversion; the first data processing device 20 and the second data processing device 40 cooperate to form a slice-by-slice scheme. The first data processing device 20 and the second data processing device 40 are designed in a product comprising two chips of a chip-on-chip scheme, which are typically outsourced, with a proprietary data communication protocol between them for data transmission, i.e. the second data communication protocol may be a proprietary data communication protocol provided by the chip vendor.

In an embodiment, the image sensor 10 may generate image data with MIPI (Mobile Industry Processor Interface ) specification, for example, the data output channel of the image sensor is MIPI CSI interface, and the first data communication protocol is MIPI CSI protocol. Specifically, the data output channel of the image sensor may be an MIPI CSI-2 interface. MIPI is a consortium established in 2003 by companies such as ARM, nokia, ST, and TI, usa, texas instruments, and the like, with the aim of standardizing interfaces inside a mobile phone such as cameras, display interfaces, radio frequency/baseband interfaces, and the like, thereby reducing the complexity of the mobile phone design and increasing the design flexibility. There are different working groups under the MIPI alliance, which respectively define a series of handset internal interface standards, such as a camera interface CSI (Camera Serial Interface), a display interface DSI (Display Serial Interface), a radio frequency interface DigRF, a microphone/speaker interface SLIMbus, etc. The end market demands lower power consumption, higher data transmission rates and smaller PCB footprints, and the MIPI interface is suitable for use in devices that are power sensitive while requiring high performance, among several standards-based serial differential interfaces that are now in use. Specifically, as described above, CSI is an interface standard specified by the camera work group under MIPI alliance; the CSI-2 is a second version of MIPI CSI, mainly comprises an application layer, a protocol layer and a physical layer, and generally supports 4-channel data transmission, has a single-line transmission speed of up to 1Gb/s and also supports 8-channel data transmission. In addition to ground, the MIPI CSI-2 interface typically has 1 pair of I2C communication pins, 1 pair of MIPI differential clock pins and 1-4 pairs of MIPI differential data signal pins.

Typically, the camera of an endoscopic camera system is hand-held, and a physician holds the camera while performing an operation on a patient to adjust the viewing site and control parameters. Because the image sensor transmits a large amount of data, the transmission power is high, and simultaneously, larger heat is generated. The camera needs to be designed to have the characteristics of low power consumption, less heating and the like. The image sensor of the MIPI interface has the characteristic of low power consumption, and the design requirement of a camera is just met.

Therefore, in order to reduce the power consumption of the related devices, the present embodiment employs an image sensor with a MIPI CSI interface having lower power consumption. However, when the optical fiber transmission assembly converts the image data from an electrical signal to an optical signal, a signal swing of the image data is required. For example, the optical fiber transmission assembly converts data from an electrical signal to an optical signal, which generally requires that the signal swing of the data is 200mv at a lower limit, whereas the signal swing of MIPI CSI data is generally less than 200mv, and the optical fiber transmission assembly cannot be supported to convert the data from an electrical signal to an optical signal due to loss in the transmission process. Thus, in this embodiment, the second data communication protocol satisfies: the image data based on the second data communication protocol has a signal amplitude required for converting the image data from an electrical signal to an optical signal by the optical fiber transmission assembly.

In an embodiment, when the image sensor is an 8-channel MIPI interface, the image sensor outputs image data to the first data processing device through the 8-channel MIPI interface, and the first data processing device processes the acquired image data, converts the processed image data into one-channel image data, and outputs the one-channel image data. Correspondingly, the optical fiber transmission assembly converts the image data into optical signals from electric signals and transmits the optical signals through one optical fiber.

In another embodiment, when the image sensor is an 8-channel MIPI interface, the image sensor outputs image data to the first data processing device through the 8-channel MIPI interface, and the first data processing device processes the acquired image data, converts the processed image data into two paths of image data, and outputs the two paths of image data. Correspondingly, the optical fiber transmission assembly converts the two paths of image data from electric signals to optical signals and transmits the optical signals through the two paths of optical fibers respectively.

In some embodiments, the third data communication protocol is the same as the first data communication protocol. For example, when the image sensor 10 outputs image data of MIPI specification, the image data converted by the second data processing device 40 is also image data of MIPI specification. In some embodiments, the third data communication protocol is different from the first data communication protocol as long as data based on the third data communication protocol can be recognized and processed by the image processing section.

In one embodiment, the number of channels of the image data processed by the first data processing device is smaller than the number of channels of the image data input by the first data processing device.

One data output channel of the image sensor 10 may correspond to one pin of the image sensor 10, or may correspond to a plurality of pins (for example, 2 pins) of the image sensor 10. Because the image data generated by the image sensor is larger and is difficult to output through one data output channel, the generated image data is usually output through a plurality of data output channels so as to reduce the speed of each output channel; at the same time, after the rate of each output channel is reduced, the data processing capability and the power consumption requirement of the next stage processing unit (such as the first data processing device 20) can be adapted to a larger extent.

In some examples, a V-by-One signaling interface standard is employed between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the V-by-One signaling interface standard. V-by-One is a signal transmission interface standard developed by Saen electronics (THine Electronics) of Japan, which can be used for high definition digital image signal transmission, and consists of 1 to 8 sets of pairing signals; specifically, the V-by-One adopts 8B/10B coding, and is better compatible with other high-speed serial such as PCI Express and USB 3.0; and the direct current balance can be effectively solved by converting the 8Bit data into the 10Bit data. The V-by-One solves the problem of wiring time lag, greatly reduces EMI interference, improves the maximum transmission speed of each group of signals (for example, 3.75 Gbps), greatly reduces the number of transmission lines and saves PCB space. For example, an ultra-high-definition UHD display screen with resolution not lower than 3840X2160, if the LVDS protocol standard is adopted, the data line can be up to 48 pairs; if the V-by-On protocol standard is adopted, only 8 pairs of data lines are needed.

In some examples, the FPD-LINK signal transmission interface standard is used between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard. Further, the FPD-LINK III/IV signal transmission interface standard is adopted between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the FPD-LINK III/IV signal transmission interface standard. FPD-Link III is an iteration performed on the basis of FPD-Link II, the main function of which is to embed in the same bad bi-directional communication channel pair. The III of FPD-Link reduces the cable cost further than II by eliminating control channel cables, such as I2C and CAN buses. FPD-Link III, compared to II, ceases to use LVDS technology and uses only CML for the serialized high-speed signals. This makes it possible to easily support data transmission rates of more than 3 Gbit/s over transmission lines with cables longer than 10 m. The advantage of using CML for FPD-Link III is that the coaxial cable driving capability is utilized, and in addition, since the coaxial cable can be made very good in controlling impedance and noise, the differential signal can be reduced, and the required interference of impedance discontinuity and noise can be better tolerated. The FPD-LINK III is characterized by supporting full duplex control for high-speed video data transmission and bi-directional control communication through a single differential LINK, and integrating video data and control through a single differential pair can reduce the size and weight of interconnect lines while also eliminating the problem of deviation and simplifying the system design. FPD-LINK III is commonly used as an interface for automotive applications, enabling point-to-point video transmission. Compared with the FPD-LINK III, the FPD-LINK IV improves the data transmission rate of a single channel.

Referring to fig. 3, the image data transmission apparatus in some embodiments includes a plurality of image sensors, such as a first image sensor 11 and a second image sensor 12, and further includes a first data processing device 20, an optical fiber transmission component 30, and a second data processing device 40, which will be described in detail below. It should be noted that, although the image data transmission device includes two image sensors in the drawings, this is not intended to limit the number of image sensors to only two, in actual situations, N image sensors may be configured according to the requirement, where N may be 2 or an integer greater than 2.

The first image sensor 11 and the second image sensor 12 are both used for generating image data; the first image sensor 11 and the second image sensor 12 each include at least two data output channels, e.g., 10a and 10b, and the first image sensor 11 and the second image sensor 12 output image data based on the first data communication protocol through the respective at least two data output channels. For example, the first image sensor 11 outputs and images data through at least two data output channels thereof, for example, 10a and 10b, and the first image sensor 12 outputs image data through at least two data output channels thereof, for example, 10a and 10 b.

The first image sensor 11 and the second image sensor 12 can generate image data generated in accordance with specifications which can be processed by an AP (Application Processor ) as a CPU for a mobile device. For example, the first image sensor 11 and the second image sensor 12 may generate image data with MIPI (Mobile Industry Processor Interface ) specification, and accordingly, the data output channel of the image sensor may be MIPI CSI-2 interface.

The first data processing device 20 is communicatively connected to the data output channels of the first image sensor 11 and the second image sensor 12. The first data processing device 20 comprises at least a first set of data outputs 20a and a second set of data outputs 20b; the first data processing device 20 is configured to convert the image data output by the first image sensor 11 into a first set of image data based on the second data communication protocol and output the first set of image data through the first set of data output terminals 20a, and convert the image data output by the second image sensor 12 into a second set of image data based on the second data communication protocol and output the second set of image data through the second set of data output terminals 20 b. In an embodiment, the second data communication protocol is different from the first data communication protocol.

Referring to fig. 4, the optical fiber transmission assembly 30 includes an electro-optical converter 31, an optical-electrical converter 35, and at least a first optical fiber transmission channel 39b and a second optical fiber transmission channel 39c. The electro-optical converter 31 is configured to convert the first set of image data output by the first data processing device 20 from an electrical signal to an optical signal, and transmit the optical signal to the electro-optical converter 35 through the first optical fiber transmission channel 39b, and the electro-optical converter 35 converts the received first set of image data from the optical signal to an electrical signal and outputs the electrical signal; the electro-optical converter 31 is further configured to convert the second set of image data output by the first data processing device 20 from an electrical signal to an optical signal, and transmit the optical signal to the electro-optical converter 35 through the second optical fiber transmission channel 39c, and the electro-optical converter 35 converts the received second set of image data from the optical signal to an electrical signal and outputs the electrical signal. It can be seen that the first fiber optic transmission channel 39b and the second fiber optic transmission channel 39c are two separate signal transmission channels, as described in detail below.

Referring to fig. 5, in some embodiments, the first optical fiber transmission channel 39b includes a first optical fiber 39bg, and the second optical fiber transmission channel 39c includes a second optical fiber 39cg. Specifically, the electro-optic transducer 32 includes at least a first input 32a, a second input 32b, a first output 32c, and a second output 32d; the photoelectric converter 36 includes at least a first input terminal 36a, a second input terminal 36b, a first output terminal 36c, and a second output terminal 36d; the first output 32c of the electro-optic transducer 32 is connected to the first input 36a of the electro-optic transducer 36 by a first optical fiber 39 bg; the second output 32b of the electro-optic transducer 32 is connected to the second input 36b of the electro-optic transducer 36 by a second optical fiber 39cg. The first input 32a, the second input 32b of the electro-optic transducer 32 are connected to the first set of data outputs 20a, the second set of data outputs 20b of the first data processing device 20, respectively.

Thus, the electro-optic transducer 32 receives the first set of image data output by the first data processing device 20 via its first input 32a and converts the first set of image data from an electrical signal to an optical signal; the electro-optic transducer 32 outputs a first set of image data converted into optical signals through a first output end 32c thereof, and transmits the first set of image data through a first optical fiber 39 bg; the photoelectric converter 36 receives the first set of image data converted into an optical signal transmitted from the first optical fiber 39bg through the first input end 36a thereof, and converts the first set of image data from an optical signal into an electrical signal to be output through the first output end 36c thereof. Similarly, the electro-optic transducer 32 receives a second set of image data output by the first data processing device 20 via its second input 32b and converts the second set of image data from an electrical signal to an optical signal; the electro-optic transducer 32 outputs the second set of image data converted into optical signals through its second output end 32b and transmits it through the second optical fiber 39 cg; the photoelectric converter 36 receives the second set of image data converted into an optical signal transmitted from the second optical fiber 39cg through its second input terminal 36b, and converts the second set of image data from an optical signal into an electrical signal to be output through its second output terminal 36 d.

Referring to fig. 6, in some embodiments, the electro-optic transducer 31 includes a first electro-optic transducer 33 and a second electro-optic transducer 34; the photoelectric converter 35 includes a first photoelectric converter 37 and a first photoelectric converter 38; the first optical fiber transmission channel 39b includes a first optical fiber 39bg, and the second optical fiber transmission channel 39c includes a second optical fiber 39cg. Specifically, the first electro-optic transducer 33 comprises a first input 33a and a first output 33b; the second electro-optic transducer 34 includes a second input 34a and a second output 34b; the first photoelectric converter 37 includes a first input terminal 37a and a first output terminal 37b; the second photoelectric converter 38 includes a second input terminal 38a and a second output terminal 38b; the first output 33b of the first electro-optic converter 33 is connected to the first input 37a of the first electro-optic converter 37 by a first optical fiber 39 bg; the second output 34a of the second electro-optic converter 34 is connected to the second input 38a of the second electro-optic converter 38 via a second optical fiber 39cg. The first input 33a of the first electro-optical converter 33 is connected to the first set of data outputs 20a of the first data processing device 20 and the second input 34a of the second electro-optical converter 34 is connected to the second set of data outputs 20b of the first data processing device 20.

Thus, the first electro-optic transducer 33 receives, via its first input 33a, a first set of image data output by the first data processing device 20 and converts the first set of image data from an electrical signal to an optical signal; the first electro-optic converter 33 outputs the first set of image data converted into optical signals through the first output end 33b thereof, and transmits the first set of image data through the first optical fiber 39 bg; the first photoelectric converter 37 receives the first set of image data converted into an optical signal transmitted from the first optical fiber 39bg through the first input end 37a thereof, and converts the first set of image data from an optical signal into an electrical signal to be output through the first output end 37b thereof. Similarly, the second electro-optic transducer 34 receives a second set of image data output by the first data processing device 20 via its second input 34a and converts the second set of image data from an electrical signal to an optical signal; the second electro-optic transducer 34 outputs a second set of image data converted into optical signals through a second output end 34b thereof and transmits the second set of image data through a second optical fiber 39 cg; the second photoelectric converter 38 receives the second set of image data converted into optical signals transmitted from the second optical fiber 39cg through the second input terminal 38a thereof, and converts the second set of image data from optical signals into electrical signals for output through the second output terminal 38b thereof.

The above are some illustrations of the fiber optic transmission assembly 30.

The second data processing device 40 comprises at least a first set of data inputs 40a and a second set of data inputs 40b; the second data processing device 40 is configured to receive the first set of image data output by the photoelectric converter 20 through the first set of data input terminals 40a thereof, and convert the first set of image data into image data based on the third data communication protocol for output; and the second data processing device 40 is further configured to receive the second set of image data output by the photoelectric converter through the second set of data input terminals 40b thereof, and convert the second set of image data into image data based on the third data communication protocol for output.

It can be seen that the first data processing device 20, the optical fiber transmission assembly 30 and the second data processing device 40 cooperate to transmit the image data generated by the image sensors, for example, the first image sensor 11 and the second image sensor 12, for example, to an image processing unit, which may be an FPGA or other CPU processing platform. The optical fiber transmission assembly 30 mainly performs electro-optical conversion, transmits signals through an optical fiber, and performs photoelectric conversion; the first data processing device 20 and the second data processing device 40 cooperate to form a slice-by-slice scheme. The first data processing device 20 and the second data processing device 40 are designed in a product comprising two chips of a chip-on-chip scheme, which are typically outsourced, with a proprietary data communication protocol between them for data transmission, i.e. the second data communication protocol may be a proprietary data communication protocol provided by the chip vendor.

In some examples, a V-by-One signaling interface standard is employed between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the V-by-One signaling interface standard.

In some examples, the FPD-LINK signal transmission interface standard is used between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard. Further, the FPD-LINK III/IV signal transmission interface standard is adopted between the first data processing device 20 and the second data processing device 40, and the second data communication protocol is a protocol based on the FPD-LINK III/IV signal transmission interface standard.

The above description is given of the image data transmission device according to the present invention, and the image data transmission device according to the present invention can be applied to a field and a product such as an endoscopic imaging system, and the following description is given by taking an endoscopic imaging field as an example.

Referring to fig. 7, 8 and 9, the endoscope image capturing system in some embodiments includes a light source section 100, a light source control section 200, an endoscope 300, an image capturing section 320, an endoscope data transmission device 400, an image processing section 500 and a display 600, which will be described in detail below.

The light source section 100 is for providing an illumination light source to a site to be observed. The light source section 100 may provide light necessary for normal light imaging, and may also provide light necessary for special light imaging. For example, the illumination light source provided by the light source unit 100 to the site to be observed may be a normal light illumination based on a wide-band light and a special light illumination based on a narrow-band light, the endoscope system generates a color image in the normal illumination mode, generates a monochrome image having a blood vessel enhancement effect in the special illumination mode, and generates a color image based on the gradation value of the monochrome image—it is understood that such a color image generated from a monochrome image such as a gradation image is a pseudo-color image, that is, the special light image is a pseudo-color image at this time.

In some embodiments, the light source part 100 may include a first light source 110 and a second light source 120. In the normal illumination mode, the first light source 110 may provide a plurality of monochromatic lights in different wavelength ranges in a time-sharing manner, for example, the first light source 110 may be a semiconductor light source or an LED light source, and the provided monochromatic lights may be blue light, green light, red light, and the like. In other embodiments, the first light source 110 may also provide a combined light of the plurality of monochromatic lights, or a broad spectrum white light source. The monochromatic light has a wavelength in the range of approximately 400nm to 700nm. In the special illumination mode, the second light source 120 provides narrowband light. For example, the second light source 120 may be a laser emitting a narrow-band blue laser, and the peak wavelength takes at least any 1 value of blue light in the 390nm-460nm range. In other embodiments, the second light source 120 may also be an LED light source or a laser LED, and the emitted narrowband light may be a narrowband green laser, or the like.

In some embodiments, the light source part 100 may further include a dichroic mirror 130, and the first and second light sources 110 and 120 are operated time-divisionally under the control of the light source control part 200; that is, when the first light source 110 is turned on, the second light source 120 is turned off. And vice versa. The dichroic mirror 130 is disposed on the transmission paths of the plurality of monochromatic lights and the narrowband light, and the paths of the plurality of monochromatic lights and the narrowband light are combined into the same path through the dichroic mirror 130. For example, a plurality of monochromatic lights may transmit the dichroic mirror 130, and narrowband light may be reflected by the dichroic mirror 130 so that the optical paths of both are combined into the same optical path; and vice versa. On the optical path after the dichroic mirror 130, the narrowband light and the plurality of monochromatic lights are transmitted in the direction of the endoscope 300 along the same optical path combined in a time-sharing manner.

In some embodiments, the light source section 100 further includes a coupling mirror 140 disposed at the dichroic mirror 130 and the light source introduction port of the endoscope 300. The coupling mirror 140 focuses the light transmitted from the dichroic mirror 130 to be better guided into the endoscope 300, thereby reducing light loss as much as possible and improving the overall illumination quality of the system. Both the light path combining action of the dichroic mirror 130 and the focusing action of the coupling mirror 140 can better guide light into the endoscope 300. Meanwhile, the use of the dichroic mirror 130 can make the overall structure of the light source section 100 more compact and the light propagation path shorter.

The above is some description of the light source section 100. The light source control section 200 is used to control the light source section 100, for example, to control the light source section 100 to provide light necessary for normal light imaging, and to control the light source section 100 to provide light necessary for special light imaging.

The endoscope 300 is used to transmit optical signals. In some embodiments, endoscope 300 may include an insertion portion 310. In some embodiments, the insertion portion 310 can be inserted into a living body, for example, a mirror body of which the insertion portion 310 is a part, and can be inserted into the living body by an operator. The insertion portion 310 can transmit light generated by the light source portion 100 to an introduction portion (which may be a light guide fiber) of a site to be observed.

The image pickup unit 320 includes at least one sensor for generating image data. For example, in some examples, the image capturing section 320 may include the image sensor 10. For another example, the image capturing unit 320 may include the first image sensor 11 and the second image sensor 12. For another example, the image capturing unit 320 may include N sensors for generating image data, where N may be an integer greater than 2.

Referring to fig. 10, an example in which the image pickup unit 320 includes the image sensor 10 is shown. In some examples, the image sensor 10 is configured to generate image data based on a first data communication protocol. Specifically, the image sensor 10 may include at least two data output channels, e.g., 10a and 10b, through which the image sensor outputs image data. In some examples, the image sensor 10 may generate image data according to specifications that can be handled by an AP (Application Processor ) as a CPU for a mobile device. For example, the image sensor 10 may generate image data of MIPI specification, and accordingly, the data output channel of the image sensor may be MIPI CSI-2 interface.

Referring to fig. 11, an example is shown in which the image pickup unit 320 includes a first image sensor 11 and a second image sensor 12. In some examples, the first image sensor 11 and the second image sensor 12 are both configured to generate image data, and the first image sensor 11 and the second image sensor 12 each include at least two data output channels, such as 10a and 10b, through which the first image sensor 11 and the second image sensor 12 output the image data based on the first data communication protocol. For example, the first image sensor 11 outputs image data through at least two data output channels thereof, for example, 10a and 10b, and the first image sensor 12 outputs image data through at least two data output channels thereof, for example, 10a and 10 b. In some examples, the first image sensor 11 and the second image sensor 12 may generate image data according to specifications that can be processed by an AP (Application Processor ) as a CPU for a mobile device. For example, the first image sensor 11 and the second image sensor 12 may generate image data of MIPI specification, and accordingly, the data output channel of the image sensor may be MIPI CSI-2 interface.

In some examples, one end of the image capturing unit 320 is connected to the endoscope data transmission device 400, so that the image processing unit 500 can provide image data, one end of the image capturing unit 320 can be clamped to the endoscope, the light source unit 100 provides a light source for the endoscope 300, and the image capturing unit 320 can obtain an optical signal of the endoscope. The following describes the endoscope data transmission apparatus 400.

The endoscope data transmission device 400 is used to transmit the image data generated by the image pickup unit 320 to the subsequent image processing unit 500 for processing. There are various implementations of the endoscope data transmission device 400, and the detailed description will be given below.

The configuration and function of the endoscope data transmission device 400 in this embodiment will be described with reference to an example in which the image pickup unit 320 includes the image sensor 10. Referring to fig. 12, in this case, an endoscopic data transmission apparatus 400 may include a first data processing device 20, an optical fiber transmission assembly 30, and a second data processing device 40.

The optical fiber transmission assembly 30 is configured to convert the image data output by the first data processing device 20 from an electrical signal to an optical signal for transmission, and then convert the image data from the optical signal to an electrical signal for output.

Referring to fig. 13, in some embodiments, the fiber optic transmission assembly 30 may include an electrical-to-optical converter 32, a fiber optic transmission channel 39a, and an optical-to-electrical converter 36. The electro-optical converter 32 receives the image data output by the first data processing device 20 and converts the image data from an electrical signal to an optical signal for output to the optical fiber transmission channel 39a; the optical fiber transmission channel 39a is used for transmitting the image data converted into the optical signal, and in some examples, the optical fiber transmission channel 39a includes an optical fiber; the photoelectric converter 36 receives the image data converted into an optical signal transmitted from the optical fiber transmission path 39a, converts the optical signal into an electrical signal, and outputs the electrical signal.

It can be seen that the first data processing device 20, the optical fiber transmission assembly 30 and the second data processing device 40 cooperate to transmit the image data generated by the image sensor 10, for example, to an image processing unit, which may be an FPGA or other CPU processing platform. The optical fiber transmission assembly 30 mainly performs electro-optical conversion, transmits signals through an optical fiber, and performs photoelectric conversion; the first data processing device 20 and the second data processing device 40 cooperate to form a slice-by-slice scheme. The first data processing device 20 and the second data processing device 40 are designed in a product comprising two chips of a chip-on-chip scheme, which are typically outsourced, with a proprietary data communication protocol between them for data transmission, i.e. the second data communication protocol may be a proprietary data communication protocol provided by the chip vendor.

The configuration and function of the endoscope data transmission device 400 in this embodiment will be described with respect to an example in which the image pickup unit 320 includes the first image sensor 11 and the second image sensor 12. Referring to fig. 14, in this case, an endoscopic data transmission apparatus 400 may include a first data processing device 20, an optical fiber transmission assembly 30, and a second data processing device 40.

The first data processing device 20 is communicatively connected to the data output channels of the first image sensor 11 and the second image sensor 12. The first data processing device 20 comprises at least a first set of data outputs 20a and a second set of data outputs 20b; the first data processing device 20 is configured to convert the image data output by the first image sensor 11 into a first set of image data based on the second data communication protocol and output the first set of image data through the first set of data output terminals 20a, and convert the image data output by the second image sensor 12 into a second set of image data based on the second data communication protocol and output the second set of image data through the second set of data output terminals 20 b.

It should be noted that the first set of data output terminals 20a and the second set of data output terminals 20b may be one output terminal or may be a plurality of output terminals. Correspondingly, the first group of image data and the second group of image data can be one-path image data or multi-path image data.

Referring to fig. 15, the optical fiber transmission assembly 30 includes an electro-optical converter 31, an optical-electrical converter 35, and at least a first optical fiber transmission channel 39b and a second optical fiber transmission channel 39c. The electro-optical converter 31 is configured to convert the first set of image data output by the first data processing device 20 from an electrical signal to an optical signal, and transmit the optical signal to the electro-optical converter 35 through the first optical fiber transmission channel 39b, and the electro-optical converter 35 converts the received first set of image data from the optical signal to an electrical signal and outputs the electrical signal; the electro-optical converter 31 is further configured to convert the second set of image data output by the first data processing device 20 from an electrical signal to an optical signal, and transmit the optical signal to the electro-optical converter 35 through the second optical fiber transmission channel 39c, and the electro-optical converter 35 converts the received second set of image data from the optical signal to an electrical signal and outputs the electrical signal. It can be seen that the first fiber optic transmission channel 39b and the second fiber optic transmission channel 39c are two separate signal transmission channels, as described in detail below.

In one embodiment, when the first image sensor and the second image sensor are 8-channel MIPI interfaces, the first image sensor and the second image sensor output image data to the first data processing device through the 8-channel MIPI interfaces. The first data processing device processes the acquired image data of the first image sensor, converts the processed image data into one path of image data (first group of image data) and outputs the one path of image data. Correspondingly, the optical fiber transmission assembly converts the image data into optical signals from electric signals and transmits the optical signals through one optical fiber. The first data processing device processes the acquired image data of the second image sensor, converts the processed image data into one path of image data (second group of image data) and outputs the one path of image data. Correspondingly, the optical fiber transmission assembly converts the image data into optical signals from electric signals and transmits the optical signals through one optical fiber.

In another embodiment, when the first image sensor and the second image sensor are 8-channel MIPI interfaces, the image sensor outputs image data to the first data processing device through the 8-channel MIPI interfaces. The first data processing device processes the acquired image data of the first image sensor, converts the processed image data into two paths of image data (first group of image data) and outputs the two paths of image data. Correspondingly, the optical fiber transmission assembly converts the two paths of image data from electric signals to optical signals and transmits the optical signals through the two paths of optical fibers respectively. The first data processing device processes the acquired image data of the second image sensor, converts the processed image data into two paths of image data (second group of image data) and outputs the two paths of image data. Correspondingly, the optical fiber transmission assembly converts the two paths of image data from electric signals to optical signals and transmits the optical signals through the two paths of optical fibers respectively.

Generally, when there are a plurality of image sensors, the first data processing device converts the image data of each image sensor into a set of corresponding image data.

Referring to fig. 16, in some embodiments, the electro-optic transducer 31 includes an electro-optic transducer 32, and the electro-optic transducer 35 includes an electro-optic transducer 36; the first optical fiber transmission channel 39b includes a first optical fiber 39bg, and the second optical fiber transmission channel 39c includes a second optical fiber 39cg. Specifically, the electro-optic transducer 32 includes at least a first input 32a, a second input 32b, a first output 32c, and a second output 32d; the photoelectric converter 36 includes at least a first input terminal 36a, a second input terminal 36b, a first output terminal 36c, and a second output terminal 36d; the first output 32c of the electro-optic transducer 32 is connected to the first input 36a of the electro-optic transducer 36 by a first optical fiber 39 bg; the second output 32b of the electro-optic transducer 32 is connected to the second input 36b of the electro-optic transducer 36 by a second optical fiber 39cg. The first input 32a, the second input 32b of the electro-optic transducer 32 are connected to the first set of data outputs 20a, the second set of data outputs 20b of the first data processing device 20, respectively.

Referring to fig. 17, in some embodiments, the electro-optic transducer 31 includes a first electro-optic transducer 33 and a second electro-optic transducer 34; the photoelectric converter 35 includes a first photoelectric converter 37 and a first photoelectric converter 38; the first optical fiber transmission channel 39b includes a first optical fiber 39bg, and the second optical fiber transmission channel 39c includes a second optical fiber 39cg. Specifically, the first electro-optic transducer 33 comprises a first input 33a and a first output 33b; the second electro-optic transducer 34 includes a second input 34a and a second output 34b; the first photoelectric converter 37 includes a first input terminal 37a and a first output terminal 37b; the second photoelectric converter 38 includes a second input terminal 38a and a second output terminal 38b; the first output 33b of the first electro-optic converter 33 is connected to the first input 37a of the first electro-optic converter 37 by a first optical fiber 39 bg; the second output 34a of the second electro-optic converter 34 is connected to the second input 38a of the second electro-optic converter 38 via a second optical fiber 39cg. The first input 33a of the first electro-optical converter 33 is connected to the first set of data outputs 20a of the first data processing device 20 and the second input 34a of the second electro-optical converter 34 is connected to the second set of data outputs 20b of the first data processing device 20.

The above are some illustrations of the fiber optic transmission assembly 30.

In one embodiment, the number of first set of data inputs 40a is the same as the number of first set of data outputs 20a, and the number of second set of data inputs 40b is the same as the number of second set of data outputs 20 b.

The above are some illustrations of an endoscopic data transmission device 400. In other embodiments, the image capturing portion 320 may further include 3 or more image sensors, and the transmission principle of the image data can be seen from 2 image sensors (the first image sensor 11 and the second image sensor 12).

The image processing unit 500 may be used as an image processing host of the endoscope image capturing system, and the image processing unit 500 may be configured to receive and process the image data output from the endoscope data transmission device 400 to generate data for displaying an image. In some embodiments, the image processing portion 500 includes an FPGA or a CPU.

The display 600 is used for displaying the data for displaying images.

The foregoing is illustrative of an endoscopic imaging system in some embodiments of the invention. It will be appreciated by those skilled in the art that fig. 7-17 are merely examples of an endoscopic imaging system and are not limiting of an endoscopic imaging system, and an endoscopic imaging system may include more or fewer components than shown in fig. 7-17, or may combine certain components, or different components, e.g., an endoscopic imaging system may also include dilators, smoke control devices, input-output devices, network access devices, etc. In addition, the same reference numerals in FIGS. 7-17 as in FIGS. 1-6 may be used for the same components, and specific description may be found in the corresponding embodiment of FIGS. 1-6.

Reference is made to various exemplary embodiments herein. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope herein. For example, the various operational steps and components used to perform the operational steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or combined into other steps) depending on the particular application or taking into account any number of cost functions associated with the operation of the system.

While the principles herein have been shown in various embodiments, many modifications of structure, arrangement, proportions, elements, materials, and components, which are particularly adapted to specific environments and operative requirements, may be used without departing from the principles and scope of the present disclosure. The above modifications and other changes or modifications are intended to be included within the scope of this document.

The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes may be made without departing from the scope of the present disclosure. Accordingly, the present disclosure is to be considered as illustrative and not restrictive in character, and all such modifications are intended to be included within the scope thereof. Also, advantages, other advantages, and solutions to problems have been described above with regard to various embodiments. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, system, article, or apparatus. Furthermore, the term "couple" and any other variants thereof are used herein to refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections, and/or any other connection.

Those skilled in the art will recognize that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. Accordingly, the scope of the invention should be determined only by the following claims.

Claims

An image data transmission device for an endoscopic imaging system, the image data transmission device being configured to be connected to an image processing section in the endoscopic imaging system so as to transmit image data to the image processing section; characterized by comprising the following steps:

at least a first image sensor and a second image sensor, wherein the first image sensor and the second image sensor are used for generating image data, and the first image sensor and the second image sensor output the image data based on a first data communication protocol through respective data output channels;

the first data processing device is in communication connection with the data output channels of the first image sensor and the second image sensor; the first data processing device comprises at least a first set of data outputs and a second set of data outputs; the first data processing device is used for converting the image data output by the first image sensor into a first group of image data based on a second data communication protocol and outputting the first group of image data through the first group of data output end, and converting the image data output by the second image sensor into a second group of image data based on a second data communication protocol and outputting the second group of image data through the second group of data output end; the second data communication protocol is different from the first data communication protocol;

An optical fiber transmission assembly comprising an electro-optic transducer, an optical-to-electrical transducer, and at least a first optical fiber transmission channel and a second optical fiber transmission channel; the photoelectric converter is used for converting the first group of image data output by the first data processing device from an electric signal to an optical signal, transmitting the optical signal to the photoelectric converter through the first optical fiber transmission channel, and converting the received first group of image data from the optical signal to the electric signal and outputting the electric signal; the photoelectric converter is further used for converting the second group of image data output by the first data processing device from an electric signal to an optical signal, transmitting the optical signal to the photoelectric converter through the second optical fiber transmission channel, and converting the received second group of image data from the optical signal to the electric signal and outputting the electric signal;

a second data processing device comprising at least a first set of data inputs and a second set of data inputs; the second data processing device is used for receiving a first group of image data output by the photoelectric converter through the first group of data input end and converting the first group of image data into image data based on a third data communication protocol to be output; and the second data processing device is further used for receiving a second group of image data output by the photoelectric converter through the second group of data input end and converting the second group of image data into image data based on a third data communication protocol to be output.
The image data transmission device of claim 1, wherein,

the first group of data output ends and the second group of data output ends of the first data processing device adopt a V-by-One signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt a V-by-One signal transmission interface standard, and the second data communication protocol is a protocol based on the V-by-One signal transmission interface standard;

or the first group of data output ends and the second group of data output ends of the first data processing device adopt an FPD-LINK signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt the FPD-LINK signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard.
The image data transmission device according to claim 2, wherein:

the first group of data output ends and the second group of data output ends of the first data processing device adopt an FPD-LINK III signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt an FPD-LINK III signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK III signal transmission interface standard;

Or the first group of data output ends and the second group of data output ends of the first data processing device adopt an FPD-LINK IV signal transmission interface standard, the first group of data input ends and the second group of data input ends of the second data processing device adopt an FPD-LINK IV signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK IV signal transmission interface standard.
The video data transmission device according to claim 1, wherein the third data communication protocol is the same as the first data communication protocol or the third data communication protocol is different from the first data communication protocol.
The image data transmission device according to any one of claims 1 to 4, wherein the data output channels of the first image sensor and the second image sensor are MIPI CSI interfaces, and the first data communication protocol is MIPI CSI protocol.
The image data transmission device of claim 5, wherein the second data communication protocol satisfies: the image data based on the second data communication protocol has a signal amplitude required for converting the image data from an electrical signal to an optical signal by the electro-optical converter.
The image data transmission device of claim 1, wherein the first optical fiber transmission channel comprises a first optical fiber and the second optical fiber transmission channel comprises a second optical fiber;

the electro-optic transducer comprises at least a first input end, a second input end, a first output end and a second output end; the photoelectric converter comprises at least a first input end, a second input end, a first output end and a second output end; the first output end of the photoelectric converter is connected with the first input end of the photoelectric converter through the first optical fiber; the second output end of the photoelectric converter is connected with the second input end of the photoelectric converter through the second optical fiber;

the electro-optic converter receives a first group of image data output by the first data processing device through a first input end of the electro-optic converter and converts the first group of image data from an electric signal to an optical signal; the electro-optic converter outputs a first group of image data converted into optical signals through a first output end of the electro-optic converter and transmits the first group of image data through the first optical fiber; the photoelectric converter receives a first group of image data which is transmitted by the first optical fiber and is converted into an optical signal through a first input end of the photoelectric converter, and converts the first group of image data into an electric signal from the optical signal so as to be output through a first output end of the photoelectric converter;

The electro-optic converter receives a second group of image data output by the first data processing device through a second input end of the electro-optic converter and converts the second group of image data from an electric signal to an optical signal; the electro-optic converter outputs a second group of image data converted into optical signals through a second output end of the electro-optic converter and transmits the second group of image data through the second optical fiber; the photoelectric converter receives a second group of image data which is transmitted by the second optical fiber and is converted into optical signals through a second input end of the photoelectric converter, and converts the second group of image data into electric signals from the optical signals so as to be output through a second output end of the photoelectric converter.
The image data transmission device of claim 1, wherein the electro-optic transducer comprises a first electro-optic transducer and a second electro-optic transducer; the photoelectric converter includes a first photoelectric converter and a first photoelectric converter; the first optical fiber transmission channel comprises a first optical fiber, and the second optical fiber transmission channel comprises a second optical fiber;

the first electro-optic transducer includes a first input and a first output; the second electro-optic transducer includes a second input and a second output; the first photoelectric converter comprises a first input end and a first output end; the second photoelectric converter comprises a second input end and a second output end; the first output end of the first photoelectric converter is connected with the first input end of the first photoelectric converter through the first optical fiber; the second output end of the second photoelectric converter is connected with the second input end of the second photoelectric converter through the second optical fiber;

The first electro-optic converter receives a first group of image data output by the first data processing device through a first input end of the first electro-optic converter and converts the first group of image data from an electric signal to an optical signal; the first electro-optic converter outputs a first group of image data converted into optical signals through a first output end of the first electro-optic converter and transmits the first group of image data through the first optical fiber; the first photoelectric converter receives a first group of image data which is transmitted by the first optical fiber and is converted into an optical signal through a first input end of the first photoelectric converter, and converts the first group of image data into an electric signal from the optical signal so as to be output through a first output end of the first photoelectric converter;

the second electro-optical converter receives a second group of image data output by the first data processing device through a second input end of the second electro-optical converter and converts the second group of image data from an electric signal to an optical signal; the second electro-optic converter outputs a second group of image data converted into optical signals through a second output end of the second electro-optic converter and transmits the second group of image data through the second optical fiber; the second photoelectric converter receives the second group of image data converted into optical signals transmitted by the second optical fiber through a second input end of the second photoelectric converter, and converts the second group of image data into electric signals from the optical signals so as to be output through a second output end of the second photoelectric converter.
An image data transmission device for an endoscopic imaging system, the image data transmission device being configured to be connected to an image processing section in the endoscopic imaging system so as to transmit image data to the image processing section; characterized by comprising the following steps:

an image sensor for generating and outputting image data based on a first data communication protocol;

the first data processing device is at least used for converting the image data output by the image sensor into image data based on a second data communication protocol and outputting the image data; the second data communication protocol is different from the first data communication protocol;

the optical fiber transmission assembly is used for converting the image data output by the first data processing device into an optical signal from an electric signal and transmitting the optical signal, and then converting the optical signal into the electric signal and outputting the electric signal;

a second data processing device for receiving the image data transmitted from the optical fiber transmission assembly and converting the image data into image data based on a third data communication protocol for output;

wherein:

the first data processing device and the second data processing device adopt a V-by-One signal transmission interface standard, and the second data communication protocol is a protocol based on the V-by-One signal transmission interface standard;

Alternatively, an FPD-LINK signal transmission interface standard is adopted between the first data processing device and the second data processing device, and the second data communication protocol is a protocol based on the FPD-LINK signal transmission interface standard.
The image data transmission device according to claim 9, wherein:

the first data processing device and the second data processing device adopt an FPD-LINK III signal transmission interface standard, and the second data communication protocol is a protocol based on the FPD-LINK III signal transmission interface standard;

or, an FPD-LINK IV signal transmission interface standard is adopted between the first data processing device and the second data processing device, and the second data communication protocol is a protocol based on the FPD-LINK IV signal transmission interface standard.
The image data transmission device according to claim 9, wherein the third data communication protocol is the same as the first data communication protocol or the third data communication protocol is different from the first data communication protocol.
The image data transmission device according to any one of claims 9 to 11, wherein the first data communication protocol is MIPI CSI protocol.
The image data transmission device of claim 12, wherein the second data communication protocol satisfies: the image data based on the second data communication protocol has a signal amplitude required for converting the image data from an electrical signal to an optical signal by the optical fiber transmission assembly.
The image data transmission device of claim 9, wherein the optical fiber transmission assembly comprises an electro-optic transducer, an optical fiber transmission channel, and an optical-to-electrical transducer;

the electro-optical converter receives the image data output by the first data processing device and converts the image data from an electric signal to an optical signal to output to the optical fiber transmission channel;

the optical fiber transmission channel is used for transmitting the image data converted into optical signals; the optical fiber transmission channel comprises an optical fiber;

the photoelectric converter receives the image data converted into the optical signal transmitted by the optical fiber transmission channel, converts the image data into an electric signal from the optical signal and outputs the electric signal.
An endoscopic imaging system, comprising:

a light source section;

a light source control section for controlling the light source section to provide light required for imaging;

An endoscope including an insertion section that can be inserted into a living body;

an image pickup section including the image data transmission device according to any one of claims 1 to 14;

an image processing unit for receiving and processing the image data outputted from the endoscope data transmission device to generate data for displaying an image;

and the display is used for displaying the data for displaying the image.
The endoscopic imaging system according to claim 15, wherein said image processing section comprises an FPGA or a CPU.