WO2021051222A1

WO2021051222A1 - Endoscope system, mixed light source, video acquisition device and image processor

Info

Publication number: WO2021051222A1
Application number: PCT/CN2019/105889
Authority: WO
Inventors: 迟崇巍; 田捷
Original assignee: 北京数字精准医疗科技有限公司
Priority date: 2019-09-16
Filing date: 2019-09-16
Publication date: 2021-03-25

Abstract

An endoscope system, a mixed light source, a video acquisition device and an image processor. The endoscope system comprises a mixed light source, which is used for synchronously emitting visible light and near-infrared light to a target detection area containing a contrast medium; a video image acquisition device, which is used for acquiring image mixing data of the target detection area, the image mixing data comprising visible light image data and near-infrared light image data of the target detection area; an image processor, which is used for generating contrastographic images of the target detection area according to the image mixing data; a video output unit, which is used for converting the contrastographic images into a video format and outputting same; and a display unit, which is used for displaying in real time the contrastographic images in the video format. The present invention may increase the accuracy of the endoscope system distinguishing normal tissue and pathological tissue.

Description

Endoscope system, hybrid light source, video capture device and image processor

Technical field

This application relates to the field of medical equipment, and in particular to an endoscope system, a hybrid light source, a video acquisition device and an image processor.

Background technique

Molecular imaging (molecular imaging) is a science that uses imaging methods to display specific molecules at the tissue level, cell and subcellular level, reflect changes at the molecular level in the living body, and conduct qualitative and quantitative research on their biological behavior in imaging. An endoscope based on molecular imaging technology is a commonly used medical device. It is a testing instrument that integrates traditional optics, ergonomics, precision machinery, modern electronics, mathematics, and software. Doctors can observe the lesions (such as ulcers or tumors and other diseased tissues/organs) in the patient with the help of an endoscope, and formulate a better treatment plan accordingly.

Traditional endoscopy systems use natural light to stimulate imaging, but the appearance of some diseased tissues is not obvious; and some normal tissues have very similar appearances to diseased tissues under certain circumstances. Therefore, if the judgment is made only by visually observing the image displayed by the endoscope system, it is easy to cause misdiagnosis. Therefore, how to accurately distinguish between normal tissue and diseased tissue has become an urgent technical problem for the endoscopic system.

Summary of the invention

The purpose of the embodiments of the present application is to provide an endoscope system, a hybrid light source, a video acquisition device, and an image processor, so as to improve the accuracy of the endoscope system in distinguishing normal tissues from diseased tissues. To achieve the foregoing objective, on the one hand, an embodiment of the present application provides an endoscope system, including:

Hybrid light source, which is used to simultaneously emit visible light and near-infrared light to the target detection area containing the contrast agent;

A video image acquisition device for acquiring image mixing data of the target detection area; the image mixing data includes visible light image data and near-infrared light image data of the target detection area;

An image processor, configured to generate a contrast image of the target detection area according to the image mixing data;

A video output unit for converting the contrast image into a video format and outputting;

The display unit is used to display the contrast image in the video format in real time.

In an embodiment of the present application, the hybrid light source includes: a white laser light source and a near-infrared laser light source that are close to each other and independently arranged.

In an embodiment of the present application, the video image acquisition device includes:

An endoscope, which is used to collect the image mixed light signal of the target detection area;

A focusing lens, which is used to adjust the focus of the image mixed optical signal;

The filter is used to filter out the mixed optical signal of the image after focusing, the wavelength of which is outside the visible light and the near-infrared light;

The photoelectric conversion module is used to convert the filtered image mixed light signal into a corresponding electrical signal, which is used as the image mixed data.

In an embodiment of the present application, the photoelectric conversion module includes any one of the following:

Charge coupled device

Complementary metal oxide semiconductor image sensor.

In an embodiment of the present application, the hybrid light source, the video output unit, and the image processor are integrated into an integrated structure.

On the other hand, the embodiments of the present application also provide a hybrid light source, the hybrid light source is applied to an endoscope system, and the hybrid light source is used to simultaneously emit visible light and near-infrared light to a target detection area containing a contrast agent.

On the other hand, an embodiment of the present application also provides a video capture device, the video capture device is applied to an endoscope system, the video image capture device is used to capture image mixed data of a target detection area; the image The mixed data includes visible light image data and near-infrared light image data of the target detection area.

Charge coupled device

Complementary metal oxide semiconductor image sensor.

On the other hand, an embodiment of the present application also provides an image processor, including:

An image acquisition module for acquiring image mixing data of a target detection area; the image mixing data includes visible light image data and near-infrared light image data of the target detection area;

Recognition and marking module for recognizing near-infrared light image data and visible light image data from the image mixed data, and marking the near-infrared light image data;

The image synthesis module is used to synthesize the marked near-infrared light image data and the recognized visible light image data into a contrast image of the target detection area.

In an embodiment of the present application, the image processor includes a field programmable gate array.

It can be seen from the technical solutions provided by the above embodiments of the present application that in the endoscope system based on molecular imaging technology in the embodiments of the present application, since the endoscope system can acquire the near-infrared light image of the target detection area, the near-infrared light image It can penetrate deeper tissues of the target detection area. Therefore, the endoscopic system of the embodiment of the present application outputs the displayed contrast image, which can more accurately distinguish normal tissues and diseased tissues in the target detection area, thereby improving the endoscope The accuracy of the system to distinguish normal tissues from diseased tissues.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments described in this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. In the attached picture:

Fig. 1 is a structural block diagram of an endoscope system in some embodiments of the application;

FIG. 2 is a schematic diagram of the structure of an image processor in some embodiments of the application;

FIG. 3 is a schematic diagram of an image processor in some other embodiments of the application.

detailed description

In order to enable those skilled in the art to better understand the technical solutions in the application, the following will clearly and completely describe the technical solutions in the embodiments of the application in conjunction with the drawings in the embodiments of the application. Obviously, the described The embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of this application. For example, in the following description, forming the second part above the first part may include an embodiment in which the first part and the second part are formed in a direct contact manner, and may also include the first part and the second part in a non-direct contact manner ( That is, between the first component and the second component, additional components may be included).

Moreover, for ease of description, some embodiments of the present application may use spatially relative terms such as "above", "below", "top", "below", etc., to describe what is shown in the drawings of the embodiments. The relationship between one element or component and another (or other) elements or components. It should be understood that in addition to the orientation described in the drawings, the spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the drawings is turned over, elements or parts described as "below" or "beneath" other elements or parts will then be positioned as "above" or "above" other elements or parts. Above".

Studies have shown that the ability of light to penetrate tissues is related to the strength of the tissues to absorb light, the characteristics of light waves, the structure of biological tissues and their physical and chemical properties. Compared with visible light, near-infrared light (Near-Infrared, referred to as NIR) with a wavelength range of 650-900nm has: (1) The absorption and scattering effect of near-infrared light in this band is the smallest by biological tissues, and near-infrared light can Penetrate deeper tissues; ⑵Because the autofluorescence of biological tissues in this band of near-infrared light is small, the signal-to-background ratio (SBR) is relatively high.

As shown in Figure 1, based on the above principles, in order to improve the ability of the endoscope system to distinguish between normal tissue and diseased tissue, the endoscope system of some embodiments of the present application may include a hybrid light source, a video image acquisition device, an image processor, and a video Output unit and display unit, etc. Among them, the hybrid light source can be used to simultaneously emit visible light and near-infrared light to the target detection area containing the contrast agent. The video image acquisition device can be used to acquire image mixed data of the target detection area. The image mixing data includes visible light image data and near-infrared light image data of the target detection area. The image processor may be used to generate a contrast image of the target detection area according to the image mixing data. The video output unit can be used to convert the contrast image into a video format and output it. The display unit can be used to display contrast images in video format in real time.

In the endoscope system of the foregoing embodiment of the present application, since the endoscope system can obtain the near-infrared light image of the target detection area, and the near-infrared light image can penetrate deeper tissues of the target detection area, therefore, the present application The endoscopic system of the above-mentioned embodiment outputs and displays the contrast image, which can more accurately distinguish the normal tissue and the diseased tissue in the target detection area.

In an embodiment of the present application, the hybrid light source may be a white light laser light source and a near-infrared laser light source that are arranged close to each other and independently arranged, and the white light laser light source and the near-infrared laser light source can be synchronized to the target detection area. Emit visible light and near-infrared light. In another embodiment of the present application, the hybrid light source may also be a single laser source, and the single laser source emits laser light whose wavelength range covers at least the visible light waveband and the near-infrared light waveband. Therefore, in some embodiments of the present application, the structure of the hybrid light source is not limited, and it can be specifically selected according to needs.

In an embodiment of the present application, the video image acquisition device may include an endoscope, a focusing lens, a filter, and a photoelectric conversion module. Wherein, the endoscope can be used to collect the image mixed light signal of the target detection area. Focusing The lens can be used to focus the mixed optical signal of the image to obtain a clear image. The filter can be used to filter out the image mixed light signal after focusing, and its wavelength is outside the visible light and near-infrared light, that is, the image mixed light signal filtered by the filter can only retain the visible light part and the near-infrared light. Light part. The photoelectric conversion module can be used to convert the filtered image mixed optical signal into a corresponding electrical signal, which is used as image mixed data for subsequent processing. In some embodiments of the present application, the photoelectric conversion module may be single to reduce power consumption and volume.

In some exemplary embodiments, the photoelectric conversion module may include, but is not limited to, a Charge Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (Complementary Metal Oxide Semiconductor, CMOS) image sensor, for example.

In some embodiments of the present application, the image processor may adopt any suitable hardware or a combination of software and hardware according to needs. For example, in some exemplary embodiments, the image processor may be implemented by a hardware structure such as a Field Programmable Gate Array (FPGA) or a Complex Programmable Logic Device (CPLD). Among them, FPGA is a hardware-based processing system. Its advantage lies in the parallel processing of data. Multiple data can be processed simultaneously in one clock cycle, which can shorten the processing time and reduce the image delay. In the foregoing hardware implementation manner, as shown in FIG. 2, the image processor may include an image acquisition module 21, an identification and marking module 22, and an image synthesis module 23. among them:

The image acquisition module 21 may be used to acquire image mixing data of the target detection area. The image mixing data includes visible light image data and near-infrared light image data of the target detection area.

The recognition and marking module 22 may be used to recognize near-infrared light image data and visible light image data from the image mixed data, and mark the near-infrared light image data. In an embodiment of the present application, under the irradiation of near-visible light, the target detection area will present a visible light image; because the target detection area contains a contrast agent, at the same time, under the irradiation of near-infrared light, the target detection area The part containing the contrast agent will also show another near-infrared light image. In the image mixing data acquired by the image acquisition module 21: the color value of the visible light image corresponding to the visible light image data and the color value of the near-infrared light image corresponding to the near-infrared light image data are significantly different. Therefore, based on this color value The recognition and marking module 22 recognizes the near-infrared light image data and the visible light image data from the image mixed data. On this basis, in order to enhance or highlight the display effect and facilitate operation, the recognized near-infrared light image data can be marked (for example, fluorescent marking, etc.).

The image synthesis module 23 can be used to synthesize the marked near-infrared light image data and the recognized visible light image data into a contrast image of the target detection area.

In some embodiments of the present application, the image processor may further include an image defogging module and an image noise reduction module. among them:

The image defogging module can be used to perform dark channel defogging processing on the mixed image data to reduce or eliminate fogging of the picture. Due to the process of surgery and other applications, the image may be fogged and blurred due to steam or other factors. The image defogging module based on the dark channel defogging algorithm can reduce or even eliminate the fogging and blur of the image to a certain extent.

The image noise reduction module can be used to perform multi-frame noise reduction processing on the image processed by the image defogging module to improve the signal-to-noise ratio of the image. In the process of image acquisition, the image noise may increase due to factors such as light conditions and exposure time, making the image unclear. Through the image defogging module that contains multi-frame noise reduction algorithms, it is possible to find different pixels with noise properties under different frame numbers for the collected multi-frame images for the same target area, and through repeated weighting and replacement, After the final synthesis, a relatively clean and pure image is obtained.

In some embodiments of the present application, the image synthesis module 23 may include:

The RGB conversion sub-module can be used to perform RGB conversion on the marked near-infrared light image data in the target area to convert it into an image with a designated fluorescent effect.

The image overlay sub-module can be used to cover the image with the designated fluorescent effect to the image of the visible light portion in the target area, so as to avoid the interference of visible light.

In an exemplary embodiment of the present application, the above-mentioned designated fluorescent effect image may be, for example, a green fluorescent effect image. Because the green fluorescence effect is more intuitive and obvious, it is convenient for the doctor to distinguish normal tissues and diseased tissues more clearly during operation (or in other application scenarios).

In another exemplary embodiment of the present application, the above-mentioned designated fluorescent effect image can be displayed in multiple levels, that is, the brightness of the displayed fluorescent light can be changed according to the strength of the original data, which is beneficial for the doctor to treat the lesion of the diseased tissue. Distinguish and judge the degree.

For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing this application, the functions of each unit can be implemented in one or more hardware modules.

In other exemplary embodiments, as shown in FIG. 3, the image processor may also include a processor and a computer program stored in a memory, and the computer program executes the following steps when the computer program is run by the processor:

Acquiring image mixing data of the target detection area; the image mixing data includes visible light image data and near-infrared light image data of the target detection area;

Identifying near-infrared light image data and visible light image data from the image mixing data, and marking the near-infrared light image data;

The marked near-infrared light image data and the recognized visible light image data are synthesized into a contrast image of the target detection area.

Although the process flow described above includes multiple operations appearing in a specific order, it should be clearly understood that these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel (for example, using parallel processors or Multi-threaded environment).

In some embodiments of the present application, the example video output module may be a video capture card, such as an HDMI capture card, a VGA video capture card, a PCI video card, a PCI-E video capture card, and so on.

In some embodiments of the present application, the display device may be a display, such as a cathode ray tube (i.e. display), a plasma display, a liquid crystal display (i.e. LCD display), a light-emitting diode display (i.e. LED display), an organic light-emitting diode display ( Namely OLED display) and so on.

In some embodiments of the present application, in order to make the endoscope system compact and easy to use, the hybrid light source, the video output unit and the image processor are integrated into an integrated structure.

The present invention is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

In a typical configuration, the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-permanent memory in a computer readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer readable medium.

Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

It should also be noted that the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, or device including a series of elements not only includes those elements, but also includes no The other elements clearly listed may also include elements inherent to the process, method, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, or device that includes the element.

Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

This application may be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments. In these distributed computing environments, tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.

The foregoing descriptions are only examples of the present application, and are not used to limit the present application. For those skilled in the art, this application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the scope of the claims of this application.

Claims

An endoscope system, characterized in that it comprises:

Hybrid light source, which is used to simultaneously emit visible light and near-infrared light to the target detection area containing the contrast agent;

A video image acquisition device for acquiring image mixing data of the target detection area; the image mixing data includes visible light image data and near-infrared light image data of the target detection area;

An image processor, configured to generate a contrast image of the target detection area according to the image mixing data;

A video output unit for converting the contrast image into a video format and outputting;

The display unit is used to display the contrast image in the video format in real time.
The endoscope system according to claim 1, wherein the hybrid light source comprises: a white light laser light source and a near-infrared laser light source which are arranged close to each other and independently arranged.
3. The endoscope system of claim 2, wherein the video image acquisition device comprises:

An endoscope, which is used to collect the image mixed light signal of the target detection area;

A focusing lens, which is used to adjust the focus of the image mixed optical signal;

The filter is used to filter out the mixed optical signal of the image after focusing, the wavelength of which is outside the visible light and the near-infrared light;

The photoelectric conversion module is used to convert the filtered image mixed light signal into a corresponding electrical signal, which is used as the image mixed data.
5. The endoscope system of claim 3, wherein the photoelectric conversion module comprises any one of the following:

Charge coupled device

Complementary metal oxide semiconductor image sensor.
3. The endoscope system according to claim 3, wherein the hybrid light source, the video output unit and the image processor are integrated into an integrated structure.
A hybrid light source, characterized in that the hybrid light source is applied to an endoscope system, and the hybrid light source is used to simultaneously emit visible light and near-infrared light to a target detection area containing a contrast agent.
7. The hybrid light source according to claim 6, wherein the hybrid light source comprises: a white light laser light source and a near-infrared laser light source which are arranged close to each other and independently arranged.
A video acquisition device, characterized in that the video acquisition device is applied to an endoscope system, the video image acquisition device is used to collect image mixing data of a target detection area; the image mixing data includes the target detection Visible light image data and near-infrared light image data of the area.
8. The video capture device of claim 8, wherein the video image capture device comprises:

An endoscope, which is used to collect the image mixed light signal of the target detection area;

A focusing lens, which is used to adjust the focus of the image mixed optical signal;

The filter is used to filter out the mixed optical signal of the image after focusing, the wavelength of which is outside the visible light and the near-infrared light;

The photoelectric conversion module is used to convert the filtered image mixed light signal into a corresponding electrical signal, which is used as the image mixed data.
9. The video capture device of claim 9, wherein the photoelectric conversion module comprises any one of the following:

Charge coupled device

Complementary metal oxide semiconductor image sensor.
An image processor, characterized in that it comprises:

An image acquisition module for acquiring image mixing data of a target detection area; the image mixing data includes visible light image data and near-infrared light image data of the target detection area;

Recognition and marking module for recognizing near-infrared light image data and visible light image data from the image mixed data, and marking the near-infrared light image data;

The image synthesis module is used to synthesize the marked near-infrared light image data and the recognized visible light image data into a contrast image of the target detection area.
11. The image processor of claim 11, wherein the image processor comprises a field programmable gate array.
The image processor of claim 11, wherein the image processor further comprises:

An image defogging module for performing dark channel defogging processing on the mixed image data to reduce or eliminate fogging of the picture;

The image noise reduction module is used to perform multi-frame noise reduction processing on the image processed by the image defogging module to improve the signal-to-noise ratio of the image.
The image processor of claim 11, wherein the image synthesis module comprises:

The RGB conversion sub-module is used to convert the marked near-infrared light image data in the target area into an image with a designated fluorescent effect; the image overlay sub-module is used to cover the image with a designated fluorescent effect The visible light part of the image in the target area is dropped.