CN114376491A

CN114376491A - Endoscopic imaging device, method and system thereof, and electronic device

Info

Publication number: CN114376491A
Application number: CN202011111762.0A
Authority: CN
Inventors: 高原; 徐文杰
Original assignee: Sunny Optical Zhejiang Research Institute Co Ltd
Current assignee: Sunny Optical Zhejiang Research Institute Co Ltd
Priority date: 2020-10-16
Filing date: 2020-10-16
Publication date: 2022-04-22

Abstract

An endoscopic imaging apparatus, a method, a system and an electronic device thereof are suitable for acquiring an image of a target object. The endoscopic imaging device includes: an optical lens for receiving and focusing the illumination light reflected back through the target object; a spectrum chip assembly, wherein the spectrum chip assembly is correspondingly arranged on the focal plane of the optical lens and is used for receiving the illuminating light focused by the optical lens to acquire original signal data; and a data processor, wherein the data processor is communicably connected to the spectral chip assembly for performing data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object.

Description

Endoscopic imaging device, method and system thereof, and electronic device

Technical Field

The present invention relates to the field of endoscope technologies, and in particular, to an endoscopic imaging apparatus, a method and a system thereof, and an electronic device.

Background

An endoscope refers to a detection device comprising an image sensor, an optical lens, an illumination structure, a mechanical device, and the like, which can go deep into the digestive tract or other tract to observe the internal environment and transmit images for a doctor or a professional to make a diagnosis. Medical endoscopes have high image quality requirements because clear images allow doctors and professionals to make more accurate judgments and manipulations of patients. In addition, since the environment inside the digestive tract or other ducts is complex, images of medical endoscopes can clearly bring great help to doctors and professionals if mucous membranes, tissues, blood vessels, and the like can be visually distinguished.

Conventional electronic endoscopes typically employ RGB sensors that cover the visible spectrum from 400nm to 800nm, similar to normal lighting, and do not improve the contrast between capillaries and subcutaneous microvessels, although the image is realistically sharp.

Currently, to improve contrast, one of the existing endoscopic imaging techniques is narrow band optical imaging (NBI) technique, which employs a narrow band optical filter added to the illumination system to generate light source illumination with central wavelengths of 415nm and 540nm, respectively, and a bandwidth of 30 nm. Because the penetration depth of the two lights with the wavelengths of 415nm and 540nm in the mucous membrane tissue is different, the light with the wavelength of 415nm mainly shows superficial blood vessels and is brown, and the light with the wavelength of 540nm can show the blood vessels in the submucosa as blue-green, so the blood vessel distribution condition in the mucous membrane can be easily identified. However, such a narrow-band light imaging technique not only has insufficient brightness due to the use of narrow-band light filters, but also cannot combine images of other wavelengths due to the fact that the filters are fixed.

Another existing endoscopic imaging technology is an intelligent spectral colorimetric technology (FICE), which does not use a narrow-band optical filter, but decomposes an RGB image acquired through a general electronic endoscope into spectral images with a single wavelength, generates spectral images with independent wavelengths by selecting a combination of red, green, and blue lights with arbitrary wavelengths, and processes and reconstructs the spectral images through a spectral estimation algorithm to obtain FICE images. Although the intelligent spectral colorimetry can arbitrarily select wavelengths at intervals of 5nm between 400nm and 600nm, and at most 50 wavelengths can be combined for the purpose of electronic staining, the FICE technology has insufficient image definition due to spectral estimation using RGB images to obtain images at specific wavelengths, so that the intelligent spectral colorimetry is inferior to the narrow-band optical imaging technology in the aspect of displaying the morphology of the microvasculature.

Disclosure of Invention

An advantage of the present invention is to provide an endoscopic imaging apparatus, a method and a system thereof, and an electronic device, which can acquire a spectral image of any wavelength band, and thus contribute to achieving a better electronic staining effect.

Another advantage of the present invention is to provide an endoscopic imaging apparatus, a method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the endoscopic imaging apparatus can directly acquire a hyperspectral image of a multispectral region from a physical layer by using a spectrum chip, so as to achieve higher electronic staining quality by combining a spectrum recovery algorithm.

Another advantage of the present invention is to provide an endoscopic imaging apparatus, a method and a system thereof, and an electronic device, wherein in an embodiment of the present invention, the endoscopic imaging apparatus can acquire an image with higher brightness, and simultaneously acquire richer and more accurate spectral information, thereby facilitating to achieve better electronic dyeing effect.

Another advantage of the present invention is to provide an endoscopic imaging apparatus, a method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the endoscopic imaging apparatus can provide a doctor and a professional with better quality endoscopic images for more accurate judgment and operation of a patient.

It is another advantage of the present invention to provide an endoscopic imaging apparatus, method, system and electronic device thereof, wherein expensive materials or complex structures are not required in the present invention in order to achieve the above objects. The present invention thus successfully and effectively provides a solution that not only provides a simple endoscopic imaging apparatus and method, system, and electronic device, but also increases the practicality and reliability of the endoscopic imaging apparatus and method, system, and electronic device.

To achieve at least one of the above advantages or other advantages and objects, the present invention provides an endoscopic imaging device adapted to be disposed at a front end of an endoscope to acquire an image of a target object, wherein the endoscopic imaging device includes:

an optical lens for receiving and focusing the illumination light reflected back through the target object;

a spectrum chip assembly, wherein the spectrum chip assembly is correspondingly arranged on the focal plane of the optical lens and is used for receiving the illuminating light focused by the optical lens to acquire original signal data; and

a data processor, wherein the data processor is communicatively connected to the spectral chip assembly for performing data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object.

According to an embodiment of the present invention, the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on a pixel of the image sensor, and the spectral filter is located between the optical lens and the image sensor, so that the illumination light focused by the optical lens passes through the spectral filter before being received by the image sensor.

According to an embodiment of the present invention, the spectral filter is a narrow spectral band filter.

According to an embodiment of the invention, the narrow-band filter includes a plurality of filter arrays having the same specification, and the plurality of filter arrays are respectively and correspondingly attached to a plurality of pixel arrays of the image sensor.

According to an embodiment of the present invention, the data processor includes a direct reading module and a reconstruction module communicatively connected to each other, wherein the direct reading module is configured to directly read raw signal data obtained via the image sensor to obtain a series of raw spectral data with specific wavelengths, and the reconstruction module is configured to reconstruct an electronic staining effect by overlapping and restoring a plurality of raw spectral data with selected wavelengths into multispectral image data.

According to an embodiment of the invention, the spectral filter is a quantum dot thin film filter or a micro-nano structure filter with a specific spectral response curve.

According to an embodiment of the present invention, the data processor includes a spectrum reconstruction module and a reconstruction module communicably connected to each other, wherein the spectrum reconstruction module is configured to perform spectrum reconstruction processing on raw signal data obtained through the image sensor according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data with specific wavelengths, and the reconstruction module is configured to reconstruct an electronic staining effect by superimposing and restoring a plurality of the reconstructed spectrum data with selected wavelengths into multispectral image data.

According to an embodiment of the present invention, the spectral reconstruction algorithm model is L ═ S × P, where L is the light response input of one pixel; s is a system matrix to represent the characteristics of the whole endoscope system; and P is the reconstructed spectrum of the pixel point.

According to an embodiment of the present invention, the endoscopic imaging apparatus further includes a light source module, wherein the light source module is correspondingly disposed near the optical lens for emitting the illumination light to the target object.

According to an embodiment of the present invention, the light emitting direction of the light source module is substantially parallel to the optical axis direction of the optical lens.

According to another aspect of the present invention, the present invention further provides an electronic device adapted to acquire an image of a target object, comprising:

at least one endoscopic imaging device, wherein the endoscopic imaging device comprises:

a data processor, wherein the data processor is communicatively connected to the spectral chip assembly for performing data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object; and

an electronic device body, wherein the electronic device body comprises:

a monitor component for displaying a corresponding image based on the spectral image data; and

an endoscope tube, wherein the endoscope tube communicatively connects the endoscopic imaging device with the monitor, and the endoscopic imaging device is disposed at a forward end of the endoscope tube, wherein the endoscope tube is configured to transmit the spectral image data obtained via the at least one endoscopic imaging device to the monitor.

According to another aspect of the present invention, there is further provided an endoscopic imaging method comprising the steps of:

acquiring raw signal data obtained by imaging illumination light which is received by a spectral chip assembly and reflected back from a target object and focused by an optical lens, wherein the spectral chip assembly comprises a spectral filter and an image sensor, the spectral filter is correspondingly arranged on a pixel of the image sensor, and the spectral filter is positioned between the optical lens and the image sensor;

and performing data processing on the original signal data to obtain spectral image data corresponding to the target object.

According to an embodiment of the present invention, the step of performing data processing on the raw signal data to obtain spectral image data corresponding to the target object includes the steps of:

directly reading the acquired original signal data to obtain a series of original spectral data with specific wavelengths, wherein the spectral filter is a narrow spectral band filter; and

and reconstructing an electronic dyeing effect by superposing and restoring a plurality of original spectral data with selected wavelengths into multispectral image data.

according to the spectral reconstruction algorithm model, performing spectral reconstruction processing on the acquired original signal data to obtain a series of reconstructed spectral data with specific wavelengths, wherein the spectral filter is a wide spectral band filter; and

and reconstructing an electronic dyeing effect by superposing and restoring a plurality of reconstructed spectral data with selected wavelengths into multispectral image data.

According to another aspect of the present invention, the present invention further provides an endoscopic imaging system for acquiring an image of a target object, wherein the endoscopic imaging system comprises, communicably connected to each other:

the acquisition module is used for acquiring raw signal data obtained by imaging illumination light which is received by a spectral chip assembly and reflected back from a target object and focused by an optical lens, wherein the spectral chip assembly comprises a spectral filter and an image sensor, the spectral filter is correspondingly arranged on a pixel of the image sensor, and the spectral filter is positioned between the optical lens and the image sensor;

and the data processing module is used for carrying out data processing on the original signal data to obtain spectral image data corresponding to the target object.

According to an embodiment of the present invention, the data processing module includes a direct reading module and a reconstruction module communicably connected to each other, wherein the direct reading module is configured to directly read the acquired raw signal data to obtain a series of raw spectral data with specific wavelengths, and the spectral filter is a narrow spectral filter; the reconstruction module is used for reconstructing an electronic dyeing effect by superposing and restoring a plurality of original spectral data with selected wavelengths into multispectral image data.

According to an embodiment of the present invention, the data processing module includes a spectrum reconstruction module and a reconstruction module communicably connected to each other, wherein the spectrum reconstruction module is configured to perform spectrum reconstruction processing on the acquired raw signal data according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data with specific wavelengths, and the spectrum filter is a wide-band filter; the reconstruction module is used for reconstructing an electronic dyeing effect by superposing and restoring a plurality of the reconstructed spectral data with the selected wavelength into multispectral image data.

According to another aspect of the present invention, the present invention further provides an electronic device comprising:

at least one processor configured to execute instructions; and

a memory communicatively coupled to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executable by the at least one processor to cause the at least one processor to perform some or all of the steps of an endoscopic imaging method, wherein the endoscopic imaging method comprises the steps of:

Further objects and advantages of the invention will be fully apparent from the ensuing description and drawings.

These and other objects, features and advantages of the present invention will become more fully apparent from the following detailed description, the accompanying drawings and the claims.

Drawings

FIG. 1 is a block diagram schematic of an endoscopic imaging device in accordance with an embodiment of the present invention.

Fig. 2 shows a schematic structural diagram of the spectral chip assembly of the endoscopic imaging device according to the above-described embodiment of the present invention.

Fig. 3 shows a schematic structural diagram of the spectral filter of the spectral chip assembly according to the above-described embodiment of the invention.

FIG. 4A shows an example of a narrow spectral band filter of the spectral chip assembly according to the above-described embodiments of the present invention.

Fig. 4B shows an example of the micro-nano structured filter of the spectral chip assembly according to the above embodiment of the present invention.

FIG. 5A shows an example of a data processor of the spectroscopic chip assembly according to the above-described embodiment of the invention.

Fig. 5B shows a variant example of the data processor according to the above-described embodiment of the invention.

Fig. 6 shows a schematic diagram comparing the output of the endoscopic imaging device according to the above-described embodiment of the present invention with the conventional RGB chip.

FIG. 7 is a flow chart diagram of an endoscopic imaging method in accordance with an embodiment of the present invention.

Fig. 8 shows a flow chart of one of the steps in the endoscopic imaging method according to the above-described embodiment of the present invention.

FIG. 9 is one example of an endoscopic imaging system according to an embodiment of the present invention.

FIG. 10 is another example of the endoscopic imaging system according to the above-described embodiments of the present invention.

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the invention.

Fig. 12 is a schematic structural diagram of another electronic device according to an embodiment of the invention.

Detailed Description

The following description is presented to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.

In the present invention, the terms "a" and "an" in the claims and the description should be understood as meaning "one or more", that is, one element may be one in number in one embodiment, and the element may be more than one in number in another embodiment. The terms "a" and "an" should not be construed as limiting the number unless the number of such elements is explicitly recited as one in the present disclosure, but rather the terms "a" and "an" should not be construed as being limited to only one of the number.

In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless explicitly stated or limited otherwise, the terms "connected" and "connected" are to be interpreted broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Since the ordinary electronic endoscope adopts the RGB sensor, although the RGB sensor covers the spectrum range of visible light 400nm to 800nm, and is similar to ordinary illumination, the image can be vivid and clear, but the RGB sensor cannot improve the contrast of the capillary vessels and the subcutaneous micro-vessels. Therefore, the invention provides an endoscope imaging device, a method and a system thereof as well as electronic equipment, wherein a spectrum chip is adopted instead of a traditional RGB sensor, and a spectrum image of any waveband is reconstructed through a spectrum estimation algorithm by combining the operation relation between pixels through special materials on the pixels, so that a better electronic dyeing effect can be achieved.

Referring to fig. 1-6 of the drawings, an endoscopic imaging device adapted to be disposed at a forward end of an endoscope to acquire an image of a target object in accordance with an embodiment of the present invention is illustrated. Specifically, as shown in fig. 1 and 2, the endoscopic imaging device 10 may include an optical lens 11 and a spectral chip assembly 12, wherein the optical lens 11 is configured to receive and focus the illumination light reflected back through the target object, and the spectral chip assembly 12 is correspondingly disposed at a focal plane of the optical lens 11, and is configured to receive the illumination light focused through the optical lens 11 to obtain raw signal data, so as to obtain spectral image data corresponding to the target object.

It should be noted that the endoscopic imaging device 10 of the present application uses the spectral chip assembly 12 to replace the conventional RGB sensor, so that the endoscopic imaging device 10 can obtain spectral information of more spectral bands from the physical aspect, and thus the information of the specific wavelength estimated by the spectrum is more accurate and the image is clearer.

More specifically, as shown in fig. 1 to 3, the spectral chip assembly 12 of the endoscopic imaging device 10 may include a spectral filter 121 and an image sensor 122, wherein the spectral filter 121 is correspondingly disposed on a pixel of the image sensor 122, and the spectral filter 121 is located between the optical lens 11 and the image sensor 122, such that the illumination light focused by the optical lens 11 passes through the spectral filter 121 before being received by the image sensor 122 to obtain raw signal data.

Illustratively, in an example of the present invention, as shown in fig. 3, the spectral filter 121 of the spectral chip assembly 12 can be, but is not limited to be, implemented as a narrow-band filter 1211, wherein the narrow-band filter 1211 includes a plurality of filter arrays having the same specification, and the plurality of filter arrays are respectively and correspondingly attached to the plurality of pixel arrays of the image sensor 122. For example, as shown in fig. 4A, M × K pixels are selected on the image sensor 122 as the pixel array, and then a filter with a fixed specification is respectively attached to each pixel in the pixel array to form a corresponding M × K filter array. In particular, the filter arrays attached to all the pixel arrays are identical.

In other examples of the present invention, as shown in fig. 3, the spectral filter 121 of the spectral chip assembly 12 may also be implemented as, but not limited to, a wide spectral band filter having a specific spectral response curve, such as a quantum dot thin film filter 1212 or a micro-nano structure filter 1213, etc. For example, as shown in fig. 4B, when the spectral filter 121 is implemented as the micro-nano structure filter 1213, the micro-nano structure filter 1213 is generally composed of photosensitive units (i.e., micro-nano structure units) made of a special material such as photonic crystal or plasma, and the specifications of the photosensitive units are completely consistent, and the spatial dimensions of the photosensitive units are generally in the micrometer level, but the photosensitive units and the pixels of the image sensor do not form a one-to-one correspondence relationship, that is, one photosensitive unit may correspond to a plurality of pixels.

Notably, since endoscopes typically require a deep inside alimentary or other tract for viewing the internal environment (i.e., the target object is typically located in a dark environment), it is desirable to specially equip it with a light source to provide illumination light. Therefore, according to the above-mentioned embodiment of the present invention, as shown in fig. 1, the endoscopic imaging device 10 may further include a light source assembly 13, wherein the light source assembly 13 is correspondingly disposed near the optical lens 11 for emitting the illumination light to the target object, so that the illumination light reflected by the target object can be received by the optical lens 11 for focusing.

Preferably, the light emitting direction of the light source assembly 13 is substantially parallel to the optical axis direction of the optical lens 11, so that the illumination area of the light source assembly 13 and the field of view area of the optical lens 11 have a large overlapping range, so as to ensure that the target object illuminated by the light source assembly 13 can be captured by the endoscopic imaging device 10 to acquire corresponding spectral image data.

Illustratively, the light source assembly 13 may be, but is not limited to being, implemented as a white light source, such as an LED, a halogen lamp, or a laser light source, and combinations thereof, as long as the illumination requirements of the endoscope application scene are met. In addition, the optical lens 11 of the present application can also be implemented as, but not limited to, a conventional lens as long as the optical requirements of the endoscope application scene are met, and the details of the present application are not repeated herein.

It should be noted that, since the data acquired by the image sensor 122 in the endoscopic imaging apparatus 10 of the present application is raw signal data, and the spectral image data can be acquired after the raw signal data is subjected to data processing, so as to be able to accurately observe different target objects such as a diseased organ or a lesion according to the spectral image data, as shown in fig. 1 and fig. 2, the endoscopic imaging apparatus 10 of the present application may further include a data processor 14, wherein the data processor 14 is communicably connected to the spectral chip assembly 12 for performing data processing on the raw signal data acquired via the spectral chip assembly 12 to acquire the spectral image data. It is understood that the data processor 14 may be integrated with the image sensor 122 of the spectroscopic chip assembly 12 or may be separately disposed; in other words, the data processor 14 of the present invention may be integrally designed with the spectroscopic chip assembly 12 or may be designed separately from the spectroscopic chip assembly 12.

It is noted that the types of the raw signal data acquired by the image sensor 122 of the spectral chip assembly 12 are different according to the type difference of the spectral filter 121 in the spectral chip assembly 12. For example, when the spectral filters 121 of the spectral chip assembly 12 are implemented as the narrow spectral band filters 1211, the corresponding narrow-band multispectral data is suitable for direct pixel readout methods, since the illumination light will directly produce narrow-band transmission spectra after passing through the narrow spectral band filters 1211, and a spectral reconstruction process is generally not required. When the spectral filter 121 of the spectral chip assembly 12 is implemented as the wide spectral band filter (such as the quantum dot thin film filter 1212 or the micro-nano structure filter 1213), since each quantum dot or micro-nano structure unit forms a broadband projection spectral filter, it is necessary to reconstruct the spectral data of the sample according to the measured optical response and the known transmission spectrum of each filter.

Illustratively, in an example of the present invention, as shown in fig. 5A, the spectral filter 121 of the spectral chip assembly 12 is implemented as the narrow spectral band filter 1211, at this time, the data processor 14 of the endoscopic imaging device 10 may include a direct reading module 141 and a reconstruction module 142 which are communicably connected to each other, wherein the direct reading module 141 is communicably connected to the image sensor 122 for directly reading the raw signal data obtained by the image sensor 122 to obtain a series of raw spectral data of a specific wavelength, and wherein the reconstruction module 142 is configured to reconstruct an electronic dyeing effect by superimposing and restoring a plurality of the raw spectral data of a selected wavelength into multispectral image data.

In another example of the present invention, as shown in fig. 5B, the spectral filter 121 of the spectral chip assembly 12 is implemented as the wide spectral band filter, in this case, the data processor 14 of the endoscopic imaging device 10 may include a spectral reconstruction module 41 'and a reconstruction module 142' that are communicably connected to each other, wherein the spectral reconstruction module 141 'is communicably connected to the image sensor 122 for performing a spectral reconstruction process on the raw signal data acquired via the image sensor 122 according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of a specific wavelength, and wherein the reconstruction module 142' is configured to reconstruct an electronic staining effect by superimposing and restoring a plurality of the reconstructed spectral data of a selected wavelength into multispectral image data.

Preferably, the spectral reconstruction algorithm model may be implemented, but is not limited to: l ═ S × P, where L is the light response input of one pixel; s is a system matrix to represent the characteristics of the whole endoscope system; and P is the reconstructed spectrum of the pixel point.

Illustratively, the form of the optically responsive input L may be particularly expressed as

Wherein M represents the input light response dimension of each pixel, and the dimension is equal to the number of quantum dots of the quantum dot thin film filter 1212 or the number of micro-nano structure units of the micro-nano structure filter 1213.

The form of the system matrix S is specifically expressed as

Wherein E represents a light source radiation distribution matrix, C represents a CMOS spectral response matrix, F represents an inherent spectral transmission rate matrix of the optical filter, and N is a reconstructed spectral data latitude.

The reconstructed spectrum P is expressed in a specific form

It is noted that, the reconstruction spectrum P is solved by the spectrum reconstruction algorithm model, and a nonlinear optimization method is generally adopted, which includes a simulated annealing algorithm, a non-negative least square method, a gradient descent method, a convex optimization method, and the like. It will be appreciated that in general the spectral recovery effect of subsequent reconstruction algorithms will increase as the number of filters within the filter increases.

In addition, the selected wavelength of the present invention is preferably selected according to the different pathological organs and lesions to be observed, so that the electronic staining effect (as shown in fig. 6) reconstructed based on the multispectral image data can clearly display the pathological organs and lesions to be observed by selecting the combination of the different wavelengths, which is beneficial for a doctor or a professional to perform more accurate judgment and operation on a patient.

Illustrative method

Referring to fig. 7 and 8 of the drawings, an endoscopic imaging method according to an embodiment of the present invention is illustrated. Specifically, as shown in fig. 7, the endoscopic imaging method may include the steps of:

s100: acquiring raw signal data obtained by imaging illumination light which is received by a spectral chip assembly and reflected back from a target object and focused by an optical lens, wherein the spectral chip assembly comprises a spectral filter and an image sensor, the spectral filter is correspondingly arranged on a pixel of the image sensor, and the spectral filter is positioned between the optical lens and the image sensor; and

s200: and carrying out data processing on the original signal data to obtain spectral image data corresponding to the target object.

More specifically, in an example of the present invention, as shown in fig. 8, the step S200 of the endoscopic imaging method may include the steps of:

s210: directly reading the acquired original signal data to obtain a series of original spectral data with specific wavelengths, wherein the spectral filter is a narrow spectral band filter; and

s220: and reconstructing an electronic dyeing effect by superposing and restoring a plurality of original spectral data with selected wavelengths into multispectral image data.

In another example of the present invention, as shown in fig. 8, the step S200 of the endoscopic imaging method may further include the steps of:

s210': according to the spectral reconstruction algorithm model, performing spectral reconstruction processing on the acquired original signal data to obtain a series of reconstructed spectral data with specific wavelengths, wherein the spectral filter is a wide spectral band filter; and

s220': and reconstructing an electronic dyeing effect by superposing and restoring a plurality of reconstructed spectral data with selected wavelengths into multispectral image data.

Notably, the spectral reconstruction algorithm model may be implemented as, but is not limited to: l ═ S × P, where L is the light response input of one pixel; s is a system matrix to represent the characteristics of the whole endoscope system; and P is the reconstructed spectrum of the pixel point.

It should be noted that, according to the above-mentioned embodiment of the present invention, the endoscopic imaging method may further include, before the step S100, the steps of:

controlling the opening of a light source component to emit the illumination light to the target object.

Illustrative System

Referring to fig. 9 and 10 of the drawings, an endoscopic imaging system in accordance with an embodiment of the present invention is illustrated. Specifically, as shown in fig. 9 and 10, the endoscopic imaging system 300 may include:

an acquiring module 310, configured to acquire raw signal data of an image received by a spectral chip assembly, wherein the raw signal data is received by the spectral chip assembly from illumination light reflected from a target object and focused by an optical lens, and the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on a pixel of the image sensor, and the spectral filter is located between the optical lens and the image sensor; and

and a data processing module 320, configured to perform data processing on the raw signal data to obtain spectral image data corresponding to the target object.

More specifically, in an example of the present invention, as shown in fig. 9, the data processing module 320 of the endoscopic imaging system 300 may include a direct reading module 321 and a reconstruction module 322 communicably connected to each other, wherein the direct reading module 321 is configured to directly read the acquired raw signal data to obtain a series of raw spectral data of a specific wavelength, wherein the spectral filter is a narrow spectral filter; the reconstruction module 322 is configured to reconstruct an electronic staining effect by overlapping and restoring a plurality of the original spectral data with selected wavelengths into multispectral image data.

In another example of the present invention, as shown in fig. 10, the data processing module 320 of the endoscopic imaging system 300 may include a spectrum reconstruction module 321 ' and a reconstruction module 322 ', wherein the spectrum reconstruction module 321 ' is configured to perform a spectrum reconstruction process on the acquired raw signal data according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data with specific wavelengths, wherein the spectrum filter is a wide spectrum band filter; the reconstruction module 322' is configured to reconstruct an electronic staining effect by overlaying and restoring a plurality of the reconstructed spectral data with selected wavelengths into multispectral image data.

It is noted that, according to the above-mentioned embodiment of the present invention, as shown in fig. 9 and 10, the endoscopic imaging system 300 further includes a control module 330, wherein the control module 330 is used for controlling the activation of a light source assembly to emit the illumination light to the target object.

Illustrative electronic device

Next, an electronic apparatus according to an embodiment of the present invention is described with reference to fig. 11. As shown in fig. 11, the electronic device 90 includes one or more processors 91 and memory 92.

The processor 91 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions. In other words, the processor 91 comprises one or more physical devices configured to execute instructions. For example, the processor 91 may be configured to execute instructions that are part of: one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, implement a technical effect, or otherwise arrive at a desired result.

The processor 91 may include one or more processors configured to execute software instructions. Additionally or alternatively, the processor 91 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processors of the processor 91 may be single core or multicore, and the instructions executed thereon may be configured for serial, parallel, and/or distributed processing. The various components of the processor 91 may optionally be distributed over two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the processor 91 may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

The memory 92 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 91 to implement some or all of the steps of the above-described exemplary methods of the present invention, and/or other desired functions.

In other words, the memory 92 comprises one or more physical devices configured to hold machine-readable instructions executable by the processor 91 to implement the methods and processes described herein. In implementing these methods and processes, the state of the memory 92 may be transformed (e.g., to hold different data). The memory 92 may include removable and/or built-in devices. The memory 92 may include optical memory (e.g., CD, DVD, HD-DVD, blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. The memory 92 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.

It is understood that the memory 92 comprises one or more physical devices. However, aspects of the instructions described herein may alternatively be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a limited period of time. Aspects of the processor 91 and the memory 92 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, Field Programmable Gate Arrays (FPGAs), program and application specific integrated circuits (PASIC/ASIC), program and application specific standard products (PSSP/ASSP), system on a chip (SOC), and Complex Programmable Logic Devices (CPLDs).

In one example, as shown in FIG. 11, the electronic device 90 may also include an input device 93 and an output device 94, which may be interconnected via a bus system and/or other form of connection mechanism (not shown). The input device 93 may be, for example, a camera module or the like for capturing image data or video data. As another example, the input device 93 may include or interface with one or more user input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input device 93 may include or interface with a selected Natural User Input (NUI) component. Such component parts may be integrated or peripheral and the transduction and/or processing of input actions may be processed on-board or off-board. Example NUI components may include a microphone for speech and/or voice recognition; infrared, color, stereo display and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer and/or gyroscope for motion detection and/or intent recognition; and an electric field sensing component for assessing brain activity and/or body movement; and/or any other suitable sensor.

The output device 94 may output various information including the classification result and the like to the outside. The output devices 94 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, the electronic device 90 may further comprise the communication means, wherein the communication means may be configured to communicatively couple the electronic device 90 with one or more other computer devices. The communication means may comprise wired and/or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured for communication via a wireless telephone network or a wired or wireless local or wide area network. In some embodiments, the communications device may allow the electronic device 90 to send and/or receive messages to and/or from other devices via a network such as the internet.

It will be appreciated that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Also, the order of the above-described processes may be changed.

Of course, for the sake of simplicity, only some of the components of the electronic device 90 relevant to the present invention are shown in fig. 11, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 90 may include any other suitable components, depending on the particular application.

According to another aspect of the present invention, an embodiment of the present invention further provides another electronic device. Illustratively, as shown in fig. 12, the electronic device includes an electronic device body 400 and at least one endoscopic imaging apparatus 10 as described above. The electronic device body 400 may include an endoscopic tube 410 and a monitor assembly 420. The endoscopic imaging apparatus 10 is communicably disposed at a front end of the endoscopic tube 410 of the electronic device body 400, and the endoscopic imaging apparatus 10 may include an optical lens 11 and a spectral chip assembly 12, wherein the optical lens 11 is configured to receive and focus illumination light reflected back through a target object, and the spectral chip assembly 12 is correspondingly disposed at a focal plane of the optical lens 11, and is configured to receive the illumination light focused through the optical lens 11 to be subjected to photoimaging, thereby obtaining spectral image data corresponding to the target object, wherein the endoscopic tube 410 transmits the spectral image data obtained through the endoscopic imaging apparatus 10 to the monitor assembly 420, and the monitor assembly 420 is configured to display a corresponding image based on the spectral image data.

More specifically, as shown in fig. 12, the endoscopic imaging device 10 may further include a light source assembly 13, wherein the light source assembly 13 is correspondingly disposed near the optical lens 11 for emitting the illumination light to the target object, so that the illumination light reflected by the target object can be received by the optical lens 11 for focusing.

Preferably, the optical lens 11 and the light source assembly 13 are arranged side by side at the front end of the endoscope tube 410 of the electronic device body 400 so as to observe the target object inside the alimentary canal or other ducts.

In addition, the monitor assembly 420 of the electronic device body 400 of the present application may include an endoscope controller 421, an image processor 422, and a display 423 communicably connected to each other so as to implement various functions of an endoscope.

It is noted that, as shown in fig. 12, the electronic apparatus body 400 of the present application may be, but is not limited to being, implemented as a medical endoscope such as a gastroscope, an enteroscope, a bronchoscope, or the like. Of course, in other examples of the present application, the electronic device body 400 may also be implemented as an industrial endoscope.

It should also be noted that in the apparatus, devices and methods of the present invention, the components or steps may be broken down and/or re-combined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

It will be appreciated by persons skilled in the art that the embodiments of the invention described above and shown in the drawings are given by way of example only and are not limiting of the invention. The objects of the invention have been fully and effectively accomplished. The functional and structural principles of the present invention have been shown and described in the examples, and any variations or modifications of the embodiments of the present invention may be made without departing from the principles.

Claims

1. An endoscopic imaging device adapted to be disposed at a forward end of an endoscope to acquire an image of a target object, wherein the endoscopic imaging device comprises:

2. The endoscopic imaging device according to claim 1, wherein said spectral chip assembly comprises a spectral filter and an image sensor, wherein said spectral filter is disposed on a pixel of said image sensor correspondingly, and said spectral filter is located between said optical lens and said image sensor, such that the illumination light focused by said optical lens passes through said spectral filter before being received by said image sensor.

3. The endoscopic imaging device according to claim 2, wherein said spectral filter is a narrow spectral band filter.

4. The endoscopic imaging device according to claim 3, wherein the narrow spectral band filter comprises a plurality of filter arrays having the same specifications, and the plurality of filter arrays are respectively correspondingly attached to a plurality of pixel arrays of the image sensor.

5. The endoscopic imaging device according to claim 3, wherein said data processor comprises a direct reading module and a reconstruction module communicatively connected to each other, wherein said direct reading module is configured to directly read raw signal data obtained via said image sensor to obtain a series of raw spectral data of a specific wavelength, wherein said reconstruction module is configured to reconstruct an electronic staining effect by superposing a plurality of said raw spectral data of a selected wavelength back into multispectral image data.

6. The endoscopic imaging device according to claim 2, wherein the spectral filter is a quantum dot thin film filter or a micro-nanostructured filter having a specific spectral response curve.

7. The endoscopic imaging apparatus as defined in claim 6, wherein the data processor includes a spectral reconstruction module and a reconstruction module communicatively connected to each other, wherein the spectral reconstruction module is configured to perform a spectral reconstruction process on the raw signal data obtained via the image sensor according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of a specific wavelength, and wherein the reconstruction module is configured to reconstruct an electronic staining effect by superposing and restoring a plurality of the reconstructed spectral data of a selected wavelength to multispectral image data.

8. The endoscopic imaging apparatus according to claim 7, wherein said spectral reconstruction algorithm model is L-S x P, where L is the light response input of one pixel; s is a system matrix to represent the characteristics of the whole endoscope system; and P is the reconstructed spectrum of the pixel point.

9. The endoscopic imaging device according to any one of claims 1 to 8, further comprising a light source assembly, wherein said light source assembly is disposed in the vicinity of said optical lens, respectively, for emitting the illumination light to the target object.

10. The endoscopic imaging apparatus according to claim 9, wherein a light emitting direction of said light source assembly is substantially parallel to an optical axis direction of said optical lens.

11. An electronic device adapted to acquire an image of a target object, comprising:

an electronic device body, wherein the electronic device body comprises:

12. An endoscopic imaging method, comprising the steps of:

13. The endoscopic imaging method of claim 12, wherein said step of data processing the raw signal data to obtain spectral image data corresponding to the target object comprises the steps of:

14. The endoscopic imaging method of claim 12, wherein said step of data processing the raw signal data to obtain spectral image data corresponding to the target object comprises the steps of:

15. An endoscopic imaging system for acquiring an image of a target object, wherein the endoscopic imaging system comprises, communicatively coupled to each other:

16. The endoscopic imaging system of claim 15, wherein the data processing module comprises a direct reading module and a reconstruction module communicatively connected to each other, wherein the direct reading module is configured to directly read the acquired raw signal data to obtain a series of raw spectral data of a specific wavelength, wherein the spectral filter is a narrow spectral filter; the reconstruction module is used for reconstructing an electronic dyeing effect by superposing and restoring a plurality of original spectral data with selected wavelengths into multispectral image data.

17. The endoscopic imaging system of claim 15, wherein the data processing module comprises a spectral reconstruction module and a reconstruction module communicatively connected to each other, wherein the spectral reconstruction module is configured to perform a spectral reconstruction process on the acquired raw signal data according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of specific wavelengths, wherein the spectral filter is a wide spectral filter; the reconstruction module is used for reconstructing an electronic dyeing effect by superposing and restoring a plurality of the reconstructed spectral data with the selected wavelength into multispectral image data.

18. An electronic device, comprising:

at least one processor configured to execute instructions; and