CN111127578B

CN111127578B - Intelligent programmable light source DOT imaging system, method and device

Info

Publication number: CN111127578B
Application number: CN201911361416.5A
Authority: CN
Inventors: 王洪涛; 张国旺
Original assignee: Zhejiang Dolby Medical Technology Co ltd
Current assignee: Zhejiang Dolby Medical Technology Co ltd
Priority date: 2019-12-25
Filing date: 2019-12-25
Publication date: 2023-04-07
Anticipated expiration: 2039-12-25
Also published as: CN111127578A

Abstract

The application discloses an intelligent programmable light source DOT imaging system, method and device, and the scheme of using a preset graphic template and matching with an intelligent image recognition mode to dynamically generate the template can generate a static or dynamic graphical light source with an unlimited shape, can be widely applied to various DOT imaging applications, has high reliability and long service life of the light source, greatly reduces maintenance cost, and causes better social benefit.

Description

Intelligent programmable light source DOT imaging system, method and device

Technical Field

The application relates to the technical field of medical equipment image processing, in particular to an intelligent programmable light source DOT imaging system, method and device.

Background

DOT imaging is increasingly used in the field of medical devices, and in DOT imaging systems, the role of a light source is very important, and a light source meeting requirements is needed for obtaining an image with qualified quality. With the use of DOT imaging in various medical fields, the requirements of light sources are more and more complex, and the trend of diversification is shown. The method is not only diverse in form, but also has higher requirements on dynamic parameters such as scanning tracks, scanning periods and the like. The traditional light source solution generally adopts the permutation and combination of lamps and mechanical motion to accomplish the scanning process, and the lamp bank of discretization can't be applicable to the application that requires the illumination resolution to be high, and too complicated mechanical device causes higher fault rate easily, and life is also lower moreover, causes higher maintenance cost for the user.

Disclosure of Invention

In order to solve the above technical problems, embodiments of the present application provide an intelligent programmable light source DOT imaging system, method, and apparatus.

A first aspect of an embodiment of the present application provides an intelligent programmable light source DOT imaging system, which may include:

the image acquisition module is used for acquiring image data;

the image processing module is used for receiving the image data of the image acquisition module;

the light beam shaping module is used for receiving an external light source, converting the external light source into a light beam form suitable for the imaging engine module and transmitting the converted light beam to the imaging engine module;

the imaging engine module is used for converting the light beam emitted by the light beam shaping module into a light source graph template based on the result of the image processing module and then emitting the light beam to the projection lens;

and the projection lens module is used for acquiring a light source graphic template of the imaging engine module, projecting the light source graphic template to an object field of the image acquisition module and irradiating the detected target.

Further, the image processing module specifically includes:

the DOT image processing sub-module is used for realizing the function of medical DOT imaging and carrying out medical image processing on a detection target image acquired by the camera;

the light source pattern template submodule library is used for storing preset pattern templates of light sources in various forms;

the light source template image identification submodule is used for dynamically generating a light source graphic template;

and the imaging engine interface sub-module is used for transmitting the light source graphic template to an imaging engine to form a light source with a corresponding shape to be projected to the detected target.

Furthermore, each graphic module in the light source graphic template sub-module library is correspondingly provided with a unique label unit; and the imaging engine module is used for selecting the corresponding graphic module according to the label.

Further, the light source template image identification submodule acquires an image of a camera, performs boundary detection and contour extraction on a real object scene in the image, generates a light source template according to an extraction result, and stores the generated light source pattern template into a light source pattern template submodule library.

Further, the imaging engine module comprises:

the DMD module is internally provided with a micro-mirror array and is used for acquiring the light image emitted by the light beam shaping module and emitting a light source to the projection lens;

the DMD control module is connected with the DMD module through an image interface bus; the data used for obtaining the pattern template of the light source; the DMD control module uses the received source pattern template data to control the deflection angle of each mirror plate on the micro-mirror array of the DMD module.

A second aspect of the embodiments of the present application provides an intelligent programmable light source DOT imaging method, including:

acquiring a light source graph based on a light source;

based on the formed light source graph, adjusting the light emitting shape and the light emitting angle of the imaging engine, and illuminating the bottom of the tray where the detected object is located;

acquiring image data of a detected target;

and acquiring analysis image data, and adjusting the light source pattern received by the bottom of the tray where the detected target is located.

Further, the adjusting the light-emitting shape and the light-emitting angle of the imaging engine based on the formed light source pattern to illuminate the bottom of the tray where the detected object is located includes:

acquiring an external light source, converting the external light source into a light beam form suitable for an imaging engine module, and transmitting the converted light beam to an imaging engine;

and adjusting a micro-mirror array in the imaging engine based on the image data of the detected target, and controlling the deflection angle of each lens on the micro-mirror array by using the received data.

Further, the acquiring an external light source and converting the external light source into a light beam form suitable for the imaging engine module, and the directing the converted light beam to the imaging engine includes:

acquiring an external light source illumination image;

shaping a light beam image, setting labels on the obtained different light source graphic data to form a unique light source graphic, and storing the unique light source graphic in a database form;

different light source patterns in the database are called as light source patterns for irradiating the tray.

Further, the shaping the beam image, labeling the obtained different light source pattern data to form a unique light source pattern, and storing in a database form includes:

carrying out boundary detection and contour extraction on a real object scene in the image;

generating light source graphic data based on the extracted result, and internally setting a unique label;

and forming a database by the generated light source graph data.

In a third aspect, an embodiment of the present application provides an imaging apparatus, which includes a memory and a processor, where the memory has stored thereon computer-executable instructions, and the processor implements the method of the first aspect when executing the computer-executable instructions on the memory.

In the embodiment of the application, the scheme of dynamically generating the template by using the preset graphic template and matching with an intelligent image recognition mode can generate the static or dynamic graphic light source with an unlimited shape, can be widely applied to various DOT imaging applications, and has the advantages of high reliability and long service life of the light source, greatly reduced maintenance cost and better social benefit.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a flow chart of an imaging method to which the present application is directed;

FIG. 2 is a block diagram of a DOT imaging system programmable light source device system;

FIG. 3 is a block diagram of functional sub-modules of an image processing module;

FIG. 4 is a schematic diagram showing the correspondence between a light source pattern template and an object field light source illumination surface of an image acquisition module;

FIG. 5 is an image processing module flow diagram;

FIG. 6 is a schematic diagram of a DLP engine based programmable light source DOT imaging system;

FIG. 7 is a schematic diagram of a LCD engine based programmable light source DOT imaging system;

fig. 8 is a schematic structural diagram of an imaging device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Fig. 1 is a flowchart of an intelligent programmable light source DOT imaging method according to the present application, including:

s101, acquiring a light source graph based on a light source.

It can be understood that the selection of the light source pattern is generally selected according to the actual detection target shape and the part to be detected, and as an initial step of the method, the light source pattern is mainly adjustable in brightness, and it is only necessary to directly adjust the light source of the light beam.

And S102, adjusting the light-emitting shape and the light-emitting angle of the imaging engine based on the formed light source pattern, and illuminating the bottom of the tray where the detected object is located.

It is understood that the main function of this step is to adjust the shape of the light source, and generally, the converted light beam is emitted to the imaging engine according to the form of the light beam converted to be suitable for the imaging engine module after the external light source is obtained. And selecting a proper light source image model in the imaging engine, and illuminating the tray.

Different obtained light source graph data are arranged in the imaging engine, and a unique light source graph is formed by setting a label, so that a light source image corresponding to the scheme can be quickly called in the subsequent detection or similar detection. All official image data are stored in the imaging engine in a database form, generally, the database of the scheme is larger along with the implementation of the scheme for longer and longer time, and in the subsequent imaging process, the corresponding light source image data basically only need to be directly called.

When the light source image data is extracted, the boundary detection and contour extraction are carried out on a real scene in the image, the light source graph data is generated based on the extraction result, a unique label is arranged in the light source graph data, and the generated light source graph data forms a database.

After the appropriate light source pattern in the database is acquired, the light source image is also used as the light source pattern to illuminate the tray.

In the hardware of the imaging engine, the variation of the light source pattern is generally realized in the form of a micro mirror array in a manner of controlling the deflection angle of each lens on the micro mirror array.

S103, acquiring the image data of the detected target.

It will be appreciated that there are generally three purposes here based on the acquired image data, the first being for subsequent inspection by image acquisition of the current object under test, the second being for expansion of the database as data of the light source image, and the third being for the operator to select as feedback whether to adjust the local angle of the micro mirror array inside the imaging engine as well.

And S104, acquiring analysis image data and adjusting a light source pattern received by the bottom of the tray where the detected target is located.

The embodiment of the application also provides an intelligent programmable light source DOT imaging system, which is used for executing any one of the above identification methods. Specifically, referring to fig. 2, it includes: the device comprises an image acquisition module 1-6, an image processing module 1-1, an imaging engine module 1-2, a light beam shaping module 1-3, a projection lens module 1-5 and a light source 1-4.

The image acquisition module 1-6 is used for acquiring image data;

the image processing module 1-1 is configured to receive image data of the image acquisition module;

the beam shaping module 1-3 is used for receiving an external light source, converting the external light source into a beam form suitable for the imaging engine module, and transmitting the converted beam to the imaging engine module;

the imaging engine module 1-2 converts the light beam emitted by the light beam shaping module into a light source pattern template based on the result of the image processing module and then emits the light source pattern template to the projection lens;

the projection lens module 1-5 obtains a light source graph template of the imaging engine module, projects the light source graph template to an object field of the image acquisition module, and irradiates the detected target.

The image processing module 1-1 is an embedded image processor module, and can be realized by using an ARM _ DSP image processing chip of 66AK2x series produced by TI company, the image processing module 1-1 is connected with the image acquisition module 1-6 through a USB cable or other video buses, sets and controls the image acquisition module 1-6, and receives image data from the image acquisition module 1-6; the imaging engine module 1-2 can be implemented by using a DLP optical imaging module of TI company, or can be implemented by using an LCD optical imaging module or an LCOS optical imaging module, the imaging engine module 1-2 is connected to the image processing module 1-1 through an LVDS signal or an HDMI signal, receives image data from the image processing module 1-1, and the imaging engine module 1-2 receives the light beam irradiated from the beam shaping module 1-3, and converts the light beam into an optical image by using the image data received from the image processing module 1-1 to emit the optical image to the projection lens; the light beam shaping module 1-3 receives the light beam emitted by the light source 1-4, converts the light beam into a light beam form suitable for an imaging engine, and emits the light beam to the imaging engine 1-3, and the light source 1-4 emits a light beam to the light beam shaping module 1-3, wherein the light source can be an LED (light emitting diode) or laser type light source; the projection lens 1-5 receives the light image emitted by the imaging engine module 1-2, projects the light image to an object field of the image acquisition module 1-6, and irradiates the detected target.

As shown in fig. 3, which is a block diagram of a functional sub-module of the image processing module, the DOT image processing sub-module 2-1 is responsible for implementing the function of medical DOT imaging and performing medical image processing on a detection target image acquired by a camera; the light source pattern template sub-module library 2-2 is responsible for storing preset pattern templates of light sources in various forms, and a template in a certain form is required in practical application and can be directly called from the pattern templates and output to an imaging engine; the light source template image identification submodule 2-3 is responsible for dynamically generating a graph template of a light source, when the light source graph template needs to be dynamically generated, the light source template image identification submodule 2-3 reads in an image of a camera, carries out boundary detection, contour extraction and other processing on a real object scene in the image, and generates the light source template according to an extraction result; storing the generated light source graphic template into a light source graphic template sub-module library 2-2; and the imaging engine interface sub-module 2-4 is responsible for transmitting the light source graphic template to the imaging engine to form a light source with a corresponding shape to be projected to the DOT detection target.

Fig. 4 is a schematic diagram showing a corresponding relationship between a light source graphic template and an object field light source illumination surface of an image acquisition module, where a selected area 3-1 and a non-selected area 3-3 on the light source graphic template respectively represent an area to be lit and a non-light area on an object field illumination surface, the area of the selected area 3-1 corresponding to the object field light source illumination surface of the camera is a light-emitting area 3-2, that is, a light source of the DOT system, and the area of the non-selected area 3-3 corresponding to the object field light source illumination surface of the camera is a non-light area 3-4.

As shown in fig. 5, which is a flow chart of an image processing module, firstly, it is determined whether to use an existing light source graphic template for illumination, and if so, the light source graphic template is directly called from a light source graphic template sub-module library; if not, the template is formed in real time in an image extraction mode, firstly, the pattern layer of the whole light source template is set as a selected area and displayed through an imaging engine, at the moment, the whole camera object field is a light emitting area, the camera collects images, the area of a detected target is shielded to form a dark area, the dark area is extracted as the selected area by utilizing edge detection and contour extraction, and the rest is used as a non-selected area, so that a light source of the corresponding light emitting area is formed on the light source irradiation surface of the camera object field.

As shown in fig. 6, which is a schematic diagram of a programmable light source DOT imaging system based on a DLP engine, an image processing module 1-1 is connected to a camera 1-8 (irradiation area is a) through a USB cable, the camera is controlled to collect an image of a target to be detected and read the image, the image processing module 1-1 is connected to a DMD control module 5-2 in the imaging engine through an LVDS cable and transmits data of a graphic template of a light source to the DMD control module 5-2, the DMD control module 5-2 is connected to a DMD module 5-3 through an image interface bus, and the DMD control module 5-2 controls a deflection angle of each lens on a micromirror array of the DMD module 5-3 by using the received graphic template data.

Light beams emitted by the light source 1-4 are adjusted by the light beam shaping module 1-3 to be projected onto a micro-mirror array of the DMD module 5-3, the micro-mirror array forms light images according to control data of the DMD control module 5-2 and the incident light, the light images are reflected to a projection lens and irradiate on a breast tray 5-7, and the breast tray 5-7 is a camera object field light source irradiation surface where a measured object B is located. As mentioned above, the microlens corresponding to the data of the selected area on the graphic template data will reflect the light received by itself to the projection lens, forming a light emitting area, i.e. a light source; the microlens sheeting for the non-selected area data will not reflect the incident light, thus forming a non-illuminated area in the direction of the projection lens.

Fig. 7 is a schematic diagram of a DLP engine-based programmable light source DOT imaging system, which works very similar to the DLP engine-based programmable light source DOT imaging system, except that the control module in the imaging engine is replaced by the LCD control module 6-2, and the dmd module 5-3 is replaced by the LCD module 6-3.

Fig. 8 is a schematic structural diagram of an imaging device according to an embodiment of the present application. The object detection apparatus 4000 comprises a processor 41 and may further comprise an input device 42, an output device 43 and a memory 44. The input device 42, the output device 43, the memory 44 and the processor 41 are connected to each other via a bus.

The memory includes, but is not limited to, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), which is used for storing instructions and data.

The input means are for inputting data and/or signals and the output means are for outputting data and/or signals. The output means and the input means may be separate devices or may be an integral device.

The processor may include one or more processors, for example, one or more Central Processing Units (CPUs), and in the case of one CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor may also include one or more special purpose processors, which may include GPUs, FPGAs, etc., for accelerated processing.

The memory is used to store program codes and data of the network device.

The processor is used for calling the program codes and data in the memory and executing the steps in the method embodiment. Specifically, reference may be made to the description of the method embodiment, which is not repeated herein.

It will be appreciated that fig. 8 only shows a simplified design of the object detection device. In practical applications, the motion recognition devices may further include other necessary components, including but not limited to any number of input/output devices, processors, controllers, memories, etc., and all motion recognition devices that can implement the embodiments of the present application are within the scope of the present application.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the division of the unit is only one logical function division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. The shown or discussed mutual coupling, direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are wholly or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a read-only memory (ROM), or a Random Access Memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape, a magnetic disk, or an optical medium, such as a Digital Versatile Disk (DVD), or a semiconductor medium, such as a Solid State Disk (SSD).

Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to the details of the foregoing embodiments, and various equivalent changes (such as number, shape, position, etc.) may be made to the technical solution of the present invention within the technical spirit of the present invention, and the equivalents are protected by the present invention.

Claims

1. An intelligent programmable light source DOT imaging system, comprising:

the image acquisition module is used for acquiring image data;

the projection lens module is used for acquiring a light source graphic template of the imaging engine module, projecting the light source graphic template to an object field of the image acquisition module and irradiating a detected target;

the image processing module specifically comprises:

the DOT image processing submodule is used for realizing the function of medical DOT imaging and processing medical images of a detection target image acquired by the camera;

the light source graphic template submodule library is used for storing preset graphic templates of light sources in various forms;

2. The intelligent programmable light source DOT imaging system of claim 1, wherein each graphic module in the light source graphic template sub-module library is provided with a unique tag unit; and the imaging engine module is used for selecting the corresponding graphic module according to the label.

3. The intelligent programmable light source DOT imaging system of claim 2, wherein the light source template image recognition sub-module obtains an image of a camera, performs boundary detection and contour extraction on a real object scene in the image, generates a light source template according to the extraction result, and stores the generated light source pattern template in the light source pattern template sub-module library.

4. The intelligent programmable light source DOT imaging system of claim 3, wherein the imaging engine module comprises:

5. An intelligent programmable light source DOT imaging method is characterized by comprising the following steps:

acquiring a light source graph based on an actual detection target shape;

acquiring image data of a detected target;

acquiring analysis image data, and adjusting a light source graph received by the bottom of a tray where a detected target is located;

the light-emitting shape and the light-emitting angle of the imaging engine are adjusted based on the formed light source pattern, and the step of illuminating the bottom of the tray where the detected target is located comprises the following steps:

adjusting a micro-mirror array in an imaging engine based on image data of the detected target, and controlling the deflection angle of each lens on the micro-mirror array by using the received data;

the acquiring an external light source and converting the external light source into a light beam form suitable for the imaging engine module, and the transmitting the converted light beam to the imaging engine comprises:

acquiring an external light source illumination image;

6. An intelligent programmable light source DOT imaging method as claimed in claim 5, wherein the shaping the beam image, labeling the resulting different light source pattern data to form a unique light source pattern, the storing in database form comprises:

and forming a database by the generated light source graph data.

7. An imaging device comprising a memory having computer-executable instructions stored thereon and a processor that, when executing the computer-executable instructions on the memory, performs the method of any one of claims 1-4.